streaming

[navaneethstabilix]

async def stream_to_websocket(llm, websocket, prompt):
async for chunk in llm.astream(prompt):
await websocket.send(chunk.content)
yield chunk.content

Usage example

Assuming websocket is your WebSocket connection object

async for content in stream_to_websocket(llm, websocket, "write an essay on Sachin in 200 words"):
# Process each chunk as needed
pass

how to add flush= true and end ="" here

[dosubot[bot]]

To incorporate flush=True and end="" in your asynchronous function, you can use the print function to output each chunk immediately. However, since flush and end are parameters of the print function, they don't directly apply to the websocket.send method. Instead, you can use them when printing the content to the console. Here's how you can modify your function:

async def stream_to_websocket(llm, websocket, prompt):
    async for chunk in llm.astream(prompt):
        await websocket.send(chunk.content)
        print(chunk.content, end="", flush=True)  # Print each chunk immediately
        yield chunk.content

# Usage example
# Assuming `websocket` is your WebSocket connection object
async for content in stream_to_websocket(llm, websocket, "write an essay on Sachin in 200 words"):
    # Process each chunk as needed
    pass

In this code, print(chunk.content, end="", flush=True) ensures that each chunk is printed to the console immediately without buffering, and without adding a newline after each chunk.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}