How to import the stream_options parameter in chatopenapi of langchain 0.0.352 version. Let the large model type output token usage information.

[XiaozanZhang]

How to import the stream_options parameter in chatopenapi of langchain0.0.352 version. Let the large model type output token usage information.

[dosubot[bot]]

@XiaozanZhang Hello there! I'm here to help you with any bugs, questions, or contribution assistance you might need while waiting for a human maintainer. Let's work together to tackle this!

To use the stream_options parameter in the chatopenapi module of LangChain version 0.0.352 to output token usage information, you need to configure it to include usage metadata in the streaming output. You can do this by setting the include_usage option within stream_options. Here's a basic example of how you might set it up:

from langchain.chatopenapi import YourChatModelClass

# Initialize your chat model
chat_model = YourChatModelClass(...)

# Set stream_options to include usage
stream_options = {"include_usage": True}

# Use the model with stream_options
for chunk in chat_model._stream(stream_options=stream_options):
    # Process each chunk
    print(chunk)

This setup ensures that the streaming output includes token usage information by setting include_usage to True in the stream_options ^[1].

To continue talking to Dosu, mention @dosu.

[XiaozanZhang]

if my code is like photo, how can i solve? tks

[feijoes]

@XiaozanZhang, are you trying to get the response as a stream? Try this:

async def event_stream(llm_chain: Runnable, message: str):
    async for event in llm_chain.astream_events({"input": message}, version="v2"):
        yield content.encode('utf-8')

Or, if I understand correctly, you want to import the typing stream options?

from openai.types.chat.chat_completion_stream_options_param import ChatCompletionStreamOptionsParam