Skip to content

Missing Handling for delta.reasoning_content in agents.models.chatcmpl_stream_handler.ChatCmplStreamHandler.handle_stream #578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kingmming opened this issue Apr 23, 2025 · 3 comments
Labels
bug Something isn't working

Comments

@kingmming
Copy link

Issue Title: Missing Handling for delta.reasoning_content in agents.models.chatcmpl_stream_handler.ChatCmplStreamHandler.handle_stream

Description:

In the ChatCmplStreamHandler.handle_stream method, the case where delta.reasoning_content is present is not handled. This could lead to incomplete or incorrect handling of the stream response when reasoning content is provided by the model.

Problematic Code Example:

class ChatCmplStreamHandler:
    @classmethod
    async def handle_stream(
        cls,
        response: Response,
        stream: AsyncStream[ChatCompletionChunk],
    ) -> AsyncIterator[TResponseStreamEvent]:
        usage: CompletionUsage | None = None
        state = StreamingState()

        async for chunk in stream:
            
            if not state.started:
                state.started = True
                yield ResponseCreatedEvent(
                    response=response,
                    type="response.created",
                )

            usage = chunk.usage

            if not chunk.choices or not chunk.choices[0].delta:
                continue

            delta = chunk.choices[0].delta

            # Handle text
            if delta.content:
		…… 

            # Handle refusals (model declines to answer)
            if delta.refusal:

Steps to Reproduce:

  1. Use a model that provides reasoning content in its responses.
  2. Trigger a stream response that includes delta.reasoning_content.
  3. Observe that the ChatCmplStreamHandler.handle_stream method does not process or emit events related to the reasoning content.

Actual Behavior:

The delta.reasoning_content is ignored, and no events are emitted to handle or notify consumers of the reasoning content.

Impact:

This issue prevents the proper handling and display of reasoning content in the stream, which is crucial for models that provide explanations or reasoning behind their responses.

Proposed Solution:

Add handling for delta.reasoning_content in the ChatCmplStreamHandler.handle_stream method. Example code snippet:

class ChatCmplStreamHandler:
    @classmethod
    async def handle_stream(
        cls,
        response: Response,
        stream: AsyncStream[ChatCompletionChunk],
    ) -> AsyncIterator[TResponseStreamEvent]:
        usage: CompletionUsage | None = None
        state = StreamingState()

        async for chunk in stream:
            
            if not state.started:
                state.started = True
                yield ResponseCreatedEvent(
                    response=response,
                    type="response.created",
                )

            usage = chunk.usage

            if not chunk.choices or not chunk.choices[0].delta:
                continue

            delta = chunk.choices[0].delta

            # Handle text
            if delta.content:

            # Handle reasoning content text
            if delta.reasoning_content:
   
@kingmming kingmming added the bug Something isn't working label Apr 23, 2025
@rm-openai
Copy link
Collaborator

@kingmming which model produces delta.reasoning_content?

@kingmming
Copy link
Author

@kingmming which model produces delta.reasoning_content?

Hi @rm-openai ! The delta.reasoning_content is generated by the DeepSeek-R1 model.

When the enable-reasoning option is enabled during the OpenAI-compatible service startup, the DeepSeek-R1 model provides reasoning content in the response stream.

Here is an example of how it appears in the response data:

{
    "choices": [
        {
            "delta": {
                "content": null,
                "reasoning_content": " smoothly"
            },
            "finish_reason": null,
            "index": 0,
            "logprobs": null
        }
    ],
    "object": "chat.completion.chunk",
    "usage": null,
    "created": 1745454612,
    "system_fingerprint": null,
    "model": "DeepSeek-R1",
    "id": "chatcmpl-281b5461-bfa3-99f5-b3cb-650b2306e712"
}
{
    "choices": [
        {
            "delta": {
                "content": " assistant",
                "reasoning_content": null
            },
            "finish_reason": null,
            "index": 0,
            "logprobs": null
        }
    ],
    "object": "chat.completion.chunk",
    "usage": null,
    "created": 1745454612,
    "system_fingerprint": null,
    "model": "DeepSeek-R1",
    "id": "chatcmpl-281b5461-bfa3-99f5-b3cb-650b2306e712"
}

@Ddper
Copy link
Contributor

Ddper commented Apr 24, 2025

I'm working on this issue, see this PR #494

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants