Invalid tool calls when using streamEvents and a node calls model.withStructuredOutput()

[sebastianmulders]

Hello! When using structured outputs within a node, I am getting back AIMessageChunks containing invalid_tool_calls with an error stating Malformed args. and error type being invalid_tool_call. Following the LangChain docs on .withStructuredOutput() from here and using that in LangGraph, results in AIMessageChunks containing for example:

AIMessageChunk {
  "id": "chatcmpl-AT7lKLldnGMv0PmRhp0xdQwN5934F",
  "content": "",
  "additional_kwargs": {
    "tool_calls": [
      {
        "index": 0,
        "function": "[Object]"
      }
    ]
  },
  "response_metadata": {
    "usage": {}
  },
  "tool_calls": [],
  "tool_call_chunks": [
    {
      "args": "rating",
      "index": 0,
      "type": "tool_call_chunk"
    }
  ],
  "invalid_tool_calls": [
    {
      "args": "rating",
      "error": "Malformed args.",
      "type": "invalid_tool_call"
    }
  ]
}
AIMessageChunk {
  "id": "chatcmpl-AT7pv0xRkHB6QFhe2z5KKTS1AjfuF",
  "content": "",
  "additional_kwargs": {
    "tool_calls": [
      {
        "index": 0,
        "function": "[Object]"
      }
    ]
  },
  "response_metadata": {
    "usage": {}
  },
  "tool_calls": [],
  "tool_call_chunks": [
    {
      "args": "}",
      "index": 0,
      "type": "tool_call_chunk"
    }
  ],
  "invalid_tool_calls": [
    {
      "args": "}",
      "error": "Malformed args.",
      "type": "invalid_tool_call"
    }
  ]
}

The response/output of the chain run is complete and useful, but I noticed these invalid_tool_calls when using .streamEvents(). It does not seem to happen when using .stream(). When not using withStructuredOutput(), everything works as expected. Not sure if this is potentially related to #470.

Here's an example:

import { Annotation, START, StateGraph, END } from '@langchain/langgraph';
import { BaseMessage, isAIMessageChunk } from '@langchain/core/messages';
import { ChatOpenAI } from '@langchain/openai';
import { z } from 'zod';

const GraphState = Annotation.Root({
  messages: Annotation<BaseMessage[]>({
    reducer: (x, y) => x.concat(y),
  }),
});

const joke = z.object({
  setup: z.string().describe('The setup of the joke'),
  punchline: z.string().describe('The punchline to the joke'),
  rating: z.number().optional().describe('How funny the joke is, from 1 to 10'),
});

const model = new ChatOpenAI({
  model: 'gpt-4o-mini',
  temperature: 0,
  streaming: true,
});

const nodeFn = async (state: typeof GraphState.State) => {
  const responseMessages = await model.withStructuredOutput(joke).invoke(state.messages);

  return { messages: [responseMessages] };
};

const workflow = new StateGraph(GraphState).addNode('node', nodeFn).addEdge(START, 'node').addEdge('node', END);

const app = workflow.compile();

const stream = app.streamEvents(
  { messages: [{ role: 'user', content: 'Tell me a joke about cats.' }] },
  { version: 'v2' },
);

for await (const { event, data } of stream) {
  if (event === 'on_chat_model_stream' && isAIMessageChunk(data.chunk)) {
    console.log(data.chunk);
  }
}

Using @langchain/core ^0.3.18, @langchain/langgraph ^0.2.20, and @langchain/openai ^0.3.13.
LangSmith trace: https://smith.langchain.com/public/e2ee1092-6315-4bd4-a11d-9950d540a015/r

[danielpgauer]

up!