langgraph: add structured output to create_react_agent #2848

vbarda · 2024-12-20T20:31:20Z

class WeatherResponse(BaseModel):
    """Respond to the user with this"""

    temperature: float = Field(description="The temperature in fahrenheit")
    wind_direction: str = Field(
        description="The direction of the wind in abbreviated form"
    )
    wind_speed: float = Field(description="The speed of the wind in mph")

@tool
def get_weather(city: Literal["nyc", "sf"]):
    """Use this to get weather information."""
    if city == "nyc":
        return "It is cloudy in NYC, with 5 mph winds in the North-East direction and a temperature of 70 degrees"
    elif city == "sf":
        return "It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction"
    else:
        raise AssertionError("Unknown city")

model = ChatOpenAI()
tools = [get_weather]
agent_with_structured_output = create_react_agent(model, tools, response_format=WeatherResponse)
agent_with_structured_output.invoke({"messages": [("user", "what's the weather in nyc?")]})

{
    'messages': [...],
    'structured_response': WeatherResponse(temperature=70.0, wind_directon='NE', wind_speed=5.0)
}

eyurtsev · 2024-12-21T02:12:18Z

@vbarda one question with this approach -- are users going to need to control the prompt used for the model is used for extracting structured data (e.g., w/ few shot examples, further formatting instructions etc.)? Do we know whether this is good enough for the average case?

eyurtsev · 2024-12-21T02:13:59Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

-        ):
+        required_keys = {"messages", "remaining_steps"}
+        if response_format is not None:
+            required_keys.add("structured_response")


did we decide on structured_response vs. parsed? I'm OK with current

eyurtsev · 2024-12-21T02:16:00Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

+                `response_format` requires the model to support `.with_structured_output`
+
+            !!! Note
+                The graph will make a separate call to the LLM to generate the structured response after the agent loop is finished.


Does it make sense to make a note that other strategies are possible? (We could note to a guide on how to customize? or can also just not worry about it)

eyurtsev · 2024-12-21T02:17:26Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

    if not tool_calling_enabled:
        # Define a new graph
        workflow = StateGraph(state_schema or AgentState)
        workflow.add_node("agent", RunnableCallable(call_model, acall_model))
        workflow.set_entry_point("agent")
+        if response_format is not None:
+            workflow.add_node(
+                "generate_structured_response",


could also use a shorter name like structure_response -- or can have chatty g brain storm a 1-2 word name

eyurtsev · 2024-12-21T02:18:01Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

@@ -201,6 +226,7 @@ def create_react_agent(
    state_schema: Optional[StateSchemaType] = None,
    messages_modifier: Optional[MessagesModifier] = None,
    state_modifier: Optional[StateModifier] = None,
+    response_format: Optional[Union[dict, type[BaseModel]]] = None,


is there a unit test for this?

achrafmam2

Hi, my name is Achraf. I wanted to learn the codebase and thought of starting with looking at recent pull requests.

I don't mean to add extra work. Please consider all my comments as optional. If you think it adds noise please let me know so to stop.

achrafmam2 · 2024-12-22T11:00:15Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

+        )
+        # NOTE: we exclude the last message because there is enough information
+        # for the LLM to generate the structured response
+        response = model_with_structured_output.invoke(state["messages"][:-1], config)


Is there a downside in sending the full history? I am worried that this could cause bugs in the future.

achrafmam2 · 2024-12-22T11:14:34Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

@@ -162,6 +174,19 @@ def _should_bind_tools(model: LanguageModelLike, tools: Sequence[BaseTool]) -> b
    return False


+def _get_model(model: LanguageModelLike) -> BaseChatModel:


Consider _unwrap_chat_model or _extract_chat_model or something similar that reflects the purpose of the function.

achrafmam2 · 2024-12-22T11:18:49Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

+        state: AgentState, config: RunnableConfig
+    ) -> AgentState:
+        model_with_structured_output = _get_model(model).with_structured_output(
+            cast(Union[dict, type[BaseModel]], response_format)


nit: Starting PEP604 one can use the shortcut dict | type[BaseModel].

By the way you may get away from using this cast by adding a defensive check.

assert response_format is not None, "Internal error: calling generate_structured_response when response_format is None".

The | operator is only supported in python 3.10+, never mind the suggestion.

achrafmam2 · 2024-12-22T11:22:35Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

@@ -643,12 +720,14 @@ async def acall_model(state: AgentState, config: RunnableConfig) -> AgentState:
        )

    # Define the function that determines whether to continue or not
-    def should_continue(state: AgentState) -> Literal["tools", "__end__"]:


Why not extend the Literal this provides extra safety and readability.

achrafmam2 · 2024-12-22T11:30:59Z

libs/langgraph/langgraph/prebuilt/chat_agent_executor.py

+                generate_structured_response, agenerate_structured_response
+            ),
+        )
+        workflow.add_edge("generate_structured_response", "__end__")


Consider using the END constant. This is both safer, and helps in future refactoring.

jorgeutd · 2024-12-22T16:31:32Z

❤️

vbarda added 4 commits December 20, 2024 15:02

langgraph: add structured output to create_react_agent

0c0574a

update

9f9753c

lint

862f968

test

6fe052f

eyurtsev approved these changes Dec 21, 2024

View reviewed changes

eyurtsev reviewed Dec 21, 2024

View reviewed changes

achrafmam2 reviewed Dec 22, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

langgraph: add structured output to create_react_agent #2848

langgraph: add structured output to create_react_agent #2848

vbarda commented Dec 20, 2024

eyurtsev commented Dec 21, 2024

eyurtsev Dec 21, 2024

eyurtsev Dec 21, 2024

eyurtsev Dec 21, 2024

eyurtsev Dec 21, 2024

achrafmam2 left a comment

achrafmam2 Dec 22, 2024

achrafmam2 Dec 22, 2024

achrafmam2 Dec 22, 2024

achrafmam2 Dec 23, 2024

achrafmam2 Dec 22, 2024

achrafmam2 Dec 22, 2024

jorgeutd commented Dec 22, 2024

		@@ -162,6 +174,19 @@ def _should_bind_tools(model: LanguageModelLike, tools: Sequence[BaseTool]) -> b
		return False


		def _get_model(model: LanguageModelLike) -> BaseChatModel:

langgraph: add structured output to create_react_agent #2848

Are you sure you want to change the base?

langgraph: add structured output to create_react_agent #2848

Conversation

vbarda commented Dec 20, 2024

eyurtsev commented Dec 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

achrafmam2 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgeutd commented Dec 22, 2024