OpenAI Structured Output with Langchain Or Langgraph

[mbenisti-ledger]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

#no code needed / functional question

Description

Hi,
Langgraph is great. However, I could not find in the documentation a way to leverage OpenAI Structured Output Json Schema (see documentation here: https://openai.com/index/introducing-structured-outputs-in-the-api/)
Have I missed it? And if not, is there any plan to add it in the future ? This is definitely an important feature if I want to output structured data to be used in a data pipeline where we expect a specific format.

System Info

Irrelevant

[dosubot[bot]]

Hey there, @mbenisti-ledger! I'm here to help you with any bugs, questions, or contribution guidance while we wait for a human maintainer. Let's tackle this together!

Yes, LangChain supports leveraging OpenAI Structured Output JSON Schema. You can use the create_structured_output_runnable function with the mode set to "openai-json" to achieve this.

Here is an example:

from typing import Optional
from langchain.chains import create_structured_output_runnable
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field

class Dog(BaseModel):
    '''Identifying information about a dog.'''
    name: str = Field(..., description="The dog's name")
    color: str = Field(..., description="The dog's color")
    fav_food: Optional[str] = Field(None, description="The dog's favorite food")

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
structured_llm = create_structured_output_runnable(Dog, llm, mode="openai-json")
system = '''You are a world class assistant for extracting information in structured JSON formats. \
Extract a valid JSON blob from the user input that matches the following JSON Schema:
{output_schema}'''
prompt = ChatPromptTemplate.from_messages(
    [("system", system), ("human", "{input}"),]
)
chain = prompt | structured_llm
chain.invoke({"input": "Harry was a chubby brown beagle who loved chicken"})

This example demonstrates how to use the create_structured_output_runnable function with the mode parameter set to "openai-json" to extract structured outputs in JSON format ^[1].

Additionally, LangChain provides other methods to handle structured outputs, such as using the with_structured_output method. For example:

from typing import Optional
from langchain_core.pydantic_v1 import BaseModel, Field

class Joke(BaseModel):
    '''Joke to tell user.'''
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(description="How funny the joke is, from 1 to 10")

structured_llm = llm.with_structured_output(Joke)
structured_llm.invoke("Tell me a joke about cats")

This will return a structured output in the form of a Joke object ^[2].

LangChain's BaseChatModel class provides several imperative and declarative methods to facilitate structured outputs, including with_structured_output. This method creates a wrapper that structures model output using a schema, making it easier to handle structured data in various formats ^[3].

To continue talking to Dosu, mention @dosu.

[robbiemu]

why does Hermes llama3.1 (fine tuned for tool calling) not work? https://smith.langchain.com/public/0dc88908-9a04-4f44-b5bc-d3cb68ae2638/r It is returning raw JSON. I notice Mistral-nemo does work, and it returns yaml.

[derekhsu]

According to the API documentation, create_structured_output_runnable has been deprecated since version 0.1.17 and will be removed since version 1.0. Is the method create_structured_output_runnable still recommended?