Support for Structured Outputs in LangChain using OpenAI's API in langchain_openai

[CD-rajveer]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

OpenAI recently announced support for structured outputs in their API. This feature enables developers to define JSON schemas and constrain the outputs based on these schemas, enhancing the performance and reliability of applications using large language models (LLMs). This can be implemented using function calling and custom JSON schemas, making it an invaluable tool for developers.

Here is the announcement: OpenAI: Introducing Structured Outputs in the API.

This feature can be integrated into LangChain, allowing users to define and utilize structured outputs seamlessly. Tools such as Instructor, Outlines, and Guidance can be leveraged for this purpose.

Motivation

The introduction of structured outputs in the OpenAI API addresses a critical need for improving the reliability and performance of applications using LLMs. By constraining the output to adhere to predefined schemas, developers can ensure more predictable and accurate responses from the models.

Proposal (If applicable)

To integrate support for structured outputs in LangChain, the following steps can be taken:

API Integration: Extend LangChain to support OpenAI's API for structured outputs. This will involve adding methods to define JSON schemas and function calls that enforce these schemas.
Tool Utilization: Incorporate tools like Instructor, Outlines, and Guidance to aid in defining and managing structured outputs.
Documentation: Provide comprehensive documentation and examples to help users understand how to define schemas, make function calls, and utilize structured outputs in their applications.
Tutorials and Examples: Create tutorials and practical examples to demonstrate the benefits and usage of structured outputs in various use cases.

[ntakouris]

It's already there

llm: BaseChatModel = ChatOpenAI(model="gpt-4o")

if isinstance(llm, ChatOpenAI):
    llm = llm.with_structured_output(strict=True)

[hansvdam]

That is not the way; it gets translated into a tools-call under the hood if just with a strict-param added. The following causes true openAI structured output:

llm: BaseChatModel = ChatOpenAI(model="gpt-4o")

if isinstance(llm, ChatOpenAI):
    llm = llm.with_structured_output(model="gpt-4o", method="json_schema", strict=True)

as per the documentation in: langchain_openai:

            method: The method for steering model generation, one of:

                - "function_calling":
                    Uses OpenAI's tool-calling (formerly called function calling)
                    API: https://platform.openai.com/docs/guides/function-calling
                - "json_schema":
                    Uses OpenAI's Structured Output API: https://platform.openai.com/docs/guides/structured-outputs
                    Supported for "gpt-4o-mini", "gpt-4o-2024-08-06", and later
                    models.
                - "json_mode":
                    Uses OpenAI's JSON mode. Note that if using JSON mode then you
                    must include instructions for formatting the output into the
                    desired schema into the model call:
                    https://platform.openai.com/docs/guides/structured-outputs/json-mode