langchain.core : Use shallow copy for schema manipulation in JsonOutputParser.get_format_instructions

[L-cloud]

• Description :

Fix: Use shallow copy for schema manipulation in get_format_instructions

Prevents side effects on the original schema object by using a dictionary comprehension for a safer and more controlled manipulation of schema key-value pairs, enhancing code reliability.

• Issue: Issue: JsonOutputParser's get_format_instructions() Modifying Pydantic Class Schema #17161
• Dependencies: None
• Twitter handle: None

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Feb 13, 2024 6:40am

[hwchase17]

can we add a comment about why this is neccessary, and a quick unit test

[L-cloud]

@hwchase17
Thank you for your comment! I will now provide a detailed example of the specific changes required, drawn from an actual case I have encountered. This involves the use of the same Base Class in two scenarios: one utilizing an OpenAI function call and the other employing a JsonOutputParser.

class Joke(BaseModel):
    setup: str = Field(description="question to set up a joke")
    punchline: str = Field(description="answer to resolve the joke")

joke_query = "Tell me a joke."

parser = JsonOutputParser(pydantic_object=Joke)

prompt = PromptTemplate(
    template="Answer the user query.\n{format_instructions}\n{query}\n",
    input_variables=["query"],
    partial_variables={"format_instructions": parser.get_format_instructions()}, #cause
)

chain = prompt | model | parser

chain.invoke({"query": joke_query})

openai_functions = [convert_to_openai_function(Joke)]

parser = JsonOutputFunctionsParser()

chain2 = prompt | model.bind(functions=openai_functions) | parser

chain2.invoke({"query": "tell me a joke"})

In this code, chain2 triggers an error stating: openai.BadRequestError: Error code: 400 - {'error': {'message': "'' does not match '^[a-zA-Z0-9_-]{1,64}$' - 'functions.0.name'", 'type': 'invalid_request_error', 'param': None, 'code': None}}. The root cause of this error is the direct deletion of variables from the BaseModel schema via the schema() function in the get_format_instructions function, as shown below:

schema = self.pydantic_object.schema()
reduced_schema = schema
            if "title" in reduced_schema:
                del reduced_schema["title"]
            if "type" in reduced_schema:
                del reduced_schema["type"]

The use of BaseModel's .schema() function in convert_to_openai_function results in default values of "" being assigned to the name and type, leading to the aforementioned error. Although this is the only case I have identified, I believe other functions utilizing BaseModel could similarly be at risk. While there may be a slight degradation in performance, utilizing schema_str appears to present no significant issues.

            schema_str = json.dumps(reduced_schema)
            return JSON_FORMAT_INSTRUCTIONS.format(schema=schema_str)

Please let me know if there is anything that needs clarification, if there's anything I've missed, or if the test code is inadequate or lacking :)

[hwchase17]

a comment here would be great! otherwise looks good

[L-cloud]

@hwchase17
I've applied the changes, thank you :)

[L-cloud]

I have also made the same changes in /langchain/langchain/output_parsers/yaml.py and /langchain/langchain/output_parsers/pydantic.py.