Please integrate json_repair package to fix JSON from LLMs

[ArslanKAS]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

I've tried LangChain, LLaMaIndex and Ollama as well. They all struggle with JSON output. Out of 100 calls at least 2,3 calls will still have broken JSON that needs manual code to get fixed even If we use Pydantic and nicely ask LLMs to follow the proper JSON schema.

There's a package called "json_repair" that can easily fix any broken LLM JSON response:
https://github.com/mangiucugna/json_repair

Please integrate it into LangChain so we can easily get good JSON as response

Motivation

The following is the package for the motivation:
https://github.com/mangiucugna/json_repair

Please have a look at them or maybe talk to them. This new feature would be a big help.

Proposal (If applicable)

No response

[mainpyp]

Using OpenAIs response_format={ "type": "json_object" } will do the trick when the token limit is not exceeded. However this is just the case for OpenAI and presumably won't help you if you are using another model, however I just wanted to leave it here!

[vladman-25]

I have the solution for you:

from langchain_core.runnables import RunnableLambda
from langchain_core.output_parsers import JsonOutputParser
from json_repair import repair_json

def wrapper_repair_json(input):
    input.content = repair_json(input.content)
    return input

chain = construction_prompt | llm | RunnableLambda(wrapper_repair_json) | JsonOutputParser()