When Migrating From ChatOpenAI to ChatGroq Binding Functions Raise an Error

[ardakocaman100101]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

options = ["FINISH"]
function_def = {
    "name": "route",
    "description": "Select the next role.",
    "parameters": {
        "title": "routeSchema",
        "type": "object",
        "properties": {
            "next": {
                "title": "Next",
                "anyOf": [
                    {"enum": options},
                ],
            }
        },
        "required": ["next"],
    },
}


supervisor_chain = (
    prompt
    | model.bind_functions(functions=[function_def], function_call="route")
)
supervisor_chain.invoke({"input": ["what's the date today?"]})

Description

I built a langgraph multiagent system similar to Supervisor architecture and it works without any errors with chatgpt
I decided to migrate to llama3 with groq's api and used chatgroq (from langchain_groq import ChatGroq) it worked fine with a test agent. create_tool_calling_agent and AgentExecutor methods are used for creating the agent.
I got error when I tried to create a chain that binds function to the model.
model.bind_functions(functions=[function_def], function_call="route")
this part seems problematic. I got an error saying:
TypeError: Completions.create() got an unexpected keyword argument 'functions'
I also tried with .bind() method but no luck. Is this because langchain depreciated function binding with tool calling? If that's the problem can you recommend me a sample work that illustrates migrating from functions to tools with required parameters?
Thank you.

System Info

langchain==0.1.16
langchain-community==0.0.32
langchain-core==0.1.45
langchain-experimental==0.0.57
langchain-groq==0.1.3
langchain-openai==0.1.3
langchain-pinecone==0.1.0
langchain-text-splitters==0.0.1
langgraph==0.0.38

[dosubot[bot]]

To address the TypeError: Completions.create() got an unexpected keyword argument 'functions' issue you're encountering after migrating to ChatGroq, it's necessary to switch from using functions to tools. This change is due to updates in the API that now requires the use of tools and tool_choice instead of functions and function_call.

Here's a revised approach using bind_tools to integrate tools with your model, which aligns with the updated API expectations:

from langchain_community.tools import MoveFileTool
from langchain_core.messages import HumanMessage
from langchain_openai import ChatOpenAI

# Initialize your model
model = ChatOpenAI(model="gpt-3.5-turbo")

# Define your tools
tools = [MoveFileTool()]

# Bind tools to the model
model_with_tools = model.bind_tools(tools)

# Invoke the model with tools
message = model_with_tools.invoke([HumanMessage(content="move file foo to bar")])

This code snippet demonstrates how to correctly bind tools to your model, which should eliminate the TypeError by using the bind_tools method. This method is specifically designed for the updated API, facilitating the integration of function-like objects with your model.

For further guidance and examples on migrating from functions to tools with the required parameters, refer to the LangChain documentation, particularly the section on using tools as OpenAI functions.

Details

Sources

• docs/docs/modules/agents/tools/tools_as_openai_functions.ipynb
• libs/langchain/langchain/chains/openai_tools/extraction.py
• libs/core/langchain_core/utils/function_calling.py
• libs/community/tests/unit_tests/tools/file_management/test_move.py
• libs/partners/groq/langchain_groq/chat_models.py
• docs/docs/integrations/chat/groq.ipynb
• libs/core/langchain_core/tools.py
• docs/docs/modules/tools/custom_tools.ipynb
• libs/langchain/langchain/chains/router/base.py
• docs/docs/expression_language/how_to/routing.ipynb
• libs/core/langchain_core/runnables/router.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[touhi99]

unfortunately doesn't help. how to do the routing here among multiple agents?

[ardakocaman100101]

I've never seen this bot solve any decent problem anyway

[touhi99]

Probably its just a RAG on documentation added, but I dont think it can identify critical issue intelligently yet. Hope it gets better in future release.

[eyurtsev]

OpenAI introduced functions first, then they deprecated functions and replace them with tools.

Use bind_tools rather than bind_functions and treat bind_functions as deprecated.

[ardakocaman100101]

thank you Eugene

[ardakocaman100101]

Hello again,
I replaced bind_functions with bind_tools (and also function_call with tool_choice) but still having problems with parsing. What would be the best parser for function calling? I used to use the JsonOutputFunctionsParser but now I switch my model to ChatGroq and usel the tools so I gotta find another one. Do you have any recommendations?

[touhi99]

worked for me:

supervisor_chain = (
            prompt
            | llm.bind_tools(tools=[function_def]) 
            | JsonOutputToolsParser(first_tool_only=True)
            | flatten_output
)


def flatten_output(data):
    if 'args' in data and isinstance(data['args'], dict):
        args_content = data.pop('args')  
        data.update(args_content) 
    return data

Although ChatGroq Llama models not handling agents properly for me, not sure yet. but first pass works so far

[ardakocaman100101]

I realized that too, although 70B model should easily beat gpt3.5 it doesn't feel like it when I tried an A/B test. Would you say sources other than chatgroq changes anything?
I'll try using it with few-shot prompting

[touhi99]

no, they will be definitely better options than gpt3.5 maybe. I had a case where previously working with gpt-4-turbo, quite smooth (but i didnt want to use the paid api and slow process) - so trying out with groq (llama3) . But realized, apart from wrong routing, I am hitting a rate limit error. Probably something inefficient, I need to work on removing any overhead.

[MoMoWongHK]

It seems it won't stop (I mean the agent keep talking)

[touhi99]

maybe some few-shot prompting helps, I also have similar case when using a openai turbo 01-25 vs azure openai 11-06 version. Azure openai one seems stuck in some loop, I am just not sure if the other LLMs are just bad for agentic approahc, or maybe something I am doing wrong. so far with openai turbo seems most of cases good, but trying to optimize for llama/mixtral in groq

[jacktang]

@touhi99 Hello, I applied the code snippet to agent supervisor example. And run the test code:

for s in graph.stream(
    {
        "messages": [
            HumanMessage(content="Code hello world and print it to the terminal")
        ]
    }
):
    if "__end__" not in s:
        print(s)
        print("----")

It throws error:

{'supervisor': {'next': 'Coder'}}
----
[22](vscode-notebook-cell:?execution_count=3&line=22) def agent_node(state, agent, name):
---> [23](vscode-notebook-cell:?execution_count=3&line=23)     result = agent.invoke(state)
     [24](vscode-notebook-cell:?execution_count=3&line=24)     return {"messages": [HumanMessage(content=result["output"], name=name)]}

AttributeError: 'NoneType' object has no attribute 'invoke'

Do you know how to fix it?

[jacktang]

Is there any simple implementation for route tool? I am confused why it is so complex

[hobbytp]

I found groq llama3-groq-70b-8192-tool-use-preview model can work good for tool usage in langchain/langgraph