Tool calling agent invokes undefined tool: 'describe'

[maryshgh]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

RunnableWithMessageHistory(AgentExecutor(agent=create_tool_calling_agent(llm_with_tools,self.tools, system_prompt))).invoke(input_prompt, config={ 'configurable': {'session_id': session_id} })

Description

We are using a set of tools and we have prompted model through tool_calling_agent system prompt to only invoke tools from the given list, and one of the tools we use is named 'fallback', for specific questions where model is supposed to use this fallback tool with the following format:

Invoking: fallback with {'question': 'please answer the following question'}

The model uses the following and fails to respond, does anyone know why is this happening?

Invoking: describe with {'extension': 'fallback'}

System Info

Vertex AI
Python: 3.10.12

[dosubot[bot]]

The issue of the tool-calling agent invoking an undefined tool named 'describe' instead of the expected 'fallback' tool is likely due to the prompt configuration or the tools binding process. Specifically, the prompt must have an agent_scratchpad key, and the language model (LLM) must correctly bind the tools.

To resolve this issue, ensure the following:

1. Prompt Configuration: Verify that the prompt includes the agent_scratchpad key as a MessagesPlaceholder. This is crucial for the agent to correctly handle intermediate steps and tool outputs.

2. Tool Binding: Ensure that the LLM has a bind_tools method implemented and that it correctly binds the provided tools.

Here is an example of how to correctly set up the prompt and bind the tools:

from langchain.agents import AgentExecutor, create_tool_calling_agent, tool
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate

# Define the prompt with the required `agent_scratchpad` key
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a helpful assistant"),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
        ("placeholder", "{agent_scratchpad}"),
    ]
)

# Initialize the language model
model = ChatAnthropic(model="claude-3-opus-20240229")

# Define the tools
@tool
def magic_function(input: int) -> int:
    """Applies a magic function to an input."""
    return input + 2

tools = [magic_function]

# Create the tool-calling agent
agent = create_tool_calling_agent(model, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Invoke the agent with the correct input
agent_executor.invoke({"input": "what is the value of magic_function(3)?"})

Additionally, ensure that the tools are correctly defined and imported. For example, if you are using tools from the langchain_community package, make sure they are properly imported and available for the agent to use:

from langchain_community.tools import (
    BraveSearch,
    HumanInputRun,
    # Add other tools as needed
)

Ensure that the bind_tools method on the LLM correctly binds the tools and that the prompt is correctly configured with the necessary placeholders. This should resolve the issue of the agent invoking an undefined tool.

Note that importing tools directly from langchain is deprecated. You should import tools from langchain_community instead:

from langchain_community.tools import BraveSearch, HumanInputRun

If you encounter any deprecation warnings, follow the instructions to update your imports accordingly.

In addition, you can refer to the test_agent_invalid_tool function from the LangChain test suite, which demonstrates how to handle invalid tool invocations and provide correct suggestions:

def test_agent_invalid_tool() -> None:
    """Test agent invalid tool and correct suggestions."""
    fake_llm = FakeListLLM(responses=["FooBarBaz\nAction: Foo\nAction Input: Bar"])
    tools = [
        Tool(
            name="Search",
            func=lambda x: x,
            description="Useful for searching",
            return_direct=True,
        ),
    ]
    agent = initialize_agent(
        tools=tools,
        llm=fake_llm,
        agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
        return_intermediate_steps=True,
        max_iterations=1,
    )

    resp = agent("when was langchain made")
    resp["intermediate_steps"][0][1] == "Foo is not a valid tool, try one of [Search]."

This example shows how to handle cases where an invalid tool is invoked and suggests the correct tool to use ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.