How can I pass additional arguments to a tool that are not generated by the LLM?

[ldorigo]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class SearchKBToolInput(BaseModel):
    query: str = Field(default="The contextualized question to search for in the knowledge base.")

class SearchKBToolOutput(BaseModel):
    sources: dict[str, str] = Field(default="The sources of the documents retrieved from the knowledge base.")
    documents: list[str] = Field(default="The documents retrieved from the knowledge base.")    


class SearchKBTool(BaseTool):
    name: str = "search_knowledge_base"
    description: str = Field(
        default="Searches the knowledge base for answers. The query parameter should contain the contextualized question to search for in the knowledge base."
    )
    return_schema = SearchKBToolOutput
    args_schema :Type[BaseModel]= SearchKBToolInput
    pinecone: Pinecone
    retrieval_chain: Runnable[str, SearchKBToolOutput] = None
    
    def __init__(self,  *args, **kwargs):
        super().__init__(*args, **kwargs)

        retriever = self.pinecone.as_retriever(
            search_type="mmr",
            search_kwargs={"lambda_mult": 0.9},
        ).configurable_fields(
            search_kwargs=ConfigurableField(
                id="search_kwargs",
                name="Search Kwargs",
                description="The search kwargs to use",
            )
        )

        retrieve_documents_chain = RunnablePassthrough.assign(
            documents=lambda x: x["query"] | retriever,
        ).with_config(run_name="Retrieve Documents Chain")

        # Output Formatter
        output_formatter_chain = RunnableParallel(
            sources=itemgetter("documents") | documents_to_sources,
        ).with_config(run_name="Output Formatter Chain")

        # Chain
        retrieval_chain = (
            retrieve_documents_chain
            | output_formatter_chain
        )
        self.retrieval_chain = retrieval_chain

    async def _arun(self, query: str, assistant_id: str, **kwargs) -> SearchKBToolOutput:
        result= await self.retrieval_chain.ainvoke(
            {"query": query},
            {"configurable": {"search_kwargs": {"namespace": assistant_id}}}
        )

        return result



    def _run(self, query: str, assistant_id: str, **kwargs) -> SearchKBToolOutput:
        result = self.retrieval_chain.invoke(
            {"query": query},
            {"configurable": {"search_kwargs": {"namespace": assistant_id}}}
        )
        return result

Description

I followed the examples from langgraph and in the docs about "tools", but I can't figure out how to let some input parameters of the tool be set by hand by the others need to be filled by the LLM.

I have an assistant_id parameter that determines the pinecone namespace and I want to pass it manually to the tool when I invoke it. If i make the Input model something like:

class SearchKBToolInput(BaseModel):
    query: str = Field(default="The contextualized question to search for in the knowledge base.")
    assistant_id: str

Then the LLM will see the assistant_id argument and try to generate it.
However if I don't include it in the input model, the method langchain_core.tool.BaseTool._parse_input discards any parameter that I pass to the tool. How should I do it? This is how I invoke the tool:

# In __init__.py:
        self.tool_executor = ToolExecutor(self.tools, verbose=True)


# in call_tool():
        parsed_args = json.loads(last_message.additional_kwargs["function_call"]["arguments"])
        if tool_name == self.search_kb_tool.name:
            tool_input = {
                    "query": parsed_args["query"],
                    "assistant_id": state["assistant_id"],
            }
       # ...

        action = ToolInvocation(
            tool=tool_name,
            tool_input=tool_input,
        )

        response = await self.tool_executor.ainvoke(action)

System Info

System Information

OS: Linux
OS Version: #1 ZEN SMP PREEMPT_DYNAMIC Sat, 13 Jan 2024 14:36:54 +0000
Python Version: 3.11.6 (main, Nov 14 2023, 09:36:21) [GCC 13.2.1 20230801]

Package Information

langchain_core: 0.1.18
langchain: 0.1.5
langchain_community: 0.0.17
langchain_openai: 0.0.5
langgraph: 0.0.21

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langserve

[dosubot[bot]]

🤖

Hey @ldorigo! Great to see you diving deep into the intricacies of LangChain again. Looking forward to tackling this puzzle together. 🚀

Based on your code and the problem you're facing, it seems like you want to pass an additional argument (assistant_id) to your tool that is not generated by the Language Learning Model (LLM). However, you're facing an issue because the _parse_input method in the BaseTool class discards any parameter that is not included in the args_schema.

One way to handle this is to include the assistant_id in the args_schema but make it an optional field. This way, the LLM won't try to generate it, and you can still pass it manually when invoking the tool. Here's how you can modify your SearchKBToolInput class:

class SearchKBToolInput(BaseModel):
    query: str = Field(default="The contextualized question to search for in the knowledge base.")
    assistant_id: Optional[str] = Field(default=None)

Then, in your _arun and _run methods, you can check if assistant_id is None and if so, set it to the default value you want to use:

async def _arun(self, query: str, assistant_id: Optional[str] = None, **kwargs) -> SearchKBToolOutput:
    if assistant_id is None:
        assistant_id = "default_assistant_id"  # replace with your default value

    result= await self.retrieval_chain.ainvoke(
        {"query": query},
        {"configurable": {"search_kwargs": {"namespace": assistant_id}}}
    )

    return result

def _run(self, query: str, assistant_id: Optional[str] = None, **kwargs) -> SearchKBToolOutput:
    if assistant_id is None:
        assistant_id = "default_assistant_id"  # replace with your default value

    result = self.retrieval_chain.invoke(
        {"query": query},
        {"configurable": {"search_kwargs": {"namespace": assistant_id}}}
    )
    return result

This way, you can pass the assistant_id manually when invoking the tool, and the LLM won't try to generate it because it's an optional field in the args_schema. If you don't pass it manually, it will be set to the default value you specify in the _arun and _run methods.

Please note that this solution assumes that you're okay with having a default value for assistant_id when it's not passed manually. If this is not the case, you might need to modify your tool's logic to handle the case when assistant_id is None.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• libs/langchain/langchain/agents/openai_assistant/base.py
• libs/langchain/langchain/agents/load_tools.py
• libs/core/langchain_core/tools.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ldorigo]

This is just a hacky workaround - I don't want the assistant_id parameter to be sent to the LLM at all.

[eyurtsev]

One way to do this is to create the tool dynamically:

def create_my_tool(my_arg) -> Tool:
    @tool # <-- Use tool decorator to make your life easier
    def tool_name(arg1, arg2, ..):
      """ Tool Description""" 
      ... use my_arg, arg1, arg2 ...

[benm5678]

Cool, thanks for that...I just had to add return tool_name to make it work.

[ldorigo]

Thanks, but this is not optimal at all for tools that require some (slow) initialization logic (which is the reason I subclassed tool rather than using a function in the first place) - It requires re-initializing the tool on every single call...

[benm5678]

In my case, we use it to create a toolset. The toolset can be reused once initialized.

[ldorigo]

Found a solution - just use extra: "allow" in the pydantic model's config.

[eyurtsev]

Function calling and sub-classing should provide identical expressive power since you can bind to variables from outer scope.

my_slow_initlaization_code = ... # <-- Do slow stuff here

def create_my_tool(my_arg) -> Tool:
    @tool # <-- Use tool decorator to make your life easier
    def tool_name(arg1, arg2, ..):
      """ Tool Description""" 
      ... use my_arg, arg1, arg2 ...
       use the result of slow stuff here

If your'e able re-write your init to be fast -- as a rule of thumb init should involve no network requests or heavy compute (i know that a lot of langchain tools don't follow this design currently, but we'll try to push them toward this pattern over time)

[fatimatayeb]

How will you do this with an agent?
The agent invokes the tool but before it does, I want to pass a parameter to it (I don't want the LLM to see this parameter).

[ldorigo]

I found an easy general solution - just mark the pydantic model with extra: allow:

from langchain.pydantic_v1 import BaseModel
from langchain.pydantic_v1 import Extra
from langchain.pydantic_v1 import Field


class MyToolInput(BaseModel):
    query: str = Field(default="The query")

    class Config:
        extra = Extra.allow

[ldorigo]

When you allow extra fields, you can then use different input parameters for your actual call. e.g.:

class MyTool(BaseTool):
    name = "my_tool"
    args_schema: Type[BaseModel] = MyToolInput

    def _arun(self, query, some_hidden_param: str):
            ...

This will allow you to still use this tool as an actual LLM tool/function call (so the LLM will only generate the "official" input parameter, query), but it will still work if you pass additinal arguments to the tool executor (e.g. :

tool_input = {
  "query": "foo",
  "some_hidden_param": "bar"
}

action = ToolInvocation(
  tool = "my_tool",
  tool_input = tool_input
)

result = tool_executor.ainvoke(action)

(note this is within langgraph, for agents you probably need to adapt a little)

[GaborCC]

Has anyone solved this with agents?

[ldorigo]

Your easiest shot (although a little brittle) is probably just to subclass the relevant agent class and modify the function that actually calls the tool.

[sduncker1]

Struggling to implement the ideas above into a ToolNode context. I basically need to patch through a sessionID to the tool function for further usage there. Have tried a number of methods but no luck so far, including binding to the tool. Last resort will be to route via the LLM call which is not great however. The session_ID is only available right before the tool invocation inside the node, so somehow need to be able to pass on as the add'l param.

Any add'l ideas anyone? Below shortened code to highlight the issue

class MyToolInput(BaseModel):
    query: str = Field(default="The query")

    class Config:
        extra = Extra.allow

class MyTool(BaseTool):
    name = "MyTool"
    description="Ask a question"
    args_schema = MyToolInput

    def _arun(self, question:str, session_id:str):
        print(session_id)
        answer = input(question + ":\n ")

        question_and_answer = f"{question} Answer:{answer}"
        return question_and_answer

    def _run(self, question:str, session_id:str):
        print(session_id)
        answer = input(question + ":\n ")

        question_and_answer = f"{question} Answer:{answer}"
        return question_and_answer


new_tool = MyTool()

tool_node = ToolNode([new_tool])


def execute(state: State):
    #this node executes the function calling

    ...setting up prompt and more....

    new_tool_with_sessionID = new_tool.bind(session_id=state["session_id"])

    #configure model
    llm_model = ChatOpenAI(model="gpt-4o", temperature=0, streaming=True)
    llm_model_with_tools = llm_model.bind_tools([new_tool_with_sessionID])

    response = llm_model_with_tools.invoke(messages)

    return {"messages": [response]}

[DrBilitski]

I make a member variable and initialize it in the constructor

class MyTool(BaseTool):
    myOtherParam = "Some default value"  # <--------



    def _run(self, question:str, session_id:str):
        print(session_id)
        answer = input(question + ":\n ")

        print (self.myOtherParam)   # <--------

        question_and_answer = f"{question} Answer:{answer}"
        return question_and_answer


new_tool = MyTool(myOtherParam="some value")  # <--------

[pariwartan]

how to pass assistant_id across all tools in Langgraph

[ch-liuzhide]

You can try using the feature of closures to temporarily store assistantId without worrying about memory leaks.

def get_tool(assistant_id):
    return {
        "SearchKBTool": factory(assistant_id = assistant_id),
    }

and your tool with closures like below:

def factory(assistant_id: str):
    assistant_id = assistant_id

    @tool(parse_docstring=True)
    def SearchKBTool(
        query: str,
    ) -> str:
        """Search for information based on the query. '.

        Args:
            query: The user's question.
        """
        try:
            return retrieval.search_knowledge(query, assistant_id)
        except Exception as e:
            print(f"An error occurred: {e}")
            return None

    return search_knowledge

[epistoteles]

This thread is a bit older but still pops up when people search for this topic in search engines.

I want to add my two cents, because the built-in solution of langchain is not mentioned here: InjectedToolArg. This type was added to LangChain in August 2024.

This is how InjectedToolArg is used (in this example the LLM should generate the expression but not the auth_token):

from langchain_core.tools import tool, InjectedToolArg

class CalculatorInput(BaseModel):
    expression: str = Field(description="The math expression to calculate'")
    auth_token: Annotated[str, InjectedToolArg]  # this will not be included in the tool schema

@tool(args_schema=CalculatorInput)
def calculator(expression: str, auth_token: Annotated[str, InjectedToolArg]):
    """Use this tool to calculate an expression (e.g. '1+1')."""
    ...

In the background, LangChain will keep the argument for the input schema:

>>> calculator.get_input_schema().schema()
{
    'properties': {
        'expression': {
            'description': "The math expression to calculate",
            'title': 'Expression',
            'type': 'string'
        },
        'auth_token': {'title': 'Auth Token', 'type': 'string'}
    },
    'required': ['expression', 'auth_token'],
    'title': 'CalculatorInput',
    'type': 'object'
}

but hide it from the LLM in the tool call schema:

>>> calculator.tool_call_schema.schema()
{
    'description': "Use this tool to calculate an expression (e.g. '1+1').",
    'properties': {
        'expression': {
            'description': "The math expression to calculate",
            'title': 'Expression',
            'type': 'string'
        }
    },
    'required': ['expression'],
    'title': 'calculator',
    'type': 'object'
}

As a result, the additional argument auth_token will not be filtered out by the input parser, but passed to the tool when it is invoked.

I hope this helps anyone looking for a solution post August 2024.