-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into brace/better-tools
- Loading branch information
Showing
1 changed file
with
229 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,229 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# How to pass run time values to a tool\n", | ||
| "\n", | ||
| ":::info Prerequisites\n", | ||
| "\n", | ||
| "This guide assumes familiarity with the following concepts:\n", | ||
| "- [Chat models](/docs/concepts/#chat-models)\n", | ||
| "- [LangChain Tools](/docs/concepts/#tools)\n", | ||
| "- [How to create tools](/docs/how_to/custom_tools)\n", | ||
| "- [How to use a model to call tools](/docs/how_to/tool_calling/)\n", | ||
| ":::\n", | ||
| "\n", | ||
| ":::info Supported models\n", | ||
| "\n", | ||
| "This how-to guide uses models with native tool calling capability.\n", | ||
| "You can find a [list of all models that support tool calling](/docs/integrations/chat/).\n", | ||
| "\n", | ||
| ":::\n", | ||
| "\n", | ||
| "You may need to bind values to a tool that are only known at runtime. For example, the tool logic may require using the ID of the user who made the request.\n", | ||
| "\n", | ||
| "Most of the time, such values should not be controlled by the LLM. In fact, allowing the LLM to control the user ID may lead to a security risk.\n", | ||
| "\n", | ||
| "Instead, the LLM should only control the parameters of the tool that are meant to be controlled by the LLM, while other parameters (such as user ID) should be fixed by the application logic.\n", | ||
| "\n", | ||
| "This how-to guide shows a simple design pattern that creates the tool dynamically at run time and binds to them appropriate values." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We can bind them to chat models as follows:\n", | ||
| "\n", | ||
| "```{=mdx}\n", | ||
| "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", | ||
| "\n", | ||
| "<ChatModelTabs\n", | ||
| " customVarName=\"llm\"\n", | ||
| "/>\n", | ||
| "```" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Passing request time information\n", | ||
| "\n", | ||
| "The idea is to create the tool dynamically at request time, and bind to it the appropriate information. For example,\n", | ||
| "this information may be the user ID as resolved from the request itself." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 9, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { z } from \"zod\";\n", | ||
| "import { StructuredTool } from \"@langchain/core/tools\";\n", | ||
| "\n", | ||
| "const userToPets: Record<string, string[]> = {};\n", | ||
| "\n", | ||
| "function generateToolsForUser(userId: string): StructuredTool[] {\n", | ||
| " class UpdateFavoritePets extends StructuredTool {\n", | ||
| " name = \"update_favorite_pets\";\n", | ||
| " description = \"Add the list of favorite pets.\";\n", | ||
| " schema = z.object({\n", | ||
| " pets: z.array(z.string())\n", | ||
| " })\n", | ||
| "\n", | ||
| " async _call(input: { pets: string[] }): Promise<string> {\n", | ||
| " userToPets[userId] = input.pets;\n", | ||
| " return \"update_favorite_pets called.\"\n", | ||
| " }\n", | ||
| " }\n", | ||
| "\n", | ||
| " class DeleteFavoritePets extends StructuredTool {\n", | ||
| " name = \"delete_favorite_pets\";\n", | ||
| " description = \"Delete the list of favorite pets.\";\n", | ||
| "\n", | ||
| " schema = z.object({\n", | ||
| " no_op: z.boolean().optional().describe(\"No operation.\")\n", | ||
| " })\n", | ||
| "\n", | ||
| " async _call(input: never): Promise<string> {\n", | ||
| " if (userId in userToPets) {\n", | ||
| " delete userToPets[userId];\n", | ||
| " }\n", | ||
| " return \"delete_favorite_pets called.\"\n", | ||
| " }\n", | ||
| " }\n", | ||
| "\n", | ||
| " class ListFavoritePets extends StructuredTool {\n", | ||
| " name = \"list_favorite_pets\";\n", | ||
| " description = \"List favorite pets if any.\";\n", | ||
| " schema = z.object({\n", | ||
| " no_op: z.boolean().optional().describe(\"No operation.\")\n", | ||
| " })\n", | ||
| "\n", | ||
| " async _call(input: never): Promise<string> {\n", | ||
| " return JSON.stringify(userToPets[userId]) || JSON.stringify([]);\n", | ||
| " }\n", | ||
| " }\n", | ||
| "\n", | ||
| " return [new UpdateFavoritePets(), new DeleteFavoritePets(), new ListFavoritePets()];\n", | ||
| "}" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Verify that the tools work correctly" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 10, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "{ brace: [ 'cat', 'dog' ] }\n", | ||
| "[\"cat\",\"dog\"]\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "const [updatePets, deletePets, listPets] = generateToolsForUser(\"brace\")\n", | ||
| "await updatePets.invoke({ pets: [\"cat\", \"dog\"] })\n", | ||
| "console.log(userToPets)\n", | ||
| "console.log(await listPets.invoke({}))" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 11, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import { BaseChatModel } from \"@langchain/core/language_models/chat_models\";\n", | ||
| "\n", | ||
| "async function handleRunTimeRequest(userId: string, query: string, llm: BaseChatModel): Promise<any> {\n", | ||
| " if (!llm.bindTools) {\n", | ||
| " throw new Error(\"Language model does not support tools.\");\n", | ||
| " }\n", | ||
| " const tools = generateToolsForUser(userId);\n", | ||
| " const llmWithTools = llm.bindTools(tools);\n", | ||
| " return llmWithTools.invoke(query);\n", | ||
| " }" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "This code will allow the LLM to invoke the tools, but the LLM is **unaware** of the fact that a **user ID** even exists!" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 12, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "{\n", | ||
| " name: 'update_favorite_pets',\n", | ||
| " args: { pets: [ 'cats', 'parrots' ] },\n", | ||
| " id: 'call_to1cbIVqMNuahHCdFO9oQzpN'\n", | ||
| "}\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "const aiMessage = await handleRunTimeRequest(\n", | ||
| " \"brace\", \"my favorite animals are cats and parrots.\", llm,\n", | ||
| ")\n", | ||
| "console.log(aiMessage.tool_calls[0])" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| ":::tip\n", | ||
| "Click [here](https://smith.langchain.com/public/3d766ecc-8f28-400b-8636-632e6f1598c7/r) to see the LangSmith trace for the above run.\n", | ||
| ":::\n", | ||
| "\n", | ||
| ":::tip\n", | ||
| "Chat models only output requests to invoke tools, they don't actually invoke the underlying tools.\n", | ||
| "\n", | ||
| "To see how to invoke the tools, please refer to [how to use a model to call tools](/docs/how_to/tool_calling/).\n", | ||
| ":::" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "TypeScript", | ||
| "language": "typescript", | ||
| "name": "tslab" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "mode": "typescript", | ||
| "name": "javascript", | ||
| "typescript": true | ||
| }, | ||
| "file_extension": ".ts", | ||
| "mimetype": "text/typescript", | ||
| "name": "typescript", | ||
| "version": "3.7.2" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 4 | ||
| } |