From 74adcbc9f9d0b36413a8d33c87246223adef274e Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Wed, 22 Jan 2025 17:21:17 +0800 Subject: [PATCH 01/11] create structure and add basic usage page --- fern/pages/v2/tool-use/tool-use-citations.mdx | 21 +++ .../v2/tool-use/tool-use-function-calling.mdx | 14 ++ .../pages/v2/tool-use/tool-use-multi-step.mdx | 14 ++ fern/pages/v2/tool-use/tool-use-overview.mdx | 120 ++++++++++++++++++ fern/pages/v2/tool-use/tool-use-schema.mdx | 14 ++ fern/pages/v2/tool-use/tool-use-streaming.mdx | 14 ++ .../tool-use/tool-use-structured-outputs.mdx | 14 ++ fern/v2.yml | 35 +++-- 8 files changed, 237 insertions(+), 9 deletions(-) create mode 100644 fern/pages/v2/tool-use/tool-use-citations.mdx create mode 100644 fern/pages/v2/tool-use/tool-use-function-calling.mdx create mode 100644 fern/pages/v2/tool-use/tool-use-multi-step.mdx create mode 100644 fern/pages/v2/tool-use/tool-use-overview.mdx create mode 100644 fern/pages/v2/tool-use/tool-use-schema.mdx create mode 100644 fern/pages/v2/tool-use/tool-use-streaming.mdx create mode 100644 fern/pages/v2/tool-use/tool-use-structured-outputs.mdx diff --git a/fern/pages/v2/tool-use/tool-use-citations.mdx b/fern/pages/v2/tool-use/tool-use-citations.mdx new file mode 100644 index 000000000..3e8ff0f96 --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-citations.mdx @@ -0,0 +1,21 @@ +--- +title: "Citations for tool use" +slug: "v2/docs/tool-use-citations" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- + +## Accessing citations + +## Citation modes + +### Fast citations + +### Accurate citations \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-function-calling.mdx b/fern/pages/v2/tool-use/tool-use-function-calling.mdx new file mode 100644 index 000000000..c0193b508 --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-function-calling.mdx @@ -0,0 +1,14 @@ +--- +title: "Function calling" +slug: "v2/docs/tool-use-function-calling" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- +TBD \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-multi-step.mdx b/fern/pages/v2/tool-use/tool-use-multi-step.mdx new file mode 100644 index 000000000..2df467b0b --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-multi-step.mdx @@ -0,0 +1,14 @@ +--- +title: "Multi-step tool use (agents)" +slug: "v2/docs/tool-use-multi-step" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- +TBD \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx new file mode 100644 index 000000000..232366298 --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -0,0 +1,120 @@ +--- +title: "Tool use - basic usage" +slug: "v2/docs/tool-use-overview" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- + +## Overview + +Tool use is a technique which allows developers to connect Cohere’s Command R family of models to external tools like search engines, APIs, functions, databases, etc. + +This opens up a richer set of behaviors by leveraging data stored in tools, taking actions through APIs, interacting with a vector database, querying a search engine, etc., and is particularly valuable for enterprise developers, since a lot of enterprise data lives in external sources. + +The Chat endpoint comes with built-in tool use capabilities such as function calling, multi-step reasoning, and citation generation. + +## Setup + +First, import the Cohere library and create a client. + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys +``` + +## Tool definition + +The pre-requisite, or Step 0, before we can run a tool use workflow, is to define the tools. We can break this further into two steps: + +- Creating the tool +- Defining the tool schema + +### Creating the tool + +A tool can be any function that you create or extrenal services that return an object for a given input. Some examples: a web search engine, an email service, an SQL database, a vector database, a weather data service, a sports data service, or even another LLM. + +In this example, we define a `get_weather` function that returns the temperature for a given query, which is the location. You can implement any logic here, but to simplify the example, here we are hardcoding the return value to be the same for all queries. + +```python PYTHON +def get_weather(location): + # Implement any logic here + return [{"temperature": "20C"}] + # Return a list of objects e.g. [{"url": "abc.com", "text": "..."}, {"url": "xyz.com", "text": "..."}] + +functions_map = {"get_weather": get_weather} +``` + +### Defining the tool schema + +We also need to define the tool schemas in a format that can be passed to the Chat endpoint. The schema follows the JSON Schema specification and must contain the following fields: +- `name`: the name of the tool. +- `description`: a description of what the tool is and what it is used for. +- `parameters`: a list of parameters that the tool accepts. For each parameter, we need to define the following fields: + - `type`: the type of the parameter. + - `properties`: the name of the parameter and the following fields: + - `type`: the type of the parameter. + - `description`: a description of what the parameter is and what it is used for. + - `required`: a list of required parameters from the `properties` field. + +This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on the information that it contains. Therefore, the more descriptive and clear the schema, the more likely the LLM will make the right tool call decisions. + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "the location to get weather, example: San Fransisco, CA", + } + }, + "required": ["location"], + }, + }, + }, +] +``` + +## Running the tool use workflow + +We can think of a tool use system as consisting of four components: + +- The user +- The application +- The LLM +- The tools + +At its most basic, these four components interact in a workflow through four steps: + +- Step 1: **Get user message**: The LLM gets the user message (via the application). +- Step 2: **Generate tool calls**: The LLM decides which tools to call (if any) and generates the tool calls. +- Step 3: **Get tool results**: The application executes the tools, and the results are sent to the LLM. +- Step 4: **Generate response and citations**: The LLM generates the response and citations back to the user. + +Tool use workflow + +### Step 1: Get user message +... + +### Step 2: Generate tool calls +... + +### Step 3: Get tool results +... + +### Step 4: Generate response and citations +... \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-schema.mdx b/fern/pages/v2/tool-use/tool-use-schema.mdx new file mode 100644 index 000000000..f5617be7d --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-schema.mdx @@ -0,0 +1,14 @@ +--- +title: "Tool use schema" +slug: "v2/docs/tool-use-schema" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- +TBD \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx new file mode 100644 index 000000000..d5ba503e1 --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -0,0 +1,14 @@ +--- +title: "Streaming for tool use" +slug: "v2/docs/tool-use-streaming" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- +TBD \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-structured-outputs.mdx b/fern/pages/v2/tool-use/tool-use-structured-outputs.mdx new file mode 100644 index 000000000..8df3be578 --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-structured-outputs.mdx @@ -0,0 +1,14 @@ +--- +title: "Structured outputs for tool use" +slug: "v2/docs/tool-use-structured-outputs" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- +TBD \ No newline at end of file diff --git a/fern/v2.yml b/fern/v2.yml index cd4331694..a8597b4ac 100644 --- a/fern/v2.yml +++ b/fern/v2.yml @@ -80,16 +80,33 @@ navigation: - page: Retrieval Augmented Generation (RAG) path: pages/v2/text-generation/retrieval-augmented-generation-rag.mdx - section: Tool Use - path: pages/v2/text-generation/tools.mdx contents: - - page: Tool Use - path: pages/v2/text-generation/tools/tool-use.mdx - - page: Multi-step Tool Use (Agents) - path: pages/v2/text-generation/tools/multi-step-tool-use.mdx - - page: Implementing a Multi-Step Agent with Langchain - path: pages/v2/text-generation/tools/implementing-a-multi-step-agent-with-langchain.mdx - - page: Parameter Types in Tool Use - path: pages/v2/text-generation/tools/parameter-types-in-tool-use.mdx + - page: Basic usage + path: pages/v2/tool-use/tool-use-overview.mdx + - page: Function calling + path: pages/v2/tool-use/tool-use-function-calling.mdx + - page: Tool schema + path: pages/v2/tool-use/tool-use-schema.mdx + - page: Structured outputs + path: pages/v2/tool-use/tool-use-structured-outputs.mdx + - page: Multi-step tool use (agents) + path: pages/v2/tool-use/tool-use-multi-step.mdx + - page: Citations + path: pages/v2/tool-use/tool-use-citations.mdx + - page: Streaming + path: pages/v2/tool-use/tool-use-streaming.mdx + + # - section: Tool Use + # path: pages/v2/text-generation/tools.mdx + # contents: + # - page: Tool Use + # path: pages/v2/text-generation/tools/tool-use.mdx + # - page: Multi-step Tool Use (Agents) + # path: pages/v2/text-generation/tools/multi-step-tool-use.mdx + # - page: Implementing a Multi-Step Agent with Langchain + # path: pages/v2/text-generation/tools/implementing-a-multi-step-agent-with-langchain.mdx + # - page: Parameter Types in Tool Use + # path: pages/v2/text-generation/tools/parameter-types-in-tool-use.mdx - page: Tokens and Tokenizers path: pages/v2/text-generation/tokens-and-tokenizers.mdx - section: Prompt Engineering From 2a8c6689c38bc401c4332c7dfe8741fe48575a52 Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Thu, 23 Jan 2025 16:44:09 +0800 Subject: [PATCH 02/11] initial set of pages --- fern/pages/v2/tool-use/tool-use-citations.mdx | 14 +- ...{tool-use-schema.mdx => tool-use-faqs.mdx} | 6 +- .../pages/v2/tool-use/tool-use-multi-step.mdx | 15 +- fern/pages/v2/tool-use/tool-use-overview.mdx | 134 +++++++++++++++++- ...lling.mdx => tool-use-parameter-types.mdx} | 8 +- fern/pages/v2/tool-use/tool-use-streaming.mdx | 9 +- .../v2/tool-use/tool-use-tool-definition.mdx | 35 +++++ fern/v2.yml | 18 +-- 8 files changed, 215 insertions(+), 24 deletions(-) rename fern/pages/v2/tool-use/{tool-use-schema.mdx => tool-use-faqs.mdx} (82%) rename fern/pages/v2/tool-use/{tool-use-function-calling.mdx => tool-use-parameter-types.mdx} (62%) create mode 100644 fern/pages/v2/tool-use/tool-use-tool-definition.mdx diff --git a/fern/pages/v2/tool-use/tool-use-citations.mdx b/fern/pages/v2/tool-use/tool-use-citations.mdx index 3e8ff0f96..93481ebcd 100644 --- a/fern/pages/v2/tool-use/tool-use-citations.mdx +++ b/fern/pages/v2/tool-use/tool-use-citations.mdx @@ -13,9 +13,19 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- ## Accessing citations +[[TODO - describe citations and how to access them, what do they contain]] + +### Non-streaming +[[TODO - show how to access citations from the response object in a non-streaming scenario]] + +### Streaming +[[TODO - show how to access citations from the response object in a streaming scenario]] ## Citation modes +[[TODO - describe citation modes - fast, accurate, etc]] -### Fast citations +### Accurate citations +[[TODO - show code example of fast citations]] -### Accurate citations \ No newline at end of file +### Fast citations +[[TODO - show code example of fast citations]] diff --git a/fern/pages/v2/tool-use/tool-use-schema.mdx b/fern/pages/v2/tool-use/tool-use-faqs.mdx similarity index 82% rename from fern/pages/v2/tool-use/tool-use-schema.mdx rename to fern/pages/v2/tool-use/tool-use-faqs.mdx index f5617be7d..feadaa673 100644 --- a/fern/pages/v2/tool-use/tool-use-schema.mdx +++ b/fern/pages/v2/tool-use/tool-use-faqs.mdx @@ -1,6 +1,6 @@ --- -title: "Tool use schema" -slug: "v2/docs/tool-use-schema" +title: "Tool use - FAQs" +slug: "v2/docs/tool-use-faqs" hidden: false description: >- @@ -11,4 +11,4 @@ keywords: "Cohere, text generation, LLMs, generative AI" createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- -TBD \ No newline at end of file +[[TODO - FAQs]] \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-multi-step.mdx b/fern/pages/v2/tool-use/tool-use-multi-step.mdx index 2df467b0b..c8390259b 100644 --- a/fern/pages/v2/tool-use/tool-use-multi-step.mdx +++ b/fern/pages/v2/tool-use/tool-use-multi-step.mdx @@ -11,4 +11,17 @@ keywords: "Cohere, text generation, LLMs, generative AI" createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- -TBD \ No newline at end of file +## Overview +[[TODO - describe multi-step tool use (agents)]] + +## State management +[[TODO - describe how the messages list construction for multi-step differs from single-step]] + +## Multi-step reasoning +[[TODO - example multi-step tool use wrt multi-step reasoning]] + +## Self-correction +[[TODO - example multi-step tool use wrt self-correction]] + +## Multi-step, parallel tool use +[[TODO - example of multi-step, parallel tool use]] \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index 232366298..4a771ae2a 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -20,6 +20,8 @@ This opens up a richer set of behaviors by leveraging data stored in tools, taki The Chat endpoint comes with built-in tool use capabilities such as function calling, multi-step reasoning, and citation generation. +Tool use overview + ## Setup First, import the Cohere library and create a client. @@ -38,6 +40,8 @@ The pre-requisite, or Step 0, before we can run a tool use workflow, is to defin - Creating the tool - Defining the tool schema +Tool definition + ### Creating the tool A tool can be any function that you create or extrenal services that return an object for a given input. Some examples: a web search engine, an email service, an SQL database, a vector database, a weather data service, a sports data service, or even another LLM. @@ -89,7 +93,7 @@ tools = [ ] ``` -## Running the tool use workflow +## Tool use workflow We can think of a tool use system as consisting of four components: @@ -108,13 +112,133 @@ At its most basic, these four components interact in a workflow through four ste Tool use workflow ### Step 1: Get user message -... +In the first step, we get the user's message and put it in the `messages` list with the `role` set to `user`. + +```python PYTHON +messages = [{"role": "user", "content": "What's the weather in Toronto?"}] +``` + + +Optional: If you want to define a system message, you can add it to the `messages` list with the `role` set to `system`. + +```python PYTHON +system_message = """## Task & Context +You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging. + +## Style Guide +Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling. +""" + +messages = [ + {"role": "system", "content": system_message}, + {"role": "user", "content": "What's the weather in Toronto?"}, +] +``` + + ### Step 2: Generate tool calls -... + +Next, we call the Chat endpoint to generate the list of tool calls. This is done by passing the parameters `model`, `messages`, and `tools` to the Chat endpoint. + +The endpoint will send back a list of tool calls to be made if the model determines that tools are required. If it does, it will return two types of messages: +- `tool_plan`: its reflection on the next steps it should take, given the user query. +- `tool_calls`: a list of tool calls to be made (if any), together with the tool call IDs. + +We then add this information to the `messages` list with the `role` set to `assistant`. + +```python PYTHON +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_calls": response.message.tool_calls, + "tool_plan": response.message.tool_plan, + } + ) + print(response.message.tool_calls) +``` + +```mdx wordWrap +[ToolCallV2(id='get_weather_776n8ctsgycn', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Toronto"}'))] +``` ### Step 3: Get tool results -... +During this step, we perform the function calling. We call the necessary tools based on the tool call payloads given by the endpoint. + +For each tool call, we append the tool results to the `tool_content` list with `type` set to `document` and `document` set to the tool results (in JSON string format). + +We then add this information to the `messages` list with the `role` set to `tool`, together with the tool call IDs that were generated in the previous step. + +```python PYTHON +import json + +if response.message.tool_calls: + for tc in response.message.tool_calls: + tool_result = functions_map[tc.function.name]( + **json.loads(tc.function.arguments) + ) + tool_content = [] + for data in tool_result: + tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) + # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated + messages.append( + {"role": "tool", "tool_call_id": tc.id, "content": tool_content} + ) +``` ### Step 4: Generate response and citations -... \ No newline at end of file +By this time, the tool call has already been executed, and the result has been returned to the LLM. + +In this step, we call the Chat endpoint to generate the response to the user, again by passing the parameters `model`, `messages` (which has now been updated with information fromthe tool calling and tool execution steps), and `tools`. + +The model generates a response to the user, grounded on the information provided by the tool. + +It also generates fine-grained citations, which are included out-of-the-box with the Command family of models. Here, we see the model generating two citations, one for each specific span in its response, where it uses the tool result to answer the question. + +```python PYTHON +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) +print(response.message.content[0].text) +``` +```mdx wordWrap +It is 20C in Toronto. +``` +```mdx wordWrap +start=6 end=9 text='20C' sources=[ToolSource(type='tool', id='get_weather_776n8ctsgycn:0', tool_output={'temperature': '20C'})] +``` + +## Parallel tool calling +[[TODO - demonstrate example of parallel tool calling]] + +## Directly answering +[[TODO - describe the scenario where the LLM decides not to use tools but instead directly answers the user]] + +## Forcing tool usage +[[TODO - describe the tool choice parameter]] + +## Response object + +### Tool calling step +[[TODO - describe the response object for tool calling step]] + +### Response generation step +[[TODO - describe the response object for response generation step]] + +## State management +[[TODO - describe the state management via the messages list - single and multi turn. show examples]] + +### Single turn +[[TODO - describe state management wrt single turn scenarios]] + +### Multi turn +[[TODO - describe state management wrt multi turn scenarios]] diff --git a/fern/pages/v2/tool-use/tool-use-function-calling.mdx b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx similarity index 62% rename from fern/pages/v2/tool-use/tool-use-function-calling.mdx rename to fern/pages/v2/tool-use/tool-use-parameter-types.mdx index c0193b508..f3f1c71d4 100644 --- a/fern/pages/v2/tool-use/tool-use-function-calling.mdx +++ b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx @@ -1,6 +1,6 @@ --- -title: "Function calling" -slug: "v2/docs/tool-use-function-calling" +title: "Tool use parameter types" +slug: "v2/docs/tool-use-parameter-types" hidden: false description: >- @@ -11,4 +11,6 @@ keywords: "Cohere, text generation, LLMs, generative AI" createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- -TBD \ No newline at end of file + +### Parameter types +[[TODO - copy and paste this page - https://docs.cohere.com/docs/parameter-types-in-tool-use]] \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index d5ba503e1..0480a13c8 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -11,4 +11,11 @@ keywords: "Cohere, text generation, LLMs, generative AI" createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- -TBD \ No newline at end of file +## Overview +[[TODO - describe streaming in the tool use context - e.g. event types]] + +## Tool calling step +[[TODO - describe handling streaming objects for the tool calling step]] + +## Response generation step +[[TODO - describe handling streaming objects for the response generation step]] \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx new file mode 100644 index 000000000..a654ba48d --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx @@ -0,0 +1,35 @@ +--- +title: "Tool use - tool definition" +slug: "v2/docs/tool-use-definition" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- +## Overview +[[TODO - elaborate from the basic usage section - which gave the simplest possible way to define a tool]] + +## Tool creation + +### Example: Custom functions +[[TODO - an example of a tool with basic working logic - e.g. sales database]] + +### Example: External services +[[TODO - an example of a tool with basic working logic - e.g. web search]] + +## Tool schema + +### JSON schema +[[TODO - overview of the JSON schema]] + +### Tool schema prompting +[[TODO - how to write good tool schema e.g. descriptions, etc]] + +## Structured outputs + +[[TODO - how to use the strict_tools parameter]] \ No newline at end of file diff --git a/fern/v2.yml b/fern/v2.yml index a8597b4ac..7f1c50b72 100644 --- a/fern/v2.yml +++ b/fern/v2.yml @@ -83,18 +83,18 @@ navigation: contents: - page: Basic usage path: pages/v2/tool-use/tool-use-overview.mdx - - page: Function calling - path: pages/v2/tool-use/tool-use-function-calling.mdx - - page: Tool schema - path: pages/v2/tool-use/tool-use-schema.mdx - - page: Structured outputs - path: pages/v2/tool-use/tool-use-structured-outputs.mdx - - page: Multi-step tool use (agents) + - page: Multi-step (agents) path: pages/v2/tool-use/tool-use-multi-step.mdx - - page: Citations - path: pages/v2/tool-use/tool-use-citations.mdx + - page: Tool definition + path: pages/v2/tool-use/tool-use-tool-definition.mdx + - page: Parameter types + path: pages/v2/tool-use/tool-use-parameter-types.mdx - page: Streaming path: pages/v2/tool-use/tool-use-streaming.mdx + - page: Citations + path: pages/v2/tool-use/tool-use-citations.mdx + - page: FAQs + path: pages/v2/tool-use/tool-use-faqs.mdx # - section: Tool Use # path: pages/v2/text-generation/tools.mdx From a248a78ecf7d04139d05b106fb80a08ea04d8b8c Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Tue, 28 Jan 2025 12:55:31 +0800 Subject: [PATCH 03/11] update tool use overview --- fern/pages/v2/tool-use/tool-use-overview.mdx | 250 ++++++++++++++++-- .../v2/tool-use/tool-use-tool-definition.mdx | 2 +- 2 files changed, 230 insertions(+), 22 deletions(-) diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index 4a771ae2a..cf2478f4c 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -44,7 +44,7 @@ The pre-requisite, or Step 0, before we can run a tool use workflow, is to defin ### Creating the tool -A tool can be any function that you create or extrenal services that return an object for a given input. Some examples: a web search engine, an email service, an SQL database, a vector database, a weather data service, a sports data service, or even another LLM. +A tool can be any function that you create or external services that return an object for a given input. Some examples: a web search engine, an email service, an SQL database, a vector database, a weather data service, a sports data service, or even another LLM. In this example, we define a `get_weather` function that returns the temperature for a given query, which is the location. You can implement any logic here, but to simplify the example, here we are hardcoding the return value to be the same for all queries. @@ -112,7 +112,7 @@ At its most basic, these four components interact in a workflow through four ste Tool use workflow ### Step 1: Get user message -In the first step, we get the user's message and put it in the `messages` list with the `role` set to `user`. +In the first step, we get the user's message and append it to the `messages` list with the `role` set to `user`. ```python PYTHON messages = [{"role": "user", "content": "What's the weather in Toronto?"}] @@ -145,7 +145,7 @@ The endpoint will send back a list of tool calls to be made if the model determi - `tool_plan`: its reflection on the next steps it should take, given the user query. - `tool_calls`: a list of tool calls to be made (if any), together with the tool call IDs. -We then add this information to the `messages` list with the `role` set to `assistant`. +We then append this information to the `messages` list with the `role` set to `assistant`. ```python PYTHON response = co.chat( @@ -158,15 +158,26 @@ if response.message.tool_calls: messages.append( { "role": "assistant", - "tool_calls": response.message.tool_calls, "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, } ) + print(response.message.tool_plan, "\n") print(response.message.tool_calls) ``` ```mdx wordWrap -[ToolCallV2(id='get_weather_776n8ctsgycn', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Toronto"}'))] +I will search for the weather in Toronto. + +[ + ToolCallV2( + id="get_weather_1byjy32y4hvq", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Toronto"}' + ), + ) +] ``` ### Step 3: Get tool results @@ -174,7 +185,7 @@ During this step, we perform the function calling. We call the necessary tools b For each tool call, we append the tool results to the `tool_content` list with `type` set to `document` and `document` set to the tool results (in JSON string format). -We then add this information to the `messages` list with the `role` set to `tool`, together with the tool call IDs that were generated in the previous step. +We then append this information to the `messages` list with the `role` set to `tool`, together with the tool call IDs that were generated in the previous step. ```python PYTHON import json @@ -200,7 +211,7 @@ In this step, we call the Chat endpoint to generate the response to the user, ag The model generates a response to the user, grounded on the information provided by the tool. -It also generates fine-grained citations, which are included out-of-the-box with the Command family of models. Here, we see the model generating two citations, one for each specific span in its response, where it uses the tool result to answer the question. +We then append the response to the `messages` list with the `role` set to `assistant`. ```python PYTHON response = co.chat( @@ -208,37 +219,234 @@ response = co.chat( messages=messages, tools=tools ) + +messages.append( + {"role": "assistant", "content": response.message.content[0].text} +) + print(response.message.content[0].text) ``` ```mdx wordWrap -It is 20C in Toronto. +It's 20°C in Toronto. +``` +It also generates fine-grained citations, which are included out-of-the-box with the Command family of models. Here, we see the model generating two citations, one for each specific span in its response, where it uses the tool result to answer the question. + +```python PYTHON +print(response.message.citations) ``` ```mdx wordWrap -start=6 end=9 text='20C' sources=[ToolSource(type='tool', id='get_weather_776n8ctsgycn:0', tool_output={'temperature': '20C'})] +[Citation(start=5, end=9, text='20°C', sources=[ToolSource(type='tool', id='get_weather_1byjy32y4hvq:0', tool_output={'temperature': '20C'})], type='TEXT_CONTENT')] ``` ## Parallel tool calling -[[TODO - demonstrate example of parallel tool calling]] +The LLM can also determine that more than one tool call is required, where it will call multiple tools in parallel. This can be calling the same tool multiple times or different tools for any number of calls. + +In the example below, the user asks for the weather in Toronto and New York. This requires calling the `get_weather` function twice, one for each location. This is reflected in the model's response, where two parallel tool calls are generated. + +```python PYTHON +messages = [{"role": "user", "content": "What's the weather in Toronto and New York?"}] + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) +``` +```mdx wordWrap +I will search for the weather in Toronto and New York. + +[ + ToolCallV2( + id="get_weather_9b0nr4kg58a8", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Toronto"}' + ), + ), + ToolCallV2( + id="get_weather_0qq0mz9gwnqr", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"New York"}' + ), + ), +] +``` ## Directly answering -[[TODO - describe the scenario where the LLM decides not to use tools but instead directly answers the user]] +A key attribute of tool use systems is the model’s ability to choose the right tools for a task. This includes the ability to decide to *not* use any tool, and instead, respond to a user message directly. + +In the example below, the user asks for a simple arithmetic question. The model determines that it does not need to use any of the available tools (only one, `get_weather`, in this case), and instead, directly answers the user. + +```python PYTHON +messages = [{"role": "user", "content": "What's 2+2?"}] + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) + +else: + print(response.message.content[0].text) +``` +```mdx wordWrap +The answer to 2+2 is 4. +``` ## Forcing tool usage -[[TODO - describe the tool choice parameter]] -## Response object +This feature is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models. -### Tool calling step -[[TODO - describe the response object for tool calling step]] +As we saw in the previous examples, during the tool calling step, the model may decide to either: +- make tool call(s) +- or, respond to a user message directly. -### Response generation step -[[TODO - describe the response object for response generation step]] +You can, however, force the model to choose one of these options. This is done via the `tool_choice` parameter. +- You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`. +- Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`. +By default, if you don’t specify the `tool_choice` parameter, then the model will decide between making tool calls or responding directly. + +```python PYTHON {5} +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools, + tool_choice="REQUIRED" # optional, to force tool calls + # tool_choice="NONE" # optional, to force a direct response +) +``` ## State management -[[TODO - describe the state management via the messages list - single and multi turn. show examples]] +This section gives a more detailed look at how the state is managed via the `messages` list. + +As described in the [tool use workflow](#tool-use-workflow) section above, at each step of the workflow, the endpoint requires that we append specific types of information to the `messages` list. This is to ensure that the model has the necessary context to generate its response at a given point. ### Single turn -[[TODO - describe state management wrt single turn scenarios]] +In summary, each single turn of a conversation that involves tool calling consists of: +1. a `user` message containing the user message (`content`) +2. an `assistant` message, containing the tool calling information (`tool_plan` and `tool_calls`) +3. a `tool` message, containing the tool results (`tool_call_id` and `content`) +4. a final `assistant` message, containing the response to the user (`content`) + +These correspond to the four steps described in the [tool use workflow](#tool-use-workflow) section above. + +The following is the list of messages from the example in the [tool use workflow](#tool-use-workflow) section. + +```python PYTHON +for message in messages: + print(message, "\n") +``` + +```json +{ + "role": "user", + "content": "What's the weather in Toronto?" +} + +{ + "role": "assistant", + "tool_plan": "I will search for the weather in Toronto.", + "tool_calls": [ + ToolCallV2( + id="get_weather_1byjy32y4hvq", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Toronto"}' + ), + ) + ], +} + +{ + "role": "tool", + "tool_call_id": "get_weather_1byjy32y4hvq", + "content": [{"type": "document", "document": {"data": '{"temperature": "20C"}'}}], +} + +{ + "role": "assistant", + "content": "It's 20°C in Toronto." +} +``` + +Note that this sequence of messages will be different in these scenarios: +- **Multi-step**: In a multi-step tool use scenario, instead of just one occurence of assistant-tool messages (items 2 and 3 above), there will be a sequence of assistant-tool messages to reflect the multiple steps of tool calling involved. This is further described in the [multi-step tool use documentation]([[TODO - add link]]) +- **Directly answering**: When the model decides to respond directly to the user, there will be no items 2 and 3 above (the tool calling and tool response messages). Instead, the final `assistant` message will contain the model's direct response to the user. + +### Chatbots + +When building chatbots, we'll need to maintain the state of a conversation over multiple turns. In this case, we can keep appending to the `messages` list with the sequence of described in the [state management](#state-management) section above. + +As an example, here's the messages list from the first conversation turn. +```python PYTHON +from cohere import ToolCallV2, ToolCallV2Function + +messages = [ + {"role": "user", "content": "What's the weather in Toronto?"}, + { + "role": "assistant", + "tool_plan": "I will search for the weather in Toronto.", + "tool_calls": [ + ToolCallV2( + id="get_weather_1byjy32y4hvq", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Toronto"}' + ), + ) + ], + }, + { + "role": "tool", + "tool_call_id": "get_weather_1byjy32y4hvq", + "content": [ + {"type": "document", "document": {"data": '{"temperature": "20C"}'}} + ], + }, + {"role": "assistant", "content": "It's 20°C in Toronto."}, +] +``` +Then, given a follow-up user message, the model correctly infers that the context of the user's message is about the weather. + +```python PYTHON +messages.append({"role": "user", "content": "What about London?"}) + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) +``` +```mdx wordWrap +I will search for the weather in London. + +[ToolCallV2(id='get_weather_8hwpm7d4wr14', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"London"}'))] +``` -### Multi turn -[[TODO - describe state management wrt multi turn scenarios]] diff --git a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx index a654ba48d..fe9472908 100644 --- a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx +++ b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx @@ -17,7 +17,7 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ## Tool creation ### Example: Custom functions -[[TODO - an example of a tool with basic working logic - e.g. sales database]] +[[TODO - an example of a tool with basic working logic - e.g. sales database - example more than 1 tool]] ### Example: External services [[TODO - an example of a tool with basic working logic - e.g. web search]] From aba6b9f6d8d6ba895d40f1cea7de18a4fc850e33 Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Mon, 3 Feb 2025 16:26:56 +0800 Subject: [PATCH 04/11] add more sections --- fern/pages/v2/tool-use/tool-use-citations.mdx | 272 +++++++- fern/pages/v2/tool-use/tool-use-overview.mdx | 173 +---- .../v2/tool-use/tool-use-parameter-types.mdx | 580 ++++++++++++++++- fern/pages/v2/tool-use/tool-use-streaming.mdx | 347 +++++++++- .../v2/tool-use/tool-use-tool-definition.mdx | 8 +- ...step.mdx => tool-use-usage-multi-step.mdx} | 11 +- .../v2/tool-use/tool-use-usage-patterns.mdx | 598 ++++++++++++++++++ fern/v2.yml | 6 +- 8 files changed, 1820 insertions(+), 175 deletions(-) rename fern/pages/v2/tool-use/{tool-use-multi-step.mdx => tool-use-usage-multi-step.mdx} (53%) create mode 100644 fern/pages/v2/tool-use/tool-use-usage-patterns.mdx diff --git a/fern/pages/v2/tool-use/tool-use-citations.mdx b/fern/pages/v2/tool-use/tool-use-citations.mdx index 93481ebcd..63a75f89d 100644 --- a/fern/pages/v2/tool-use/tool-use-citations.mdx +++ b/fern/pages/v2/tool-use/tool-use-citations.mdx @@ -13,19 +13,279 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- ## Accessing citations -[[TODO - describe citations and how to access them, what do they contain]] + +The Chat endpoint generates fine-grained citations for its tool use response. This capability is included out-of-the-box with the Command family of models. + +The following sections describe how to access the citations in both the non-streaming and streaming modes. ### Non-streaming -[[TODO - show how to access citations from the response object in a non-streaming scenario]] +First, define the tool and its associated schema. + +```python PYTHON +# ! pip install -U cohere +import cohere +import json + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys + +def get_weather(location): + temperature = { + "bern": "22°C", + "madrid": "24°C", + "brasilia": "28°C" + } + loc = location.lower() + if loc in temperature: + return [{"temperature": {loc: temperature[loc]}}] + return [{"temperature": {loc: "Unknown"}}] + +functions_map = {"get_weather": get_weather} + +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "the location to get weather. Provide just the city name without the country name.", + } + }, + "required": ["location"], + }, + }, + } +] +``` + +Next, run the tool calling and execution steps. + +```python PYTHON +messages = [{"role": "user", "content": "What's the weather in Madrid and Brasilia?"}] + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + + for tc in response.message.tool_calls: + tool_result = functions_map[tc.function.name]( + **json.loads(tc.function.arguments) + ) + tool_content = [] + for data in tool_result: + tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) + messages.append( + {"role": "tool", "tool_call_id": tc.id, "content": tool_content} + ) +``` + +In the non-streaming mode (using `chat` to generate the model response), the citations are provided in the `message.citations` field of the response object. + +Each citation object contains: +- `start` and `end`: the start and end indices of the text that cites a source(s) +- `text`: its corresponding span of text +- `sources`: the source(s) that it references + +```python PYTHON +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +messages.append( + {"role": "assistant", "content": response.message.content[0].text} +) + +print(response.message.content[0].text) + +for citation in response.message.citations: + print(citation, "\n") +``` + +```mdx wordWrap +It is currently 24°C in Madrid and 28°C in Brasilia. + +start=16 end=20 text='24°C' sources=[ToolSource(type='tool', id='get_weather_14brd1n2kfqj:0', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' + +start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='get_weather_vdr9cvj619fk:0', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' +``` ### Streaming -[[TODO - show how to access citations from the response object in a streaming scenario]] +In a streaming scenario (using `chat_stream` to generate the model response), the citations are provided in the `citation-start` events. + +Each citation object contains the same fields as the [non-streaming scenario](###non-streaming). + +```python PYTHON +response = co.chat_stream( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +response_text = "" +citations = [] +for chunk in response: + if chunk: + if chunk.type == "content-delta": + response_text += chunk.delta.message.content.text + print(chunk.delta.message.content.text, end="") + if chunk.type == "citation-start": + citations.append(chunk.delta.message.citations) + +messages.append( + {"role": "assistant", "content": response_text} +) + +for citation in citations: + print(citation, "\n") +``` + +```mdx wordWrap +It is currently 24°C in Madrid and 28°C in Brasilia. + +start=16 end=20 text='24°C' sources=[ToolSource(type='tool', id='get_weather_dkf0akqdazjb:0', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' + +start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='get_weather_gh65bt2tcdy1:0', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' +``` ## Citation modes -[[TODO - describe citation modes - fast, accurate, etc]] +When ruuning tool use in streaming mode, it’s possible to configure how citations are generated and presented. You can choose between fast citations or accurate citations, depending on your latency and precision needs: ### Accurate citations -[[TODO - show code example of fast citations]] +The model produces its answer first, and then, after the entire response is generated, it provides citations that map to specific segments of the response text. This approach may incur slightly higher latency, but it ensures the citation indices are more precisely aligned with the final text segments of the model’s answer. This is the default option, or you can explicitly specify it by adding the `citation_options={"mode": "accurate"}` argument in the API call. + +Here is an example. To make things concise, let's start with a pre-defined list of `messages` with the user query, tool calling, and tool results are already available. + +```python PYTHON +# ! pip install -U cohere +import cohere +import json + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys + +from cohere import ToolCallV2, ToolCallV2Function + +messages = [ + {"role": "user", "content": "What's the weather in Madrid and Brasilia?"}, + { + "role": "assistant", + "tool_plan": "I will search for the weather in Madrid and Brasilia.", + "tool_calls": [ + ToolCallV2( + id="get_weather_dkf0akqdazjb", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Madrid"}' + ), + ), + ToolCallV2( + id="get_weather_gh65bt2tcdy1", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Brasilia"}' + ), + ), + ], + }, + { + "role": "tool", + "tool_call_id": "get_weather_dkf0akqdazjb", + "content": [ + { + "type": "document", + "document": { + "data": '{"temperature": {"madrid": "24\\u00b0C"}}', + "id" : "1" + }, + } + ], + }, + { + "role": "tool", + "tool_call_id": "get_weather_gh65bt2tcdy1", + "content": [ + { + "type": "document", + "document": { + "data": '{"temperature": {"brasilia": "28\\u00b0C"}}', + "id" : "2" + }, + } + ], + }, +] +``` + +With the `citation_options` mode set to `accurate`, we get the citations after the entire response is generated. + +```python PYTHON +response = co.chat_stream( + model="command-r-plus-08-2024", + messages=messages, + tools=tools, + citation_options={"mode": "accurate"} +) + +response_text = "" +citations = [] +for chunk in response: + if chunk: + if chunk.type == "content-delta": + response_text += chunk.delta.message.content.text + print(chunk.delta.message.content.text, end="") + if chunk.type == "citation-start": + citations.append(chunk.delta.message.citations) + +print("\n") +for citation in citations: + print(citation, "\n") +``` +```mdx wordWrap +It is currently 24°C in Madrid and 28°C in Brasilia. + +start=16 end=20 text='24°C' sources=[ToolSource(type='tool', id='1', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' + +start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='2', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' +``` ### Fast citations -[[TODO - show code example of fast citations]] +The model generates citations inline, as the response is being produced. In streaming mode, you will see citations injected at the exact moment the model uses a particular piece of external context. This approach provides immediate traceability at the expense of slightly less precision in citation relevance. You can specify it by adding the `citation_options={"mode": "fast"}` argument in the API call. + +Here is an example using the same list of pre-defined`messages` as the above. With the `citation_options` mode set to `fast`, we get the citations inlineas the model generates the response. + +```python PYTHON +response = co.chat_stream( + model="command-r-plus-08-2024", + messages=messages, + tools=tools, + citation_options={"mode": "fast"} +) + +response_text = "" +for chunk in response: + if chunk: + if chunk.type == "content-delta": + response_text += chunk.delta.message.content.text + print(chunk.delta.message.content.text, end="") + if chunk.type == "citation-start": + print(f" [{chunk.delta.message.citations.sources[0].id}]", end="") +``` +```mdx wordWrap +It is currently 24°C [1] in Madrid and 28°C [2] in Brasilia. +``` \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index cf2478f4c..e0368b48c 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -59,7 +59,7 @@ functions_map = {"get_weather": get_weather} ### Defining the tool schema -We also need to define the tool schemas in a format that can be passed to the Chat endpoint. The schema follows the JSON Schema specification and must contain the following fields: +We also need to define the tool schemas in a format that can be passed to the Chat endpoint. The schema follows the [JSON Schema specification](https://json-schema.org/) and must contain the following fields: - `name`: the name of the tool. - `description`: a description of what the tool is and what it is used for. - `parameters`: a list of parameters that the tool accepts. For each parameter, we need to define the following fields: @@ -69,7 +69,7 @@ We also need to define the tool schemas in a format that can be passed to the Ch - `description`: a description of what the parameter is and what it is used for. - `required`: a list of required parameters from the `properties` field. -This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on the information that it contains. Therefore, the more descriptive and clear the schema, the more likely the LLM will make the right tool call decisions. +This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on the information that it contains. Therefore, the more descriptive and clear the schema, the more likely the LLM will make the right tool call decisions. In a typical development cycle, some fields such as `name`, `description`, and `properties` will likely require a few rounds of iterations in order to get the best results (a similar idea to prompt engineering). ```python PYTHON tools = [ @@ -238,106 +238,12 @@ print(response.message.citations) [Citation(start=5, end=9, text='20°C', sources=[ToolSource(type='tool', id='get_weather_1byjy32y4hvq:0', tool_output={'temperature': '20C'})], type='TEXT_CONTENT')] ``` -## Parallel tool calling -The LLM can also determine that more than one tool call is required, where it will call multiple tools in parallel. This can be calling the same tool multiple times or different tools for any number of calls. -In the example below, the user asks for the weather in Toronto and New York. This requires calling the `get_weather` function twice, one for each location. This is reflected in the model's response, where two parallel tool calls are generated. - -```python PYTHON -messages = [{"role": "user", "content": "What's the weather in Toronto and New York?"}] - -response = co.chat( - model="command-r-plus-08-2024", - messages=messages, - tools=tools -) - -if response.message.tool_calls: - messages.append( - { - "role": "assistant", - "tool_plan": response.message.tool_plan, - "tool_calls": response.message.tool_calls, - } - ) - print(response.message.tool_plan, "\n") - print(response.message.tool_calls) -``` -```mdx wordWrap -I will search for the weather in Toronto and New York. - -[ - ToolCallV2( - id="get_weather_9b0nr4kg58a8", - type="function", - function=ToolCallV2Function( - name="get_weather", arguments='{"location":"Toronto"}' - ), - ), - ToolCallV2( - id="get_weather_0qq0mz9gwnqr", - type="function", - function=ToolCallV2Function( - name="get_weather", arguments='{"location":"New York"}' - ), - ), -] -``` - -## Directly answering -A key attribute of tool use systems is the model’s ability to choose the right tools for a task. This includes the ability to decide to *not* use any tool, and instead, respond to a user message directly. - -In the example below, the user asks for a simple arithmetic question. The model determines that it does not need to use any of the available tools (only one, `get_weather`, in this case), and instead, directly answers the user. - -```python PYTHON -messages = [{"role": "user", "content": "What's 2+2?"}] - -response = co.chat( - model="command-r-plus-08-2024", - messages=messages, - tools=tools -) - -if response.message.tool_calls: - print(response.message.tool_plan, "\n") - print(response.message.tool_calls) - -else: - print(response.message.content[0].text) -``` -```mdx wordWrap -The answer to 2+2 is 4. -``` - -## Forcing tool usage - -This feature is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models. - -As we saw in the previous examples, during the tool calling step, the model may decide to either: -- make tool call(s) -- or, respond to a user message directly. - -You can, however, force the model to choose one of these options. This is done via the `tool_choice` parameter. -- You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`. -- Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`. - -By default, if you don’t specify the `tool_choice` parameter, then the model will decide between making tool calls or responding directly. - -```python PYTHON {5} -response = co.chat( - model="command-r-plus-08-2024", - messages=messages, - tools=tools, - tool_choice="REQUIRED" # optional, to force tool calls - # tool_choice="NONE" # optional, to force a direct response -) -``` ## State management This section gives a more detailed look at how the state is managed via the `messages` list. As described in the [tool use workflow](#tool-use-workflow) section above, at each step of the workflow, the endpoint requires that we append specific types of information to the `messages` list. This is to ensure that the model has the necessary context to generate its response at a given point. -### Single turn In summary, each single turn of a conversation that involves tool calling consists of: 1. a `user` message containing the user message (`content`) 2. an `assistant` message, containing the tool calling information (`tool_plan` and `tool_calls`) @@ -385,68 +291,25 @@ for message in messages: } ``` -Note that this sequence of messages will be different in these scenarios: -- **Multi-step**: In a multi-step tool use scenario, instead of just one occurence of assistant-tool messages (items 2 and 3 above), there will be a sequence of assistant-tool messages to reflect the multiple steps of tool calling involved. This is further described in the [multi-step tool use documentation]([[TODO - add link]]) -- **Directly answering**: When the model decides to respond directly to the user, there will be no items 2 and 3 above (the tool calling and tool response messages). Instead, the final `assistant` message will contain the model's direct response to the user. - -### Chatbots +The sequence of messages is represented in the diagram below. -When building chatbots, we'll need to maintain the state of a conversation over multiple turns. In this case, we can keep appending to the `messages` list with the sequence of described in the [state management](#state-management) section above. +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query)
"] + B["
Assistant (tool call)
"] + C["
Tool (tool result)
"] + D["
Assistant (response)
"] -As an example, here's the messages list from the first conversation turn. -```python PYTHON -from cohere import ToolCallV2, ToolCallV2Function + A -.-> B + B -.-> C + C -.-> D -messages = [ - {"role": "user", "content": "What's the weather in Toronto?"}, - { - "role": "assistant", - "tool_plan": "I will search for the weather in Toronto.", - "tool_calls": [ - ToolCallV2( - id="get_weather_1byjy32y4hvq", - type="function", - function=ToolCallV2Function( - name="get_weather", arguments='{"location":"Toronto"}' - ), - ) - ], - }, - { - "role": "tool", - "tool_call_id": "get_weather_1byjy32y4hvq", - "content": [ - {"type": "document", "document": {"data": '{"temperature": "20C"}'}} - ], - }, - {"role": "assistant", "content": "It's 20°C in Toronto."}, -] + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 + style C fill:#fff,stroke:#000,color:#000 + style D fill:#fff,stroke:#000,color:#000 ``` -Then, given a follow-up user message, the model correctly infers that the context of the user's message is about the weather. - -```python PYTHON -messages.append({"role": "user", "content": "What about London?"}) - -response = co.chat( - model="command-r-plus-08-2024", - messages=messages, - tools=tools -) -if response.message.tool_calls: - messages.append( - { - "role": "assistant", - "tool_plan": response.message.tool_plan, - "tool_calls": response.message.tool_calls, - } - ) - print(response.message.tool_plan, "\n") - print(response.message.tool_calls) -``` -```mdx wordWrap -I will search for the weather in London. - -[ToolCallV2(id='get_weather_8hwpm7d4wr14', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"London"}'))] -``` +Note that this sequence of messages represents a basic usage pattern in tool use. The next page [[TODO - add link]] will describe how this will be adapted for other scenarios. diff --git a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx index f3f1c71d4..7ce6f1deb 100644 --- a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx +++ b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx @@ -12,5 +12,581 @@ createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- -### Parameter types -[[TODO - copy and paste this page - https://docs.cohere.com/docs/parameter-types-in-tool-use]] \ No newline at end of file +## Structured Outputs (Tools) + +The [Structured Outputs](https://docs.cohere.com/docs/structured-outputs) feature guarantees that an LLM’s response will strictly follow a schema specified by the user. + +While this feature is supported in two scenarios (JSON and tools), this page will focus on the tools scenario. + +### Usage + +When you use the Chat API with `tools`, setting the `strict_tools` parameter to `True` will guarantee that every generated tool call follows the specified tool schema. + +Concretely, this means: + +- No hallucinated tool names +- No hallucinated tool parameters +- Every `required` parameter is included in the tool call +- All parameters produce the requested data types + +With `strict_tools` enabled, the API will ensure that the tool names and tool parameters are generated according to the tool definitions. This eliminates tool name and parameter hallucinations, ensures that each parameter matches the specified data type, and that all required parameters are included in the model response. + +Additionally, this results in faster development. You don’t need to spend a lot of time prompt engineering the model to avoid hallucinations. + +In the example below, we create a tool that can retrieve weather data for a given location. The tool is called `get_weather` which contains a parameter called `location`. We then invoke the Chat API with `strict_tools` set to `True` to ensure that the generated tool calls always include the correct function and parameter names. + +When the `strict_tools` parameter is set to `True`, you can define a maximum of 200 fields across all tools being passed to an API call. + +```python PYTHON {4} +response = co.chat(model="command-r-plus-08-2024", + messages=[{"role": "user", "content": "What's the weather in Toronto?"}], + tools=tools, + strict_tools=True) + +print(response.message.tool_calls) +``` + + +### Important notes + +When using `strict_tools`, the following notes apply: +- This parameter is only supported in Chat API V2 via the strict_tools parameter (not API V1). +- You must specify at least one `required` parameter. Tools with only optional parameters are not supported in this mode. +- You can define a maximum of 200 fields across all tools in a single Chat API call. + +## Supported parameter types + +Structured Outputs supports a subset of the JSON Schema specification. Refer to the [Structured Outputs documentation](https://docs.cohere.com/docs/structured-outputs#parameter-types-support) for the list of supported and unsupported parameters. + +## Usage examples + +This section provides usage examples of the JSON Schema [parameters that are supported](https://docs.cohere.com/v2/docs/tool-use#structured-outputs-tools) in Structured Outputs (Tools). + + + +The examples on this page each provide a tool schema and a `message` (the user message). To get an output, pass those values to a Chat endpoint call, as shown in the helper code below. + +```python PYTHON PYTHON +import cohere +co = cohere.ClientV2(api_key="YOUR API KEY") + +res = co.chat( + # The model name. Example: command-r-plus-08-2024 + model="MODEL_NAME", + # The user message. Optional - you can first add a `system_message` role + messages=[ + { + "role": "user", + "content": message, + } + ], + # The tool schema that you define + tools=tools, + # This guarantees that the output will adhere to the schema + strict_tools=True, + # Typically, you'll need a low temperature for more deterministic outputs + temperature=0 +) + +for tc in response.message.tool_calls: + print(f"{tc.function.name} | Parameters: {tc.function.arguments}") +``` + + + +### Basic types + +#### String + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "The location to get weather.", + } + }, + "required": ["location"], + }, + }, + }, +] + +message = "What's the weather in Toronto?" +``` + +Example output: + +```mdx wordWrap +get_weather +{ + "location": "Toronto" +} +``` + +#### Integer + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "add_numbers", + "description": "Adds two numbers", + "parameters": { + "type": "object", + "properties": { + "first_number": { + "type": "integer", + "description": "The first number to add.", + }, + "second_number": { + "type": "integer", + "description": "The second number to add.", + }, + }, + "required": ["first_number", "second_number"], + }, + }, + } +] + +message = "What is five plus two" +``` + +Example output: + +```mdx wordWrap +add_numbers +{ + "first_number": 5, + "second_number": 2 +} +``` + +#### Float + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "add_numbers", + "description": "Adds two numbers", + "parameters": { + "type": "object", + "properties": { + "first_number": { + "type": "number", + "description": "The first number to add.", + }, + "second_number": { + "type": "number", + "description": "The second number to add.", + }, + }, + "required": ["first_number", "second_number"], + }, + }, + } +] + +message = "What is 5.3 plus 2" +``` + +Example output: + +```mdx wordWrap +add_numbers +{ + "first_number": 5.3, + "second_number": 2 +} +``` + +#### Boolean + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "reserve_tickets", + "description": "Reserves a train ticket", + "parameters": { + "type": "object", + "properties": { + "quantity": { + "type": "integer", + "description": "The quantity of tickets to reserve.", + }, + "trip_protection": { + "type": "boolean", + "description": "Indicates whether to add trip protection.", + }, + }, + "required": ["quantity", "trip_protection"], + }, + }, + } +] + +message = "Book me 2 tickets. I don't need trip protection." +``` + +Example output: + +```mdx wordWrap +reserve_tickets +{ + "quantity": 2, + "trip_protection": false +} +``` + +### Array + +#### With specific types + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "locations": { + "type": "array", + "items": {"type": "string"}, + "description": "The locations to get weather.", + } + }, + "required": ["locations"], + }, + }, + }, +] + +message = "What's the weather in Toronto and New York?" +``` + +Example output: + +```mdx wordWrap +get_weather +{ + "locations": [ + "Toronto", + "New York" + ] +} +``` + +#### Without specific types + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "locations": { + "type": "array", + "description": "The locations to get weather.", + } + }, + "required": ["locations"], + }, + }, + }, +] + +message = "What's the weather in Toronto and New York?" +``` + +Example output: + +```mdx wordWrap +get_weather +{ + "locations": [ + "Toronto", + "New York" + ] +} +``` + +#### Lists of lists + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "maxPoints", + "description": "Finds the maximum number of points on a line.", + "parameters": { + "type": "object", + "properties": { + "points": { + "type": "array", + "description": "The list of points. Points are 2 element lists [x, y].", + "items": { + "type": "array", + "items": {"type": "integer"}, + "description": "A point represented by a 2 element list [x, y].", + }, + } + }, + "required": ["points"], + }, + }, + } +] + +message = "Please provide the maximum number of collinear points for this set of coordinates - [[1,1],[2,2],[3,4],[5,5]]." +``` + +Example output: + +```mdx wordWrap +maxPoints +{ + "points": [ + [1,1], + [2,2], + [3,4], + [5,5] + ] +} +``` + +### Others + +#### Nested objects + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "search_furniture_products", + "description": "Searches for furniture products given the user criteria.", + "parameters": { + "type": "object", + "properties": { + "product_type": { + "type": "string", + "description": "The type of the product to search for.", + }, + "features": { + "type": "object", + "properties": { + "material": {"type": "string"}, + "style": {"type": "string"}, + }, + "required": ["style"], + }, + }, + "required": ["product_type"], + }, + }, + } +] + +message = "I'm looking for a dining table made of oak in Scandinavian style." +``` + +Example output: + +```mdx wordWrap +search_furniture_products +{ + "features": { + "material": "oak", + "style": "Scandinavian" + }, + "product_type": "dining table" +} +``` + +#### Enums + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "fetch_contacts", + "description": "Fetch a contact by type", + "parameters": { + "type": "object", + "properties": { + "contact_type": { + "type": "string", + "description": "The type of contact to fetch.", + "enum": ["customer", "supplier"], + } + }, + "required": ["contact_type"], + }, + }, + } +] + +message = "Give me vendor contacts." +``` + +Example output: + +```mdx wordWrap +fetch_contacts +{ + "contact_type": "supplier" +} +``` + +#### Const + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "The location to get weather.", + }, + "country": { + "type": "string", + "description": "The country for the weather lookup", + "const": "Canada", + }, + }, + "required": ["location", "country"], + }, + }, + }, +] + +message = "What's the weather in Toronto and Vancouver?" +``` + +Example output: + +```mdx wordWrap +get_weather +{ + "country": "Canada", + "location": "Toronto" +} +--- +get_weather +{ + "country": "Canada", + "location": "Vancouver" +} +--- +``` + +#### Pattern + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "query_product_by_sku", + "description": "Queries products by SKU pattern", + "parameters": { + "type": "object", + "properties": { + "sku_pattern": { + "type": "string", + "description": "Pattern to match SKUs", + "pattern": "[A-Z]{3}[0-9]{4}", + } + }, + "required": ["sku_pattern"], + }, + }, + } +] + +message = "Check the stock level of this product - 7374 hgY" +``` + +Example output: + +```mdx wordWrap +query_product_by_sku +{ + "sku_pattern": "HGY7374" +} +``` + +#### Format + +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "book_hotel", + "description": "Books a hotel room for a specific check-in date", + "parameters": { + "type": "object", + "properties": { + "hotel_name": { + "type": "string", + "description": "Name of the hotel", + }, + "check_in_date": { + "type": "string", + "description": "Check-in date for the hotel", + "format": "date", + }, + }, + "required": ["hotel_name", "check_in_date"], + }, + }, + } +] + +message = "Book a room at the Grand Hotel with check-in on Dec 2 2024" +``` + +Example output: + +```mdx wordWrap +book_hotel +{ + "check_in_date": "2024-12-02", + "hotel_name": "Grand Hotel" +} +``` + + + + + + diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index 0480a13c8..05d147e47 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -12,10 +12,347 @@ createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- ## Overview -[[TODO - describe streaming in the tool use context - e.g. event types]] -## Tool calling step -[[TODO - describe handling streaming objects for the tool calling step]] +To enable response streaming in tool use, use the `chat_stream` endpoint instead of `chat` when making the API call. -## Response generation step -[[TODO - describe handling streaming objects for the response generation step]] \ No newline at end of file +You can stream responses in both the tool calling and the response generation steps. This allows your application to receive token streams as the model plans and executes tool calls and generates its response using the tool results. + +## Events stream + +In tool use, the events streamed by the endpoint follows the structure of a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events) but contains additional events for tool calling and response generation together with the corresponding contents. This section describes the stream of events and their contents. + +### Tool calling step + +#### Event types +`message-start` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +`tool-plan-delta` + +Emitted when the next token of the tool plan is generated. + +`tool-call-start` + +Emitted when the model generates tool calls that require actioning upon. The event contains a list of `tool_calls` containing the tool name and tool call ID of the tool. + +`tool-call-delta` + +Emitted when the next token of the the tool call is generated. + +`tool-call-end` + +Emitted when the tool call is finished. + +When there are more than one tool calls being made (i.e. parallel tool calls), the sequence of `tool-call-start`, `tool-call-delta`, and `tool-call-end` events will repeat. + +`message-end` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +#### Example stream + +The following is an example stream in the tool calling step. + +```mdx wordWrap +# User message +"What's the weather in Madrid and Brasilia?" + +# Events stream +type='message-start' id='fba98ad3-e5a1-413c-a8de-84fbf9baabf7' delta=ChatMessageStartEventDelta(message=ChatMessageStartEventDeltaMessage(role='assistant', content=[], tool_plan='', tool_calls=[], citations=[])) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan='I')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' will')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' search')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' for')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' the')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' weather')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' in')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' Madrid')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' and')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan=' Brasilia')) + -------------------------------------------------- +type='tool-plan-delta' delta=ChatToolPlanDeltaEventDelta(message=ChatToolPlanDeltaEventDeltaMessage(tool_plan='.')) + -------------------------------------------------- +type='tool-call-start' index=0 delta=ChatToolCallStartEventDelta(message=ChatToolCallStartEventDeltaMessage(tool_calls=ToolCallV2(id='get_weather_p1t92w7gfgq7', type='function', function=ToolCallV2Function(name='get_weather', arguments='')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='{\n "')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='location')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='":')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments=' "')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='Madrid')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='"')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='\n')))) + -------------------------------------------------- +type='tool-call-delta' index=0 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='}')))) + -------------------------------------------------- +type='tool-call-end' index=0 + -------------------------------------------------- +type='tool-call-start' index=1 delta=ChatToolCallStartEventDelta(message=ChatToolCallStartEventDeltaMessage(tool_calls=ToolCallV2(id='get_weather_ay6nmvjgp9vn', type='function', function=ToolCallV2Function(name='get_weather', arguments='')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='{\n "')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='location')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='":')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments=' "')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='Bras')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='ilia')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='"')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='\n')))) + -------------------------------------------------- +type='tool-call-delta' index=1 delta=ChatToolCallDeltaEventDelta(message=ChatToolCallDeltaEventDeltaMessage(tool_calls=ChatToolCallDeltaEventDeltaMessageToolCalls(function=ChatToolCallDeltaEventDeltaMessageToolCallsFunction(arguments='}')))) + -------------------------------------------------- +type='tool-call-end' index=1 + -------------------------------------------------- +type='message-end' id=None delta=ChatMessageEndEventDelta(finish_reason='TOOL_CALL', usage=Usage(billed_units=UsageBilledUnits(input_tokens=37.0, output_tokens=28.0, search_units=None, classifications=None), tokens=UsageTokens(input_tokens=913.0, output_tokens=83.0))) + -------------------------------------------------- +``` + + +### Response generation step + +#### Event types + +`message-start` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +`content-start` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +`content-delta` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +`citation-start` + +Emitted for every citation generated in the response. + +`citation-end` + +Emitted to indicate the end of a citation. If there are multiple citations generated, the events will come as a sequence of `citation-start` and `citation-end` pairs. + +`content-end` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +`message-end` + +Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events). + +#### Example stream + +The following is an example stream in the response generation step. + +```mdx wordWrap +"What's the weather in Madrid and Brasilia?" + +type='message-start' id='e8f9afc1-0888-46f0-a9ed-eb0e5a51e17f' delta=ChatMessageStartEventDelta(message=ChatMessageStartEventDeltaMessage(role='assistant', content=[], tool_plan='', tool_calls=[], citations=[])) + -------------------------------------------------- +type='content-start' index=0 delta=ChatContentStartEventDelta(message=ChatContentStartEventDeltaMessage(content=ChatContentStartEventDeltaMessageContent(text='', type='text'))) + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='It'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' is'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' currently'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' 2'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='4'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='°'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='C in'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' Madrid'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' and'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' 2'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='8'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='°'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='C in'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text=' Brasilia'))) logprobs=None + -------------------------------------------------- +type='content-delta' index=0 delta=ChatContentDeltaEventDelta(message=ChatContentDeltaEventDeltaMessage(content=ChatContentDeltaEventDeltaMessageContent(text='.'))) logprobs=None + -------------------------------------------------- +type='citation-start' index=0 delta=CitationStartEventDelta(message=CitationStartEventDeltaMessage(citations=Citation(start=16, end=20, text='24°C', sources=[ToolSource(type='tool', id='get_weather_m3kdvxncg1p8:0', tool_output={'temperature': '{"madrid":"24°C"}'})], type='TEXT_CONTENT'))) + -------------------------------------------------- +type='citation-end' index=0 + -------------------------------------------------- +type='citation-start' index=1 delta=CitationStartEventDelta(message=CitationStartEventDeltaMessage(citations=Citation(start=35, end=39, text='28°C', sources=[ToolSource(type='tool', id='get_weather_cfwfh3wzkbrs:0', tool_output={'temperature': '{"brasilia":"28°C"}'})], type='TEXT_CONTENT'))) + -------------------------------------------------- +type='citation-end' index=1 + -------------------------------------------------- +type='content-end' index=0 + -------------------------------------------------- +type='message-end' id=None delta=ChatMessageEndEventDelta(finish_reason='COMPLETE', usage=Usage(billed_units=UsageBilledUnits(input_tokens=87.0, output_tokens=19.0, search_units=None, classifications=None), tokens=UsageTokens(input_tokens=1061.0, output_tokens=85.0))) + -------------------------------------------------- +``` + +## Usage example + +This section provides an example of handling streamed objects in the response generation step in tool use. + +### Setup + +First, import the Cohere library and create a client. + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys +``` +### Tool definition + +Next, define the tool and its associated schema. + +```python PYTHON +def get_weather(location): + temperature = { + "bern": "22°C", + "madrid": "24°C", + "brasilia": "28°C" + } + loc = location.lower() + if loc in temperature: + return [{"temperature": {loc: temperature[loc]}}] + return [{"temperature": {loc: "Unknown"}}] + +functions_map = {"get_weather": get_weather} + +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "the location to get weather. Provide just the city name without the country name.", + } + }, + "required": ["location"], + }, + }, + } +] +``` + +### Streaming the response + +Before streaming the response, first run through the tool calling and execution steps. + +```python PYTHON +messages = [{"role": "user", "content": "What's the weather in Madrid and Brasilia?"}] + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) + +import json + +if response.message.tool_calls: + for tc in response.message.tool_calls: + tool_result = functions_map[tc.function.name]( + **json.loads(tc.function.arguments) + ) + tool_content = [] + for data in tool_result: + tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) + # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated + messages.append( + {"role": "tool", "tool_call_id": tc.id, "content": tool_content} + ) +``` + +```mdx wordWrap +I will use the get_weather tool to find the weather in Madrid and Brasilia. + +[ToolCallV2(id='get_weather_15c2p6g19s8f', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Madrid"}')), ToolCallV2(id='get_weather_n01pkywy0p2w', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Brasilia"}'))] +``` + +Once the tool results have been received, we can now stream the response using the `chat_stream` endpoint. The events are passed as `chunk` objects. + +In the example below, out of all the `chunk` stream of events, we pick `content-delta` to display the text response and `citation-start` to display the citations. + +```python PYTHON +response = co.chat_stream( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +response_text = "" +citations = [] +for chunk in response: + if chunk: + if chunk.type == "content-delta": + response_text += chunk.delta.message.content.text + print(chunk.delta.message.content.text, end="") + if chunk.type == "citation-start": + citations.append(chunk.delta.message.citations) + +messages.append( + {"role": "assistant", "content": response_text} +) +``` + +```mdx wordWrap +It's currently 24°C in Madrid and 28°C in Brasilia. +``` + +```python PYTHON +for citation in citations: + print(citation, "\n") +``` + +```mdx wordWrap +start=5 end=9 text='24°C' sources=[ToolSource(type='tool', id='get_weather_15c2p6g19s8f:0', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' + +start=24 end=28 text='28°C' sources=[ToolSource(type='tool', id='get_weather_n01pkywy0p2w:0', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' +``` \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx index fe9472908..56870bdc6 100644 --- a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx +++ b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx @@ -12,7 +12,11 @@ createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- ## Overview -[[TODO - elaborate from the basic usage section - which gave the simplest possible way to define a tool]] +Tools allow you to extend the model's capabilities by giving it access to external functions and services. While the basic usage [[TODO link]] provides basic tool examples, this section expands on the topic of tools and how to set them up in more practical scenarios. + +At its most basic, setting up a tool consists of: +- **Tool creation**: A function that implements the tool's logic. This can be a custom function or a call to an external service. +- **Tool schema**: A schema that describes the tool's interface to the model. This defines the arguments that the tool accepts and the format of the output it returns. ## Tool creation @@ -27,7 +31,7 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ### JSON schema [[TODO - overview of the JSON schema]] -### Tool schema prompting +### Prompting [[TODO - how to write good tool schema e.g. descriptions, etc]] ## Structured outputs diff --git a/fern/pages/v2/tool-use/tool-use-multi-step.mdx b/fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx similarity index 53% rename from fern/pages/v2/tool-use/tool-use-multi-step.mdx rename to fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx index c8390259b..261bf83fb 100644 --- a/fern/pages/v2/tool-use/tool-use-multi-step.mdx +++ b/fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx @@ -1,5 +1,5 @@ --- -title: "Multi-step tool use (agents)" +title: "Multi-step tool use" slug: "v2/docs/tool-use-multi-step" hidden: false @@ -14,6 +14,15 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ## Overview [[TODO - describe multi-step tool use (agents)]] +The Chat endpoint supports multi-step tool use, which enables the model to perform sequential reasoning. This is especially useful in agentic workflows that require multiple steps to complete a task. + +As an example, suppose a tool use application has access to a web search tool. Given the following question, it will need to perform a series of steps in a sequence, because one step is dependent on the result of the previous step. + +"What was the revenue of the most valuable company in the US in 2023?” + +1. First, it needs to identify the most valuable company in the US in 2023 +2. Then only it can get the revenue figure now that the company has been identified + ## State management [[TODO - describe how the messages list construction for multi-step differs from single-step]] diff --git a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx new file mode 100644 index 000000000..301dbe47c --- /dev/null +++ b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx @@ -0,0 +1,598 @@ +--- +title: "Usage patterns" +slug: "v2/docs/tool-use-usage-patterns" + +hidden: false +description: >- + TBD +image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" +keywords: "Cohere, text generation, LLMs, generative AI" + +createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" +updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" +--- + +The tool use feature of the Chat endpoint comes with a set of capabilities that enable developers to implement a variety of tool use scenarios. This section describes the different patterns of tool use implementation supported by these capabilities. Each pattern can be implemented in isolation or in combination with the others. + +## Setup + +First, import the Cohere library and create a client. + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys +``` + +Let's also use the same tool defined from the example in the previous example[[TODO - link to basic example page]]. + +```python PYTHON +def get_weather(location): + # Implement any logic here + return [{"temperature": "20C"}] + # Return a list of objects e.g. [{"url": "abc.com", "text": "..."}, {"url": "xyz.com", "text": "..."}] + +functions_map = {"get_weather": get_weather} + +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "the location to get weather, example: San Fransisco, CA", + } + }, + "required": ["location"], + }, + }, + }, +] +``` + +## Parallel tool calling +The model can determine that more than one tool call is required, where it will call multiple tools in parallel. This can be calling the same tool multiple times or different tools for any number of calls. + +In the example below, the user asks for the weather in Toronto and New York. This requires calling the `get_weather` function twice, one for each location. This is reflected in the model's response, where two parallel tool calls are generated. + +```python PYTHON +messages = [{"role": "user", "content": "What's the weather in Toronto and New York?"}] + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) +``` +```mdx wordWrap +I will search for the weather in Toronto and New York. + +[ + ToolCallV2( + id="get_weather_9b0nr4kg58a8", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Toronto"}' + ), + ), + ToolCallV2( + id="get_weather_0qq0mz9gwnqr", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"New York"}' + ), + ), +] +``` +**State management** + +When tools are called in parallel, we append the messages list with one single `assistant` message containing all the tool calls and one `tool` message for each tool call. + +```python PYTHON +import json + +if response.message.tool_calls: + for tc in response.message.tool_calls: + tool_result = functions_map[tc.function.name]( + **json.loads(tc.function.arguments) + ) + tool_content = [] + for data in tool_result: + tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) + # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated + messages.append( + {"role": "tool", "tool_call_id": tc.id, "content": tool_content} + ) +``` + +The sequence of messages is represented in the diagram below. + +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query)
"] + B["
Assistant (tool calls)
"] + C["
Tool (tool result 1)
"] + D["
Tool (tool result 2)
"] + E["
Tool (tool result N)
"] + F["
Assistant (response)
"] + + A -.-> B + B -.-> C + C -.-> D + D -.-> E + E -.-> F + + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 + style C fill:#fff,stroke:#000,color:#000 + style D fill:#fff,stroke:#000,color:#000 + style E fill:#fff,stroke:#000,color:#000 + style F fill:#fff,stroke:#000,color:#000 +``` + +## Directly answering +A key attribute of tool use systems is the model’s ability to choose the right tools for a task. This includes the model's ability to decide to *not* use any tool, and instead, respond to a user message directly. + +In the example below, the user asks for a simple arithmetic question. The model determines that it does not need to use any of the available tools (only one, `get_weather`, in this case), and instead, directly answers the user. + +```python PYTHON +messages = [{"role": "user", "content": "What's 2+2?"}] + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) + +else: + print(response.message.content[0].text) +``` +```mdx wordWrap +The answer to 2+2 is 4. +``` + +**State management** + +When the model opts to respond directly to the user, there will be no items 2 and 3 above (the tool calling and tool response messages). Instead, the final `assistant` message will contain the model's direct response to the user. + +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query)
"] + B["
Assistant (response)
"] + + A -.-> B + + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 +``` + +## Multi-step tool use + +The Chat endpoint supports multi-step tool use, which enables the model to perform sequential reasoning. This is especially useful in agentic workflows that require multiple steps to complete a task. + +As an example, suppose a tool use application has access to a web search tool. Given the question "What was the revenue of the most valuable company in the US in 2023?”, it will need to perform a series of steps in a specific order: +- Identify the most valuable company in the US in 2023 +- Then only get the revenue figure now that the company has been identified + +To illustrate this, let's start with the same weather example and add another tool called `get_capital_city`, which returns the capital city of a given country. + +Here's the function definitions for the tools: +```python PYTHON +def get_weather(location): + temperature = { + "bern": 22, + "madrid": 24, + "brasilia": 28 + } + loc = location.lower() + if loc in temperature: + return [{"temperature": {loc: temperature[loc]}}] + return [{"temperature": {loc: "Unknown"}}] + +def get_capital_city(country): + capital_city = { + "switzerland": "bern", + "spain": "madrid", + "brazil": "brasilia" + } + country = country.lower() + if country in capital_city: + return [{"capital_city": {country: capital_city[country]}}] + return [{"capital_city": {country: "Unknown"}}] + +functions_map = { + "get_capital_city": get_capital_city, + "get_weather": get_weather +} +``` + +And here are the corresponding tool schemas: +```python PYTHON +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "gets the weather of a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "the location to get weather. Provide just the city name without the country name.", + } + }, + "required": ["location"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "get_capital_city", + "description": "gets the capital city of a given country", + "parameters": { + "type": "object", + "properties": { + "country": { + "type": "string", + "description": "the country to get the capital city for", + } + }, + "required": ["country"], + }, + }, + }, +] +``` + +Next, we implement the four-step tool use workflow as described in the previous page [[TODO - link to basic example page]]. + +The key difference here is the second (tool calling) and third (tool execution) steps are put in a `while` loop, which means that a sequence of this pair can happen for a number of times. This stops when the model decides in the tool calling step that no more tool calls are needed, which then triggers the fourth step (response generation). + +In this example, the user asks for the temperature in Brazil's capital city. + +```python PYTHON +# Step 1: Get the user message +messages = [{"role": "user", "content": "What's the temperature in Brazil's capital city?"}] + +# Step 2: Generate tool calls (if any) +model = "command-r-plus-08-2024" +response = co.chat( + model=model, + messages=messages, + tools=tools, + temperature=0.3 +) + +while response.message.tool_calls: + print("TOOL PLAN:") + print(response.message.tool_plan, "\n") + print("TOOL CALLS:") + for tc in response.message.tool_calls: + print( + f"Tool name: {tc.function.name} | Parameters: {tc.function.arguments}" + ) + print("=" * 50) + + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + + # Step 3: Get tool results + print("TOOL RESULT:") + for tc in response.message.tool_calls: + tool_result = functions_map[tc.function.name]( + **json.loads(tc.function.arguments) + ) + tool_content = [] + print(tool_result) + for data in tool_result: + tool_content.append( + { + "type": "document", + "document": {"data": json.dumps(data)}, + } + ) + # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated + messages.append( + { + "role": "tool", + "tool_call_id": tc.id, + "content": tool_content, + } + ) + + # Step 4: Generate response and citations + response = co.chat( + model=model, + messages=messages, + tools=tools, + temperature=0.1, + ) + +messages.append( + { + "role": "assistant", + "content": response.message.content[0].text, + } +) + +# Print final response +print("RESPONSE:") +print(response.message.content[0].text) +print("=" * 50) + +# Print citations (if any) +verbose_source = ( + True # Change to True to display the contents of a source +) +if response.message.citations: + print("CITATIONS:\n") + for citation in response.message.citations: + print( + f"Start: {citation.start}| End:{citation.end}| Text:'{citation.text}' " + ) + print("Sources:") + for idx, source in enumerate(citation.sources): + print(f"{idx+1}. {source.id}") + if verbose_source: + print(f"{source.tool_output}") + print("\n") +``` +The model first determines that it first needs to find out the capital city of Brazil. Using this information, it can proceed with the next step in the sequence, which is to look up the temperature of that city. + +This is reflected in the model's response, where two tool calls are generated in a sequence. + +```mdx wordWrap +TOOL PLAN: +I will first find out the capital city of Brazil and then I will find out the temperature there. + +TOOL CALLS: +Tool name: get_capital_city | Parameters: {"country":"Brazil"} +================================================== +TOOL RESULT: +[{'capital_city': {'brazil': 'brasilia'}}] +TOOL PLAN: +I found out that the capital city of Brazil is Brasilia. Now I will find out the temperature there. + +TOOL CALLS: +Tool name: get_weather | Parameters: {"location":"Brasilia"} +================================================== +TOOL RESULT: +[{'temperature': {'brasilia': 28}}] +RESPONSE: +The temperature in Brasilia, the capital city of Brazil, is 28°C. +================================================== +CITATIONS: + +Start: 19| End:27| Text:'Brasilia' +Sources: +1. get_capital_city_xvvhzenxpraf:0 +{'capital_city': '{"brazil":"brasilia"}'} + + +Start: 60| End:65| Text:'28°C.' +Sources: +1. get_weather_tj79nmn7agmq:0 +{'temperature': '{"brasilia":28}'} +``` + +**State management** +In a multi-step tool use scenario, instead of just one occurence of assistant-tool messages, there will be a sequence of assistant-tool messages to reflect the multiple steps of tool calling involved. + +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query)
"] + B["
Assistant (tool call(s) - step 1)
"] + C["
Tool (tool result(s) - step 1)
"] + D["
Assistant (tool call(s) - step 2)
"] + E["
Tool (tool result(s) - step 2)
"] + F["
Assistant (tool call(s) - step N)
"] + G["
Tool (tool result(s) - step N)
"] + H["
Assistant (response)
"] + + A -.-> B + B -.-> C + C -.-> D + D -.-> E + E -.-> F + F -.-> G + G -.-> H + + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 + style C fill:#fff,stroke:#000,color:#000 + style D fill:#fff,stroke:#000,color:#000 + style E fill:#fff,stroke:#000,color:#000 + style F fill:#fff,stroke:#000,color:#000 + style G fill:#fff,stroke:#000,color:#000 + style H fill:#fff,stroke:#000,color:#000 +``` + +## Forcing tool usage + +This feature is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models. + +As we saw in the previous examples, during the tool calling step, the model may decide to either: +- make tool call(s) +- or, respond to a user message directly. + +You can, however, force the model to choose one of these options. This is done via the `tool_choice` parameter. +- You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`. +- Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`. + +By default, if you don’t specify the `tool_choice` parameter, then the model will decide between making tool calls or responding directly. + +```python PYTHON {5} +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools, + tool_choice="REQUIRED" # optional, to force tool calls + # tool_choice="NONE" # optional, to force a direct response +) +``` + +**State management** + +Here's the sequence of messages when `tool_choice` is set to `REQUIRED`. +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query)
"] + B["
Assistant (tool call(s))
"] + C["
Tool (tool result(s))
"] + D["
Assistant (response)
"] + + A -.-> B + B -.-> C + C -.-> D + + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 + style C fill:#fff,stroke:#000,color:#000 + style D fill:#fff,stroke:#000,color:#000 +``` + +Here's the sequence of messages when `tool_choice` is set to `NONE`. + +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query)
"] + B["
Assistant (response)
"] + + A -.-> B + + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 +``` + + +## Chatbots (multi-turn) + +When building chatbots, we'll need to maintain the memory or state of a conversation over multiple turns. In this case, we can keep appending each turn of a conversation to the `messages` list. + +As an example, here's the messages list from the first turn of a conversation. + +```python PYTHON +from cohere import ToolCallV2, ToolCallV2Function + +messages = [ + {"role": "user", "content": "What's the weather in Toronto?"}, + { + "role": "assistant", + "tool_plan": "I will search for the weather in Toronto.", + "tool_calls": [ + ToolCallV2( + id="get_weather_1byjy32y4hvq", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Toronto"}' + ), + ) + ], + }, + { + "role": "tool", + "tool_call_id": "get_weather_1byjy32y4hvq", + "content": [ + {"type": "document", "document": {"data": '{"temperature": "20C"}'}} + ], + }, + {"role": "assistant", "content": "It's 20°C in Toronto."}, +] +``` +Then, in the second turn, given a rather vague follow-up user message, the model correctly infers that the context of the user's message is about the weather. + +```python PYTHON +messages.append({"role": "user", "content": "What about London?"}) + +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +if response.message.tool_calls: + messages.append( + { + "role": "assistant", + "tool_plan": response.message.tool_plan, + "tool_calls": response.message.tool_calls, + } + ) + print(response.message.tool_plan, "\n") + print(response.message.tool_calls) +``` +```mdx wordWrap +I will search for the weather in London. + +[ToolCallV2(id='get_weather_8hwpm7d4wr14', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"London"}'))] +``` + +**State management** + +The sequence of messages is represented in the diagram below. + +```mermaid +%%{init: {'htmlLabels': true}}%% +flowchart TD + A["
User (query) - #1
"] + B["
Assistant (tool call(s) - #1)
"] + C["
Tool (tool result(s) - #1)
"] + D["
Assistant (response - #1)
"] + E["
User (query) - #2
"] + F["
Assistant (tool call(s) - #2)
"] + G["
Tool (tool result(s) - #2)
"] + H["
Assistant (response - #2)
"] + I["
...
"] + + A -.-> B + B -.-> C + C -.-> D + D -.-> E + E -.-> F + F -.-> G + G -.-> H + H -.-> I + + style A fill:#fff,stroke:#000,color:#000 + style B fill:#fff,stroke:#000,color:#000 + style C fill:#fff,stroke:#000,color:#000 + style D fill:#fff,stroke:#000,color:#000 + style E fill:#fff,stroke:#000,color:#000 + style F fill:#fff,stroke:#000,color:#000 + style G fill:#fff,stroke:#000,color:#000 + style H fill:#fff,stroke:#000,color:#000 + style I fill:#fff,stroke:#000,color:#000 +``` diff --git a/fern/v2.yml b/fern/v2.yml index 7f1c50b72..4901861f7 100644 --- a/fern/v2.yml +++ b/fern/v2.yml @@ -83,10 +83,8 @@ navigation: contents: - page: Basic usage path: pages/v2/tool-use/tool-use-overview.mdx - - page: Multi-step (agents) - path: pages/v2/tool-use/tool-use-multi-step.mdx - - page: Tool definition - path: pages/v2/tool-use/tool-use-tool-definition.mdx + - page: Usage patterns + path: pages/v2/tool-use/tool-use-usage-patterns.mdx - page: Parameter types path: pages/v2/tool-use/tool-use-parameter-types.mdx - page: Streaming From c65e9213ca5672396afcf15b5dd3aec68738502f Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Tue, 4 Feb 2025 16:35:07 +0800 Subject: [PATCH 05/11] update sections --- fern/pages/v2/tool-use/tool-use-citations.mdx | 36 +++- fern/pages/v2/tool-use/tool-use-overview.mdx | 33 +++- .../v2/tool-use/tool-use-parameter-types.mdx | 39 ++-- fern/pages/v2/tool-use/tool-use-streaming.mdx | 30 +++- .../v2/tool-use/tool-use-usage-patterns.mdx | 166 +++++++++--------- 5 files changed, 194 insertions(+), 110 deletions(-) diff --git a/fern/pages/v2/tool-use/tool-use-citations.mdx b/fern/pages/v2/tool-use/tool-use-citations.mdx index 63a75f89d..0516694d0 100644 --- a/fern/pages/v2/tool-use/tool-use-citations.mdx +++ b/fern/pages/v2/tool-use/tool-use-citations.mdx @@ -22,13 +22,33 @@ The following sections describe how to access the citations in both the non-stre First, define the tool and its associated schema. + + + ```python PYTHON # ! pip install -U cohere import cohere import json co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys +``` + + + +```python PYTHON +# ! pip install -U cohere +import cohere +import json +co = cohere.ClientV2( + api_key="", # Leave this blank + base_url="" +) +``` + + + +```python PYTHON def get_weather(location): temperature = { "bern": "22°C", @@ -129,7 +149,7 @@ start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='get_weather_vd ### Streaming In a streaming scenario (using `chat_stream` to generate the model response), the citations are provided in the `citation-start` events. -Each citation object contains the same fields as the [non-streaming scenario](###non-streaming). +Each citation object contains the same fields as the [non-streaming scenario](#non-streaming). ```python PYTHON response = co.chat_stream( @@ -165,12 +185,14 @@ start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='get_weather_gh ``` ## Citation modes -When ruuning tool use in streaming mode, it’s possible to configure how citations are generated and presented. You can choose between fast citations or accurate citations, depending on your latency and precision needs: +When running tool use in streaming mode, it’s possible to configure how citations are generated and presented. You can choose between fast citations or accurate citations, depending on your latency and precision needs. ### Accurate citations -The model produces its answer first, and then, after the entire response is generated, it provides citations that map to specific segments of the response text. This approach may incur slightly higher latency, but it ensures the citation indices are more precisely aligned with the final text segments of the model’s answer. This is the default option, or you can explicitly specify it by adding the `citation_options={"mode": "accurate"}` argument in the API call. +The model produces its answer first, and then, after the entire response is generated, it provides citations that map to specific segments of the response text. This approach may incur slightly higher latency, but it ensures the citation indices are more precisely aligned with the final text segments of the model’s answer. + +This is the default option, or you can explicitly specify it by adding the `citation_options={"mode": "accurate"}` argument in the API call. -Here is an example. To make things concise, let's start with a pre-defined list of `messages` with the user query, tool calling, and tool results are already available. +Here is an example. To keep it concise, let's start with a pre-defined list of `messages` with the user query, tool calling, and tool results are already available. ```python PYTHON # ! pip install -U cohere @@ -265,9 +287,11 @@ start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='2', tool_outpu ``` ### Fast citations -The model generates citations inline, as the response is being produced. In streaming mode, you will see citations injected at the exact moment the model uses a particular piece of external context. This approach provides immediate traceability at the expense of slightly less precision in citation relevance. You can specify it by adding the `citation_options={"mode": "fast"}` argument in the API call. +The model generates citations inline, as the response is being produced. In streaming mode, you will see citations injected at the exact moment the model uses a particular piece of external context. This approach provides immediate traceability at the expense of slightly less precision in citation relevance. + +You can specify it by adding the `citation_options={"mode": "fast"}` argument in the API call. -Here is an example using the same list of pre-defined`messages` as the above. With the `citation_options` mode set to `fast`, we get the citations inlineas the model generates the response. +Here is an example using the same list of pre-defined`messages` as the above. With the `citation_options` mode set to `fast`, we get the citations inline as the model generates the response. ```python PYTHON response = co.chat_stream( diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index e0368b48c..f243df2a2 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -26,12 +26,29 @@ The Chat endpoint comes with built-in tool use capabilities such as function cal First, import the Cohere library and create a client. + + + ```python PYTHON # ! pip install -U cohere import cohere co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys ``` + + + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2( + api_key="", # Leave this blank + base_url="" +) +``` + + ## Tool definition @@ -296,20 +313,20 @@ The sequence of messages is represented in the diagram below. ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query)
"] - B["
Assistant (tool call)
"] - C["
Tool (tool result)
"] - D["
Assistant (response)
"] + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query
"] + B["
ASSISTANT
Tool call
"] + C["
TOOL
Tool result
"] + D["
ASSISTANT
Response
"] A -.-> B B -.-> C C -.-> D - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 - style C fill:#fff,stroke:#000,color:#000 - style D fill:#fff,stroke:#000,color:#000 + class A,B,C,D defaultStyle; ``` + Note that this sequence of messages represents a basic usage pattern in tool use. The next page [[TODO - add link]] will describe how this will be adapted for other scenarios. diff --git a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx index 7ce6f1deb..d5040511f 100644 --- a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx +++ b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx @@ -33,17 +33,14 @@ With `strict_tools` enabled, the API will ensure that the tool names and tool pa Additionally, this results in faster development. You don’t need to spend a lot of time prompt engineering the model to avoid hallucinations. -In the example below, we create a tool that can retrieve weather data for a given location. The tool is called `get_weather` which contains a parameter called `location`. We then invoke the Chat API with `strict_tools` set to `True` to ensure that the generated tool calls always include the correct function and parameter names. - When the `strict_tools` parameter is set to `True`, you can define a maximum of 200 fields across all tools being passed to an API call. ```python PYTHON {4} response = co.chat(model="command-r-plus-08-2024", - messages=[{"role": "user", "content": "What's the weather in Toronto?"}], - tools=tools, - strict_tools=True) - -print(response.message.tool_calls) + messages=[{"role": "user", "content": "What's the weather in Toronto?"}], + tools=tools, + strict_tools=True +) ``` @@ -66,11 +63,33 @@ This section provides usage examples of the JSON Schema [parameters that are sup The examples on this page each provide a tool schema and a `message` (the user message). To get an output, pass those values to a Chat endpoint call, as shown in the helper code below. -```python PYTHON PYTHON + + + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys +``` + + + +```python PYTHON +# ! pip install -U cohere import cohere -co = cohere.ClientV2(api_key="YOUR API KEY") -res = co.chat( +co = cohere.ClientV2( + api_key="", # Leave this blank + base_url="" +) +``` + + + + +```python PYTHON +response = co.chat( # The model name. Example: command-r-plus-08-2024 model="MODEL_NAME", # The user message. Optional - you can first add a `system_message` role diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index 05d147e47..96c0577cc 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -13,13 +13,13 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- ## Overview -To enable response streaming in tool use, use the `chat_stream` endpoint instead of `chat` when making the API call. +To enable response streaming in tool use, use the `chat_stream` endpoint instead of `chat`. -You can stream responses in both the tool calling and the response generation steps. This allows your application to receive token streams as the model plans and executes tool calls and generates its response using the tool results. +You can stream responses in both the tool calling and the response generation steps. This allows your application to receive token streams as the model plans and executes tool calls and finally generates its response. ## Events stream -In tool use, the events streamed by the endpoint follows the structure of a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events) but contains additional events for tool calling and response generation together with the corresponding contents. This section describes the stream of events and their contents. +In tool use, the events streamed by the endpoint follows the structure of a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming#basic-chat-stream-events) but contains additional events for tool calling and response generation with the associated contents. This section describes the stream of events and their contents. ### Tool calling step @@ -219,18 +219,36 @@ type='message-end' id=None delta=ChatMessageEndEventDelta(finish_reason='COMPLET ## Usage example -This section provides an example of handling streamed objects in the response generation step in tool use. +This section provides an example of handling streamed objects in the tool use response generation step. ### Setup First, import the Cohere library and create a client. + + + ```python PYTHON # ! pip install -U cohere import cohere co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys ``` + + + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2( + api_key="", # Leave this blank + base_url="" +) +``` + + + ### Tool definition Next, define the tool and its associated schema. @@ -316,9 +334,9 @@ I will use the get_weather tool to find the weather in Madrid and Brasilia. [ToolCallV2(id='get_weather_15c2p6g19s8f', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Madrid"}')), ToolCallV2(id='get_weather_n01pkywy0p2w', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Brasilia"}'))] ``` -Once the tool results have been received, we can now stream the response using the `chat_stream` endpoint. The events are passed as `chunk` objects. +Once the tool results have been received, we can now stream the response using the `chat_stream` endpoint. -In the example below, out of all the `chunk` stream of events, we pick `content-delta` to display the text response and `citation-start` to display the citations. +The events are streamed as `chunk` objects. In the example below, we pick `content-delta` to display the text response and `citation-start` to display the citations. ```python PYTHON response = co.chat_stream( diff --git a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx index 301dbe47c..b357dcc07 100644 --- a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx +++ b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx @@ -12,20 +12,37 @@ createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" --- -The tool use feature of the Chat endpoint comes with a set of capabilities that enable developers to implement a variety of tool use scenarios. This section describes the different patterns of tool use implementation supported by these capabilities. Each pattern can be implemented in isolation or in combination with the others. +The tool use feature of the Chat endpoint comes with a set of capabilities that enable developers to implement a variety of tool use scenarios. This section describes the different patterns of tool use implementation supported by these capabilities. Each pattern can be implemented on its own or in combination with the others. ## Setup First, import the Cohere library and create a client. + + + ```python PYTHON # ! pip install -U cohere import cohere co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys ``` + + + +```python PYTHON +# ! pip install -U cohere +import cohere + +co = cohere.ClientV2( + api_key="", # Leave this blank + base_url="" +) +``` + + -Let's also use the same tool defined from the example in the previous example[[TODO - link to basic example page]]. +Let's use the same `get_weather` tool as in the previous example[[TODO - link to basic example page]]. ```python PYTHON def get_weather(location): @@ -127,12 +144,14 @@ The sequence of messages is represented in the diagram below. ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query)
"] - B["
Assistant (tool calls)
"] - C["
Tool (tool result 1)
"] - D["
Tool (tool result 2)
"] - E["
Tool (tool result N)
"] - F["
Assistant (response)
"] + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query
"] + B["
ASSISTANT
Tool calls
"] + C["
TOOL
Tool result #1
"] + D["
TOOL
Tool result #2
"] + E["
TOOL
Tool result #N
"] + F["
ASSISTANT
Response
"] A -.-> B B -.-> C @@ -140,12 +159,7 @@ flowchart TD D -.-> E E -.-> F - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 - style C fill:#fff,stroke:#000,color:#000 - style D fill:#fff,stroke:#000,color:#000 - style E fill:#fff,stroke:#000,color:#000 - style F fill:#fff,stroke:#000,color:#000 + class A,B,C,D,E,F defaultStyle; ``` ## Directly answering @@ -180,13 +194,14 @@ When the model opts to respond directly to the user, there will be no items 2 an ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query)
"] - B["
Assistant (response)
"] + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query
"] + B["
ASSISTANT
Response
"] A -.-> B - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 + class A,B defaultStyle; ``` ## Multi-step tool use @@ -367,9 +382,9 @@ if response.message.citations: print(f"{source.tool_output}") print("\n") ``` -The model first determines that it first needs to find out the capital city of Brazil. Using this information, it can proceed with the next step in the sequence, which is to look up the temperature of that city. +The model first determines that it needs to find out the capital city of Brazil. Once it has this information, it proceeds with the next step in the sequence, which is to look up the temperature of that city. -This is reflected in the model's response, where two tool calls are generated in a sequence. +This is reflected in the model's response, where two tool calling-result pairs are generated in a sequence. ```mdx wordWrap TOOL PLAN: @@ -406,43 +421,39 @@ Sources: ``` **State management** -In a multi-step tool use scenario, instead of just one occurence of assistant-tool messages, there will be a sequence of assistant-tool messages to reflect the multiple steps of tool calling involved. + +In a multi-step tool use scenario, instead of just one occurence of `assistant`-`tool` messages, there will be a sequence of `assistant`-`tool` messages to reflect the multiple steps of tool calling involved. ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query)
"] - B["
Assistant (tool call(s) - step 1)
"] - C["
Tool (tool result(s) - step 1)
"] - D["
Assistant (tool call(s) - step 2)
"] - E["
Tool (tool result(s) - step 2)
"] - F["
Assistant (tool call(s) - step N)
"] - G["
Tool (tool result(s) - step N)
"] - H["
Assistant (response)
"] - - A -.-> B - B -.-> C - C -.-> D - D -.-> E - E -.-> F - F -.-> G - G -.-> H - - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 - style C fill:#fff,stroke:#000,color:#000 - style D fill:#fff,stroke:#000,color:#000 - style E fill:#fff,stroke:#000,color:#000 - style F fill:#fff,stroke:#000,color:#000 - style G fill:#fff,stroke:#000,color:#000 - style H fill:#fff,stroke:#000,color:#000 + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query
"] + B1["
ASSISTANT
Tool call step #1
"] + C1["
TOOL
Tool result step #1
"] + B2["
ASSISTANT
Tool call step #2
"] + C2["
TOOL
Tool result step #2
"] + BN["
ASSISTANT
Tool call step #N
"] + CN["
TOOL
Tool result step #N
"] + D["
ASSISTANT
Response
"] + + A -.-> B1 + B1 -.-> C1 + C1 -.-> B2 + B2 -.-> C2 + C2 -.-> BN + BN -.-> CN + CN -.-> D + + class A,B1,C1,B2,C2,BN,CN,D defaultStyle; ``` ## Forcing tool usage This feature is only compatible with the [Command R7B](https://docs.cohere.com/v2/docs/command-r7b) and newer models. -As we saw in the previous examples, during the tool calling step, the model may decide to either: +As shown in the previous examples, during the tool calling step, the model may decide to either: - make tool call(s) - or, respond to a user message directly. @@ -450,7 +461,7 @@ You can, however, force the model to choose one of these options. This is done v - You can force the model to make tool call(s), i.e. to not respond directly, by setting the `tool_choice` parameter to `REQUIRED`. - Alternatively, you can force the model to respond directly, i.e. to not make tool call(s), by setting the `tool_choice` parameter to `NONE`. -By default, if you don’t specify the `tool_choice` parameter, then the model will decide between making tool calls or responding directly. +By default, if you don’t specify the `tool_choice` parameter, then it is up to the model to decide whether to make tool calls or respond directly. ```python PYTHON {5} response = co.chat( @@ -468,19 +479,18 @@ Here's the sequence of messages when `tool_choice` is set to `REQUIRED`. ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query)
"] - B["
Assistant (tool call(s))
"] - C["
Tool (tool result(s))
"] - D["
Assistant (response)
"] + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query
"] + B["
ASSISTANT
Tool call
"] + C["
TOOL
Tool result
"] + D["
ASSISTANT
Response
"] A -.-> B B -.-> C C -.-> D - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 - style C fill:#fff,stroke:#000,color:#000 - style D fill:#fff,stroke:#000,color:#000 + class A,B,C,D defaultStyle; ``` Here's the sequence of messages when `tool_choice` is set to `NONE`. @@ -488,19 +498,20 @@ Here's the sequence of messages when `tool_choice` is set to `NONE`. ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query)
"] - B["
Assistant (response)
"] + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query
"] + B["
ASSISTANT
Response
"] A -.-> B - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 + class A,B defaultStyle; ``` ## Chatbots (multi-turn) -When building chatbots, we'll need to maintain the memory or state of a conversation over multiple turns. In this case, we can keep appending each turn of a conversation to the `messages` list. +Building chatbots requires maintaining the memory or state of a conversation over multiple turns. To do this, we can keep appending each turn of a conversation to the `messages` list. As an example, here's the messages list from the first turn of a conversation. @@ -532,7 +543,7 @@ messages = [ {"role": "assistant", "content": "It's 20°C in Toronto."}, ] ``` -Then, in the second turn, given a rather vague follow-up user message, the model correctly infers that the context of the user's message is about the weather. +Then, in the second turn, when provided with a rather vague follow-up user message, the model correctly infers that the context is about the weather. ```python PYTHON messages.append({"role": "user", "content": "What about London?"}) @@ -567,15 +578,17 @@ The sequence of messages is represented in the diagram below. ```mermaid %%{init: {'htmlLabels': true}}%% flowchart TD - A["
User (query) - #1
"] - B["
Assistant (tool call(s) - #1)
"] - C["
Tool (tool result(s) - #1)
"] - D["
Assistant (response - #1)
"] - E["
User (query) - #2
"] - F["
Assistant (tool call(s) - #2)
"] - G["
Tool (tool result(s) - #2)
"] - H["
Assistant (response - #2)
"] - I["
...
"] + classDef defaultStyle fill:#fff,stroke:#000,color:#000; + + A["
USER
Query - turn #1
"] + B["
ASSISTANT
Tool call - turn #1
"] + C["
TOOL
Tool result - turn #1
"] + D["
ASSISTANT
Response - turn #1
"] + E["
USER
Query - turn #2
"] + F["
ASSISTANT
Tool call - turn #2
"] + G["
TOOL
Tool result - turn #2
"] + H["
ASSISTANT
Response - turn #2
"] + I["
USER
...
"] A -.-> B B -.-> C @@ -586,13 +599,6 @@ flowchart TD G -.-> H H -.-> I - style A fill:#fff,stroke:#000,color:#000 - style B fill:#fff,stroke:#000,color:#000 - style C fill:#fff,stroke:#000,color:#000 - style D fill:#fff,stroke:#000,color:#000 - style E fill:#fff,stroke:#000,color:#000 - style F fill:#fff,stroke:#000,color:#000 - style G fill:#fff,stroke:#000,color:#000 - style H fill:#fff,stroke:#000,color:#000 - style I fill:#fff,stroke:#000,color:#000 + class A,B,C,D,E,F,G,H,I defaultStyle; ``` + From 91d1d4858d82dac5546783b11857c0ce16fd4926 Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Fri, 7 Feb 2025 08:48:31 +0800 Subject: [PATCH 06/11] update tool use pages --- fern/pages/v2/tool-use/tool-use-citations.mdx | 2 +- fern/pages/v2/tool-use/tool-use-overview.mdx | 78 +++++++++++++------ .../v2/tool-use/tool-use-parameter-types.mdx | 2 +- fern/pages/v2/tool-use/tool-use-streaming.mdx | 21 ++++- .../v2/tool-use/tool-use-tool-definition.mdx | 39 ---------- .../v2/tool-use/tool-use-usage-patterns.mdx | 8 +- 6 files changed, 79 insertions(+), 71 deletions(-) delete mode 100644 fern/pages/v2/tool-use/tool-use-tool-definition.mdx diff --git a/fern/pages/v2/tool-use/tool-use-citations.mdx b/fern/pages/v2/tool-use/tool-use-citations.mdx index 0516694d0..f2fc2756e 100644 --- a/fern/pages/v2/tool-use/tool-use-citations.mdx +++ b/fern/pages/v2/tool-use/tool-use-citations.mdx @@ -73,7 +73,7 @@ tools = [ "properties": { "location": { "type": "string", - "description": "the location to get weather. Provide just the city name without the country name.", + "description": "the location to get the weather, example: San Francisco.", } }, "required": ["location"], diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index f243df2a2..cd43a6673 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -14,9 +14,9 @@ updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ## Overview -Tool use is a technique which allows developers to connect Cohere’s Command R family of models to external tools like search engines, APIs, functions, databases, etc. +Tool use is a technique which allows developers to connect Cohere’s Command family models to external tools like search engines, APIs, functions, databases, etc. -This opens up a richer set of behaviors by leveraging data stored in tools, taking actions through APIs, interacting with a vector database, querying a search engine, etc., and is particularly valuable for enterprise developers, since a lot of enterprise data lives in external sources. +This opens up a richer set of behaviors by leveraging tools to access external data sources, taking actions through APIs, interacting with a vector database, querying a search engine, etc., and is particularly valuable for enterprise developers, since a lot of enterprise data lives in external sources. The Chat endpoint comes with built-in tool use capabilities such as function calling, multi-step reasoning, and citation generation. @@ -76,7 +76,7 @@ functions_map = {"get_weather": get_weather} ### Defining the tool schema -We also need to define the tool schemas in a format that can be passed to the Chat endpoint. The schema follows the [JSON Schema specification](https://json-schema.org/) and must contain the following fields: +We also need to define the tool schemas in a format that can be passed to the Chat endpoint. The schema follows the [JSON Schema specification](https://json-schema.org/understanding-json-schema) and must contain the following fields: - `name`: the name of the tool. - `description`: a description of what the tool is and what it is used for. - `parameters`: a list of parameters that the tool accepts. For each parameter, we need to define the following fields: @@ -84,9 +84,13 @@ We also need to define the tool schemas in a format that can be passed to the Ch - `properties`: the name of the parameter and the following fields: - `type`: the type of the parameter. - `description`: a description of what the parameter is and what it is used for. - - `required`: a list of required parameters from the `properties` field. + - `required`: a list of required properties by name, which appear as keys in the `properties` object -This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on the information that it contains. Therefore, the more descriptive and clear the schema, the more likely the LLM will make the right tool call decisions. In a typical development cycle, some fields such as `name`, `description`, and `properties` will likely require a few rounds of iterations in order to get the best results (a similar idea to prompt engineering). +This schema informs the LLM about what the tool does, and the LLM decides whether to use a particular tool based on the information that it contains. + +Therefore, the more descriptive and clear the schema, the more likely the LLM will make the right tool call decisions. + +In a typical development cycle, some fields such as `name`, `description`, and `properties` will likely require a few rounds of iterations in order to get the best results (a similar approach to prompt engineering). ```python PYTHON tools = [ @@ -100,7 +104,7 @@ tools = [ "properties": { "location": { "type": "string", - "description": "the location to get weather, example: San Fransisco, CA", + "description": "the location to get the weather, example: San Francisco.", } }, "required": ["location"], @@ -110,6 +114,8 @@ tools = [ ] ``` +The endpoint supports a subset of the JSON Schema specification. Refer to the [Structured Outputs documentation](https://docs.cohere.com/docs/structured-outputs#parameter-types-support) for the list of supported and unsupported parameters. + ## Tool use workflow We can think of a tool use system as consisting of four components: @@ -158,11 +164,11 @@ messages = [ Next, we call the Chat endpoint to generate the list of tool calls. This is done by passing the parameters `model`, `messages`, and `tools` to the Chat endpoint. -The endpoint will send back a list of tool calls to be made if the model determines that tools are required. If it does, it will return two types of messages: +The endpoint will send back a list of tool calls to be made if the model determines that tools are required. If it does, it will return two types of information: - `tool_plan`: its reflection on the next steps it should take, given the user query. -- `tool_calls`: a list of tool calls to be made (if any), together with the tool call IDs. +- `tool_calls`: a list of tool calls to be made (if any), together with auto-generated tool call IDs. -We then append this information to the `messages` list with the `role` set to `assistant`. +We then append these to the `messages` list with the `role` set to `assistant`. ```python PYTHON response = co.chat( @@ -197,12 +203,25 @@ I will search for the weather in Toronto. ] ``` + + +The model can decide to *not* make any tool call, and instead, respond to a user message directly. This is described [here](https://docs.cohere.com/docs/tool-use-usage-patterns#directly-answering). + + +The model can determine that more than one tool call is required. This can be calling the same tool multiple times or different tools for any number of calls. This is described [here](https://docs.cohere.com/docs/tool-use-usage-patterns#parallel-tool-calling). + + + ### Step 3: Get tool results During this step, we perform the function calling. We call the necessary tools based on the tool call payloads given by the endpoint. -For each tool call, we append the tool results to the `tool_content` list with `type` set to `document` and `document` set to the tool results (in JSON string format). - -We then append this information to the `messages` list with the `role` set to `tool`, together with the tool call IDs that were generated in the previous step. +For each tool call, we append the `messages` list with: +- the `tool_call_id` generated in the previous step. +- the `content` of each tool result with the following fields: + - `type` which is `document` + - `document` containing + - `data`: which stores the contents of the tool result. + - `id` (optional): you can provide each document with a unique ID for use in citations, otherwise auto-generated ```python PYTHON import json @@ -214,13 +233,28 @@ if response.message.tool_calls: ) tool_content = [] for data in tool_result: - tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) - # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated + # Optional: the "document" object can take an "id" field for use in citations, otherwise auto-generated + tool_content.append( + { + "type": "document", + "document": { + "data": json.dumps(data) + } + } + ) messages.append( - {"role": "tool", "tool_call_id": tc.id, "content": tool_content} + { + "role": "tool", + "tool_call_id": tc.id, + "content": tool_content + } ) ``` + +The Chat endpoint supports multi-step tool use, allowing the model to perform agentic tasks that require sequential reasoning. This means that steps 2 and 3 will run multiple times in loop. This is described [here](https://docs.cohere.com/docs/tool-use-usage-patterns#multi-step-tool-use). + + ### Step 4: Generate response and citations By this time, the tool call has already been executed, and the result has been returned to the LLM. @@ -256,10 +290,10 @@ print(response.message.citations) ``` -## State management -This section gives a more detailed look at how the state is managed via the `messages` list. +### State management +This section provides a more detailed look at how the state is managed via the `messages` list as described in the [tool use workflow](#tool-use-workflow) above. -As described in the [tool use workflow](#tool-use-workflow) section above, at each step of the workflow, the endpoint requires that we append specific types of information to the `messages` list. This is to ensure that the model has the necessary context to generate its response at a given point. +At each step of the workflow, the endpoint requires that we append specific types of information to the `messages` list. This is to ensure that the model has the necessary context to generate its response at a given point. In summary, each single turn of a conversation that involves tool calling consists of: 1. a `user` message containing the user message (`content`) @@ -267,9 +301,7 @@ In summary, each single turn of a conversation that involves tool calling consis 3. a `tool` message, containing the tool results (`tool_call_id` and `content`) 4. a final `assistant` message, containing the response to the user (`content`) -These correspond to the four steps described in the [tool use workflow](#tool-use-workflow) section above. - -The following is the list of messages from the example in the [tool use workflow](#tool-use-workflow) section. +These correspond to the four steps described above. The list of `messages` is shown below. ```python PYTHON for message in messages: @@ -308,7 +340,7 @@ for message in messages: } ``` -The sequence of messages is represented in the diagram below. +The sequence of `messages` is represented in the diagram below. ```mermaid %%{init: {'htmlLabels': true}}%% @@ -328,5 +360,5 @@ flowchart TD ``` -Note that this sequence of messages represents a basic usage pattern in tool use. The next page [[TODO - add link]] will describe how this will be adapted for other scenarios. +Note that this sequence represents a basic usage pattern in tool use. The next page [[TODO - add link]] will describe how this is adapted for other scenarios. diff --git a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx index d5040511f..cc738b25d 100644 --- a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx +++ b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx @@ -129,7 +129,7 @@ tools = [ "properties": { "location": { "type": "string", - "description": "The location to get weather.", + "description": "the location to get the weather, example: San Francisco.", } }, "required": ["location"], diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index 96c0577cc..f7ae7b036 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -278,7 +278,7 @@ tools = [ "properties": { "location": { "type": "string", - "description": "the location to get weather. Provide just the city name without the country name.", + "description": "the location to get the weather, example: San Francisco.", } }, "required": ["location"], @@ -321,8 +321,8 @@ if response.message.tool_calls: ) tool_content = [] for data in tool_result: + # Optional: the "document" object can take an "id" field for use in citations, otherwise auto-generated tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) - # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated messages.append( {"role": "tool", "tool_call_id": tc.id, "content": tool_content} ) @@ -331,7 +331,22 @@ if response.message.tool_calls: ```mdx wordWrap I will use the get_weather tool to find the weather in Madrid and Brasilia. -[ToolCallV2(id='get_weather_15c2p6g19s8f', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Madrid"}')), ToolCallV2(id='get_weather_n01pkywy0p2w', type='function', function=ToolCallV2Function(name='get_weather', arguments='{"location":"Brasilia"}'))] +[ + ToolCallV2( + id="get_weather_15c2p6g19s8f", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Madrid"}' + ), + ), + ToolCallV2( + id="get_weather_n01pkywy0p2w", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Brasilia"}' + ), + ), +] ``` Once the tool results have been received, we can now stream the response using the `chat_stream` endpoint. diff --git a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx b/fern/pages/v2/tool-use/tool-use-tool-definition.mdx deleted file mode 100644 index 56870bdc6..000000000 --- a/fern/pages/v2/tool-use/tool-use-tool-definition.mdx +++ /dev/null @@ -1,39 +0,0 @@ ---- -title: "Tool use - tool definition" -slug: "v2/docs/tool-use-definition" - -hidden: false -description: >- - TBD -image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" -keywords: "Cohere, text generation, LLMs, generative AI" - -createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" -updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ---- -## Overview -Tools allow you to extend the model's capabilities by giving it access to external functions and services. While the basic usage [[TODO link]] provides basic tool examples, this section expands on the topic of tools and how to set them up in more practical scenarios. - -At its most basic, setting up a tool consists of: -- **Tool creation**: A function that implements the tool's logic. This can be a custom function or a call to an external service. -- **Tool schema**: A schema that describes the tool's interface to the model. This defines the arguments that the tool accepts and the format of the output it returns. - -## Tool creation - -### Example: Custom functions -[[TODO - an example of a tool with basic working logic - e.g. sales database - example more than 1 tool]] - -### Example: External services -[[TODO - an example of a tool with basic working logic - e.g. web search]] - -## Tool schema - -### JSON schema -[[TODO - overview of the JSON schema]] - -### Prompting -[[TODO - how to write good tool schema e.g. descriptions, etc]] - -## Structured outputs - -[[TODO - how to use the strict_tools parameter]] \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx index b357dcc07..9204ddd25 100644 --- a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx +++ b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx @@ -63,7 +63,7 @@ tools = [ "properties": { "location": { "type": "string", - "description": "the location to get weather, example: San Fransisco, CA", + "description": "the location to get the weather, example: San Francisco.", } }, "required": ["location"], @@ -132,8 +132,8 @@ if response.message.tool_calls: ) tool_content = [] for data in tool_result: + # Optional: the "document" object can take an "id" field for use in citations, otherwise auto-generated tool_content.append({"type": "document", "document": {"data": json.dumps(data)}}) - # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated messages.append( {"role": "tool", "tool_call_id": tc.id, "content": tool_content} ) @@ -257,7 +257,7 @@ tools = [ "properties": { "location": { "type": "string", - "description": "the location to get weather. Provide just the city name without the country name.", + "description": "the location to get the weather, example: San Francisco.", } }, "required": ["location"], @@ -330,13 +330,13 @@ while response.message.tool_calls: tool_content = [] print(tool_result) for data in tool_result: + # Optional: the "document" object can take an "id" field for use in citations, otherwise auto-generated tool_content.append( { "type": "document", "document": {"data": json.dumps(data)}, } ) - # Optional: add an "id" field in the "document" object, otherwise IDs are auto-generated messages.append( { "role": "tool", From cb765ccb275efd08a18620513e58d61877b0191a Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Fri, 7 Feb 2025 12:01:10 +0800 Subject: [PATCH 07/11] update citation IDs, messages list --- fern/pages/v2/tool-use/tool-use-citations.mdx | 102 +++++++++++++----- fern/pages/v2/tool-use/tool-use-overview.mdx | 21 +++- .../v2/tool-use/tool-use-parameter-types.mdx | 2 +- fern/pages/v2/tool-use/tool-use-streaming.mdx | 2 +- .../v2/tool-use/tool-use-usage-patterns.mdx | 2 +- 5 files changed, 94 insertions(+), 35 deletions(-) diff --git a/fern/pages/v2/tool-use/tool-use-citations.mdx b/fern/pages/v2/tool-use/tool-use-citations.mdx index f2fc2756e..b7a404888 100644 --- a/fern/pages/v2/tool-use/tool-use-citations.mdx +++ b/fern/pages/v2/tool-use/tool-use-citations.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-citations" hidden: false description: >- - TBD + Guide on accessing and utilizing citations generated by the Cohere Chat endpoint for tool use. It covers both non-streaming and streaming modes (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI" @@ -184,15 +184,12 @@ start=16 end=20 text='24°C' sources=[ToolSource(type='tool', id='get_weather_dk start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='get_weather_gh65bt2tcdy1:0', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' ``` -## Citation modes -When running tool use in streaming mode, it’s possible to configure how citations are generated and presented. You can choose between fast citations or accurate citations, depending on your latency and precision needs. - -### Accurate citations -The model produces its answer first, and then, after the entire response is generated, it provides citations that map to specific segments of the response text. This approach may incur slightly higher latency, but it ensures the citation indices are more precisely aligned with the final text segments of the model’s answer. +## Document ID +When passing the tool results from the tool execution step, you can optionally add custom IDs to the `id` field in the `document` object. These IDs will be used by the endpoint as the citation reference. -This is the default option, or you can explicitly specify it by adding the `citation_options={"mode": "accurate"}` argument in the API call. +If you don't provide the `id` field, the ID will be auto-generated in the the format of `:`. Example: `get_weather_1byjy32y4hvq:0`. -Here is an example. To keep it concise, let's start with a pre-defined list of `messages` with the user query, tool calling, and tool results are already available. +Here is an example of using custom IDs. To keep it concise, let's start with a pre-defined list of `messages` with the user query, tool calling, and tool results are already available. ```python PYTHON # ! pip install -U cohere @@ -201,28 +198,28 @@ import json co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys -from cohere import ToolCallV2, ToolCallV2Function - messages = [ {"role": "user", "content": "What's the weather in Madrid and Brasilia?"}, { "role": "assistant", "tool_plan": "I will search for the weather in Madrid and Brasilia.", "tool_calls": [ - ToolCallV2( - id="get_weather_dkf0akqdazjb", - type="function", - function=ToolCallV2Function( - name="get_weather", arguments='{"location":"Madrid"}' - ), - ), - ToolCallV2( - id="get_weather_gh65bt2tcdy1", - type="function", - function=ToolCallV2Function( - name="get_weather", arguments='{"location":"Brasilia"}' - ), - ), + { + "id": "get_weather_dkf0akqdazjb", + "type": "function", + "function": { + "name": "get_weather", + "arguments": '{"location":"Madrid"}' + }, + }, + { + "id": "get_weather_gh65bt2tcdy1", + "type": "function", + "function": { + "name": "get_weather", + "arguments": '{"location":"Brasilia"}' + }, + }, ], }, { @@ -232,7 +229,7 @@ messages = [ { "type": "document", "document": { - "data": '{"temperature": {"madrid": "24\\u00b0C"}}', + "data": '{"temperature": {"madrid": "24°C"}}', "id" : "1" }, } @@ -245,7 +242,7 @@ messages = [ { "type": "document", "document": { - "data": '{"temperature": {"brasilia": "28\\u00b0C"}}', + "data": '{"temperature": {"brasilia": "28°C"}}', "id" : "2" }, } @@ -254,9 +251,60 @@ messages = [ ] ``` +When document IDs are provided, the citation will refer to the documents using these IDs. + +```python PYTHON +response = co.chat( + model="command-r-plus-08-2024", + messages=messages, + tools=tools +) + +print(response.message.content[0].text) + +for citation in response.message.citations: + print(citation, "\n") +``` + +Note the `id` fields in the citations, which refer to the IDs in the `document` object. + +```mdx wordWrap +It's 24°C in Madrid and 28°C in Brasilia. + +start=5 end=9 text='24°C' sources=[ToolSource(type='tool', id='1', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' + +start=24 end=28 text='28°C' sources=[ToolSource(type='tool', id='2', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' +``` + +In contrast, here's an example citation when the IDs are not provided. + +```mdx wordWrap +It is currently 24°C in Madrid and 28°C in Brasilia. + +start=16 end=20 text='24°C' sources=[ToolSource(type='tool', id='get_weather_dkf0akqdazjb:0', tool_output={'temperature': '{"madrid":"24°C"}'})] type='TEXT_CONTENT' + +start=35 end=39 text='28°C' sources=[ToolSource(type='tool', id='get_weather_gh65bt2tcdy1:0', tool_output={'temperature': '{"brasilia":"28°C"}'})] type='TEXT_CONTENT' +``` + +## Citation modes +When running tool use in streaming mode, it’s possible to configure how citations are generated and presented. You can choose between fast citations or accurate citations, depending on your latency and precision needs. + +### Accurate citations +The model produces its answer first, and then, after the entire response is generated, it provides citations that map to specific segments of the response text. This approach may incur slightly higher latency, but it ensures the citation indices are more precisely aligned with the final text segments of the model’s answer. + +This is the default option, or you can explicitly specify it by adding the `citation_options={"mode": "accurate"}` argument in the API call. + +Here is an example using the same list of pre-defined `messages` as the above. + With the `citation_options` mode set to `accurate`, we get the citations after the entire response is generated. ```python PYTHON +# ! pip install -U cohere +import cohere +import json + +co = cohere.ClientV2("COHERE_API_KEY") # Get your free API key here: https://dashboard.cohere.com/api-keys + response = co.chat_stream( model="command-r-plus-08-2024", messages=messages, @@ -291,7 +339,7 @@ The model generates citations inline, as the response is being produced. In stre You can specify it by adding the `citation_options={"mode": "fast"}` argument in the API call. -Here is an example using the same list of pre-defined`messages` as the above. With the `citation_options` mode set to `fast`, we get the citations inline as the model generates the response. +With the `citation_options` mode set to `fast`, we get the citations inline as the model generates the response. ```python PYTHON response = co.chat_stream( diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index cd43a6673..32078e8a3 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-overview" hidden: false description: >- - TBD + An overview of using Cohere's Command family models to integrate with external tools, enabling developers to enhance application functionality and build agentic workflows (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI" @@ -296,10 +296,21 @@ This section provides a more detailed look at how the state is managed via the ` At each step of the workflow, the endpoint requires that we append specific types of information to the `messages` list. This is to ensure that the model has the necessary context to generate its response at a given point. In summary, each single turn of a conversation that involves tool calling consists of: -1. a `user` message containing the user message (`content`) -2. an `assistant` message, containing the tool calling information (`tool_plan` and `tool_calls`) -3. a `tool` message, containing the tool results (`tool_call_id` and `content`) -4. a final `assistant` message, containing the response to the user (`content`) +1. A `user` message containing the user message + - `content` +2. An `assistant` message, containing the tool calling information + - `tool_plan` + - `tool_calls` + - `id` + - `type` + - `function` (consisting of `name` and `arguments`) +3. A `tool` message, containing the tool results + - `tool_call_id` + - `content` containing a list of documents where each document contains the following fields: + - `type` + - `document` (consisting of `data` and optionally `id`) +4. A final `assistant` message, containing the model's response + - `content` These correspond to the four steps described above. The list of `messages` is shown below. diff --git a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx index cc738b25d..cb44bb7e2 100644 --- a/fern/pages/v2/tool-use/tool-use-parameter-types.mdx +++ b/fern/pages/v2/tool-use/tool-use-parameter-types.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-parameter-types" hidden: false description: >- - TBD + Guide on using structured outputs with tool parameters in the Cohere Chat API. Includes guide on supported parameter types and usage examples (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI" diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index f7ae7b036..dec2d9127 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-streaming" hidden: false description: >- - TBD + Gguide on implementing streaming for tool use in Cohere's platform. Includes the necessary steps to enable response streaming and details the structure of streamed events (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI" diff --git a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx index 9204ddd25..e8eacd7f2 100644 --- a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx +++ b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-usage-patterns" hidden: false description: >- - TBD + Guide on implementing various tool use patterns with the Cohere Chat endpoint such as parallel tool calling, multi-step tool use, and more (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI" From a829b428b6c469ad643704602c07abc12c4eeceb Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Mon, 10 Feb 2025 14:51:00 +0800 Subject: [PATCH 08/11] update overview and usage patterns examples --- fern/pages/v2/tool-use/tool-use-overview.mdx | 73 ++++++++++++++++++- .../v2/tool-use/tool-use-usage-patterns.mdx | 22 ++---- 2 files changed, 80 insertions(+), 15 deletions(-) diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index 32078e8a3..1a68c9faa 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -166,7 +166,10 @@ Next, we call the Chat endpoint to generate the list of tool calls. This is done The endpoint will send back a list of tool calls to be made if the model determines that tools are required. If it does, it will return two types of information: - `tool_plan`: its reflection on the next steps it should take, given the user query. -- `tool_calls`: a list of tool calls to be made (if any), together with auto-generated tool call IDs. +- `tool_calls`: a list of tool calls to be made (if any), together with auto-generated tool call IDs. Each generated tool call contains: + - `id`: the tool call ID + - `type`: the type of the tool call (`function`) + - `function`: the function to be called, which contains the function's `name` and `arguments` to be passed to the function. We then append these to the `messages` list with the `role` set to `assistant`. @@ -203,6 +206,74 @@ I will search for the weather in Toronto. ] ``` +By default, when using the Python SDK, the endpoint passes the tool calls as objects of type `ToolCallV2` and `ToolCallV2Function`. With these, you get built-in type safety and validation that helps prevent common errors during development. + +Alternatively, you can use plain dictionaries to structure the tool call message. + +These two options are shown below. + + + +```python PYTHON +messages = [ + {"role": "user", "content": "What's the weather in Madrid and Brasilia?"}, + { + "role": "assistant", + "tool_plan": "I will search for the weather in Madrid and Brasilia.", + "tool_calls": [ + ToolCallV2( + id="get_weather_dkf0akqdazjb", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Madrid"}' + ), + ), + ToolCallV2( + id="get_weather_gh65bt2tcdy1", + type="function", + function=ToolCallV2Function( + name="get_weather", arguments='{"location":"Brasilia"}' + ), + ), + ], + }, +] +``` + + + + +```python PYTHON +messages = [ + {"role": "user", "content": "What's the weather in Madrid and Brasilia?"}, + { + "role": "assistant", + "tool_plan": "I will search for the weather in Madrid and Brasilia.", + "tool_calls": [ + { + "id": "get_weather_dkf0akqdazjb", + "type": "function", + "function": { + "name": "get_weather", + "arguments": '{"location":"Madrid"}' + }, + }, + { + "id": "get_weather_gh65bt2tcdy1", + "type": "function", + "function": { + "name": "get_weather", + "arguments": '{"location":"Brasilia"}' + }, + }, + ], + }, +] +``` + + + + The model can decide to *not* make any tool call, and instead, respond to a user message directly. This is described [here](https://docs.cohere.com/docs/tool-use-usage-patterns#directly-answering). diff --git a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx index e8eacd7f2..9d124d6a0 100644 --- a/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx +++ b/fern/pages/v2/tool-use/tool-use-usage-patterns.mdx @@ -218,9 +218,9 @@ Here's the function definitions for the tools: ```python PYTHON def get_weather(location): temperature = { - "bern": 22, - "madrid": 24, - "brasilia": 28 + "bern": "22°C", + "madrid": "24°C", + "brasilia": "28°C" } loc = location.lower() if loc in temperature: @@ -388,7 +388,7 @@ This is reflected in the model's response, where two tool calling-result pairs a ```mdx wordWrap TOOL PLAN: -I will first find out the capital city of Brazil and then I will find out the temperature there. +First, I will search for the capital city of Brazil. Then, I will search for the temperature in that city. TOOL CALLS: Tool name: get_capital_city | Parameters: {"country":"Brazil"} @@ -396,28 +396,22 @@ Tool name: get_capital_city | Parameters: {"country":"Brazil"} TOOL RESULT: [{'capital_city': {'brazil': 'brasilia'}}] TOOL PLAN: -I found out that the capital city of Brazil is Brasilia. Now I will find out the temperature there. +I have found that the capital city of Brazil is Brasilia. Now, I will search for the temperature in Brasilia. TOOL CALLS: Tool name: get_weather | Parameters: {"location":"Brasilia"} ================================================== TOOL RESULT: -[{'temperature': {'brasilia': 28}}] +[{'temperature': {'brasilia': '28°C'}}] RESPONSE: The temperature in Brasilia, the capital city of Brazil, is 28°C. ================================================== CITATIONS: -Start: 19| End:27| Text:'Brasilia' -Sources: -1. get_capital_city_xvvhzenxpraf:0 -{'capital_city': '{"brazil":"brasilia"}'} - - Start: 60| End:65| Text:'28°C.' Sources: -1. get_weather_tj79nmn7agmq:0 -{'temperature': '{"brasilia":28}'} +1. get_weather_p0dage9q1nv4:0 +{'temperature': '{"brasilia":"28°C"}'} ``` **State management** From 1ad6f85f1b7d42ef30ab625106ad0ac6274b39b9 Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Mon, 10 Feb 2025 17:25:41 +0800 Subject: [PATCH 09/11] a few clean up updates --- fern/pages/v2/tool-use/tool-use-faqs.mdx | 14 -------- fern/pages/v2/tool-use/tool-use-overview.mdx | 23 ++++++++++-- fern/pages/v2/tool-use/tool-use-streaming.mdx | 2 +- .../v2/tool-use/tool-use-usage-multi-step.mdx | 36 ------------------- fern/v2.yml | 2 -- 5 files changed, 21 insertions(+), 56 deletions(-) delete mode 100644 fern/pages/v2/tool-use/tool-use-faqs.mdx delete mode 100644 fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx diff --git a/fern/pages/v2/tool-use/tool-use-faqs.mdx b/fern/pages/v2/tool-use/tool-use-faqs.mdx deleted file mode 100644 index feadaa673..000000000 --- a/fern/pages/v2/tool-use/tool-use-faqs.mdx +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Tool use - FAQs" -slug: "v2/docs/tool-use-faqs" - -hidden: false -description: >- - TBD -image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" -keywords: "Cohere, text generation, LLMs, generative AI" - -createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" -updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ---- -[[TODO - FAQs]] \ No newline at end of file diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index 1a68c9faa..b2dffe055 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -68,11 +68,21 @@ In this example, we define a `get_weather` function that returns the temperature ```python PYTHON def get_weather(location): # Implement any logic here - return [{"temperature": "20C"}] - # Return a list of objects e.g. [{"url": "abc.com", "text": "..."}, {"url": "xyz.com", "text": "..."}] + return [{"temperature": "20°C"}] + # Return a JSON object string, or a list of tool content blocks e.g. [{"url": "abc.com", "text": "..."}, {"url": "xyz.com", "text": "..."}] functions_map = {"get_weather": get_weather} ``` +The Chat endpoint accepts [a string or a list of objects](https://docs.cohere.com/reference/chat#request.body.messages.tool.content) as the tool content. Thus, you should format the return value in this way. The following are some examples. + +```python PYTHON +# Example: String +weather_search_results = "20°C" + +# Example: List of objects +weather_search_results = [{"city": "Toronto", "date": "250207", "temperature": "20°C"}, + {"city": "Toronto", "date": "250208", "temperature": "21°C"}] +``` ### Defining the tool schema @@ -126,12 +136,19 @@ We can think of a tool use system as consisting of four components: - The tools At its most basic, these four components interact in a workflow through four steps: - - Step 1: **Get user message**: The LLM gets the user message (via the application). - Step 2: **Generate tool calls**: The LLM decides which tools to call (if any) and generates the tool calls. - Step 3: **Get tool results**: The application executes the tools, and the results are sent to the LLM. - Step 4: **Generate response and citations**: The LLM generates the response and citations back to the user. +As an example, a weather search workflow might looks like the following: +- Step 1: **Get user message**: A user asks, "What's the weather in Toronto?" +- Step 2: **Generate tool calls**: A tool call is made to an external weather service with something like `get_weather(“toronto”)`. +- Step 3: **Get tool results**: The weather service returns the results, e.g. "20°C". +- Step 4: **Generate response and citations**: The model provides the answer, "The weather in Toronto is 20 degrees Celcius". + +The following sections go through the implementation of these steps in detail. + Tool use workflow ### Step 1: Get user message diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index dec2d9127..c8041156b 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -148,7 +148,7 @@ Same as in a [basic chat stream event](https://docs.cohere.com/v2/docs/streaming `citation-start` -Emitted for every citation generated in the response. +Emitted for every citation generated in the response. This event contains the details about a citation such as the `start` and `end` indices of the text that cites a source(s), the corresponding `text`, and the list of `sources`. `citation-end` diff --git a/fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx b/fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx deleted file mode 100644 index 261bf83fb..000000000 --- a/fern/pages/v2/tool-use/tool-use-usage-multi-step.mdx +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: "Multi-step tool use" -slug: "v2/docs/tool-use-multi-step" - -hidden: false -description: >- - TBD -image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" -keywords: "Cohere, text generation, LLMs, generative AI" - -createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" -updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ---- -## Overview -[[TODO - describe multi-step tool use (agents)]] - -The Chat endpoint supports multi-step tool use, which enables the model to perform sequential reasoning. This is especially useful in agentic workflows that require multiple steps to complete a task. - -As an example, suppose a tool use application has access to a web search tool. Given the following question, it will need to perform a series of steps in a sequence, because one step is dependent on the result of the previous step. - -"What was the revenue of the most valuable company in the US in 2023?” - -1. First, it needs to identify the most valuable company in the US in 2023 -2. Then only it can get the revenue figure now that the company has been identified - -## State management -[[TODO - describe how the messages list construction for multi-step differs from single-step]] - -## Multi-step reasoning -[[TODO - example multi-step tool use wrt multi-step reasoning]] - -## Self-correction -[[TODO - example multi-step tool use wrt self-correction]] - -## Multi-step, parallel tool use -[[TODO - example of multi-step, parallel tool use]] \ No newline at end of file diff --git a/fern/v2.yml b/fern/v2.yml index 4901861f7..279e16c1c 100644 --- a/fern/v2.yml +++ b/fern/v2.yml @@ -91,8 +91,6 @@ navigation: path: pages/v2/tool-use/tool-use-streaming.mdx - page: Citations path: pages/v2/tool-use/tool-use-citations.mdx - - page: FAQs - path: pages/v2/tool-use/tool-use-faqs.mdx # - section: Tool Use # path: pages/v2/text-generation/tools.mdx From 6a6f47153485a8e3ce95476e4589ed43711d2446 Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Mon, 10 Feb 2025 17:27:00 +0800 Subject: [PATCH 10/11] remove redundant page --- .../v2/tool-use/tool-use-structured-outputs.mdx | 14 -------------- 1 file changed, 14 deletions(-) delete mode 100644 fern/pages/v2/tool-use/tool-use-structured-outputs.mdx diff --git a/fern/pages/v2/tool-use/tool-use-structured-outputs.mdx b/fern/pages/v2/tool-use/tool-use-structured-outputs.mdx deleted file mode 100644 index 8df3be578..000000000 --- a/fern/pages/v2/tool-use/tool-use-structured-outputs.mdx +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Structured outputs for tool use" -slug: "v2/docs/tool-use-structured-outputs" - -hidden: false -description: >- - TBD -image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" -keywords: "Cohere, text generation, LLMs, generative AI" - -createdAt: "Thu Feb 29 2024 18:05:29 GMT+0000 (Coordinated Universal Time)" -updatedAt: "Tue Jun 18 2024 07:20:15 GMT+0000 (Coordinated Universal Time)" ---- -TBD \ No newline at end of file From 521a6cb9c141219872dab297c2d24b3bebef0e53 Mon Sep 17 00:00:00 2001 From: mrmer1 Date: Mon, 10 Feb 2025 17:30:14 +0800 Subject: [PATCH 11/11] update meta descriptions --- fern/pages/v2/tool-use/tool-use-overview.mdx | 2 +- fern/pages/v2/tool-use/tool-use-streaming.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fern/pages/v2/tool-use/tool-use-overview.mdx b/fern/pages/v2/tool-use/tool-use-overview.mdx index b2dffe055..447c864ee 100644 --- a/fern/pages/v2/tool-use/tool-use-overview.mdx +++ b/fern/pages/v2/tool-use/tool-use-overview.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-overview" hidden: false description: >- - An overview of using Cohere's Command family models to integrate with external tools, enabling developers to enhance application functionality and build agentic workflows (API v2). + An overview of using Cohere's tool use capabilities, enabling developers to build agentic workflows (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI" diff --git a/fern/pages/v2/tool-use/tool-use-streaming.mdx b/fern/pages/v2/tool-use/tool-use-streaming.mdx index c8041156b..98d2137ae 100644 --- a/fern/pages/v2/tool-use/tool-use-streaming.mdx +++ b/fern/pages/v2/tool-use/tool-use-streaming.mdx @@ -4,7 +4,7 @@ slug: "v2/docs/tool-use-streaming" hidden: false description: >- - Gguide on implementing streaming for tool use in Cohere's platform. Includes the necessary steps to enable response streaming and details the structure of streamed events (API v2). + Guide on implementing streaming for tool use in Cohere's platform and details on the events stream (API v2). image: "../../../assets/images/4a5325a-cohere_meta_image.jpg" keywords: "Cohere, text generation, LLMs, generative AI"