Skip to content

Commit

Permalink
docs[minor]: bedrock tool calling/WSO docs (#5735)
Browse files Browse the repository at this point in the history
* community[minor]: Add withStructuredOutput to bedrock

* implement .bind, fix call options, add docs

* chore: lint files

* merge main
  • Loading branch information
bracesproul authored Jun 23, 2024
1 parent b737245 commit ff0ac6e
Show file tree
Hide file tree
Showing 3 changed files with 125 additions and 0 deletions.
28 changes: 28 additions & 0 deletions docs/core_docs/docs/integrations/chat/bedrock.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,31 @@ Anthropic Claude-3 models hosted on Bedrock have multimodal capabilities and can
import BedrockMultimodalExample from "@examples/models/chat/integration_bedrock_multimodal.ts";

<CodeBlock language="typescript">{BedrockMultimodalExample}</CodeBlock>

### Tool calling

:::info
Not all Bedrock models support tool calling. Please refer to the [model documentation](https://docs.aws.amazon.com/bedrock/latest/APIReference/welcome.html) for more information.
:::

The examples below demonstrate how to use tool calling, along with the `withStructuredOutput` method to easily compose structured output LLM calls.

import ToolCalling from "@examples/models/chat/integration_bedrock_tools.ts";

<CodeBlock language="typescript">{ToolCalling}</CodeBlock>

:::tip
See the LangSmith trace [here](https://smith.langchain.com/public/003a684d-90eb-406e-a146-8ee5e617921b/r)
:::

#### `.withStructuredOutput({ ... })`

Using the `.withStructuredOutput` method, you can easily make the LLM return structured output, given only a Zod or JSON schema:

import WSOExample from "@examples/models/chat/integration_bedrock_wso.ts";

<CodeBlock language="typescript">{WSOExample}</CodeBlock>

:::tip
See the LangSmith trace [here](https://smith.langchain.com/public/1f7b1ad8-e4ac-4965-8ce1-fae06005f3d7/r)
:::
63 changes: 63 additions & 0 deletions examples/src/models/chat/integration_bedrock_tools.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { BedrockChat } from "@langchain/community/chat_models/bedrock";
// Or, from web environments:
// import { BedrockChat } from "@langchain/community/chat_models/bedrock/web";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

const model = new BedrockChat({
region: process.env.BEDROCK_AWS_REGION,
model: "anthropic.claude-3-sonnet-20240229-v1:0",
maxRetries: 0,
credentials: {
secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
},
});

const weatherSchema = z
.object({
city: z.string().describe("The city to get the weather for"),
state: z.string().describe("The state to get the weather for").optional(),
})
.describe("Get the weather for a city");

const modelWithTools = model.bindTools([
{
name: "weather_tool",
description: weatherSchema.description,
input_schema: zodToJsonSchema(weatherSchema),
},
]);
// Optionally, you can bind tools via the `.bind` method:
// const modelWithTools = model.bind({
// tools: [
// {
// name: "weather_tool",
// description: weatherSchema.description,
// input_schema: zodToJsonSchema(weatherSchema),
// },
// ],
// });

const res = await modelWithTools.invoke("What's the weather in New York?");
console.log(res);

/*
AIMessage {
additional_kwargs: { id: 'msg_bdrk_01JF7hb4PNQPywP4gnBbgpHi' },
response_metadata: {
stop_reason: 'tool_use',
usage: { input_tokens: 300, output_tokens: 85 }
},
tool_calls: [
{
name: 'weather_tool',
args: {
city: 'New York',
state: 'NY'
},
id: 'toolu_bdrk_01AtEZRTCKioFXqhoNcpgaV7'
}
],
}
*/
34 changes: 34 additions & 0 deletions examples/src/models/chat/integration_bedrock_wso.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { BedrockChat } from "@langchain/community/chat_models/bedrock";
// Or, from web environments:
// import { BedrockChat } from "@langchain/community/chat_models/bedrock/web";
import { z } from "zod";

const model = new BedrockChat({
region: process.env.BEDROCK_AWS_REGION,
model: "anthropic.claude-3-sonnet-20240229-v1:0",
maxRetries: 0,
credentials: {
secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
},
});

const weatherSchema = z
.object({
city: z.string().describe("The city to get the weather for"),
state: z.string().describe("The state to get the weather for").optional(),
})
.describe("Get the weather for a city");

const modelWithStructuredOutput = model.withStructuredOutput(weatherSchema, {
name: "weather_tool", // Optional, defaults to 'extract'
});

const res = await modelWithStructuredOutput.invoke(
"What's the weather in New York?"
);
console.log(res);

/*
{ city: 'New York', state: 'NY' }
*/

0 comments on commit ff0ac6e

Please sign in to comment.