Skip to content

Commit

Permalink
docs[minor]: Add TogetherAI updated integration doc (#6314)
Browse files Browse the repository at this point in the history 
* docs[minor]: Add TogetherAI updated integration doc

* drop old file
  • Loading branch information
@bracesproul
bracesproul authored Aug 1, 2024
1 parent abda1e8 commit 6187ab4
Show file tree
Hide file tree
Showing 2 changed files with 343 additions and 65 deletions.
343 changes: 343 additions & 0 deletions docs/core_docs/docs/integrations/chat/togetherai.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,343 @@
{
"cells": [
{
"cell_type": "raw",
"id": "afaf8039",
"metadata": {
"vscode": {
"languageId": "raw"
}
},
"source": [
"---\n",
"sidebar_label: Together\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "e49f1e0d",
"metadata": {},
"source": [
"# ChatTogetherAI\n",
"\n",
"This will help you getting started with `ChatTogetherAI` [chat models](/docs/concepts/#chat-models). For detailed documentation of all `ChatTogetherAI` features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain_community_chat_models_togetherai.ChatTogetherAI.html).\n",
"\n",
"## Overview\n",
"### Integration details\n",
"\n",
"| Class | Package | Local | Serializable | [PY support](https://python.langchain.com/v0.2/docs/integrations/chat/togetherai) | Package downloads | Package latest |\n",
"| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
"| [ChatTogetherAI](https://api.js.langchain.com/classes/langchain_community_chat_models_togetherai.ChatTogetherAI.html) | [@langchain/community](https://api.js.langchain.com/modules/langchain_community_chat_models_togetherai.html) | ❌ | ✅ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/community?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/community?style=flat-square&label=%20&) |\n",
"\n",
"### Model features\n",
"| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
"| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",
"| ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | \n",
"\n",
"## Setup\n",
"\n",
"To access `ChatTogetherAI` models you'll need to create a Together account, get an API key [here](https://api.together.xyz/), and install the `@langchain/community` integration package.\n",
"\n",
"### Credentials\n",
"\n",
"Head to [api.together.ai](https://api.together.ai/) to sign up to TogetherAI and generate an API key. Once you've done this set the `TOGETHER_AI_API_KEY` environment variable:\n",
"\n",
"```bash\n",
"export TOGETHER_AI_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
"\n",
"```bash\n",
"# export LANGCHAIN_TRACING_V2=\"true\"\n",
"# export LANGCHAIN_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"### Installation\n",
"\n",
"The LangChain ChatTogetherAI integration lives in the `@langchain/community` package:\n",
"\n",
"```{=mdx}\n",
"import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
"import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
"\n",
"<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
"\n",
"<Npm2Yarn>\n",
" @langchain/community\n",
"</Npm2Yarn>\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "a38cde65-254d-4219-a441-068766c0d4b5",
"metadata": {},
"source": [
"## Instantiation\n",
"\n",
"Now we can instantiate our model object and generate chat completions:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
"metadata": {},
"outputs": [],
"source": [
"import { ChatTogetherAI } from \"@langchain/community/chat_models/togetherai\"\n",
"\n",
"const llm = new ChatTogetherAI({\n",
" model: \"mistralai/Mixtral-8x7B-Instruct-v0.1\",\n",
" temperature: 0,\n",
" maxTokens: undefined,\n",
" timeout: undefined,\n",
" maxRetries: 2,\n",
" // other params...\n",
"})"
]
},
{
"cell_type": "markdown",
"id": "2b4f3e15",
"metadata": {},
"source": [
"## Invocation"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "62e0dbc3",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"AIMessage {\n",
" \"id\": \"chatcmpl-9rT9qEDPZ6iLCk6jt3XTzVDDH6pcI\",\n",
" \"content\": \"J'adore la programmation.\",\n",
" \"additional_kwargs\": {},\n",
" \"response_metadata\": {\n",
" \"tokenUsage\": {\n",
" \"completionTokens\": 8,\n",
" \"promptTokens\": 31,\n",
" \"totalTokens\": 39\n",
" },\n",
" \"finish_reason\": \"stop\"\n",
" },\n",
" \"tool_calls\": [],\n",
" \"invalid_tool_calls\": [],\n",
" \"usage_metadata\": {\n",
" \"input_tokens\": 31,\n",
" \"output_tokens\": 8,\n",
" \"total_tokens\": 39\n",
" }\n",
"}\n"
]
}
],
"source": [
"const aiMsg = await llm.invoke([\n",
" [\n",
" \"system\",\n",
" \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
" ],\n",
" [\"human\", \"I love programming.\"],\n",
"])\n",
"aiMsg"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"J'adore la programmation.\n"
]
}
],
"source": [
"console.log(aiMsg.content)"
]
},
{
"cell_type": "markdown",
"id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",
"metadata": {},
"source": [
"## Chaining\n",
"\n",
"We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"AIMessage {\n",
" \"id\": \"chatcmpl-9rT9wolZWfJ3xovORxnkdf1rcPbbY\",\n",
" \"content\": \"Ich liebe das Programmieren.\",\n",
" \"additional_kwargs\": {},\n",
" \"response_metadata\": {\n",
" \"tokenUsage\": {\n",
" \"completionTokens\": 6,\n",
" \"promptTokens\": 26,\n",
" \"totalTokens\": 32\n",
" },\n",
" \"finish_reason\": \"stop\"\n",
" },\n",
" \"tool_calls\": [],\n",
" \"invalid_tool_calls\": [],\n",
" \"usage_metadata\": {\n",
" \"input_tokens\": 26,\n",
" \"output_tokens\": 6,\n",
" \"total_tokens\": 32\n",
" }\n",
"}\n"
]
}
],
"source": [
"import { ChatPromptTemplate } from \"@langchain/core/prompts\"\n",
"\n",
"const prompt = ChatPromptTemplate.fromMessages(\n",
" [\n",
" [\n",
" \"system\",\n",
" \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
" ],\n",
" [\"human\", \"{input}\"],\n",
" ]\n",
")\n",
"\n",
"const chain = prompt.pipe(llm);\n",
"await chain.invoke(\n",
" {\n",
" input_language: \"English\",\n",
" output_language: \"German\",\n",
" input: \"I love programming.\",\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d1ee55bc-ffc8-4cfa-801c-993953a08cfd",
"metadata": {},
"source": [
"## Tool calling & JSON mode\n",
"\n",
"The TogetherAI chat supports JSON mode and calling tools.\n",
"\n",
"### Tool calling"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "8de584a8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\n",
" {\n",
" name: 'calculator',\n",
" args: { input: '2 + 3' },\n",
" type: 'tool_call',\n",
" id: 'call_nhtnmganqJPAG9I1cN8ULI9R'\n",
" }\n",
"]\n"
]
}
],
"source": [
"import { ChatTogetherAI } from \"@langchain/community/chat_models/togetherai\";\n",
"import { ChatPromptTemplate } from \"@langchain/core/prompts\";\n",
"import { convertToOpenAITool } from \"@langchain/core/utils/function_calling\";\n",
"import { Calculator } from \"@langchain/community/tools/calculator\";\n",
"\n",
"// Use a pre-built tool\n",
"const calculatorTool = convertToOpenAITool(new Calculator());\n",
"\n",
"const modelWithCalculator = new ChatTogetherAI({\n",
" temperature: 0,\n",
" // This is the default env variable name it will look for if none is passed.\n",
" apiKey: process.env.TOGETHER_AI_API_KEY,\n",
" // Together JSON mode/tool calling only supports a select number of models\n",
" model: \"mistralai/Mixtral-8x7B-Instruct-v0.1\",\n",
"}).bind({\n",
" // Bind the tool to the model.\n",
" tools: [calculatorTool],\n",
" tool_choice: calculatorTool, // Specify what tool the model should use\n",
"});\n",
"\n",
"const promptForTools = ChatPromptTemplate.fromMessages([\n",
" [\"system\", \"You are a super not-so-smart mathmatician.\"],\n",
" [\"human\", \"Help me out, how can I add {math}?\"],\n",
"]);\n",
"\n",
"// Use LCEL to chain the prompt to the model.\n",
"const responseWithTool = await promptForTools.pipe(modelWithCalculator).invoke({\n",
" math: \"2 plus 3\",\n",
"});\n",
"\n",
"console.dir(responseWithTool.tool_calls, { depth: null });"
]
},
{
"cell_type": "markdown",
"id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
"metadata": {},
"source": [
"Behind the scenes, TogetherAI uses the OpenAI SDK and OpenAI compatible API, with some caveats:\n",
"\n",
"- Certain properties are not supported by the TogetherAI API, see [here](https://docs.together.ai/reference/chat-completions).\n",
"\n",
"## API reference\n",
"\n",
"For detailed documentation of all ChatTogetherAI features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_community_chat_models_togetherai.ChatTogetherAI.html"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "TypeScript",
"language": "typescript",
"name": "tslab"
},
"language_info": {
"codemirror_mode": {
"mode": "typescript",
"name": "javascript",
"typescript": true
},
"file_extension": ".ts",
"mimetype": "text/typescript",
"name": "typescript",
"version": "3.7.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading

0 comments on commit 6187ab4

Please sign in to comment.