Skip to content

Commit

Permalink
docs[minor]: Update mistral embedding docs (#6362)
Browse files Browse the repository at this point in the history
  • Loading branch information
@bracesproul
bracesproul authored Aug 5, 2024
1 parent 18f5923 commit 134301e
Show file tree
Hide file tree
Showing 2 changed files with 344 additions and 28 deletions.
344 changes: 344 additions & 0 deletions docs/core_docs/docs/integrations/text_embedding/mistralai.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,344 @@
{
"cells": [
{
"cell_type": "raw",
"id": "afaf8039",
"metadata": {
"vscode": {
"languageId": "raw"
}
},
"source": [
"---\n",
"sidebar_label: MistralAI\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "9a3d6f34",
"metadata": {},
"source": [
"# MistralAIEmbeddings\n",
"\n",
"This will help you get started with MistralAIEmbeddings [embedding models](/docs/concepts#embedding-models) using LangChain. For detailed documentation on `MistralAIEmbeddings` features and configuration options, please refer to the [API reference](https://api.js.langchain.com/classes/langchain_mistralai.MistralAIEmbeddings.html).\n",
"\n",
"## Overview\n",
"### Integration details\n",
"\n",
"| Class | Package | Local | [Py support](https://python.langchain.com/docs/integrations/text_embedding/mistralai/) | Package downloads | Package latest |\n",
"| :--- | :--- | :---: | :---: | :---: | :---: |\n",
"| [MistralAIEmbeddings](https://api.js.langchain.com/classes/langchain_mistralai.MistralAIEmbeddings.html) | [@langchain/mistralai](https://api.js.langchain.com/modules/langchain_mistralai.html) | ❌ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/mistralai?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/mistralai?style=flat-square&label=%20&) |\n",
"\n",
"## Setup\n",
"\n",
"To access MistralAI embedding models you'll need to create a MistralAI account, get an API key, and install the `@langchain/mistralai` integration package.\n",
"\n",
"### Credentials\n",
"\n",
"Head to [console.mistral.ai](https://console.mistral.ai/) to sign up to `MistralAI` and generate an API key. Once you've done this set the `MISTRAL_API_KEY` environment variable:\n",
"\n",
"```bash\n",
"export MISTRAL_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
"\n",
"```bash\n",
"# export LANGCHAIN_TRACING_V2=\"true\"\n",
"# export LANGCHAIN_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"### Installation\n",
"\n",
"The LangChain MistralAIEmbeddings integration lives in the `@langchain/mistralai` package:\n",
"\n",
"```{=mdx}\n",
"import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
"import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
"\n",
"<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
"\n",
"<Npm2Yarn>\n",
" @langchain/mistralai\n",
"</Npm2Yarn>\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "45dd1724",
"metadata": {},
"source": [
"## Instantiation\n",
"\n",
"Now we can instantiate our model object and generate chat completions:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "9ea7a09b",
"metadata": {},
"outputs": [],
"source": [
"import { MistralAIEmbeddings } from \"@langchain/mistralai\";\n",
"\n",
"const embeddings = new MistralAIEmbeddings({\n",
" model: \"mistral-embed\", // Default value\n",
"});"
]
},
{
"cell_type": "markdown",
"id": "77d271b6",
"metadata": {},
"source": [
"## Indexing and Retrieval\n",
"\n",
"Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our RAG tutorials under the [working with external knowledge tutorials](/docs/tutorials/#working-with-external-knowledge).\n",
"\n",
"Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document using the demo [`MemoryVectorStore`](/docs/integrations/vectorstores/memory)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d817716b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LangChain is the framework for building context-aware reasoning applications\n"
]
}
],
"source": [
"// Create a vector store with a sample text\n",
"import { MemoryVectorStore } from \"langchain/vectorstores/memory\";\n",
"\n",
"const text = \"LangChain is the framework for building context-aware reasoning applications\";\n",
"\n",
"const vectorstore = await MemoryVectorStore.fromDocuments(\n",
" [{ pageContent: text, metadata: {} }],\n",
" embeddings,\n",
");\n",
"\n",
"// Use the vector store as a retriever that returns a single document\n",
"const retriever = vectorstore.asRetriever(1);\n",
"\n",
"// Retrieve the most similar text\n",
"const retrievedDocuments = await retriever.invoke(\"What is LangChain?\");\n",
"\n",
"retrievedDocuments[0].pageContent;"
]
},
{
"cell_type": "markdown",
"id": "e02b9855",
"metadata": {},
"source": [
"## Direct Usage\n",
"\n",
"Under the hood, the vectorstore and retriever implementations are calling `embeddings.embedDocument(...)` and `embeddings.embedQuery(...)` to create embeddings for the text(s) used in `fromDocuments` and the retriever's `invoke` operations, respectively.\n",
"\n",
"You can directly call these methods to get embeddings for your own use cases.\n",
"\n",
"### Embed single texts\n",
"\n",
"You can embed queries for search with `embedQuery`. This generates a vector representation specific to the query:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "0d2befcd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\n",
" -0.04443359375, 0.01885986328125, 0.018035888671875,\n",
" -0.00864410400390625, 0.049652099609375, -0.001190185546875,\n",
" 0.028900146484375, -0.035675048828125, -0.00702667236328125,\n",
" 0.00016105175018310547, -0.027587890625, 0.029388427734375,\n",
" -0.053253173828125, -0.0003020763397216797, -0.046112060546875,\n",
" 0.0258026123046875, -0.0010776519775390625, 0.02703857421875,\n",
" 0.040985107421875, -0.004547119140625, -0.020172119140625,\n",
" -0.02606201171875, -0.01457977294921875, 0.01220703125,\n",
" -0.0078582763671875, -0.0084228515625, -0.02056884765625,\n",
" -0.071044921875, -0.0404052734375, 0.00923919677734375,\n",
" 0.01407623291015625, -0.0210113525390625, 0.0006284713745117188,\n",
" -0.01465606689453125, 0.0186309814453125, -0.015838623046875,\n",
" 0.0007920265197753906, -0.04437255859375, 0.008758544921875,\n",
" -0.0172119140625, 0.01312255859375, -0.01358795166015625,\n",
" -0.0212860107421875, -0.000035822391510009766, -0.0226898193359375,\n",
" -0.01390838623046875, -0.007659912109375, -0.016021728515625,\n",
" 0.025909423828125, -0.034515380859375, -0.0372314453125,\n",
" 0.020355224609375, -0.02606201171875, -0.0158843994140625,\n",
" -0.037994384765625, 0.00450897216796875, 0.0142822265625,\n",
" -0.012725830078125, -0.0770263671875, 0.02630615234375,\n",
" -0.048614501953125, 0.006072998046875, 0.00417327880859375,\n",
" -0.005138397216796875, 0.02557373046875, 0.0311279296875,\n",
" 0.026519775390625, -0.0103607177734375, -0.0108489990234375,\n",
" -0.029510498046875, 0.022186279296875, 0.0256500244140625,\n",
" -0.0186309814453125, 0.0443115234375, -0.0304107666015625,\n",
" -0.03131103515625, 0.007427215576171875, 0.0234527587890625,\n",
" 0.0224761962890625, 0.00463104248046875, -0.0037021636962890625,\n",
" 0.0302581787109375, 0.0733642578125, -0.0121612548828125,\n",
" -0.0172576904296875, 0.019317626953125, 0.029052734375,\n",
" -0.0024871826171875, 0.0174713134765625, 0.026092529296875,\n",
" 0.04425048828125, -0.0004563331604003906, 0.0146026611328125,\n",
" -0.00748443603515625, 0.06146240234375, 0.02294921875,\n",
" -0.016845703125, -0.0014057159423828125, -0.01435089111328125,\n",
" 0.06097412109375\n",
"]\n"
]
}
],
"source": [
"const singleVector = await embeddings.embedQuery(text);\n",
"\n",
"console.log(singleVector.slice(0, 100));"
]
},
{
"cell_type": "markdown",
"id": "1b5a7d03",
"metadata": {},
"source": [
"### Embed multiple texts\n",
"\n",
"You can embed multiple texts for indexing with `embedDocuments`. The internals used for this method may (but do not have to) differ from embedding queries:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2f4d6e97",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[\n",
" -0.04443359375, 0.01885986328125, 0.0180511474609375,\n",
" -0.0086517333984375, 0.049652099609375, -0.00121307373046875,\n",
" 0.0289154052734375, -0.03570556640625, -0.007015228271484375,\n",
" 0.0001499652862548828, -0.0276641845703125, 0.0294036865234375,\n",
" -0.05322265625, -0.0002808570861816406, -0.04608154296875,\n",
" 0.02581787109375, -0.0011072158813476562, 0.027099609375,\n",
" 0.040985107421875, -0.004547119140625, -0.0201873779296875,\n",
" -0.0260772705078125, -0.0146026611328125, 0.0121917724609375,\n",
" -0.007843017578125, -0.0084381103515625, -0.0205535888671875,\n",
" -0.07110595703125, -0.04046630859375, 0.00931549072265625,\n",
" 0.01409912109375, -0.02099609375, 0.0006232261657714844,\n",
" -0.014678955078125, 0.0186614990234375, -0.0158233642578125,\n",
" 0.000812530517578125, -0.04437255859375, 0.00873565673828125,\n",
" -0.0172119140625, 0.013092041015625, -0.0135498046875,\n",
" -0.0212860107421875, -0.000006735324859619141, -0.0226898193359375,\n",
" -0.01389312744140625, -0.0076751708984375, -0.0160064697265625,\n",
" 0.0259246826171875, -0.0345458984375, -0.037200927734375,\n",
" 0.020355224609375, -0.0260009765625, -0.0159149169921875,\n",
" -0.03802490234375, 0.004489898681640625, 0.0143280029296875,\n",
" -0.01274871826171875, -0.07708740234375, 0.0263214111328125,\n",
" -0.04864501953125, 0.00608062744140625, 0.004192352294921875,\n",
" -0.005115509033203125, 0.0255889892578125, 0.0311279296875,\n",
" 0.0265045166015625, -0.0103607177734375, -0.01084136962890625,\n",
" -0.0294952392578125, 0.022186279296875, 0.0256500244140625,\n",
" -0.0186767578125, 0.044342041015625, -0.030426025390625,\n",
" -0.03131103515625, 0.007396697998046875, 0.0234527587890625,\n",
" 0.0224609375, 0.004634857177734375, -0.003643035888671875,\n",
" 0.0302886962890625, 0.07342529296875, -0.01221466064453125,\n",
" -0.017303466796875, 0.0193023681640625, 0.029052734375,\n",
" -0.0024890899658203125, 0.0174407958984375, 0.026123046875,\n",
" 0.044219970703125, -0.0004944801330566406, 0.01462554931640625,\n",
" -0.007450103759765625, 0.06146240234375, 0.022979736328125,\n",
" -0.016845703125, -0.001445770263671875, -0.0143890380859375,\n",
" 0.06097412109375\n",
"]\n",
"[\n",
" -0.02032470703125, 0.02606201171875, 0.051605224609375,\n",
" -0.0281982421875, 0.055755615234375, 0.001987457275390625,\n",
" 0.031982421875, -0.0131378173828125, -0.0252685546875,\n",
" 0.001010894775390625, -0.024017333984375, 0.053375244140625,\n",
" -0.042816162109375, 0.005584716796875, -0.04132080078125,\n",
" 0.03021240234375, 0.01324462890625, 0.016876220703125,\n",
" 0.041961669921875, -0.004299163818359375, -0.0273895263671875,\n",
" -0.039642333984375, -0.021575927734375, 0.0309295654296875,\n",
" -0.0099945068359375, -0.0163726806640625, -0.00968170166015625,\n",
" -0.07733154296875, -0.030364990234375, -0.003864288330078125,\n",
" 0.016387939453125, -0.0389404296875, -0.0026702880859375,\n",
" -0.0176544189453125, 0.0264434814453125, -0.01226806640625,\n",
" -0.0022220611572265625, -0.039703369140625, -0.00907135009765625,\n",
" -0.0260467529296875, 0.03155517578125, -0.0004324913024902344,\n",
" -0.019500732421875, -0.0120697021484375, -0.008544921875,\n",
" -0.01654052734375, 0.00067138671875, -0.0134735107421875,\n",
" 0.01080322265625, -0.034759521484375, -0.06201171875,\n",
" 0.012359619140625, -0.006237030029296875, -0.0168914794921875,\n",
" -0.0183563232421875, 0.0236053466796875, -0.0021419525146484375,\n",
" -0.0164947509765625, -0.052581787109375, 0.022125244140625,\n",
" -0.045745849609375, -0.0009088516235351562, 0.0097808837890625,\n",
" -0.0009326934814453125, 0.041656494140625, 0.0269775390625,\n",
" 0.016845703125, -0.0022335052490234375, -0.0182342529296875,\n",
" -0.0245208740234375, 0.0036602020263671875, -0.0188751220703125,\n",
" -0.0023956298828125, 0.0238800048828125, -0.034942626953125,\n",
" -0.033782958984375, 0.0046234130859375, 0.0318603515625,\n",
" 0.0251007080078125, -0.0023288726806640625, -0.0225677490234375,\n",
" 0.0004394054412841797, 0.064208984375, -0.0254669189453125,\n",
" -0.0234222412109375, 0.0009264945983886719, 0.01464080810546875,\n",
" 0.006626129150390625, -0.007450103759765625, 0.02642822265625,\n",
" 0.0260009765625, 0.00536346435546875, 0.01479339599609375,\n",
" -0.0032253265380859375, 0.0498046875, 0.048248291015625,\n",
" -0.01519012451171875, 0.00605010986328125, 0.019744873046875,\n",
" 0.0296478271484375\n",
"]\n"
]
}
],
"source": [
"const text2 = \"LangGraph is a library for building stateful, multi-actor applications with LLMs\";\n",
"\n",
"const vectors = await embeddings.embedDocuments([text, text2]);\n",
"\n",
"console.log(vectors[0].slice(0, 100));\n",
"console.log(vectors[1].slice(0, 100));"
]
},
{
"cell_type": "markdown",
"id": "8938e581",
"metadata": {},
"source": [
"## API reference\n",
"\n",
"For detailed documentation of all MistralAIEmbeddings features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_mistralai.MistralAIEmbeddings.html"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "TypeScript",
"language": "typescript",
"name": "tslab"
},
"language_info": {
"codemirror_mode": {
"mode": "typescript",
"name": "javascript",
"typescript": true
},
"file_extension": ".ts",
"mimetype": "text/typescript",
"name": "typescript",
"version": "3.7.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
28 changes: 0 additions & 28 deletions docs/core_docs/docs/integrations/text_embedding/mistralai.mdx
View file

This file was deleted.

0 comments on commit 134301e

Please sign in to comment.