Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

langchain_community:async_documentdb_vectorstore #20825

Closed
wants to merge 4 commits into from
Closed

langchain_community:async_documentdb_vectorstore #20825

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1be6cca
async_documentdb_vectorstore
alon-parag-taiwa Apr 24, 2024
f6d3699
Merge branch 'master' into async-documentdb
Alonoparag Apr 24, 2024
6c89276
Formatting and moving the notebook to the docs
alon-parag-taiwa Apr 24, 2024
16c9c56
Merge branch 'master' into async-documentdb
baskaryan Apr 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
241 changes: 241 additions & 0 deletions docs/docs/integrations/async_rag_with_aws_documentdb.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install nest-asyncio"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"env_data = {\n",
" \"OPENAI_API_KEY\":\"<your-api-key>\",\n",
" \"DOCUMENTDB_CONNECTION_STRING\":r\"<your-connection-string>\"\n",
" }\n",
"\n",
"os.environ.update(env_data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Assume that you are passionate about baking cakes, and you are looking for some recipes for your next cake. Fortunately you have a large database of cakes. Which cake would be best suited for you?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_core.documents import Document\n",
"recipes = [\n",
" Document(page_content=\"In order to make the famous langchain chocolate cake Blend dark chocolate and cocoa powder for a rich chocolate cake base.\", metadata={\"recipeId\": \"101\"}),\n",
" Document(page_content=\"In order to make the famous langchain cheese cake Combine cream cheese, sugar, and eggs to prepare the cheesecake batter.\", metadata={\"recipeId\": \"102\"}),\n",
" Document(page_content=\"In order to make the famous langchain orange cake Stir together flour, fresh orange zest, and orange juice for a zesty orange cake.\", metadata={\"recipeId\": \"103\"}),\n",
" Document(page_content=\"In order to make the famous langchain banan cake Mix mashed bananas with cinnamon and nutmeg for a spiced banana cake.\", metadata={\"recipeId\": \"104\"}),\n",
" Document(page_content=\"In order to make the famous langchain cardmom cake Whisk eggs, almond flour, and ground cardamom for a fragrant cardamom cake.\", metadata={\"recipeId\": \"105\"})\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# You can create the collection using afrom_documents"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_community.vectorstores.async_documentdb import AsyncDocumentDBVectorSearch\n",
"from langchain_community.embeddings.openai import OpenAIEmbeddings\n",
"from motor.motor_asyncio import AsyncIOMotorClient\n",
"import os\n",
"\n",
"connection_string = os.environ.get(\"DOCUMENTDB_CONNECTION_STRING\")\n",
"\n",
"namespace = \"cookbook.cakes\"\n",
"db_name, collection_name = namespace.split(\".\")\n",
"motor_client = AsyncIOMotorClient(connection_string)\n",
"collection = motor_client[db_name][collection_name]\n",
"if collection_name in await motor_client[db_name].list_collection_names():\n",
" await collection.drop()\n",
"index_name = \"cakes\"\n",
"\n",
"embedding_model = OpenAIEmbeddings(model=\"text-embedding-ada-002\")\n",
"\n",
"# Vector dimension for the index similarity\n",
"dimensions = 1536\n",
"vectorstore = await AsyncDocumentDBVectorSearch.afrom_documents(\n",
" recipes,\n",
" embedding_model,\n",
" collection=collection,\n",
" index_name=index_name\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Lets create the index ove the documents\n",
"from langchain_community.vectorstores.async_documentdb import DocumentDBSimilarityType\n",
"\n",
"\n",
"await vectorstore.acreate_index(dimensions, DocumentDBSimilarityType.COS)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#Lets create a chain that can call the vectorstore\n",
"from langchain_community.chat_models.openai import ChatOpenAI\n",
"from langchain_core.prompts.chat import ChatPromptTemplate\n",
"from langchain_core.runnables.passthrough import RunnablePassthrough\n",
"\n",
"# We define a prompt here, `recipe` is the placeholder for the most suitable recipe\n",
"prompt = \"\"\"\n",
"You are an expert conditor, Please recommend a suitable cake recipe from langchain.\n",
"{recipe}\n",
"\"\"\"\n",
"\n",
"prompt_template = ChatPromptTemplate.from_messages(\n",
" [('system', prompt),\n",
" (\"human\",\"{message}\")]\n",
")\n",
"\n",
"# Vectorstores can also create a retriever object which could be used inside a chain to retrieve relevant objects.\n",
"# Additionaly, you can add aggregation commands such as $match to this call to filter on specific fields.\n",
"# for example\n",
"# retriever = vectorstore.as_retriever([{\n",
" # \"$match\": {\n",
" # \"recipeId\": 123\n",
" # }\n",
" # }])\n",
"\n",
"retriever = vectorstore.as_retriever()\n",
"\n",
"# We would need a small helper function to turn the results of the search to a string\n",
"def format_docs(docs) -> str:\n",
" \"\"\"Helper function to format docs as input for RAG chaing\n",
" Args:\n",
" docs (list): list of Document objects retrieved from vector store\n",
" \"\"\"\n",
" return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
" \n",
"# We need a function to extract the message as string from the input object, I recommend you to check how the input looks like by printing it. You can also write this expression as a lambda function\n",
"def input_formatter(_input) -> str:\n",
" return _input[\"message\"]\n",
"\n",
"# Let's create the chain\n",
"llm = ChatOpenAI()\n",
"chain = (\n",
" RunnablePassthrough.assign(\n",
" recipe=input_formatter | retriever | format_docs\n",
" )\n",
" | prompt_template\n",
" | llm\n",
")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Now Let's ask our chain to find us a recipe with cheescake\n",
"message = dict(\n",
" message=\"Recommend me a good cheescake\"\n",
")\n",
"res = await chain.ainvoke(input=message)\n",
"print(res.content)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
">>> I recommend trying the famous Langchain cheesecake recipe. To make it, combine cream cheese, sugar, and eggs to prepare the cheesecake batter. You can also add a hint of vanilla extract for extra flavor. Bake the cheesecake until it is set and then chill it in the fridge before serving. Enjoy this delicious and creamy dessert!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"message = dict(\n",
" message=\"Recommend me a recipe with banana\"\n",
")\n",
"res = await chain.ainvoke(input=message)\n",
"print(res.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
">>> I recommend trying the famous langchain banana cake recipe! To make this delicious spiced banana cake, mix mashed bananas with cinnamon and nutmeg for a flavorful twist. It's a perfect treat for any occasion and is sure to be a crowd-pleaser. Enjoy baking!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading
Loading