From 9b7d49f7dae44f41f36f527279449035b34128d5 Mon Sep 17 00:00:00 2001 From: Pooja Kamath <60406274+Pookam90@users.noreply.github.com> Date: Tue, 10 Dec 2024 08:30:10 +0530 Subject: [PATCH] docs: Adding Docs for new SQLServer Vector store package (#28173) **Description:** Adding Documentation for new SQL Server Vector Store Package. Changed files - Added new Vector Store - docs\docs\integrations\vectorstores\sqlserver.ipynb FeatureTable.Js - docs\src\theme\FeatureTables.js Microsoft.mdx - docs\docs\integrations\providers\microsoft.mdx Detailed documentation on API - https://python.langchain.com/api_reference/sqlserver/index.html --------- Co-authored-by: Erick Friis --- .../docs/integrations/providers/microsoft.mdx | 25 + .../integrations/vectorstores/sqlserver.ipynb | 959 ++++++++++++++++++ docs/src/theme/FeatureTables.js | 15 +- 3 files changed, 998 insertions(+), 1 deletion(-) create mode 100644 docs/docs/integrations/vectorstores/sqlserver.ipynb diff --git a/docs/docs/integrations/providers/microsoft.mdx b/docs/docs/integrations/providers/microsoft.mdx index a63c2fe898ffc..518d4869d47f2 100644 --- a/docs/docs/integrations/providers/microsoft.mdx +++ b/docs/docs/integrations/providers/microsoft.mdx @@ -343,6 +343,31 @@ See a [usage example](/docs/integrations/memory/postgres_chat_message_history/). Since Azure Database for PostgreSQL is open-source Postgres, you can use the [LangChain's Postgres support](/docs/integrations/vectorstores/pgvector/) to connect to Azure Database for PostgreSQL. +### Azure SQL Database + +>[Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/database/sql-database-paas-overview?view=azuresql) is a robust service that combines scalability, security, and high availability, providing all the benefits of a modern database solution. It also provides a dedicated Vector data type & built-in functions that simplifies the storage and querying of vector embeddings directly within a relational database. This eliminates the need for separate vector databases and related integrations, increasing the security of your solutions while reducing the overall complexity. + +By leveraging your current SQL Server databases for vector search, you can enhance data capabilities while minimizing expenses and avoiding the challenges of transitioning to new systems. + +##### Installation and Setup + +See [detail configuration instructions](/docs/integrations/vectorstores/sqlserver). + +We need to install the `langchain-sqlserver` python package. + +```bash +!pip install langchain-sqlserver==0.1.1 +``` + +##### Deploy Azure SQL DB on Microsoft Azure + +[Sign Up](https://learn.microsoft.com/azure/azure-sql/database/free-offer?view=azuresql) for free to get started today. + +See a [usage example](/docs/integrations/vectorstores/sqlserver). + +```python +from langchain_sqlserver import SQLServer_VectorStore +``` ### Azure AI Search diff --git a/docs/docs/integrations/vectorstores/sqlserver.ipynb b/docs/docs/integrations/vectorstores/sqlserver.ipynb new file mode 100644 index 0000000000000..2e6ee2a33c950 --- /dev/null +++ b/docs/docs/integrations/vectorstores/sqlserver.ipynb @@ -0,0 +1,959 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "3fe4f4a9-8810-428c-90cb-147ad8563025", + "language": "python" + }, + "source": [ + "# SQLServer " + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "f791e7da-9710-4f15-93f0-6ea61840a25f", + "language": "python" + }, + "source": [ + ">Azure SQL provides a dedicated [Vector data type](https:\\learn.microsoft.com\\sql\\t-sql\\data-types\\vector-data-type?view=azuresqldb-current&viewFallbackFrom=sql-server-ver16&tabs=csharp-sample) that simplifies the creation, storage, and querying of vector embeddings directly within a relational database. This eliminates the need for separate vector databases and related integrations, increasing the security of your solutions while reducing the overall complexity.\n", + "\n", + "Azure SQL is a robust service that combines scalability, security, and high availability, providing all the benefits of a modern database solution. It leverages a sophisticated query optimizer and enterprise features to perform vector similarity searches alongside traditional SQL queries, enhancing data analysis and decision-making. \n", + " \n", + "Read more on using [Intelligent applications with Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/database/ai-artificial-intelligence-intelligent-applications?view=azuresql)\n", + "\n", + "This notebook shows you how to leverage this integrated SQL [vector database](https://devblogs.microsoft.com/azure-sql/exciting-announcement-public-preview-of-native-vector-support-in-azure-sql-database/) to store documents and perform vector search queries using Cosine (cosine distance), L2 (Euclidean distance), and IP (inner product) to locate documents close to the query vectors" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "320f08b1-2fac-46fe-8e3a-273b6bf6ca8d", + "language": "python" + }, + "source": [ + "## Setup\n", + " \n", + "Install the `langchain-sqlserver` python package.\n", + "\n", + "The code lives in an integration package called:[langchain-sqlserver](https:\\github.com\\langchain-ai\\langchain-azure\\tree\\main\\libs\\sqlserver)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "5fa6ff09-79d5-4023-9005-91a217f91a5b", + "language": "python" + }, + "outputs": [], + "source": [ + "!pip install langchain-sqlserver==0.1.1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Credentials\n", + "\n", + "There are no credentials needed to run this notebook, just make sure you downloaded the `langchain_sqlserver` package\n", + "If you want to get best in-class automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n", + "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialization" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "4113da9c-b0fe-4e01-bc06-cafe05634fb6", + "language": "python" + }, + "outputs": [], + "source": [ + "from langchain_sqlserver import SQLServer_VectorStore" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "458deaef-f985-4efe-957c-7840509fdfa3", + "language": "python" + }, + "source": [ + "Find your Azure SQL DB connection string in the Azure portal under your database settings\n", + "\n", + "For more info: [Connect to Azure SQL DB - Python](https:\\learn.microsoft.com\\en-us\\azure\\azure-sql\\database\\connect-query-python?view=azuresql)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "azdata_cell_guid": "d3439463-899e-48aa-88a1-ba6bdedbdc9d", + "language": "python" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "import pyodbc\n", + "\n", + "# Define your SQLServer Connection String\n", + "_CONNECTION_STRING = (\n", + " \"Driver={ODBC Driver 18 for SQL Server};\"\n", + " \"Server=.database.windows.net,1433;\"\n", + " \"Database=test;\"\n", + " \"TrustServerCertificate=yes;\"\n", + " \"Connection Timeout=60;\"\n", + " \"LongAsMax=yes;\"\n", + ")\n", + "\n", + "# Connection string can vary:\n", + "# \"mssql+pyodbc://:/?driver=ODBC+Driver+18+for+SQL+Server\" -> With Username and Password specified\n", + "# \"mssql+pyodbc:///?driver=ODBC+Driver+18+for+SQL+Server&Trusted_connection=yes\" -> Uses Trusted connection\n", + "# \"mssql+pyodbc:///?driver=ODBC+Driver+18+for+SQL+Server\" -> Uses EntraID connection\n", + "# \"mssql+pyodbc:///?driver=ODBC+Driver+18+for+SQL+Server&Trusted_connection=no\" -> Uses EntraID connection" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "dcbdafc3-71ec-4e73-b768-ffb49dae2aee", + "language": "python" + }, + "source": [ + "In this example we use Azure OpenAI to generate embeddings , however you can use different embeddings provided in LangChain.\n", + "\n", + "You can deploy a version of Azure OpenAI instance on Azure Portal following this [guide](https:\\learn.microsoft.com\\en-us\\azure\\ai-services\\openai\\how-to\\create-resource?pivots=web-portal). Once you have your instance running, make sure you have the name of your instance and key. You can find the key in the Azure Portal, under the \"Keys and Endpoint\" section of your instance." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "a65110ff-cfa4-498c-bb7a-d937c04872c0", + "language": "python" + }, + "outputs": [], + "source": [ + "!pip install langchain-openai" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "3bd306b1-f346-4c01-93f4-039827e4f2e6", + "language": "python" + }, + "outputs": [], + "source": [ + "# Import the necessary Libraries\n", + "from langchain_openai import AzureChatOpenAI, AzureOpenAIEmbeddings\n", + "\n", + "# Set your AzureOpenAI details\n", + "azure_endpoint = \"https://.openai.azure.com/\"\n", + "azure_deployment_name_embedding = \"text-embedding-3-small\"\n", + "azure_deployment_name_chatcompletion = \"chatcompletion\"\n", + "azure_api_version = \"2023-05-15\"\n", + "azure_api_key = \"YOUR_KEY\"\n", + "\n", + "\n", + "# Use AzureChatOpenAI for chat completions\n", + "llm = AzureChatOpenAI(\n", + " azure_endpoint=azure_endpoint,\n", + " azure_deployment=azure_deployment_name_chatcompletion,\n", + " openai_api_version=azure_api_version,\n", + " openai_api_key=azure_api_key,\n", + ")\n", + "\n", + "# Use AzureOpenAIEmbeddings for embeddings\n", + "embeddings = AzureOpenAIEmbeddings(\n", + " azure_endpoint=azure_endpoint,\n", + " azure_deployment=azure_deployment_name_embedding,\n", + " openai_api_version=azure_api_version,\n", + " openai_api_key=azure_api_key,\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "f1f10145-06db-4cab-853f-9eb3b6fa8ada", + "language": "python" + }, + "source": [ + "## Manage vector store  " + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "azdata_cell_guid": "c4033f67-bea2-4859-af4d-b41f3b929978", + "language": "python" + }, + "outputs": [], + "source": [ + "from langchain_community.vectorstores.utils import DistanceStrategy\n", + "from langchain_sqlserver import SQLServer_VectorStore\n", + "\n", + "# Initialize the vector store\n", + "vector_store = SQLServer_VectorStore(\n", + " connection_string=_CONNECTION_STRING,\n", + " distance_strategy=DistanceStrategy.COSINE, # optional, if not provided, defaults to COSINE\n", + " embedding_function=embeddings, # you can use different embeddings provided in LangChain\n", + " embedding_length=1536,\n", + " table_name=\"langchain_test_table\", # using table with a custom name\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "525f611b-2bd5-4fd4-9192-93d588c5ad0b", + "language": "python" + }, + "source": [ + "### Add items to vector store" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "azdata_cell_guid": "6410813d-0ff1-44dd-b6bb-32fd74772e4f", + "language": "python" + }, + "outputs": [], + "source": [ + "## we will use some artificial data for this example\n", + "query = [\n", + " \"I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labrador is finicky and she appreciates this product better than most.\",\n", + " \"The candy is just red , No flavor . Just plan and chewy . I would never buy them again\",\n", + " \"Arrived in 6 days and were so stale i could not eat any of the 6 bags!!\",\n", + " \"Got these on sale for roughly 25 cents per cup, which is half the price of my local grocery stores, plus they rarely stock the spicy flavors. These things are a GREAT snack for my office where time is constantly crunched and sometimes you can't escape for a real meal. This is one of my favorite flavors of Instant Lunch and will be back to buy every time it goes on sale.\",\n", + " \"If you are looking for a less messy version of licorice for the children, then be sure to try these! They're soft, easy to chew, and they don't get your hands all sticky and gross in the car, in the summer, at the beach, etc. We love all the flavos and sometimes mix these in with the chocolate to have a very nice snack! Great item, great price too, highly recommend!\",\n", + " \"We had trouble finding this locally - delivery was fast, no more hunting up and down the flour aisle at our local grocery stores.\",\n", + " \"Too much of a good thing? We worked this kibble in over time, slowly shifting the percentage of Felidae to national junk-food brand until the bowl was all natural. By this time, the cats couldn't keep it in or down. What a mess. We've moved on.\",\n", + " \"Hey, the description says 360 grams - that is roughly 13 ounces at under $4.00 per can. No way - that is the approximate price for a 100 gram can.\",\n", + " \"The taste of these white cheddar flat breads is like a regular cracker - which is not bad, except that I bought them because I wanted a cheese taste.

What was a HUGE disappointment? How misleading the packaging of the box is. The photo on the box (I bought these in store) makes it look like it is full of long flatbreads (expanding the length and width of the box). Wrong! The plastic tray that holds the crackers is about 2\"\n", + " \" smaller all around - leaving you with about 15 or so small flatbreads.

What is also bad about this is that the company states they use biodegradable and eco-friendly packaging. FAIL! They used a HUGE box for a ridiculously small amount of crackers. Not ecofriendly at all.

Would I buy these again? No - I feel ripped off. The other crackers (like Sesame Tarragon) give you a little
more bang for your buck and have more flavor.\",\n", + " \"I have used this product in smoothies for my son and he loves it. Additionally, I use this oil in the shower as a skin conditioner and it has made my skin look great. Some of the stretch marks on my belly has disappeared quickly. Highly recommend!!!\",\n", + " \"Been taking Coconut Oil for YEARS. This is the best on the retail market. I wish it was in glass, but this is the one.\",\n", + "]\n", + "\n", + "query_metadata = [\n", + " {\"id\": 1, \"summary\": \"Good Quality Dog Food\"},\n", + " {\"id\": 8, \"summary\": \"Nasty No flavor\"},\n", + " {\"id\": 4, \"summary\": \"stale product\"},\n", + " {\"id\": 11, \"summary\": \"Great value and convenient ramen\"},\n", + " {\"id\": 5, \"summary\": \"Great for the kids!\"},\n", + " {\"id\": 2, \"summary\": \"yum falafel\"},\n", + " {\"id\": 9, \"summary\": \"Nearly killed the cats\"},\n", + " {\"id\": 6, \"summary\": \"Price cannot be correct\"},\n", + " {\"id\": 3, \"summary\": \"Taste is neutral, quantity is DECEITFUL!\"},\n", + " {\"id\": 7, \"summary\": \"This stuff is great\"},\n", + " {\"id\": 10, \"summary\": \"The reviews don't lie\"},\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "azdata_cell_guid": "03e8161a-6cdd-415d-8261-b6b99982726c", + "language": "python" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[1, 8, 4, 11, 5, 2, 9, 6, 3, 7, 10]" + ] + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "vector_store.add_texts(texts=query, metadatas=query_metadata)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "a2838ad1-64a1-409e-b97d-7883b42a0b33", + "language": "python" + }, + "source": [ + "## Query vector store\n", + "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent.\n", + "\n", + "Performing a simple similarity search can be done as follows:" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "azdata_cell_guid": "1baa2857-167e-4873-ad9c-e67649ef39bf", + "language": "python" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[Document(metadata={'id': 1, 'summary': 'Good Quality Dog Food'}, page_content='I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labrador is finicky and she appreciates this product better than most.'), Document(metadata={'id': 7, 'summary': 'This stuff is great'}, page_content='I have used this product in smoothies for my son and he loves it. Additionally, I use this oil in the shower as a skin conditioner and it has made my skin look great. Some of the stretch marks on my belly has disappeared quickly. Highly recommend!!!'), Document(metadata={'id': 5, 'summary': 'Great for the kids!'}, page_content=\"If you are looking for a less messy version of licorice for the children, then be sure to try these! They're soft, easy to chew, and they don't get your hands all sticky and gross in the car, in the summer, at the beach, etc. We love all the flavos and sometimes mix these in with the chocolate to have a very nice snack! Great item, great price too, highly recommend!\")]\n" + ] + } + ], + "source": [ + "# Perform a similarity search between the embedding of the query and the embeddings of the documents\n", + "simsearch_result = vector_store.similarity_search(\"Good reviews\", k=3)\n", + "print(simsearch_result)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "f92f0a1b-19aa-46d1-ad1a-c2e52f9114d0", + "language": "python" + }, + "source": [ + "### Filtering Support:\n", + "\n", + "The vectorstore supports a set of filters that can be applied against the metadata fields of the documents.This feature enables developers and data analysts to refine their queries, ensuring that the search results are accurately aligned with their needs. By applying filters based on specific metadata attributes, users can limit the scope of their searches, concentrating only on the most relevant data subsets." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "azdata_cell_guid": "24fabd60-0b29-4ed9-9d5e-38c68fe05dfa", + "language": "python" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[Document(metadata={'id': 7, 'summary': 'This stuff is great'}, page_content='I have used this product in smoothies for my son and he loves it. Additionally, I use this oil in the shower as a skin conditioner and it has made my skin look great. Some of the stretch marks on my belly has disappeared quickly. Highly recommend!!!'), Document(metadata={'id': 5, 'summary': 'Great for the kids!'}, page_content=\"If you are looking for a less messy version of licorice for the children, then be sure to try these! They're soft, easy to chew, and they don't get your hands all sticky and gross in the car, in the summer, at the beach, etc. We love all the flavos and sometimes mix these in with the chocolate to have a very nice snack! Great item, great price too, highly recommend!\"), Document(metadata={'id': 3, 'summary': 'Taste is neutral, quantity is DECEITFUL!'}, page_content='The taste of these white cheddar flat breads is like a regular cracker - which is not bad, except that I bought them because I wanted a cheese taste.

What was a HUGE disappointment? How misleading the packaging of the box is. The photo on the box (I bought these in store) makes it look like it is full of long flatbreads (expanding the length and width of the box). Wrong! The plastic tray that holds the crackers is about 2 smaller all around - leaving you with about 15 or so small flatbreads.

What is also bad about this is that the company states they use biodegradable and eco-friendly packaging. FAIL! They used a HUGE box for a ridiculously small amount of crackers. Not ecofriendly at all.

Would I buy these again? No - I feel ripped off. The other crackers (like Sesame Tarragon) give you a little
more bang for your buck and have more flavor.')]\n" + ] + } + ], + "source": [ + "# hybrid search -> filter for cases where id not equal to 1.\n", + "hybrid_simsearch_result = vector_store.similarity_search(\n", + " \"Good reviews\", k=3, filter={\"id\": {\"$ne\": 1}}\n", + ")\n", + "print(hybrid_simsearch_result)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "449c4cde-e303-4856-8deb-6e6ad56f9501", + "language": "python" + }, + "source": [ + "### Similarity Search with Score:\n", + "If you want to execute a similarity search and receive the corresponding scores you can run:" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "azdata_cell_guid": "382fa5d4-6da1-46c1-987f-6d0ec050be99", + "language": "python", + "tags": [ + "hide_input" + ] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[(Document(metadata={'id': 3, 'summary': 'Taste is neutral, quantity is DECEITFUL!'}, page_content='The taste of these white cheddar flat breads is like a regular cracker - which is not bad, except that I bought them because I wanted a cheese taste.

What was a HUGE disappointment? How misleading the packaging of the box is. The photo on the box (I bought these in store) makes it look like it is full of long flatbreads (expanding the length and width of the box). Wrong! The plastic tray that holds the crackers is about 2 smaller all around - leaving you with about 15 or so small flatbreads.

What is also bad about this is that the company states they use biodegradable and eco-friendly packaging. FAIL! They used a HUGE box for a ridiculously small amount of crackers. Not ecofriendly at all.

Would I buy these again? No - I feel ripped off. The other crackers (like Sesame Tarragon) give you a little
more bang for your buck and have more flavor.'), 0.651870006770711), (Document(metadata={'id': 8, 'summary': 'Nasty No flavor'}, page_content='The candy is just red , No flavor . Just plan and chewy . I would never buy them again'), 0.6908952973052638), (Document(metadata={'id': 4, 'summary': 'stale product'}, page_content='Arrived in 6 days and were so stale i could not eat any of the 6 bags!!'), 0.7360955776468822), (Document(metadata={'id': 1, 'summary': 'Good Quality Dog Food'}, page_content='I have bought several of the Vitality canned dog food products and have found them all to be of good quality. The product looks more like a stew than a processed meat and it smells better. My Labrador is finicky and she appreciates this product better than most.'), 0.7408823529514486), (Document(metadata={'id': 9, 'summary': 'Nearly killed the cats'}, page_content=\"Too much of a good thing? We worked this kibble in over time, slowly shifting the percentage of Felidae to national junk-food brand until the bowl was all natural. By this time, the cats couldn't keep it in or down. What a mess. We've moved on.\"), 0.782995248991772), (Document(metadata={'id': 7, 'summary': 'This stuff is great'}, page_content='I have used this product in smoothies for my son and he loves it. Additionally, I use this oil in the shower as a skin conditioner and it has made my skin look great. Some of the stretch marks on my belly has disappeared quickly. Highly recommend!!!'), 0.7912681479906212), (Document(metadata={'id': 2, 'summary': 'yum falafel'}, page_content='We had trouble finding this locally - delivery was fast, no more hunting up and down the flour aisle at our local grocery stores.'), 0.809213468778896), (Document(metadata={'id': 10, 'summary': \"The reviews don't lie\"}, page_content='Been taking Coconut Oil for YEARS. This is the best on the retail market. I wish it was in glass, but this is the one.'), 0.8281482301097155), (Document(metadata={'id': 5, 'summary': 'Great for the kids!'}, page_content=\"If you are looking for a less messy version of licorice for the children, then be sure to try these! They're soft, easy to chew, and they don't get your hands all sticky and gross in the car, in the summer, at the beach, etc. We love all the flavos and sometimes mix these in with the chocolate to have a very nice snack! Great item, great price too, highly recommend!\"), 0.8283754326400574), (Document(metadata={'id': 6, 'summary': 'Price cannot be correct'}, page_content='Hey, the description says 360 grams - that is roughly 13 ounces at under $4.00 per can. No way - that is the approximate price for a 100 gram can.'), 0.8323967822635847), (Document(metadata={'id': 11, 'summary': 'Great value and convenient ramen'}, page_content=\"Got these on sale for roughly 25 cents per cup, which is half the price of my local grocery stores, plus they rarely stock the spicy flavors. These things are a GREAT snack for my office where time is constantly crunched and sometimes you can't escape for a real meal. This is one of my favorite flavors of Instant Lunch and will be back to buy every time it goes on sale.\"), 0.8387189489406939)]\n" + ] + } + ], + "source": [ + "simsearch_with_score_result = vector_store.similarity_search_with_score(\n", + " \"Not a very good product\", k=12\n", + ")\n", + "print(simsearch_with_score_result)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "620e29bd-02f8-4dc7-91a2-52537cb08886", + "language": "python" + }, + "source": [ + "For a full list of the different searches you can execute on a Azure SQL vector store, please refer to the [API reference](https://python.langchain.com/api_reference/sqlserver/index.html)." + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "ff48b371-b94f-4a3a-bd66-cce856baf6c4", + "language": "python" + }, + "source": [ + "### Similarity Search when you already have embeddings you want to search on" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "35afb4cd-0682-4525-9ba8-625fecc59bb4", + "language": "python", + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[Document(metadata={'id': 8, 'summary': 'Nasty No flavor'}, page_content='The candy is just red , No flavor . Just plan and chewy . I would never buy them again'), Document(metadata={'id': 4, 'summary': 'stale product'}, page_content='Arrived in 6 days and were so stale i could not eat any of the 6 bags!!'), Document(metadata={'id': 3, 'summary': 'Taste is neutral, quantity is DECEITFUL!'}, page_content='The taste of these white cheddar flat breads is like a regular cracker - which is not bad, except that I bought them because I wanted a cheese taste.

What was a HUGE disappointment? How misleading the packaging of the box is. The photo on the box (I bought these in store) makes it look like it is full of long flatbreads (expanding the length and width of the box). Wrong! The plastic tray that holds the crackers is about 2 smaller all around - leaving you with about 15 or so small flatbreads.

What is also bad about this is that the company states they use biodegradable and eco-friendly packaging. FAIL! They used a HUGE box for a ridiculously small amount of crackers. Not ecofriendly at all.

Would I buy these again? No - I feel ripped off. The other crackers (like Sesame Tarragon) give you a little
more bang for your buck and have more flavor.'), Document(metadata={'id': 6, 'summary': 'Price cannot be correct'}, page_content='Hey, the description says 360 grams - that is roughly 13 ounces at under $4.00 per can. No way - that is the approximate price for a 100 gram can.')]\n" + ] + } + ], + "source": [ + "# if you already have embeddings you want to search on\n", + "simsearch_by_vector = vector_store.similarity_search_by_vector(\n", + " [-0.0033353185281157494, -0.017689190804958344, -0.01590404286980629, ...]\n", + ")\n", + "print(simsearch_by_vector)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "8a7083fd-ddb2-4187-a315-744b7a623178", + "language": "python" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[(Document(metadata={'id': 8, 'summary': 'Nasty No flavor'}, page_content='The candy is just red , No flavor . Just plan and chewy . I would never buy them again'), 0.9648153551769503), (Document(metadata={'id': 4, 'summary': 'stale product'}, page_content='Arrived in 6 days and were so stale i could not eat any of the 6 bags!!'), 0.9655108580341948), (Document(metadata={'id': 3, 'summary': 'Taste is neutral, quantity is DECEITFUL!'}, page_content='The taste of these white cheddar flat breads is like a regular cracker - which is not bad, except that I bought them because I wanted a cheese taste.

What was a HUGE disappointment? How misleading the packaging of the box is. The photo on the box (I bought these in store) makes it look like it is full of long flatbreads (expanding the length and width of the box). Wrong! The plastic tray that holds the crackers is about 2 smaller all around - leaving you with about 15 or so small flatbreads.

What is also bad about this is that the company states they use biodegradable and eco-friendly packaging. FAIL! They used a HUGE box for a ridiculously small amount of crackers. Not ecofriendly at all.

Would I buy these again? No - I feel ripped off. The other crackers (like Sesame Tarragon) give you a little
more bang for your buck and have more flavor.'), 0.9840511208615808), (Document(metadata={'id': 6, 'summary': 'Price cannot be correct'}, page_content='Hey, the description says 360 grams - that is roughly 13 ounces at under $4.00 per can. No way - that is the approximate price for a 100 gram can.'), 0.9915737524649991)]\n" + ] + } + ], + "source": [ + "# Similarity Search with Score if you already have embeddings you want to search on\n", + "simsearch_by_vector_with_score = vector_store.similarity_search_by_vector_with_score(\n", + " [-0.0033353185281157494, -0.017689190804958344, -0.01590404286980629, ...]\n", + ")\n", + "print(simsearch_by_vector_with_score)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Delete items from vector store" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "01f30a69-76cb-4137-bb80-1061abc095be", + "language": "python" + }, + "source": [ + "### Delete Row by ID" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "azdata_cell_guid": "1b42828c-0850-4d89-a1b5-a463bae0f143", + "language": "python" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# delete row by id\n", + "vector_store.delete([\"3\", \"7\"])" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "51b9a47e-a17a-4427-8abe-90d87fd63389", + "language": "python" + }, + "source": [ + "### Drop Vector Store" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "cc9a281a-d204-4830-83d0-fcdd890c7f9c", + "language": "python" + }, + "outputs": [], + "source": [ + "# drop vectorstore\n", + "vector_store.drop()" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "2d1b942b-f1ca-4fb5-abb7-bb2855631962", + "language": "python" + }, + "source": [ + "## Load a Document from Azure Blob Storage" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "cab89a29-e5e3-44b6-8f29-b4470d26f5d4", + "language": "python" + }, + "source": [ + "Below is example of loading a file from Azure Blob Storage container into the SQL Vector store after splitting the document into chunks.\n", + "[Azure Blog Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) is Microsoft's object storage solution for the cloud. Blob Storage is optimized for storing massive amounts of unstructured data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "azdata_cell_guid": "6cff6a17-89b6-4d73-a92d-cf289dea4294", + "language": "python" + }, + "outputs": [], + "source": [ + "pip install azure-storage-blob" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "azdata_cell_guid": "d9127900-0942-48f1-bd4d-081c7fa3fcae", + "language": "python" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of split documents: 528\n" + ] + } + ], + "source": [ + "from langchain.document_loaders import AzureBlobStorageFileLoader\n", + "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", + "from langchain_core.documents import Document\n", + "\n", + "# Define your connection string and blob details\n", + "conn_str = \"DefaultEndpointsProtocol=https;AccountName=;AccountKey===;EndpointSuffix=core.windows.net\"\n", + "container_name = \" 100\n", + " else doc.page_content\n", + " for doc in response[\"context\"]\n", + " ],\n", + " }\n", + "\n", + " # Create a DataFrame\n", + " df = pd.DataFrame(data)\n", + "\n", + " # Print the table\n", + " print(\"\\nSources:\")\n", + " print(df.to_markdown(index=False))" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "azdata_cell_guid": "3cab0661-2351-4164-952f-67670addd99b", + "language": "python", + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer: When Harry first learned that he was a wizard, he felt quite sure there had been a horrible mistake. He struggled to believe it because he had spent his life being bullied and mistreated by the Dursleys. If he was really a wizard, he wondered why he hadn't been able to use magic to defend himself. This disbelief and surprise were evident when he gasped, “I’m a what?”\n", + "\n", + "Sources:\n", + "| Doc ID | Content |\n", + "|:--------------------------------------------|:------------------------------------------------------|\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | Harry was wondering what a wizard did once he’d fi... |\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | Harry realized his mouth was open and closed it qu... |\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | “Most of us reckon he’s still out there somewhere ... |\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | “Ah, go boil yer heads, both of yeh,” said Hagrid.... |\n", + "\n" + ] + } + ], + "source": [ + "# Define the user query\n", + "user_query = \"How did Harry feel when he first learnt that he was a Wizard?\"\n", + "\n", + "# Call the function to get the answer and sources\n", + "get_answer_and_sources(user_query)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "azdata_cell_guid": "1e1939d8-671f-4063-906c-89ee6813f12b", + "language": "python" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Yes, Harry had a pet owl named Hedwig. He decided to call her Hedwig after finding the name in a book titled *A History of Magic*.\n", + "\n", + "Sources:\n", + "| Doc ID | Content |\n", + "|:--------------------------------------------|:------------------------------------------------------|\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | Harry sank down next to the bowl of peas. “What di... |\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | Harry kept to his room, with his new owl for compa... |\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | As the snake slid swiftly past him, Harry could ha... |\n", + "| 01 Harry Potter and the Sorcerers Stone.txt | Ron reached inside his jacket and pulled out a fat... |\n", + "\n" + ] + } + ], + "source": [ + "# Define the user query\n", + "user_query = \"Did Harry have a pet? What was it\"\n", + "\n", + "# Call the function to get the answer and sources\n", + "get_answer_and_sources(user_query)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "d1f01a01-1e1d-4af6-95a3-82bad34419fe" + }, + "source": [ + "## API reference \n", + "\n", + "For detailed documentation of SQLServer Vectorstore features and configurations head to the API reference: [https://python.langchain.com/api\\_reference/sqlserver/index.html](https:\\python.langchain.com\\api_reference\\sqlserver\\index.html)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": { + "azdata_cell_guid": "f04dd9d6-d4f2-4425-9c6c-2275ff65c594" + }, + "source": [ + "## Related\n", + "- Vector store [conceptual guide](https://python.langchain.com/docs/concepts/vectorstores/)\n", + "- Vector store [how-to guides](https://python.langchain.com/docs/how_to/#vector-stores)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/docs/src/theme/FeatureTables.js b/docs/src/theme/FeatureTables.js index afa27742aaee9..74b3a4525e10e 100644 --- a/docs/src/theme/FeatureTables.js +++ b/docs/src/theme/FeatureTables.js @@ -1138,7 +1138,20 @@ const FEATURE_TABLES = { multiTenancy: true, local: true, idsInAddDocuments: false, - } + }, + { + name: "SQLServer", + link: "sqlserver", + deleteById: true, + filtering: true, + searchByVector: true, + searchWithScore: true, + async: false, + passesStandardTests: false, + multiTenancy: false, + local: false, + idsInAddDocuments: false, + }, ], } };