Creata an agent that allows for Q&A over documents in Azure Blob Storage

[artemvk7]

Hi,

I want to create an agent that can query documents stored in Azure Blob Storage and help answer questions. The documents are documentation on software products. The agent has to have memory. How can I achieve this?

This is what I have so far:

import os

import openai
from langchain.agents.agent_toolkits import (VectorStoreInfo,
                                             VectorStoreToolkit,
                                             create_vectorstore_agent)
from langchain.cache import InMemoryCache
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.chains import ConversationalRetrievalChain
from langchain.chat_models import AzureChatOpenAI
from langchain.document_loaders import (AzureBlobStorageContainerLoader)
from langchain.embeddings import OpenAIEmbeddings
from langchain.globals import set_llm_cache
from langchain.prompts import PromptTemplate
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma
set_llm_cache(InMemoryCache())


llm = AzureChatOpenAI(
    cache=True,
    openai_api_base=openai.api_base,
    openai_api_key=openai.api_key,
    openai_api_version=openai.api_version,
    streaming=True,
    max_tokens=1000,
    deployment_name="gpt-4-turbo",
    model_version="1106-Preview",
    callbacks=[StreamingStdOutCallbackHandler()],
)

embeddings = OpenAIEmbeddings(
    api_key=openai.api_key,
    deployment=embeddings_deployment,
    model="embeddings",
    openai_api_base=openai.api_base,
    openai_api_type=openai.api_type,
)


loader = AzureBlobStorageContainerLoader(
    conn_str=connection_str_azrStorage,
    container=container_name,
)

# Connection string and container name
documents = loader.load()
# Split the documents using CharacterTextSplitter
text_splitter = CharacterTextSplitter(chunk_size=20000, chunk_overlap=1000)
split_documents = text_splitter.split_documents(documents)

# Convert the document chunks to embedding and save them to the vector store
# (Assuming split_documents are in the correct format for Chroma.from_documents)
vectordb = Chroma.from_documents(
    split_documents, embeddings, persist_directory="./data", 
)

vectorstore_info = VectorStoreInfo(
    name="projectum_documentation",
    description="Documentation, how-to, guides, limitations on Projectum products",
    vectorstore=vectordb,
)

toolkit = VectorStoreToolkit(vectorstore_info=vectorstore_info, llm=llm)

vectordb.persist()
retriever = vectordb.as_retriever(search_kwargs={"k": 15})

[dosubot[bot]]

🤖

Hello @artemvk7,

Nice to see you again! I hope you're doing well. I appreciate your continued interest in the LangChain project.

Based on the context provided, it seems like you're on the right track with your current implementation. However, to create an agent that can query and answer questions from documents stored in Azure Blob Storage, with the agent having memory capabilities, you might need to make a few modifications.

Firstly, you need to create an instance of AgentWithMemory and pass the LanguageModel and documents as parameters. This will allow the agent to remember the context of the conversation and provide more accurate answers. Here's an example of how you can do this:

from langchain_core.models import LanguageModel
from langchain_core.agents import AgentWithMemory

# Initialize the language model
model = LanguageModel()

# Initialize the agent with memory capabilities
agent = AgentWithMemory(model=model, documents=documents)

Next, you can use the answer method of the AgentWithMemory instance to get the response to a question:

response = agent.answer("your_question")

Please replace "your_question" with the actual question you want to ask.

This code assumes that the LanguageModel and AgentWithMemory classes are part of the langchain_core package and that they have the necessary methods for processing documents and answering questions. If these classes do not exist or do not have the necessary methods, you will need to create them or modify them accordingly.

Also, it's important to note that the AzureChatOpenAI class in the LangChain codebase is a wrapper for the Azure OpenAI Chat Completion API. It is used to interact with a deployed model on Azure OpenAI. The class does not directly interact with Azure Blob Storage. It interacts with the Azure OpenAI API, which may internally use Azure Blob Storage for storing and retrieving models, but that is abstracted away from the user of the AzureChatOpenAI class.

For more details, you can refer to the following sources:

• libs/langchain/langchain/document_loaders/azure_blob_storage_file.py
• libs/langchain/langchain/chat_models/azure_openai.py

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

• Is there any way to combine chatbot and question answering over docs?
• libs/langchain/langchain/document_loaders/azure_blob_storage_file.py
• libs/langchain/langchain/chat_models/azure_openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[artemvk7]

@dosu-bot

What if I want external memory, such as the SQLChatMessageHistory? And how do I add tools, etc?

[dosubot[bot]]

🤖

To implement SQLChatMessageHistory for external memory in your agent, you can follow these steps:

1. First, you need to import the SQLChatMessageHistory class from the langchain.memory.chat_message_histories module:

from langchain.memory.chat_message_histories import SQLChatMessageHistory

2. Then, create an instance of SQLChatMessageHistory. You need to provide a session_id and a connection_string to your SQL database. Optionally, you can also provide a table_name, a session_id_field_name, and a custom_message_converter:

sql_chat_history = SQLChatMessageHistory(
    session_id="my_session_id",
    connection_string="sqlite:///my_database.db",
    table_name="my_table_name",
    session_id_field_name="my_session_id_field_name",
    custom_message_converter=None,
)

3. You can add messages to your SQLChatMessageHistory instance using the add_message method. This method takes a BaseMessage instance as an argument:

from langchain.schema.messages import TextMessage

message = TextMessage(content="Hello, world!")
sql_chat_history.add_message(message)

4. You can retrieve all messages from the SQLChatMessageHistory instance using the messages property:

messages = sql_chat_history.messages
for message in messages:
    print(message.content)

5. If you want to clear the session memory from the database, you can use the clear method:

sql_chat_history.clear()

6. Finally, you can add the SQLChatMessageHistory instance to your agent as an external memory:

agent = MyAgent(external_memory=sql_chat_history)

Please replace MyAgent with the actual class name of your agent. The external_memory argument should be an instance of a class that implements the BaseExternalMemory interface. In this case, SQLChatMessageHistory is a suitable class because it extends BaseChatMessageHistory, which in turn implements BaseExternalMemory.

For more details, you can refer to the SQLChatMessageHistory class in the LangChain codebase.

As for adding tools to your agent, it seems like you have already added a VectorStoreToolkit to your agent. If you want to add more tools, you can follow a similar process. You just need to create an instance of the tool and then add it to your agent. The exact process might vary depending on the specific tool you want to add. If you could provide more details about the tools you want to add, I could give more specific guidance.

Sources

• libs/langchain/langchain/memory/chat_message_histories/sql.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[BubuDavid]

I there any update on this issue but with v0.3 and langgraph?... How do we persist the chroma db without saving it forever in disk? Is there a way to persist the chromadb in an Azure Storage Blob and just to call it every time we do not have the local persist version?