Should Qdrant METADATA_KEY be "payload" by default?

[thekie]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

N/A

Description

Hello,
I m currently implementing RAG using Qdrant as a.

I had an issue, where I didn't get the attached as metadata in my Documents. I dug a little deeper and saw that the METADATA_KEY is set to metadata by default. The Qdrant PointStruct Object returned by the search method returns the metadata in an attribute named payload.

langchain/libs/community/langchain_community/vectorstores/qdrant.py

Line 85 in 53b8c86

METADATA_KEY = "metadata"

I was able to work around this issue by handing over the correct metadata_payload_key to the Qdrant vector store instance like this:

from langchain.vectorstores.qdrant import Qdrant

qdrant = Qdrant(
            client,
            config.qdrant_collection_name,
            embeddings=get_embeddings_engine(config),
            metadata_payload_key="payload"
        )

But I am wondering if the default value should be changed?

I would file a bug, or create a PR, but I was not sure if this is by design, or indeed a mistake.
Any guidance is appreciated.

Thank you.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.2.0: Wed Nov 15 21:53:18 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6000
Python Version: 3.12.1 (main, Dec 7 2023, 20:45:44) [Clang 15.0.0 (clang-1500.0.40.1)]

Package Information

langchain_core: 0.1.22
langchain: 0.1.6
langchain_community: 0.0.19
langsmith: 0.0.87
langchain_openai: 0.0.5
langchainhub: 0.1.14

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph
langserve

[dosubot[bot]]

🤖

Hello @thekie,

Nice to meet you! I'm Dosu, a bot here to help you solve bugs, answer questions, and guide you on your journey to becoming a contributor. While we wait for a human maintainer, I'm here to assist you with any inquiries you might have. I'm taking a look at your issue regarding the METADATA_KEY in Qdrant and will provide you with a full answer shortly.

[wgcv]

Why i want only one field of the metadata and not all the metadata??

[thekie]

I misunderstood the implementation. It turns out the metadata must be one field in the payload dictionary of the PointStruct

[dosubot[bot]]

🤖

Yes, your implementation is correct. In the LangChain framework, the metadata_payload_key is used to retrieve the metadata from the payload dictionary of the PointStruct. By default, this key is set to metadata, but it can be overridden by providing a different value when creating the Qdrant vector store instance, as you have done.

Here is the relevant code from the Qdrant class in the LangChain framework:

class Qdrant(VectorStore):
    METADATA_KEY = "metadata"

    def __init__(self, client, collection_name, embeddings, metadata_payload_key=None):
        super().__init__(client, collection_name, embeddings)
        self.metadata_payload_key = metadata_payload_key or self.METADATA_KEY

As you can see, the metadata_payload_key is set to the value of metadata_payload_key if it is provided, otherwise it defaults to self.METADATA_KEY, which is metadata.

So, in your case, setting metadata_payload_key to "payload" when creating the Qdrant vector store instance is the correct way to ensure that the metadata is correctly retrieved from the payload dictionary of the PointStruct.

Please note that this is based on the current implementation of the LangChain framework and may change in future versions. Always refer to the latest documentation or source code for the most accurate information.

You can find the source code for the Qdrant class in the LangChain repository at libs/community/langchain_community/vectorstores/qdrant.py.

Regarding your question about the default value of METADATA_KEY, it is set to metadata in the Qdrant class. However, I wasn't able to find any information about whether this value is used elsewhere in the LangChain framework. If you have any further questions or need more information, please let me know.

Sources

• libs/community/langchain_community/vectorstores/qdrant.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[alexvcodesphere]

Is it possible to somehow get the full payload? (I am using langchain js)