partners[pinecone]: Prevent race condition in add_texts() method

[orkhank]

Problem

When invoking the add_texts() method on the vector store, subsequent actions that depend on the existence of the newly added documents may result in errors or unexpected behavior. For example:

from langchain_pinecone import PineconeVectorStore

# Example code demonstrating the problem
vectore_store = PineconeVectorStore.from_texts(["Document 1", "Document 2"], ...)

# Immediately trying to retrieve the added documents
retrieved_docs = vector_store.similarity_search("Document 1")
# This may raise an error or return unexpected results if the documents are not yet indexed.

Solution

This pull request introduces a mechanism to block the addition of text to the index until the expected number of documents has been reached. The added code ensures that all insertions are completed only when the vector count in the index matches the anticipated document count.

Considerations

I have implemented this solution based on the assumption that no additional updates will occur on the index during this process. However, I am open to feedback regarding this interpretation of the problem and any alternative approaches that may be more effective.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Nov 11, 2024 3:00pm

[efriis]

appreciate the thought! however, this is actually the same behavior as calling the pinecone api directly due to an eventual consistency guarantee, and we don't want to by-default block even longer.

I would prefer to document how folks can implement this loop themselves to overcome eventual consistency rather than merge this into the integration.

[orkhank]

Yeah, that makes sense. Thanks for the reply.