Error when using Deletion Mode with OpenSearch Vectorstore

[snarum00]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.indexes import SQLRecordManager, index
from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores.opensearch_vector_search import (
    OpenSearchVectorSearch,
)
from requests_aws4auth import AWS4Auth
import boto3
from opensearchpy import RequestsHttpConnection

collection_name = "test_collection"

embedding = OpenAIEmbeddings()

vectorstore = OpenSearchVectorSearch(
    embedding_function=OpenAIEmbeddings(
        openai_api_key="",
        model="text-embedding-ada-002",
    ),
    opensearch_url="",
    http_auth=AWS4Auth(
        refreshable_credentials=boto3.Session().get_credentials(),
        region=boto3.Session().region_name,
        service="aoss",
    ),
    timeout=300,
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection,
    index_name="test_index",
    engine="faiss",
)

namespace = f"opensearch/{collection_name}"
record_manager = SQLRecordManager(
    namespace, db_url="sqlite:///record_manager_cache.sql"
)

# record_manager.create_schema()

doc1 = Document(page_content="kitty", metadata={"source": "kitty.txt"})
doc2 = Document(page_content="doggy", metadata={"source": "doggy.txt"})
doc3 = Document(page_content="bird", metadata={"source": "bird.txt"})

## add all documents
index(
    [doc1, doc2, doc3],
    record_manager,
    vectorstore,
    cleanup="full",
    source_id_key="source",
)

## this should cause doc1 and doc3 to be deleted from vectorstore
index([doc2], record_manager, vectorstore, cleanup="full", source_id_key="source")

Description

I am trying to test out using index deletion modes with my opensearch vectorstore but when trying to run through this quickstart guide I am running into an error when it tries to delete documents from opensearch.

File "/usr/local/lib/python3.9/site-packages/langchain/indexes/_api.py", line 373, in index vector_store.delete(uids_to_delete) File "/usr/local/lib/python3.9/site-packages/langchain_community/vectorstores/opensearch_vector_search.py", line 635, in delete raise e File "/usr/local/lib/python3.9/site-packages/langchain_community/vectorstores/opensearch_vector_search.py", line 632, in delete bulk(self.client, body, refresh=refresh_indices, ignore_status=404) File "/usr/local/lib/python3.9/site-packages/opensearchpy/helpers/actions.py", line 424, in bulk for ok, item in streaming_bulk(client, actions, ignore_status=ignore_status, *args, **kwargs): # type: ignore File "/usr/local/lib/python3.9/site-packages/opensearchpy/helpers/actions.py", line 337, in streaming_bulk for data, (ok, info) in zip( File "/usr/local/lib/python3.9/site-packages/opensearchpy/helpers/actions.py", line 272, in _process_bulk_chunk for item in gen: File "/usr/local/lib/python3.9/site-packages/opensearchpy/helpers/actions.py", line 201, in _process_bulk_chunk_success raise BulkIndexError("%i document(s) failed to index." % len(errors), errors) opensearchpy.helpers.errors.BulkIndexError: ('2 document(s) failed to index.', [{'delete': {'_index': 'test_index', '_id': '2a7c9f28-7763-52c1-b3ea-9aef92133476', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Invalid external document id:[2a7c9f28-7763-52c1-b3ea-9aef92133476] for index type: [VECTORSEARCH].'}}}, {'delete': {'_index': 'test_index', '_id': '71f94ed4-56f0-56c1-9193-667149806f60', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Invalid external document id:[71f94ed4-56f0-56c1-9193-667149806f60] for index type: [VECTORSEARCH].'}}}])

The documentation claims this is compatible with OpenSearchVectorSearch.

System Info

langchain==0.1.15
langchain-community==0.0.32
langchain-core==0.1.41
langchain-openai==0.0.2.post1
langchain-text-splitters==0.0.1
langchainhub==0.1.14

platform: mac
python version: Python 3.9.16 (also tried with most recent version)

[dosubot[bot]]

The error you're encountering is due to the document IDs not being recognized as valid by the OpenSearch VECTORSEARCH index during the delete operation. This could be because of the format or the existence of the IDs in the index. Here's how you can address this:

1. Check Document ID Format: Ensure the UUIDs generated for document IDs are compatible with your OpenSearch VECTORSEARCH index. The error suggests an issue with the format or validity of the IDs.

2. Verify ID Existence: Before attempting to delete, confirm that the document IDs exist in the index. Attempting to delete non-existent IDs could cause errors.

3. Review Index Configuration: Make sure your VECTORSEARCH index is set up to accept the UUID format used for document IDs. If there are specific format requirements, adjust your ID generation logic accordingly.

4. Enhance Error Handling: Improve error handling around delete operations to catch detailed error messages from OpenSearch. This can help identify specific issues with the operation or IDs.

5. Update Implementation: Ensure the delete method in your VectorStore class is correctly formatted for the VECTORSEARCH index type in OpenSearch.

6. Consult Documentation: Review OpenSearch documentation for any additional requirements or limitations on deleting documents or the VECTORSEARCH index type.

The document IDs are expected to be strings, as indicated by the use of str(uuid.uuid4()) for generating document IDs if not provided. This suggests that the format itself should generally be compatible, assuming the VECTORSEARCH index does not impose unique constraints beyond those typical of OpenSearch indices.

If these steps do not resolve the issue, further investigation into the specific error messages or logs might be necessary for more targeted troubleshooting.

Sources

• libs/langchain/langchain/indexes/_api.py
• libs/community/langchain_community/vectorstores/opensearch_vector_search.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[bryanjknight]

FWIW, I ran into this as well and had to hack the code a bit such that the ID is not the autogenerated one from LangChain but rather the id from the OpenSearch hit

[k4r4n2002]

Could you elaborate how you fixed it
I am using bedrockembeddings for the embedding
And in OpenSearchVectorSearch wrapper class, I used add_documents function but I was not sure whether the uuids ids list that I created of same size as documents list is being assigned as it should.
I wanted to implement delete functionality but it seems not to work
I receive a bulkindexerror

[arjunsohanlal]

Hey @bryanjknight, could you share a snippet of what worked for you?

[arjunsohanlal]

I'm facing the exact same issue on my application as well - basically that delete operations don't work with OpenSearch Serverless via LangChain's Indexing API. Anybody got any thoughts on this? Here's a snippet of my traceback -

Traceback (most recent call last):
  File "xxxxxxx\opensearchserverless-poc\lc-opensearch-test.py", line 198, in <module>
    result = index(
             ^^^^^^
  File "xxxxxxx\opensearch\Lib\site-packages\langchain_core\indexing\api.py", line 434, in index
    destination.delete(uids_to_delete)
  File "xxxxxxx\opensearch\Lib\site-packages\langchain_community\vectorstores\opensearch_vector_search.py", line 710, in delete
    raise e
  File "xxxxxxx\opensearch\Lib\site-packages\langchain_community\vectorstores\opensearch_vector_search.py", line 707, in delete
    bulk(self.client, body, refresh=refresh_indices, ignore_status=404)
  File "xxxxxxx\Envs\opensearch\Lib\site-packages\opensearchpy\helpers\actions.py", line 423, in bulk
    for ok, item in streaming_bulk(client, actions, ignore_status=ignore_status, *args, **kwargs):  # type: ignore
  File "xxxxxxx\Envs\opensearch\Lib\site-packages\opensearchpy\helpers\actions.py", line 336, in streaming_bulk
    for data, (ok, info) in zip(
  File "xxxxxxx\Envs\opensearch\Lib\site-packages\opensearchpy\helpers\actions.py", line 272, in _process_bulk_chunk
    yield from gen
  File "xxxxxxx\Envs\opensearch\Lib\site-packages\opensearchpy\helpers\actions.py", line 201, in _process_bulk_chunk_success
    raise BulkIndexError(f"{len(errors)} document(s) failed to index.", errors)
opensearchpy.helpers.errors.BulkIndexError: ('500 document(s) failed to index.', [{'delete': {'_index': 'test-index', '_id': '434ea98a-36c0-5ebe-92ad-8d8ec9d32b6a', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Invalid external document id:[434ea98a-36c0-5ebe-92ad-8d8ec9d32b6a] for index type: [VECTORSEARCH].'}}}, {'delete': {'_index': 'test-index', 
'_id': '1fae487e-5efb-506c-8c72-02b8723d2aa8', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Invalid external document id:[1fae487e-5efb-506c-8c72-02b8723d2aa8] for index type: [VECTORSEARCH].'}}}, ...

and the list goes on.