partner: Astra DB, add indexing support for Vector Store class #17767
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
the
size:L
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
dosubot
bot
added
Ɑ: vector store
|
@cbornet FYI |
cbornet
reviewed
Feb 20, 2024
libs/partners/astradb/langchain_astradb/vectorstores/astradb.py
Outdated
Show resolved
Hide resolved
cbornet
reviewed
Feb 20, 2024
libs/partners/astradb/langchain_astradb/vectorstores/astradb.py
Outdated
Show resolved
Hide resolved
cbornet
reviewed
Feb 20, 2024
libs/partners/astradb/langchain_astradb/vectorstores/astradb.py
Outdated
Show resolved
Hide resolved
- removed a raise from _validate_create_collection_indexing - rename metadata-indexing init parameters - reformat docstring for _validate_create_collection_indexing - generic error/warning messages for _validate_create_collection_indexing - improved check-for-no-warning logic for indexing integration tests
|
@efriis the above checks seem to show that integration tests were not performed. Is it something to be expected for a code-related PR on a partner package? |
dosubot
bot
added
size:XL
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
libs/partners/astradb/tests/integration_tests/test_vectorstores.py
Outdated
Show resolved
Hide resolved
cbornet
reviewed
Feb 21, 2024
cbornet
reviewed
Feb 21, 2024
eyurtsev
pushed a commit
that referenced
this pull request
Feb 28, 2024
…h LangChain package (#18131) **Description** This PR sets the "caller identity" of the Astra DB clients used by the integration plugins (`AstraDBChatMessageHistory`, `AstraDBStore`, `AstraDBByteStore` and, pending #17767 , `AstraDBVectorStore`). In this way, the requests to the Astra DB Data API coming from within LangChain are identified as such (the purpose is anonymous usage stats to best improve the Astra DB service).
|
Package has moved! Could you reopen here? https://github.com/langchain-ai/langchain-datastax |
|
Reopened in the new repo - thank you! |
gkorland
pushed a commit
to FalkorDB/langchain
that referenced
this pull request
Mar 30, 2024
…h LangChain package (langchain-ai#18131) **Description** This PR sets the "caller identity" of the Astra DB clients used by the integration plugins (`AstraDBChatMessageHistory`, `AstraDBStore`, `AstraDBByteStore` and, pending langchain-ai#17767 , `AstraDBVectorStore`). In this way, the requests to the Astra DB Data API coming from within LangChain are identified as such (the purpose is anonymous usage stats to best improve the Astra DB service).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support to the "Indexing" options when creating Astra DB collection for fine-grained control over which fields are indexed to be later used as search filters.
The (sensible) default is to index all and only the contents of the "metadata" map (plus the vectors themselves for the ANN search of course).
But one can choose to exclude certain metadata fields from being indexed (e.g. very long unique strings), or conversely specify an allowlist - or even provide a fully custom indexing prescription fed directly to the API.
In case the collection is detected on DB already, an error is raised if the requested indexing options are incompatible. But if a legacy collection (without indexing options altogether) is detected, a warning is raised and the execution can proceed.
Added unit + integration tests about this behaviour.