Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Pinecone - remove legacy filter support #1069

Merged
merged 6 commits into from
Sep 12, 2024
Merged

chore: Pinecone - remove legacy filter support #1069

Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
141fba0
Remove legacy filter support
vblagoje Sep 9, 2024
7adb47c
Linting
vblagoje Sep 9, 2024
16ab80e
Improve error message
vblagoje Sep 11, 2024
a5ca26c
Improve error message - lint
vblagoje Sep 11, 2024
3337b39
Change message to be more generic
vblagoje Sep 11, 2024
b3331ac
Error msg fmt
vblagoje Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions integrations/pinecone/src/haystack_integrations/document_stores/pinecone/document_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
from haystack.dataclasses import Document
from haystack.document_stores.types import DuplicatePolicy
from haystack.utils import Secret, deserialize_secrets_inplace
from haystack.utils.filters import convert

from pinecone import Pinecone, PodSpec, ServerlessSpec

Expand Down Expand Up @@ -201,6 +200,10 @@ def filter_documents(self, filters: Optional[Dict[str, Any]] = None) -> List[Doc
:returns: A list of Documents that match the given filters.
"""

if filters and "operator" not in filters and "conditions" not in filters:
msg = "Legacy filters support has been removed. Please see documentation for new filter syntax."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest adding a link to the documentation in addition to the message to see the documentation

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done @davidsbatista thanks for the suggestion, now applied across all relevant integrations

raise ValueError(msg)

# Pinecone only performs vector similarity search
# here we are querying with a dummy vector and the max compatible top_k
documents = self._embedding_retrieval(query_embedding=self._dummy_vector, filters=filters, top_k=TOP_K_LIMIT)
Expand Down Expand Up @@ -253,7 +256,8 @@ def _embedding_retrieval(
raise ValueError(msg)

if filters and "operator" not in filters and "conditions" not in filters:
filters = convert(filters)
msg = "Legacy filters support has been removed. Please see documentation for new filter syntax."
raise ValueError(msg)
filters = _normalize_filters(filters) if filters else None

result = self.index.query(
Expand Down