Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error occurs at update_embeddings #56

Open
3CE8D2BAC65BDD6AA9 opened this issue Feb 15, 2023 · 2 comments
Open

error occurs at update_embeddings #56

3CE8D2BAC65BDD6AA9 opened this issue Feb 15, 2023 · 2 comments

Comments

@3CE8D2BAC65BDD6AA9
Copy link

I have been trying https://github.com/weaviate/weaviate-examples/blob/main/harrypotter-qa-haystack-weaviate/JUPYTER-HarryPotter-QA-Haystack-Weaviate.ipynb a few times at my local machine.

My code is:

from haystack.utils

import clean_wiki_text
import pandas as pd

harry = pd.read_csv("https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/harry_potter_wiki.csv")

dicts = []

for ix, row in harry.iterrows():
dic = {

    'content': clean_wiki_text(row.text),
    'meta': {
        'name': row['name'],
        'url': row.url
    }
}
dicts.append(dic)

from haystack.utils import launch_weaviate
launch_weaviate()
from haystack.document_stores import WeaviateDocumentStore
document_store = WeaviateDocumentStore() # assumes Weaviate is running on http://localhost:8080
document_store.write_documents(documents=dicts, batch_size=100)
from haystack.nodes import EmbeddingRetriever

retriever = EmbeddingRetriever(document_store=document_store, model_format="sentence_transformers", embedding_model="sentence-transformers/multi-qa-mpnet-base-dot-v1",)
document_store.update_embeddings(retriever)
from haystack.nodes import FARMReader
reader = FARMReader(model_name_or_path="deepset/tinyroberta-squad2", use_gpu=True)

everything went smooth until this line:

document_store.update_embeddings(retriever)

The error message is:

Exception has occurred: WeaviateDocumentStoreError
Query results contain errors: [{'locations': [{'column': 6, 'line': 1}], 'message': 'explorer: list class: search: invalid pagination params: query maximum results exceeded', 'path': ['Get', 'Document']}]
File "/home/ky/weaviate/harrypotter.py", line 26, in
document_store.update_embeddings(retriever)
haystack.document_stores.weaviate.WeaviateDocumentStoreError: Query results contain errors: [{'locations': [{'column': 6, 'line': 1}], 'message': 'explorer: list class: search: invalid pagination params: query maximum results exceeded', 'path': ['Get', 'Document']}]

Any idea how to resolve this?

@3CE8D2BAC65BDD6AA9 3CE8D2BAC65BDD6AA9 changed the title error occurs error occurs at update_embeddings Feb 15, 2023
@Heucles
Copy link

Heucles commented Jul 19, 2023

I am also facing the same issue

@Heucles
Copy link

Heucles commented Jul 25, 2023

Hey @3CE8D2BAC65BDD6AA9 I was able to get it to work increasing the value of QUERY_MAXIMUM_RESULTS env, the default is 10K, for me doubling it made it work.

I've also was able to find a couple of threads discussing the issue and I'm currently looking for a better solution, here are the links bellow:
weaviate/weaviate#1947
deepset-ai/haystack#2517
deepset-ai/haystack#3390

Hope those will help you.

Heucles

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants