Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OllamaChatGenerator return error in 'streaming_callback' argument #791

Closed
NILICK opened this issue Jun 6, 2024 · 2 comments
Closed

OllamaChatGenerator return error in 'streaming_callback' argument #791

NILICK opened this issue Jun 6, 2024 · 2 comments

Comments

@NILICK
Copy link

NILICK commented Jun 6, 2024

Describe the bug
Based on this document, I can use streaming_callback as argument in OllamaChatGenerator.

My code:

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.ollama import OllamaChatGenerator
from haystack.components.generators.utils import print_streaming_chunk

generator = OllamaChatGenerator(model="phi3",
                                url = "http://localhost:11434/api/chat",
                                streaming_callback = print_streaming_chunk,
                                generation_kwargs={
                                    "num_predict": 100,
                                    "temperature": 0.9,
                                    },
                                )

messages = [ChatMessage.from_system("\nYou are a helpful, respectful and honest assistant"),
ChatMessage.from_user("What's Natural Language Processing?")]

print(generator.run(messages=messages))

But it return error

Error message

TypeError: OllamaChatGenerator.init() got an unexpected keyword argument 'streaming_callback'

System:

  • OS: Ubuntu
  • Haystack version (commit or version number): 2.2.1
@anakin87 anakin87 transferred this issue from deepset-ai/haystack Jun 6, 2024
@anakin87
Copy link
Member

anakin87 commented Jun 6, 2024

Hello!

I have tried your code with the latest version of ollama-haystack (0.0.7) and works for me.

Please try reinstalling the latest version (pip install -U ollama-haystack) and let us know.

@NILICK
Copy link
Author

NILICK commented Jun 6, 2024

Hi @anakin87, Thanks for your fast and correct reply.

@NILICK NILICK closed this as completed Jun 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants