refactor!: use ollama python library instead of calling the API with requests

[lbux]

Related Issues

• part of Ollama: use the python library instead of the API #1062

Proposed Changes:

I got the idea to rewrite the library to use the Python library when I ran into tools issues. For some reason, it worked directly through through the api (using CURL) and using Ollama's Python library, but it did not work through Haystack's implementation of the api calls. After some troubleshooting, I rewrote the generator that I needed to use the Python library as a backend, and it worked.

If Ollama already handles the requests for us, why try to reinvent it? I believe using the Python library will allow for easier additions in the future and make the code easier to understand for those who want to modify and extend the generators/embedders.

How did you test it?

I slightly modified the tests due to a nuance with how the urls are handled (Ollama just wants the url and port as each generator/embedder calls its respective endpoint), so it is not 1:1 but pretty close. I also removed tests for components that we no longer need.

Notes for the reviewer

This is likely a breaking change due to the changes of url.

Some type checking tests will fail while I work out a workaround.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[CLAassistant]

All committers have signed the CLA.

[anakin87]

This is a good idea!

I wanted to do the same thing...

The only reason we don't use Ollama python library is that it didn't exist when the integration was created.

[anakin87]

I took the liberty to push some minor changes.

Thank you for your work!

[lbux]

Awesome! Nice to see everything was wrapped up by the time I woke up

[jaimevalero]

I do not know if this helps anyone, but in my app, I had to change:

# From this
pipe.add_component("llm_specialist", OllamaGenerator(model='llama3.1:8b', url='http://127.0.0.1:11434/api/generate'))

# To this
pipe.add_component("llm_specialist", OllamaGenerator(model='llama3.1:8b', url='http://127.0.0.1:11434'))

After updating the package from ollama-haystack-0.0.6 to ollama-haystack-1.0.0, to keep things working.

[lbux]

I do not know if this helps anyone, but in my app, I had to change:

# From this
pipe.add_component("llm_specialist", OllamaGenerator(model='llama3.1:8b', url='http://127.0.0.1:11434/api/generate'))

# To this
pipe.add_component("llm_specialist", OllamaGenerator(model='llama3.1:8b', url='http://127.0.0.1:11434'))

After updating the package from ollama-haystack-0.0.6 to ollama-haystack-1.0.0, to keep things working.

Yes, this is what I meant by breaking change. This should be the only thing impacted (provided that everything else was written correctly). I'm sure there is a way to "catch" these types of errors and if needed, I can try and implement it, but I want to know:

Did you find it easy to debug the error and correct the code?

[anakin87]

Thanks!
I updated the docs to reflect this change.

[jaimevalero]

Did you find it easy to debug the error and correct the code?

At first sight I tought it could be related to the new haystack pipeline logic because the exception was during the pipeline run.

Also, I discarded the ollama service by doing:

curl -s 127.0.0.1:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"why is the sky blue?","stream":false}' | jq -r .response

As it was returning data, that discarded the ollama service itself.

Then, I started to "freeze", each package of the requirements.txt file in my app, one by one . Finally, I realized it was the ollama-haystack package and not the haystack-ai.

The issue was very well documented (big thank you).