-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add OptimumEmbedder #137
Comments
Reopening as the docs are still missing. Nor have we announced this on social media (to the best of my knowledge). |
The docs are live: https://docs.haystack.deepset.ai/docs/optimumtextembedder, https://docs.haystack.deepset.ai/docs/optimumdocumentembedder I believe only the social media announcement is left |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
Huggin Face's Optimum library provides faster inference through ONNX and TensorRT. This can be used to create blazing fast embedding components. The concepts used in Optimum also play well with some of the concepts that we have in Haystack. For example:
Loading non-ONNX checkpoints requires a conversion step, this takes some time. We can do that step in our warmup function (https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models#loading-a-vanilla-transformers-model).
Describe the solution you'd like
Describe alternatives you've considered
Additional context
https://colab.research.google.com/drive/10UAtpz26Gv2LtamT8j33LmI5UFQFwF4T?usp=sharing
https://github.com/huggingface/optimum-benchmark/tree/main/examples/fast-mteb
Tasks
The text was updated successfully, but these errors were encountered: