Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OptimumEmbedder #137

Closed
10 tasks done
Tracked by #6669
mathislucka opened this issue Dec 22, 2023 · 2 comments · Fixed by #379
Closed
10 tasks done
Tracked by #6669

Add OptimumEmbedder #137

mathislucka opened this issue Dec 22, 2023 · 2 comments · Fixed by #379
Assignees
Labels

Comments

@mathislucka
Copy link
Member

mathislucka commented Dec 22, 2023

Is your feature request related to a problem? Please describe.
Huggin Face's Optimum library provides faster inference through ONNX and TensorRT. This can be used to create blazing fast embedding components. The concepts used in Optimum also play well with some of the concepts that we have in Haystack. For example:

Loading non-ONNX checkpoints requires a conversion step, this takes some time. We can do that step in our warmup function (https://huggingface.co/docs/optimum/onnxruntime/usage_guides/models#loading-a-vanilla-transformers-model).

Describe the solution you'd like

Describe alternatives you've considered

Additional context
https://colab.research.google.com/drive/10UAtpz26Gv2LtamT8j33LmI5UFQFwF4T?usp=sharing
https://github.com/huggingface/optimum-benchmark/tree/main/examples/fast-mteb

Tasks

Preview Give feedback
  1. P1 integration:optimum type:documentation
    dfokina
@mathislucka mathislucka added the P2 label Dec 22, 2023
@mathislucka mathislucka transferred this issue from deepset-ai/haystack Dec 22, 2023
@mathislucka mathislucka added this to the 2.0 Embedders milestone Dec 22, 2023
@masci masci added the contributions wanted! Looking for external contributions label Jan 11, 2024
@mathislucka mathislucka added P1 and removed P2 labels Feb 1, 2024
@masci masci assigned vblagoje and unassigned vblagoje Feb 5, 2024
@shadeMe shadeMe self-assigned this Feb 9, 2024
@anakin87 anakin87 removed the contributions wanted! Looking for external contributions label Feb 12, 2024
@shadeMe shadeMe reopened this Feb 21, 2024
@shadeMe shadeMe closed this as completed Mar 12, 2024
@shadeMe
Copy link
Contributor

shadeMe commented Apr 25, 2024

Reopening as the docs are still missing. Nor have we announced this on social media (to the best of my knowledge).

@shadeMe shadeMe reopened this Apr 25, 2024
@masci masci added P2 and removed P1 labels Apr 29, 2024
@masci masci added P1 and removed P2 labels May 21, 2024
@dfokina
Copy link
Contributor

dfokina commented May 23, 2024

The docs are live: https://docs.haystack.deepset.ai/docs/optimumtextembedder, https://docs.haystack.deepset.ai/docs/optimumdocumentembedder

I believe only the social media announcement is left

@shadeMe shadeMe removed their assignment May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants