Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use AVX-512 #16

Open
thomasahle opened this issue Apr 24, 2023 · 1 comment
Open

Use AVX-512 #16

thomasahle opened this issue Apr 24, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@thomasahle
Copy link
Owner

AVX-512 has some nice features, such as support for fast float16 operations. This might allow us to do rescoring very fast.
The Quicker ADC paper also mentions some uses of AVX-512: https://arxiv.org/pdf/1812.09162.pdf such as {5,6,7} bit lookup tables. Though I don't think any of the top libraries, like ScaNN or Faiss actually uses that.

@thomasahle thomasahle added the enhancement New feature or request label Apr 24, 2023
@thomasahle
Copy link
Owner Author

thomasahle commented May 2, 2023

There's also the simple idea of using shuffle 512 in four lanes instead of two with AVX and 1 with SSE:
https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm512_shuffle_epi8&ig_expand=6664,6629,6629

It should probably look something like this: https://gist.github.com/thomasahle/dad66753ffecda62f86b6e6eaf0ec8e5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant