Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create tooling to measure quantization error #310

Open
shubhamvishu opened this issue Oct 24, 2024 · 2 comments
Open

Create tooling to measure quantization error #310

shubhamvishu opened this issue Oct 24, 2024 · 2 comments

Comments

@shubhamvishu
Copy link

@mikemccand mentioned it would be I'd good to have a tooling to measure/benchmark the overall quantization error for a given index or vector dataset. It could go over each float to check the induced error converting float -> quantized half/full byte -> float (rescaled value) the induced error and report the average induced quantization error/noise. Maybe also also playing with the confidenceInterval values and plot some nice looking charts?

@mikemccand
Copy link
Owner

KnnGraphTester.java already reports some interesting stats about the HNSW graph ... maybe it could also measure/aggregate/report the quantization error?

@shubhamvishu
Copy link
Author

@mikemccand Indeed it would be useful to include this in the KnnGraphTester output itself. I'll put out a PR for this(sometime this week).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants