Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unpredictable quantization with quantized_bits and an alpha value of None #129

Open
qdlmcfresh opened this issue Mar 14, 2024 · 0 comments

Comments

@qdlmcfresh
Copy link

If None, the scaling factor is 1 for all channels.
states that leaving alpha at the default value of None should result in alpha being set to 1, but that's not the case which leads to unexpected behavior in quantization. The result is quantized values that are out of range a lot.

Manually setting alpha to 1 in the constructor leads to the expected results after quantization

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant