You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently when trying to export a quantized model LLaMA Factory requires auto_gptq to be installed.
As per the project's page:
🚨 AutoGPTQ development has stopped. Please switch to GPTQModel as drop-in replacement. 🚨
The problem I have because of this is that I can't build AutoGPTQ with the latest Pytorch in a CUDA 12.8 environment, as it gives a compilation error. On the other hand, I am able to build GPTQModel in that environment. Also, it seems LLaMA Factory tries to import auto_gptq regardless of the quantization method chosen in the config.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Currently when trying to export a quantized model LLaMA Factory requires
auto_gptq
to be installed.As per the project's page:
The problem I have because of this is that I can't build AutoGPTQ with the latest Pytorch in a CUDA 12.8 environment, as it gives a compilation error. On the other hand, I am able to build GPTQModel in that environment. Also, it seems LLaMA Factory tries to import
auto_gptq
regardless of the quantization method chosen in the config.Beta Was this translation helpful? Give feedback.
All reactions