Skip to content
This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

Commit

Permalink
fix int4_fullrange dtype loading
Browse files Browse the repository at this point in the history
Signed-off-by: Wang, Chang <[email protected]>
  • Loading branch information
changwangss authored Jul 18, 2024
1 parent d917388 commit ef87c04
Showing 1 changed file with 2 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1833,6 +1833,7 @@ def load_low_bit(cls, pretrained_model_name_or_path, *model_args, **kwargs):
if quantization_config.weight_dtype not in [
"fp8_e5m2",
"fp8_e4m3",
"int4_fullrange"
]:
model = build_woq_model(model, quantization_config)
else:
Expand Down Expand Up @@ -1949,6 +1950,7 @@ def replace_ipex_cpu_woq_linear(model, current_name=[]):
if quantization_config.weight_dtype not in [
"fp8_e5m2",
"fp8_e4m3",
"int4_fullrange"
] and not quantization_config.use_ipex:
model = replace_linear(
model,
Expand Down

0 comments on commit ef87c04

Please sign in to comment.