[DOC]update README (#301)

* update README Signed-off-by: intellinjun <[email protected]> * Update README.md --------- Signed-off-by: intellinjun <[email protected]>
intel · Jun 25, 2024 · 97c8190 · 97c8190
1 parent 9f8924d
commit 97c8190
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -133,6 +133,11 @@ python scripts/run.py model-path --weight_dtype int4 -p "She opened the door and
 ```bash
 # skip the step if GGUF model is from Hugging Face or generated by llama.cpp
 python scripts/convert.py --outtype f32 --outfile ne-f32.bin EleutherAI/gpt-j-6b
+# Using the quantize script requires a binary installation of Neural Speed
+mkdir build&&cd build
+cmake ..&&make -j
+cd ..
+python scripts/quantize.py  --model_name gptj --model_file ne-f32.bin  --out_file ne-q4_j.bin  --build_dir ./build --weight_dtype int4 --alg sym
 ```
 
 #### Inference