Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fp16 and int8 to OpenVINO models and export CLI #443

Merged
merged 16 commits into from
Oct 4, 2023
Prev Previous commit
Next Next commit
fix
  • Loading branch information
echarlaix committed Oct 4, 2023
commit eefec6c064057111e17c77e60083d78594da5755
19 changes: 4 additions & 15 deletions optimum/exporters/openvino/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
from requests.exceptions import ConnectionError as RequestsConnectionError
from transformers import AutoTokenizer

from openvino import Core
from optimum.exporters import TasksManager
from optimum.exporters.onnx import __main__ as optimum_main
from optimum.exporters.onnx.base import OnnxConfig, OnnxConfigWithPast
Expand All @@ -31,9 +30,6 @@
from ...intel.utils.modeling_utils import patch_decoder_attention_mask
from .convert import export_models


core = Core()

OV_XML_FILE_NAME = "openvino_model.xml"

_MAX_UNCOMPRESSED_SIZE = 1e9
Expand Down Expand Up @@ -126,17 +122,10 @@ def main_export(
>>> main_export("gpt2", output="gpt2_onnx/")
```
"""
if int8:
if fp16:
raise ValueError(
"Both `fp16` and `int8` were both set to `True`, please select only one of these options."
)

if not is_nncf_available():
raise ImportError(
"Quantization of the weights to int8 requires nncf, please install it with `pip install nncf`"
)

if int8 and not is_nncf_available():
raise ImportError(
"Quantization of the weights to int8 requires nncf, please install it with `pip install nncf`"
)

if model_kwargs is None:
model_kwargs = {}
Expand Down