OV migrate model export on pytorch frontend #397

eaidova · 2023-08-02T10:02:17Z

What does this PR do?

use openvino pyotrch frontend for exporting model. It should reduce memory consumption and speedup conversion of model in comparison with exporting models with intermediate step to convert to ONNX
model conversion pipeline preserved as close as it is possible to onnx path and reuse configuration, dummy input creation and patching from optimum ort

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

HuggingFaceDocBuilderDev · 2023-08-02T10:11:10Z

The documentation is not available anymore as the PR was closed or merged.

optimum/intel/openvino/modeling_decoder.py

echarlaix

Thanks a lot @eaidova

optimum/intel/openvino/export.py

echarlaix · 2023-08-02T12:55:08Z

optimum/intel/openvino/export.py

+                f"The task could not be automatically inferred as this is available only for models hosted on the Hugging Face Hub. Please provide the argument --task with the relevant task from {', '.join(TasksManager.get_all_tasks())}. Detailed error: {e}"
+            )
+
+    model = TasksManager.get_model_from_task(


It would be super practical to let users export a model directly and not only from a model path that will be used to load the model (like in this example) we could have an export_model function that could be used by main_export (and that would call _get_submodels_and_onnx_configs directly), what do you think ? (we could also have a get_model_from_task and a save_preprocessors functions that could be used when creating / saving an OVModel)

optimum/intel/openvino/export.py

setup.py

optimum/intel/openvino/quantization.py

fxmarty · 2023-08-17T07:38:52Z

Let me know if you'd like me to review!

optimum/intel/openvino/quantization.py

eaidova · 2023-08-21T07:32:49Z

Let me know if you'd like me to review!

@fxmarty could you please review?

fxmarty

Looks great thanks a lot! I left a few first comments.

.github/workflows/test_openvino.yml

fxmarty · 2023-08-21T15:46:13Z

setup.py

+    "openvino": ["openvino==2023.1.0.dev20230811", "onnx", "onnxruntime"],
+    "nncf": ["nncf @ git+https://github.com/openvinotoolkit/nncf.git"],


Do you know when the release of openvino 2023.1 and nncf are planned? Maybe it would make sense to wait for their release first?

these changes very important for us for enabling LLMs (we see large benefit in memory consumption and conversion time for them in the first order), also as I can see this PR blocks several other PRs from merge (and some ongoing optimizations too). So I want to bring it as soon as possible without waiting official release

optimum/exporters/openvino/convert.py

fxmarty · 2023-08-21T15:53:25Z

optimum/exporters/openvino/__main__.py

+        device=device,
+    )
+
+    custom_architecture = False


If hard-coded, I think all the following logic relying on it can be removed.

do you suggest to remove possibility to provide custom config?
Because its value comes to True if it provided

optimum/exporters/openvino/__main__.py

optimum/exporters/openvino/convert.py

optimum/intel/openvino/modeling_base_seq2seq.py

optimum/intel/openvino/quantization.py

tests/openvino/test_modeling.py

tests/openvino/test_quantization.py

eaidova · 2023-08-23T05:16:23Z

@fxmarty looks like ipex testing pipeline broken after release transformers 4.32.0. this failure does not connected with my pr, could you please take a look?

echarlaix · 2023-08-25T08:16:38Z

looks like ipex testing pipeline broken after release transformers 4.32.0. this failure does not connected with my pr, could you please take a look?

Now fixed !

echarlaix

Huge work @eaidova !! Thanks a lot

tests/openvino/test_quantization.py

optimum/exporters/openvino/utils.py

echarlaix · 2023-08-25T10:39:41Z

optimum/exporters/openvino/__main__.py

+    import torch
+
+
+def main_export(


A lot seems to be reused from https://github.com/huggingface/optimum/blob/v1.12.0/optimum/exporters/onnx/__main__.py#L132, it would make sense to create functions usable for both the ONNX, OpenVINO export (and others) but can be done in a following PR

optimum/exporters/openvino/convert.py

echarlaix · 2023-08-25T12:23:06Z

optimum/intel/openvino/modeling_decoder.py


        # Export the model to the ONNX format
        export(model=model, config=onnx_config, output=save_dir_path / ONNX_WEIGHTS_NAME)

        return cls._from_pretrained(
            model_id=save_dir_path,
            config=config,
-            from_onnx=True,
+            from_onnx=not is_torch_model(model),


It would make sense to have an OpenVINO model resulting from export for both pytorch / tensorflow wdyt ? Also could make sense to deprecate from_onnx in the future

yes, openvino supports tf model conversion directly as well, but I can not find any testing examples in optimum-intel in support tf models to test this approach, that is why I keep it like in optimum used.
Do you have some models that I can take a look regarding tf support?

optimum/exporters/openvino/convert.py

optimum/intel/openvino/quantization.py

AlexKoff88 · 2023-09-05T06:23:36Z

It's been lasting for too long and is getting hard to rebase this PR. Do we really have major comments that should be resolved? If not, I suggest doing a quick check again, fixing CI and proceeding with the merge. Minor comments can be fixed post-merge.
@fxmarty, @echarlaix, what do you think?

helena-intel · 2023-09-05T08:33:41Z

It's been lasting for too long and is getting hard to rebase this PR. Do we really have major comments that should be resolved? If not, I suggest doing a quick check again, fixing CI and proceeding with the merge. Minor comments can be fixed post-merge. @fxmarty, @echarlaix, what do you think?

IMO we should be really careful with requiring an OpenVINO prerelease for optimum-intel. Prereleases often have issues, and it's a reasonable expectation that Optimum will just work with the latest OpenVINO release. We recommend people to install optimum-intel from source in some places too. Final OpenVINO 2023.1 release should be soon. Can we wait for that?

echarlaix · 2023-09-05T09:51:32Z

There is still a couple of things to add (currently not tests to check the tf model export even if enabled with this PR for example) but can be added in a following PR so that we don't delay this PR even more as it's been a while already. I agree with @helena-intel, it makes sense to wait for openvino v2023.1 if the release date is previewed for soon, also ok to merge it now in case that makes it easier for you @eaidova (we won't do any minor release on our side in the meantime)
Thanks again for the awesome work @eaidova 🔥

echarlaix · 2023-09-05T10:06:17Z

fixing CI and proceeding with the merge

Also to fix the openvino tests, https://github.com/eaidova/optimum-intel/blob/ea/pt_fe/optimum/intel/openvino/modeling.py#L552 should be modified to include the renaming of _to_onnx_to_load to _to_load done in this PR cc @eaidova

echarlaix · 2023-09-05T10:09:24Z

Also the documentation related tests can be ignored as unrelated

eaidova · 2023-09-18T10:32:17Z

@echarlaix @AlexKoff88 @helena-intel openvino release 2023.1 has been published several hour ago. Could you please take a look one more time?
Thanks

AlexKoff88 · 2023-09-19T08:56:48Z

@echarlaix, I think we should merge this since quite a lot of new stuff depends on this feature. Thanks!

echarlaix · 2023-09-19T10:22:08Z

@echarlaix, I think we should merge this since quite a lot of new stuff depends on this feature. Thanks!

Works for me!

eaidova force-pushed the ea/pt_fe branch from 0c14457 to 0e01c08 Compare August 2, 2023 10:09

eaidova commented Aug 2, 2023

View reviewed changes

optimum/intel/openvino/modeling_decoder.py Outdated Show resolved Hide resolved

AlexKoff88 requested review from echarlaix and helena-intel August 2, 2023 11:07

eaidova force-pushed the ea/pt_fe branch from d2924d7 to a26cbfa Compare August 2, 2023 12:25

echarlaix reviewed Aug 2, 2023

View reviewed changes

eaidova force-pushed the ea/pt_fe branch 3 times, most recently from 86e4765 to 2348020 Compare August 9, 2023 19:06

AlexKoff88 reviewed Aug 11, 2023

View reviewed changes

optimum/intel/openvino/quantization.py Outdated Show resolved Hide resolved

eaidova force-pushed the ea/pt_fe branch 4 times, most recently from 98bedd9 to bced686 Compare August 13, 2023 14:44

AlexKoff88 mentioned this pull request Aug 14, 2023

Fix train_text_to_image_qat.py #401

Closed

AlexKoff88 mentioned this pull request Aug 17, 2023

Timm Models integration to Optimum-intel #404

Merged

ljaljushkin reviewed Aug 17, 2023

View reviewed changes

optimum/intel/openvino/quantization.py Outdated Show resolved Hide resolved

eaidova force-pushed the ea/pt_fe branch from bced686 to 308e2f1 Compare August 18, 2023 09:50

eaidova mentioned this pull request Aug 18, 2023

Fix import error is_safetensors_available #407

Merged

eaidova force-pushed the ea/pt_fe branch 3 times, most recently from ae67cc0 to 91798e8 Compare August 21, 2023 07:10

fxmarty reviewed Aug 21, 2023

View reviewed changes

eaidova force-pushed the ea/pt_fe branch from a9df276 to 3e12d5c Compare August 23, 2023 06:15

echarlaix reviewed Aug 25, 2023

View reviewed changes

eaidova added 4 commits September 4, 2023 17:48

remove flag from_onnx

6ccb6d7

refactoring

8775ab2

docstrings and typehints

0cd1c02

small fixes

1985e21

eaidova force-pushed the ea/pt_fe branch from e86ad91 to f769526 Compare September 4, 2023 16:42

add docstring to main_export

6857029

eaidova force-pushed the ea/pt_fe branch from f769526 to 6857029 Compare September 4, 2023 16:48

eaidova requested review from echarlaix and AlexKoff88 September 4, 2023 16:49

fix timm models

a127e43

Merge branch 'main' into ea/pt_fe

90e67ae

eaidova force-pushed the ea/pt_fe branch from d24e552 to 849a6a2 Compare September 13, 2023 07:56

fix circular imports

d96bf75

eaidova force-pushed the ea/pt_fe branch from 849a6a2 to d96bf75 Compare September 13, 2023 08:45

eaidova added 2 commits September 18, 2023 14:15

update ov version

ab8be3f

revert excluding deberta

20d99e9

update nncf on package

cce69f6

eaidova mentioned this pull request Sep 19, 2023

Added weight compression for Dolly 2.0 openvinotoolkit/openvino_notebooks#1319

Merged

echarlaix approved these changes Sep 19, 2023

View reviewed changes

AlexKoff88 approved these changes Sep 19, 2023

View reviewed changes

echarlaix merged commit 4b8ed24 into huggingface:main Sep 19, 2023

eaidova deleted the ea/pt_fe branch September 19, 2023 13:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OV migrate model export on pytorch frontend #397

OV migrate model export on pytorch frontend #397

eaidova commented Aug 2, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Aug 2, 2023 •

edited

Loading

echarlaix left a comment

echarlaix Aug 2, 2023

fxmarty commented Aug 17, 2023

eaidova commented Aug 21, 2023

fxmarty left a comment

fxmarty Aug 21, 2023

eaidova Aug 22, 2023

fxmarty Aug 21, 2023

eaidova Aug 23, 2023

eaidova commented Aug 23, 2023

echarlaix commented Aug 25, 2023

echarlaix left a comment

echarlaix Aug 25, 2023

echarlaix Aug 25, 2023

eaidova Aug 30, 2023

AlexKoff88 commented Sep 5, 2023

helena-intel commented Sep 5, 2023

echarlaix commented Sep 5, 2023

echarlaix commented Sep 5, 2023

echarlaix commented Sep 5, 2023

eaidova commented Sep 18, 2023

AlexKoff88 commented Sep 19, 2023

echarlaix commented Sep 19, 2023

		"openvino": ["openvino==2023.1.0.dev20230811", "onnx", "onnxruntime"],
		"nncf": ["nncf @ git+https://github.com/openvinotoolkit/nncf.git"],

OV migrate model export on pytorch frontend #397

OV migrate model export on pytorch frontend #397

Conversation

eaidova commented Aug 2, 2023 • edited Loading

What does this PR do?

Before submitting

HuggingFaceDocBuilderDev commented Aug 2, 2023 • edited Loading

echarlaix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fxmarty commented Aug 17, 2023

eaidova commented Aug 21, 2023

fxmarty left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eaidova commented Aug 23, 2023

echarlaix commented Aug 25, 2023

echarlaix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexKoff88 commented Sep 5, 2023

helena-intel commented Sep 5, 2023

echarlaix commented Sep 5, 2023

echarlaix commented Sep 5, 2023

echarlaix commented Sep 5, 2023

eaidova commented Sep 18, 2023

AlexKoff88 commented Sep 19, 2023

echarlaix commented Sep 19, 2023

eaidova commented Aug 2, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Aug 2, 2023 •

edited

Loading