Integrate weight-only quantizaion of INC

[mengniwang95]

This PR integrate weight-only quantization of neural compressor into optimum-intel.

Notice: Need to use the master branch for test

[hshen14]

@echarlaix Could you please help review this PR? INC supports production-level quality of weight-only quantization including INT8 and INT4 for LLMs in latest master (also be released in INC v2.3 in early Sep). Thanks.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[echarlaix]

Thanks a lot for the addition @mengniwang95

[echarlaix]

Looks great, thanks for the addition @mengniwang95 !

[mengniwang95]

Hi Ella, currently UT fails since it doesn't use the latest master code. Do we need to wait neural-compressor 2.3 release for UT test and merge this PR after all test passing? @echarlaix

[echarlaix]

Hi Ella, currently UT fails since it doesn't use the latest master code. Do we need to wait neural-compressor 2.3 release for UT test and merge this PR after all test passing? @echarlaix

For when is the neural-compressor release planned ? This PR is compatible with the current INC latest version so I'm ok to with merging it now (the test can be added in an other PR with INC being installed from source)

[mengniwang95]

Hi Ella, currently UT fails since it doesn't use the latest master code. Do we need to wait neural-compressor 2.3 release for UT test and merge this PR after all test passing? @echarlaix

For when is the neural-compressor release planned ? This PR is compatible with the current INC latest version so I'm ok to with merging it now (the test can be added in an other PR with INC being installed from source)

Hi Ella, neural-compressor release is planned on 9/15. I add INT4 UT in this branch, but it is not triggered due to neural-compressor < 2.3

[hshen14]

@echarlaix it seems some tests failed, while they may not be related with the changes. Could you please help check, or is it okay to get this PR merged?

[echarlaix]

@echarlaix it seems some tests failed, while they may not be related with the changes. Could you please help check, or is it okay to get this PR merged?

Could you update your branch by rebasing from main ? This will fix all unrelated tests. The INC tests are failing, because the release is previewed for tomorrow I think we should install neural-compressor from source here to verify all the tests are passing and then we can merge