Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update sharing memory flags for OpenVINO #429

Closed

Conversation

jiwaszki
Copy link
Contributor

Update deprecated shared_memory flag with new share_inputs which works in the same way. Removes warnings.

Added new flag share_outputs to utilize new feature of OpenVINO to reduce copies to minimum.

@jiwaszki
Copy link
Contributor Author

Hi, @echarlaix this PR should go together with OpenVINO 2023.1 (now only pre-release versions are available: https://pypi.org/project/openvino/#history). Will the newest version update trigger automatically when OpenVINO drops? (ofc I will trigger branch update)

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@AlexKoff88
Copy link
Collaborator

I think we should also update the quantization part where the infer request is wrapped. Here, we should only use share_inputs.

@echarlaix
Copy link
Collaborator

Thanks for the addition @jiwaszki

Will the newest version update trigger automatically when OpenVINO drops? (ofc I will trigger branch update)

The latest OpenVINO version should be installed, but because these modifications are compatible with v2023.1 or higher we should also update it here

Copy link
Collaborator

@echarlaix echarlaix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @jiwaszki !

@AlexKoff88
Copy link
Collaborator

@jiwaszki, can you please rebase?

@jiwaszki jiwaszki mentioned this pull request Jan 5, 2024
3 tasks
@IlyasMoutawwakil
Copy link
Member

done in #499

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants