Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: bump cuda base version to a more recent one #248

Merged
merged 1 commit into from
Oct 2, 2024
Merged

chore: bump cuda base version to a more recent one #248

merged 1 commit into from
Oct 2, 2024

Conversation

SdgJlbl
Copy link
Contributor

@SdgJlbl SdgJlbl commented Sep 17, 2024

Fixes FL-1740

@SdgJlbl SdgJlbl requested a review from a team as a code owner September 17, 2024 15:05
@linear Linear
Copy link

linear bot commented Sep 17, 2024

FL-1740 Bump cuda base image version in substrafl

@SdgJlbl SdgJlbl force-pushed the chore/bump-cuda-base-image branch from 271b4a9 to 8551a2e Compare September 17, 2024 15:06
@SdgJlbl
Copy link
Contributor Author

SdgJlbl commented Sep 17, 2024

/e2e --tests=camelyon,mnist,doc

@Owlfred
Copy link

Owlfred commented Sep 17, 2024 •
edited
Loading

End to end tests: ✔️ SUCCESS

Awesome! 🎉

thbcmlowk
thbcmlowk reviewed Sep 17, 2024
View reviewed changes
Copy link
Contributor

@thbcmlowk thbcmlowk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Just wondering, has it been tested on dev with --use-gpu?

substrafl/remote/register/register.py
@@ -37,7 +37,7 @@
"""

_GPU_BASE_IMAGE = """
FROM nvidia/cuda:11.8.0-runtime-ubuntu22.04
FROM nvidia/cuda:12.6.1-runtime-ubuntu24.04
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11.8.0 -> 12.6.1 seems like a huge bump - are we sure there was not a reason for using 11.8.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11.8 is not supported in Ubuntu 24.04.
Having to use Ubuntu 22.04 is forcing us to either install a vulnerable version of libpixman-1-dev (a dependency of openslides-tool) or to install libpixman-1-dev from source (I must admit, I haven't investigated much further this option).
Current versions of pytorch are still supporting CUDA 11.8, but it's the oldest version supported.

I can try with cuda 12.5.1, which is the oldest CUDA supported by Ubuntu 24.04 https://gitlab.com/nvidia/container-images/cuda/blob/master/doc/supported-tags.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on dev ✅

@SdgJlbl SdgJlbl force-pushed the chore/bump-cuda-base-image branch from 8551a2e to a9ce6fd Compare September 18, 2024 07:50
ThibaultFy
ThibaultFy approved these changes Oct 2, 2024
View reviewed changes
Copy link
Member

@ThibaultFy ThibaultFy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@SdgJlbl SdgJlbl merged commit 4d0a472 into main Oct 2, 2024
8 checks passed
@SdgJlbl SdgJlbl deleted the chore/bump-cuda-base-image branch October 2, 2024 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thbcmlowk thbcmlowk thbcmlowk left review comments

@ThibaultFy ThibaultFy ThibaultFy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@SdgJlbl @Owlfred @ThibaultFy @thbcmlowk