Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix volume attachment with dstack backend #2175

Merged
merged 2 commits into from
Jan 6, 2025

Conversation

un-def
Copy link
Collaborator

@un-def un-def commented Jan 3, 2025

Fixes: #2172
See: #1681
See: #1682

@un-def un-def requested a review from jvstme January 3, 2025 16:03
Copy link
Collaborator

@jvstme jvstme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the same change needed in get_job_mount_point_volume? Otherwise, I think, the volumes will be attached to the instance but not mounted

@un-def
Copy link
Collaborator Author

un-def commented Jan 6, 2025

@jvstme good catch!

It turned out that JobProvisioningData in process_submitted_jobs (the one I fixed) and in process_running_jobs are not the same, we have additional logic for running jobs:

if (
job_provisioning_data.backend == BackendType.DSTACK
and job_provisioning_data.backend_data is not None
):
backend_data = json.loads(job_provisioning_data.backend_data)
job_provisioning_data.backend = backend_data["base_backend"]

That means that it "just" works without the additional fix your suggested, but it should be fixed indeed.

I think we no longer need this magic (since #1682), but I'll keep in for now with a comment.

@un-def un-def merged commit 163298e into master Jan 6, 2025
24 checks passed
@un-def un-def deleted the issue_2172_fix_dstack_backend_volume_attach branch January 6, 2025 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Cannot attach an AWS volume on dstack Sky
2 participants