Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: prediction pipelines with multiple GPUs fails #92

Open
favyen2 opened this issue Jan 21, 2025 · 0 comments
Open

Bug: prediction pipelines with multiple GPUs fails #92

favyen2 opened this issue Jan 21, 2025 · 0 comments

Comments

@favyen2
Copy link
Collaborator

favyen2 commented Jan 21, 2025

I think it is because LightningCLI is spawning another process that is running the whole prediction pipeline again instead of just the LightningCLI call.

An easy fix would be to use subprocess in rslp.utils.rslearn to do tho prediction instead of calling CustomLightningCLI directly. This is less clean but not sure there's a way to fix it otherwise.

Anyway needs some verification first about the source of the issue, I think it is what I said but not 100% sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant