Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set default workers=0 for Windows #205

Merged
merged 1 commit into from
Mar 13, 2024
Merged

Set default workers=0 for Windows #205

merged 1 commit into from
Mar 13, 2024

Conversation

BramVanroy
Copy link
Contributor

Currently the default num_workers is set to 2*gpus. However, on Windows there is an issue with this default behavior (but only when gpus>0).

Massive shout-out to @awaelchli who helped in pin-pointing the issue and confirming that the issue cannot be inherently contributed to PyTorch Lightning but that instead this is a consequence of how self is passed to the data loader, which will then be parallellized but spawning new processes is different on Windows and Linux, which lead to the issues present here. This issue is therefore very similar to other common errors that are often encountered, related the PyTorch DataLoader. num_workers=0 is the best bet to avoid issues on Windows.

So this PR sets num_workers=0 if we're on Windows, by default. It will also throw a warning to the user when they have manually set num_workers to a higher number when using the GPU and when on Windows, as that is the use-case that will not work.

closes #190

@ricardorei
Copy link
Collaborator

Thanks Bram! This is very useful. Ill publish a new version fixing it

@ricardorei ricardorei merged commit e8daab9 into Unbabel:master Mar 13, 2024
3 checks passed
BramVanroy added a commit to BramVanroy/mateo-demo that referenced this pull request Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[QUESTION] Keep getting scores of '0' no matter what input used
2 participants