Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn of process over-allocation #105

Open
phargogh opened this issue Oct 7, 2024 · 0 comments
Open

Warn of process over-allocation #105

phargogh opened this issue Oct 7, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@phargogh
Copy link
Member

phargogh commented Oct 7, 2024

I have a case right now where I'm running taskgraph graph on a slurm cluster, which requires that I request the number of CPU cores that I think I'm going to need.

Looking at glances or top while I'm running the graph, I can see that the most tasks that I'm actually executing in parallel are about 5 at a time. It would be really great if taskgraph could warn be about egregious over-allocation of workers, like if only 2 or 3 processes are ever used at a time but n_workers was set to 20 or something, then we could see a warning in the logs.

The specific case I'm encountering is that there are 20 CPUs allocated, but the most I'm ever using (as far as I can tell) is 12.

@phargogh phargogh added the enhancement New feature or request label Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant