Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove max_tasks_per_child=1 limitation from processes executor #515

Merged
merged 3 commits into from
Jul 23, 2024

Conversation

tomwhite
Copy link
Member

I originally imposed this constraint in #411 since it allowed a convenient way to test that tasks never used more memory than projected (see test_mem_utilization.py).

However, on re-running the benchmark in #492 (comment) I saw that the processes never exceeded allowed_mem, and also the tests in this PR also check that invariant for a variety of workloads.

This also means that we can use processes on Python < 3.11.

@tomwhite tomwhite force-pushed the set-max-tasks-per-child branch from af708bd to e6599d6 Compare July 23, 2024 08:30
@tomwhite tomwhite merged commit 69e9f94 into main Jul 23, 2024
10 checks passed
@tomwhite tomwhite deleted the set-max-tasks-per-child branch July 23, 2024 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant