Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Input generators should perhaps be able to deal with specificities of scheduler #174

Open
sphuber opened this issue Mar 24, 2021 · 0 comments

Comments

@sphuber
Copy link
Collaborator

sphuber commented Mar 24, 2021

Not all scheduler plugins function the same or require/support the same inputs. Most of us are familiar with PBS and SLURM, which both require the num_machines to be defined, and so most of our protocols define this in the resources option. However, there are other schedulers, such as the LSF for example, that do not accept this input and one has to set tot_num_mpiprocs. The question now is how to deal with this. In principle the generators could deal with this, but the question should they. Given the fact that they receive the code as input, they can get the type of the scheduler from the Computer for which that code is configured. In principle then, they could react on this, for example set num_machines for PBS and SLURM and tot_num_mpiprocs. The problem, however, is that it is not only impractical having to add this logic in all generators, it is also never going to be complete. Since new schedulers can be added through plugins, the protocols can never be complete for all potential schedulers out there.

The only way to generalize this is if it were possible to find the set of parameters that would cover all schedulers at a minimum and then add methods to the classes that define which parameters can and have to be set. Clients can then introspect this and act accordingly. Still I doubt whether it is feasible to nail down this set of parameters that should work for (almost) all schedulers. Other ideas?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant