Plugin architecture for job dependency module #3550
Unanswered
SteVwonder
asked this question in
Design
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In previous ☕ hours, we thought it would be cool to have a job dependency module in flux-core that supported plugins from flux-core as well as other projects (e.g.,
flux-depend
) or from vendors. The analog here would be tojobtap
. The depend module could handle registering the appropriate services, parsing RPC payloads, extracting thedepend
section from jobspec, and then passing the relevant info to registered/discovered plugins.The depend module could then be responsible for waiting for a "thumbs up" from every plugin for each job. Once all of the plugins have signed off, the job can move out of the
DEPEND
state.In RFC26, we have the concept of
scheme
(e.g.,string
orfluid
). Assuming thatscheme
gets extended to include things likeevent
orfile
, I thought it might be cool for plugins to register with the module what scheme they support. Then if a job comes in with a dependency scheme that no plugin is registered for, the dependency module can raise an exception and reject the job.If people like the general idea, then I figure we can iterate on the exact idea/protocol in this discussion thread.
Beta Was this translation helpful? Give feedback.
All reactions