ENH: Ignore n_jobs in config if flags specifying single recording are passed

[berkgercek]

Describe the new feature or enhancement

Currently when n_jobs is specified in the config file and is not 1, processes are restricted to use only 1 core even when the command mne_bids_pipeline is called with the --subject MySub flag, and only one subject is being processed. It would be nice if, should the command-level override be passed, the n_jobs parameter would also be ignored and a single python process run that has access to all available cores.

Describe your proposed implementation

mne_bids_pipeline --config myconf.py should respect the n_jobs: int parameter in myconf.py

mne_bids_pipeline --config myconf.py --subject MySub --session 01 --task MyTask should ignore the n_jobs parameter and implicitly set n_jobs = 1

possibly: mne_bids_pipeline --config myconf.py --subject MySub --task MyTask should respect the n_jobs parameter to process sessions in parallel, or tasks in parallel should there be one session specified and multiple possible tasks. My use case only has one session and task per subject so I'm not sure of the mechanics here.

Describe possible alternatives

The current solution is to also set the flag --n_jobs 1 when calling with a single subject/session/task, which is more explicit but causes possible confusion when omitted as the pipeline will only use a single core despite processing a single datum.

[welcome]

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

[hoechenberger]

Parallelization is hard, this will require quite some thinking and fiddling, I'm afraid. But I would welcome such a change