You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
From @TaiSakuma : add a Job option for a post-execution script, before files are transferred to HDFS (à la setup_script).
An example workflow would be: setup (conda activate) => run python script => postrun (zip up output) => transfer to HDFS.
Thinking of calling it postrun_script. Other suggestions: teardown_script, although this is a misnomer as (a) the user can use it for anything, (b) it isn't necessary to remove anything from the worker node - the script does this automatically
The text was updated successfully, but these errors were encountered:
From @TaiSakuma : add a
Job
option for a post-execution script, before files are transferred to HDFS (à la setup_script).An example workflow would be: setup (conda activate) => run python script => postrun (zip up output) => transfer to HDFS.
Thinking of calling it
postrun_script
. Other suggestions:teardown_script
, although this is a misnomer as (a) the user can use it for anything, (b) it isn't necessary to remove anything from the worker node - the script does this automaticallyThe text was updated successfully, but these errors were encountered: