Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable asynchronous jobs #54

Merged
merged 299 commits into from
Feb 7, 2024
Merged

Enable asynchronous jobs #54

merged 299 commits into from
Feb 7, 2024

Conversation

mjaehn
Copy link
Contributor

@mjaehn mjaehn commented Dec 12, 2023 •
edited
Loading

TODOs:

  • Add a "case logfile", i.e., where the times are logged when a certain job started and finished
  • Print an information whenever a sbatch job is submitted (like it was there previously without --parsable)
  • move logfiles to finished directory
  • check resume
  • Delete wait script and its output file afterwards
  • Enable option to add workflow directly in config.yaml case file
  • make icon-art-oem-test async
  • make icon-art-global async
  • Disentangle prepare_data and add dependencies
  • Create prepare_cosmo and prepare_icon jobs from prepare_data
  • Fix logging times for all jobs
  • Add a -c / --chunk option to run_chain.py to target a specific chunk
  • Make all jobs being called via sbatch
  • Distinguish python and simulation jobs
  • Don't pass logfile and use cfg.logfile instead
  • Error handling in run_chain(): Get exitcodes from cfg.get_job_info()
  • Remove sequential part
  • Adapt Jenkins test scripts

@mjaehn mjaehn added the v3.1 label Dec 12, 2023
@mjaehn
Copy link
Contributor Author

mjaehn commented Dec 15, 2023

launch jenkins

mjaehn and others added 28 commits December 20, 2023 17:18
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
0c01df7
GitHub Action: Apply Pep8-formatting
d2fa410
@mjaehn
Small fix
47da701
GitHub Action: Apply Pep8-formatting
1c65c5b
@mjaehn
Format duration
b0dd12a
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
d2df141
@mjaehn
Remove --wait from submit function
54b21a6
@mjaehn
Let geosp be a separate job
5769f8a
GitHub Action: Apply Pep8-formatting
df4fb2c
@mjaehn
Add run_chain argument to process specified chunk
f9a8060
GitHub Action: Apply Pep8-formatting
bbad6e9
@mjaehn
Fix datetime comparison in int2lm
f6e75b3
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
a68b4ae
@mjaehn
Submit all jobs as sbatch
d885c39
GitHub Action: Apply Pep8-formatting
3181262
@mjaehn
Use existing submit() function and only return job_id
f1c757d
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
6661ce5
GitHub Action: Apply Pep8-formatting
bb8844f
@mjaehn
Fix prepare_data
0b5faf1
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
81047bf
@mjaehn
Disable logging for individual jobs
e64cd48
GitHub Action: Apply Pep8-formatting
efd94d6
@mjaehn
Write casename into logfile
6871eff
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
d7d8c08
@mjaehn
Fix timers for jobs
cee8c44
GitHub Action: Apply Pep8-formatting
f60cd9d
@mjaehn
Use UTC times
aefab40
@mjaehn
Merge branch 'async' of github.com:C2SM/processing-chain into async
7f2982d
@mjaehn
Copy link
Contributor Author

mjaehn commented Jan 31, 2024

launch jenkins

@mjaehn mjaehn marked this pull request as ready for review January 31, 2024 15:56
@mjaehn mjaehn requested a review from leclairm January 31, 2024 15:56
@mjaehn
Copy link
Contributor Author

mjaehn commented Jan 31, 2024

launch jenkins

leclairm
leclairm reviewed Feb 5, 2024
View reviewed changes
Copy link
Contributor

@leclairm leclairm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not check all the jobs, I leave it up to you there.

config.py Outdated Show resolved Hide resolved
config.py Show resolved Hide resolved
jobs/__init__.py Outdated Show resolved Hide resolved
jobs/icon.py Outdated Show resolved Hide resolved
jobs/tools/__init__.py Outdated Show resolved Hide resolved
run_chain.py Outdated Show resolved Hide resolved
@mjaehn mjaehn mentioned this pull request Feb 6, 2024
Merged
@leclairm
ref: remove unnecessary if levels
d74b596
@leclairm
del: remove unused imports
069502a
@leclairm
ref(basic python jobs): merge generation of script and submission
5541007 
The create_sbatch_script method is used nowhere else => No need to
seperate generation of job script from its submission
@leclairm
Copy link
Contributor

leclairm commented Feb 6, 2024

Ok, I stop pushing not important stuff now ...

@mjaehn
Copy link
Contributor Author

mjaehn commented Feb 6, 2024

launch jenkins

@mjaehn mjaehn mentioned this pull request Feb 7, 2024
Open
@mjaehn mjaehn linked an issue Feb 7, 2024 that may be closed by this pull request
async branch: error if I don't add the project account (which is not the default that daint picks up) #56
Open
@mjaehn mjaehn merged commit 008bf17 into v3.1-rc Feb 7, 2024
2 checks passed
@mjaehn mjaehn deleted the async branch February 7, 2024 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leclairm leclairm leclairm left review comments

Assignees
No one assigned
Labels
v3.1
Projects
None yet
Milestone
No milestone
2 participants
@mjaehn @leclairm