v0.1.6

Bug fix release: pin pymongo version to lower than 4.11 to avoid incompatibility with maggma

Changelog

v0.1.6 (2025-02-07)

Full Changelog

Closed issues:

• Replacing failed jobs and adding new jobs to a flow #250

Merged pull requests:

• Bug fix for Remote Host #253 (gpetretto)
• Hot fix to pin pymongo version to lower than 4.11. #251 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator

v0.1.5

Several updates, new features and bug fixes to improve user experience, configuration and maintenance.

Most notable changes:

• Added an upgrade to handle backward incompatible changes. From now on, after updating to a later version of jobflow-remote, users are suggested to run the jf admin upgrade command for all the projects that may be related.
• The batch submission mode can run multiple Jobs in parallel
• An experimental version of a GUI has been developed based on FastHTML. Start it with jf gui.
• A backup functionality for the queue database, accessible through jf backup in the CLI
• Reporting of the executed jobs over specific time intervals for Jobs and Flows: jf job report and jf flow report

Changelog

v0.1.5 (2025-01-29)

Full Changelog

Implemented enhancements:

• Optionally delay download after job end #210
• Add a backup feature #165

Closed issues:

• Print only with no wait #245
• Mismatched dumped taskdoc and database state #231
• Check status for simulation code in jf job list #227
• Successful Response not being written to jfremote_out.json #226
• jf admin upgrade causes a lot of errors #224
• QToolKit is currently pinned to development version #218
• Full-split : Remote Runner setup #214
• An Error occurred during the command execution: ValueError No daemon runner document. #211
• "Error executing jf project check --errors to connect to <cluster> (no additional info)" #208
• how to rerun a job but with improved memory #198
• Delete VASP files when rerunnning jobs? #197
• call jf from container #196
• how to limit number of jobs #195
• non-unique uuids as a problem in jobflow-remote #193
• Set projects_folder via env var #188
• Support for heterogenous computing resources? #184
• Jobflow remote logo #178
• How could I use SGE for job submission? #159
• Check there is not already a runner running #140
• How to add metadata to flows docs? #124
• Job submission fails based on job name #112
• preventing reset issues #102
• Missing docs: batch mode #96
• Does jobflow-remote support the pilot job model? #86

Merged pull requests:

• bump qtoolkit 0.1.6 #248 (gpetretto)
• Make JobDoc available at runtime #244 (gpetretto)
• Finalize gui implementation #243 (gpetretto)
• fix typing for submit_flow #242 (FabiPi3)
• Web GUI for jobflow remote #241 (fraricci)
• Allow switching off daemon if no running runner doc #240 (gpetretto)
• Logo #239 (gpetretto)
• [WIP] Save db dumps for test failures #237 (gpetretto)
• Various updates #234 (gpetretto)
• [WIP] test direct execution of docker commands and addition of PBS container #233 (gpetretto)
• Testing adding flags for codecov, separating unit and integration tests. #232 (davidwaroquiers)
• Option for stored data in jf job list #228 (FabiPi3)
• Delete job files when rerunning; customizable execution command #201 (gpetretto)
• Add sanitization option to host outputs #191 (gpetretto)
• Add backup functionality #190 (gpetretto)
• Require date to reset DB #189 (gpetretto)
• Allow list of jobs from scheduler using username #187 (gpetretto)
• Option to set Job priority #183 (gpetretto)
• New CLI functionalities: tree, report, job info #180 (gpetretto)
• Parallel batch submission #172 (gpetretto)
• Generalize integration tests to other queue systems (SGE) and test more Python versions #160 (ml-evs)
• Project check #158 (davidwaroquiers)
• WIP: Check only one runner #150 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator

v0.1.4

Bug fix release. See PR #177 for details. Prevents insertion of jobs/flows with the same uuid.
Users of previous versions are encouraged to upgrade and run jf admin index rebuild to regenerate the indexes in the queue collection.

Changelog

v0.1.4 (2024-09-13)

Full Changelog

Closed issues:

• Remote errors in interactive mode when larger workflows are run #174
• Add --query (or --worker) flag to jf job rerun/retry #170

Merged pull requests:

• Bugfix: improve collection indexes to prevent duplications #177 (gpetretto)
• Add --worker and --query option in CLI #171 (gpetretto)

* This Changelog was automatically generated by github_changelog_generator

v0.1.3

Linting based on ruff. New functionalities and bug fixes.

Changelog

v0.1.3 (2024-09-02)

Full Changelog

Closed issues:

• Cannot force delete more than 10 flows #169
• Issue with some db in DOWNLOAD state #161
• Jobs fail with error Remote error: file path/to/job_dir/remote_job_data.json for job xxx does not exist #157
• Correct setup of jfremote config project.yaml file for nested gateways #153
• Is there a way to visualize the workflow based on jobflow-remote info? #149
• Flow information is shared even when using different queue stores #144
• a suggestion to improve the documentation #143
• Delete job functionality #141
• Queue-out interactive mode #90
• Writing to home dir is crashing my submission. #81

Merged pull requests:

• Minor updates #156 (gpetretto)
• delete_job functionality #154 (gpetretto)
• Fix ruamel yaml dump #151 (davidwaroquiers)
• Add hyperlink for slurm and pbs resource keywords as a note in the documentation #148 (QuantumChemist)
• CLI add --pid option to jf job info #142 (janosh)
• Fix legacy ruff errors and enable corresponding rules for future linting #134 (janosh)

* This Changelog was automatically generated by github_changelog_generator

v0.1.2

Updates and bug fixes

Changelog

v0.1.2 (2024-06-26)

Full Changelog

Implemented enhancements:

• Allow setting resources for FAILED and COMPLETED jobs #119
• SUGGESTION: Catchall state for errors #111
• Flow information #19

Fixed bugs:

• Flow state stays COMPLETED when a job is rerun #118

Closed issues:

• Feature request: show more information about the flow(s) that will be deleted #130
• Feature request: optionally delete outputs when doing jf flow delete #129
• Feature request: support psutil 6.0.0 #128
• SUGGESTION: Add an example for querying results from the database #115

Merged pull requests:

• Small updates to JobController and CLI #132 (gpetretto)
• Migrate linting from flake8, isort, autoflake to ruff #122 (janosh)
• Minor updates #121 (gpetretto)
• Better error message on missing project name #120 (janosh)

* This Changelog was automatically generated by github_changelog_generator

v0.1.1

v0.1.1 (2024-03-20)

Full Changelog

Closed issues:

• Two projects, job submission #89
• Add guard for missing additional_store before executing job #76
• PyPI release checklist #53

Merged pull requests:

• Tests and updates #92 (gpetretto)
• Add options for remote JobStore #88 (gpetretto)
• Updates #87 (gpetretto)
• Interactive login for MFA #83 (gpetretto)
• Add testing and runtime checks for additional stores #59 (ml-evs)
• Remove errant __init__.py that prevents mypy from working #54 (ml-evs)

* This Changelog was automatically generated by github_changelog_generator

v0.1.0

This is the first public release of Jobflow Remote! Thank you to all those who have been testing the various previous iterations.

Although there are no breaking changes from the first public version (i.e., the publicizing of this GitHub repository), it is RECOMMENDED that existing users update to v0.1.0 as it will pull in the latest compatible versions of the dependencies, in particular QToolKit v0.1.3.

Changelog

v0.1.0 (2024-02-05)

Full Changelog

Closed issues:

• jf job list shows errors randomly #63
• Multiple lines to pre_run #61
• Add tests for Jobs with a function arg or kwarg #45
• Question about supported data types #44
• Error during retry #40
• 'REMOTE_ERROR' state for running job #36
• Targeted Python versions #33
• Jobflow dependency #30
• remote_job_data.json file missing #26
• Suggestion: improve -sdate option #24
• Job info for remote failures #20
• List of flows by state does not work #16
• Runner fails when default resources are provider per worker #14
• Cryptic failures when worker's workdir is missing #13
• Worker can be set to None leaving dead jobs #11
• Jobs with RESERVED state have datetime objects for last_updated field #8
• Command line "jf project" when there is no project #5
• .jfremote yaml test unsensitive to port change #4
• config issue #3

Merged pull requests:

• Add dev setup docs #57 (ml-evs)
• Add a test that runs with a given exec_config #56 (ml-evs)
• Pin dependency versions for testing, add PyPI release and enable dependabot #55 (ml-evs)
• Add integration test for job with a callable as a kwarg #51 (ml-evs)
• Add codecov upload #49 (ml-evs)
• Fix README badge #48 (ml-evs)
• Breaking changes: db_id type, cancelled state, deserialization and documentation #47 (gpetretto)
• Added GitHub release workflow and docs builds #46 (davidwaroquiers)
• [WIP] New job management system #37 (gpetretto)
• Add index to the folder name #35 (gpetretto)
• Linting fixes and CI config for multiple Python 3.9+ #34 (ml-evs)
• Add containerized integration tests for remote and local workers using slurm #32 (ml-evs)
• Use the qtoolkit and jobflow PyPI packages, add direct pydantic dep #31 (ml-evs)
• [WIP] pydantic2 updates #29 (gpetretto)
• Run CI on PRs to develop branch #28 (ml-evs)
• fix typo in cli help message #25 (FabiPi3)
• jf flow info and other cli updates #23 (gpetretto)
• Stopped states. #22 (davidwaroquiers)
• Fixed jf flow list. #18 (davidwaroquiers)
• Restore queries based on job id #17 (gpetretto)
• Check work_dir with project check CLI and enforce absolute paths #15 (ml-evs)
• Fix handling of null worker in submit_flow #12 (ml-evs)
• Added documentation structure. #9 (davidwaroquiers)
• A few minor tweaks from first use #7 (ml-evs)
• Added dependencies in pyproject.toml. #2 (davidwaroquiers)
• WIP Config #1 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator