fix: job scheduler delayed execution window handling and info [DHIS2-15027]

[jbee]

Summary

Main fix is to not use "now" as the last execution time when a job has not been executed before when computing the next execution time because that could lead to missing an execution and thereby deferring it indefinitely. The "last" execution must always be in the past for this computation. So now is shifted back by the maximum delay. This should be safe for any setting.

This comes with a semantic change as well (not since 2.40 but since the feature was introduced in 2.41). When creating a job it will always execute as long as now is in the window between the intended execution time and the maximum delayed execution time. This means a newly created job running every day at 7pm will run if I create it 8pm with a the setting for maximum CRON delay being longer than 1h.

Further improvements:

• when a job actually runs (start) this is logged on debug
• renames the HeartbeatJob to Housekeeping (was missed when the name changed)
• updates the lastUpdated field with each transition of the jobStatus to now()
• uses the 4h default as constant to calculate the getNextExecutionTime() and getMaxDelayedExecutionTime()
• fixes a bug in the computation of the loop alignment (cosmetic, just to have 20,40,60/00 instead of some random second of the minute when the scheduling loop is active)
• SchedulerEntry now has
  • maxDelayedExecutionTime : the correct end of the execution window instant (setting dependent)
  • secondsToNextExecutionTime: seconds to next run, for convenience as this is server time based not user local time based
  • secondsToMaxDelayedExecutionTime: seconds to next run end of execution window (same as above)
• JobConfiguration now also has getMaxDelayedExecutionTime() but similar to getNextExecutionTime() both of these are only correct 100% of the time if the setting for max delay has not been changed from its default of 4h

Automatic Testing

Existing tests were adjusted and extended to reflect the change in behaviour for newly created jobs with CRON

Manual Testing

• create a job with CRON that soon should run
• wait
• check it did run

• create a job with CRON that runs daily about 1h ago to current time
• wait 1min
• check it did run (because it was in 4h window from 1h ago)

• create a job with CRON that runs that runs daily about 5h from current time
• wahit a bit (1min)
• check it did not run, maybe also check the indicated next execution time (for fully correct values use /api/scheduler/)4.

• query /api/scheduler
• check the new fields for jobs created above maxDelayedExecutionTime, secondsToNextExecutionTime and secondsToMaxDelayedExecutionTime and see if their values add up to your expectations

[codecov]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (d82df76) 66.42% compared to head (625b37c) 66.46%.
Report is 6 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #16112      +/-   ##
============================================
+ Coverage     66.42%   66.46%   +0.04%     
- Complexity    31539    31599      +60     
============================================
  Files          3507     3508       +1     
  Lines        130571   130768     +197     
  Branches      15242    15253      +11     
============================================
+ Hits          86735    86919     +184     
- Misses        36756    36770      +14     
+ Partials       7080     7079       -1     

Flag	Coverage Δ
integration	50.26% <7.31%> (-0.06%)	⬇️
integration-h2	32.52% <68.29%> (+0.15%)	⬆️
unit	30.37% <24.39%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...his/scheduling/HibernateJobConfigurationStore.java	55.75% <ø> (ø)
...rc/main/java/org/hisp/dhis/setting/SettingKey.java	87.06% <100.00%> (+0.06%)	⬆️
...s/webapi/controller/scheduling/SchedulerEntry.java	97.82% <100.00%> (+3.88%)	⬆️
...ava/org/hisp/dhis/scheduling/JobConfiguration.java	76.92% <94.11%> (+2.68%)	⬆️
...java/org/hisp/dhis/scheduling/HousekeepingJob.java	3.70% <0.00%> (ø)
...in/java/org/hisp/dhis/scheduling/JobScheduler.java	20.83% <50.00%> (+0.83%)	⬆️
...his/scheduling/DefaultJobConfigurationService.java	70.96% <0.00%> (ø)

... and 26 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d82df76...625b37c. Read the comment docs.

[sonarcloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud