Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: query API for job configurations with errors [DHIS2-15276] #15636

Merged
merged 7 commits into from
Nov 15, 2023

Conversation

jbee
Copy link
Contributor

@jbee jbee commented Nov 9, 2023

Summary

Adds an API endpoints to search and view error recordings attached to a JobConfiguration.

New APIs

  • /api/jobConfigurations/progress/errors: only the errors of the progress JSON object flattened to an array of objects
  • /api/jobConfigurations/{uid}/errors: errors of a specific job configuration as job run errors
  • /api/jobConfigurations/errors: search API to list jobs with errors that give job run error entries

A entry of a job run errors has this structure:

[
  {
    "id":"hg6LR5ls42g",
    "type":"METADATA_IMPORT",
    "user":"M5zQapPyTZI",
    "errors":[
      {
        "id":"KfdsGBcoiCa",
        "args":[
          "[brokenHdoxL] (CategoryOption)",
          "Births attended by [KfdsGBcoiCa] (Category)",
          "categoryOption"
        ],
        "code":"E5002",
        "type":"Category",
        "message":"Invalid reference [brokenHdoxL] (CategoryOption) on object Births attended by [KfdsGBcoiCa] (Category) for association `categoryOption`"
      },
      {
        "id":"SvzFH1XEfJr",
        "args":[
          "[brokenXoUN4] (CategoryOption)",
          "Labour complications [SvzFH1XEfJr] (Category)",
          "categoryOption"
        ],
        "code":"E5002",
        "type":"Category",
        "message":"Invalid reference [brokenXoUN4] (CategoryOption) on object Labour complications [SvzFH1XEfJr] (Category) for association `categoryOption`"
      }
    ],
    "created":"2023-11-14T11:05:59.374",
    "executed":"2023-11-14T11:06:13.815453",
    "filesize":11069,
    "filetype":"application/json",
    "finished":"2023-11-14T11:06:14.52768"
  }
]

Manual Testing

  • goto import/export app
  • export some metadata to a json file and change the file manually to make it invalid, e.g. change a UID in the file to one that has valid format but does not exist
  • import the broken metatda file
  • check the import is done and shows /api/jobConfigurations/gist?filter=jobType:eq:METADATA_IMPORT&order=created:desc
  • check finding the entry in job run with errors list: /api/jobConfigurations/errors
  • check per job run errors: /api/jobConfigurations/hg6LR5ls42g/errors (replace UID in path)
  • check plain job errors: http://localhost:8080/api/jobConfigurations/hg6LR5ls42g/progress/errors (replace UID in path)

@jbee jbee self-assigned this Nov 9, 2023
Copy link

codecov bot commented Nov 13, 2023

Codecov Report

Merging #15636 (bda4006) into master (520136b) will increase coverage by 16.88%.
Report is 8 commits behind head on master.
The diff coverage is 87.05%.

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #15636       +/-   ##
=============================================
+ Coverage     49.39%   66.27%   +16.88%     
- Complexity    23283    31327     +8044     
=============================================
  Files          3483     3484        +1     
  Lines        129911   129986       +75     
  Branches      15182    15192       +10     
=============================================
+ Hits          64166    86154    +21988     
+ Misses        59684    36744    -22940     
- Partials       6061     7088     +1027     
Flag Coverage Δ
integration 50.00% <87.05%> (?)
integration-h2 32.29% <2.35%> (+<0.01%) ⬆️
unit 30.26% <2.35%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...his/scheduling/DefaultJobConfigurationService.java 71.02% <100.00%> (+20.67%) ⬆️
...his/scheduling/DefaultJobSchedulerLoopService.java 53.91% <100.00%> (+53.04%) ⬆️
...mons/jackson/config/JacksonObjectMapperConfig.java 98.24% <100.00%> (+0.03%) ⬆️
.../webapi/controller/metadata/MetadataImportJob.java 88.00% <ø> (+84.00%) ⬆️
...his/scheduling/HibernateJobConfigurationStore.java 55.75% <96.55%> (+34.32%) ⬆️
...ain/java/org/hisp/dhis/scheduling/JobProgress.java 75.11% <66.66%> (+16.43%) ⬆️
...org/hisp/dhis/scheduling/RecordingJobProgress.java 72.41% <0.00%> (+14.65%) ⬆️
...is/commons/jackson/config/JsonValueSerializer.java 60.00% <60.00%> (ø)
...troller/scheduling/JobConfigurationController.java 41.07% <50.00%> (+3.57%) ⬆️

... and 1222 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 520136b...bda4006. Read the comment docs.

@jbee jbee marked this pull request as ready for review November 14, 2023 13:53
private UID user;

/** The earliest date the job ran that should be included */
@CheckForNull private Date from;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use LocalDate or LocalDateTime for new models seeing as we want to move to use these eventually? Or is there something restricting us from doing this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I should try at least, just did this out of a (bad) habit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have setup a flexible parsing for LocalDate in the same way we have for Date so for now I keep it Date. At some point we should make an attempt to switch dates in APIs over while supporting the parsing flexibility.

Copy link
Contributor

@david-mackessy david-mackessy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding a new API without tests?! If something breaks in the most simple use case, how will we know about it?

@jbee
Copy link
Contributor Author

jbee commented Nov 14, 2023

adding a new API without tests?! If something breaks in the most simple use case, how will we know about it?

You got me :) Have to add some

Copy link

sonarcloud bot commented Nov 15, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@jbee jbee merged commit 5b0cc8e into dhis2:master Nov 15, 2023
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants