Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix codecov coverage calculation #719

Merged
merged 1 commit into from
Jan 8, 2025
Merged

fix codecov coverage calculation #719

merged 1 commit into from
Jan 8, 2025

Conversation

Bai-Li-NOAA
Copy link
Contributor

@Bai-Li-NOAA Bai-Li-NOAA commented Jan 7, 2025

What is the feature?

  • fix call-calc-coverage workflow

How have you implemented the solution?

  • update the DESCRIPTION file and set Config/testthat/parallel to false.
    This addresses the GHA run-time issue.
  • skip parallel tests when running covr::package_coverage(), as these tests
    don't contribute to increasing code coverage and are redundant due to
    the reuse of wrapper functions. Integration tests are sufficient for
    coverage calculation.
  • update GHA build triggers.

Does the PR impact any other area of the project, maybe another repo?

  • No

Copy link
Contributor

github-actions bot commented Jan 7, 2025

Instructions for code reviewer

Hello reviewer, thanks for taking the time to review this PR!

  • Please use this checklist during your review, checking off items that you have verified are complete!
  • For PRs that don't make changes to code (e.g., changes to README.md or Github actions workflows), feel free to skip over items on the checklist that are not relevant. Remember it is still important to do a thorough review.
  • Then, comment on the pull request with your review indicating where you have questions or changes need to be made before merging.
  • Remember to review every line of code you’ve been asked to review, look at the context, make sure you’re improving code health, and compliment developers on good things that they do.
  • PR reviews are a great way to learn, so feel free to share your tips and tricks. However, for optional changes (i.e., not required for merging), please include nit: (for nitpicking) before making the suggestion. For example, nit: I prefer using a data.frame() instead of a matrix because...
  • Engage with the developer when they respond to comments and check off additional boxes as they become complete so the PR can be merged in when all the tasks are fulfilled. Make it clear when this has been reached by commenting on the PR with something like This PR is now ready to be merged, no changes needed.

Checklist

  • The PR is requested to be merged into the appropriate branch (typically main)
  • The code is well-designed.
  • The functionality is good for the users of the code.
  • Any User Interface changes are sensible and look good.
  • The code isn’t more complex than it needs to be.
  • Code coverage remains high, indicating the new code is tested.
  • The developer used clear names for everything.
  • Comments are clear and useful, and mostly explain why instead of what.
  • Code is appropriately documented (doxygen and roxygen).

Copy link

codecov bot commented Jan 7, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.65%. Comparing base (cb222ec) to head (d0db887).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #719      +/-   ##
==========================================
+ Coverage   65.73%   68.65%   +2.92%     
==========================================
  Files          16       44      +28     
  Lines         715     4703    +3988     
  Branches      197      197              
==========================================
+ Hits          470     3229    +2759     
- Misses        160     1427    +1267     
+ Partials       85       47      -38     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@kellijohnson-NOAA kellijohnson-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Bai-Li-NOAA for these changes and @k-doering-NOAA for helping. I have a few questions, see comments. But, I also wanted to give more context for this PR for when I go back to it years later 🤣. GitHub actions for code coverage were running for over 6 hours leading to them failing with changes in 0.3.0.0. This is not the first time that it had happened but we did not address it before. Instead, I just turned them off for the merge into main knowing that we needed to fix them. The fix it to not have code coverage of parallel tests.

Comment on lines 7 to 15
paths-ignore:
- .devcontainer
- .github
- 'LICENSE'
- 'README.md'
- 'CONTRIBUTING.md'
- 'Rbuildignore'
- '.gitignore'
- 'man'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we continue to ignore these files/folders? It seems to me like we would not care if we have code coverage of the manual files and these other non-code files. Was there something that was not working with these being ignored?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just learned yesterday that paths-ignore prevents the workflow from running if only the listed files are changed. In the future, if the workflow takes too long to run, we could consider setting up a paths filter. This would make the workflow run only when changes are made to specific folders, like inst/include, src, or R, instead of listing all the non-code files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah, right. I forgot but, still, I think that we can keep these lines so the code coverage is not ran when a PR or push is made only to these files. For example, there is a GitHub action that only changes README.md and we wouldn't want to run the codecoverage test on that PR to dev. Or, if devtools::document() is ran and only changes are made to the man/*.Rd files we wouldn't want to run the coverage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it! I just added those lines back without making any changes. I’ll leave the final edits to @k-doering-NOAA through PR #720. Basically I’m not sure where to place paths-ignore. Should it go under workflow-dispatch, push, or pull?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have to repeat them under both push and pull so that they are all ignored on both PR and pushes. For the workflow dispatch i think maybe we just want to test them all?

- 'man'
push:
branches:
- main
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to have push to main because there should only be PR to main? So, I guess I am thinking that the tests wouldn't take as long if we didn't double up on a PR and a push to main by just doing it on the PR. I am fine with leaving it though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. It should only be a PR to main. I’ve made the changes, and @k-doering-NOAA, some changes from PR #720 might also need to be updated.

.github/workflows/call-calc-coverage.yml Show resolved Hide resolved
DESCRIPTION Show resolved Hide resolved
@kellijohnson-NOAA
Copy link
Contributor

Second, this PR needs to be to dev not to main. But, I am not sure exactly how to change that.

@Bai-Li-NOAA
Copy link
Contributor Author

Bai-Li-NOAA commented Jan 8, 2025

Thanks, @kellijohnson-NOAA. I'll address the questions in the comments and add more details to the PR description for future reference. I've updated the PR to target dev. I'm not sure if there will be any updates directly to main that bypass dev. If that happens, feel free to edit (on the right side of the PR title) the PR to target main.

@Bai-Li-NOAA Bai-Li-NOAA changed the base branch from main to dev January 8, 2025 17:00
* update the DESCRIPTION file and set Config/testthat/parallel to false.
  This addresses the GHA run-time issue.
* skip parallel tests when running covr::package_coverage(), as these tests
  don't contribute to increasing code coverage and are redundant due to
  the reuse of wrapper functions. Integration tests are sufficient for
  coverage calculation.
* update GHA build triggers.

Co-authored-by: Kathryn Doering <[email protected]>
Copy link
Contributor

@kellijohnson-NOAA kellijohnson-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Bai-Li-NOAA I think this is ready. Please rebase and merge into dev whenever you are ready.

@Bai-Li-NOAA Bai-Li-NOAA merged commit dbdc3a3 into dev Jan 8, 2025
11 checks passed
@Bai-Li-NOAA Bai-Li-NOAA deleted the fix-codecov-gha branch January 8, 2025 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants