-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix codecov coverage calculation #719
Conversation
Instructions for code reviewerHello reviewer, thanks for taking the time to review this PR!
Checklist
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #719 +/- ##
==========================================
+ Coverage 65.73% 68.65% +2.92%
==========================================
Files 16 44 +28
Lines 715 4703 +3988
Branches 197 197
==========================================
+ Hits 470 3229 +2759
- Misses 160 1427 +1267
+ Partials 85 47 -38 ☔ View full report in Codecov by Sentry. |
4290bb8
to
d0db887
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Bai-Li-NOAA for these changes and @k-doering-NOAA for helping. I have a few questions, see comments. But, I also wanted to give more context for this PR for when I go back to it years later 🤣. GitHub actions for code coverage were running for over 6 hours leading to them failing with changes in 0.3.0.0. This is not the first time that it had happened but we did not address it before. Instead, I just turned them off for the merge into main knowing that we needed to fix them. The fix it to not have code coverage of parallel tests.
paths-ignore: | ||
- .devcontainer | ||
- .github | ||
- 'LICENSE' | ||
- 'README.md' | ||
- 'CONTRIBUTING.md' | ||
- 'Rbuildignore' | ||
- '.gitignore' | ||
- 'man' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we continue to ignore these files/folders? It seems to me like we would not care if we have code coverage of the manual files and these other non-code files. Was there something that was not working with these being ignored?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just learned yesterday that paths-ignore
prevents the workflow from running if only the listed files are changed. In the future, if the workflow takes too long to run, we could consider setting up a paths
filter. This would make the workflow run only when changes are made to specific folders, like inst/include
, src
, or R
, instead of listing all the non-code files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yeah, right. I forgot but, still, I think that we can keep these lines so the code coverage is not ran when a PR or push is made only to these files. For example, there is a GitHub action that only changes README.md and we wouldn't want to run the codecoverage test on that PR to dev. Or, if devtools::document() is ran and only changes are made to the man/*.Rd files we wouldn't want to run the coverage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it! I just added those lines back without making any changes. I’ll leave the final edits to @k-doering-NOAA through PR #720. Basically I’m not sure where to place paths-ignore
. Should it go under workflow-dispatch
, push
, or pull
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have to repeat them under both push and pull so that they are all ignored on both PR and pushes. For the workflow dispatch i think maybe we just want to test them all?
- 'man' | ||
push: | ||
branches: | ||
- main |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to have push to main because there should only be PR to main? So, I guess I am thinking that the tests wouldn't take as long if we didn't double up on a PR and a push to main by just doing it on the PR. I am fine with leaving it though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. It should only be a PR to main. I’ve made the changes, and @k-doering-NOAA, some changes from PR #720 might also need to be updated.
Second, this PR needs to be to dev not to main. But, I am not sure exactly how to change that. |
Thanks, @kellijohnson-NOAA. I'll address the questions in the comments and add more details to the PR description for future reference. I've updated the PR to target |
d0db887
to
a6eb09a
Compare
* update the DESCRIPTION file and set Config/testthat/parallel to false. This addresses the GHA run-time issue. * skip parallel tests when running covr::package_coverage(), as these tests don't contribute to increasing code coverage and are redundant due to the reuse of wrapper functions. Integration tests are sufficient for coverage calculation. * update GHA build triggers. Co-authored-by: Kathryn Doering <[email protected]>
a6eb09a
to
a64f057
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Bai-Li-NOAA I think this is ready. Please rebase and merge into dev whenever you are ready.
What is the feature?
How have you implemented the solution?
DESCRIPTION
file and setConfig/testthat/parallel
tofalse
.This addresses the GHA run-time issue.
covr::package_coverage()
, as these testsdon't contribute to increasing code coverage and are redundant due to
the reuse of wrapper functions. Integration tests are sufficient for
coverage calculation.
Does the PR impact any other area of the project, maybe another repo?