-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Report invalid units generated during test session (bugfix) #1560
Conversation
c7c4aae
to
9e11b88
Compare
+ use __index__ instead of UUID
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1560 +/- ##
==========================================
+ Coverage 48.80% 48.90% +0.09%
==========================================
Files 369 369
Lines 40192 40258 +66
Branches 6783 6799 +16
==========================================
+ Hits 19616 19687 +71
+ Misses 19857 19852 -5
Partials 719 719
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great stuff!
A few typos and other small comments, but otherwise good to land I think!
Co-authored-by: Pierre Equoy <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Much clearer, thanks!
Description
Currently if a template generates a unit that is invalid, a message is logged (error for some, debug for some others) and the unit is skipped. This message is visible using Checkbox local but it burried in the Agent log in remote. This is of course sub-optimal as we have no recollection of anything happening after the fact but this can lead to tests being... not executed, without an explanation or even a way for the user to notice.
This PR introduces a new unit, the
InvalidJob
unit, that allows us to report the invalidly generated units as failures, so that they are present all reports and on C3. This change is made along the newfeature
section, that allows us to toggle features like this on or off (to allow people to transition into big changes instead of having to update forcibly). The changes are both in this PR because the feature change is about 4 lines, and it makes sense to create and use it (else, I don't see how I could test it)Resolved issues
Fixes: CHECKBOX-1677
Fixes: CHECKBOX-1676
Documentation
Pending implementation approval, I think this should be mentioned in both the extended tutorial and the normal tutorial
Tests
This implements a new metabox scenario and unit tests