-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Fail CI if xfails pass unexpectedly #1888
Conversation
Be strict about tests marked with `xfail`. If they pass unexpectedly, fail the testsuite instead of loggin `XPASS`.
@tehrengruber it looks like gt4py/tests/next_tests/integration_tests/feature_tests/ffront_tests/test_execution.py Lines 264 to 277 in 3249aa5
passes with the numpy backend despite the general |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -93,6 +93,7 @@ def _process_args( | |||
el_range == range_ and el_dims == dims | |||
for (el_range, el_dims) in ranges_and_dims | |||
): | |||
print(ranges_and_dims) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Debugging leftover?
Nice! Thanks for fixing this 🚀 |
Description
Be strict about tests marked with
xfail
. If they pass unexpectedly, fail the testsuite instead of logging anXPASS
.Requirements
N/A