[6.15.z] addFinalizer in test body #14422
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherrypick of PR: #14036
Problem Statement
Turns out that
@request.addFinalizer
when used in test body (not in fixture) is dependent on the execution order, so basically when some assertion defined in the test before the finalizer part fails, finalizer is not added and so not executed.This exactly happened in
test_positive_disable_hammer_defaults
in upgrades, breaking a lot of other tests in the same run. Big kudos to @lpramuk for isolating the issue there.Solution
Moving the addFinalizer part to the start of the test, fixes the problem. Prt run won't reveal much in this case, but running the previous version of the test with induced assertion error leaves satellite with:
...which means that the finalizer didn't run. With this pr, the defaults are properly cleaned up:
Related Issues
I did a quick search for this pattern across robottelo and I found quite a bit of cases where
@request.addfinalizer
is added towards the end of the test body, with some preceding assertions that could possibly break it:robottelo/tests/foreman/cli/test_errata.py
Line 692 in d9c5ea8
robottelo/tests/foreman/api/test_discoveryrule.py
Line 188 in d9c5ea8
robottelo/tests/foreman/api/test_computeresource_libvirt.py
Line 256 in d9c5ea8
robottelo/tests/foreman/api/test_parameters.py
Line 58 in d9c5ea8
robottelo/tests/foreman/maintain/test_maintenance_mode.py
Line 146 in d9c5ea8
robottelo/tests/foreman/api/test_http_proxy.py
Line 316 in d9c5ea8
robottelo/tests/foreman/maintain/test_packages.py
Line 219 in d9c5ea8
robottelo/tests/foreman/ui/test_computeresource_gce.py
Line 217 in d9c5ea8
robottelo/tests/foreman/maintain/test_advanced.py
Line 104 in d9c5ea8
robottelo/tests/foreman/destructive/test_capsule_loadbalancer.py
Line 234 in d9c5ea8
robottelo/tests/foreman/maintain/test_health.py
Line 219 in d9c5ea8
robottelo/tests/foreman/ui/test_ansible.py
Line 233 in d9c5ea8
This doesn't include
request.addFinalizer(...)
calls that likely suffer from the same problem. Now, I could go and reorder all those occurrences, but I feel we should discuss this team wide and maybe abolish this practice all together