Tests random order #16508
Open
Tests random order #16508
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You can achieve this now by installing pytest-random-order: pip3 install pytest-random-order Then running pytest with this: python3 -m pytest --random-order=package -nauto --dist=loadfile Tests will be run in a random order. Signed-off-by: Donald Sharp <[email protected]>
I used sed to get the pytest changes in place, run black to get the formatting right. Signed-off-by: Donald Sharp <[email protected]>
Signed-off-by: Donald Sharp <[email protected]>
ton31337
reviewed
Aug 12, 2024
There was a problem hiding this comment.
Can't we just add something like that? To avoid touching every test file we have.
[pytest]
...
random_order_disable = true
Even more, reading the manual I see:
From v1.0.0 onwards, this plugin no longer randomises tests by default. To enable randomisation, you have to run
pytest in one of the following ways:
pytest --random-order
pytest --random-order-bucket=<bucket_type>
pytest --random-order-seed=
So, this is not needed at all to disable the plugin in every file? Which version did you test?
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add the ability to the test suites to run the tests in a random order.
pip3 install pytest-random-order
sudo -E python3 -m pytest -s -vv -n --dist=loadfile --random-order=package
Consult the pytest-random-order module for further instructions.