Skip to content

Commit

Permalink
upstream ci: Use a single random seed for spliting tests
Browse files Browse the repository at this point in the history
Dependind on how long it took for the jobs to start, a different seed
would be used to group tests and tests could either repeat or not be
selected at all.

By using a seed based on the day the test run reduces the chance of
using different random seeds, and still allow for the tests to be
executed in a different order.

The execution in different order is important to identify tests that
work or fail only if executed after other tests.
  • Loading branch information
rjeffman committed Oct 5, 2023
1 parent 8efe156 commit eab7d6a
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 0 deletions.
1 change: 1 addition & 0 deletions tests/azure/templates/galaxy_script.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ jobs:
--color=yes \
--splits=${{ parameters.number_of_groups }} \
--group=${{ parameters.group_number }} \
--randomly-seed=$(date "+%Y%m%d") \
--junit-xml=TEST-results-group-${{ parameters.group_number }}.xml
displayName: Run playbook tests
env:
Expand Down
1 change: 1 addition & 0 deletions tests/azure/templates/playbook_fast.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ jobs:
--suppress-no-test-exit-code \
--splits=${{ parameters.number_of_groups }} \
--group=${{ parameters.group_number }} \
--randomly-seed=$(date "+%Y%m%d") \
--junit-xml=TEST-results-group-${{ parameters.group_number }}.xml
then
[ $? -eq 5 ] && true || false
Expand Down
1 change: 1 addition & 0 deletions tests/azure/templates/playbook_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ jobs:
--color=yes \
--splits=${{ parameters.number_of_groups }} \
--group=${{ parameters.group_number }} \
--randomly-seed=$(date "+%Y%m%d") \
--junit-xml=TEST-results-group-${{ parameters.group_number }}.xml
displayName: Run playbook tests
env:
Expand Down

0 comments on commit eab7d6a

Please sign in to comment.