[8.16](backport #5900) Introduce filter by group and dry-run for integration tests #6398
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
* add filtering integration tests by group * add dry-run integration test flag * mage check for define.Require should skip TestMain * Add define filter on sudo requirements (cherry picked from commit 7978189)
mergify
bot
requested review from
blakerouse and
andrzej-stencel
and removed request for
a team
|
|
Tests are failing due to unrelated reasons with |
pchila
requested review from
pchila
and removed request for
blakerouse and
andrzej-stencel
|
8.16.3-SNAPSHOT should be available now, relaunched the k8s tests as well hoping that OGC behaves on this retry... |
pchila
approved these changes
Dec 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds a
TestMain()to thetesting/integrationpackage to allow passing some command-line arguments when running integration tests usinggo test:-integration.groupsallows to specify a comma-separated list of groups to filter on-integration.dry-runwill validate define constraints but will skip the real tests that containdefinedirectives and add a placeholder<Test name>/dry-runwhich will pass if all thedefinechecks passThe
groupsflag allows for a simple way of selecting a subset of tests (for either real running or dry-runs)The
sudoflag allows to filter tests based on their sudo requirements (if those need to run as root/administrator or not)The
dry-runargument allows for quick execution when experimenting with group filtering (making sure that what would be executed matches expectations)Why is it important?
Running by groups should simplify initial implementation of running Integration tests on CI
Checklist
./changelog/fragmentsusing the changelog toolDisruptive User Impact
How to test this PR locally
(since it's a dry-run it won't actually run the tests)
=== RUN TestLongRunningAgentForLeaks define.go:170: not running as root and test requires root --- SKIP: TestLongRunningAgentForLeaks (0.00s) === RUN TestAPMConfig === RUN TestAPMConfig/dry-run define_flags.go:52: Test dry-run successful === NAME TestAPMConfig define_flags.go:54: Skipped because dry-run mode has been specified. --- SKIP: TestAPMConfig (0.00s) --- PASS: TestAPMConfig/dry-run (0.00s) === RUN TestBeatsServerless define.go:170: not running as root and test requires root --- SKIP: TestBeatsServerless (0.00s) === RUN TestContainerCMD define.go:149: group container not found in [fleet default]. Skipping --- SKIP: TestContainerCMD (0.00s) === RUN TestContainerCMDWithAVeryLongStatePath define.go:149: group container not found in [fleet default]. Skipping --- SKIP: TestContainerCMDWithAVeryLongStatePath (0.00s) ... a long list of tests is omitted here ... === RUN TestStandaloneUpgradeFailsWhenUpgradeIsInProgress define.go:149: group upgrade not found in [fleet default]. Skipping --- SKIP: TestStandaloneUpgradeFailsWhenUpgradeIsInProgress (0.00s) === RUN TestStandaloneUpgradeRetryDownload define.go:149: group upgrade not found in [fleet default]. Skipping --- SKIP: TestStandaloneUpgradeRetryDownload (0.00s) === RUN TestStandaloneUpgradeSameCommit define.go:149: group upgrade not found in [fleet default]. Skipping --- SKIP: TestStandaloneUpgradeSameCommit (0.00s) === RUN TestStandaloneUpgrade define.go:149: group upgrade not found in [fleet default]. Skipping --- SKIP: TestStandaloneUpgrade (0.00s) === RUN TestStandaloneUpgradeUninstallKillWatcher define.go:149: group upgrade not found in [fleet default]. Skipping --- SKIP: TestStandaloneUpgradeUninstallKillWatcher (0.00s) PASS ok github.com/elastic/elastic-agent/testing/integration 1.123sAdding a sudo filter for running only test that do not require sudo:
Related issues
Questions to ask yourself
This is an automatic backport of pull request #5900 done by [Mergify](https://mergify.com).