Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky drift correction tests #2858

Open
weyfonk opened this issue Sep 17, 2024 · 1 comment
Open

Fix flaky drift correction tests #2858

weyfonk opened this issue Sep 17, 2024 · 1 comment
Assignees
Milestone

Comments

@weyfonk
Copy link
Contributor

weyfonk commented Sep 17, 2024

Drift correction end-to-end tests are notoriously flaky. They currently get in the way of smoothly running CI workflows.

Here is an example of such failures, which mostly involve this test case.

@weyfonk weyfonk self-assigned this Sep 17, 2024
@weyfonk weyfonk added this to Fleet Sep 17, 2024
@github-project-automation github-project-automation bot moved this to 🆕 New in Fleet Sep 17, 2024
@weyfonk weyfonk moved this from 🆕 New to 🏗 In progress in Fleet Sep 17, 2024
@weyfonk weyfonk added this to the v2.10.0 milestone Sep 17, 2024
@weyfonk
Copy link
Contributor Author

weyfonk commented Sep 17, 2024

First hypothesis: the Fleet agent's garbage collection, acting on Helm releases missing a bundle deployment, might interfere with drift correction which installs releases, upgrades them and rolls them back.
This has been tested through #2861, by:

  1. Deploying Fleet with a longer garbage collection interval, which was then expected to reduce the likelihood of forced drift correction failing
  2. Disabling garbage collection altogether

In both cases, drift correction would still fail on the 1st or 2nd run (unlike locally, where it always passes), suggesting that garbage collection is not the culprit here.

@weyfonk weyfonk moved this from 🏗 In progress to ✅ Done in Fleet Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

1 participant