Remove the agent version override in tests. #3563

cmacknz · 2023-10-06T21:05:55Z

We now have working 8.12.0, 8.11.0, and 8.10.3 snapshots.

elasticmachine · 2023-10-06T21:05:58Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

elasticmachine · 2023-10-06T21:34:11Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-10-16T14:51:14.307+0000
Duration: 25 min 57 sec

Test stats 🧪

Test	Results
Failed	0
Passed	6501
Skipped	59
Total	6560

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2023-10-06T21:34:19Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	98.81% (`83/84`)	👍 0.029
Files	67.105% (`204/304`)	👎 -0.453
Classes	65.719% (`370/563`)	👎 -0.468
Methods	52.997% (`1167/2202`)	👎 -0.541
Lines	39.288% (`13604/34626`)	👎 -0.422
Conditionals	100.0% (`0/0`)	💚

We now have working 8.12.0 and 8.11.0 snapshots.

pchila · 2023-10-09T08:45:39Z

buildkite test this

pchila

The change is ok but the latest buildkite run is worrying: https://buildkite.com/elastic/elastic-agent/builds/3864

The TestAPM and TestStandaloneUpgradeWithGPGFallback fail again (symptoms look very similar to the fleet bootstrap issue we already experienced during 8.11.0 FF)

I relaunched an integration test run but this should not be merged until we get a clean run

Edit: me and @michalpristas had a look on main and this does not reintroduce the bootstrap mode we saw last week
However the failures for the tests need to be investigated anyways (the GPG ones look likethey are coming from 8.11.0 not being released but being selected as a version for the downgrade)

faec · 2023-10-09T18:56:50Z

It looks like the explicit test failure is trying to grab a snapshot URL that doesn't exist (maybe it's getting the hash wrong?) but I also noticed this line which looks weird to me:

>>> (linux-amd64-ubuntu-2204) Test output (sudo) (stdout): upgrader.go:190: Upgrading from version "8.12.0-SNAPSHOT" to version "8.11.0"

Is this an intentional "upgrade" to a lower version, or is the test misbehaving?

cmacknz · 2023-10-10T18:35:05Z

the GPG ones look likethey are coming from 8.11.0 not being released but being selected as a version for the downgrade

When we call https://artifacts-api.elastic.co/v1/versions/ it reports that 8.11 is a known version which isn't expected. This is because an 8.11 BC exists and we would need to download that.

{"versions":["7.17.10","7.17.11","7.17.12","7.17.13","7.17.14-SNAPSHOT","7.17.14","8.7.1","8.8.0","8.8.1","8.8.2","8.9.0","8.9.1","8.9.2","8.10.0-SNAPSHOT","8.10.0","8.10.1-SNAPSHOT","8.10.1","8.10.2-SNAPSHOT","8.10.2","8.10.3-SNAPSHOT","8.10.3","8.11.0-SNAPSHOT","8.11.0","8.12.0-SNAPSHOT"],"aliases":["7.17-SNAPSHOT","7.17","8.7","8.8","8.9","8.10-SNAPSHOT","8.10","8.11-SNAPSHOT","8.11","8.12-SNAPSHOT"],"manifests":{"last-update-time":"Tue, 10 Oct 2023 18:24:10 UTC","seconds-since-last-update":14}}

The problem is it comes from https://staging.elastic.co/8.11.0-8f52d632/summary-8.11.0.html#elastic-agent which we have no special handling for in the agent. This is because there is no obvious qualifier in the version itself to tell us to look in staging like there is for snapshots:

elastic-agent/internal/pkg/agent/application/upgrade/artifact/download/snapshot/downloader.go

Line 23 in 7e653eb

const snapshotURIFormat = "https://snapshots.elastic.co/%s-%s/downloads/"

We can either unconditionally use the snapshot version or look at the https://artifacts-api.elastic.co/v1/versions/8.11 to confirm the URL to use. I don't see a reason not to use snapshots since BCs have a larger time window between fixes if we need to fix a problem.

cmacknz · 2023-10-10T18:54:47Z

I think what we can do is check if the highest version number only has a snapshot available (8.12.0-SNAPSHOT). If it does it is unreleased, and the previous version must also be a snapshot to guarantee it exists.

The only caveat to this is we would prefer 8.11.1-SNAPSHOT to 8.11.0 once 8.11.0 is actually released, but I think that is actually preferable if we are still actively iterating on the code for the 8.11 branch.

cmacknz · 2023-10-11T13:08:07Z

And now 8.12.0-SNAPSHOT is broken.

cmacknz · 2023-10-12T13:52:55Z

buildkite test it

pchila

The only thing missing is a successful run of integration tests...

cmacknz · 2023-10-13T19:53:58Z

The RetryDownload upgrade test appears to be failing because of a bug where the the updated source URI is ignored in favour of the snapshot path. Probably this has to do with the fact that a dedicated snapshot downloader exists and gets tried before the http one so this will always fail for snapshots.

{"log.level":"info","@timestamp":"2023-10-12T21:13:48.976Z","log.origin":{"file.name":"upgrade/step_download.go","file.line":59},"message":"Downloading upgrade artifact","log":{"source":"elastic-agent"},"version":"8.11.0-SNAPSHOT","source_uri":"http://localhost:35383","drop_path":"","target_path":"/opt/Elastic/Agent/data/elastic-agent-f0c25c/downloads","install_path":"/opt/Elastic/Agent/data/elastic-agent-f0c25c/install","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-12T21:13:48.976Z","log.origin":{"file.name":"upgrade/step_download.go","file.line":188},"message":"download attempt 1","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-12T21:14:26.665Z","log.origin":{"file.name":"http/downloader.go","file.line":319},"message":"download from https://snapshots.elastic.co/8.11.0-60b804d4/downloads/beats/elastic-agent/elastic-agent-8.11.0-SNAPSHOT-linux-arm64.tar.gz completed in 37 seconds @ 17.52MBps","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-10-12T21:14:26.727Z","log.origin":{"file.name":"http/downloader.go","file.line":319},"message":"download from https://snapshots.elastic.co/8.11.0-60b804d4/downloads/beats/elastic-agent/elastic-agent-8.11.0-SNAPSHOT-linux-arm64.tar.gz.sha512 completed in Less than a second @ +InfYBps","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}

Use the agent core version in the condition in the policy because the variable evaluation strips the snapshot suffix.

elastic-sonarqube · 2023-10-16T15:02:45Z

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

100.0% Coverage
0.0% Duplication

* Remove the agent version override in tests. We now have working 8.12.0 and 8.11.0 snapshots. * Add failing unit test for upgradeable minor during FF. * Return 8.11.0-SNAPSHOT as the previous minor. * Fix broken condition in upgrade rollback test. Use the agent core version in the condition in the policy because the variable evaluation strips the snapshot suffix. * Don't auto overwrite sourceURI if it isn't the default. (cherry picked from commit 9adbac2)

* Remove the agent version override in tests. We now have working 8.12.0 and 8.11.0 snapshots. * Add failing unit test for upgradeable minor during FF. * Return 8.11.0-SNAPSHOT as the previous minor. * Fix broken condition in upgrade rollback test. Use the agent core version in the condition in the policy because the variable evaluation strips the snapshot suffix. * Don't auto overwrite sourceURI if it isn't the default. (cherry picked from commit 9adbac2) Co-authored-by: Craig MacKenzie <[email protected]>

cmacknz added the Team:Elastic-Agent Label for the Agent team label Oct 6, 2023

cmacknz self-assigned this Oct 6, 2023

cmacknz requested a review from a team as a code owner October 6, 2023 21:05

cmacknz requested review from faec and pchila October 6, 2023 21:05

cmacknz added skip-changelog backport-v8.11.0 Automated backport with mergify backport-v8.10.0 Automated backport with mergify labels Oct 6, 2023

cmacknz enabled auto-merge (squash) October 6, 2023 21:06

cmacknz force-pushed the remove-stack-version-override branch from 341a84a to 24f8690 Compare October 7, 2023 02:09

Remove the agent version override in tests.

7e653eb

We now have working 8.12.0 and 8.11.0 snapshots.

cmacknz force-pushed the remove-stack-version-override branch from 24f8690 to 7e653eb Compare October 7, 2023 02:09

pchila disabled auto-merge October 9, 2023 08:45

pchila requested changes Oct 9, 2023

View reviewed changes

pchila mentioned this pull request Oct 9, 2023

[8.10](backport #3518) Fix: Secondary fallback is not used for agent with local agent #3533

Merged

cmacknz added 3 commits October 10, 2023 15:52

Add failing unit test for upgradeable minor during FF.

89ad961

Correctly detect versions during FF.

b97766e

Fix typo in comment.

f088cc6

Fix make check.

219b110

pchila approved these changes Oct 12, 2023

View reviewed changes

Return 8.11.0-SNAPSHOT as the previous minor.

5a6ec52

cmacknz added 3 commits October 13, 2023 16:22

Fix broken condition in upgrade rollback test.

20ccbe4

Use the agent core version in the condition in the policy because the variable evaluation strips the snapshot suffix.

Don't auto overwrite sourceURI if it isn't the default.

a680be2

Fix snapshot source URI override, add test.

2329b71

cmacknz merged commit 9adbac2 into elastic:main Oct 16, 2023
8 checks passed

cmacknz deleted the remove-stack-version-override branch October 16, 2023 17:45

This was referenced Oct 16, 2023

[8.10](backport #3563) Remove the agent version override in tests. #3612

Merged

[8.11](backport #3563) Remove the agent version override in tests. #3613

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove the agent version override in tests. #3563

Remove the agent version override in tests. #3563

cmacknz commented Oct 6, 2023

elasticmachine commented Oct 6, 2023

elasticmachine commented Oct 6, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Oct 6, 2023 •

edited

Loading

pchila commented Oct 9, 2023

pchila left a comment •

edited

Loading

faec commented Oct 9, 2023

cmacknz commented Oct 10, 2023 •

edited

Loading

cmacknz commented Oct 10, 2023

cmacknz commented Oct 11, 2023

cmacknz commented Oct 12, 2023

pchila left a comment

cmacknz commented Oct 13, 2023

elastic-sonarqube bot commented Oct 16, 2023

Remove the agent version override in tests. #3563

Remove the agent version override in tests. #3563

Conversation

cmacknz commented Oct 6, 2023

elasticmachine commented Oct 6, 2023

elasticmachine commented Oct 6, 2023 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Oct 6, 2023 • edited Loading

🌐 Coverage report

pchila commented Oct 9, 2023

pchila left a comment • edited Loading

Choose a reason for hiding this comment

faec commented Oct 9, 2023

cmacknz commented Oct 10, 2023 • edited Loading

cmacknz commented Oct 10, 2023

cmacknz commented Oct 11, 2023

cmacknz commented Oct 12, 2023

pchila left a comment

Choose a reason for hiding this comment

cmacknz commented Oct 13, 2023

elastic-sonarqube bot commented Oct 16, 2023

elasticmachine commented Oct 6, 2023 •

edited

Loading

elasticmachine commented Oct 6, 2023 •

edited

Loading

pchila left a comment •

edited

Loading

cmacknz commented Oct 10, 2023 •

edited

Loading