Ess terraform + run integration tests on BK agents #5113

pazone · 2024-07-10T22:25:33Z

What does this PR do?

A dedicated pipeline that runs Ubuntu integration tests directly on Buildkite runners without OGC. Currently it's not triggered by any of Github events.
Sudo tests build example

Improves groupping transparency. We can now immediately see long-running groups;
Adds the ability to rerun failed groups instead of the entire pipeline build;
Introduces a simple terraform configuration for ESS stack instead of a complex stack provisioner;
Uses prebuilt VM images platform-ingest-elastic-agent-ubuntu-2204. All required packages are pre-installed and we don't need to install them in runtime;
Adds the ability to use all the gobld-supported CSPs and VMs incl. ARM64 and MacOS.

Shared ESS stack and test reruns

The Start ESS stack for integration tests spins up an ESS stack that is used by following integration tests. When the tests are complete, the stack will be destroyed regardless of the test results. When we restart a single test group, it will start a dedicated stack and destroy it when the tests are complete.

This type of pipeline is supposed to replace the existing single-step approach. Further upcoming changes a listed in the the meta-issue.

Why is it important?

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in ./changelog/fragments using the changelog tool
I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

Related issues

Questions to ask yourself

How are we going to support this in production?
How are we going to measure its adoption?
How are we going to debug this?
What are the metrics I should take care of?
...

mergify · 2024-07-10T22:26:17Z

This pull request does not have a backport label. Could you fix it @pazone? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elastic-sonarqube · 2024-07-10T23:24:05Z

Quality Gate passed

Issues
4 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

mergify · 2024-07-16T11:28:40Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b ess_terraform upstream/ess_terraform
git merge upstream/main
git push upstream ess_terraform

… ess_terraform

elastic-sonarqube · 2024-10-07T21:51:50Z

Quality Gate passed

Issues
4 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

* ESS by terraforms * Use bk runners * added state to artifacts * Add common.sh to scripts * Add depends_on * Inlined paclage variables * tmp disbale unit tests * tmp disbale unit tests * install tf * install tf * install tf * install tf * fix ec creds * Added auto-approve * Test run * package again * package again * package again * package again * package * package * added mage build:testBinaries * Install mage * Install mage * Install mage * Install mage * Install mage * fix go * Merged * Merged * fix * fix * Fix tests * Debug * Use custom image * New suod tests * Test run * Test run * Test run * Debug * Debug * Debug * Debug * Debug * Cleanup * Using subshell in newgrp * Using subshell in newgrp * Test new custom VM image * Test new custom VM image * Set terraform version * Cleanup * terraform 1.9.3 * terraform 1.9.3 * Debug * Explicit image version * Explicit image version * Explicit image version * Sudo tests * +x * Try mage integration:local * Use gotestsum * Use gotestsum * Run sudo tests * Run sudo tests * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Increased test timeout * Groupped sudo tests * Quoted regexps * BK steps by groups * BK steps by groups * BK steps by groups * BK steps by groups * fix param propagation * Debug * Big run * reuse EC_API_KEY * Debug test * Debug a test * Revert debug * Debug * Html report * groupping, auto-retry, packaging * removed reruns * Reuse the initial ESS stack. Start a new stack for retries * Fix ess start path * Fix ess start path * Fix ess start path * Fix ess start path * Debug * Debug * Debug * Fix EC_KEY recovery * Added lock to artifacts * Fix artifacts download path * Fix artifacts download path * Fix artifacts download path * Removed unnecessaru build dependencies * Added build id * Separated BK integration tests to a dedicated pipeline * CLeanup * Applied proposed changes * Applied proposed changes * Applied proposed changes (cherry picked from commit 7aa8bb2)

* ESS by terraforms * Use bk runners * added state to artifacts * Add common.sh to scripts * Add depends_on * Inlined paclage variables * tmp disbale unit tests * tmp disbale unit tests * install tf * install tf * install tf * install tf * fix ec creds * Added auto-approve * Test run * package again * package again * package again * package again * package * package * added mage build:testBinaries * Install mage * Install mage * Install mage * Install mage * Install mage * fix go * Merged * Merged * fix * fix * Fix tests * Debug * Use custom image * New suod tests * Test run * Test run * Test run * Debug * Debug * Debug * Debug * Debug * Cleanup * Using subshell in newgrp * Using subshell in newgrp * Test new custom VM image * Test new custom VM image * Set terraform version * Cleanup * terraform 1.9.3 * terraform 1.9.3 * Debug * Explicit image version * Explicit image version * Explicit image version * Sudo tests * +x * Try mage integration:local * Use gotestsum * Use gotestsum * Run sudo tests * Run sudo tests * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Increased test timeout * Groupped sudo tests * Quoted regexps * BK steps by groups * BK steps by groups * BK steps by groups * BK steps by groups * fix param propagation * Debug * Big run * reuse EC_API_KEY * Debug test * Debug a test * Revert debug * Debug * Html report * groupping, auto-retry, packaging * removed reruns * Reuse the initial ESS stack. Start a new stack for retries * Fix ess start path * Fix ess start path * Fix ess start path * Fix ess start path * Debug * Debug * Debug * Fix EC_KEY recovery * Added lock to artifacts * Fix artifacts download path * Fix artifacts download path * Fix artifacts download path * Removed unnecessaru build dependencies * Added build id * Separated BK integration tests to a dedicated pipeline * CLeanup * Applied proposed changes * Applied proposed changes * Applied proposed changes (cherry picked from commit 7aa8bb2) Co-authored-by: Pavel Zorin <[email protected]>

* ESS by terraforms * Use bk runners * added state to artifacts * Add common.sh to scripts * Add depends_on * Inlined paclage variables * tmp disbale unit tests * tmp disbale unit tests * install tf * install tf * install tf * install tf * fix ec creds * Added auto-approve * Test run * package again * package again * package again * package again * package * package * added mage build:testBinaries * Install mage * Install mage * Install mage * Install mage * Install mage * fix go * Merged * Merged * fix * fix * Fix tests * Debug * Use custom image * New suod tests * Test run * Test run * Test run * Debug * Debug * Debug * Debug * Debug * Cleanup * Using subshell in newgrp * Using subshell in newgrp * Test new custom VM image * Test new custom VM image * Set terraform version * Cleanup * terraform 1.9.3 * terraform 1.9.3 * Debug * Explicit image version * Explicit image version * Explicit image version * Sudo tests * +x * Try mage integration:local * Use gotestsum * Use gotestsum * Run sudo tests * Run sudo tests * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Increased test timeout * Groupped sudo tests * Quoted regexps * BK steps by groups * BK steps by groups * BK steps by groups * BK steps by groups * fix param propagation * Debug * Big run * reuse EC_API_KEY * Debug test * Debug a test * Revert debug * Debug * Html report * groupping, auto-retry, packaging * removed reruns * Reuse the initial ESS stack. Start a new stack for retries * Fix ess start path * Fix ess start path * Fix ess start path * Fix ess start path * Debug * Debug * Debug * Fix EC_KEY recovery * Added lock to artifacts * Fix artifacts download path * Fix artifacts download path * Fix artifacts download path * Removed unnecessaru build dependencies * Added build id * Separated BK integration tests to a dedicated pipeline * CLeanup * Applied proposed changes * Applied proposed changes * Applied proposed changes (cherry picked from commit 7aa8bb2)

oakrizan · 2024-12-12T10:21:53Z

@Mergifyio backport 8.x

mergify · 2024-12-12T10:22:08Z

backport 8.x

✅ Backports have been created

#5736 [8.x](backport #5113) Ess terraform + run integration tests on BK agents has been created for branch 8.x

oakrizan · 2024-12-13T06:55:05Z

https://github.com/Mergifyio backport 8.16

mergify · 2024-12-13T06:55:24Z

backport 8.16

✅ Backports have been created

#6321 Ess terraform + run integration tests on BK agents (backport #5113) has been created for branch 8.16 but encountered conflicts

* ESS by terraforms * Use bk runners * added state to artifacts * Add common.sh to scripts * Add depends_on * Inlined paclage variables * tmp disbale unit tests * tmp disbale unit tests * install tf * install tf * install tf * install tf * fix ec creds * Added auto-approve * Test run * package again * package again * package again * package again * package * package * added mage build:testBinaries * Install mage * Install mage * Install mage * Install mage * Install mage * fix go * Merged * Merged * fix * fix * Fix tests * Debug * Use custom image * New suod tests * Test run * Test run * Test run * Debug * Debug * Debug * Debug * Debug * Cleanup * Using subshell in newgrp * Using subshell in newgrp * Test new custom VM image * Test new custom VM image * Set terraform version * Cleanup * terraform 1.9.3 * terraform 1.9.3 * Debug * Explicit image version * Explicit image version * Explicit image version * Sudo tests * +x * Try mage integration:local * Use gotestsum * Use gotestsum * Run sudo tests * Run sudo tests * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Increased test timeout * Groupped sudo tests * Quoted regexps * BK steps by groups * BK steps by groups * BK steps by groups * BK steps by groups * fix param propagation * Debug * Big run * reuse EC_API_KEY * Debug test * Debug a test * Revert debug * Debug * Html report * groupping, auto-retry, packaging * removed reruns * Reuse the initial ESS stack. Start a new stack for retries * Fix ess start path * Fix ess start path * Fix ess start path * Fix ess start path * Debug * Debug * Debug * Fix EC_KEY recovery * Added lock to artifacts * Fix artifacts download path * Fix artifacts download path * Fix artifacts download path * Removed unnecessaru build dependencies * Added build id * Separated BK integration tests to a dedicated pipeline * CLeanup * Applied proposed changes * Applied proposed changes * Applied proposed changes (cherry picked from commit 7aa8bb2) # Conflicts: # .buildkite/bk.integration.pipeline.yml # .buildkite/scripts/steps/ess.sh # .buildkite/scripts/sudo-integration-tests.sh

oakrizan · 2024-12-13T06:57:14Z

https://github.com/Mergifyio backport 8.17

mergify · 2024-12-13T06:57:49Z

backport 8.17

✅ Backports have been created

Backport to branch 8.17 not needed, change already in branch 8.17

* ESS by terraforms * Use bk runners * added state to artifacts * Add common.sh to scripts * Add depends_on * Inlined paclage variables * tmp disbale unit tests * tmp disbale unit tests * install tf * install tf * install tf * install tf * fix ec creds * Added auto-approve * Test run * package again * package again * package again * package again * package * package * added mage build:testBinaries * Install mage * Install mage * Install mage * Install mage * Install mage * fix go * Merged * Merged * fix * fix * Fix tests * Debug * Use custom image * New suod tests * Test run * Test run * Test run * Debug * Debug * Debug * Debug * Debug * Cleanup * Using subshell in newgrp * Using subshell in newgrp * Test new custom VM image * Test new custom VM image * Set terraform version * Cleanup * terraform 1.9.3 * terraform 1.9.3 * Debug * Explicit image version * Explicit image version * Explicit image version * Sudo tests * +x * Try mage integration:local * Use gotestsum * Use gotestsum * Run sudo tests * Run sudo tests * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Increased test timeout * Groupped sudo tests * Quoted regexps * BK steps by groups * BK steps by groups * BK steps by groups * BK steps by groups * fix param propagation * Debug * Big run * reuse EC_API_KEY * Debug test * Debug a test * Revert debug * Debug * Html report * groupping, auto-retry, packaging * removed reruns * Reuse the initial ESS stack. Start a new stack for retries * Fix ess start path * Fix ess start path * Fix ess start path * Fix ess start path * Debug * Debug * Debug * Fix EC_KEY recovery * Added lock to artifacts * Fix artifacts download path * Fix artifacts download path * Fix artifacts download path * Removed unnecessaru build dependencies * Added build id * Separated BK integration tests to a dedicated pipeline * CLeanup * Applied proposed changes * Applied proposed changes * Applied proposed changes (cherry picked from commit 7aa8bb2) # Conflicts: # .buildkite/bk.integration.pipeline.yml # .buildkite/scripts/steps/ess.sh # .buildkite/scripts/sudo-integration-tests.sh

* ESS by terraforms * Use bk runners * added state to artifacts * Add common.sh to scripts * Add depends_on * Inlined paclage variables * tmp disbale unit tests * tmp disbale unit tests * install tf * install tf * install tf * install tf * fix ec creds * Added auto-approve * Test run * package again * package again * package again * package again * package * package * added mage build:testBinaries * Install mage * Install mage * Install mage * Install mage * Install mage * fix go * Merged * Merged * fix * fix * Fix tests * Debug * Use custom image * New suod tests * Test run * Test run * Test run * Debug * Debug * Debug * Debug * Debug * Cleanup * Using subshell in newgrp * Using subshell in newgrp * Test new custom VM image * Test new custom VM image * Set terraform version * Cleanup * terraform 1.9.3 * terraform 1.9.3 * Debug * Explicit image version * Explicit image version * Explicit image version * Sudo tests * +x * Try mage integration:local * Use gotestsum * Use gotestsum * Run sudo tests * Run sudo tests * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Debug * Increased test timeout * Groupped sudo tests * Quoted regexps * BK steps by groups * BK steps by groups * BK steps by groups * BK steps by groups * fix param propagation * Debug * Big run * reuse EC_API_KEY * Debug test * Debug a test * Revert debug * Debug * Html report * groupping, auto-retry, packaging * removed reruns * Reuse the initial ESS stack. Start a new stack for retries * Fix ess start path * Fix ess start path * Fix ess start path * Fix ess start path * Debug * Debug * Debug * Fix EC_KEY recovery * Added lock to artifacts * Fix artifacts download path * Fix artifacts download path * Fix artifacts download path * Removed unnecessaru build dependencies * Added build id * Separated BK integration tests to a dedicated pipeline * CLeanup * Applied proposed changes * Applied proposed changes * Applied proposed changes (cherry picked from commit 7aa8bb2)

pazone added 3 commits July 5, 2024 17:21

ESS by terraforms

6f538df

Use bk runners

03a4cef

added state to artifacts

44bff94

pazone added the enhancement New feature or request label Jul 10, 2024

pazone changed the title ~~Ess terraform~~ Ess terraform + run integration tests on BK agents Jul 10, 2024

mergify bot assigned pazone Jul 10, 2024

mergify bot added the backport-skip label Jul 10, 2024

pazone added 3 commits July 11, 2024 00:33

Add common.sh to scripts

e80884b

Add depends_on

d438c5b

Inlined paclage variables

a72f93c

pazone added 7 commits July 11, 2024 01:24

tmp disbale unit tests

77c7946

tmp disbale unit tests

f194932

install tf

3c9b37c

install tf

37ac771

install tf

8d52711

install tf

5a6a00c

fix ec creds

bda90aa

ycombinator added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team and removed Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team labels Jul 11, 2024

Added auto-approve

1078f3c

pazone added 6 commits July 16, 2024 13:40

Merged

607fc55

Test run

4901bef

package again

e31792b

package again

a0a3f38

package again

b141cb9

package again

4f16839

Merge branch 'ess_terraform' of github.com:elastic/elastic-agent into…

5097ddc

… ess_terraform

pazone merged commit 7aa8bb2 into main Oct 8, 2024
14 checks passed

pazone deleted the ess_terraform branch October 8, 2024 08:35

mergify bot mentioned this pull request Oct 8, 2024

[8.x](backport #5113) Ess terraform + run integration tests on BK agents #5736

Merged

7 tasks

pazone mentioned this pull request Oct 15, 2024

[CI] Pipeline resource for BK integration tests #5717

Merged

7 tasks

oakrizan added backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify backport-8.15 Automated backport to the 8.15 branch with mergify and removed backport-skip labels Dec 11, 2024

mergify bot mentioned this pull request Dec 11, 2024

[8.15](backport #5113) Ess terraform + run integration tests on BK agents #6282

Closed

7 tasks

oakrizan added backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify and removed backport-8.15 Automated backport to the 8.15 branch with mergify backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify labels Dec 11, 2024

mergify bot mentioned this pull request Dec 13, 2024

Ess terraform + run integration tests on BK agents (backport #5113) #6321

Draft

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ess terraform + run integration tests on BK agents #5113

Ess terraform + run integration tests on BK agents #5113

pazone commented Jul 10, 2024 •

edited

Loading

mergify bot commented Jul 10, 2024

elastic-sonarqube bot commented Jul 10, 2024

mergify bot commented Jul 16, 2024

elastic-sonarqube bot commented Oct 7, 2024

oakrizan commented Dec 12, 2024

mergify bot commented Dec 12, 2024

oakrizan commented Dec 13, 2024

mergify bot commented Dec 13, 2024 •

edited

Loading

oakrizan commented Dec 13, 2024

mergify bot commented Dec 13, 2024 •

edited

Loading

Ess terraform + run integration tests on BK agents #5113

Ess terraform + run integration tests on BK agents #5113

Conversation

pazone commented Jul 10, 2024 • edited Loading

What does this PR do?

Shared ESS stack and test reruns

Why is it important?

Checklist

Disruptive User Impact

How to test this PR locally

Related issues

Questions to ask yourself

mergify bot commented Jul 10, 2024

elastic-sonarqube bot commented Jul 10, 2024

Quality Gate passed

mergify bot commented Jul 16, 2024

elastic-sonarqube bot commented Oct 7, 2024

Quality Gate passed

oakrizan commented Dec 12, 2024

mergify bot commented Dec 12, 2024

✅ Backports have been created

oakrizan commented Dec 13, 2024

mergify bot commented Dec 13, 2024 • edited Loading

✅ Backports have been created

oakrizan commented Dec 13, 2024

mergify bot commented Dec 13, 2024 • edited Loading

✅ Backports have been created

pazone commented Jul 10, 2024 •

edited

Loading

mergify bot commented Dec 13, 2024 •

edited

Loading

mergify bot commented Dec 13, 2024 •

edited

Loading