Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch back to Ubuntu-based Docker images in the cloud #6389

Closed
wants to merge 325 commits into from

Conversation

rdner
Copy link
Member

@rdner rdner commented Dec 18, 2024

According to the support matrix, ECE versions up to 3.6 are marked as compatible with Docker 19.03 and the Wolfi images require Docker version of 20.10.10.

Before the change

Size

docker.elastic.co/beats-ci/elastic-agent-cloud 9.0.0 74efb1bb7e5f 8 minutes ago  1.19GB
Inspect ``` [ { "Id": "sha256:74efb1bb7e5f306faa072c9e43c7b5543e254bef6273d75062b7f86974349c83", "RepoTags": [ "docker.elastic.co/beats-ci/elastic-agent-cloud:9.0.0" ], "RepoDigests": [], "Parent": "", "Comment": "buildkit.dockerfile.v0", "Created": "2024-12-18T17:35:53.494813844Z", "DockerVersion": "", "Author": "github.com/chainguard-dev/apko", "Config": { "Hostname": "", "Domainname": "", "User": "elastic-agent", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/share/elastic-agent:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt", "BEAT_SETUID_AS=elastic-agent", "ELASTIC_CONTAINER=true", "GODEBUG=madvdontneed=1", "LIBBEAT_MONITORING_CGROUPS_HIERARCHY_OVERRIDE=/" ], "Cmd": [ "/app/apm.sh" ], "ArgsEscaped": true, "Image": "", "Volumes": null, "WorkingDir": "/usr/share/elastic-agent", "Entrypoint": [ "/usr/bin/tini", "--" ], "OnBuild": null, "Labels": { "description": "Elastic Agent - single, unified way to add monitoring for logs, metrics, and other types of data to a host.", "io.k8s.description": "Elastic Agent - single, unified way to add monitoring for logs, metrics, and other types of data to a host.", "io.k8s.display-name": "Elastic-Agent image", "license": "Elastic License", "name": "elastic-agent", "org.label-schema.build-date": "2024-12-18T17:32:48Z", "org.label-schema.license": "Elastic License", "org.label-schema.name": "elastic-agent", "org.label-schema.schema-version": "1.0", "org.label-schema.url": "https://www.elastic.co/beats/elastic-agent", "org.label-schema.vcs-ref": "c7c5ba4132605f9bdef4b1bd63ab6337fef4abba", "org.label-schema.vcs-url": "github.com/elastic/elastic-agent", "org.label-schema.vendor": "Elastic", "org.label-schema.version": "9.0.0", "org.opencontainers.image.authors": "[email protected]", "org.opencontainers.image.created": "2024-12-18T17:32:48Z", "org.opencontainers.image.licenses": "Elastic License", "org.opencontainers.image.source": "https://github.com/chainguard-images/images/tree/main/images/wolfi-base", "org.opencontainers.image.title": "Elastic-Agent", "org.opencontainers.image.url": "https://images.chainguard.dev/directory/image/wolfi-base/overview", "org.opencontainers.image.vendor": "Elastic", "release": "1", "summary": "elastic-agent", "url": "https://www.elastic.co/beats/elastic-agent", "vendor": "Elastic", "version": "9.0.0" } }, "Architecture": "arm64", "Os": "linux", "Size": 1194750494, "GraphDriver": { "Data": { "LowerDir": "/var/lib/docker/overlay2/sdg87mdq4svpd7wpkl11sh842/diff:/var/lib/docker/overlay2/axphtz8sdlrn9zleog4gjnxpq/diff:/var/lib/docker/overlay2/gv8meh3jyzhot8w9g3usn9g5h/diff:/var/lib/docker/overlay2/t6rl6nhz1uspyh5o6lkqu4yuf/diff:/var/lib/docker/overlay2/7vkoxbttkkm2wyolfqebfdx1a/diff:/var/lib/docker/overlay2/f6bujzjazl6j1bvd2ms0ja4de/diff:/var/lib/docker/overlay2/zyxzjmufyrqyt22jg8hf8xpwj/diff:/var/lib/docker/overlay2/cpgqhyvdig306s49ltflhudbv/diff:/var/lib/docker/overlay2/dybe5bv4s0aekp192evrpo5xe/diff:/var/lib/docker/overlay2/r6k3ajqdfjtf9hri5esfr9196/diff:/var/lib/docker/overlay2/k6ydhzhf9l4lqge8d2j5ch3sr/diff:/var/lib/docker/overlay2/j450cel3vt1dj7x3i9jzvpayg/diff:/var/lib/docker/overlay2/19d1789725c2bd0f98f49a62b0741133b362cbb5922d94e0205ed66fc2b0048c/diff", "MergedDir": "/var/lib/docker/overlay2/y2fbx2i3nr65gxbdcaz837wt8/merged", "UpperDir": "/var/lib/docker/overlay2/y2fbx2i3nr65gxbdcaz837wt8/diff", "WorkDir": "/var/lib/docker/overlay2/y2fbx2i3nr65gxbdcaz837wt8/work" }, "Name": "overlay2" }, "RootFS": { "Type": "layers", "Layers": [ "sha256:f43404bb77009eea1482839f0cf091b83c81f150204978d4851b13328404b4f3", "sha256:d791d6174769d58a3213539eb0b93ee943bb1778d89131c6f29f2e87383b312a", "sha256:95a92be37c3b843ad872e5c079422575e67a38163783b23d9623a798bb92461f", "sha256:2fb92e7b84a20c65899807a21a01fe47269edca71a30fab6a78c2ebe7be3330b", "sha256:4a06365095c50dcdf68a0744c4873b89db22e56354c34bc9193752ee3f875540", "sha256:e923d5b7583a12ea602b0ac8acd8c70f0adcfbcaa5a9e15fa6ba5d367dc7fe4b", "sha256:e80ff90499224bd249d5242ea793ecda1433f0fd7de5c63d40c930e6fd39280a", "sha256:1c0b1baf72397b2d8274a7b9f3a3618ea9e89c740c277379bfd320583e999b13", "sha256:850900da5654a0f88542d226f6e2159b863cdc04782f39913f1fa1c64dd90729", "sha256:bdac68a04cbdf845c2cb5acb44ce9146d5ade2e0742ff2eb8e874f84e0b59797", "sha256:45f4ae444093f5270d263da4d82a17880c6312e572ae77e3af0e7906efdd5d69", "sha256:4fd67414b59e254b220c3651ebeeae082bb4c3427e44483286a997e23b05b900", "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef", "sha256:c57f2d1b12e7ef385e65f55e804afd3540d83a3fd4338cd0658cc813d1bde865" ] }, "Metadata": { "LastTagTime": "2024-12-18T17:35:54.500567761Z" } } ] ```
History
IMAGE          CREATED         CREATED BY                                      SIZE      COMMENT
74efb1bb7e5f   6 minutes ago   RUN /bin/sh -c echo -e '#!/bin/sh\nexec /usr…   48B       buildkit.dockerfile.v0
<missing>      6 minutes ago   CMD ["/app/apm.sh"]                             0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   ENTRYPOINT ["/usr/bin/tini" "--"]               0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   WORKDIR /usr/share/elastic-agent                0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   ENV LIBBEAT_MONITORING_CGROUPS_HIERARCHY_OVE…   0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   USER elastic-agent                              0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   RUN /bin/sh -c mkdir /app &&     chown elast…   0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   COPY /opt /opt # buildkit                       384MB     buildkit.dockerfile.v0
<missing>      6 minutes ago   COPY /usr/share/elastic-agent/NOTICE.txt /li…   5.46MB    buildkit.dockerfile.v0
<missing>      6 minutes ago   COPY /usr/share/elastic-agent/LICENSE.txt /l…   3.86kB    buildkit.dockerfile.v0
<missing>      6 minutes ago   RUN /bin/sh -c mkdir /licenses # buildkit       0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   RUN /bin/sh -c chmod 0777 /usr/share/elastic…   101kB     buildkit.dockerfile.v0
<missing>      6 minutes ago   COPY --chown=elastic-agent:elastic-agent /us…   743MB     buildkit.dockerfile.v0
<missing>      6 minutes ago   RUN /bin/sh -c groupadd --gid 1000 elastic-a…   4.18kB    buildkit.dockerfile.v0
<missing>      6 minutes ago   COPY docker-entrypoint /usr/local/bin/docker…   509B      buildkit.dockerfile.v0
<missing>      6 minutes ago   RUN /bin/sh -c set -e ;   TINI_BIN="";   TIN…   23.9kB    buildkit.dockerfile.v0
<missing>      6 minutes ago   ENV GODEBUG=madvdontneed=1                      0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   ENV PATH=/usr/share/elastic-agent:/usr/local…   0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   ENV ELASTIC_CONTAINER=true                      0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   LABEL org.label-schema.build-date=2024-12-18…   0B        buildkit.dockerfile.v0
<missing>      6 minutes ago   RUN /bin/sh -c for iter in {1..10}; do      …   48.8MB    buildkit.dockerfile.v0
<missing>      6 minutes ago   ENV BEAT_SETUID_AS=elastic-agent                0B        buildkit.dockerfile.v0
<missing>      6 weeks ago     apko                                            13.6MB    This is an apko single-layer image

After the change

Size

docker.elastic.co/beats-ci/elastic-agent-cloud 9.0.0 d0e958e84ab7 2 minutes ago 1.29GB
Inspect ``` [ { "Id": "sha256:d0e958e84ab7a0f32d0e43ff2c39384949b3968f47b2a1e100e068e74fd91644", "RepoTags": [ "docker.elastic.co/beats-ci/elastic-agent-cloud:9.0.0" ], "RepoDigests": [], "Parent": "", "Comment": "buildkit.dockerfile.v0", "Created": "2024-12-18T17:48:25.635877762Z", "DockerVersion": "", "Author": "", "Config": { "Hostname": "", "Domainname": "", "User": "elastic-agent", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/share/elastic-agent:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "BEAT_SETUID_AS=elastic-agent", "ELASTIC_CONTAINER=true", "GODEBUG=madvdontneed=1", "LIBBEAT_MONITORING_CGROUPS_HIERARCHY_OVERRIDE=/" ], "Cmd": [ "/app/apm.sh" ], "ArgsEscaped": true, "Image": "", "Volumes": null, "WorkingDir": "/usr/share/elastic-agent", "Entrypoint": [ "/usr/bin/tini", "--" ], "OnBuild": null, "Labels": { "description": "Elastic Agent - single, unified way to add monitoring for logs, metrics, and other types of data to a host.", "io.k8s.description": "Elastic Agent - single, unified way to add monitoring for logs, metrics, and other types of data to a host.", "io.k8s.display-name": "Elastic-Agent image", "license": "Elastic License", "name": "elastic-agent", "org.label-schema.build-date": "2024-12-18T17:46:10Z", "org.label-schema.license": "Elastic License", "org.label-schema.name": "elastic-agent", "org.label-schema.schema-version": "1.0", "org.label-schema.url": "https://www.elastic.co/beats/elastic-agent", "org.label-schema.vcs-ref": "cec09bd03a1ca5617ad1a7434ed8f7ce51fe66db", "org.label-schema.vcs-url": "github.com/elastic/elastic-agent", "org.label-schema.vendor": "Elastic", "org.label-schema.version": "9.0.0", "org.opencontainers.image.authors": "[email protected]", "org.opencontainers.image.created": "2024-12-18T17:46:10Z", "org.opencontainers.image.licenses": "Elastic License", "org.opencontainers.image.ref.name": "ubuntu", "org.opencontainers.image.title": "Elastic-Agent", "org.opencontainers.image.vendor": "Elastic", "org.opencontainers.image.version": "24.04", "release": "1", "summary": "elastic-agent", "url": "https://www.elastic.co/beats/elastic-agent", "vendor": "Elastic", "version": "9.0.0" } }, "Architecture": "arm64", "Variant": "v8", "Os": "linux", "Size": 1289212891, "GraphDriver": { "Data": { "LowerDir": "/var/lib/docker/overlay2/zqb2zub8ti249yla316c400r7/diff:/var/lib/docker/overlay2/i6814mqjaj0kyqac8b82dffpz/diff:/var/lib/docker/overlay2/q9zpgnfwhgrlkhz9jj9fr4lqr/diff:/var/lib/docker/overlay2/ul5e3a5ql5tpa2ojgu5b1yw4u/diff:/var/lib/docker/overlay2/zh8w3h7nanu0phy7011ruhw8c/diff:/var/lib/docker/overlay2/zyp8j2prxuc95xd83ghlc9c62/diff:/var/lib/docker/overlay2/vx954xu959wd59nbu58w8yd0k/diff:/var/lib/docker/overlay2/762zpt0itxo4hby8fwdadb6kc/diff:/var/lib/docker/overlay2/pm2845rl4cbiypfw2230o5mvz/diff:/var/lib/docker/overlay2/ojottaod38q96s3qg0kby3kzo/diff:/var/lib/docker/overlay2/aq2k7cy1i867st82jo8ci34du/diff:/var/lib/docker/overlay2/1qh0xmg17kbbski7aozhj0wgf/diff:/var/lib/docker/overlay2/enxj36jrm8kv3fmxnvg61o1sg/diff:/var/lib/docker/overlay2/1813c99c723780173339a1fecc28a299baad1665955cdc48fa176167c64de487/diff", "MergedDir": "/var/lib/docker/overlay2/kb0qegonzjdf3tfwagc1gxs94/merged", "UpperDir": "/var/lib/docker/overlay2/kb0qegonzjdf3tfwagc1gxs94/diff", "WorkDir": "/var/lib/docker/overlay2/kb0qegonzjdf3tfwagc1gxs94/work" }, "Name": "overlay2" }, "RootFS": { "Type": "layers", "Layers": [ "sha256:1575723d84b80a2963cb6b2bf2ae5cefd3859bda7470710cff2b703d40a92000", "sha256:e264862abcae2e6b0a473ee6fb70c0f2e93aca368e86af2ed37079bcfa2e987e", "sha256:47a2620b091b009ecfd194ebbd45b259731860ee2b58f9d545e4a7a22573bb13", "sha256:d549795f698d16e410fd37d7b2ae7d79ee3af7afa8151488a46bb5f0e942c71a", "sha256:4655a57d2a6de858f64a40214036c3ab34e7b9158ea9276e11f6de7a1b966302", "sha256:83ac2eae372f96c12241dffdceaa37efef41ff0076a409a6b083aa3382b02d79", "sha256:eb41815406f33e93f68f81de6a4eeccca5f90c67544dbf4af248596af51d7904", "sha256:51e942e7cbb5ab926d5c481f5ada6d6dd677636d05b0b93846f8c57911e10d00", "sha256:e92a51c2f935ab6ec4dbb9b9747dfa5fec7f356647339b5bc1ff5e4a24de5afd", "sha256:b571c858d7043f134662c5bfcc26ea036433291a4ba6a86f4ecba42421051018", "sha256:176d4b6dfbffe7a735d7b85e459925fc1bef0107595f67bcaf771f378fbcd634", "sha256:b725525f3f4ea6597f4308a4ecee26f1411c440b43d3dda9e481408a145b44d7", "sha256:af9db6d5ddcc7238f19ff66e3ed96703725a669a1974a21585416ee6f26a340e", "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef", "sha256:a8e53c288bdcdc5c816ea00fc056abb539dd1a67ea3c9762c11a6acaf9686a78" ] }, "Metadata": { "LastTagTime": "2024-12-18T17:48:26.630738345Z" } } ] ```
History
IMAGE          CREATED         CREATED BY                                      SIZE      COMMENT
d0e958e84ab7   2 minutes ago   RUN /bin/sh -c echo -e '#!/bin/sh\nexec /usr…   51B       buildkit.dockerfile.v0
<missing>      2 minutes ago   CMD ["/app/apm.sh"]                             0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   ENTRYPOINT ["/usr/bin/tini" "--"]               0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   WORKDIR /usr/share/elastic-agent                0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   ENV LIBBEAT_MONITORING_CGROUPS_HIERARCHY_OVE…   0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   USER elastic-agent                              0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c mkdir /app &&     chown elast…   0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   COPY /opt /opt # buildkit                       384MB     buildkit.dockerfile.v0
<missing>      2 minutes ago   COPY /usr/share/elastic-agent/NOTICE.txt /li…   5.46MB    buildkit.dockerfile.v0
<missing>      2 minutes ago   COPY /usr/share/elastic-agent/LICENSE.txt /l…   3.86kB    buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c mkdir /licenses # buildkit       0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c chmod 0777 /usr/share/elastic…   101kB     buildkit.dockerfile.v0
<missing>      2 minutes ago   COPY --chown=elastic-agent:elastic-agent /us…   743MB     buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c groupadd --gid 1000 elastic-a…   4.98kB    buildkit.dockerfile.v0
<missing>      2 minutes ago   COPY docker-entrypoint /usr/local/bin/docker…   509B      buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c set -e ;   TINI_BIN="";   TIN…   23.9kB    buildkit.dockerfile.v0
<missing>      2 minutes ago   ENV GODEBUG=madvdontneed=1                      0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   ENV PATH=/usr/share/elastic-agent:/usr/local…   0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   ENV ELASTIC_CONTAINER=true                      0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   LABEL org.label-schema.build-date=2024-12-18…   0B        buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c for iter in {1..10}; do      …   56.1MB    buildkit.dockerfile.v0
<missing>      2 minutes ago   RUN /bin/sh -c touch /var/mail/ubuntu && cho…   4.47kB    buildkit.dockerfile.v0
<missing>      2 minutes ago   ENV BEAT_SETUID_AS=elastic-agent                0B        buildkit.dockerfile.v0
<missing>      4 weeks ago     /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B        
<missing>      4 weeks ago     /bin/sh -c #(nop) ADD file:765dfd09ec2ac4870…   101MB     
<missing>      4 weeks ago     /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B        
<missing>      4 weeks ago     /bin/sh -c #(nop)  LABEL org.opencontainers.…   0B        
<missing>      4 weeks ago     /bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH     0B        
<missing>      4 weeks ago     /bin/sh -c #(nop)  ARG RELEASE                  0B        

kruskall and others added 30 commits September 17, 2024 19:56
* Force agent version to 8.16.0 to fix CI

* Fix beats serverless tests

* fix k8s integration tests
* Read Kibana version to build dashboard export folder

TestBeatsServerless/TestSetupDashboards/export_dashboards checks for
the exported dashboards files, however the folder depends on the
Kibana version, the test is updated to read the Kibana version and
build the folder path using it.

* Update testing/integration/beats_serverless_test.go

Co-authored-by: Paolo Chilà <[email protected]>

* Fetch Kibana version upon initial connection

---------

Co-authored-by: Paolo Chilà <[email protected]>
Co-authored-by: Craig MacKenzie <[email protected]>
Co-authored-by: Shaunak Kashyap <[email protected]>
…lastic#5494)

It adds support for encrypted client certificate key during install/enroll, which done by the cli flag `--elastic-agent-cert-key-passphrase`.
* Clarify commit message

* Convert file to YML and add header comment

* Reintroducing rand/v2 import cleaned up by IDE

* Use YAML v3

* Replace .agent-versions.json with .agent-versions.yml

* Running mage fmt

* Adding a newline in the header comment

* Fix comment

* Fix commit message

* Look for changes in new file

* Moving + renaming agent versions file

* Updating code to refer to new file location + name

* Running mage integration:updateVersions
* Add pprofextension from the OTel-contrilib

This commit adds the pprofextension to the OTel collector.

Signed-off-by: Alexandros Sapranidis <[email protected]>
…... (elastic#5576)

... iscovery/updatecli" Updatecli version policy

Made with ❤️️ by updatecli

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Change maintainer label to opencontainers in Dockerfile

The old way of specifying maintainers in a Dockerfile was to use the
MAINTAINER instruction. This is now deprecated and the maintainer label was
used instead.

Nowadays the recommended way to set a maintainer is through the
`org.opencontainers.image.authors` label.

See https://docs.docker.com/reference/build-checks/maintainer-deprecated/.

* add changelog entry
* feat: implement values.schema.json for elastic-agent helm chart

* fix: update rendered examples to latest elastic-agent version

* feat: split ESPlainAuth into ESPlainAuthBasic and ESPlainAuthAPI for better validation

* fix: omit redundant keys for ESSecretAuthBasic and ESSecretAuthAPI

* fix: update output type in TestKubernetesAgentHelm according to new values.schema.json
* Update elastic-agent-client dependency to 7.16.0

* Add support for APM sampling rate config

* fixup! Add support for APM sampling rate config

* Add unit tests for APM config triggering component changes

* make notice
* chore: Update from dockerfiles

Made with ❤️️ by updatecli

* chore: Update version.asciidoc

Made with ❤️️ by updatecli

* chore: Update .golangci.yml

Made with ❤️️ by updatecli

* chore: Update .go-version

Made with ❤️️ by updatecli

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pierre HILBERT <[email protected]>
…lastic#5547)

* Add ability to run the same tests with multi docker image variants.

* Fix buildkite.

* Add back distro for kubernetes types.

* Refactor how kubernetes tests are selected.

* Allow specific docker variant.

* Add kubernetes testing to the dev guide.

* Add back K8S_VERSION to buildkite job.

* Fix docker image location.

* Don't create so many clusters.

* Ensure basic is always picked first.

* Fix cloud variant image name.

* revert builkite changes

* Move kubernetes integration tests in CI next to other integration tests.

* Rename wolfi-complete to complete-wolfi.

* Reduce docker variants on some tests.

* Only test basic and wolfi.
…... (elastic#5589)

... olicies/ironbank/templates" Updatecli version policy

Made with ❤️️ by updatecli

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The documentation claims this should work, but it currently doesn't.
* add kubernetes v1.31 to test matrix

* add 1.31 to buildkite pipeline
…framework (elastic#5391)

Adds the ability to generate the integration testing workload as buildkite pipelines.

This doesn't actually complete the enablement. This just provides a great starting point for generating the buildkite steps. More work is still to be done to actually perform the work in the steps.
Adds the elastic-agent-service container with the py-connectors component enabled.
* Skip merge coverage reports step

* Add debugging echo

* Soft fail instead of skip

* Soft fail instead of skip
Bumps [github.com/elastic/elastic-agent-libs](https://github.com/elastic/elastic-agent-libs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/elastic/elastic-agent-libs/releases)
- [Commits](elastic/elastic-agent-libs@v0.10.1...v0.11.0)

---
updated-dependencies:
- dependency-name: github.com/elastic/elastic-agent-libs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
goja is not used in elastic agent

see go mod graph | grep groja
…elastic#5603)

* build(deps): bump github.com/elastic/go-sysinfo from 1.14.1 to 1.14.2

Bumps [github.com/elastic/go-sysinfo](https://github.com/elastic/go-sysinfo) from 1.14.1 to 1.14.2.
- [Release notes](https://github.com/elastic/go-sysinfo/releases)
- [Commits](elastic/go-sysinfo@v1.14.1...v1.14.2)

---
updated-dependencies:
- dependency-name: github.com/elastic/go-sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update NOTICE.txt

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…5536)

Also remove defaults for username and password set this way.
* Remove unused symbols

* Fix bodycloser linter errors

* Fix errcheck linter errors

* Fix staticcheck linter errors

* Use default configuration for nakedret linter

* Fix various linter errors
…hives (elastic#5647)

This commit modifies movePackagesToArchive() function to include python
wheel packages introduced with commit 701f8b9 .
This solves an issue when packaging docker images using a DROP_PATH env
var: prior to this change, the python wheel archives present in
DROP_PATH were not moved to `<DROP PATH>/archive/<platform>` failing to
create an image for which the Dockerfile expects to install the python
wheel package
blakerouse and others added 19 commits December 11, 2024 12:21
* Write integration test.

* Skip notifyFleetAuditUninstall on Windows.

* Re-enable notifyFleetAuditUninstall on Windows.

* Update test to be in Fleet group.

* Adjust OGC supported.

* Add changelog.

* Update changelog.

* Add check for connected to Fleet.

* Remove TestRepeatedInstallUninstall.
…2.3.2 (elastic#6271)

* build(deps): bump github.com/elastic/go-docappender/v2

Bumps [github.com/elastic/go-docappender/v2](https://github.com/elastic/go-docappender) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/elastic/go-docappender/releases)
- [Commits](elastic/go-docappender@v2.3.1...v2.3.2)

---
updated-dependencies:
- dependency-name: github.com/elastic/go-docappender/v2
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update NOTICE.txt

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…#6310)

Now the script can be run on a matrix for multiple release branches.
These files are used for picking the starting (pre-upgrade) or ending (post-upgrade) agent versions in upgrade integration tests.

The content is based on responses from https://www.elastic.co/api/product_versions and https://snapshots.elastic.co

The current update is generated based on the following requirements:

Package version: 9.0.0

```json
{
  "UpgradeToVersion": "9.0.0",
  "CurrentMajors": 1,
  "PreviousMajors": 1,
  "PreviousMinors": 2,
  "SnapshotBranches": [
    "8.x",
    "8.17",
    "7.17"
  ]
}
```

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Before we had to manually trigger the update version automation for
all branches other than `main`.
…lastic#6293)

* chore: update version and appVersion in the elastic-agent helm chart

* chore: render examples

* fix: correct agentVersion

* fix: mark elastic-agent helm chart with -beta suffix

* fix: update chart version and render examples
* feat: transition kubernetes integration tests to adapter pattern

* fix: count nodes without NoSchedule taints
* Fix audit/unenroll calls when agent runs fleet-server

* Fix integration test

* Increase output for failed test

* move notification attempt

* move notification attempt

* audit/uninstall will call localhost with certonly tls

* Fix linter

* remove tls setting

* Add PR link to changelog

* stop checking if we need to notify in an error is encountered

* Shorten non-fatal error messages to prevent output issues in terminals

* Don't load agentinfo, use ID value from config instead

* Add unprivilged tests to audit unenroll integration tests

* Add comments for setting hosts
…o 0.17.5 (elastic#6339)

* build(deps): bump github.com/elastic/elastic-agent-libs

Bumps [github.com/elastic/elastic-agent-libs](https://github.com/elastic/elastic-agent-libs) from 0.17.4 to 0.17.5.
- [Release notes](https://github.com/elastic/elastic-agent-libs/releases)
- [Commits](elastic/elastic-agent-libs@v0.17.4...v0.17.5)

---
updated-dependencies:
- dependency-name: github.com/elastic/elastic-agent-libs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update NOTICE.txt

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Michel Laterman <[email protected]>
…iles from 0.115.0 to 0.116.0 (elastic#6369)

* build(deps): bump go.opentelemetry.io/collector/pipeline/pipelineprofiles

Bumps [go.opentelemetry.io/collector/pipeline/pipelineprofiles](https://github.com/open-telemetry/opentelemetry-collector) from 0.115.0 to 0.116.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md)
- [Commits](open-telemetry/opentelemetry-collector@v0.115.0...v0.116.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/collector/pipeline/pipelineprofiles
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update NOTICE.txt

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…to 8.17.0 (elastic#6368)

* build(deps): bump github.com/elastic/go-elasticsearch/v8

Bumps [github.com/elastic/go-elasticsearch/v8](https://github.com/elastic/go-elasticsearch) from 8.16.0 to 8.17.0.
- [Release notes](https://github.com/elastic/go-elasticsearch/releases)
- [Changelog](https://github.com/elastic/go-elasticsearch/blob/main/CHANGELOG.md)
- [Commits](elastic/go-elasticsearch@v8.16.0...v8.17.0)

---
updated-dependencies:
- dependency-name: github.com/elastic/go-elasticsearch/v8
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update NOTICE.txt

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* fix(5163): added context to the verify interface, updated relevant calls, added backoff roundtripper

* fix(5163): added backoff roundtripper, added tests for asc file download

* fix(5163): remove log

* fix(5163): ran mage fmt

* fix(5163): removed unused roundtripper

* fix(5163): closing response body on unsuccesful responses

* fix(5163): added changelog

* fix(5163): added comment explaining the backoff function, added nolint comment

* fix(5163): added logic to handle resetting request body in the retry roundtripper, added unit tests, updated comments

* fix(5163): fixed linting errors

* fix(5164): ran mage check
* feat: Prevent leaking secrets when logging component model

* add changelog entry

* test: add test checking that secrets are not leaked when components are marhsaled to JSON
These files are used for picking the starting (pre-upgrade) or ending (post-upgrade) agent versions in upgrade integration tests.

The content is based on responses from https://www.elastic.co/api/product_versions and https://snapshots.elastic.co

The current update is generated based on the following requirements:

Package version: 9.0.0

```json
{
  "UpgradeToVersion": "9.0.0",
  "CurrentMajors": 1,
  "PreviousMajors": 1,
  "PreviousMinors": 2,
  "SnapshotBranches": [
    "8.x",
    "8.17",
    "7.17"
  ]
}
```

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat: add kubernetes hints test

* feat: utilise labelselector for checking redis pods

* feat: check redis pods for correct hint annotations
According to the support matrix, ECE versions up to 3.6 are marked as
compatible with Docker 19.03 and the Wolfi images require Docker version of 20.10.10.
@rdner rdner added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team skip-changelog labels Dec 18, 2024
@rdner rdner self-assigned this Dec 18, 2024
Copy link
Contributor

mergify bot commented Dec 18, 2024

This pull request does not have a backport label. Could you fix it @rdner? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

Copy link
Contributor

mergify bot commented Dec 18, 2024

backport-v8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

@mergify mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Dec 18, 2024
@rdner rdner changed the base branch from main to 8.17 December 18, 2024 17:54
@rdner rdner closed this Dec 18, 2024
@rdner rdner deleted the revert-cloud-ubuntu branch December 18, 2024 17:54
@rdner
Copy link
Member Author

rdner commented Dec 18, 2024

Sorry for the wrong pings, closing this in favor of #6393 sine the base branch should be 8.17.

Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.x Automated backport to the 8.x branch with mergify skip-changelog Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.