[helm] fleet mode fixes #6345

pkoutsovasilis · 2024-12-16T19:02:21Z

What does this PR do?

This PR introduces several fixes to the Elastic Agent Helm chart regarding Fleet Mode deployments:

Fleet mode now includes the necessary Kubernetes changes (e.g., volume mounts, resource requests) for enabled integrations.
Leader election is enabled by default for Fleet mode unless explicitly disabled by the user in the agent preset.
Kubernetes integration is now enabled by default.
Sets hostNetwork=true for perNode (Daemonset) agent preset, used by kubernetes and system integrations

Why is it important?

These changes are critical as they address bugs 🙂

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
~~[ ] I have added an entry in ./changelog/fragments using the changelog tool~~
~~[ ] I have added an integration test or an E2E test~~

Disruptive User Impact

N/A

How to test this PR locally

You can see in the ./deploy/helm/examples/fleet-managed/rendered/manifest.yaml that now fleet-managed agent has the appropriate volume mounts and leader election enabled.
Follow this example and you should be able to see container logs correctly

Related issues

pkoutsovasilis · 2024-12-18T17:42:05Z

/test

…egrations

… by user

pkoutsovasilis · 2024-12-18T19:48:35Z

buildkite test this

elasticmachine · 2024-12-19T13:36:43Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

pkoutsovasilis · 2024-12-19T13:37:12Z

cc @eedugon in case you wanna give this PR a go before it gets merged 🙂

blakerouse

lgtm!

eedugon · 2024-12-19T15:38:02Z

I'll give it a try!

mergify · 2024-12-19T16:13:40Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b helm/fix_fleet_k8s_mounts upstream/helm/fix_fleet_k8s_mounts
git merge upstream/main
git push upstream helm/fix_fleet_k8s_mounts

…unts # Conflicts: # deploy/helm/elastic-agent/examples/kubernetes-hints-autodiscover/rendered/manifest.yaml # deploy/helm/elastic-agent/examples/multiple-integrations/rendered/manifest.yaml # deploy/helm/elastic-agent/templates/integrations/_kubernetes/_preset_pernode.tpl

elastic-sonarqube · 2024-12-19T17:23:39Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

eedugon · 2024-12-20T08:17:33Z

@pkoutsovasilis : I've tried to test the PR. It looks promising and I think everything is fixed, however I've found a problem.

From your branch I have run the following:

helm upgrade --install demo5 ./deploy/helm/elastic-agent \
--set agent.fleet.enabled=true \
--set agent.fleet.url=https://fleet-svc.default.svc:443 \
--set agent.fleet.token=MVpCSXRwTUJEWE90bU9YWVh6dmg6ekhDV0lna3FRaUM2SUhTRGg2UUpNZw== \
--set agent.fleet.preset=perNode \
--set agent.version=8.16.2 \
--set agent.fleet.insecure=true

Note: I have to set agent.version=8.16.2 because my stack is in that version.
The manifest and installed agent runs 9.x.x:

        image: docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT

So it looks the parameter agent.version is ignored.

Besides that, the generated manifest looks OK, and I would expect it to solve the issues of the PR:

HostPath volume mounts for logs look ok.
HostNetwork looks ok
Leader election looks ok.

Thanks for this!

eedugon

LGTM, check out the comment about agent.version just in case (not related with this PR though).

pkoutsovasilis · 2024-12-20T09:32:28Z

LGTM, check out the comment about agent.version just in case (not related with this PR though).

thanks for the testing @eedugon , the value to change the tag of the agent image is agent.image.tag; agent.version was used when the user enabled hints, to find and download the correct released archive. That said, since these got embedded in the agent container image maybe it is time to deprecate it 🙂

* fix: allow fleet mode to do the necessary k8s changes for enabled integrations * fix: make kubernetes integration enabled by default * fix: enable leader election for fleet mode unless explicitly disabled by user * fix: enable hostNetwork for perNode preset (cherry picked from commit 0d94ead) # Conflicts: # deploy/helm/elastic-agent/examples/kubernetes-hints-autodiscover/rendered/manifest.yaml # deploy/helm/elastic-agent/examples/multiple-integrations/rendered/manifest.yaml # deploy/helm/elastic-agent/templates/integrations/_kubernetes/_preset_pernode.tpl

* fix: allow fleet mode to do the necessary k8s changes for enabled integrations * fix: make kubernetes integration enabled by default * fix: enable leader election for fleet mode unless explicitly disabled by user * fix: enable hostNetwork for perNode preset (cherry picked from commit 0d94ead)

* fix: allow fleet mode to do the necessary k8s changes for enabled integrations * fix: make kubernetes integration enabled by default * fix: enable leader election for fleet mode unless explicitly disabled by user * fix: enable hostNetwork for perNode preset (cherry picked from commit 0d94ead) # Conflicts: # deploy/helm/elastic-agent/examples/kubernetes-hints-autodiscover/rendered/manifest.yaml # deploy/helm/elastic-agent/examples/multiple-integrations/rendered/manifest.yaml # deploy/helm/elastic-agent/templates/integrations/_kubernetes/_preset_pernode.tpl

* fix: allow fleet mode to do the necessary k8s changes for enabled integrations * fix: make kubernetes integration enabled by default * fix: enable leader election for fleet mode unless explicitly disabled by user * fix: enable hostNetwork for perNode preset (cherry picked from commit 0d94ead) Co-authored-by: Panos Koutsovasilis <[email protected]>

* [helm] fleet mode fixes (#6345) * fix: allow fleet mode to do the necessary k8s changes for enabled integrations * fix: make kubernetes integration enabled by default * fix: enable leader election for fleet mode unless explicitly disabled by user * fix: enable hostNetwork for perNode preset (cherry picked from commit 0d94ead) # Conflicts: # deploy/helm/elastic-agent/examples/kubernetes-hints-autodiscover/rendered/manifest.yaml # deploy/helm/elastic-agent/examples/multiple-integrations/rendered/manifest.yaml # deploy/helm/elastic-agent/templates/integrations/_kubernetes/_preset_pernode.tpl * fix: resolve conflicts --------- Co-authored-by: Panos Koutsovasilis <[email protected]>

pkoutsovasilis added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team skip-changelog backport-8.x Automated backport to the 8.x branch with mergify backport-8.17 Automated backport with mergify labels Dec 16, 2024

pkoutsovasilis self-assigned this Dec 16, 2024

pkoutsovasilis force-pushed the helm/fix_fleet_k8s_mounts branch from 81e13d6 to 53b7cc1 Compare December 18, 2024 16:21

pkoutsovasilis added 4 commits December 18, 2024 21:33

fix: allow fleet mode to do the necessary k8s changes for enabled int…

8cf2732

…egrations

fix: make kubernetes integration enabled by default

77f54e7

fix: enable leader election for fleet mode unless explicitly disabled…

4203638

… by user

fix: enable hostNetwork for perNode preset

a3ad6c7

pkoutsovasilis force-pushed the helm/fix_fleet_k8s_mounts branch from 53b7cc1 to a3ad6c7 Compare December 18, 2024 19:33

pkoutsovasilis marked this pull request as ready for review December 19, 2024 13:36

pkoutsovasilis requested a review from a team as a code owner December 19, 2024 13:36

pkoutsovasilis requested review from michalpristas and andrzej-stencel December 19, 2024 13:36

blakerouse approved these changes Dec 19, 2024

View reviewed changes

pkoutsovasilis added the backport-8.16 Automated backport with mergify label Dec 19, 2024

eedugon self-requested a review December 20, 2024 08:19

eedugon approved these changes Dec 20, 2024

View reviewed changes

pkoutsovasilis merged commit 0d94ead into elastic:main Dec 23, 2024
9 checks passed

pkoutsovasilis deleted the helm/fix_fleet_k8s_mounts branch December 23, 2024 08:52

mergify bot mentioned this pull request Dec 23, 2024

[8.16](backport #6345) [helm] fleet mode fixes #6424

Merged

5 tasks

This was referenced Dec 23, 2024

[8.x](backport #6345) [helm] fleet mode fixes #6425

Merged

[8.17](backport #6345) [helm] fleet mode fixes #6426

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[helm] fleet mode fixes #6345

[helm] fleet mode fixes #6345

pkoutsovasilis commented Dec 16, 2024 •

edited

Loading

pkoutsovasilis commented Dec 18, 2024

pkoutsovasilis commented Dec 18, 2024

elasticmachine commented Dec 19, 2024

pkoutsovasilis commented Dec 19, 2024

blakerouse left a comment

eedugon commented Dec 19, 2024

mergify bot commented Dec 19, 2024

elastic-sonarqube bot commented Dec 19, 2024

eedugon commented Dec 20, 2024

eedugon left a comment

pkoutsovasilis commented Dec 20, 2024 •

edited

Loading

[helm] fleet mode fixes #6345

[helm] fleet mode fixes #6345

Conversation

pkoutsovasilis commented Dec 16, 2024 • edited Loading

What does this PR do?

Why is it important?

Checklist

Disruptive User Impact

How to test this PR locally

Related issues

pkoutsovasilis commented Dec 18, 2024

pkoutsovasilis commented Dec 18, 2024

elasticmachine commented Dec 19, 2024

pkoutsovasilis commented Dec 19, 2024

blakerouse left a comment

Choose a reason for hiding this comment

eedugon commented Dec 19, 2024

mergify bot commented Dec 19, 2024

elastic-sonarqube bot commented Dec 19, 2024

Quality Gate passed

eedugon commented Dec 20, 2024

eedugon left a comment

Choose a reason for hiding this comment

pkoutsovasilis commented Dec 20, 2024 • edited Loading

pkoutsovasilis commented Dec 16, 2024 •

edited

Loading

pkoutsovasilis commented Dec 20, 2024 •

edited

Loading