[helm] fleet mode fixes

[pkoutsovasilis]

What does this PR do?

This PR introduces several fixes to the Elastic Agent Helm chart regarding Fleet Mode deployments:

• Fleet mode now includes the necessary Kubernetes changes (e.g., volume mounts, resource requests) for enabled integrations.
• Leader election is enabled by default for Fleet mode unless explicitly disabled by the user in the agent preset.
• Kubernetes integration is now enabled by default.
• Sets hostNetwork=true for perNode (Daemonset) agent preset, used by kubernetes and system integrations

Why is it important?

These changes are critical as they address bugs 🙂

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• [ ] I have added an entry in ./changelog/fragments using the changelog tool
• [ ] I have added an integration test or an E2E test

Disruptive User Impact

N/A

How to test this PR locally

• You can see in the ./deploy/helm/examples/fleet-managed/rendered/manifest.yaml that now fleet-managed agent has the appropriate volume mounts and leader election enabled.
• Follow this example and you should be able to see container logs correctly

Related issues

• Closes [Helm] Enable leader election by default for Fleet Managed agent installation with perNode preset #6284
• Closes [Helm] Chart does not mount /var/log/containers/ on Elastic-Agent pod, thus no logs are ingested #6204
• Closes [Helm] Missing hostNetwork: true for managed agent in perNode preset #6324

[pkoutsovasilis]

/test

[pkoutsovasilis]

buildkite test this

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[pkoutsovasilis]

cc @eedugon in case you wanna give this PR a go before it gets merged 🙂

[blakerouse]

lgtm!

[eedugon]

I'll give it a try!

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b helm/fix_fleet_k8s_mounts upstream/helm/fix_fleet_k8s_mounts
git merge upstream/main
git push upstream helm/fix_fleet_k8s_mounts

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[eedugon]

@pkoutsovasilis : I've tried to test the PR. It looks promising and I think everything is fixed, however I've found a problem.

From your branch I have run the following:

helm upgrade --install demo5 ./deploy/helm/elastic-agent \
--set agent.fleet.enabled=true \
--set agent.fleet.url=https://fleet-svc.default.svc:443 \
--set agent.fleet.token=MVpCSXRwTUJEWE90bU9YWVh6dmg6ekhDV0lna3FRaUM2SUhTRGg2UUpNZw== \
--set agent.fleet.preset=perNode \
--set agent.version=8.16.2 \
--set agent.fleet.insecure=true

Note: I have to set agent.version=8.16.2 because my stack is in that version.
The manifest and installed agent runs 9.x.x:

        image: docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT

So it looks the parameter agent.version is ignored.

Besides that, the generated manifest looks OK, and I would expect it to solve the issues of the PR:

• HostPath volume mounts for logs look ok.
• HostNetwork looks ok
• Leader election looks ok.

Thanks for this!

[eedugon]

LGTM, check out the comment about agent.version just in case (not related with this PR though).

[pkoutsovasilis]

LGTM, check out the comment about agent.version just in case (not related with this PR though).

thanks for the testing @eedugon , the value to change the tag of the agent image is agent.image.tag; agent.version was used when the user enabled hints, to find and download the correct released archive. That said, since these got embedded in the agent container image maybe it is time to deprecate it 🙂