[v0.10] [SURE-8309] fleet-agent in rke2 cluster repeatedly deploying managed-system-agent bundle

[weyfonk]

Backport of #2856 to v0.10.0.

[weyfonk]

Additional QA

Problem

Adding an RKE2 cluster to a Fleet setup would lead to multiple log messages about the Fleet agent bundle being deployed, although that deployment would actually happen only once.

Solution

Moved the log message to ensure that it is only produced when a deployment actually happened, not when it was skipped due to the bundle and release already being deployed.

Testing

Engineering Testing

Manual Testing

Observed a single Deployed bundle log for a Fleet agent bundle, vs multiple identical logs before the fix.
Tests were run against a local k3s cluster, as this issue is expected to be independent from the underlying K8s distribution.

Automated Testing

N/A (tests do not cover logging)

QA Testing Considerations

We should validate that this fix works on RKE2 as well (single Deployed bundle log for Fleet agent deployment).

⚠️ The log message appears as Deploying bundle in the issue description, because it involves an older Fleet version. From Fleet 0.10 onwards, that message is Deployed bundle, as it is produced right after a deployment, not before starting it.

Regressions Considerations

N/A

[sbulage]

System Information

Before Upgrade

Rancher Version	Fleet Version
Prime:2.9.2	Fleet: 0.10.2

After Upgrade

Rancher Version	Fleet Version
v2.9-af2463ff43418024ff86cfd9304836b7074a924a-head	fleet-104.1.0+up0.10.4-rc.3

---

Before Upgrade

• Deploy a Rancher Prime latest released version (I installed prime/2.9.2)
• Import downstream cluster. I used k3d and k3s
• Observed the logs of the fleet-agent installed in the downstream cluster.

---

In the fleet-agent pod logs you'll see the repeated messages related to Deployed bundle

Note: I filtered with the string Deployed bundle to see log repetition behavior.

Logs:

Fleet agent logs before upgrade

{"level":"info","ts":"2024-10-10T13:02:18Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"7c95e429-7533-433e-bb74-c30833cff1be","deploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"","release":"cattle-fleet-system/fleet-agent-imported-0:1","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}
{"level":"info","ts":"2024-10-10T13:02:18Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"afcdcee8-1c87-4b87-b8e1-e19cf029e8c1","deploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","release":"cattle-fleet-system/fleet-agent-imported-0:1","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}
{"level":"info","ts":"2024-10-10T13:02:19Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"d2ae448a-bada-49f8-b382-b4abc75e3df2","deploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","release":"cattle-fleet-system/fleet-agent-imported-0:1","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}
{"level":"info","ts":"2024-10-10T13:02:19Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"dc332e34-f685-4a14-8dff-e9f1b00c2aa8","deploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","release":"cattle-fleet-system/fleet-agent-imported-0:1","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}
{"level":"info","ts":"2024-10-10T13:02:23Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"f728bb68-eb17-4058-85c2-618ad6546142","deploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","release":"cattle-fleet-system/fleet-agent-imported-0:1","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}
{"level":"info","ts":"2024-10-10T13:02:24Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"3e90c4d7-4965-4120-bb30-7dfb269d9353","deploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","release":"cattle-fleet-system/fleet-agent-imported-0:1","appliedDeploymentID":"s-898541053ef22777eeb082aa2288c4af9fbc8dd8a8ed5604e17404d340156:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}

---

After Upgrade

• Upgraded same cluster to 2.9-head which installs Fleet version: 0.10.4-rc.3
• Observe the fleet-agent pod logs on the imported clusters.

In the fleet-agent pod logs you'll see the only single entry for Deployed bundle

Note: I filtered with the string Deployed bundle to see log repetition behavior.

---

Logs:

Fleet agent logs After upgrade

{"level":"info","ts":"2024-10-10T14:54:03Z","logger":"bundledeployment.DeployBundle","msg":"Deployed bundle","controller":"bundledeployment","controllerGroup":"fleet.cattle.io","controllerKind":"BundleDeployment","BundleDeployment":{"name":"fleet-agent-imported-0","namespace":"cluster-fleet-default-imported-0-f1158df57e01"},"namespace":"cluster-fleet-default-imported-0-f1158df57e01","name":"fleet-agent-imported-0","reconcileID":"ab927aa1-e10e-4634-8a72-d70608954b97","deploymentID":"s-3f93a063ea103800d1695529d247a6fd229b7d2150e653ad20095a81018a8:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","appliedDeploymentID":"s-2dbe734b124db9ecd44d99019b99fd47170b4ddab7bf8639cc202a94b11e8:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21","release":"cattle-fleet-system/fleet-agent-imported-0:4","DeploymentID":"s-3f93a063ea103800d1695529d247a6fd229b7d2150e653ad20095a81018a8:8eaf3c183fe289136e7a2da3ceafa5c435e8f098e386324237baeb8811cf8b21"}