Calico static routes not updated when nodes added/removed from cluster

[mikeoleary]

Setup Details

CIS Version : 2.18.1
Build: f5networks/k8s-bigip-ctlr:latest
Agent Mode: AS3
Orchestration: K8S
Pool Mode: Cluster
Additional Setup details: Customer has Calico CNI

Description

Submitting on behalf of customer, pls reach out if cust details are required. Customer reports that not all Calico routes show up on BIG-IP when using static routing mode. Customer reports that CIS is not updating routes when Calico issues a new /26 CIDR block to a node.

Customer reports this behavior with CIS 2.18.1

Customer is aware of how to correctly deploy CIS with static-routing-mode and is using --static-routing-mode=true and --orchestration-cni=calico-k8s

Customer reports Calico version as follows:

Here is the Calico Enterprise version info:
Cluster Calico Version: v3.26.0 ß Opensource version
Cluster Calico Enterprise Version: v3.18.4 ß Calico Enterprise version

Steps To Reproduce

1. Deploy CIS 2.18.1 with a K8s cluster running Calico CNI, and use --static-routing-mode=true and --orchestration-cni=calico-k8s
2. See that multiple static routes are created. In this case, a /26 CIDR block for each node.
3. Add or remove nodes from the cluster. Calico will assign new CIDR blocks for new nodes and remove/reclaim blocks from deleted nodes.
4. CIS does not update the static routes on BIG-IP for added/removed nodes.

Expected Result

CIS should update the static routes on BIG-IP for added/removed nodes.

Actual Result

CIS does not update the static routes as nodes are added or removed.

Diagnostic Information

<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

Observations (if any)

[mikeoleary]

Update from customer:

It may not be when nodes are added to the cluster, but when Calico assigns an additional /26 CIDR block to an existing node when the pod count requires more IP addresses. Calico will also reclaim that /26 pool once the Pod count goes down and the pool is cleared out.

[trinaths]

Created [CONTCNTR-5009] for internal tracking.

[vklohiya]

@mikeoleary , Did customer updated the required RBAC needed for Calilco CNI as per the document

[mikeoleary]

@vklohiya yes they did according to customer. This does not appear to be an RBAC issue.

[jmorello-mdsol]

I can confirm. Yes, we updated the required RBAC for the Calico CNI.

[mikeoleary]

To keep the status of this github issue up to date, we've been told to expect an update in cis v2.20 to fix this. I can revisit this issue after 2.20 is released.

[lavanya-f5]

@mikeoleary @jmorello-mdsol dev build for this fix quay.io/f5networks/k8s-bigip-ctlr-devel:974b12e6735b39b63929e772659f57efbf007d1c.