Allow kubernetes_service to serve only dynamic kube cluster resources

[programmerq]

Expected Behavior

Helm chart for teleport-kube-agent should support service dynamic clusters only.

Teleport itself supports this already. A valid kuberenetes_service section doesn't need kube_cluster_name or kubeconfig_file if it has a labels section. The chart should be able to produce a teleport.yaml with labels only, if the user has that use case.

Current Behavior

The kube_cluster_name is always set in the teleport.yaml generated by the chart. In a case where a user has both discovery_service and kubernetes_service, the discovery service can incorporate EKS tags into the labels of the dynamic cluster. If the dynamic object has the same name as the kube_cluster_name, then the dynamic object will be skipped/ignored from the user's perspective.

Use Case

For EKS auto-discovery, this can be confusing for the end-user:

actual output:

% tsh kube ls
Kube Cluster Name  Labels                                                                                                                                                                                                                                                                                    Selected
------------------ -------------
dev-eks-1          TeleportKubernetesName=dev-eks-1,account-id=000000000000,env=dev,region=eu-west-1,service=eks,terraform=true
dev-eks-2          env=dev
dev-eks-3          TeleportKubernetesName=dev-eks-3,account-id=000000000000,env=dev,region=us-east-2,terraform-aws-modules=eks,terraform=true

expected:

% tsh kube ls
Kube Cluster Name  Labels                                                                                                                                                                                                                                                                                    Selected
------------------ -------------
dev-eks-1          TeleportKubernetesName=dev-eks-1,account-id=000000000000,env=dev,region=eu-west-1,service=eks,terraform=true
dev-eks-2          TeleportKubernetesName=dev-eks-2,account-id=000000000000,env=dev,region=us-west-2,terraform-aws-modules=eks,service=eks,terraform=true
dev-eks-3          TeleportKubernetesName=dev-eks-3,account-id=000000000000,env=dev,region=us-east-2,terraform-aws-modules=eks,terraform=true

In this example, the dev-eks-2 is running the discovery+kube agent. It has kubeClusterName: dev-eks-2. The dynamic object for dev-eks-2 isn't loaded. If the dynamic dev-eks-2 object were loaded, it would show the AWS tag-derived labels.

Workarounds

Instead of setting kubeClusterName: dev-eks-2 in the helm values, an alternate name could be specified. Teleport RBAC could deny users from seeing the alternate name, allowing the discovered object to come through.

Manually edit the generated ConfigMap to remove kube_cluster_name from teleport.yaml. The MustMergeOverwrite helm function that the teleport-kube-agent chart uses to specify teleport.yaml overrides cannot remove the key from kubernetes_service.

[tigrato]

I think you can use:

kubeClusterName: somethingElse

teleportConfig: 
  kubernetes_service:
    kube_cluster_name: "" <- this will reset the value but still satisfies the chart requirements