Addons: AWS Pod Identity Webhook doesn't support sidecar containers.

[eliasscosta]

/kind bug

1. What kops version are you running? The command kops version, will display
this information.

Client version: 1.29.0 (git-v1.29.0)

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.

Server Version: v1.29.4

3. What cloud provider are you using?

AWS

4. What commands did you run? What is the simplest way to reproduce this issue?

When we create a pod with sidecar containers, the mutate webhook from amazon-eks-pod-identity-webhook removes the restartPolicy=Always from our initContainer.

Ref:

• https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/

5. What happened after the commands executed?

Throw an error making it impossible to use the sidecar containers.

Pod "wp-01j6vwf91vw2rmhqmdrtyk76zv" is invalid: spec.initContainers[0].readinessProbe: Forbidden: may not be set for init containers without restartPolicy=Always

6. What did you expect to happen?

Expect the mutate don't remove the field from initContainers

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

Not applicable, the addon is a very old version.

9. Anything else do we need to know?

The fix release on version 0.5.3 of the Pod Webhook Identity

• Update k8s.io libraries to v0.29.1 to support sidecar containers aws/amazon-eks-pod-identity-webhook#209
• https://github.com/aws/amazon-eks-pod-identity-webhook/releases/tag/v0.5.3