Handle shoot-oidc-service extension when provisioning kyma [KIM/feature]

[kwiatekus]

Description

Provisioner should always enable shoot-oidc-service feature flag when provisioning kyma runtime
When additional OIDC issuer is requested, provisioner should create this OIDC resource in the provisioned shoot cluster
https://community.sap.com/t5/additional-blogs-by-sap/using-github-actions-openid-connect-in-kubernetes/ba-p/13542513
and configure cluster admin role binding for the subject of the token issued by the additional issuer.

AC

• OIDC is part of the technical contract for KIM (= part of Runtime-CR)
• Following two fields are required in the final Runtime-CR, but for KEB is is not mandatory to set them initially. If one of these field is missing or empty, a default values will be set by KIM (default values are static values in the KIM configuration):
  • oidcConfig (default value is Kyma-OIDC provider - it will be only set if no configuration is provided by KEB)
  • additionalOidcConfigs (is a list, if list is empty KIM has to set one default entry to the list)

Reasons

It is required for the kyma-project/kyma#18305

Attachments
kyma-project/kyma#18519 (comment)
https://github.com/kyma-project/control-plane/tree/main/components/provisioner

Sample resource:

apiVersion: authentication.gardener.cloud/v1alpha1
kind: OpenIDConnect
metadata:
  name: actions-oidc
spec:
  issuerURL: https://token.actions.githubusercontent.com
  clientID: my-kubernetes-cluster
  usernameClaim: sub
  usernamePrefix: "actions-oidc:"
  requiredClaims:
    repository: myOrg/myRepo
    workflow: deploy-kubernetes
    ref: refs/heads/main

[tobiscr]

PR from @kyma-project/otters for provisioner: kyma-project/control-plane#3358

Docs from Gardener: https://github.com/gardener/gardener-extension-shoot-oidc-service/blob/master/docs/usage/openidconnects.md

[tobiscr]

We have it on our TODO list, but right now - based on the current ranked backlog it will be considered as part of our deliverables in Q2/24.

[tobiscr]

We agreed on following implementation on KIM side:

• KIM will store the default values for some fields in our RuntimeCR (e.g. oidcConfig)
• We will have a few fields in the RuntimeCR , which won't be necessarily provided by KEB but have to be filled out with default values by KIM (<< having them in the RuntimeCR included is helpful for later operational actions applied by SRE - they will sometimes have to adjust these values when major cluster-upgrades are applied)
  • oidcConfig: if missing, KIM will set the operator-OIDC configuration (operator=on-call engineer) as default
  • additionalOidcConfigs (is a list): if the list is empty, KIM will add the first entry to be the Kyma OIDC configuration
• It's KIMs responsibility, to check for each creation/update of a RuntimeCR instance, that missing fields will be added to this CR with their default values (could be done by an webhook executed before KIM starts processing the values etc.)

See also kyma-project/kyma#18305 (comment)

[tobiscr]

First implementation is ready for OIDC extender: https://github.com/kyma-project/infrastructure-manager/blob/main/internal/gardener/shoot/extender/oidc.go

[tobiscr]

To unblock customers who are waiting for the enablement of the OIDC extension in GArdener, we updated the Provisioner last week to activate this extension per default for all new created clusters. So, any new created SKR cluster will have the OIDC extension now enabled and customers can configure their own OIDC provider by creating the corresponding CR in their SKR clusters.

It is planned to start the replacement of the Provisioner with KIM (Kyma Infrasturcture Manager) by end of this month. KIM will also per default enable the OIDC extension for all managed clusters.

[Disper]

Technical workplan

• oidc extension enabled
• plug it in to the state machine without actually doing anything
• update condition checking to go into this state only if necessary (extension enabled )
• If user provides no OIDC config KIM defaults the additionalOidcConfigs[0] with the default User-facing OIDC config (should be the current kyma OIDC config)
  • OIDC config is properly defaulted
  • additionalOIDC config is properly defaulted
  • KIM defines Operator-facing OIDC explicitely in shoot CR
  • defaults configured in all landscapes
• double check CRBs if there is no bug that there are set not in the right context (wrong client)
• create OpenID resource on the shoot
• create OpenID resource for each additionalOidc config
• openID resource created using non-fake values
• clarify creation of OpenIDConnect CR without verification if they were indeed created on the shoot (fire and forget) - it's fine, follow-up created - OpenIDConnect CR are created without proper verification if operation succeeded #410
• double check if the right CRBs are applied correctly and also with the right group (confirm with KK/SRE)
• clarify ensure prefixes in Operator-facing OIDC (for username and groups) part (confirm with KK/SRE)
• manual verification of full OIDC e2e flow (what could be the scenario?)
• adjust sequence tests
• adjust unit tests
• oidc configuration should be updated accordingly when user updates it configuration (changes in runtime CR)
  • add logic that removes existing "additionalOidcs" from the Shoot spec (in case they're changed/removed by the user)
  • recreate "additionalOidcs" in the Shoot spec (in case they're changed/removed by the user)
• golangci-lint
• fix trivy (github.com/gardener/[email protected] gopkg.in/square/[email protected])
• consider merging with CRB state
• move before CRB state
• prepare config changes for DEV/STAGE/PROD/internal_doc_example
• the condition set should reflect team's internal conditions strategy - created follow-up for that - Strategy for KIM's statuses&condition handling #411
  • proper error handling in case of a failure