Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Service Account Issuer Migration doc #16711

Merged
merged 1 commit into from
Oct 15, 2024
Merged

Update Service Account Issuer Migration doc #16711

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 39 additions & 9 deletions docs/operations/service_account_issuer_migration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,46 @@ As noted in the Kubernetes Docs when the `--service-account-issuer` flag is spec

So with this feature we can migrate to a new Service Account Issuer without disruption to cluster operations.

**Note**: There is official kOps support for this in the forthcoming feature - [koordinates/kops#16497](https://github.com/kubernetes/kops/pull/16497).
**Note**: These procedures can be adapted to enable IAM Roles for Service Accounts (IRSA) without disruption to cluster operations.

## Migrate using Instancegroup Hooks (prior kOps v1.28+)
## Migrate using kOps `additionalServiceAccountIssuers`

In [kubernetes/kops#16497](https://github.com/kubernetes/kops/pull/16497) kOps added support for specifying an additional Service Account Issuer. This was released in kOps v1.28.5 so it requires at least this version to follow this procedure. If you are using an earlier version you can follow the [Migrate using Instancegroup Hooks](#migrate-using-instancegroup-hooks) procedure.

**Warning**: This procedure is manual. We recommend testing this on a staging cluster before proceeding on a production cluster.

In this example we are switching from `master.[cluster-name].[domain]` to `api.internal.[cluster-name].[domain]`.

1. Add new SAI as additional (existing SAI as primary) to the Cluster resource
```yaml
kubeAPIServer:
serviceAccountIssuer: https://master.[cluster-name].[domain]
additionalServiceAccountIssuers:
- https://api.internal.[cluster-name].[domain]
```
2. Apply the changes to the cluster
3. Roll the control-plane nodes
4. Switch the primary/secondary SAI on the Cluster resource
```yaml
kubeAPIServer:
serviceAccountIssuer: https://api.internal.[cluster-name].[domain]
additionalServiceAccountIssuers:
- https://master.[cluster-name].[domain]
```
5. Apply the changes to the cluster
6. Roll the control-plane nodes
7. Wait 24 hours until the dynamic SA tokens have refreshed
8. Remove the `additionalServiceAccountIssuers` from the Cluster resource
9. Apply the changes to the cluster
10. Roll the control-plane nodes

## Migrate using Instancegroup Hooks

**Warning**: This procedure is manual and involves some tricky modification of manifest files. We recommend testing this on a staging cluster before proceeding on a production cluster.

In this example we are switching from `master.[cluster-name].[domain]` to `api.internal.[cluster-name].[domain]`.

1. Add the `modify-kube-api-manifest` (existing SAI as primary) hook to the control-plane instancegroups
1. Add the `modify-kube-api-manifest` (existing SAI as primary) hook to the control-plane InstanceGroup resources
```yaml
hooks:
- name: modify-kube-api-manifest
Expand All @@ -27,7 +58,7 @@ In this example we are switching from `master.[cluster-name].[domain]` to `api.i
```
2. Apply the changes to the cluster
3. Roll the control-plane nodes
4. Update the `modify-kube-api-manifest` (switch the primary/secondary SAI) hook on the control-plane instancegroups
4. Update the `modify-kube-api-manifest` (switch the primary/secondary SAI) hook on the control-plane InstanceGroup resources
```yaml
hooks:
- name: modify-kube-api-manifest
Expand All @@ -40,10 +71,9 @@ In this example we are switching from `master.[cluster-name].[domain]` to `api.i
```
5. Apply the changes to the cluster
6. Roll the control-plane nodes
7. Roll all other nodes in the cluster
8. Wait 24 hours until the dynamic SA tokens have refreshed
9. Remove the `modify-kube-api-manifest` hook on the control-plane instancegroups
10. Apply the changes to the cluster
11. Roll the control-plane nodes
7. Wait 24 hours until the dynamic SA tokens have refreshed
8. Remove the `modify-kube-api-manifest` hook on the control-plane InstanceGroup resources
9. Apply the changes to the cluster
10. Roll the control-plane nodes

This procedure was originally posted in a GitHub issue [here](https://github.com/kubernetes/kops/issues/16488#issuecomment-2084325891) with inspiration from [this comment](https://github.com/kubernetes/kops/issues/14201#issuecomment-1732035655).
Loading