From 20c5a75efd323ea4f9fd7e4c1e8d09de47f33d1f Mon Sep 17 00:00:00 2001 From: Poornima Krishnasamy Date: Thu, 14 Dec 2023 18:08:02 +0000 Subject: [PATCH] Add steps to re-run the account pipeline and integration tests --- runbooks/source/node-group-changes.html.md.erb | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/runbooks/source/node-group-changes.html.md.erb b/runbooks/source/node-group-changes.html.md.erb index 95434f3f..93e371fe 100644 --- a/runbooks/source/node-group-changes.html.md.erb +++ b/runbooks/source/node-group-changes.html.md.erb @@ -16,11 +16,17 @@ You may need to make a change to an EKS [cluster node group] or [instance type c To avoid bringing down all the nodes at once is to follow these steps: 1. add a new node group with your [updated changes] -1. lookup the old node group name (you can find this in the aws gui) -1. once merged in you can drain the old node group using the command below: -1. raise a new [pr deleting] the old node group +2. re-run the [infrastructure-account/terraform-apply] pipeline to update the Modsecurity Audit logs cluster to map roles to both old and new node group IAM Role + This is to avoid losing modsec audit logs from the new node group +3. lookup the old node group name (you can find this in the aws gui) +4. once merged in you can drain the old node group using the command below: + + > cloud-platform pipeline cordon-and-drain --cluster-name --node-group + +5. raise a new [pr deleting] the old node group +6. re-run the [infrastructure-account/terraform-apply] pipeline to again to update the Modsecurity Audit logs cluster to map roles with only the new node group IAM Role +7. run the integration tests to ensure the cluster is healthy -> cloud-platform pipeline cordon-and-drain --cluster-name --node-group [script source] because this command runs remotely in concourse you can't use this command to drain default ng on the manager cluster. @@ -36,3 +42,4 @@ To avoid bringing down all the nodes at once is to follow these steps: [updated changes]: https://github.com/ministryofjustice/cloud-platform-infrastructure/pull/2657 [cordons and drains nodes]: https://github.com/ministryofjustice/cloud-platform-terraform-concourse/blob/main/pipelines/manager/main/cordon-and-drain-nodes.yaml [script source]: https://github.com/ministryofjustice/cloud-platform-terraform-concourse/blob/7851f741e6c180ed868a97d51cec0cf1e109de8d/pipelines/manager/main/cordon-and-drain-nodes.yaml#L50 +[infrastructure-account/terraform-apply]: https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/infrastructure-account/jobs/terraform-apply