Split Infrastructure Components

[sj-williams]

Background

With the introduction of Gatekeeper pod mutations for support of PSP removal in k8s 1.25, we have reached the conclusion it is time for us to split our terraform infrastructure components into 2 separate layers, in order to ensure 'core' services are stood up in advance of additional cluster supporting services.

To do:

Add a core-components (or appropriately named) folder & corresponding s3 state layer and move foundational components into this:

• Gatekeeper
• Calico
• RBAC
• Storage
• Custom system namespace labelling resources

(and review other appropriate candidates)

Once we have this structural change in code, test builds and terraform state move for existing clusters.

• Create new action to publish none main releases on cloud-platform-cli #5365
• Add support for core terraform in cloud-platform-cli #5359
• Add core folder under eks and move components #5315
• Move gatekeeper into core #5316
• Move eks_csi into core #5317
• Update create-cluster pipeline to be more live-like #5375
• Move networking into core #5318
• Move rbac into core #5319
• Move modsec ingress and ingress into core #5320
• Move cert-manager into core #5321
• Clean up cloud-platform-infrastructure repo #5322
• Move Prometheus crds to core #5323
• Create new concourse pipeline to test SCI clusters #5361
• Test and document plan for merging core into live environments #5376
• Plan next steps for Split Components #5300
• Create ADR for Split Components Infrastructure #5335

[sj-williams]

Notes:
https://docs.google.com/document/d/1xngnLF7wtnxnqL0TvaSfFRrPDc9JKpNQ5gz5BeR0twE/edit#heading=h.th49plkg8gop

labels move branch:
ministryofjustice/cloud-platform-infrastructure@main...move-default-ns-labels

[mikebell]

Closing in favour of milestones