-
Notifications
You must be signed in to change notification settings - Fork 55
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ci: add e2e test for self-managed infrastructure (#2472)
* add self-managed infra e2e test * self-managed terminatio Signed-off-by: Moritz Sanft <[email protected]> * fix upgrade test Signed-off-by: Moritz Sanft <[email protected]> * fix indentation Signed-off-by: Moritz Sanft <[email protected]> * use -r when copying dir Signed-off-by: Moritz Sanft <[email protected]> * add terraform variable parsing * copy constellation conf Signed-off-by: Moritz Sanft <[email protected]> * remove unnecessary line breaks * add missing value Signed-off-by: Moritz Sanft <[email protected]> * add image fetching for CSP Signed-off-by: Moritz Sanft <[email protected]> * fix quoting Signed-off-by: Moritz Sanft <[email protected]> * add missing input to internal lb test * normalize Azure URLs.. Of course * tidy Signed-off-by: Moritz Sanft <[email protected]> * fix expressions * initsecret to hex * update hexdump cmd * add build test Signed-off-by: Moritz Sanft <[email protected]> * add node / pod cidr outputs Signed-off-by: Moritz Sanft <[email protected]> * explicitly delete the state file Signed-off-by: Moritz Sanft <[email protected]> * add missing license header Signed-off-by: Moritz Sanft <[email protected]> * always write all outputs Signed-off-by: Moritz Sanft <[email protected]> * fix list output Signed-off-by: Moritz Sanft <[email protected]> * remove state-file and admin-conf on destroy * dont use test payload Signed-off-by: Moritz Sanft <[email protected]> * [remove] use self managed infra in manual e2e for testing Signed-off-by: Moritz Sanft <[email protected]> * init: always skip infrastructure phase * patch maa in workflow Signed-off-by: Moritz Sanft <[email protected]> * default to Constellation-created infra in e2e test --------- Signed-off-by: Moritz Sanft <[email protected]>
- Loading branch information
Showing
24 changed files
with
417 additions
and
11 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
name: Self-managed infrastructure creation | ||
description: "Create the required infrastructure for a Constellation cluster manually." | ||
|
||
inputs: | ||
cloudProvider: | ||
description: "The cloud provider the test runs on." | ||
required: true | ||
|
||
runs: | ||
using: "composite" | ||
steps: | ||
- name: Copy Terraform configuration and Constellation config | ||
shell: bash | ||
working-directory: | ||
run: | | ||
cp -r ${{ github.workspace }}/cli/internal/terraform/terraform/${{ inputs.cloudProvider }} ${{ github.workspace }}/e2e-infra | ||
cp ${{ github.workspace }}/constellation-conf.yaml ${{ github.workspace }}/e2e-infra | ||
- name: Get CSP image reference | ||
id: get_image | ||
shell: bash | ||
working-directory: ${{ github.workspace }}/e2e-infra | ||
run: | | ||
echo "image_ref=$(bazel run //hack/image-fetch:image-fetch)" >> $GITHUB_OUTPUT | ||
- name: Write Terraform variables | ||
shell: bash | ||
working-directory: ${{ github.workspace }}/e2e-infra | ||
run: | | ||
echo "name = \"$(yq '.name' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "debug = $(yq '.debugCluster' constellation-conf.yaml)" >> terraform.tfvars | ||
echo "custom_endpoint = \"$(yq '.customEndpoint' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "image_id = \"${{ steps.get_image.outputs.image_ref }}\"" >> terraform.tfvars | ||
echo "node_groups = { | ||
control_plane_default = { | ||
role = \"$(yq '.nodeGroups.control_plane_default.role' constellation-conf.yaml)\" | ||
zone = \"$(yq '.nodeGroups.control_plane_default.zone' constellation-conf.yaml)\" | ||
instance_type = \"$(yq '.nodeGroups.control_plane_default.instanceType' constellation-conf.yaml)\" | ||
disk_size = \"$(yq '.nodeGroups.control_plane_default.stateDiskSizeGB' constellation-conf.yaml)\" | ||
disk_type = \"$(yq '.nodeGroups.control_plane_default.stateDiskType' constellation-conf.yaml)\" | ||
initial_count = \"$(yq '.nodeGroups.control_plane_default.initialCount' constellation-conf.yaml)\" | ||
} | ||
worker_default = { | ||
role = \"$(yq '.nodeGroups.worker_default.role' constellation-conf.yaml)\" | ||
zone = \"$(yq '.nodeGroups.worker_default.zone' constellation-conf.yaml)\" | ||
instance_type = \"$(yq '.nodeGroups.worker_default.instanceType' constellation-conf.yaml)\" | ||
disk_size = \"$(yq '.nodeGroups.worker_default.stateDiskSizeGB' constellation-conf.yaml)\" | ||
disk_type = \"$(yq '.nodeGroups.worker_default.stateDiskType' constellation-conf.yaml)\" | ||
initial_count = \"$(yq '.nodeGroups.worker_default.initialCount' constellation-conf.yaml)\" | ||
} | ||
}" >> terraform.tfvars | ||
if [[ "${{ inputs.cloudProvider }}" == 'aws' ]]; then | ||
echo "iam_instance_profile_control_plane = \"$(yq '.provider.aws.iamProfileControlPlane' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "iam_instance_profile_worker_nodes = \"$(yq '.provider.aws.iamProfileWorkerNodes' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "region = \"$(yq '.provider.aws.region' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "zone = \"$(yq '.provider.aws.zone' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "ami = \"${{ steps.get_image.outputs.image_ref }}\"" >> terraform.tfvars | ||
echo "enable_snp = $(yq '.attestation | has("awsSEVSNP")' constellation-conf.yaml)" >> terraform.tfvars | ||
elif [[ "${{ inputs.cloudProvider }}" == 'azure' ]]; then | ||
echo "location = \"$(yq '.provider.azure.location' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "create_maa = $(yq '.attestation | has("azureSEVSNP")' constellation-conf.yaml)" >> terraform.tfvars | ||
echo "confidential_vm = $(yq '.attestation | has("azureSEVSNP")' constellation-conf.yaml)" >> terraform.tfvars | ||
echo "secure_boot = $(yq '.provider.azure.secureBoot' constellation-conf.yaml)" >> terraform.tfvars | ||
echo "resource_group = \"$(yq '.provider.azure.resourceGroup' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "user_assigned_identity = \"$(yq '.provider.azure.userAssignedIdentity' constellation-conf.yaml)\"" >> terraform.tfvars | ||
elif [[ "${{ inputs.cloudProvider }}" == 'gcp' ]]; then | ||
echo "project = \"$(yq '.provider.gcp.project' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "region = \"$(yq '.provider.gcp.region' constellation-conf.yaml)\"" >> terraform.tfvars | ||
echo "zone = \"$(yq '.provider.gcp.zone' constellation-conf.yaml)\"" >> terraform.tfvars | ||
fi | ||
terraform fmt terraform.tfvars | ||
echo "Using Terraform variables:" | ||
cat terraform.tfvars | ||
- name: Apply Terraform configuration | ||
shell: bash | ||
working-directory: ${{ github.workspace }}/e2e-infra | ||
run: | | ||
terraform init | ||
terraform apply -auto-approve | ||
- name: Patch MAA Policy | ||
shell: bash | ||
working-directory: ${{ github.workspace }}/e2e-infra | ||
if: ${{ inputs.cloudProvider }} == 'azure' | ||
run: | | ||
bazel run //hack/maa-patch:maa-patch $(terraform output attestationURL | jq -r) | ||
- name: Write outputs to state file | ||
shell: bash | ||
working-directory: ${{ github.workspace }}/e2e-infra | ||
run: | | ||
yq eval '.version ="v1"' --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.initSecret =\"$(terraform output initSecret | jq -r | tr -d '\n' | hexdump -ve '/1 "%02x"' && echo '')\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.clusterEndpoint =\"$(terraform output out_of_cluster_endpoint | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.inClusterEndpoint =\"$(terraform output in_cluster_endpoint | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.ipCidrNode =\"$(terraform output ip_cidr_nodes | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.uid =\"$(terraform output uid | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.name =\"$(terraform output name | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.apiServerCertSANs =$(terraform output -json api_server_cert_sans)" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
if [[ "${{ inputs.cloudProvider }}" == 'azure' ]]; then | ||
yq eval ".infrastructure.azure.resourceGroup =\"$(terraform output resource_group | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.azure.subscriptionID =\"$(terraform output subscription_id | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.azure.networkSecurityGroupName =\"$(terraform output network_security_group_name | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.azure.loadBalancerName =\"$(terraform output loadbalancer_name | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.azure.userAssignedIdentity =\"$(terraform output user_assigned_identity_client_id | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.azure.attestationURL =\"$(terraform output attestationURL | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
elif [[ "${{ inputs.cloudProvider }}" == 'gcp' ]]; then | ||
yq eval ".infrastructure.gcp.projectID =\"$(terraform output project | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
yq eval ".infrastructure.gcp.ipCidrPod =\"$(terraform output ip_cidr_pods | jq -r)\"" --inplace ${{ github.workspace }}/constellation-state.yaml | ||
fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.