terraform: Terraform module for AWS

[elchead]

Context

This is the first implementation to manage constellation through Terraform using a module. It internally still relies on the CLI to call constellation apply.
However, the CLI is automatically fetched (and yq), so it's hidden from the user.

Proposed change(s)

• create a terraform-module for aws (shipped as zipped artifact)
• add an e2e test for the setup and run during weekly tests

Additional info

• AB#3502
• Successful E2E run older here
• Terraform changes will be rebased once reviewed. Since the files moved to a new location any Terraform change will cause conflicts right now.
• should we fetch yq like the constellation CLI in the background? It was found that yq has incompatible changes in different minor releases. E.g. one version required - to be present to process stdout input and another didn't. I'm not aware of any incompatibility issues in the current code, but I think it's safer to ensure one yq version.
• consider publication to module registry? https://developer.hashicorp.com/terraform/registry/modules/publish~ -> not until asked for
• How should we solve the reference to the constellation CLI until the release?
• Is it ok to use yq? and potentially install as dependency too?
  You need to perform a bazel devbuild inside aws-constellation such that the binary is available.

Usage

1. Download the terraform-module artifact.
2. cd terraform-module/aws-constellation
3. Create a terraform.tfvars file with the input (see E2E test)
4. (Optional): If you want to use a custom constellation CLI, you can place the binary inside the terraform-module/aws-constellation. Otherwise the latest CLI release is fetched during the execution.
5. Run terraform apply:

terraform apply -var-file=terraform.tfvars  -auto-approve
...
module.constellation.terraform_data.apply: Still creating... [15m40s elapsed]
module.constellation.terraform_data.apply (local-exec): Your Constellation cluster was successfully initialized.

module.constellation.terraform_data.apply (local-exec): Constellation cluster identifier  ff9a90dff66b2d4fe6043593bdef6abe28b26cd9d73b30febf709c1820f818e6
module.constellation.terraform_data.apply (local-exec): Kubernetes configuration          constellation-admin.conf

module.constellation.terraform_data.apply (local-exec): You can now connect to your cluster by executing:
module.constellation.terraform_data.apply (local-exec): 	export KUBECONFIG="/Users/adria/work/constellation/cli/internal/terraform/terraform/constellation-admin.conf"

module.constellation.terraform_data.apply: Creation complete after 15m49s [id=7451e54c-6620-1f58-29b6-f88472b1109f]

Apply complete! Resources: 3 added, 0 changed, 1 destroyed.
21.35s user 4.51s system 2% cpu 15:55.62s total

Checklist

• Update docs
• Add labels (e.g., for changelog category)
• Is PR title adequate for changelog?
• Link to Milestone

[netlify]

✅ Deploy Preview for constellation-docs canceled.

Name	Link
🔨 Latest commit	7d9fd37
🔍 Latest deploy log	https://app.netlify.com/sites/constellation-docs/deploys/654a87ee8ef5c70008443810

[3u13r]

What is the reason why we use --force here?

[elchead]

no reason. was just handy during testing. This should be replaced by apply as annotated before merging at any rate.

[elchead]

EDIT: it is needed for devbuilds of the CLI (since currently only release images are supported).

[msanft]

IMO, as we also discussed yesterday, we shouldn't place the module in the CLI directory. Here is my idea:

Move all of our Terraform configurations to a top-level directory called terraform. In that directory, place an assets.go, similar to what we use in the debugd. This file should just embed all the Terraform configurations that we currently embed in the CLI Terraform code directly. Then, in the CLI, we can import this assets-file. What do you think?

[msanft]

IMO, as we also discussed yesterday, we shouldn't place the module in the CLI directory. Here is my idea:

Move all of our Terraform configurations to a top-level directory called terraform. In that directory, place an assets.go, similar to what we use in the debugd. This file should just embed all the Terraform configurations that we currently embed in the CLI Terraform code directly. Then, in the CLI, we can import this assets-file. What do you think?

done

done

done

[msanft]

Looking good from a first glance! Will give this one more thorough review on monday, then we should be gtm.

[thomasten]

Last commit addressing my suggestions looks good. I'm not familiar enough with our TF code to approve the whole PR.

[msanft]

LGTM apart from the comments I made. Still, I feel unsure about approving this alone. I would like if @3u13r or someone else could have another look

done

[github-actions]

Coverage report

Package	Old	New	Trend
cli/internal/terraform	72.30%	72.30%	↔️
internal/constants	[no test files]	[no test files]	🚧
terraform	0.00%	[no test files]	🚨

[3u13r]

LGTM