Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rfc: Terraform provider #2613

Merged
merged 11 commits into from
Nov 23, 2023
99 changes: 99 additions & 0 deletions rfc/terraform-provider.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Constellation Terraform Provider

The Constellation Terraform Provider allows its user to manage the full lifecycle of a Constellation cluster -- namely initialization and updates (`constellation apply`) -- via Terraform.

## Design Goals

- The User needs to be able to perform initialization and upgrades of a Cluster through the provider. Deletion is covered by deleting the underlying infrastructure through Terraform.
- Infrastructure provisioning should explicitly **not** be performed by the Constellation Terraform provider.
- The User needs to receive some sort of Access Token to the Cluster (e.g. a Kubeconfig, whether long- or short-lived)
when applying a configuration containing the provider and the resources listed below.

elchead marked this conversation as resolved.
Show resolved Hide resolved
## Terraform Configuration Layout for the Constellation Resource

This resembles an examplary configuration of a Constellation cluster through Terraform. While the exact naming and struct layout
of the individual components may change until the implementation, the structure of the overall configuration looks like this:

```hcl
terraform {
required_providers {
constellation = {
source = "tbd/constellation"
version = "2.13.0"
}
}
}

provider "constellation" { }

resource "constellation_cluster" "foo" {
uid = "bar"
name = "baz"
image = data.constellation_image.ref # or provide manually crafted values
kubernetes_version = "v1.27.6"
constellation_microservice_version = "lockstep" # or pinned to a specific version
debug = false
init_endpoint = "10.10.10.10" # should use public ip of LB resource, ideally also provisioned through TF
kubernetes_api_endpoint = "10.10.10.10" # should use public ip of LB resource, ideally also provisioned through TF
burgerdev marked this conversation as resolved.
Show resolved Hide resolved
extra_microservices = {
csi_driver = true
# + more
# possiblly also constellation microservices with version and maybe service options,
# which would make constellation_microservice_version obsolete.
# exact API TBD
}
master_secret = "foo" # updating this would force recreation of the cluster
init_secret = "bar" # maybe derive from master_secret, updating this would force recreation of the cluster
msanft marked this conversation as resolved.
Show resolved Hide resolved
Comment on lines +45 to +46
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that the user generates these secrets and pastes them into this TF config?
If so, is it common to have such secrets in a TF config? Or should they come from input variables, env variables, etc.?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use the Terraform random provider (what we currently do) to generate this, load them from your HashiCorp vault by using the corresponding provider, etc.

I don't think you would expose them in the configuration files usually, but only in the state, which can be stored remotely, etc. and only have a reference to it, e.g. by using the random provider, in the configuration files - But the RFC should be agnostic of where the value comes from actually. When documenting our provider we could mention some recommendations of how secrets can be provisioned.

network_config = {
# TBD
# should contain CIDRs for pod network, service cidr, node network... for Cilium
# the aforementioned values might be outputs of infrastructure that is also provisioned
# through Terraform, such as a VPC.
# and in-cluster Kubernetes API endpoint, e.g. for Kubelets
}
attestation = data.constellation_attestation.attestation # or provide manually crafted values
}

# constellation_cluster provides:
# constellation_cluster.foo.kubeconfig
burgerdev marked this conversation as resolved.
Show resolved Hide resolved
# constellation_cluster.foo.owner_id
# constellation_cluster.foo.cluster_id


data "constellation_attestation" "foo" {
attestation_variant = "GCP-SEV-ES"
image_version = "v2.13.0" # or "lockstep"
maa_url = "https://www.example.com" # optional, for Azure only
}

# constellation_attestation provides:
# data.constellation_attestation.foo.measurements
# data.constellation_attestation.foo.attestation


data "constellation_image" "foo" {
image_version = "v2.21.0" # or "lockstep"
attestation_variant = "GCP-SEV-ES"
csp = "GCP"
region = "us-central1" # optional, for AWS only
}

# constellation_image provides:
# constellation_image.foo.reference (CSP-specific image reference)
```

The Terraform state is to be considered sensitive, as it contains the Master- & Init-Secret, as well as the Kubeconfig.

The `constellation_cluster` resource is the main resource implemented by the provider.
It declares a Constellation cluster with a specific configuration.
Applying it will create the cluster if not existing, upgrade the cluster when the changes can be performed in place (e.g. K8s / node image / microservice update) *or*
recreate the resource when the update can't be performed in-place (e.g. changing the master secret), update it with the according configuration if already existing,
or deletes it ("it" as in the state, but not the underlying infrastructure) if not present in the configuration but in the state. If resource recreation is necessary (i.e. if the resources cannot be updated in-place), an
[error](https://developer.hashicorp.com/terraform/plugin/framework/migrating/attributes-blocks/force-new#framework) is thrown that indicates that content on previously created
persistent volumes (encrypted with the old mastersecret) cannot be retrieved with the post-recreation / new mastersecret.

The "constellation_attestation" and "constellation_image" objects are [data sources](https://developer.hashicorp.com/terraform/language/data-sources),
which are objects that should be evaluated by the Provider each time the state is refreshed (i.e. each time any Terraform command that evaluates configuration against state),
but have no observable side effects. For image and attestation, this is required as the provider need to evaluate `latest` values or map CSP-agnostic image references (e.g. `v2.13.0`)
to CSP-specific image references (e.g. `/CommunityGalleries/.../Image` for Azure). This is implemented as an nilpotent API call and thus has no observable side-effects, but needs
to be re-evaluated as the values returned by the API might change between evaluations.