diff --git a/dev-docs/user-manual.md b/dev-docs/user-manual.md new file mode 100644 index 000000000..7a757b78d --- /dev/null +++ b/dev-docs/user-manual.md @@ -0,0 +1,175 @@ +# Contrast + +Contrast runs confidential container deployments on untrusted Kubernetes at scale. + +Contrast is based on the [Kata Containers](https://github.com/kata-containers/kata-containers) and +[Confidential Containers](https://github.com/confidential-containers) projects. Confidential Containers are +Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation +from the surrounding environment. This works with unmodified containers in a lift-and-shift approach. +It currently targets the [CoCo preview on AKS](https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-containers-on-aks-preview). + +## The Contrast Coordinator + +The Contrast Coordinator is the central remote attestation component of a Contrast deployment. It's a certificate +authority and issues certificates for workload pods running inside confidential containers. The Coordinator +is configured with a *manifest*, a configuration file that holds the reference values of all other parts of +a deployment. The Coordinator ensures that your app's topology adheres to your specified manifest. It verifies +the identity and integrity of all your services and establishes secure, encrypted communication channels between +the different parts of your deployment. As your app needs to scale, the Coordinator transparently verifies new +instances and then provides them with mesh credentials. + +To verify your deployment, the remote attestation of the Coordinator and its manifest offers a single remote +attestation statement for your entire deployment. Anyone can use this to verify the integrity of your distributed +app, making it easier to assure stakeholders of your app's security. + +## The Contrast Initializer + +Contrast provides an Initializer that handles the remote attestation on the workload side transparently and +fetches the workload certificate. The Initializer runs as init container before your workload is started. + +## How to deploy emojivoto in Contrast + +### Prerequisites + +Install the latest version of the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/). +[Login to your account](https://docs.microsoft.com/en-us/cli/azure/authenticate-azure-cli) which has permissions to create an AKS cluster, by +executing: + +```sh +az login +``` + +Create an AKS cluster with Confidential Container support: + +```sh +az extension add \ + --name aks-preview + +az aks create \ + --resource-group myResourceGroup \ + --name myAKSCluster \ + --kubernetes-version 1.29 \ + --os-sku AzureLinux \ + --node-vm-size Standard_DC4as_cc_v5 \ + --node-count 1 \ + --generate-ssh-keys + +az aks nodepool add \ + --resource-group myResourceGroup \ + --name nodepool2 \ + --cluster-name myAKSCluster \ + --mode System \ + --node-count 1 \ + --os-sku AzureLinux \ + --node-vm-size Standard_DC4as_cc_v5 \ + --workload-runtime KataCcIsolation + +az aks get-credentials \ + --resource-group myResourceGroup \ + --name myAKSCluster +``` + +### Download the latest Contrast release + +Since Contrast has not yet been publicly released, a download URL will be +provided by Edgeless. + +```sh +wget +unzip contrast.zip +``` + +### Deploy the Contrast Coordinator + +Install the latest Contrast Coordinator release, comprising a single replica +deployment and a LoadBalancer service, into your cluster: + +```sh +kubectl apply -f coordinator.yml +``` + +### Annotate the emojivoto deployment + +Run the generate command generate the execution policies and add them as +annotations to your deployment files. A manifest.json with the reference values +of your deployment will be created: + +```sh +./contrast generate deployment/* +``` + +### Set the manifest + +Attest the Coordinator and set the manifest. It might take up to a few minutes +for the load balancer to be created. + +```sh +coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') +echo $coordinator +./contrast set -c "${coordinator}:1313" -m manifest.json deployment/ +``` + +### Deploy emojivoto + +Since the coordinator has a manifest set, which defines the amojivoto deployment +as an allowed workload, we can deploy the application: + +```sh +kubectl apply -f deployment/ +``` + +### Verify the Coordinator + +An end user (data owner) can verify the Contrast deployment using the verify +command: + +```sh +./contrast verify -c "${coordinator}:1313" +``` + +The CLI will attest the Coordinator using embedded reference values. The CLI +will write the service mesh root certificate and the history of manifests into +the `verify/` directory. In addition, the policies referenced in the manifest are +also written to the directory. + +### Connect and verify the workload + +Connect to the workloads using the Coordinator's mesh root as a trusted CA certificate. For example, with curl: + +```sh +kubectl patch svc web-svc -p '{"spec": {"type": "LoadBalancer"}}' +lbip=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') +echo $lbip +curl --cacert ./verify/mesh-root.pem -k "https://${lbip}" +``` + +The workload certificate is a DNS wildcard certificate, `curl`, but SAN +verification fails when accessing the workload via an IP address. +On Azure all load balancers automatically get ephemeral DNS entries, so either +use that or configure DNS yourself. + +To validate the certificate locally, use `openssl`: + +```sh +openssl s_client -showcerts -connect ${lbip}:443 certChain.pem +awk 'BEGIN {c=0;} /BEGIN CERT/{c++} { print > "cert." c ".pem"}' < certChain.pem +openssl verify -verbose -trusted verify/MeshCA.pem -- cert.1.pem +``` + +## Current limitations + +Contrast is in an early development stage and most underlying projects are under development, too. +As a result there are currently certain limitations, from which we try to document the most significant +ones here: + +- Only availabile on AKS with CoCo preview (AMD SEV-SNP) +- Persistent volumes currently not supported in CoCo +- While workload policies are functional in general, but [not covering all edge cases](https://github.com/microsoft/kata-containers/releases/tag/genpolicy-0.6.2-5) +- Port-forwarding isn't supported by Kata Containers yet +- CLI only available for Linux (mostly because upstream dependencies are not availabile for other platforms) + +## Upcoming Contrast features + +- Transparent service mesh (apps can currently use mTLS with Coordinator certs for secure communication) +- Plugin key management service (KMS) for attestation/coordinator certificate based key release +- High availability (distributed Contrast Coordinator)