Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.5, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.5
Contrast
-
Welcome to the documentation of Contrast! Contrast runs confidential container deployments on Kubernetes at scale.
-
Contrast is based on the Kata Containers and
-Confidential Containers projects.
-Confidential Containers are Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation from the surrounding environment.
-This works with unmodified containers in a lift-and-shift approach.
-Contrast currently targets the CoCo preview on AKS.
-
tip
See the 📄whitepaper for more information on confidential computing.
Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer. It removes the infrastructure provider from the trusted computing base (TCB). This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, for example, malicious co-tenants escalating their privileges.
-
Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.5, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.5
Confidential Containers
-
Contrast uses some building blocks from Confidential Containers (CoCo), a CNCF Sandbox project that aims to standardize confidential computing at the pod level.
-The project is under active development and many of the high-level features are still in flux.
-Contrast uses the more stable, core primitive provided by CoCo: its Kubernetes runtime.
Kubernetes can be extended to use more than one container runtime with RuntimeClass objects.
-The Container Runtime Interface (CRI) implementation, for example containerd, dispatches pod management API calls to the appropriate RuntimeClass.
-RuntimeClass implementations are usually based on an OCI runtime, such as runc, runsc or crun.
-In CoCo's case, the runtime is Kata Containers with added confidential computing capabilities.
Kata Containers is an OCI runtime that runs pods in VMs.
-The guest VM spawns an agent process that accepts management commands from the Kata runtime running on the host.
-There are two options for creating pod VMs: local to the Kubernetes node, or remote VMs created with cloud provider APIs.
-Using local VMs requires either bare metal servers or VMs with support for nested virtualization.
-Local VMs communicate with the host over a virtual socket.
-For remote VMs, host-to-agent communication is tunnelled through the cloud provider's network.
-
Kata Containers was originally designed to isolate the guest from the host, but it can also run pods in confidential VMs (CVMs) to shield pods from their underlying infrastructure.
-In confidential mode, the guest agent is configured with an Open Policy Agent (OPA) policy to authorize API calls from the host.
-This policy also contains checksums for the expected container images.
-It's derived from Kubernetes resource definitions and its checksum is included in the attestation report.
Azure Kubernetes Service (AKS) provides CoCo-enabled node pools as a preview offering.
-These node pools leverage Azure VM types capable of nested virtualization (CVM-in-VM) and the CoCo stack is pre-installed.
-Contrast can be deployed directly into a CoCo-enabled AKS cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.5, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.5
Product Features
-
Contrast simplifies the deployment and management of Confidential Containers, offering optimal data security for your workloads while integrating seamlessly with your existing Kubernetes environment.
-
From a security perspective, Contrast employs the Confidential Containers concept and provides security benefits that go beyond individual containers, shielding your entire deployment from the underlying infrastructure.
-
From an operational perspective, Contrast provides the following key features:
-
-
-
Managed Kubernetes Compatibility: Initially compatible with Azure Kubernetes Service (AKS), Contrast is designed to support additional platforms such as AWS EKS and Google Cloud GKE as they begin to accommodate confidential containers.
-
-
-
Lightweight Installation: Contrast can be integrated as a day-2 operation within existing clusters, adding minimal components to your setup. This facilitates straightforward deployments using your existing YAML configurations, Helm charts, or Kustomization, enabling native Kubernetes orchestration of your applications.
-
-
-
Remote Attestation: Contrast generates a concise attestation statement that verifies the identity, authenticity, and integrity of your deployment both internally and to external parties. This architecture ensures that updates or scaling of the application don't compromise the attestation’s validity.
-
-
-
Service Mesh: Contrast securely manages a Public Key Infrastructure (PKI) for your deployments, issues workload-specific certificates, and establishes transparent mutual TLS (mTLS) connections across pods. This is done by harnessing the envoy proxy to ensure secure communications within your Kubernetes cluster.
-
-
-
-
\ No newline at end of file
diff --git a/pr-preview/pr-918/0.5/basics/security-benefits.html b/pr-preview/pr-918/0.5/basics/security-benefits.html
deleted file mode 100644
index c1b3d95b22..0000000000
--- a/pr-preview/pr-918/0.5/basics/security-benefits.html
+++ /dev/null
@@ -1,22 +0,0 @@
-
-
-
-
-
-security-benefits | Contrast
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.5, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.5
Workload deployment
-
The following instructions will guide you through the process of making an existing Kubernetes deployment
-confidential and deploying it together with Contrast.
-
A running CoCo-enabled cluster is required for these steps, see the setup guide on how to set it up.
Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files
-unchanged, you can copy the files into a separate local directory.
-You can also generate files from a Helm chart or from a Kustomization.
To specify that a workload (pod, deployment, etc.) should be deployed as confidential containers,
-add runtimeClassName: kata-cc-isolation to the pod spec (pod definition or template).
-In addition, add the Contrast Initializer as initContainers to these workloads and configure the
-workload to use the certificates written to a volumeMount named tls-certs.
Run the generate command to generate the execution policies and add them as annotations to your
-deployment files. A manifest.json with the reference values of your deployment will be created.
For the next steps, we will need to connect to the Coordinator. The released Coordinator resource
-includes a LoadBalancer definition we can use.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
-
Port-forwarding of Confidential Containers
kubectl port-forward uses a Container Runtime Interface (CRI) method that isn't supported by the Kata shim.
-If you can't use a public load balancer, you can deploy a port-forwarder.
-The port-forwarder relays traffic from a CoCo pod and can be accessed via kubectl port-forward.
After this step, the Coordinator will start issuing TLS certs to the workloads. The init container
-will fetch a certificate for the workload and the workload is started.
An end user (data owner) can verify the Contrast deployment using the verify command.
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using embedded reference values. The CLI will write the service mesh
-root certificate and the history of manifests into the verify/ directory. In addition, the policies referenced
-in the manifest are also written to the directory.
You can securely connect to the workloads using the Coordinator's mesh-root.pem as a trusted CA certificate.
-First, expose the service on a public IP address via a LoadBalancer service:
-
kubectl patch svc ${MY_SERVICE}-p'{"spec": {"type": "LoadBalancer"}}' timeout 30s bash-c'until kubectl get service/${MY_SERVICE} --output=jsonpath='{.status.loadBalancer}' | grep "ingress"; do sleep 2 ; done' lbip=$(kubectl get svc ${MY_SERVICE}-o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo$lbip
-
Subject alternative names and LoadBalancer IP
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh root certificate with throw the following error:
$ curl--cacert ./verify/mesh-root.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
-
Using openssl, the certificate of the service can be validated with the mesh-root.pem:
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.5, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.5
Confidential emoji voting
-
-
This tutorial guides you through deploying emojivoto as a
-confidential Contrast deployment and validating the deployment from a voters perspective.
-
Emojivoto is an example app allowing users to vote for different emojis and view votes
-on a leader board. It has a microservice architecture consisting of a
-web frontend (web), a gRPC backend for listing available emojis (emoji), and a backend for
-the voting and leader board logic (voting). The vote-bot simulates user traffic by submitting
-votes to the frontend.
Using a voting service, users' votes are considered highly sensitive data, as we require
-a secret ballot. Also, users are likely interested in the fairness of the ballot. For
-both requirements, we can use Confidential Computing and, specifically, workload attestation
-to prove to those interested in voting that the app is running in a protected environment
-where their votes are processed without leaking to the platform provider or workload owner.
Run the generate command to generate the execution policies and add them as
-annotations to your deployment files. A manifest.json file with the reference values
-of your deployment will be created:
-
contrast generate deployment/
-
Runtime class and Initializer
The deployment YAML shipped for this demo is already configured to be used with Contrast.
-A runtime class kata-cc-isolation was added to the pods to signal they should be run
-as Confidential Containers. In addition, the Contrast Initializer was added
-as an init container to these workloads to facilitate the attestation and certificate pulling
-before the actual workload is started.
Configure the coordinator with a manifest. It might take up to a few minutes
-for the load balancer to be created and the Coordinator being available.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"The user API of your Contrast Coordinator is available at $coordinator:1313" contrast set-c"${coordinator}:1313" deployment/
-
The CLI will use the embedded reference values to attest the Coordinator deployment
-during the TLS handshake. If the connection succeeds, we're ensured that the Coordinator
-deployment hasn't been tampered with.
Now that the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload,
-we can deploy the application:
-
kubectl apply -f deployment/
-
Inter-deployment communication
The Contrast Coordinator issues mesh certificates after successfully validating workloads.
-These certificates can be used for secure inter-deployment communication. The Initializer
-sends an attestation report to the Coordinator, retrieves certificates and a private key in return
-and writes them to a volumeMount. The emojivoto version we're using is patched to only communicate
-via mTLS (the original app talks plain HTTP). The different parts of the workload are configured
-to use the credentials from the volumeMount when communicating with each other.
As voters, we want to verify the fairness and confidentiality of the deployment before
-deciding to vote. Regardless of the scale of our distributed deployment, Contrast only
-needs a single remote attestation step to verify the deployment. By doing remote attestation
-of the Coordinator, we transitively verify those systems the Coordinator has already attested
-or will attest in the future. Successful verification of the Coordinator means that
-we can be sure it will enforce the configured manifest.
A potential voter can verify the Contrast deployment using the verify
-command:
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using embedded reference values. If the command succeeds,
-the Coordinator deployment was successfully verified to be running in the expected Confidential
-Computing environment with the expected code version. The Coordinator will then return its
-configuration over the established TLS channel. The CLI will store this information, namely the root
-certificate of the mesh (mesh-root.pem) and the history of manifests, into the verify/ directory.
-In addition, the policies referenced in the manifest history are also written into the same directory.
In the next step, the Coordinator configuration that was written by the verify command needs to be audited.
-A potential voter should inspect the manifest and the referenced policies. They could delegate
-this task to an entity they trust.
After ensuring the configuration of the Coordinator fits the expectation, you can securely connect
-to the workloads using the Coordinator's mesh-root.pem as a trusted CA certificate.
-
To access the web frontend, expose the service on a public IP address via a LoadBalancer service:
-
frontendIP=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"Frontend is available at https://$frontendIP, you can visit it in your browser."
-
Using openssl, the certificate of the service can be validated with the mesh-root.pem:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh root certificate with throw the following error:
-
$ curl--cacert ./verify/mesh-root.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
The Policies section of the manifest maps policy hashes to a list of SANs. To enable certificate verification
-of the web frontend with tools like curl, edit the policy with your favorite editor and add the frontendIP to
-the list that already contains the "web" DNS entry:
Next, set the changed manifest at the coordinator with:
-
contrast set-c"${coordinator}:1313" deployment/
-
The Contrast Coordinator will rotate the mesh root certificate on the manifest update. Workload certificates issued
-after the manifest are thus issued by another certificate authority and services receiving the new CA certificate chain
-won't trust parts of the deployment that got their certificate issued before the update. This way, Contrast ensures
-that parts of the deployment that received a security update won't be infected by parts of the deployment at an older
-patch level that may have been compromised. The mesh-root.pem is updated with the new CA certificate chain.
The Coordinator has the new manifest set, but the different containers of the app are still
-using the older certificate authority. The Contrast Initializer terminates after the initial attestation
-flow and won't pull new certificates on manifest updates.
After the update has been rolled out, connecting to the frontend using curl will successfully validate
-the service certificate and return the HTML document of the voting site:
CoCo on AKS is currently in preview. An extension for the az CLI is needed to create such a cluster.
-Add the extension with the following commands:
-
az extension add\ --name aks-preview \ --allow-preview true az extension update \ --name aks-preview \ --allow-preview true
-
Then register the required feature flags in your subscription to allow access to the public preview:
-
az feature register \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"
-
The registration can take a few minutes. The status of the operation can be checked with the following
-command, which should show the registration state as Registered:
-
az feature show \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"\ --output table
-
Afterward, refresh the registration of the ContainerService provider:
-
az provider register \ --namespace"Microsoft.ContainerService"
# Select the name for your AKS cluster azClusterName="ContrastDemo" az aks create \ --resource-group "$azResourceGroup"\ --name"$azClusterName"\ --kubernetes-version 1.29\ --os-sku AzureLinux \ --node-vm-size Standard_DC4as_cc_v5 \ --node-count 1\ --generate-ssh-keys
After trying out Contrast, you might want to clean up the cloud resources created in this step.
-In case you've created a new resource group, you can just delete that group with
-
az group delete \ --name"$azResourceGroup"
-
Deleting the resource group will also delete the cluster and all other related resources.
-
To only cleanup the AKS cluster and node pools, run
-
az aks delete \ --resource-group "$azResourceGroup"\ --name"$azClusterName"
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Contrast
-
Welcome to the documentation of Contrast! Contrast runs confidential container deployments on Kubernetes at scale.
-
Contrast is based on the Kata Containers and
-Confidential Containers projects.
-Confidential Containers are Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation from the surrounding environment.
-This works with unmodified containers in a lift-and-shift approach.
-Contrast currently targets the CoCo preview on AKS.
-
tip
See the 📄whitepaper for more information on confidential computing.
Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer. It removes the infrastructure provider from the trusted computing base (TCB). This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, for example, malicious co-tenants escalating their privileges.
-
Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
CLI telemetry
-
The Contrast CLI sends telemetry data to Edgeless Systems when you use CLI commands.
-This allows to understand how Contrast is used and to improve it.
-
The CLI sends the following data:
-
-
The CLI version
-
The CLI target OS and architecture (GOOS and GOARCH)
-
The command that was run
-
The kind of error that occurred (if any)
-
-
The CLI doesn't collect sensitive information.
-The implementation is open-source and can be reviewed.
-
IP addresses may be processed or stored for security purposes.
-
The data that the CLI collects adheres to the Edgeless Systems privacy policy.
-
You can disable telemetry by setting the environment variable DO_NOT_TRACK=1 before running the CLI.
-
-
\ No newline at end of file
diff --git a/pr-preview/pr-918/0.6/architecture.html b/pr-preview/pr-918/0.6/architecture.html
deleted file mode 100644
index 976b25e662..0000000000
--- a/pr-preview/pr-918/0.6/architecture.html
+++ /dev/null
@@ -1,24 +0,0 @@
-
-
-
-
-
-Architecture | Contrast
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Attestation in Contrast
-
This document describes the attestation architecture of Contrast, adhering to the definitions of Remote ATtestation procedureS (RATS) in RFC 9334.
-The following gives a detailed description of Contrast's attestation architecture.
-At the end of this document, we included an FAQ that answers the most common questions regarding attestation in hindsight of the security benefits.
Contrast integrates with the RATS architecture, leveraging their definition of roles and processes including Attesters, Verifiers, and Relying Parties.
-
-
Figure 1: Conceptual attestation architecture. Taken from RFC 9334.
-
-
Attester: Assigned to entities that are responsible for creating Evidence which is then sent to a Verifier.
-
Verifier: These entities utilize the Evidence, Reference Values, and Endorsements. They assess the trustworthiness of the Attester by applying an Appraisal Policy for Evidence. Following this assessment, Verifiers generate Attestation Results for use by Relying Parties. The Appraisal Policy for Evidence may be provided by the Verifier Owner, programmed into the Verifier, or acquired through other means.
-
Relying Party: Assigned to entities that utilize Attestation Results, applying their own appraisal policies to make specific decisions, such as authorization decisions. This process is referred to as the "appraisal of Attestation Results." The Appraisal Policy for Attestation Results might be sourced from the Relying Party Owner, configured by the owner, embedded in the Relying Party, or obtained through other protocols or mechanisms.
This includes all Pods of the Contrast deployment that run inside Confidential Containers and generate cryptographic evidence reflecting their current configuration and state.
-Their evidence is rooted in the hardware measurements from the CPU and their confidential VM environment.
-The details of this evidence are given below in the section on evidence generation and appraisal.
-
-
Figure 2: Attestation flow of a confidential pod. Based on the layered attester graphic in RFC 9334.
-
Pods run in Contrast's runtime environment (B), effectively within a confidential VM.
-During launch, the CPU (A) measures the initial memory content of the confidential VM that contains Contrast's pod-VM image and generates the corresponding attestation evidence.
-The image is in IGVM format, encapsulating all information required to launch a virtual machine, including the kernel, the initramfs, and kernel cmdline.
-The kernel cmdline contains the root hash for dm-verity that ensures the integrity of the root filesystem.
-The root filesystem contains all components of the container's runtime environment including the guest agent (C).
-
In the userland, the guest agent takes care of enforcing the runtime policy of the pod.
-While the policy is passed in during the initialization procedure via the host, the evidence for the runtime policy is part of the CPU measurements.
-During the deployment the policy is annotated to the Kubernetes Pod resources.
-On AMD SEV-SNP the hash of the policy is then added to the attestation report via the HOSTDATA field by the hypervisor.
-When provided with the policy from the Kata host, the guest agent verifies that the policy's hash matches the one in the HOSTDATA field.
-
In summary a Pod's evidence is the attestation report of the CPU that provides evidence for runtime environment and the runtime policy.
The Coordinator acts as a verifier within the Contrast deployment, configured with a Manifest that defines the reference values and serves as an appraisal policy for all pods in the deployment.
-It also pulls endorsements from hardware vendors to verify the hardware claims.
-The Coordinator operates within the cluster as a confidential container and provides similar evidence as any other Pod when it acts as an attester.
-In RATS terminology, the Coordinator's dual role is defined as a lead attester in a composite device which spans the entire deployment: Coordinator and the workload pods.
-It collects evidence from other attesters and conveys it to a verifier, generating evidence about the layout of the whole composite device based on the Manifest as the appraisal policy.
-
-
Figure 3: Contrast deployment as a composite device. Based on the composite device in RFC 9334.
-
The CLI serves as the verifier for the Coordinator and the entire Contrast deployment, containing the reference values for the Coordinator and the endorsements from hardware vendors.
-These reference values are built into the CLI during our release process and can be reproduced offline via reproducible builds.
A relying party in the Contrast scenario could be, for example, the data owner that interacts with the application.
-The relying party can use the CLI to obtain the attestation results and Contrast's CA certificates bound to these results.
-The CA certificates can then be used by the relying party to authenticate the application, for example through TLS connections.
In Contrast, attestation evidence revolves around a hardware-generated attestation report, which contains several critical pieces of information:
-
-
The hardware attestation report: This report includes details such as the chip identifier, platform information, microcode versions, and comprehensive guest measurements. The entire report is signed by the CPU's private key, ensuring the authenticity and integrity of the data provided.
-
The launch measurements: Included within the hardware attestation report, this is a digest generated by the CPU that represents a hash of all initial guest memory pages. This includes essential components like the kernel, initramfs, and the kernel command line. Notably, it incorporates the root filesystem's dm-verity root hash, verifying the integrity of the root filesystem.
-
The runtime policy hash: Also part of the hardware attestation report, this field contains the hash of the Rego policy which dictates all expected API commands and their values from the host to the Kata guest agent. It encompasses crucial settings such as dm-verity hashes for the container image layers, environment variables, and mount points.
The appraisal of this evidence in Contrast is governed by two main components:
-
-
The Manifest: A JSON file used by the Coordinator to align with reference values. It sets the expectations for runtime policy hashes for each pod and includes what should be reported in the hardware attestation report for each component of the deployment.
-
The CLI's appraisal policy: This policy encompasses expected values of the Coordinator’s guest measurements and its runtime policy. It's embedded into the CLI during the build process and ensures that any discrepancy between the built-in values and those reported by the hardware attestation can be identified and addressed. The integrity of this policy is safeguardable through reproducible builds, allowing verification against the source code reference.
-
-
Frequently asked questions about attestation in Contrast
-
What's the purpose of remote attestation in Contrast?
-
Remote attestation in Contrast ensures that software runs within a secure, isolated confidential computing environment.
-This process certifies that the memory is encrypted and confirms the integrity and authenticity of the software running within the deployment.
-By validating the runtime environment and the policies enforced on it, Contrast ensures that the system operates in a trustworthy state and hasn't been tampered with.
-
How does Contrast ensure the security of the attestation process?
-
Contrast leverages hardware-rooted security features such as AMD SEV-SNP to generate cryptographic evidence of a pod’s current state and configuration.
-This evidence is checked against pre-defined appraisal policies to guarantee that only verified and authorized pods are part of a Contrast deployment.
Attestation confirms the integrity of the runtime environment and the identity of the workloads.
-It plays a critical role in preventing unauthorized changes and detecting potential modifications at runtime.
-The attestation provides integrity and authenticity guarantees, enabling relying parties—such as workload operators or data owners—to confirm the effective protection against potential threats, including malicious cloud insiders, co-tenants, or compromised workload operators.
-More details on the specific security benefits can be found here.
-
How can you verify the authenticity of attestation results?
-
Attestation results in Contrast are tied to cryptographic proofs generated and signed by the hardware itself.
-These proofs are then verified using public keys from trusted hardware vendors, ensuring that the results aren't only accurate but also resistant to tampering.
-For further authenticity verification, all of Contrast's code is reproducibly built, and the attestation evidence can be verified locally from the source code.
-
How are attestation results used by relying parties?
-
Relying parties use attestation results to make informed security decisions, such as allowing access to sensitive data or resources only if the attestation verifies the system's integrity.
-Thereafter, the use of Contrast's CA certificates in TLS connections provides a practical approach to communicate securely with the application.
In summary, Contrast's attestation strategy adheres to the RATS guidelines and consists of robust verification mechanisms that ensure each component of the deployment is secure and trustworthy.
-This comprehensive approach allows Contrast to provide a high level of security assurance to its users.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Certificate authority
-
The Coordinator acts as a certificate authority (CA) for the workloads
-defined in the manifest.
-After a workload pod's attestation has been verified by the Coordinator,
-it receives a mesh certificate and the mesh CA certificate.
-The mesh certificate can be used for example in a TLS connection as the server or
-client certificate to proof to the other party that the workload has been
-verified by the Coordinator. The other party can verify the mesh certificate
-with the mesh CA certificate. While the certificates can be used by the workload
-developer in different ways, they're automatically used in Contrast's service
-mesh to establish mTLS connections between workloads in the same deployment.
The Coordinator establishes a public key infrastructure (PKI) for all workloads
-contained in the manifest. The Coordinator holds three certificates: the root CA
-certificate, the intermediate CA certificate, and the mesh CA certificate.
-The root CA certificate is a long-lasting certificate and its private key signs
-the intermediate CA certificate. The intermediate CA certificate and the mesh CA
-certificate share the same private key. This intermediate private key is used
-to sign the mesh certificates. Moreover, the intermediate private key and
-therefore the intermediate CA certificate and the mesh CA certificate are
-rotated when setting a new manifest.
Depending on the configuration of the first manifest, it allows the workload
-owner to update the manifest and, therefore, the deployment.
-Workload owners and data owners can be mutually untrusted parties.
-To protect against the workload owner silently introducing malicious containers,
-the Coordinator rotates the intermediate private key every time the manifest is
-updated and, therefore, the
-intermediate CA certificate and mesh CA certificate. If the user doesn't
-trust the workload owner, they use the mesh CA certificate obtained when they
-verified the Coordinator and the manifest. This ensures that the user only
-connects to workloads defined in the manifest they verified since only those
-workloads' certificates are signed with this intermediate private key.
-
Similarly, the service mesh also uses the mesh CA certificate obtained when the
-workload was started, so the workload only trusts endpoints that have been
-verified by the Coordinator based on the same manifest. Consequently, a
-manifest update requires a fresh rollout of the services in the service mesh.
The root CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This should only be used if the data owner trusts all future updates to the
-manifest and workloads. This is, for instance, the case when the workload owner is
-the same entity as the data owner.
-
The mesh CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This certificate is bound to the manifest set when the Coordinator is verified.
-If the manifest is updated, the mesh CA certificate changes.
-New workloads will receive mesh certificates signed by the new mesh CA certificate.
-The Coordinator with the new manifest needs to be verified to retrieve the new mesh CA certificate.
-The service mesh also uses the mesh CA certificate to verify the mesh certificates.
-
The intermediate CA certificate links the root CA certificate to the
-mesh certificate so that the mesh certificate can be verified with the root CA
-certificate. It's part of the certificate chain handed out by
-endpoints in the service mesh.
-
The mesh certificate is part of the certificate chain handed out by
-endpoints in the service mesh. During the startup of a pod, the Initializer
-requests a certificate from the Coordinator. This mesh certificate will be returned if the Coordinator successfully
-verifies the workload. The mesh certificate
-contains X.509 extensions with information from the workloads attestation
-document.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Confidential Containers
-
Contrast uses some building blocks from Confidential Containers (CoCo), a CNCF Sandbox project that aims to standardize confidential computing at the pod level.
-The project is under active development and many of the high-level features are still in flux.
-Contrast uses the more stable core primitive provided by CoCo: its Kubernetes runtime.
Kubernetes can be extended to use more than one container runtime with RuntimeClass objects.
-The Container Runtime Interface (CRI) implementation, for example containerd, dispatches pod management API calls to the appropriate RuntimeClass.
-RuntimeClass implementations are usually based on an OCI runtime, such as runc, runsc or crun.
-In CoCo's case, the runtime is Kata Containers with added confidential computing capabilities.
Kata Containers is an OCI runtime that runs pods in VMs.
-The pod VM spawns an agent process that accepts management commands from the Kata runtime running on the host.
-There are two options for creating pod VMs: local to the Kubernetes node, or remote VMs created with cloud provider APIs.
-Using local VMs requires either bare metal servers or VMs with support for nested virtualization.
-Local VMs communicate with the host over a virtual socket.
-For remote VMs, host-to-agent communication is tunnelled through the cloud provider's network.
-
Kata Containers was originally designed to isolate the guest from the host, but it can also run pods in confidential VMs (CVMs) to shield pods from their underlying infrastructure.
-In confidential mode, the guest agent is configured with an Open Policy Agent (OPA) policy to authorize API calls from the host.
-This policy also contains checksums for the expected container images.
-It's derived from Kubernetes resource definitions and its checksum is included in the attestation report.
Azure Kubernetes Service (AKS) provides CoCo-enabled node pools as a preview offering.
-These node pools leverage Azure VM types capable of nested virtualization (CVM-in-VM) and the CoCo stack is pre-installed.
-Contrast can be deployed directly into a CoCo-enabled AKS cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Product features
-
Contrast simplifies the deployment and management of Confidential Containers, offering optimal data security for your workloads while integrating seamlessly with your existing Kubernetes environment.
-
From a security perspective, Contrast employs the Confidential Containers concept and provides security benefits that go beyond individual containers, shielding your entire deployment from the underlying infrastructure.
-
From an operational perspective, Contrast provides the following key features:
-
-
-
Managed Kubernetes compatibility: Initially compatible with Azure Kubernetes Service (AKS), Contrast is designed to support additional platforms such as AWS EKS and Google Cloud GKE as they begin to accommodate confidential containers.
-
-
-
Lightweight installation: Contrast can be integrated as a day-2 operation within existing clusters, adding minimal components to your setup. This facilitates straightforward deployments using your existing YAML configurations, Helm charts, or Kustomization, enabling native Kubernetes orchestration of your applications.
-
-
-
Remote attestation: Contrast generates a concise attestation statement that verifies the identity, authenticity, and integrity of your deployment both internally and to external parties. This architecture ensures that updates or scaling of the application don't compromise the attestation’s validity.
-
-
-
Service mesh: Contrast securely manages a Public Key Infrastructure (PKI) for your deployments, issues workload-specific certificates, and establishes transparent mutual TLS (mTLS) connections across pods. This is done by harnessing the envoy proxy to ensure secure communications within your Kubernetes cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Contrast security overview
-
This document outlines the security measures of Contrast and its capability to counter various threats.
-Contrast is designed to shield entire Kubernetes deployments from the infrastructure, enabling entities to manage sensitive information (such as regulated or personally identifiable information (PII)) in the public cloud, while maintaining data confidentiality and ownership.
-
Contrast is applicable in situations where establishing trust with the workload operator or the underlying infrastructure is challenging.
-This is particularly beneficial for regulated sectors looking to transition sensitive activities to the cloud, without sacrificing security or compliance.
-It allows for cloud adoption by maintaining a hardware-based separation from the cloud service provider.
Leveraging Confidential Computing technology, Contrast provides three defining security properties:
-
-
Encryption of data in use: Contrast ensures that all data processed in memory is encrypted, making it inaccessible to unauthorized users or systems, even if they have physical access to the hardware.
-
Workload isolation: Each pod runs in its isolated runtime environment, preventing any cross-contamination between workloads, which is critical for multi-tenant infrastructures.
-
Remote attestation: This feature allows data owners and workload operators to verify that the Contrast environment executing their workloads hasn't been tampered with and is running in a secure, pre-approved configuration.
-
-
The runtime encryption is transparently provided by the confidential computing hardware during the workload's lifetime.
-The workload isolation and remote attestation involves two phases:
-
-
An attestation process detects modifications to the workload image or its runtime environment during the initialization. This protects the workload's integrity pre-attestation.
-
A protected runtime environment and a policy mechanism prevents the platform operator from accessing or compromising the instance at runtime. This protects a workload's integrity and confidentiality post-attestation.
-
-
For more details on confidential computing see our whitepaper.
-The attestation architecture describes Contrast's attestation process and the resulting chain of trust in detail.
Contrast uses the Kubernetes runtime of the Confidential Containers project.
-Confidential Containers significantly decrease the size of the trusted computing base (TCB) of a Kubernetes deployment, by isolating each pod within its own confidential micro-VM environment.
-The TCB is the totality of elements in a computing environment that must be trusted not to be compromised.
-A smaller TCB results in a smaller attack surface. The following diagram shows how Confidential Containers remove the cloud & datacenter infrastructure and the physical hosts, including the hypervisor, the host OS, the Kubernetes control plane, and other components, from the TCB (red).
-In the confidential context, depicted in green, only the workload containers along with their confidential micro-VM environment are included within the TCB.
-Their integrity is verifiable through remote attestation.
-
Contrast uses hardware-based mechanisms, specifically leveraging CPU features, such as AMD SEV or Intel TDX, to provide the isolation of the workload.
-This implies that both the CPU and its microcode are integral components of the TCB.
-However, it should be noted that the CPU microcode aspects aren't depicted in the accompanying graphic.
-
-
Contrast adds the following components to a deployment that become part of the TCB.
-The components that are part of the TCB are:
-
-
The workload containers: Container images that run the actual application.
The runtime policies: Policies that enforce the runtime environments for the workload containers during their lifetime.
-
The manifest: A manifest file defining the reference values of an entire confidential deployment. It contains the policy hashes for all pods of the deployment and the expected hardware reference values for the Confidential Container runtime.
-
The Coordinator: An attestation service that runs in a Confidential Container in the Kubernetes cluster. The Coordinator is configured with the manifest. User-facing, you can verify this service and the effective manifest using remote attestation, providing you with a concise attestation for the entire deployment. Cluster-facing, it verifies all pods and their policies based on remote attestation procedures and the manifest.
In a Contrast deployment, there are three parties:
-
-
-
The container image provider, who creates the container images that represent the application that has access to the protected data.
-
-
-
The workload operator, who runs the workload in a Kubernetes cluster. The operator typically has full administrative privileges to the deployment. The operator can manage cluster resources such as nodes, volumes, and networking rules, and the operator can interact with any Kubernetes or underlying cloud API.
-
-
-
The data owner, who owns the protected data. A data owner can verify the deployment using the Coordinator attestation service. The verification includes the identity, integrity, and confidentiality of the workloads, the runtime environment and the access permissions.
-
-
-
Contrast supports a trust model where the container image provider, workload operator, and data owner are separate, mutually distrusting parties.
-
The following diagram shows the system components and parties.
Contrast is designed to defend against five possible attacks:
-
-
A malicious cloud insider: malicious employees or third-party contractors of cloud service providers (CSPs) potentially have full access to various layers of the cloud infrastructure. That goes from the physical datacenter up to the hypervisor and Kubernetes layer. For example, they can access the physical memory of the machines, modify the hypervisor, modify disk contents, intercept network communications, and attempt to compromise the confidential container at runtime. A malicious insider can expand the attack surface or restrict the runtime environment. For example, a malicious operator can add a storage device to introduce new attack vectors. As another example, a malicious operator can constrain resources such as limiting a guest's memory size, changing its disk space, or changing firewall rules.
-
A malicious cloud co-tenant: malicious cloud user ("hackers") may break out of their tenancy and access other tenants' data. Advanced attackers may even be able to establish a permanent foothold within the infrastructure and access data over a longer period. The threats are analogous to the cloud insider access scenario, without the physical access.
-
A malicious workload operator: malicious workload operators, for example Kubernetes administrators, have full access to the workload deployment and the underlying Kubernetes platform. The threats are analogously to the cloud insider access scenario, with access to everything that's above the hypervisor level.
-
A malicious attestation client: this attacker connects to the attestation service and sends malformed request.
-
A malicious container image provider: a malicious container image provider has full control over the application development itself. This attacker might release a malicious version of the workload containing harmful operations.
Contrast shields a workload from the aforementioned threats with three main components:
-
-
The runtime environment safeguards against the physical memory and disk attack surface.
-
The runtime policies safeguard against the Kubernetes control plane and container runtime attack surface.
-
The service mesh safeguards against the network attack surface.
-
-
The following tables describe concrete threats and how they're mitigated in Contrast grouped by these categories:
-
-
Attacks on the confidential container environment
-
Attacks on the attestation service
-
Attacks on workloads
-
-
Attacks on the confidential container environment
-
This table describes potential threats and mitigation strategies related to the confidential container environment.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the network connection of the launcher or image repository.
An attacker can change the image URL and control the workload binary. However these actions are reflected in the attestation report. The image repository isn't controlled using an access list, therefore the image is assumed to be viewable by everyone. You must ensure that the workload container image doesn't contain any secrets.
An attacker modifies a container's runtime environment configuration in the Kubernetes control plane.
The attestation process and the runtime policies detects unsafe configurations that load non-authentic images or perform any other modification to the expected runtime environment.
This table describes potential threats and mitigation strategies to the attestation service.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the Coordinator deployment and modifies the image or hijacks the runtime environment.
This threat is mitigated by having an attestation procedure and attested, encrypted TLS connections to the Coordinator. The attestation evidence for the Coordinator image is distributed with our releases, protected by supply chain security, and fully reproducible.
An attacker intercepts the network connection between the workload and the Coordinator and reads secret keys from the wire.
This threat is mitigated by having an attested, encrypted TLS connection. This connection helps protect the secrets from passive eavesdropping. The attacker can't create valid workload certificates that would be accepted in Contrast's service mesh. An attacker can't impersonate a valid workload container because the container's identity is guaranteed by the attestation protocol.
Within the network between your workload and the Coordinator.
An attacker exploits parsing discrepancies, which leads to undetected changes in the attestation process.
This risk is mitigated by having a parsing engine written in memory-safe Go that's tested against the attestation specification of the hardware vendor. The runtime policies are available as an attestation artifact for further inspection and audits to verify their effectiveness.
An attacker reads or modifies data written to disk via persistent volumes.
Currently, persistent volumes aren't supported in Contrast. In the future, this threat is mitigated by encrypted and integrity-protected volume mounts.
An attacker publishes a new image version containing malicious code.
The attestation process and the runtime policies require a data owner to accept a specific version of the workload and any update to the workload needs to be explicitly acknowledged.
The following table describes three example use cases and how they map to the defined threat model in this document:
-
Use Case
Example Scenario
Migrate sensitive workloads to the cloud
TechSolve Inc., a software development firm, aimed to enhance its defense-in-depth strategy for its cloud-based development environment, especially for projects involving proprietary algorithms and client data. TechSolve acts as the image provider, workload operator, and data owner, combining all three personas in this scenario. In our attestation terminology, they're the workload operator and relying party in one entity. Their threat model includes a malicious cloud insider and cloud co-tenant.
Make your SaaS more trustworthy
SaaSProviderX, a company offering cloud-based project management tools, sought to enhance its platform's trustworthiness amidst growing concerns about data breaches and privacy. Here, the relying party is the SaaS customer as the data owner. The goal is to achieve a form of operator exclusion and only allow selective operations on the deployment. Hence, their threat model includes a malicious workload operator.
Simplify regulatory compliance
HealthSecure Inc. has been managing a significant volume of sensitive patient data on-premises. With the increasing demand for advanced data analytics and the need for scalable infrastructure, the firm decides to migrate its data analytics operations to the cloud. However, the primary concern is maintaining the confidentiality and security of patient data during and after the migration, in compliance with healthcare regulations. In this compliance scenario, the regulator serves as an additional relying party. HealthSecure must implement a mechanism that ensures the isolation of patient data can be verifiably guaranteed to the regulator.
-
In each scenario, Contrast ensures exclusive data access and processing capabilities are confined to the designated workloads. It achieves this by effectively isolating the workload from the infrastructure and other components of the stack. Data owners are granted the capability to audit and approve the deployment environment before submitting their data, ensuring a secure handover. Meanwhile, workload operators are equipped to manage and operate the application seamlessly, without requiring direct access to either the workload or its associated data.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Components
-
Contrast is composed of several key components that work together to manage and scale confidential containers effectively within Kubernetes environments.
-This page provides an overview of the core components essential for deploying and managing Contrast.
The CLI serves as the primary management tool for Contrast deployments. It's designed to streamline the configuration and operation of Contrast in several ways:
-
-
Installation and setup: The CLI facilitates the installation of the necessary runtime classes required for Contrast to function within a Kubernetes cluster.
-
Policy generation: It allows users to generate runtime policies, adapt the deployment files, and generate the Contrast manifest.
-
Configuration management: Through the CLI, users can configure the Contrast Coordinator with the generated manifest.
-
Verification and attestation: The CLI provides tools to verify the integrity and authenticity of the Coordinator and the entire deployment via remote attestation.
The Contrast Coordinator is the central remote attestation service of a Contrast deployment.
-It runs inside a confidential container inside your cluster.
-The Coordinator can be verified via remote attestation, and a Contrast deployment is self-contained.
-The Coordinator is configured with a manifest, a configuration file containing the reference attestation values of your deployment.
-It ensures that your deployment's topology adheres to your specified manifest by verifying the identity and integrity of all confidential pods inside the deployment.
-The Coordinator is also a certificate authority and issues certificates for your workload pods during the attestation procedure.
-Your workload pods can establish secure, encrypted communication channels between themselves based on these certificates using the Coordinator as the root CA.
-As your app needs to scale, the Coordinator transparently verifies new instances and then provides them with their certificates to join the deployment.
-
To verify your deployment, the Coordinator's remote attestation statement combined with the manifest offers a concise single remote attestation statement for your entire deployment.
-A third party can use this to verify the integrity of your distributed app, making it easy to assure stakeholders of your app's identity and integrity.
The manifest is the configuration file for the Coordinator, defining your confidential deployment.
-It's automatically generated from your deployment by the Contrast CLI.
-It currently consists of the following parts:
-
-
Policies: The identities of your Pods, represented by the hashes of their respective runtime policies.
-
Reference Values: The remote attestation reference values for the Kata confidential micro-VM that's the runtime environment of your Pods.
-
WorkloadOwnerKeyDigest: The workload owner's public key digest. Used for authenticating subsequent manifest updates.
Runtime Policies are a mechanism to enable the use of the untrusted Kubernetes API for orchestration while ensuring the confidentiality and integrity of your confidential containers.
-They allow us to enforce the integrity of your containers' runtime environment as defined in your deployment files.
-The runtime policy mechanism is based on the Open Policy Agent (OPA) and translates the Kubernetes deployment YAML into the Rego policy language of OPA.
-The Kata Agent inside the confidential micro-VM then enforces the policy by only acting on permitted requests.
-The Contrast CLI provides the tooling for automatically translating Kubernetes deployment YAML into the Rego policy language of OPA.
Contrast provides an Initializer that handles the remote attestation on the workload side transparently and
-fetches the workload certificate. The Initializer runs as an init container before your workload is started.
-It provides the workload container and the service mesh sidecar with the workload certificates.
Contrast depends on a Kubernetes runtime class, which is installed
-by the node-installer DaemonSet.
-This runtime consists of a containerd runtime plugin, a virtual machine manager (cloud-hypervisor), and a podvm image (IGVM and rootFS).
-The installer takes care of provisioning every node in the cluster so it provides this runtime class.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Policies
-
Kata runtime policies are an integral part of the Confidential Containers preview on AKS.
-They prescribe how a Kubernetes pod must be configured to launch successfully in a confidential VM.
-In Contrast, policies act as a workload identifier: only pods with a policy registered in the manifest receive workload certificates and may participate in the confidential deployment.
-Verification of the Contrast Coordinator and its manifest transitively guarantees that all workloads meet the owner's expectations.
The Kata agent running in the confidential micro-VM exposes an RPC service AgentService to the Kata runtime.
-This service handles potentially untrustworthy requests from outside the TCB, which need to be checked against a policy.
-
Kata runtime policies are written in the policy language Rego.
-They specify what AgentService methods can be called, and the permissible parameters for each call.
-
Policies consist of two parts: a list of rules and a data section.
-While the list of rules is static, the data section is populated with information from the PodSpec and other sources.
Runtime policies are programmatically generated from Kubernetes manifests by the Contrast CLI.
-The generate subcommand inspects pod definitions and derives rules for validating the pod at the Kata agent.
-There are two important integrity checks: container image checksums and OCI runtime parameters.
-
For each of the container images used in a pod, the CLI downloads all image layers and produces a cryptographic dm-verity checksum.
-These checksums are the basis for the policy's storage data.
-
The CLI combines information from the PodSpec, ConfigMaps, and Secrets in the provided Kubernetes manifests to derive a permissible set of command-line arguments and environment variables.
-These constitute the policy's OCI data.
The generated policy document is annotated to the pod definitions in Base64 encoding.
-This annotation is propagated to the Kata runtime, which calculates the SHA256 checksum for the policy and uses that as SNP HOSTDATA for the confidential micro-VM.
-
After the VM launched, the runtime calls the agent's SetPolicy method with the full policy document.
-If the policy doesn't match the checksum in HOSTDATA, the agent rejects the policy.
-Otherwise, it applies the policy to all future AgentService requests.
Contrast verifies its confidential containers following these steps:
-
-
The Contrast CLI generates a policy and attaches it to the pod definition.
-
Kubernetes schedules the pod on a node with the confidential computing runtime.
-
Containerd invokes the Kata runtime to create the pod sandbox.
-
The Kata runtime starts a CVM with the policy's digest as HOSTDATA.
-
The Kata runtime sets the policy using the SetPolicy method.
-
The Kata agent verifies that the incoming policy's digest matches HOSTDATA.
-
The CLI sets a manifest in the Contrast Coordinator, including a list of permitted policies.
-
The Contrast Initializer sends an attestation report to the Contrast Coordinator, asking for a mesh certificate.
-
The Contrast Coordinator verifies that the started pod has a permitted policy hash in its HOSTDATA field.
-
-
After the last step, we know that the policy hasn't been tampered with and, thus, that the workload matches expectations and may receive mesh certificates.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Contrast Runtime
-
The Contrast runtime is responsible for starting pods as confidential virtual machines.
-This works by specifying the runtime class to be used in a pod spec and by registering the runtime class with the apiserver.
-The RuntimeClass resource defines a name for referencing the class and
-a handler used by the container runtime (containerd) to identify the class.
-
apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: # This name is used by pods in the runtimeClassName field name: contrast-cc-abcdef # This name is used by the # container runtime interface implementation (containerd) handler: contrast-cc-abcdef
-
Confidential pods that are part of a Contrast deployment need to specify the
-same runtime class in the runtimeClassName field, so Kubernetes uses the
-Contrast runtime instead of the default containerd / runc handler.
-
apiVersion: v1 kind: Pod spec: runtimeClassName: contrast-cc-abcdef # ...
The runtime consists of additional software components that need to be installed
-and configured on every SEV-SNP-enabled worker node.
-This installation is performed automatically by the node-installer DaemonSet.
The handler field in the Kubernetes RuntimeClass instructs containerd not to use the default runc implementation.
-Instead, containerd invokes a custom plugin called containerd-shim-contrast-cc-v2.
-This shim is described in more detail in the upstream source repository and in the containerd documentation.
The containerd shim uses cloud-hypervisor to create a confidential virtual machine for every pod.
-This requires the cloud-hypervisor binary to be installed on every node (responsibility of the node-installer).
Contrast uses a special containerd snapshotter (tardev) to provide container images as block devices to the pod-VM. This snapshotter consists of a host component that pulls container images and a guest component (kernel module) used to mount container images.
-The tardev snapshotter uses dm-verity to protect the integrity of container images.
-Expected dm-verity container image hashes are part of Contrast runtime policies and are enforced by the kata-agent.
-This enables workload attestation by specifying the allowed container image as part of the policy. Read the chapter on policies for more information.
Every pod-VM starts with the same guest image. It consists of an IGVM file and a root filesystem.
-The IGVM file describes the initial memory contents of a pod-VM and consists of:
-
-
Linux kernel image
-
initrd
-
kernel commandline
-
-
Additionally, a root filesystem image is used that contains a read-only partition with the user space of the pod-VM and a verity partition to guarantee the integrity of the root filesystem.
-The root filesystem contains systemd as the init system, and the kata agent for managing the pod.
-
This pod-VM image isn't specific to any pod workload. Instead, container images are mounted at runtime.
The RuntimeClass resource above registers the runtime with the Kubernetes api.
-The node-level installation is carried out by the Contrast node-installer
-DaemonSet that ships with every Contrast release.
-
After deploying the installer, it performs the following steps on each node:
-
-
Install the Contrast containerd shim (containerd-shim-contrast-cc-v2)
-
Install cloud-hypervisor as the virtual machine manager (VMM)
-
Install an IGVM file for pod-VMs of this class
-
Install a read only root filesystem disk image for the pod-VMs of this class
-
Reconfigure containerd by adding a runtime plugin that corresponds to the handler field of the Kubernetes RuntimeClass
-
Restart containerd to make it aware of the new plugin
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Service Mesh
-
The Contrast service mesh secures the communication of the workload by automatically
-wrapping the network traffic inside mutual TLS (mTLS) connections. The
-verification of the endpoints in the connection establishment is based on
-certificates that are part of the
-PKI of the Coordinator.
-
The service mesh can be enabled on a per-pod basis by adding the service-mesh
-container as a sidecar container.
-The service mesh container first sets up iptables
-rules based on its configuration and then starts Envoy
-for TLS origination and termination.
All TCP ingress traffic is routed over Envoy by default. Since we use
-TPROXY, the destination address
-remains the same throughout the packet handling.
-
Any incoming connection is required to present a client certificate signed by the
-mesh CA certificate.
-Envoy presents a certificate chain of the mesh
-certificate of the workload and the intermediate CA certificate as the server certificate.
-
If the deployment contains workloads which should be reachable from outside the
-Service Mesh, while still handing out the certificate chain, disable client
-authentication by setting the environment variable EDG_INGRESS_PROXY_CONFIG as
-<name>#<port>#false. Separate multiple entries with ##. You can choose any
-descriptive string identifying the service on the given port for the
-informational-only field <name>.
-
Disable redirection and TLS termination altogether by specifying
-<name>#<port>#true. This can be beneficial if the workload itself handles TLS
-on that port or if the information exposed on this port is non-sensitive.
-
The following example workload exposes a web service on port 8080 and metrics on
-port 7890. The web server is exposed to a 3rd party end-user which wants to
-verify the deployment, therefore it's still required that the server hands out
-it certificate chain signed by the mesh CA certificate. The metrics should be
-exposed via TCP without TLS.
To be able to route the egress traffic of the workload through Envoy, the remote
-endpoints' IP address and port must be configurable.
-
-
Choose an IP address inside the 127.0.0.0/8 CIDR and a port not yet in use
-by the pod.
-
Configure the workload to connect to this IP address and port.
-
Set <name>#<chosen IP>:<chosen port>#<original-hostname-or-ip>:<original-port>
-as EDG_EGRESS_PROXY_CONFIG. Separate multiple entries with ##. Choose any
-string identifying the service on the given port as <name>.
-
-
This redirects the traffic over Envoy. The endpoint must present a valid
-certificate chain which must be verifiable with the
-mesh CA certificate.
-Furthermore, Envoy uses a certificate chain with the mesh certificate of the workload
-and the intermediate CA certificate as the client certificate.
-
The following example workload has no ingress connections and two egress
-connection to different microservices. The microservices are themselves part
-of the confidential deployment. One is reachable under billing-svc:8080 and
-the other under cart-svc:8080.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Workload deployment
-
The following instructions will guide you through the process of making an existing Kubernetes deployment
-confidential and deploying it together with Contrast.
-
A running CoCo-enabled cluster is required for these steps, see the setup guide on how to set it up.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Your Kubernetes resources need some modifications to run as Confidential Containers.
-This section guides you through the process and outlines the necessary changes.
Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files
-unchanged, you can copy the files into a separate local directory.
-You can also generate files from a Helm chart or from a Kustomization.
To specify that a workload (pod, deployment, etc.) should be deployed as confidential containers,
-add runtimeClassName: contrast-cc to the pod spec (pod definition or template).
-This is a placeholder name that will be replaced by a versioned runtimeClassName when generating policies.
-In addition, add the Contrast Initializer as initContainers to these workloads and configure the
-workload to use the certificates written to a volumeMount named tls-certs.
The initializer populates the shared volume with X.509 certificates for your workload.
-These certificates are used by the Contrast Service Mesh, but can also be used by your application directly.
-The following tab group explains the setup for both scenarios.
-
Drop-in service mesh
Go integration
Contrast can be configured to handle TLS in a sidecar container.
-This is useful for workloads that are hard to configure with custom certificates, like Java applications.
Configuration of the sidecar depends heavily on the application.
-The following example is for an application with these properties:
-
The app has a main application at TCP port 8001, which should be TLS-wrapped and doesn't require client authentication.
-
The app has a metrics endpoint at TCP port 8080, which should be accessible in plain text.
-
All other endpoints require client authentication.
-
The app connects to a Kubernetes service backend.default:4001, which requires client authentication.
-
Add the following sidecar definition to your workload:
The only change required to the app itself is to let it connect to 127.0.0.2:4001 to reach the backend service.
-You can find more detailed documentation in the Service Mesh chapter.
The mesh certificate contained in certChain.pem authenticates this workload, while the mesh CA certificate mesh-ca.pem authenticates its peers.
-Your app should turn on client authentication to ensure peers are running as confidential containers, too.
-See the Certificate Authority section for detailed information about these certificates.
The following example shows how to configure a Golang app, with error handling omitted for clarity.
Run the generate command to generate the execution policies and add them as annotations to your
-deployment files. A manifest.json with the reference values of your deployment will be created.
-
contrast generate resources/
-
warning
Please be aware that runtime policies currently have some blind spots. For example, they can't guarantee the starting order of containers. See the current limitations for more details.
For the next steps, we will need to connect to the Coordinator. The released Coordinator resource
-includes a LoadBalancer definition we can use.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
-
Port-forwarding of Confidential Containers
kubectl port-forward uses a Container Runtime Interface (CRI) method that isn't supported by the Kata shim.
-If you can't use a public load balancer, you can deploy a port-forwarder.
-The port-forwarder relays traffic from a CoCo pod and can be accessed via kubectl port-forward.
After this step, the Coordinator will start issuing TLS certs to the workloads. The init container
-will fetch a certificate for the workload and the workload is started.
An end user (data owner) can verify the Contrast deployment using the verify command.
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using embedded reference values. The CLI will write the service mesh
-root certificate and the history of manifests into the verify/ directory. In addition, the policies referenced
-in the manifest are also written to the directory.
You can securely connect to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-First, expose the service on a public IP address via a LoadBalancer service:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Confidential emoji voting
-
-
This tutorial guides you through deploying emojivoto as a
-confidential Contrast deployment and validating the deployment from a voters perspective.
-
Emojivoto is an example app allowing users to vote for different emojis and view votes
-on a leader board. It has a microservice architecture consisting of a
-web frontend (web), a gRPC backend for listing available emojis (emoji), and a backend for
-the voting and leader board logic (voting). The vote-bot simulates user traffic by submitting
-votes to the frontend.
Using a voting service, users' votes are considered highly sensitive data, as we require
-a secret ballot. Also, users are likely interested in the fairness of the ballot. For
-both requirements, we can use Confidential Computing and, specifically, workload attestation
-to prove to those interested in voting that the app is running in a protected environment
-where their votes are processed without leaking to the platform provider or workload owner.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Run the generate command to generate the execution policies and add them as
-annotations to your deployment files. A manifest.json file with the reference values
-of your deployment will be created:
-
contrast generate deployment/
-
Runtime class and Initializer
The deployment YAML shipped for this demo is already configured to be used with Contrast.
-A runtime classcontrast-cc-<VERSIONHASH>
-was added to the pods to signal they should be run as Confidential Containers. In addition, the Contrast
-Initializer was added as an init container to these workloads to
-facilitate the attestation and certificate pulling before the actual workload is started.
Further, the deployment YAML is also configured with the Contrast service mesh.
-The configured service mesh proxy provides transparent protection for the communication between
-the different components of emojivoto.
Configure the coordinator with a manifest. It might take up to a few minutes
-for the load balancer to be created and the Coordinator being available.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"The user API of your Contrast Coordinator is available at $coordinator:1313" contrast set-c"${coordinator}:1313" deployment/
-
The CLI will use the embedded reference values to attest the Coordinator deployment
-during the TLS handshake. If the connection succeeds, we're ensured that the Coordinator
-deployment hasn't been tampered with.
Now that the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload,
-we can deploy the application:
-
kubectl apply -f deployment/
-
Inter-deployment communication
The Contrast Coordinator issues mesh certificates after successfully validating workloads.
-These certificates can be used for secure inter-deployment communication. The Initializer
-sends an attestation report to the Coordinator, retrieves certificates and a private key in return
-and writes them to a volumeMount. The service mesh sidecar is configured to use the credentials
-from the volumeMount when communicating with other parts of the deployment over mTLS.
-The public facing frontend for voting uses the mesh certificate without client authentication.
As voters, we want to verify the fairness and confidentiality of the deployment before
-deciding to vote. Regardless of the scale of our distributed deployment, Contrast only
-needs a single remote attestation step to verify the deployment. By doing remote attestation
-of the Coordinator, we transitively verify those systems the Coordinator has already attested
-or will attest in the future. Successful verification of the Coordinator means that
-we can be sure it will enforce the configured manifest.
A potential voter can verify the Contrast deployment using the verify
-command:
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using embedded reference values. If the command succeeds,
-the Coordinator deployment was successfully verified to be running in the expected Confidential
-Computing environment with the expected code version. The Coordinator will then return its
-configuration over the established TLS channel. The CLI will store this information, namely the root
-certificate of the mesh (mesh-ca.pem) and the history of manifests, into the verify/ directory.
-In addition, the policies referenced in the manifest history are also written into the same directory.
In the next step, the Coordinator configuration that was written by the verify command needs to be audited.
-A potential voter should inspect the manifest and the referenced policies. They could delegate
-this task to an entity they trust.
After ensuring the configuration of the Coordinator fits the expectation, you can securely connect
-to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-
To access the web frontend, expose the service on a public IP address via a LoadBalancer service:
-
frontendIP=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"Frontend is available at https://$frontendIP, you can visit it in your browser."
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
-
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
The Policies section of the manifest maps policy hashes to a list of SANs. To enable certificate verification
-of the web frontend with tools like curl, edit the policy with your favorite editor and add the frontendIP to
-the list that already contains the "web" DNS entry:
Next, set the changed manifest at the coordinator with:
-
contrast set-c"${coordinator}:1313" deployment/
-
The Contrast Coordinator will rotate the mesh ca certificate on the manifest update. Workload certificates issued
-after the manifest are thus issued by another certificate authority and services receiving the new CA certificate chain
-won't trust parts of the deployment that got their certificate issued before the update. This way, Contrast ensures
-that parts of the deployment that received a security update won't be infected by parts of the deployment at an older
-patch level that may have been compromised. The mesh-ca.pem is updated with the new CA certificate chain.
The Coordinator has the new manifest set, but the different containers of the app are still
-using the older certificate authority. The Contrast Initializer terminates after the initial attestation
-flow and won't pull new certificates on manifest updates.
After the update has been rolled out, connecting to the frontend using curl will successfully validate
-the service certificate and return the HTML document of the voting site:
CoCo on AKS is currently in preview. An extension for the az CLI is needed to create such a cluster.
-Add the extension with the following commands:
-
az extension add\ --name aks-preview \ --allow-preview true az extension update \ --name aks-preview \ --allow-preview true
-
Then register the required feature flags in your subscription to allow access to the public preview:
-
az feature register \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"
-
The registration can take a few minutes. The status of the operation can be checked with the following
-command, which should show the registration state as Registered:
-
az feature show \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"\ --output table
-
Afterward, refresh the registration of the ContainerService provider:
-
az provider register \ --namespace"Microsoft.ContainerService"
# Select the name for your AKS cluster azClusterName="ContrastDemo" az aks create \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"\ --kubernetes-version 1.29\ --os-sku AzureLinux \ --node-vm-size Standard_DC4as_cc_v5 \ --node-count 1\ --generate-ssh-keys
After trying out Contrast, you might want to clean up the cloud resources created in this step.
-In case you've created a new resource group, you can just delete that group with
-
az group delete \ --name"${azResourceGroup:?}"
-
Deleting the resource group will also delete the cluster and all other related resources.
-
To only cleanup the AKS cluster and node pools, run
-
az aks delete \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.6, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.6
Known Limitations
-
As Contrast is currently in an early development stage, it's built on several projects that are also under active development.
-This section outlines the most significant known limitations, providing stakeholders with clear expectations and understanding of the current state.
Platform Support: At present, Contrast is exclusively available on Azure AKS, supported by the Confidential Container preview for AKS. Expansion to other cloud platforms is planned, pending the availability of necessary infrastructure enhancements.
Resource Limits: There is an existing bug on AKS where container memory limits are incorrectly applied. The current workaround involves using only memory requests instead of limits.
Coverage: While the enforcement of workload policies generally functions well, there are scenarios not yet fully covered. It's crucial to review deployments specifically for these edge cases.
-
Order of events: The current policy evaluation mechanism on API requests isn't stateful, so it can't ensure a prescribed order of events. Consequently, there's no guaranteed enforcement that the service mesh sidecar container runs before the workload container. This order ensures that all traffic between pods is securely encapsulated within TLS connections.
-
Absence of events: Policies can't ensure certain events have happened. A container, such as the service mesh sidecar, can be omitted entirely. Environment variables may be missing.
-
Volume integrity checks: While persistent volumes aren't supported yet, integrity checks don't currently cover other objects such as ConfigMaps and Secrets.
-
-
warning
The policy limitations, in particular the missing guarantee that our service mesh sidecar has been started before the workload container affects the service mesh implementation of Contrast. Currently, this requires inspecting the iptables rules on startup or terminating TLS connections in the workload directly.
CLI Availability: The CLI tool is currently only available for Linux. This limitation arises because certain upstream dependencies haven't yet been ported to other platforms.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Contrast
-
Welcome to the documentation of Contrast! Contrast runs confidential container deployments on Kubernetes at scale.
-
-
Contrast is based on the Kata Containers and
-Confidential Containers projects.
-Confidential Containers are Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation from the surrounding environment.
-This works with unmodified containers in a lift-and-shift approach.
-Contrast currently targets the CoCo preview on AKS.
-
tip
See the 📄whitepaper for more information on confidential computing.
Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer. It removes the infrastructure provider from the trusted computing base (TCB). This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, for example, malicious co-tenants escalating their privileges.
-
Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
CLI telemetry
-
The Contrast CLI sends telemetry data to Edgeless Systems when you use CLI commands.
-This allows to understand how Contrast is used and to improve it.
-
The CLI sends the following data:
-
-
The CLI version
-
The CLI target OS and architecture (GOOS and GOARCH)
-
The command that was run
-
The kind of error that occurred (if any)
-
-
The CLI doesn't collect sensitive information.
-The implementation is open-source and can be reviewed.
-
IP addresses may be processed or stored for security purposes.
-
The data that the CLI collects adheres to the Edgeless Systems privacy policy.
-
You can disable telemetry by setting the environment variable DO_NOT_TRACK=1 before running the CLI.
-
-
\ No newline at end of file
diff --git a/pr-preview/pr-918/0.7/architecture.html b/pr-preview/pr-918/0.7/architecture.html
deleted file mode 100644
index 09ef8964cc..0000000000
--- a/pr-preview/pr-918/0.7/architecture.html
+++ /dev/null
@@ -1,24 +0,0 @@
-
-
-
-
-
-Architecture | Contrast
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Attestation in Contrast
-
This document describes the attestation architecture of Contrast, adhering to the definitions of Remote ATtestation procedureS (RATS) in RFC 9334.
-The following gives a detailed description of Contrast's attestation architecture.
-At the end of this document, we included an FAQ that answers the most common questions regarding attestation in hindsight of the security benefits.
Contrast integrates with the RATS architecture, leveraging their definition of roles and processes including Attesters, Verifiers, and Relying Parties.
-
-
Figure 1: Conceptual attestation architecture. Taken from RFC 9334.
-
-
Attester: Assigned to entities that are responsible for creating Evidence which is then sent to a Verifier.
-
Verifier: These entities utilize the Evidence, Reference Values, and Endorsements. They assess the trustworthiness of the Attester by applying an Appraisal Policy for Evidence. Following this assessment, Verifiers generate Attestation Results for use by Relying Parties. The Appraisal Policy for Evidence may be provided by the Verifier Owner, programmed into the Verifier, or acquired through other means.
-
Relying Party: Assigned to entities that utilize Attestation Results, applying their own appraisal policies to make specific decisions, such as authorization decisions. This process is referred to as the "appraisal of Attestation Results." The Appraisal Policy for Attestation Results might be sourced from the Relying Party Owner, configured by the owner, embedded in the Relying Party, or obtained through other protocols or mechanisms.
This includes all Pods of the Contrast deployment that run inside Confidential Containers and generate cryptographic evidence reflecting their current configuration and state.
-Their evidence is rooted in the hardware measurements from the CPU and their confidential VM environment.
-The details of this evidence are given below in the section on evidence generation and appraisal.
-
-
Figure 2: Attestation flow of a confidential pod. Based on the layered attester graphic in RFC 9334.
-
Pods run in Contrast's runtime environment (B), effectively within a confidential VM.
-During launch, the CPU (A) measures the initial memory content of the confidential VM that contains Contrast's pod-VM image and generates the corresponding attestation evidence.
-The image is in IGVM format, encapsulating all information required to launch a virtual machine, including the kernel, the initramfs, and kernel cmdline.
-The kernel cmdline contains the root hash for dm-verity that ensures the integrity of the root filesystem.
-The root filesystem contains all components of the container's runtime environment including the guest agent (C).
-
In the userland, the guest agent takes care of enforcing the runtime policy of the pod.
-While the policy is passed in during the initialization procedure via the host, the evidence for the runtime policy is part of the CPU measurements.
-During the deployment the policy is annotated to the Kubernetes Pod resources.
-On AMD SEV-SNP the hash of the policy is then added to the attestation report via the HOSTDATA field by the hypervisor.
-When provided with the policy from the Kata host, the guest agent verifies that the policy's hash matches the one in the HOSTDATA field.
-
In summary a Pod's evidence is the attestation report of the CPU that provides evidence for runtime environment and the runtime policy.
The Coordinator acts as a verifier within the Contrast deployment, configured with a Manifest that defines the reference values and serves as an appraisal policy for all pods in the deployment.
-It also pulls endorsements from hardware vendors to verify the hardware claims.
-The Coordinator operates within the cluster as a confidential container and provides similar evidence as any other Pod when it acts as an attester.
-In RATS terminology, the Coordinator's dual role is defined as a lead attester in a composite device which spans the entire deployment: Coordinator and the workload pods.
-It collects evidence from other attesters and conveys it to a verifier, generating evidence about the layout of the whole composite device based on the Manifest as the appraisal policy.
-
-
Figure 3: Contrast deployment as a composite device. Based on the composite device in RFC 9334.
-
The CLI serves as the verifier for the Coordinator and the entire Contrast deployment, containing the reference values for the Coordinator and the endorsements from hardware vendors.
-These reference values are built into the CLI during our release process and can be reproduced offline via reproducible builds.
A relying party in the Contrast scenario could be, for example, the data owner that interacts with the application.
-The relying party can use the CLI to obtain the attestation results and Contrast's CA certificates bound to these results.
-The CA certificates can then be used by the relying party to authenticate the application, for example through TLS connections.
In Contrast, attestation evidence revolves around a hardware-generated attestation report, which contains several critical pieces of information:
-
-
The hardware attestation report: This report includes details such as the chip identifier, platform information, microcode versions, and comprehensive guest measurements. The entire report is signed by the CPU's private key, ensuring the authenticity and integrity of the data provided.
-
The launch measurements: Included within the hardware attestation report, this is a digest generated by the CPU that represents a hash of all initial guest memory pages. This includes essential components like the kernel, initramfs, and the kernel command line. Notably, it incorporates the root filesystem's dm-verity root hash, verifying the integrity of the root filesystem.
-
The runtime policy hash: Also part of the hardware attestation report, this field contains the hash of the Rego policy which dictates all expected API commands and their values from the host to the Kata guest agent. It encompasses crucial settings such as dm-verity hashes for the container image layers, environment variables, and mount points.
The appraisal of this evidence in Contrast is governed by two main components:
-
-
The Manifest: A JSON file used by the Coordinator to align with reference values. It sets the expectations for runtime policy hashes for each pod and includes what should be reported in the hardware attestation report for each component of the deployment.
-
The CLI's appraisal policy: This policy encompasses expected values of the Coordinator’s guest measurements and its runtime policy. It's embedded into the CLI during the build process and ensures that any discrepancy between the built-in values and those reported by the hardware attestation can be identified and addressed. The integrity of this policy is safeguardable through reproducible builds, allowing verification against the source code reference.
-
-
Frequently asked questions about attestation in Contrast
-
What's the purpose of remote attestation in Contrast?
-
Remote attestation in Contrast ensures that software runs within a secure, isolated confidential computing environment.
-This process certifies that the memory is encrypted and confirms the integrity and authenticity of the software running within the deployment.
-By validating the runtime environment and the policies enforced on it, Contrast ensures that the system operates in a trustworthy state and hasn't been tampered with.
-
How does Contrast ensure the security of the attestation process?
-
Contrast leverages hardware-rooted security features such as AMD SEV-SNP to generate cryptographic evidence of a pod’s current state and configuration.
-This evidence is checked against pre-defined appraisal policies to guarantee that only verified and authorized pods are part of a Contrast deployment.
Attestation confirms the integrity of the runtime environment and the identity of the workloads.
-It plays a critical role in preventing unauthorized changes and detecting potential modifications at runtime.
-The attestation provides integrity and authenticity guarantees, enabling relying parties—such as workload operators or data owners—to confirm the effective protection against potential threats, including malicious cloud insiders, co-tenants, or compromised workload operators.
-More details on the specific security benefits can be found here.
-
How can you verify the authenticity of attestation results?
-
Attestation results in Contrast are tied to cryptographic proofs generated and signed by the hardware itself.
-These proofs are then verified using public keys from trusted hardware vendors, ensuring that the results aren't only accurate but also resistant to tampering.
-For further authenticity verification, all of Contrast's code is reproducibly built, and the attestation evidence can be verified locally from the source code.
-
How are attestation results used by relying parties?
-
Relying parties use attestation results to make informed security decisions, such as allowing access to sensitive data or resources only if the attestation verifies the system's integrity.
-Thereafter, the use of Contrast's CA certificates in TLS connections provides a practical approach to communicate securely with the application.
In summary, Contrast's attestation strategy adheres to the RATS guidelines and consists of robust verification mechanisms that ensure each component of the deployment is secure and trustworthy.
-This comprehensive approach allows Contrast to provide a high level of security assurance to its users.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Certificate authority
-
The Coordinator acts as a certificate authority (CA) for the workloads
-defined in the manifest.
-After a workload pod's attestation has been verified by the Coordinator,
-it receives a mesh certificate and the mesh CA certificate.
-The mesh certificate can be used for example in a TLS connection as the server or
-client certificate to proof to the other party that the workload has been
-verified by the Coordinator. The other party can verify the mesh certificate
-with the mesh CA certificate. While the certificates can be used by the workload
-developer in different ways, they're automatically used in Contrast's service
-mesh to establish mTLS connections between workloads in the same deployment.
The Coordinator establishes a public key infrastructure (PKI) for all workloads
-contained in the manifest. The Coordinator holds three certificates: the root CA
-certificate, the intermediate CA certificate, and the mesh CA certificate.
-The root CA certificate is a long-lasting certificate and its private key signs
-the intermediate CA certificate. The intermediate CA certificate and the mesh CA
-certificate share the same private key. This intermediate private key is used
-to sign the mesh certificates. Moreover, the intermediate private key and
-therefore the intermediate CA certificate and the mesh CA certificate are
-rotated when setting a new manifest.
Depending on the configuration of the first manifest, it allows the workload
-owner to update the manifest and, therefore, the deployment.
-Workload owners and data owners can be mutually untrusted parties.
-To protect against the workload owner silently introducing malicious containers,
-the Coordinator rotates the intermediate private key every time the manifest is
-updated and, therefore, the
-intermediate CA certificate and mesh CA certificate. If the user doesn't
-trust the workload owner, they use the mesh CA certificate obtained when they
-verified the Coordinator and the manifest. This ensures that the user only
-connects to workloads defined in the manifest they verified since only those
-workloads' certificates are signed with this intermediate private key.
-
Similarly, the service mesh also uses the mesh CA certificate obtained when the
-workload was started, so the workload only trusts endpoints that have been
-verified by the Coordinator based on the same manifest. Consequently, a
-manifest update requires a fresh rollout of the services in the service mesh.
The root CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This should only be used if the data owner trusts all future updates to the
-manifest and workloads. This is, for instance, the case when the workload owner is
-the same entity as the data owner.
-
The mesh CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This certificate is bound to the manifest set when the Coordinator is verified.
-If the manifest is updated, the mesh CA certificate changes.
-New workloads will receive mesh certificates signed by the new mesh CA certificate.
-The Coordinator with the new manifest needs to be verified to retrieve the new mesh CA certificate.
-The service mesh also uses the mesh CA certificate to verify the mesh certificates.
-
The intermediate CA certificate links the root CA certificate to the
-mesh certificate so that the mesh certificate can be verified with the root CA
-certificate. It's part of the certificate chain handed out by
-endpoints in the service mesh.
-
The mesh certificate is part of the certificate chain handed out by
-endpoints in the service mesh. During the startup of a pod, the Initializer
-requests a certificate from the Coordinator. This mesh certificate will be returned if the Coordinator successfully
-verifies the workload. The mesh certificate
-contains X.509 extensions with information from the workloads attestation
-document.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Observability
-
The Contrast Coordinator can expose metrics in the
-Prometheus format. These can be monitored to quickly
-identify problems in the gRPC layer or attestation errors. Prometheus metrics
-are numerical values associated with a name and additional key/values pairs,
-called labels.
The metrics can be accessed at the Coordinator pod at the port specified in the
-CONTRAST_METRICS_PORT environment variable under the /metrics endpoint. By
-default, this environment variable isn't specified, hence no metrics will be
-exposed.
-
The Coordinator starts two gRPC servers, one for the user API on port 1313 and
-one for the mesh API on port 7777. Metrics for both servers can be accessed
-using different prefixes.
-
All metric names for the user API are prefixed with contrast_userapi_grpc_server_.
-Exposed metrics include the number of handled requests of the methods
-SetManifest and GetManifest, which get called when setting the
-manifest and verifying the
-Coordinator respectively. For each method
-you can see the gRPC status code indicating whether the request succeeded or
-not and the request latency.
-
For the mesh API, the metric names are prefixed with contrast_meshapi_grpc_server_. The
-metrics include similar data to the user API for the method NewMeshCert which
-gets called by the Initializer when starting a
-new workload. Attestation failures from workloads to the Coordinator can be
-tracked with the counter contrast_meshapi_attestation_failures.
-
The current manifest generation is exposed as a
-gauge with the metric
-name contrast_coordinator_manifest_generation. If no manifest is set at the
-Coordinator, this counter will be zero.
The Service Mesh can be configured to expose
-metrics via its Envoy admin
-interface. Be
-aware that the admin interface can expose private information and allows
-destructive operations to be performed. To enable the admin interface for the
-Service Mesh, set the annotation
-contrast.edgeless.systems/servicemesh-admin-interface-port in the configuration
-of your workload. If this annotation is set, the admin interface will be started
-on this port.
-
To access the admin interface, the ingress settings of the Service Mesh have to
-be configured to allow access to the specified port (see Configuring the
-Proxy). All metrics will be
-exposed under the /stats endpoint. Metrics in Prometheus format can be scraped
-from the /stats/prometheus endpoint.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Confidential Containers
-
Contrast uses some building blocks from Confidential Containers (CoCo), a CNCF Sandbox project that aims to standardize confidential computing at the pod level.
-The project is under active development and many of the high-level features are still in flux.
-Contrast uses the more stable core primitive provided by CoCo: its Kubernetes runtime.
Kubernetes can be extended to use more than one container runtime with RuntimeClass objects.
-The Container Runtime Interface (CRI) implementation, for example containerd, dispatches pod management API calls to the appropriate RuntimeClass.
-RuntimeClass implementations are usually based on an OCI runtime, such as runc, runsc or crun.
-In CoCo's case, the runtime is Kata Containers with added confidential computing capabilities.
Kata Containers is an OCI runtime that runs pods in VMs.
-The pod VM spawns an agent process that accepts management commands from the Kata runtime running on the host.
-There are two options for creating pod VMs: local to the Kubernetes node, or remote VMs created with cloud provider APIs.
-Using local VMs requires either bare metal servers or VMs with support for nested virtualization.
-Local VMs communicate with the host over a virtual socket.
-For remote VMs, host-to-agent communication is tunnelled through the cloud provider's network.
-
Kata Containers was originally designed to isolate the guest from the host, but it can also run pods in confidential VMs (CVMs) to shield pods from their underlying infrastructure.
-In confidential mode, the guest agent is configured with an Open Policy Agent (OPA) policy to authorize API calls from the host.
-This policy also contains checksums for the expected container images.
-It's derived from Kubernetes resource definitions and its checksum is included in the attestation report.
Azure Kubernetes Service (AKS) provides CoCo-enabled node pools as a preview offering.
-These node pools leverage Azure VM types capable of nested virtualization (CVM-in-VM) and the CoCo stack is pre-installed.
-Contrast can be deployed directly into a CoCo-enabled AKS cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Product features
-
Contrast simplifies the deployment and management of Confidential Containers, offering optimal data security for your workloads while integrating seamlessly with your existing Kubernetes environment.
-
From a security perspective, Contrast employs the Confidential Containers concept and provides security benefits that go beyond individual containers, shielding your entire deployment from the underlying infrastructure.
-
From an operational perspective, Contrast provides the following key features:
-
-
-
Managed Kubernetes compatibility: Initially compatible with Azure Kubernetes Service (AKS), Contrast is designed to support additional platforms such as AWS EKS and Google Cloud GKE as they begin to accommodate confidential containers.
-
-
-
Lightweight installation: Contrast can be integrated as a day-2 operation within existing clusters, adding minimal components to your setup. This facilitates straightforward deployments using your existing YAML configurations, Helm charts, or Kustomization, enabling native Kubernetes orchestration of your applications.
-
-
-
Remote attestation: Contrast generates a concise attestation statement that verifies the identity, authenticity, and integrity of your deployment both internally and to external parties. This architecture ensures that updates or scaling of the application don't compromise the attestation’s validity.
-
-
-
Service mesh: Contrast securely manages a Public Key Infrastructure (PKI) for your deployments, issues workload-specific certificates, and establishes transparent mutual TLS (mTLS) connections across pods. This is done by harnessing the envoy proxy to ensure secure communications within your Kubernetes cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Contrast security overview
-
This document outlines the security measures of Contrast and its capability to counter various threats.
-Contrast is designed to shield entire Kubernetes deployments from the infrastructure, enabling entities to manage sensitive information (such as regulated or personally identifiable information (PII)) in the public cloud, while maintaining data confidentiality and ownership.
-
Contrast is applicable in situations where establishing trust with the workload operator or the underlying infrastructure is challenging.
-This is particularly beneficial for regulated sectors looking to transition sensitive activities to the cloud, without sacrificing security or compliance.
-It allows for cloud adoption by maintaining a hardware-based separation from the cloud service provider.
Leveraging Confidential Computing technology, Contrast provides three defining security properties:
-
-
Encryption of data in use: Contrast ensures that all data processed in memory is encrypted, making it inaccessible to unauthorized users or systems, even if they have physical access to the hardware.
-
Workload isolation: Each pod runs in its isolated runtime environment, preventing any cross-contamination between workloads, which is critical for multi-tenant infrastructures.
-
Remote attestation: This feature allows data owners and workload operators to verify that the Contrast environment executing their workloads hasn't been tampered with and is running in a secure, pre-approved configuration.
-
-
The runtime encryption is transparently provided by the confidential computing hardware during the workload's lifetime.
-The workload isolation and remote attestation involves two phases:
-
-
An attestation process detects modifications to the workload image or its runtime environment during the initialization. This protects the workload's integrity pre-attestation.
-
A protected runtime environment and a policy mechanism prevents the platform operator from accessing or compromising the instance at runtime. This protects a workload's integrity and confidentiality post-attestation.
-
-
For more details on confidential computing see our whitepaper.
-The attestation architecture describes Contrast's attestation process and the resulting chain of trust in detail.
Contrast uses the Kubernetes runtime of the Confidential Containers project.
-Confidential Containers significantly decrease the size of the trusted computing base (TCB) of a Kubernetes deployment, by isolating each pod within its own confidential micro-VM environment.
-The TCB is the totality of elements in a computing environment that must be trusted not to be compromised.
-A smaller TCB results in a smaller attack surface. The following diagram shows how Confidential Containers remove the cloud & datacenter infrastructure and the physical hosts, including the hypervisor, the host OS, the Kubernetes control plane, and other components, from the TCB (red).
-In the confidential context, depicted in green, only the workload containers along with their confidential micro-VM environment are included within the TCB.
-Their integrity is verifiable through remote attestation.
-
Contrast uses hardware-based mechanisms, specifically leveraging CPU features, such as AMD SEV or Intel TDX, to provide the isolation of the workload.
-This implies that both the CPU and its microcode are integral components of the TCB.
-However, it should be noted that the CPU microcode aspects aren't depicted in the accompanying graphic.
-
-
Contrast adds the following components to a deployment that become part of the TCB.
-The components that are part of the TCB are:
-
-
The workload containers: Container images that run the actual application.
The runtime policies: Policies that enforce the runtime environments for the workload containers during their lifetime.
-
The manifest: A manifest file defining the reference values of an entire confidential deployment. It contains the policy hashes for all pods of the deployment and the expected hardware reference values for the Confidential Container runtime.
-
The Coordinator: An attestation service that runs in a Confidential Container in the Kubernetes cluster. The Coordinator is configured with the manifest. User-facing, you can verify this service and the effective manifest using remote attestation, providing you with a concise attestation for the entire deployment. Cluster-facing, it verifies all pods and their policies based on remote attestation procedures and the manifest.
In a Contrast deployment, there are three parties:
-
-
-
The container image provider, who creates the container images that represent the application that has access to the protected data.
-
-
-
The workload operator, who runs the workload in a Kubernetes cluster. The operator typically has full administrative privileges to the deployment. The operator can manage cluster resources such as nodes, volumes, and networking rules, and the operator can interact with any Kubernetes or underlying cloud API.
-
-
-
The data owner, who owns the protected data. A data owner can verify the deployment using the Coordinator attestation service. The verification includes the identity, integrity, and confidentiality of the workloads, the runtime environment and the access permissions.
-
-
-
Contrast supports a trust model where the container image provider, workload operator, and data owner are separate, mutually distrusting parties.
-
The following diagram shows the system components and parties.
Contrast is designed to defend against five possible attacks:
-
-
A malicious cloud insider: malicious employees or third-party contractors of cloud service providers (CSPs) potentially have full access to various layers of the cloud infrastructure. That goes from the physical datacenter up to the hypervisor and Kubernetes layer. For example, they can access the physical memory of the machines, modify the hypervisor, modify disk contents, intercept network communications, and attempt to compromise the confidential container at runtime. A malicious insider can expand the attack surface or restrict the runtime environment. For example, a malicious operator can add a storage device to introduce new attack vectors. As another example, a malicious operator can constrain resources such as limiting a guest's memory size, changing its disk space, or changing firewall rules.
-
A malicious cloud co-tenant: malicious cloud user ("hackers") may break out of their tenancy and access other tenants' data. Advanced attackers may even be able to establish a permanent foothold within the infrastructure and access data over a longer period. The threats are analogous to the cloud insider access scenario, without the physical access.
-
A malicious workload operator: malicious workload operators, for example Kubernetes administrators, have full access to the workload deployment and the underlying Kubernetes platform. The threats are analogously to the cloud insider access scenario, with access to everything that's above the hypervisor level.
-
A malicious attestation client: this attacker connects to the attestation service and sends malformed request.
-
A malicious container image provider: a malicious container image provider has full control over the application development itself. This attacker might release a malicious version of the workload containing harmful operations.
Contrast shields a workload from the aforementioned threats with three main components:
-
-
The runtime environment safeguards against the physical memory and disk attack surface.
-
The runtime policies safeguard against the Kubernetes control plane and container runtime attack surface.
-
The service mesh safeguards against the network attack surface.
-
-
The following tables describe concrete threats and how they're mitigated in Contrast grouped by these categories:
-
-
Attacks on the confidential container environment
-
Attacks on the attestation service
-
Attacks on workloads
-
-
Attacks on the confidential container environment
-
This table describes potential threats and mitigation strategies related to the confidential container environment.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the network connection of the launcher or image repository.
An attacker can change the image URL and control the workload binary. However these actions are reflected in the attestation report. The image repository isn't controlled using an access list, therefore the image is assumed to be viewable by everyone. You must ensure that the workload container image doesn't contain any secrets.
An attacker modifies a container's runtime environment configuration in the Kubernetes control plane.
The attestation process and the runtime policies detects unsafe configurations that load non-authentic images or perform any other modification to the expected runtime environment.
This table describes potential threats and mitigation strategies to the attestation service.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the Coordinator deployment and modifies the image or hijacks the runtime environment.
This threat is mitigated by having an attestation procedure and attested, encrypted TLS connections to the Coordinator. The attestation evidence for the Coordinator image is distributed with our releases, protected by supply chain security, and fully reproducible.
An attacker intercepts the network connection between the workload and the Coordinator and reads secret keys from the wire.
This threat is mitigated by having an attested, encrypted TLS connection. This connection helps protect the secrets from passive eavesdropping. The attacker can't create valid workload certificates that would be accepted in Contrast's service mesh. An attacker can't impersonate a valid workload container because the container's identity is guaranteed by the attestation protocol.
Within the network between your workload and the Coordinator.
An attacker exploits parsing discrepancies, which leads to undetected changes in the attestation process.
This risk is mitigated by having a parsing engine written in memory-safe Go that's tested against the attestation specification of the hardware vendor. The runtime policies are available as an attestation artifact for further inspection and audits to verify their effectiveness.
An attacker reads or modifies data written to disk via persistent volumes.
Currently, persistent volumes aren't supported in Contrast. In the future, this threat is mitigated by encrypted and integrity-protected volume mounts.
An attacker publishes a new image version containing malicious code.
The attestation process and the runtime policies require a data owner to accept a specific version of the workload and any update to the workload needs to be explicitly acknowledged.
The following table describes three example use cases and how they map to the defined threat model in this document:
-
Use Case
Example Scenario
Migrate sensitive workloads to the cloud
TechSolve Inc., a software development firm, aimed to enhance its defense-in-depth strategy for its cloud-based development environment, especially for projects involving proprietary algorithms and client data. TechSolve acts as the image provider, workload operator, and data owner, combining all three personas in this scenario. In our attestation terminology, they're the workload operator and relying party in one entity. Their threat model includes a malicious cloud insider and cloud co-tenant.
Make your SaaS more trustworthy
SaaSProviderX, a company offering cloud-based project management tools, sought to enhance its platform's trustworthiness amidst growing concerns about data breaches and privacy. Here, the relying party is the SaaS customer as the data owner. The goal is to achieve a form of operator exclusion and only allow selective operations on the deployment. Hence, their threat model includes a malicious workload operator.
Simplify regulatory compliance
HealthSecure Inc. has been managing a significant volume of sensitive patient data on-premises. With the increasing demand for advanced data analytics and the need for scalable infrastructure, the firm decides to migrate its data analytics operations to the cloud. However, the primary concern is maintaining the confidentiality and security of patient data during and after the migration, in compliance with healthcare regulations. In this compliance scenario, the regulator serves as an additional relying party. HealthSecure must implement a mechanism that ensures the isolation of patient data can be verifiably guaranteed to the regulator.
-
In each scenario, Contrast ensures exclusive data access and processing capabilities are confined to the designated workloads. It achieves this by effectively isolating the workload from the infrastructure and other components of the stack. Data owners are granted the capability to audit and approve the deployment environment before submitting their data, ensuring a secure handover. Meanwhile, workload operators are equipped to manage and operate the application seamlessly, without requiring direct access to either the workload or its associated data.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Components
-
Contrast is composed of several key components that work together to manage and scale confidential containers effectively within Kubernetes environments.
-This page provides an overview of the core components essential for deploying and managing Contrast.
The CLI serves as the primary management tool for Contrast deployments. It's designed to streamline the configuration and operation of Contrast in several ways:
-
-
Installation and setup: The CLI facilitates the installation of the necessary runtime classes required for Contrast to function within a Kubernetes cluster.
-
Policy generation: It allows users to generate runtime policies, adapt the deployment files, and generate the Contrast manifest.
-
Configuration management: Through the CLI, users can configure the Contrast Coordinator with the generated manifest.
-
Verification and attestation: The CLI provides tools to verify the integrity and authenticity of the Coordinator and the entire deployment via remote attestation.
The Contrast Coordinator is the central remote attestation service of a Contrast deployment.
-It runs inside a confidential container inside your cluster.
-The Coordinator can be verified via remote attestation, and a Contrast deployment is self-contained.
-The Coordinator is configured with a manifest, a configuration file containing the reference attestation values of your deployment.
-It ensures that your deployment's topology adheres to your specified manifest by verifying the identity and integrity of all confidential pods inside the deployment.
-The Coordinator is also a certificate authority and issues certificates for your workload pods during the attestation procedure.
-Your workload pods can establish secure, encrypted communication channels between themselves based on these certificates using the Coordinator as the root CA.
-As your app needs to scale, the Coordinator transparently verifies new instances and then provides them with their certificates to join the deployment.
-
To verify your deployment, the Coordinator's remote attestation statement combined with the manifest offers a concise single remote attestation statement for your entire deployment.
-A third party can use this to verify the integrity of your distributed app, making it easy to assure stakeholders of your app's identity and integrity.
The manifest is the configuration file for the Coordinator, defining your confidential deployment.
-It's automatically generated from your deployment by the Contrast CLI.
-It currently consists of the following parts:
-
-
Policies: The identities of your Pods, represented by the hashes of their respective runtime policies.
-
Reference Values: The remote attestation reference values for the Kata confidential micro-VM that's the runtime environment of your Pods.
-
WorkloadOwnerKeyDigest: The workload owner's public key digest. Used for authenticating subsequent manifest updates.
Runtime Policies are a mechanism to enable the use of the untrusted Kubernetes API for orchestration while ensuring the confidentiality and integrity of your confidential containers.
-They allow us to enforce the integrity of your containers' runtime environment as defined in your deployment files.
-The runtime policy mechanism is based on the Open Policy Agent (OPA) and translates the Kubernetes deployment YAML into the Rego policy language of OPA.
-The Kata Agent inside the confidential micro-VM then enforces the policy by only acting on permitted requests.
-The Contrast CLI provides the tooling for automatically translating Kubernetes deployment YAML into the Rego policy language of OPA.
Contrast provides an Initializer that handles the remote attestation on the workload side transparently and
-fetches the workload certificate. The Initializer runs as an init container before your workload is started.
-It provides the workload container and the service mesh sidecar with the workload certificates.
Contrast depends on a Kubernetes runtime class, which is installed
-by the node-installer DaemonSet.
-This runtime consists of a containerd runtime plugin, a virtual machine manager (cloud-hypervisor), and a podvm image (IGVM and rootFS).
-The installer takes care of provisioning every node in the cluster so it provides this runtime class.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Policies
-
Kata runtime policies are an integral part of the Confidential Containers preview on AKS.
-They prescribe how a Kubernetes pod must be configured to launch successfully in a confidential VM.
-In Contrast, policies act as a workload identifier: only pods with a policy registered in the manifest receive workload certificates and may participate in the confidential deployment.
-Verification of the Contrast Coordinator and its manifest transitively guarantees that all workloads meet the owner's expectations.
The Kata agent running in the confidential micro-VM exposes an RPC service AgentService to the Kata runtime.
-This service handles potentially untrustworthy requests from outside the TCB, which need to be checked against a policy.
-
Kata runtime policies are written in the policy language Rego.
-They specify what AgentService methods can be called, and the permissible parameters for each call.
-
Policies consist of two parts: a list of rules and a data section.
-While the list of rules is static, the data section is populated with information from the PodSpec and other sources.
Runtime policies are programmatically generated from Kubernetes manifests by the Contrast CLI.
-The generate subcommand inspects pod definitions and derives rules for validating the pod at the Kata agent.
-There are two important integrity checks: container image checksums and OCI runtime parameters.
-
For each of the container images used in a pod, the CLI downloads all image layers and produces a cryptographic dm-verity checksum.
-These checksums are the basis for the policy's storage data.
-
The CLI combines information from the PodSpec, ConfigMaps, and Secrets in the provided Kubernetes manifests to derive a permissible set of command-line arguments and environment variables.
-These constitute the policy's OCI data.
The generated policy document is annotated to the pod definitions in Base64 encoding.
-This annotation is propagated to the Kata runtime, which calculates the SHA256 checksum for the policy and uses that as SNP HOSTDATA for the confidential micro-VM.
-
After the VM launched, the runtime calls the agent's SetPolicy method with the full policy document.
-If the policy doesn't match the checksum in HOSTDATA, the agent rejects the policy.
-Otherwise, it applies the policy to all future AgentService requests.
Contrast verifies its confidential containers following these steps:
-
-
The Contrast CLI generates a policy and attaches it to the pod definition.
-
Kubernetes schedules the pod on a node with the confidential computing runtime.
-
Containerd invokes the Kata runtime to create the pod sandbox.
-
The Kata runtime starts a CVM with the policy's digest as HOSTDATA.
-
The Kata runtime sets the policy using the SetPolicy method.
-
The Kata agent verifies that the incoming policy's digest matches HOSTDATA.
-
The CLI sets a manifest in the Contrast Coordinator, including a list of permitted policies.
-
The Contrast Initializer sends an attestation report to the Contrast Coordinator, asking for a mesh certificate.
-
The Contrast Coordinator verifies that the started pod has a permitted policy hash in its HOSTDATA field.
-
-
After the last step, we know that the policy hasn't been tampered with and, thus, that the workload matches expectations and may receive mesh certificates.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Contrast Runtime
-
The Contrast runtime is responsible for starting pods as confidential virtual machines.
-This works by specifying the runtime class to be used in a pod spec and by registering the runtime class with the apiserver.
-The RuntimeClass resource defines a name for referencing the class and
-a handler used by the container runtime (containerd) to identify the class.
-
apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: # This name is used by pods in the runtimeClassName field name: contrast-cc-abcdef # This name is used by the # container runtime interface implementation (containerd) handler: contrast-cc-abcdef
-
Confidential pods that are part of a Contrast deployment need to specify the
-same runtime class in the runtimeClassName field, so Kubernetes uses the
-Contrast runtime instead of the default containerd / runc handler.
-
apiVersion: v1 kind: Pod spec: runtimeClassName: contrast-cc-abcdef # ...
The runtime consists of additional software components that need to be installed
-and configured on every SEV-SNP-enabled worker node.
-This installation is performed automatically by the node-installer DaemonSet.
The handler field in the Kubernetes RuntimeClass instructs containerd not to use the default runc implementation.
-Instead, containerd invokes a custom plugin called containerd-shim-contrast-cc-v2.
-This shim is described in more detail in the upstream source repository and in the containerd documentation.
The containerd shim uses cloud-hypervisor to create a confidential virtual machine for every pod.
-This requires the cloud-hypervisor binary to be installed on every node (responsibility of the node-installer).
Contrast uses a special containerd snapshotter (tardev) to provide container images as block devices to the pod-VM. This snapshotter consists of a host component that pulls container images and a guest component (kernel module) used to mount container images.
-The tardev snapshotter uses dm-verity to protect the integrity of container images.
-Expected dm-verity container image hashes are part of Contrast runtime policies and are enforced by the kata-agent.
-This enables workload attestation by specifying the allowed container image as part of the policy. Read the chapter on policies for more information.
Every pod-VM starts with the same guest image. It consists of an IGVM file and a root filesystem.
-The IGVM file describes the initial memory contents of a pod-VM and consists of:
-
-
Linux kernel image
-
initrd
-
kernel commandline
-
-
Additionally, a root filesystem image is used that contains a read-only partition with the user space of the pod-VM and a verity partition to guarantee the integrity of the root filesystem.
-The root filesystem contains systemd as the init system, and the kata agent for managing the pod.
-
This pod-VM image isn't specific to any pod workload. Instead, container images are mounted at runtime.
The RuntimeClass resource above registers the runtime with the Kubernetes api.
-The node-level installation is carried out by the Contrast node-installer
-DaemonSet that ships with every Contrast release.
-
After deploying the installer, it performs the following steps on each node:
-
-
Install the Contrast containerd shim (containerd-shim-contrast-cc-v2)
-
Install cloud-hypervisor as the virtual machine manager (VMM)
-
Install an IGVM file for pod-VMs of this class
-
Install a read only root filesystem disk image for the pod-VMs of this class
-
Reconfigure containerd by adding a runtime plugin that corresponds to the handler field of the Kubernetes RuntimeClass
-
Restart containerd to make it aware of the new plugin
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Service mesh
-
The Contrast service mesh secures the communication of the workload by automatically
-wrapping the network traffic inside mutual TLS (mTLS) connections. The
-verification of the endpoints in the connection establishment is based on
-certificates that are part of the
-PKI of the Coordinator.
-
The service mesh can be enabled on a per-workload basis by adding a service mesh
-configuration to the workload's object annotations. During the contrast generate
-step, the service mesh is added as a sidecar
-container to
-all workloads which have a specified configuration. The service mesh container first
-sets up iptables rules based on its configuration and then starts
-Envoy for TLS origination and termination.
The service mesh container can be configured using the following object annotations:
-
-
contrast.edgeless.systems/servicemesh-ingress to configure ingress.
-
contrast.edgeless.systems/servicemesh-egress to configure egress.
-
contrast.edgeless.systems/servicemesh-admin-interface-port to configure the Envoy
-admin interface. If not specified, no admin interface will be started.
-
-
If you aren't using the automatic service mesh injection and want to configure the
-service mesh manually, set the environment variables EDG_INGRESS_PROXY_CONFIG,
-EDG_EGRESS_PROXY_CONFIG and EDG_ADMIN_PORT in the service mesh sidecar directly.
All TCP ingress traffic is routed over Envoy by default. Since we use
-TPROXY, the destination address
-remains the same throughout the packet handling.
-
Any incoming connection is required to present a client certificate signed by the
-mesh CA certificate.
-Envoy presents a certificate chain of the mesh
-certificate of the workload and the intermediate CA certificate as the server certificate.
-
If the deployment contains workloads which should be reachable from outside the
-Service Mesh, while still handing out the certificate chain, disable client
-authentication by setting the annotation contrast.edgeless.systems/servicemesh-ingress as
-<name>#<port>#false. Separate multiple entries with ##. You can choose any
-descriptive string identifying the service on the given port for the <name> field,
-as it's only informational.
-
Disable redirection and TLS termination altogether by specifying
-<name>#<port>#true. This can be beneficial if the workload itself handles TLS
-on that port or if the information exposed on this port is non-sensitive.
-
The following example workload exposes a web service on port 8080 and metrics on
-port 7890. The web server is exposed to a 3rd party end-user which wants to
-verify the deployment, therefore it's still required that the server hands out
-it certificate chain signed by the mesh CA certificate. The metrics should be
-exposed via TCP without TLS.
Note, that changing the environment variables of the sidecar container directly will
-only have an effect if the workload isn't configured to automatically generate a
-service mesh component on contrast generate. Otherwise, the service mesh sidecar
-container will be regenerated on every invocation of the command.
To be able to route the egress traffic of the workload through Envoy, the remote
-endpoints' IP address and port must be configurable.
-
-
Choose an IP address inside the 127.0.0.0/8 CIDR and a port not yet in use
-by the pod.
-
Configure the workload to connect to this IP address and port.
-
Set <name>#<chosen IP>:<chosen port>#<original-hostname-or-ip>:<original-port>
-as the contrast.edgeless.systems/servicemesh-egress workload annotation. Separate multiple
-entries with ##. Choose any string identifying the service on the given port as
-<name>.
-
-
This redirects the traffic over Envoy. The endpoint must present a valid
-certificate chain which must be verifiable with the
-mesh CA certificate.
-Furthermore, Envoy uses a certificate chain with the mesh certificate of the workload
-and the intermediate CA certificate as the client certificate.
-
The following example workload has no ingress connections and two egress
-connection to different microservices. The microservices are part
-of the confidential deployment. One is reachable under billing-svc:8080 and
-the other under cart-svc:8080.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Workload deployment
-
The following instructions will guide you through the process of making an existing Kubernetes deployment
-confidential and deploying it together with Contrast.
-
A running CoCo-enabled cluster is required for these steps, see the setup guide on how to set it up.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Your Kubernetes resources need some modifications to run as Confidential Containers.
-This section guides you through the process and outlines the necessary changes.
Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files
-unchanged, you can copy the files into a separate local directory.
-You can also generate files from a Helm chart or from a Kustomization.
To specify that a workload (pod, deployment, etc.) should be deployed as confidential containers,
-add runtimeClassName: contrast-cc to the pod spec (pod definition or template).
-This is a placeholder name that will be replaced by a versioned runtimeClassName when generating policies.
In the initialization process, the contrast-tls-certs shared volume is populated with X.509 certificates for your workload.
-These certificates are used by the Contrast Service Mesh, but can also be used by your application directly.
-The following tab group explains the setup for both scenarios.
-
Drop-in service mesh
Go integration
Contrast can be configured to handle TLS in a sidecar container.
-This is useful for workloads that are hard to configure with custom certificates, like Java applications.
Configuration of the sidecar depends heavily on the application.
-The following example is for an application with these properties:
-
The container has a main application at TCP port 8001, which should be TLS-wrapped and doesn't require client authentication.
-
The container has a metrics endpoint at TCP port 8080, which should be accessible in plain text.
-
All other endpoints require client authentication.
-
The app connects to a Kubernetes service backend.default:4001, which requires client authentication.
During the generate step, this configuration will be translated into a Service Mesh sidecar container which handles TLS connections automatically.
-The only change required to the app itself is to let it connect to 127.0.0.2:4001 to reach the backend service.
-You can find more detailed documentation in the Service Mesh chapter.
The mesh certificate contained in certChain.pem authenticates this workload, while the mesh CA certificate mesh-ca.pem authenticates its peers.
-Your app should turn on client authentication to ensure peers are running as confidential containers, too.
-See the Certificate Authority section for detailed information about these certificates.
The following example shows how to configure a Golang app, with error handling omitted for clarity.
Run the generate command to add the necessary components to your deployment files.
-This will add the Contrast Initializer to every workload with the specified contrast-cc runtime class
-and the Contrast Service Mesh to all workloads that have a specified configuration.
-After that, it will generate the execution policies and add them as annotations to your deployment files.
-A manifest.json with the reference values of your deployment will be created.
-
contrast generate resources/
-
warning
Please be aware that runtime policies currently have some blind spots. For example, they can't guarantee the starting order of containers. See the current limitations for more details.
-
If you don't want the Contrast Initializer to automatically be added to your
-workloads, there are two ways you can skip the Initializer injection step,
-depending on how you want to customize your deployment.
-
Command-line flag
Per-workload annotation
You can disable the Initializer injection completely by specifying the
---skip-initializer flag in the generate command.
contrast generate --skip-initializer resources/
If you want to disable the Initializer injection for a specific workload with
-the contrast-cc runtime class, you can do so by adding an annotation to the workload.
When disabling the automatic Initializer injection, you can manually add the
-Initializer as a sidecar container to your workload before generating the
-policies. Configure the workload to use the certificates written to the
-contrast-tls-certsvolumeMount.
For the next steps, we will need to connect to the Coordinator. The released Coordinator resource
-includes a LoadBalancer definition we can use.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
-
Port-forwarding of Confidential Containers
kubectl port-forward uses a Container Runtime Interface (CRI) method that isn't supported by the Kata shim.
-If you can't use a public load balancer, you can deploy a port-forwarder.
-The port-forwarder relays traffic from a CoCo pod and can be accessed via kubectl port-forward.
After this step, the Coordinator will start issuing TLS certificates to the workloads. The init container
-will fetch a certificate for the workload and the workload is started.
An end user (data owner) can verify the Contrast deployment using the verify command.
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using embedded reference values. The CLI will write the service mesh
-root certificate and the history of manifests into the verify/ directory. In addition, the policies referenced
-in the manifest are also written to the directory.
You can securely connect to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-First, expose the service on a public IP address via a LoadBalancer service:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.7, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.7
Confidential emoji voting
-
-
This tutorial guides you through deploying emojivoto as a
-confidential Contrast deployment and validating the deployment from a voters perspective.
-
Emojivoto is an example app allowing users to vote for different emojis and view votes
-on a leader board. It has a microservice architecture consisting of a
-web frontend (web), a gRPC backend for listing available emojis (emoji), and a backend for
-the voting and leader board logic (voting). The vote-bot simulates user traffic by submitting
-votes to the frontend.
Using a voting service, users' votes are considered highly sensitive data, as we require
-a secret ballot. Also, users are likely interested in the fairness of the ballot. For
-both requirements, we can use Confidential Computing and, specifically, workload attestation
-to prove to those interested in voting that the app is running in a protected environment
-where their votes are processed without leaking to the platform provider or workload owner.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Run the generate command to generate the execution policies and add them as
-annotations to your deployment files. A manifest.json file with the reference values
-of your deployment will be created:
-
contrast generate deployment/
-
Runtime class and Initializer
The deployment YAML shipped for this demo is already configured to be used with Contrast.
-A runtime classcontrast-cc-<VERSIONHASH>
-was added to the pods to signal they should be run as Confidential Containers. During the generation process,
-the Contrast Initializer will be added as an init container to these
-workloads to facilitate the attestation and certificate pulling before the actual workload is started.
Further, the deployment YAML is also configured with the Contrast service mesh.
-The configured service mesh proxy provides transparent protection for the communication between
-the different components of emojivoto.
Configure the coordinator with a manifest. It might take up to a few minutes
-for the load balancer to be created and the Coordinator being available.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"The user API of your Contrast Coordinator is available at $coordinator:1313" contrast set-c"${coordinator}:1313" deployment/
-
The CLI will use the embedded reference values to attest the Coordinator deployment
-during the TLS handshake. If the connection succeeds, it's ensured that the Coordinator
-deployment hasn't been tampered with.
Now that the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload,
-we can deploy the application:
-
kubectl apply -f deployment/
-
Inter-deployment communication
The Contrast Coordinator issues mesh certificates after successfully validating workloads.
-These certificates can be used for secure inter-deployment communication. The Initializer
-sends an attestation report to the Coordinator, retrieves certificates and a private key in return
-and writes them to a volumeMount. The service mesh sidecar is configured to use the credentials
-from the volumeMount when communicating with other parts of the deployment over mTLS.
-The public facing frontend for voting uses the mesh certificate without client authentication.
As voters, we want to verify the fairness and confidentiality of the deployment before
-deciding to vote. Regardless of the scale of our distributed deployment, Contrast only
-needs a single remote attestation step to verify the deployment. By doing remote attestation
-of the Coordinator, we transitively verify those systems the Coordinator has already attested
-or will attest in the future. Successful verification of the Coordinator means that
-we can be sure it will enforce the configured manifest.
A potential voter can verify the Contrast deployment using the verify
-command:
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using embedded reference values. If the command succeeds,
-the Coordinator deployment was successfully verified to be running in the expected Confidential
-Computing environment with the expected code version. The Coordinator will then return its
-configuration over the established TLS channel. The CLI will store this information, namely the root
-certificate of the mesh (mesh-ca.pem) and the history of manifests, into the verify/ directory.
-In addition, the policies referenced in the manifest history are also written into the same directory.
In the next step, the Coordinator configuration that was written by the verify command needs to be audited.
-A potential voter should inspect the manifest and the referenced policies. They could delegate
-this task to an entity they trust.
After ensuring the configuration of the Coordinator fits the expectation, you can securely connect
-to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-
To access the web frontend, expose the service on a public IP address via a LoadBalancer service:
-
frontendIP=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"Frontend is available at https://$frontendIP, you can visit it in your browser."
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the subject alternative name (SAN) field.
-Validation fails since the certificate contains no IP entries as a SAN.
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
-
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
The Policies section of the manifest maps policy hashes to a list of SANs. To enable certificate verification
-of the web frontend with tools like curl, edit the policy with your favorite editor and add the frontendIP to
-the list that already contains the "web" DNS entry:
Next, set the changed manifest at the coordinator with:
-
contrast set-c"${coordinator}:1313" deployment/
-
The Contrast Coordinator will rotate the mesh ca certificate on the manifest update. Workload certificates issued
-after the manifest update are thus issued by another certificate authority and services receiving the new CA certificate chain
-won't trust parts of the deployment that got their certificate issued before the update. This way, Contrast ensures
-that parts of the deployment that received a security update won't be infected by parts of the deployment at an older
-patch level that may have been compromised. The mesh-ca.pem is updated with the new CA certificate chain.
The Coordinator has the new manifest set, but the different containers of the app are still
-using the older certificate authority. The Contrast Initializer terminates after the initial attestation
-flow and won't pull new certificates on manifest updates.
After the update has been rolled out, connecting to the frontend using curl will successfully validate
-the service certificate and return the HTML document of the voting site:
Platform support: At present, Contrast is exclusively available on Azure AKS, supported by the Confidential Container preview for AKS. Expansion to other cloud platforms is planned, pending the availability of necessary infrastructure enhancements.
-
Bare-metal support: Support for running Contrast on bare-metal Kubernetes will be available soon for AMD SEV and Intel TDX.
Resource limits: There is an existing bug on AKS where container memory limits are incorrectly applied. The current workaround involves using only memory requests instead of limits.
Coverage: While the enforcement of workload policies generally functions well, there are scenarios not yet fully covered. It's crucial to review deployments specifically for these edge cases.
-
Order of events: The current policy evaluation mechanism on API requests isn't stateful, so it can't ensure a prescribed order of events. Consequently, there's no guaranteed enforcement that the service mesh sidecar container runs before the workload container. This order ensures that all traffic between pods is securely encapsulated within TLS connections.
-
Absence of events: Policies can't ensure certain events have happened. A container, such as the service mesh sidecar, can be omitted entirely. Environment variables may be missing.
-
Volume integrity checks: While persistent volumes aren't supported yet, integrity checks don't currently cover other objects such as ConfigMaps and Secrets.
-
-
warning
The policy limitations, in particular the missing guarantee that our service mesh sidecar has been started before the workload container affects the service mesh implementation of Contrast. Currently, this requires inspecting the iptables rules on startup or terminating TLS connections in the workload directly.
CLI availability: The CLI tool is currently only available for Linux. This limitation arises because certain upstream dependencies haven't yet been ported to other platforms.
CoCo on AKS is currently in preview. An extension for the az CLI is needed to create such a cluster.
-Add the extension with the following commands:
-
az extension add\ --name aks-preview \ --allow-preview true az extension update \ --name aks-preview \ --allow-preview true
-
Then register the required feature flags in your subscription to allow access to the public preview:
-
az feature register \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"
-
The registration can take a few minutes. The status of the operation can be checked with the following
-command, which should show the registration state as Registered:
-
az feature show \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"\ --output table
-
Afterward, refresh the registration of the ContainerService provider:
-
az provider register \ --namespace"Microsoft.ContainerService"
First, we need to create an AKS cluster. We can't directly create a CoCo-enabled cluster, so we'll need to create a
-non-CoCo cluster first, and then add a CoCo node pool, optionally replacing the non-CoCo node pool.
-
We'll first start by creating the non-CoCo cluster:
-
# Select the name for your AKS cluster azClusterName="ContrastDemo" az aks create \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"\ --kubernetes-version 1.29\ --os-sku AzureLinux \ --node-vm-size Standard_DC4as_cc_v5 \ --node-count 1\ --generate-ssh-keys
After trying out Contrast, you might want to clean up the cloud resources created in this step.
-In case you've created a new resource group, you can just delete that group with
-
az group delete \ --name"${azResourceGroup:?}"
-
Deleting the resource group will also delete the cluster and all other related resources.
-
To only cleanup the AKS cluster and node pools, run
-
az aks delete \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Contrast
-
Welcome to the documentation of Contrast! Contrast runs confidential container deployments on Kubernetes at scale.
-
-
Contrast is based on the Kata Containers and
-Confidential Containers projects.
-Confidential Containers are Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation from the surrounding environment.
-This works with unmodified containers in a lift-and-shift approach.
-Contrast currently targets the CoCo preview on AKS.
-
tip
See the 📄whitepaper for more information on confidential computing.
Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer. It removes the infrastructure provider from the trusted computing base (TCB). This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, for example, malicious co-tenants escalating their privileges.
-
Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
CLI telemetry
-
The Contrast CLI sends telemetry data to Edgeless Systems when you use CLI commands.
-This allows to understand how Contrast is used and to improve it.
-
The CLI sends the following data:
-
-
The CLI version
-
The CLI target OS and architecture (GOOS and GOARCH)
-
The command that was run
-
The kind of error that occurred (if any)
-
-
The CLI doesn't collect sensitive information.
-The implementation is open-source and can be reviewed.
-
IP addresses may be processed or stored for security purposes.
-
The data that the CLI collects adheres to the Edgeless Systems privacy policy.
-
You can disable telemetry by setting the environment variable DO_NOT_TRACK=1 before running the CLI.
-
-
\ No newline at end of file
diff --git a/pr-preview/pr-918/0.8/architecture/attestation.html b/pr-preview/pr-918/0.8/architecture/attestation.html
deleted file mode 100644
index 3ed880130e..0000000000
--- a/pr-preview/pr-918/0.8/architecture/attestation.html
+++ /dev/null
@@ -1,104 +0,0 @@
-
-
-
-
-
-Attestation in Contrast | Contrast
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Attestation in Contrast
-
This document describes the attestation architecture of Contrast, adhering to the definitions of Remote ATtestation procedureS (RATS) in RFC 9334.
-The following gives a detailed description of Contrast's attestation architecture.
-At the end of this document, we included an FAQ that answers the most common questions regarding attestation in hindsight of the security benefits.
Contrast integrates with the RATS architecture, leveraging their definition of roles and processes including Attesters, Verifiers, and Relying Parties.
-
-
Figure 1: Conceptual attestation architecture. Taken from RFC 9334.
-
-
Attester: Assigned to entities that are responsible for creating Evidence which is then sent to a Verifier.
-
Verifier: These entities utilize the Evidence, Reference Values, and Endorsements. They assess the trustworthiness of the Attester by applying an Appraisal Policy for Evidence. Following this assessment, Verifiers generate Attestation Results for use by Relying Parties. The Appraisal Policy for Evidence may be provided by the Verifier Owner, programmed into the Verifier, or acquired through other means.
-
Relying Party: Assigned to entities that utilize Attestation Results, applying their own appraisal policies to make specific decisions, such as authorization decisions. This process is referred to as the "appraisal of Attestation Results." The Appraisal Policy for Attestation Results might be sourced from the Relying Party Owner, configured by the owner, embedded in the Relying Party, or obtained through other protocols or mechanisms.
This includes all Pods of the Contrast deployment that run inside Confidential Containers and generate cryptographic evidence reflecting their current configuration and state.
-Their evidence is rooted in the hardware measurements from the CPU and their confidential VM environment.
-The details of this evidence are given below in the section on evidence generation and appraisal.
-
-
Figure 2: Attestation flow of a confidential pod. Based on the layered attester graphic in RFC 9334.
-
Pods run in Contrast's runtime environment (B), effectively within a confidential VM.
-During launch, the CPU (A) measures the initial memory content of the confidential VM that contains Contrast's pod-VM image and generates the corresponding attestation evidence.
-The image is in IGVM format, encapsulating all information required to launch a virtual machine, including the kernel, the initramfs, and kernel cmdline.
-The kernel cmdline contains the root hash for dm-verity that ensures the integrity of the root filesystem.
-The root filesystem contains all components of the container's runtime environment including the guest agent (C).
-
In the userland, the guest agent takes care of enforcing the runtime policy of the pod.
-While the policy is passed in during the initialization procedure via the host, the evidence for the runtime policy is part of the CPU measurements.
-During the deployment the policy is annotated to the Kubernetes Pod resources.
-On AMD SEV-SNP the hash of the policy is then added to the attestation report via the HOSTDATA field by the hypervisor.
-When provided with the policy from the Kata host, the guest agent verifies that the policy's hash matches the one in the HOSTDATA field.
-
In summary a Pod's evidence is the attestation report of the CPU that provides evidence for runtime environment and the runtime policy.
The Coordinator acts as a verifier within the Contrast deployment, configured with a Manifest that defines the reference values and serves as an appraisal policy for all pods in the deployment.
-It also pulls endorsements from hardware vendors to verify the hardware claims.
-The Coordinator operates within the cluster as a confidential container and provides similar evidence as any other Pod when it acts as an attester.
-In RATS terminology, the Coordinator's dual role is defined as a lead attester in a composite device which spans the entire deployment: Coordinator and the workload pods.
-It collects evidence from other attesters and conveys it to a verifier, generating evidence about the layout of the whole composite device based on the Manifest as the appraisal policy.
-
-
Figure 3: Contrast deployment as a composite device. Based on the composite device in RFC 9334.
-
The CLI serves as the verifier for the Coordinator and the entire Contrast deployment, containing the reference values for the Coordinator and the endorsements from hardware vendors.
-These reference values are built into the CLI during our release process and can be reproduced offline via reproducible builds.
A relying party in the Contrast scenario could be, for example, the data owner that interacts with the application.
-The relying party can use the CLI to obtain the attestation results and Contrast's CA certificates bound to these results.
-The CA certificates can then be used by the relying party to authenticate the application, for example through TLS connections.
In Contrast, attestation evidence revolves around a hardware-generated attestation report, which contains several critical pieces of information:
-
-
The hardware attestation report: This report includes details such as the chip identifier, platform information, microcode versions, and comprehensive guest measurements. The entire report is signed by the CPU's private key, ensuring the authenticity and integrity of the data provided.
-
The launch measurements: Included within the hardware attestation report, this is a digest generated by the CPU that represents a hash of all initial guest memory pages. This includes essential components like the kernel, initramfs, and the kernel command line. Notably, it incorporates the root filesystem's dm-verity root hash, verifying the integrity of the root filesystem.
-
The runtime policy hash: Also part of the hardware attestation report, this field contains the hash of the Rego policy which dictates all expected API commands and their values from the host to the Kata guest agent. It encompasses crucial settings such as dm-verity hashes for the container image layers, environment variables, and mount points.
The appraisal of this evidence in Contrast is governed by two main components:
-
-
The Manifest: A JSON file used by the Coordinator to align with reference values. It sets the expectations for runtime policy hashes for each pod and includes what should be reported in the hardware attestation report for each component of the deployment.
-
The CLI's appraisal policy: This policy encompasses expected values of the Coordinator’s guest measurements and its runtime policy. It's embedded into the CLI during the build process and ensures that any discrepancy between the built-in values and those reported by the hardware attestation can be identified and addressed. The integrity of this policy is safeguardable through reproducible builds, allowing verification against the source code reference.
-
-
Frequently asked questions about attestation in Contrast
-
What's the purpose of remote attestation in Contrast?
-
Remote attestation in Contrast ensures that software runs within a secure, isolated confidential computing environment.
-This process certifies that the memory is encrypted and confirms the integrity and authenticity of the software running within the deployment.
-By validating the runtime environment and the policies enforced on it, Contrast ensures that the system operates in a trustworthy state and hasn't been tampered with.
-
How does Contrast ensure the security of the attestation process?
-
Contrast leverages hardware-rooted security features such as AMD SEV-SNP to generate cryptographic evidence of a pod’s current state and configuration.
-This evidence is checked against pre-defined appraisal policies to guarantee that only verified and authorized pods are part of a Contrast deployment.
Attestation confirms the integrity of the runtime environment and the identity of the workloads.
-It plays a critical role in preventing unauthorized changes and detecting potential modifications at runtime.
-The attestation provides integrity and authenticity guarantees, enabling relying parties—such as workload operators or data owners—to confirm the effective protection against potential threats, including malicious cloud insiders, co-tenants, or compromised workload operators.
-More details on the specific security benefits can be found here.
-
How can you verify the authenticity of attestation results?
-
Attestation results in Contrast are tied to cryptographic proofs generated and signed by the hardware itself.
-These proofs are then verified using public keys from trusted hardware vendors, ensuring that the results aren't only accurate but also resistant to tampering.
-For further authenticity verification, all of Contrast's code is reproducibly built, and the attestation evidence can be verified locally from the source code.
-
How are attestation results used by relying parties?
-
Relying parties use attestation results to make informed security decisions, such as allowing access to sensitive data or resources only if the attestation verifies the system's integrity.
-Thereafter, the use of Contrast's CA certificates in TLS connections provides a practical approach to communicate securely with the application.
In summary, Contrast's attestation strategy adheres to the RATS guidelines and consists of robust verification mechanisms that ensure each component of the deployment is secure and trustworthy.
-This comprehensive approach allows Contrast to provide a high level of security assurance to its users.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Certificate authority
-
The Coordinator acts as a certificate authority (CA) for the workloads
-defined in the manifest.
-After a workload pod's attestation has been verified by the Coordinator,
-it receives a mesh certificate and the mesh CA certificate.
-The mesh certificate can be used for example in a TLS connection as the server or
-client certificate to proof to the other party that the workload has been
-verified by the Coordinator. The other party can verify the mesh certificate
-with the mesh CA certificate. While the certificates can be used by the workload
-developer in different ways, they're automatically used in Contrast's service
-mesh to establish mTLS connections between workloads in the same deployment.
The Coordinator establishes a public key infrastructure (PKI) for all workloads
-contained in the manifest. The Coordinator holds three certificates: the root CA
-certificate, the intermediate CA certificate, and the mesh CA certificate.
-The root CA certificate is a long-lasting certificate and its private key signs
-the intermediate CA certificate. The intermediate CA certificate and the mesh CA
-certificate share the same private key. This intermediate private key is used
-to sign the mesh certificates. Moreover, the intermediate private key and
-therefore the intermediate CA certificate and the mesh CA certificate are
-rotated when setting a new manifest.
Depending on the configuration of the first manifest, it allows the workload
-owner to update the manifest and, therefore, the deployment.
-Workload owners and data owners can be mutually untrusted parties.
-To protect against the workload owner silently introducing malicious containers,
-the Coordinator rotates the intermediate private key every time the manifest is
-updated and, therefore, the
-intermediate CA certificate and mesh CA certificate. If the user doesn't
-trust the workload owner, they use the mesh CA certificate obtained when they
-verified the Coordinator and the manifest. This ensures that the user only
-connects to workloads defined in the manifest they verified since only those
-workloads' certificates are signed with this intermediate private key.
-
Similarly, the service mesh also uses the mesh CA certificate obtained when the
-workload was started, so the workload only trusts endpoints that have been
-verified by the Coordinator based on the same manifest. Consequently, a
-manifest update requires a fresh rollout of the services in the service mesh.
The root CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This should only be used if the data owner trusts all future updates to the
-manifest and workloads. This is, for instance, the case when the workload owner is
-the same entity as the data owner.
-
The mesh CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This certificate is bound to the manifest set when the Coordinator is verified.
-If the manifest is updated, the mesh CA certificate changes.
-New workloads will receive mesh certificates signed by the new mesh CA certificate.
-The Coordinator with the new manifest needs to be verified to retrieve the new mesh CA certificate.
-The service mesh also uses the mesh CA certificate to verify the mesh certificates.
-
The intermediate CA certificate links the root CA certificate to the
-mesh certificate so that the mesh certificate can be verified with the root CA
-certificate. It's part of the certificate chain handed out by
-endpoints in the service mesh.
-
The mesh certificate is part of the certificate chain handed out by
-endpoints in the service mesh. During the startup of a pod, the Initializer
-requests a certificate from the Coordinator. This mesh certificate will be returned if the Coordinator successfully
-verifies the workload. The mesh certificate
-contains X.509 extensions with information from the workloads attestation
-document.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Observability
-
The Contrast Coordinator can expose metrics in the
-Prometheus format. These can be monitored to quickly
-identify problems in the gRPC layer or attestation errors. Prometheus metrics
-are numerical values associated with a name and additional key/values pairs,
-called labels.
The metrics can be accessed at the Coordinator pod at the port specified in the
-CONTRAST_METRICS_PORT environment variable under the /metrics endpoint. By
-default, this environment variable isn't specified, hence no metrics will be
-exposed.
-
The Coordinator exports gRPC metrics under the prefix contrast_grpc_server_.
-These metrics are labeled with the gRPC service name and method name.
-Metrics of interest include contrast_grpc_server_handled_total, which counts
-the number of requests by return code, and
-contrast_grpc_server_handling_seconds_bucket, which produces a histogram of
-request latency.
The meshapi.MeshAPI service records metrics for the method NewMeshCert, which
-gets called by the Initializer when starting a
-new workload. Attestation failures from workloads to the Coordinator can be
-tracked with the counter contrast_meshapi_attestation_failures.
-
The current manifest generation is exposed as a
-gauge with the metric
-name contrast_coordinator_manifest_generation. If no manifest is set at the
-Coordinator, this counter will be zero.
The Service Mesh can be configured to expose
-metrics via its Envoy admin
-interface. Be
-aware that the admin interface can expose private information and allows
-destructive operations to be performed. To enable the admin interface for the
-Service Mesh, set the annotation
-contrast.edgeless.systems/servicemesh-admin-interface-port in the configuration
-of your workload. If this annotation is set, the admin interface will be started
-on this port.
-
To access the admin interface, the ingress settings of the Service Mesh have to
-be configured to allow access to the specified port (see Configuring the
-Proxy). All metrics will be
-exposed under the /stats endpoint. Metrics in Prometheus format can be scraped
-from the /stats/prometheus endpoint.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Secrets & recovery
-
When the Coordinator is configured with the initial manifest, it generates a random secret seed.
-From this seed, it uses an HKDF to derive the CA root key and a signing key for the manifest history.
-This derivation is deterministic, so the seed can be used to bring any Coordinator to this Coordinator's state.
-
The secret seed is returned to the user on the first call to contrast set, encrypted with the user's public seed share owner key.
-If no seed share owner key is provided, a key is generated and stored in the working directory.
The Coordinator runs as a StatefulSet with a dynamically provisioned persistent volume.
-This volume stores the manifest history and the associated runtime policies.
-The manifest isn't considered sensitive information, because it needs to be passed to the untrusted infrastructure in order to start workloads.
-However, the Coordinator must ensure its integrity and that the persisted data corresponds to the manifests set by authorized users.
-Thus, the manifest is stored in plain text, but is signed with a private key derived from the Coordinator's secret seed.
When a Coordinator starts up, it doesn't have access to the signing secret and can thus not verify the integrity of the persisted manifests.
-It needs to be provided with the secret seed, from which it can derive the signing key that verifies the signatures.
-This procedure is called recovery and is initiated by the workload owner.
-The CLI decrypts the secret seed using the private seed share owner key, verifies the Coordinator and sends the seed through the Recover method.
-The Coordinator recovers its key material and verifies the manifest history signature.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Confidential Containers
-
Contrast uses some building blocks from Confidential Containers (CoCo), a CNCF Sandbox project that aims to standardize confidential computing at the pod level.
-The project is under active development and many of the high-level features are still in flux.
-Contrast uses the more stable core primitive provided by CoCo: its Kubernetes runtime.
Kubernetes can be extended to use more than one container runtime with RuntimeClass objects.
-The Container Runtime Interface (CRI) implementation, for example containerd, dispatches pod management API calls to the appropriate RuntimeClass.
-RuntimeClass implementations are usually based on an OCI runtime, such as runc, runsc or crun.
-In CoCo's case, the runtime is Kata Containers with added confidential computing capabilities.
Kata Containers is an OCI runtime that runs pods in VMs.
-The pod VM spawns an agent process that accepts management commands from the Kata runtime running on the host.
-There are two options for creating pod VMs: local to the Kubernetes node, or remote VMs created with cloud provider APIs.
-Using local VMs requires either bare metal servers or VMs with support for nested virtualization.
-Local VMs communicate with the host over a virtual socket.
-For remote VMs, host-to-agent communication is tunnelled through the cloud provider's network.
-
Kata Containers was originally designed to isolate the guest from the host, but it can also run pods in confidential VMs (CVMs) to shield pods from their underlying infrastructure.
-In confidential mode, the guest agent is configured with an Open Policy Agent (OPA) policy to authorize API calls from the host.
-This policy also contains checksums for the expected container images.
-It's derived from Kubernetes resource definitions and its checksum is included in the attestation report.
Azure Kubernetes Service (AKS) provides CoCo-enabled node pools as a preview offering.
-These node pools leverage Azure VM types capable of nested virtualization (CVM-in-VM) and the CoCo stack is pre-installed.
-Contrast can be deployed directly into a CoCo-enabled AKS cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Product features
-
Contrast simplifies the deployment and management of Confidential Containers, offering optimal data security for your workloads while integrating seamlessly with your existing Kubernetes environment.
-
From a security perspective, Contrast employs the Confidential Containers concept and provides security benefits that go beyond individual containers, shielding your entire deployment from the underlying infrastructure.
-
From an operational perspective, Contrast provides the following key features:
-
-
-
Managed Kubernetes compatibility: Initially compatible with Azure Kubernetes Service (AKS), Contrast is designed to support additional platforms such as AWS EKS and Google Cloud GKE as they begin to accommodate confidential containers.
-
-
-
Lightweight installation: Contrast can be integrated as a day-2 operation within existing clusters, adding minimal components to your setup. This facilitates straightforward deployments using your existing YAML configurations, Helm charts, or Kustomization, enabling native Kubernetes orchestration of your applications.
-
-
-
Remote attestation: Contrast generates a concise attestation statement that verifies the identity, authenticity, and integrity of your deployment both internally and to external parties. This architecture ensures that updates or scaling of the application don't compromise the attestation’s validity.
-
-
-
Service mesh: Contrast securely manages a Public Key Infrastructure (PKI) for your deployments, issues workload-specific certificates, and establishes transparent mutual TLS (mTLS) connections across pods. This is done by harnessing the envoy proxy to ensure secure communications within your Kubernetes cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Contrast security overview
-
This document outlines the security measures of Contrast and its capability to counter various threats.
-Contrast is designed to shield entire Kubernetes deployments from the infrastructure, enabling entities to manage sensitive information (such as regulated or personally identifiable information (PII)) in the public cloud, while maintaining data confidentiality and ownership.
-
Contrast is applicable in situations where establishing trust with the workload operator or the underlying infrastructure is challenging.
-This is particularly beneficial for regulated sectors looking to transition sensitive activities to the cloud, without sacrificing security or compliance.
-It allows for cloud adoption by maintaining a hardware-based separation from the cloud service provider.
Leveraging Confidential Computing technology, Contrast provides three defining security properties:
-
-
Encryption of data in use: Contrast ensures that all data processed in memory is encrypted, making it inaccessible to unauthorized users or systems, even if they have physical access to the hardware.
-
Workload isolation: Each pod runs in its isolated runtime environment, preventing any cross-contamination between workloads, which is critical for multi-tenant infrastructures.
-
Remote attestation: This feature allows data owners and workload operators to verify that the Contrast environment executing their workloads hasn't been tampered with and is running in a secure, pre-approved configuration.
-
-
The runtime encryption is transparently provided by the confidential computing hardware during the workload's lifetime.
-The workload isolation and remote attestation involves two phases:
-
-
An attestation process detects modifications to the workload image or its runtime environment during the initialization. This protects the workload's integrity pre-attestation.
-
A protected runtime environment and a policy mechanism prevents the platform operator from accessing or compromising the instance at runtime. This protects a workload's integrity and confidentiality post-attestation.
-
-
For more details on confidential computing see our whitepaper.
-The attestation architecture describes Contrast's attestation process and the resulting chain of trust in detail.
Contrast uses the Kubernetes runtime of the Confidential Containers project.
-Confidential Containers significantly decrease the size of the trusted computing base (TCB) of a Kubernetes deployment, by isolating each pod within its own confidential micro-VM environment.
-The TCB is the totality of elements in a computing environment that must be trusted not to be compromised.
-A smaller TCB results in a smaller attack surface. The following diagram shows how Confidential Containers remove the cloud & datacenter infrastructure and the physical hosts, including the hypervisor, the host OS, the Kubernetes control plane, and other components, from the TCB (red).
-In the confidential context, depicted in green, only the workload containers along with their confidential micro-VM environment are included within the TCB.
-Their integrity is verifiable through remote attestation.
-
Contrast uses hardware-based mechanisms, specifically leveraging CPU features, such as AMD SEV or Intel TDX, to provide the isolation of the workload.
-This implies that both the CPU and its microcode are integral components of the TCB.
-However, it should be noted that the CPU microcode aspects aren't depicted in the accompanying graphic.
-
-
Contrast adds the following components to a deployment that become part of the TCB.
-The components that are part of the TCB are:
-
-
The workload containers: Container images that run the actual application.
The runtime policies: Policies that enforce the runtime environments for the workload containers during their lifetime.
-
The manifest: A manifest file defining the reference values of an entire confidential deployment. It contains the policy hashes for all pods of the deployment and the expected hardware reference values for the Confidential Container runtime.
-
The Coordinator: An attestation service that runs in a Confidential Container in the Kubernetes cluster. The Coordinator is configured with the manifest. User-facing, you can verify this service and the effective manifest using remote attestation, providing you with a concise attestation for the entire deployment. Cluster-facing, it verifies all pods and their policies based on remote attestation procedures and the manifest.
In a Contrast deployment, there are three parties:
-
-
-
The container image provider, who creates the container images that represent the application that has access to the protected data.
-
-
-
The workload operator, who runs the workload in a Kubernetes cluster. The operator typically has full administrative privileges to the deployment. The operator can manage cluster resources such as nodes, volumes, and networking rules, and the operator can interact with any Kubernetes or underlying cloud API.
-
-
-
The data owner, who owns the protected data. A data owner can verify the deployment using the Coordinator attestation service. The verification includes the identity, integrity, and confidentiality of the workloads, the runtime environment and the access permissions.
-
-
-
Contrast supports a trust model where the container image provider, workload operator, and data owner are separate, mutually distrusting parties.
-
The following diagram shows the system components and parties.
Contrast is designed to defend against five possible attacks:
-
-
A malicious cloud insider: malicious employees or third-party contractors of cloud service providers (CSPs) potentially have full access to various layers of the cloud infrastructure. That goes from the physical datacenter up to the hypervisor and Kubernetes layer. For example, they can access the physical memory of the machines, modify the hypervisor, modify disk contents, intercept network communications, and attempt to compromise the confidential container at runtime. A malicious insider can expand the attack surface or restrict the runtime environment. For example, a malicious operator can add a storage device to introduce new attack vectors. As another example, a malicious operator can constrain resources such as limiting a guest's memory size, changing its disk space, or changing firewall rules.
-
A malicious cloud co-tenant: malicious cloud user ("hackers") may break out of their tenancy and access other tenants' data. Advanced attackers may even be able to establish a permanent foothold within the infrastructure and access data over a longer period. The threats are analogous to the cloud insider access scenario, without the physical access.
-
A malicious workload operator: malicious workload operators, for example Kubernetes administrators, have full access to the workload deployment and the underlying Kubernetes platform. The threats are analogously to the cloud insider access scenario, with access to everything that's above the hypervisor level.
-
A malicious attestation client: this attacker connects to the attestation service and sends malformed request.
-
A malicious container image provider: a malicious container image provider has full control over the application development itself. This attacker might release a malicious version of the workload containing harmful operations.
Contrast shields a workload from the aforementioned threats with three main components:
-
-
The runtime environment safeguards against the physical memory and disk attack surface.
-
The runtime policies safeguard against the Kubernetes control plane and container runtime attack surface.
-
The service mesh safeguards against the network attack surface.
-
-
The following tables describe concrete threats and how they're mitigated in Contrast grouped by these categories:
-
-
Attacks on the confidential container environment
-
Attacks on the attestation service
-
Attacks on workloads
-
-
Attacks on the confidential container environment
-
This table describes potential threats and mitigation strategies related to the confidential container environment.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the network connection of the launcher or image repository.
An attacker can change the image URL and control the workload binary. However these actions are reflected in the attestation report. The image repository isn't controlled using an access list, therefore the image is assumed to be viewable by everyone. You must ensure that the workload container image doesn't contain any secrets.
An attacker modifies a container's runtime environment configuration in the Kubernetes control plane.
The attestation process and the runtime policies detects unsafe configurations that load non-authentic images or perform any other modification to the expected runtime environment.
This table describes potential threats and mitigation strategies to the attestation service.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the Coordinator deployment and modifies the image or hijacks the runtime environment.
This threat is mitigated by having an attestation procedure and attested, encrypted TLS connections to the Coordinator. The attestation evidence for the Coordinator image is distributed with our releases, protected by supply chain security, and fully reproducible.
An attacker intercepts the network connection between the workload and the Coordinator and reads secret keys from the wire.
This threat is mitigated by having an attested, encrypted TLS connection. This connection helps protect the secrets from passive eavesdropping. The attacker can't create valid workload certificates that would be accepted in Contrast's service mesh. An attacker can't impersonate a valid workload container because the container's identity is guaranteed by the attestation protocol.
Within the network between your workload and the Coordinator.
An attacker exploits parsing discrepancies, which leads to undetected changes in the attestation process.
This risk is mitigated by having a parsing engine written in memory-safe Go that's tested against the attestation specification of the hardware vendor. The runtime policies are available as an attestation artifact for further inspection and audits to verify their effectiveness.
An attacker reads or modifies data written to disk via persistent volumes.
Currently, persistent volumes aren't supported in Contrast. In the future, this threat is mitigated by encrypted and integrity-protected volume mounts.
An attacker publishes a new image version containing malicious code.
The attestation process and the runtime policies require a data owner to accept a specific version of the workload and any update to the workload needs to be explicitly acknowledged.
The following table describes three example use cases and how they map to the defined threat model in this document:
-
Use Case
Example Scenario
Migrate sensitive workloads to the cloud
TechSolve Inc., a software development firm, aimed to enhance its defense-in-depth strategy for its cloud-based development environment, especially for projects involving proprietary algorithms and client data. TechSolve acts as the image provider, workload operator, and data owner, combining all three personas in this scenario. In our attestation terminology, they're the workload operator and relying party in one entity. Their threat model includes a malicious cloud insider and cloud co-tenant.
Make your SaaS more trustworthy
SaaSProviderX, a company offering cloud-based project management tools, sought to enhance its platform's trustworthiness amidst growing concerns about data breaches and privacy. Here, the relying party is the SaaS customer as the data owner. The goal is to achieve a form of operator exclusion and only allow selective operations on the deployment. Hence, their threat model includes a malicious workload operator.
Simplify regulatory compliance
HealthSecure Inc. has been managing a significant volume of sensitive patient data on-premises. With the increasing demand for advanced data analytics and the need for scalable infrastructure, the firm decides to migrate its data analytics operations to the cloud. However, the primary concern is maintaining the confidentiality and security of patient data during and after the migration, in compliance with healthcare regulations. In this compliance scenario, the regulator serves as an additional relying party. HealthSecure must implement a mechanism that ensures the isolation of patient data can be verifiably guaranteed to the regulator.
-
In each scenario, Contrast ensures exclusive data access and processing capabilities are confined to the designated workloads. It achieves this by effectively isolating the workload from the infrastructure and other components of the stack. Data owners are granted the capability to audit and approve the deployment environment before submitting their data, ensuring a secure handover. Meanwhile, workload operators are equipped to manage and operate the application seamlessly, without requiring direct access to either the workload or its associated data.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Components
-
Contrast is composed of several key components that work together to manage and scale confidential containers effectively within Kubernetes environments.
-This page provides an overview of the core components essential for deploying and managing Contrast.
The CLI serves as the primary management tool for Contrast deployments. It's designed to streamline the configuration and operation of Contrast in several ways:
-
-
Installation and setup: The CLI facilitates the installation of the necessary runtime classes required for Contrast to function within a Kubernetes cluster.
-
Policy generation: It allows users to generate runtime policies, adapt the deployment files, and generate the Contrast manifest.
-
Configuration management: Through the CLI, users can configure the Contrast Coordinator with the generated manifest.
-
Verification and attestation: The CLI provides tools to verify the integrity and authenticity of the Coordinator and the entire deployment via remote attestation.
The Contrast Coordinator is the central remote attestation service of a Contrast deployment.
-It runs inside a confidential container inside your cluster.
-The Coordinator can be verified via remote attestation, and a Contrast deployment is self-contained.
-The Coordinator is configured with a manifest, a configuration file containing the reference attestation values of your deployment.
-It ensures that your deployment's topology adheres to your specified manifest by verifying the identity and integrity of all confidential pods inside the deployment.
-The Coordinator is also a certificate authority and issues certificates for your workload pods during the attestation procedure.
-Your workload pods can establish secure, encrypted communication channels between themselves based on these certificates using the Coordinator as the root CA.
-As your app needs to scale, the Coordinator transparently verifies new instances and then provides them with their certificates to join the deployment.
-
To verify your deployment, the Coordinator's remote attestation statement combined with the manifest offers a concise single remote attestation statement for your entire deployment.
-A third party can use this to verify the integrity of your distributed app, making it easy to assure stakeholders of your app's identity and integrity.
The manifest is the configuration file for the Coordinator, defining your confidential deployment.
-It's automatically generated from your deployment by the Contrast CLI.
-It currently consists of the following parts:
-
-
Policies: The identities of your Pods, represented by the hashes of their respective runtime policies.
-
Reference Values: The remote attestation reference values for the Kata confidential micro-VM that's the runtime environment of your Pods.
-
WorkloadOwnerKeyDigest: The workload owner's public key digest. Used for authenticating subsequent manifest updates.
Runtime Policies are a mechanism to enable the use of the untrusted Kubernetes API for orchestration while ensuring the confidentiality and integrity of your confidential containers.
-They allow us to enforce the integrity of your containers' runtime environment as defined in your deployment files.
-The runtime policy mechanism is based on the Open Policy Agent (OPA) and translates the Kubernetes deployment YAML into the Rego policy language of OPA.
-The Kata Agent inside the confidential micro-VM then enforces the policy by only acting on permitted requests.
-The Contrast CLI provides the tooling for automatically translating Kubernetes deployment YAML into the Rego policy language of OPA.
Contrast provides an Initializer that handles the remote attestation on the workload side transparently and
-fetches the workload certificate. The Initializer runs as an init container before your workload is started.
-It provides the workload container and the service mesh sidecar with the workload certificates.
Contrast depends on a Kubernetes runtime class, which is installed
-by the node-installer DaemonSet.
-This runtime consists of a containerd runtime plugin, a virtual machine manager (cloud-hypervisor), and a podvm image (IGVM and rootFS).
-The installer takes care of provisioning every node in the cluster so it provides this runtime class.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Policies
-
Kata runtime policies are an integral part of the Confidential Containers preview on AKS.
-They prescribe how a Kubernetes pod must be configured to launch successfully in a confidential VM.
-In Contrast, policies act as a workload identifier: only pods with a policy registered in the manifest receive workload certificates and may participate in the confidential deployment.
-Verification of the Contrast Coordinator and its manifest transitively guarantees that all workloads meet the owner's expectations.
The Kata agent running in the confidential micro-VM exposes an RPC service AgentService to the Kata runtime.
-This service handles potentially untrustworthy requests from outside the TCB, which need to be checked against a policy.
-
Kata runtime policies are written in the policy language Rego.
-They specify what AgentService methods can be called, and the permissible parameters for each call.
-
Policies consist of two parts: a list of rules and a data section.
-While the list of rules is static, the data section is populated with information from the PodSpec and other sources.
Runtime policies are programmatically generated from Kubernetes manifests by the Contrast CLI.
-The generate subcommand inspects pod definitions and derives rules for validating the pod at the Kata agent.
-There are two important integrity checks: container image checksums and OCI runtime parameters.
-
For each of the container images used in a pod, the CLI downloads all image layers and produces a cryptographic dm-verity checksum.
-These checksums are the basis for the policy's storage data.
-
The CLI combines information from the PodSpec, ConfigMaps, and Secrets in the provided Kubernetes manifests to derive a permissible set of command-line arguments and environment variables.
-These constitute the policy's OCI data.
The generated policy document is annotated to the pod definitions in Base64 encoding.
-This annotation is propagated to the Kata runtime, which calculates the SHA256 checksum for the policy and uses that as SNP HOSTDATA for the confidential micro-VM.
-
After the VM launched, the runtime calls the agent's SetPolicy method with the full policy document.
-If the policy doesn't match the checksum in HOSTDATA, the agent rejects the policy.
-Otherwise, it applies the policy to all future AgentService requests.
Contrast verifies its confidential containers following these steps:
-
-
The Contrast CLI generates a policy and attaches it to the pod definition.
-
Kubernetes schedules the pod on a node with the confidential computing runtime.
-
Containerd invokes the Kata runtime to create the pod sandbox.
-
The Kata runtime starts a CVM with the policy's digest as HOSTDATA.
-
The Kata runtime sets the policy using the SetPolicy method.
-
The Kata agent verifies that the incoming policy's digest matches HOSTDATA.
-
The CLI sets a manifest in the Contrast Coordinator, including a list of permitted policies.
-
The Contrast Initializer sends an attestation report to the Contrast Coordinator, asking for a mesh certificate.
-
The Contrast Coordinator verifies that the started pod has a permitted policy hash in its HOSTDATA field.
-
-
After the last step, we know that the policy hasn't been tampered with and, thus, that the workload matches expectations and may receive mesh certificates.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Contrast Runtime
-
The Contrast runtime is responsible for starting pods as confidential virtual machines.
-This works by specifying the runtime class to be used in a pod spec and by registering the runtime class with the apiserver.
-The RuntimeClass resource defines a name for referencing the class and
-a handler used by the container runtime (containerd) to identify the class.
-
apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: # This name is used by pods in the runtimeClassName field name: contrast-cc-abcdef # This name is used by the # container runtime interface implementation (containerd) handler: contrast-cc-abcdef
-
Confidential pods that are part of a Contrast deployment need to specify the
-same runtime class in the runtimeClassName field, so Kubernetes uses the
-Contrast runtime instead of the default containerd / runc handler.
-
apiVersion: v1 kind: Pod spec: runtimeClassName: contrast-cc-abcdef # ...
The runtime consists of additional software components that need to be installed
-and configured on every SEV-SNP-enabled worker node.
-This installation is performed automatically by the node-installer DaemonSet.
The handler field in the Kubernetes RuntimeClass instructs containerd not to use the default runc implementation.
-Instead, containerd invokes a custom plugin called containerd-shim-contrast-cc-v2.
-This shim is described in more detail in the upstream source repository and in the containerd documentation.
The containerd shim uses cloud-hypervisor to create a confidential virtual machine for every pod.
-This requires the cloud-hypervisor binary to be installed on every node (responsibility of the node-installer).
Contrast uses a special containerd snapshotter (tardev) to provide container images as block devices to the pod-VM. This snapshotter consists of a host component that pulls container images and a guest component (kernel module) used to mount container images.
-The tardev snapshotter uses dm-verity to protect the integrity of container images.
-Expected dm-verity container image hashes are part of Contrast runtime policies and are enforced by the kata-agent.
-This enables workload attestation by specifying the allowed container image as part of the policy. Read the chapter on policies for more information.
Every pod-VM starts with the same guest image. It consists of an IGVM file and a root filesystem.
-The IGVM file describes the initial memory contents of a pod-VM and consists of:
-
-
Linux kernel image
-
initrd
-
kernel commandline
-
-
Additionally, a root filesystem image is used that contains a read-only partition with the user space of the pod-VM and a verity partition to guarantee the integrity of the root filesystem.
-The root filesystem contains systemd as the init system, and the kata agent for managing the pod.
-
This pod-VM image isn't specific to any pod workload. Instead, container images are mounted at runtime.
The RuntimeClass resource above registers the runtime with the Kubernetes api.
-The node-level installation is carried out by the Contrast node-installer
-DaemonSet that ships with every Contrast release.
-
After deploying the installer, it performs the following steps on each node:
-
-
Install the Contrast containerd shim (containerd-shim-contrast-cc-v2)
-
Install cloud-hypervisor as the virtual machine manager (VMM)
-
Install an IGVM file for pod-VMs of this class
-
Install a read only root filesystem disk image for the pod-VMs of this class
-
Reconfigure containerd by adding a runtime plugin that corresponds to the handler field of the Kubernetes RuntimeClass
-
Restart containerd to make it aware of the new plugin
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Service mesh
-
The Contrast service mesh secures the communication of the workload by automatically
-wrapping the network traffic inside mutual TLS (mTLS) connections. The
-verification of the endpoints in the connection establishment is based on
-certificates that are part of the
-PKI of the Coordinator.
-
The service mesh can be enabled on a per-workload basis by adding a service mesh
-configuration to the workload's object annotations. During the contrast generate
-step, the service mesh is added as a sidecar
-container to
-all workloads which have a specified configuration. The service mesh container first
-sets up iptables rules based on its configuration and then starts
-Envoy for TLS origination and termination.
The service mesh container can be configured using the following object annotations:
-
-
contrast.edgeless.systems/servicemesh-ingress to configure ingress.
-
contrast.edgeless.systems/servicemesh-egress to configure egress.
-
contrast.edgeless.systems/servicemesh-admin-interface-port to configure the Envoy
-admin interface. If not specified, no admin interface will be started.
-
-
If you aren't using the automatic service mesh injection and want to configure the
-service mesh manually, set the environment variables CONTRAST_INGRESS_PROXY_CONFIG,
-CONTRAST_EGRESS_PROXY_CONFIG and CONTRAST_ADMIN_PORT in the service mesh sidecar directly.
All TCP ingress traffic is routed over Envoy by default. Since we use
-TPROXY, the destination address
-remains the same throughout the packet handling.
-
Any incoming connection is required to present a client certificate signed by the
-mesh CA certificate.
-Envoy presents a certificate chain of the mesh
-certificate of the workload and the intermediate CA certificate as the server certificate.
-
If the deployment contains workloads which should be reachable from outside the
-Service Mesh, while still handing out the certificate chain, disable client
-authentication by setting the annotation contrast.edgeless.systems/servicemesh-ingress as
-<name>#<port>#false. Separate multiple entries with ##. You can choose any
-descriptive string identifying the service on the given port for the <name> field,
-as it's only informational.
-
Disable redirection and TLS termination altogether by specifying
-<name>#<port>#true. This can be beneficial if the workload itself handles TLS
-on that port or if the information exposed on this port is non-sensitive.
-
The following example workload exposes a web service on port 8080 and metrics on
-port 7890. The web server is exposed to a 3rd party end-user which wants to
-verify the deployment, therefore it's still required that the server hands out
-it certificate chain signed by the mesh CA certificate. The metrics should be
-exposed via TCP without TLS.
Note, that changing the environment variables of the sidecar container directly will
-only have an effect if the workload isn't configured to automatically generate a
-service mesh component on contrast generate. Otherwise, the service mesh sidecar
-container will be regenerated on every invocation of the command.
To be able to route the egress traffic of the workload through Envoy, the remote
-endpoints' IP address and port must be configurable.
-
-
Choose an IP address inside the 127.0.0.0/8 CIDR and a port not yet in use
-by the pod.
-
Configure the workload to connect to this IP address and port.
-
Set <name>#<chosen IP>:<chosen port>#<original-hostname-or-ip>:<original-port>
-as the contrast.edgeless.systems/servicemesh-egress workload annotation. Separate multiple
-entries with ##. Choose any string identifying the service on the given port as
-<name>.
-
-
This redirects the traffic over Envoy. The endpoint must present a valid
-certificate chain which must be verifiable with the
-mesh CA certificate.
-Furthermore, Envoy uses a certificate chain with the mesh certificate of the workload
-and the intermediate CA certificate as the client certificate.
-
The following example workload has no ingress connections and two egress
-connection to different microservices. The microservices are part
-of the confidential deployment. One is reachable under billing-svc:8080 and
-the other under cart-svc:8080.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Workload deployment
-
The following instructions will guide you through the process of making an existing Kubernetes deployment
-confidential and deploying it together with Contrast.
-
A running CoCo-enabled cluster is required for these steps, see the setup guide on how to set it up.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Your Kubernetes resources need some modifications to run as Confidential Containers.
-This section guides you through the process and outlines the necessary changes.
Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files
-unchanged, you can copy the files into a separate local directory.
-You can also generate files from a Helm chart or from a Kustomization.
To specify that a workload (pod, deployment, etc.) should be deployed as confidential containers,
-add runtimeClassName: contrast-cc to the pod spec (pod definition or template).
-This is a placeholder name that will be replaced by a versioned runtimeClassName when generating policies.
In the initialization process, the contrast-tls-certs shared volume is populated with X.509 certificates for your workload.
-These certificates are used by the Contrast Service Mesh, but can also be used by your application directly.
-The following tab group explains the setup for both scenarios.
-
Drop-in service mesh
Go integration
Contrast can be configured to handle TLS in a sidecar container.
-This is useful for workloads that are hard to configure with custom certificates, like Java applications.
Configuration of the sidecar depends heavily on the application.
-The following example is for an application with these properties:
-
The container has a main application at TCP port 8001, which should be TLS-wrapped and doesn't require client authentication.
-
The container has a metrics endpoint at TCP port 8080, which should be accessible in plain text.
-
All other endpoints require client authentication.
-
The app connects to a Kubernetes service backend.default:4001, which requires client authentication.
During the generate step, this configuration will be translated into a Service Mesh sidecar container which handles TLS connections automatically.
-The only change required to the app itself is to let it connect to 127.0.0.2:4001 to reach the backend service.
-You can find more detailed documentation in the Service Mesh chapter.
The mesh certificate contained in certChain.pem authenticates this workload, while the mesh CA certificate mesh-ca.pem authenticates its peers.
-Your app should turn on client authentication to ensure peers are running as confidential containers, too.
-See the Certificate Authority section for detailed information about these certificates.
The following example shows how to configure a Golang app, with error handling omitted for clarity.
Run the generate command to add the necessary components to your deployment files.
-This will add the Contrast Initializer to every workload with the specified contrast-cc runtime class
-and the Contrast Service Mesh to all workloads that have a specified configuration.
-After that, it will generate the execution policies and add them as annotations to your deployment files.
-A manifest.json with the reference values of your deployment will be created.
Please be aware that runtime policies currently have some blind spots. For example, they can't guarantee the starting order of containers. See the current limitations for more details.
-
If you don't want the Contrast Initializer to automatically be added to your
-workloads, there are two ways you can skip the Initializer injection step,
-depending on how you want to customize your deployment.
-
Command-line flag
Per-workload annotation
You can disable the Initializer injection completely by specifying the
---skip-initializer flag in the generate command.
If you want to disable the Initializer injection for a specific workload with
-the contrast-cc runtime class, you can do so by adding an annotation to the workload.
When disabling the automatic Initializer injection, you can manually add the
-Initializer as a sidecar container to your workload before generating the
-policies. Configure the workload to use the certificates written to the
-contrast-tls-certsvolumeMount.
For the next steps, we will need to connect to the Coordinator. The released Coordinator resource
-includes a LoadBalancer definition we can use.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
-
Port-forwarding of Confidential Containers
kubectl port-forward uses a Container Runtime Interface (CRI) method that isn't supported by the Kata shim.
-If you can't use a public load balancer, you can deploy a port-forwarder.
-The port-forwarder relays traffic from a CoCo pod and can be accessed via kubectl port-forward.
This will use the reference values from the manifest file to attest the Coordinator.
-After this step, the Coordinator will start issuing TLS certificates to the workloads. The init container
-will fetch a certificate for the workload and the workload is started.
An end user (data owner) can verify the Contrast deployment using the verify command.
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using the reference values from the given manifest file. It will then write the
-service mesh root certificate and the history of manifests into the verify/ directory. In addition, the policies
-referenced in the active manifest are also written to the directory. The verification will fail if the active
-manifest at the Coordinator doesn't match the manifest passed to the CLI.
You can securely connect to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-First, expose the service on a public IP address via a LoadBalancer service:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
If the Contrast Coordinator restarts, it enters recovery mode and waits for an operator to provide key material.
-For demonstration purposes, you can simulate this scenario by deleting the Coordinator pod.
-
kubectl delete pod -l app.kubernetes.io/name=coordinator
-
Kubernetes schedules a new pod, but that pod doesn't have access to the key material the previous pod held in memory and can't issue certificates for workloads yet.
-You can confirm this by running verify again, or you can restart a workload pod, which should stay in the initialization phase.
-However, the secret seed in your working directory is sufficient to recover the coordinator.
-
contrast recover -c"${coordinator}:1313"
-
Now that the Coordinator is recovered, all workloads should pass initialization and enter the running state.
-You can now verify the Coordinator again, which should return the same manifest you set before.
-
warning
The recovery process invalidates the mesh CA certificate:
-existing workloads won't be able to communicate with workloads newly spawned.
-All workloads should be restarted after the recovery succeeded.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.8, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.8
Confidential emoji voting
-
-
This tutorial guides you through deploying emojivoto as a
-confidential Contrast deployment and validating the deployment from a voters perspective.
-
Emojivoto is an example app allowing users to vote for different emojis and view votes
-on a leader board. It has a microservice architecture consisting of a
-web frontend (web), a gRPC backend for listing available emojis (emoji), and a backend for
-the voting and leader board logic (voting). The vote-bot simulates user traffic by submitting
-votes to the frontend.
Using a voting service, users' votes are considered highly sensitive data, as we require
-a secret ballot. Also, users are likely interested in the fairness of the ballot. For
-both requirements, we can use Confidential Computing and, specifically, workload attestation
-to prove to those interested in voting that the app is running in a protected environment
-where their votes are processed without leaking to the platform provider or workload owner.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Run the generate command to generate the execution policies and add them as
-annotations to your deployment files. A manifest.json file with the reference values
-of your deployment will be created:
The deployment YAML shipped for this demo is already configured to be used with Contrast.
-A runtime classcontrast-cc-<VERSIONHASH>
-was added to the pods to signal they should be run as Confidential Containers. During the generation process,
-the Contrast Initializer will be added as an init container to these
-workloads to facilitate the attestation and certificate pulling before the actual workload is started.
Further, the deployment YAML is also configured with the Contrast service mesh.
-The configured service mesh proxy provides transparent protection for the communication between
-the different components of emojivoto.
Configure the coordinator with a manifest. It might take up to a few minutes
-for the load balancer to be created and the Coordinator being available.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"The user API of your Contrast Coordinator is available at $coordinator:1313" contrast set-c"${coordinator}:1313" deployment/
-
The CLI will use the reference values from the manifest to attest the Coordinator deployment
-during the TLS handshake. If the connection succeeds, it's ensured that the Coordinator
-deployment hasn't been tampered with.
Now that the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload,
-we can deploy the application:
-
kubectl apply -f deployment/
-
Inter-deployment communication
The Contrast Coordinator issues mesh certificates after successfully validating workloads.
-These certificates can be used for secure inter-deployment communication. The Initializer
-sends an attestation report to the Coordinator, retrieves certificates and a private key in return
-and writes them to a volumeMount. The service mesh sidecar is configured to use the credentials
-from the volumeMount when communicating with other parts of the deployment over mTLS.
-The public facing frontend for voting uses the mesh certificate without client authentication.
As voters, we want to verify the fairness and confidentiality of the deployment before
-deciding to vote. Regardless of the scale of our distributed deployment, Contrast only
-needs a single remote attestation step to verify the deployment. By doing remote attestation
-of the Coordinator, we transitively verify those systems the Coordinator has already attested
-or will attest in the future. Successful verification of the Coordinator means that
-we can be sure it will enforce the configured manifest.
The CLI will attest the Coordinator using the reference values from a given manifest. This manifest needs
-to be communicated out of band to everyone wanting to verify the deployment, as the verify command checks
-if the currently active manifest at the Coordinator matches the manifest given to the CLI. If the command succeeds,
-the Coordinator deployment was successfully verified to be running in the expected Confidential
-Computing environment with the expected code version. The Coordinator will then return its
-configuration over the established TLS channel. The CLI will store this information, namely the root
-certificate of the mesh (mesh-ca.pem) and the history of manifests, into the verify/ directory.
-In addition, the policies referenced in the manifest history are also written into the same directory.
In the next step, the Coordinator configuration that was written by the verify command needs to be audited.
-A potential voter should inspect the manifest and the referenced policies. They could delegate
-this task to an entity they trust.
After ensuring the configuration of the Coordinator fits the expectation, you can securely connect
-to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-
To access the web frontend, expose the service on a public IP address via a LoadBalancer service:
-
frontendIP=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"Frontend is available at https://$frontendIP, you can visit it in your browser."
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the subject alternative name (SAN) field.
-Validation fails since the certificate contains no IP entries as a SAN.
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
-
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
The Policies section of the manifest maps policy hashes to a list of SANs. To enable certificate verification
-of the web frontend with tools like curl, edit the policy with your favorite editor and add the frontendIP to
-the list that already contains the "web" DNS entry:
Next, set the changed manifest at the coordinator with:
-
contrast set-c"${coordinator}:1313" deployment/
-
The Contrast Coordinator will rotate the mesh ca certificate on the manifest update. Workload certificates issued
-after the manifest update are thus issued by another certificate authority and services receiving the new CA certificate chain
-won't trust parts of the deployment that got their certificate issued before the update. This way, Contrast ensures
-that parts of the deployment that received a security update won't be infected by parts of the deployment at an older
-patch level that may have been compromised. The mesh-ca.pem is updated with the new CA certificate chain.
The Coordinator has the new manifest set, but the different containers of the app are still
-using the older certificate authority. The Contrast Initializer terminates after the initial attestation
-flow and won't pull new certificates on manifest updates.
After the update has been rolled out, connecting to the frontend using curl will successfully validate
-the service certificate and return the HTML document of the voting site:
Platform support: At present, Contrast is exclusively available on Azure AKS, supported by the Confidential Container preview for AKS. Expansion to other cloud platforms is planned, pending the availability of necessary infrastructure enhancements.
-
Bare-metal support: Support for running Contrast on bare-metal Kubernetes will be available soon for AMD SEV and Intel TDX.
Persistent volumes: Contrast only supports volumes with volumeMode: Block. These block devices are provided by the untrusted environment and should be treated accordingly. We plan to provide transparent encryption on top of block devices in a future release.
Resource limits: There is an existing bug on AKS where container memory limits are incorrectly applied. The current workaround involves using only memory requests instead of limits.
Coverage: While the enforcement of workload policies generally functions well, there are scenarios not yet fully covered. It's crucial to review deployments specifically for these edge cases.
-
Order of events: The current policy evaluation mechanism on API requests isn't stateful, so it can't ensure a prescribed order of events. Consequently, there's no guaranteed enforcement that the service mesh sidecar container runs before the workload container. This order ensures that all traffic between pods is securely encapsulated within TLS connections.
-
Absence of events: Policies can't ensure certain events have happened. A container, such as the service mesh sidecar, can be omitted entirely. Environment variables may be missing.
-
Volume integrity checks: While persistent volumes aren't supported yet, integrity checks don't currently cover other objects such as ConfigMaps and Secrets.
-
-
warning
The policy limitations, in particular the missing guarantee that our service mesh sidecar has been started before the workload container affects the service mesh implementation of Contrast. Currently, this requires inspecting the iptables rules on startup or terminating TLS connections in the workload directly.
CLI availability: The CLI tool is currently only available for Linux. This limitation arises because certain upstream dependencies haven't yet been ported to other platforms.
The Contrast Coordinator is a singleton and can't be scaled to more than one instance.
-When this instance's pod is restarted, for example for node maintenance, it needs to be recovered manually.
-In a future release, we plan to support distributed Coordinator instances that can recover automatically.
CoCo on AKS is currently in preview. An extension for the az CLI is needed to create such a cluster.
-Add the extension with the following commands:
-
az extension add\ --name aks-preview \ --allow-preview true az extension update \ --name aks-preview \ --allow-preview true
-
Then register the required feature flags in your subscription to allow access to the public preview:
-
az feature register \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"
-
The registration can take a few minutes. The status of the operation can be checked with the following
-command, which should show the registration state as Registered:
-
az feature show \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"\ --output table
-
Afterward, refresh the registration of the ContainerService provider:
-
az provider register \ --namespace"Microsoft.ContainerService"
First, we need to create an AKS cluster. We can't directly create a CoCo-enabled cluster, so we'll need to create a
-non-CoCo cluster first, and then add a CoCo node pool, optionally replacing the non-CoCo node pool.
-
We'll first start by creating the non-CoCo cluster:
-
# Select the name for your AKS cluster azClusterName="ContrastDemo" az aks create \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"\ --kubernetes-version 1.29\ --os-sku AzureLinux \ --node-vm-size Standard_DC4as_cc_v5 \ --node-count 1\ --generate-ssh-keys
After trying out Contrast, you might want to clean up the cloud resources created in this step.
-In case you've created a new resource group, you can just delete that group with
-
az group delete \ --name"${azResourceGroup:?}"
-
Deleting the resource group will also delete the cluster and all other related resources.
-
To only cleanup the AKS cluster and node pools, run
-
az aks delete \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"
Collecting logs can be a good first step to identify problems in your
-deployment. Both the CLI and the Contrast Coordinator as well as the Initializer
-can be configured to emit additional logs.
The CLI logs can be configured with the --log-level command-line flag, which
-can be set to either debug, info, warn or error. The default is info.
-Setting this to debug can get more fine-grained information as to where the
-problem lies.
The logs from the Coordinator and the Initializer can be configured via the
-environment variables CONTRAST_LOG_LEVEL, CONTRAST_LOG_FORMAT and
-CONTRAST_LOG_SUBSYSTEMS.
-
-
CONTRAST_LOG_LEVEL can be set to one of either debug, info, warn, or
-error, similar to the CLI (defaults to info).
-
CONTRAST_LOG_FORMAT can be set to text or json, determining the output
-format (defaults to text).
-
CONTRAST_LOG_SUBSYSTEMS is a comma-seperated list of subsystems that should
-be enabled for logging, which are disabled by default. Subsystems include:
-snp-issuer, kds-getter, and snp-validator. To enable all subsystems, use
-* as the value for this environment variable.
-Warnings and error messages from subsystems get printed regardless of whether
-the subsystem is listed in the CONTRAST_LOG_SUBSYSTEMS environment variable.
-
-
To configure debug logging with all subsystems for your Coordinator, add the
-following variables to your container definition.
While the Contrast Coordinator has a policy that allows certain configurations,
-the Initializer and service mesh don't. When changing environment variables of other
-parts than the Coordinator, ensure to rerun contrast generate to update the policy.
-
To access the logs generated by the Coordinator, you can use kubectl with the
-following command:
If the Coordinator or a workload pod fails to even start, it can be helpful to
-look at the events of the pod during the startup process using the describe
-command.
LAST SEEN TYPE REASON OBJECT MESSAGE 32m Warning Failed Pod/coordinator-0 kubelet Error: failed to create containerd task: failed to create shim task: "CreateContainerRequest is blocked by policy: ...
-
A common error, as in this example, is that the container creation was blocked by the
-policy. Potential reasons are a modification of the deployment YAML without updating
-the policies afterward, or a version mismatch between Contrast components.
When generating the policies, Contrast will download the images specified in your deployment
-YAML and include their cryptographic identity. If the image tag is moved to another
-container image after the policy has been generated, the image downloaded at deploy time
-will differ from the one at generation time, and the policy enforcement won't allow the
-container to be started in the pod VM.
-
To ensure the correct image is always used, pin the container image to a fixed sha256:
A version mismatch between Contrast components can cause policy validation or attestation
-to fail. Each Contrast runtime is identifiable based on its (shortened) measurement value
-used to name the runtime class version.
-
First, analyze which runtime class is currently installed in your cluster by running
-
kubectl get runtimeclasses
-
This should give you output similar to the following one.
-
NAME HANDLER AGE contrast-cc-30bfa8706b542271ec9b7762bbb400af contrast-cc-30bfa8706b542271ec9b7762bbb400af 23d contrast-cc-4d70a6e266cca46dfa8e41d92874e638 contrast-cc-4d70a6e266cca46dfa8e41d92874e638 7d contrast-cc-b817659e094106f61bf6c178c27153ba contrast-cc-b817659e094106f61bf6c178c27153ba 2d19h contrast-cc-beee79ca916b9e5dc59602788cbfb097 contrast-cc-beee79ca916b9e5dc59602788cbfb097 121m kata-cc-isolation kata-cc 45d
-
The output shows that there are four Contrast runtime classes installed (as well as the runtime class provided
-by the AKS CoCo preview, which isn't used by Contrast).
-
Next, check if the pod that won't start has the correct runtime class configured, and the
-Coordinator uses the exact same runtime:
-
kubectl -n<namespace> get -o=jsonpath='{.spec.runtimeClassName}' pod/<pod-name> kubectl -n<namespace> get -o=jsonpath='{.spec.runtimeClassName}' pod/<coordinator-pod-name>
-
The output should list the runtime class the pod is using:
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Contrast
-
Welcome to the documentation of Contrast! Contrast runs confidential container deployments on Kubernetes at scale.
-
-
Contrast is based on the Kata Containers and
-Confidential Containers projects.
-Confidential Containers are Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation from the surrounding environment.
-This works with unmodified containers in a lift-and-shift approach.
-Contrast currently targets the CoCo preview on AKS.
-
tip
See the 📄whitepaper for more information on confidential computing.
Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer. It removes the infrastructure provider from the trusted computing base (TCB). This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, for example, malicious co-tenants escalating their privileges.
-
Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
CLI telemetry
-
The Contrast CLI sends telemetry data to Edgeless Systems when you use CLI commands.
-This allows to understand how Contrast is used and to improve it.
-
The CLI sends the following data:
-
-
The CLI version
-
The CLI target OS and architecture (GOOS and GOARCH)
-
The command that was run
-
The kind of error that occurred (if any)
-
-
The CLI doesn't collect sensitive information.
-The implementation is open-source and can be reviewed.
-
IP addresses may be processed or stored for security purposes.
-
The data that the CLI collects adheres to the Edgeless Systems privacy policy.
-
You can disable telemetry by setting the environment variable DO_NOT_TRACK=1 before running the CLI.
-
-
\ No newline at end of file
diff --git a/pr-preview/pr-918/0.9/architecture/attestation.html b/pr-preview/pr-918/0.9/architecture/attestation.html
deleted file mode 100644
index 810370befd..0000000000
--- a/pr-preview/pr-918/0.9/architecture/attestation.html
+++ /dev/null
@@ -1,104 +0,0 @@
-
-
-
-
-
-Attestation in Contrast | Contrast
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Attestation in Contrast
-
This document describes the attestation architecture of Contrast, adhering to the definitions of Remote ATtestation procedureS (RATS) in RFC 9334.
-The following gives a detailed description of Contrast's attestation architecture.
-At the end of this document, we included an FAQ that answers the most common questions regarding attestation in hindsight of the security benefits.
Contrast integrates with the RATS architecture, leveraging their definition of roles and processes including Attesters, Verifiers, and Relying Parties.
-
-
Figure 1: Conceptual attestation architecture. Taken from RFC 9334.
-
-
Attester: Assigned to entities that are responsible for creating Evidence which is then sent to a Verifier.
-
Verifier: These entities utilize the Evidence, Reference Values, and Endorsements. They assess the trustworthiness of the Attester by applying an Appraisal Policy for Evidence. Following this assessment, Verifiers generate Attestation Results for use by Relying Parties. The Appraisal Policy for Evidence may be provided by the Verifier Owner, programmed into the Verifier, or acquired through other means.
-
Relying Party: Assigned to entities that utilize Attestation Results, applying their own appraisal policies to make specific decisions, such as authorization decisions. This process is referred to as the "appraisal of Attestation Results." The Appraisal Policy for Attestation Results might be sourced from the Relying Party Owner, configured by the owner, embedded in the Relying Party, or obtained through other protocols or mechanisms.
This includes all Pods of the Contrast deployment that run inside Confidential Containers and generate cryptographic evidence reflecting their current configuration and state.
-Their evidence is rooted in the hardware measurements from the CPU and their confidential VM environment.
-The details of this evidence are given below in the section on evidence generation and appraisal.
-
-
Figure 2: Attestation flow of a confidential pod. Based on the layered attester graphic in RFC 9334.
-
Pods run in Contrast's runtime environment (B), effectively within a confidential VM.
-During launch, the CPU (A) measures the initial memory content of the confidential VM that contains Contrast's pod-VM image and generates the corresponding attestation evidence.
-The image is in IGVM format, encapsulating all information required to launch a virtual machine, including the kernel, the initramfs, and kernel cmdline.
-The kernel cmdline contains the root hash for dm-verity that ensures the integrity of the root filesystem.
-The root filesystem contains all components of the container's runtime environment including the guest agent (C).
-
In the userland, the guest agent takes care of enforcing the runtime policy of the pod.
-While the policy is passed in during the initialization procedure via the host, the evidence for the runtime policy is part of the CPU measurements.
-During the deployment the policy is annotated to the Kubernetes Pod resources.
-On AMD SEV-SNP the hash of the policy is then added to the attestation report via the HOSTDATA field by the hypervisor.
-When provided with the policy from the Kata host, the guest agent verifies that the policy's hash matches the one in the HOSTDATA field.
-
In summary a Pod's evidence is the attestation report of the CPU that provides evidence for runtime environment and the runtime policy.
The Coordinator acts as a verifier within the Contrast deployment, configured with a Manifest that defines the reference values and serves as an appraisal policy for all pods in the deployment.
-It also pulls endorsements from hardware vendors to verify the hardware claims.
-The Coordinator operates within the cluster as a confidential container and provides similar evidence as any other Pod when it acts as an attester.
-In RATS terminology, the Coordinator's dual role is defined as a lead attester in a composite device which spans the entire deployment: Coordinator and the workload pods.
-It collects evidence from other attesters and conveys it to a verifier, generating evidence about the layout of the whole composite device based on the Manifest as the appraisal policy.
-
-
Figure 3: Contrast deployment as a composite device. Based on the composite device in RFC 9334.
-
The CLI serves as the verifier for the Coordinator and the entire Contrast deployment, containing the reference values for the Coordinator and the endorsements from hardware vendors.
-These reference values are built into the CLI during our release process and can be reproduced offline via reproducible builds.
A relying party in the Contrast scenario could be, for example, the data owner that interacts with the application.
-The relying party can use the CLI to obtain the attestation results and Contrast's CA certificates bound to these results.
-The CA certificates can then be used by the relying party to authenticate the application, for example through TLS connections.
In Contrast, attestation evidence revolves around a hardware-generated attestation report, which contains several critical pieces of information:
-
-
The hardware attestation report: This report includes details such as the chip identifier, platform information, microcode versions, and comprehensive guest measurements. The entire report is signed by the CPU's private key, ensuring the authenticity and integrity of the data provided.
-
The launch measurements: Included within the hardware attestation report, this is a digest generated by the CPU that represents a hash of all initial guest memory pages. This includes essential components like the kernel, initramfs, and the kernel command line. Notably, it incorporates the root filesystem's dm-verity root hash, verifying the integrity of the root filesystem.
-
The runtime policy hash: Also part of the hardware attestation report, this field contains the hash of the Rego policy which dictates all expected API commands and their values from the host to the Kata guest agent. It encompasses crucial settings such as dm-verity hashes for the container image layers, environment variables, and mount points.
The appraisal of this evidence in Contrast is governed by two main components:
-
-
The Manifest: A JSON file used by the Coordinator to align with reference values. It sets the expectations for runtime policy hashes for each pod and includes what should be reported in the hardware attestation report for each component of the deployment.
-
The CLI's appraisal policy: This policy encompasses expected values of the Coordinator’s guest measurements and its runtime policy. It's embedded into the CLI during the build process and ensures that any discrepancy between the built-in values and those reported by the hardware attestation can be identified and addressed. The integrity of this policy is safeguardable through reproducible builds, allowing verification against the source code reference.
-
-
Frequently asked questions about attestation in Contrast
-
What's the purpose of remote attestation in Contrast?
-
Remote attestation in Contrast ensures that software runs within a secure, isolated confidential computing environment.
-This process certifies that the memory is encrypted and confirms the integrity and authenticity of the software running within the deployment.
-By validating the runtime environment and the policies enforced on it, Contrast ensures that the system operates in a trustworthy state and hasn't been tampered with.
-
How does Contrast ensure the security of the attestation process?
-
Contrast leverages hardware-rooted security features such as AMD SEV-SNP to generate cryptographic evidence of a pod’s current state and configuration.
-This evidence is checked against pre-defined appraisal policies to guarantee that only verified and authorized pods are part of a Contrast deployment.
Attestation confirms the integrity of the runtime environment and the identity of the workloads.
-It plays a critical role in preventing unauthorized changes and detecting potential modifications at runtime.
-The attestation provides integrity and authenticity guarantees, enabling relying parties—such as workload operators or data owners—to confirm the effective protection against potential threats, including malicious cloud insiders, co-tenants, or compromised workload operators.
-More details on the specific security benefits can be found here.
-
How can you verify the authenticity of attestation results?
-
Attestation results in Contrast are tied to cryptographic proofs generated and signed by the hardware itself.
-These proofs are then verified using public keys from trusted hardware vendors, ensuring that the results aren't only accurate but also resistant to tampering.
-For further authenticity verification, all of Contrast's code is reproducibly built, and the attestation evidence can be verified locally from the source code.
-
How are attestation results used by relying parties?
-
Relying parties use attestation results to make informed security decisions, such as allowing access to sensitive data or resources only if the attestation verifies the system's integrity.
-Thereafter, the use of Contrast's CA certificates in TLS connections provides a practical approach to communicate securely with the application.
In summary, Contrast's attestation strategy adheres to the RATS guidelines and consists of robust verification mechanisms that ensure each component of the deployment is secure and trustworthy.
-This comprehensive approach allows Contrast to provide a high level of security assurance to its users.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Certificate authority
-
The Coordinator acts as a certificate authority (CA) for the workloads
-defined in the manifest.
-After a workload pod's attestation has been verified by the Coordinator,
-it receives a mesh certificate and the mesh CA certificate.
-The mesh certificate can be used for example in a TLS connection as the server or
-client certificate to proof to the other party that the workload has been
-verified by the Coordinator. The other party can verify the mesh certificate
-with the mesh CA certificate. While the certificates can be used by the workload
-developer in different ways, they're automatically used in Contrast's service
-mesh to establish mTLS connections between workloads in the same deployment.
The Coordinator establishes a public key infrastructure (PKI) for all workloads
-contained in the manifest. The Coordinator holds three certificates: the root CA
-certificate, the intermediate CA certificate, and the mesh CA certificate.
-The root CA certificate is a long-lasting certificate and its private key signs
-the intermediate CA certificate. The intermediate CA certificate and the mesh CA
-certificate share the same private key. This intermediate private key is used
-to sign the mesh certificates. Moreover, the intermediate private key and
-therefore the intermediate CA certificate and the mesh CA certificate are
-rotated when setting a new manifest.
Depending on the configuration of the first manifest, it allows the workload
-owner to update the manifest and, therefore, the deployment.
-Workload owners and data owners can be mutually untrusted parties.
-To protect against the workload owner silently introducing malicious containers,
-the Coordinator rotates the intermediate private key every time the manifest is
-updated and, therefore, the
-intermediate CA certificate and mesh CA certificate. If the user doesn't
-trust the workload owner, they use the mesh CA certificate obtained when they
-verified the Coordinator and the manifest. This ensures that the user only
-connects to workloads defined in the manifest they verified since only those
-workloads' certificates are signed with this intermediate private key.
-
Similarly, the service mesh also uses the mesh CA certificate obtained when the
-workload was started, so the workload only trusts endpoints that have been
-verified by the Coordinator based on the same manifest. Consequently, a
-manifest update requires a fresh rollout of the services in the service mesh.
The root CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This should only be used if the data owner trusts all future updates to the
-manifest and workloads. This is, for instance, the case when the workload owner is
-the same entity as the data owner.
-
The mesh CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This certificate is bound to the manifest set when the Coordinator is verified.
-If the manifest is updated, the mesh CA certificate changes.
-New workloads will receive mesh certificates signed by the new mesh CA certificate.
-The Coordinator with the new manifest needs to be verified to retrieve the new mesh CA certificate.
-The service mesh also uses the mesh CA certificate to verify the mesh certificates.
-
The intermediate CA certificate links the root CA certificate to the
-mesh certificate so that the mesh certificate can be verified with the root CA
-certificate. It's part of the certificate chain handed out by
-endpoints in the service mesh.
-
The mesh certificate is part of the certificate chain handed out by
-endpoints in the service mesh. During the startup of a pod, the Initializer
-requests a certificate from the Coordinator. This mesh certificate will be returned if the Coordinator successfully
-verifies the workload. The mesh certificate
-contains X.509 extensions with information from the workloads attestation
-document.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Observability
-
The Contrast Coordinator can expose metrics in the
-Prometheus format. These can be monitored to quickly
-identify problems in the gRPC layer or attestation errors. Prometheus metrics
-are numerical values associated with a name and additional key/values pairs,
-called labels.
The metrics can be accessed at the Coordinator pod at the port specified in the
-CONTRAST_METRICS_PORT environment variable under the /metrics endpoint. By
-default, this environment variable isn't specified, hence no metrics will be
-exposed.
-
The Coordinator exports gRPC metrics under the prefix contrast_grpc_server_.
-These metrics are labeled with the gRPC service name and method name.
-Metrics of interest include contrast_grpc_server_handled_total, which counts
-the number of requests by return code, and
-contrast_grpc_server_handling_seconds_bucket, which produces a histogram of
-request latency.
The meshapi.MeshAPI service records metrics for the method NewMeshCert, which
-gets called by the Initializer when starting a
-new workload. Attestation failures from workloads to the Coordinator can be
-tracked with the counter contrast_meshapi_attestation_failures_total.
-
The current manifest generation is exposed as a
-gauge with the metric
-name contrast_coordinator_manifest_generation. If no manifest is set at the
-Coordinator, this counter will be zero.
The Service Mesh can be configured to expose
-metrics via its Envoy admin
-interface. Be
-aware that the admin interface can expose private information and allows
-destructive operations to be performed. To enable the admin interface for the
-Service Mesh, set the annotation
-contrast.edgeless.systems/servicemesh-admin-interface-port in the configuration
-of your workload. If this annotation is set, the admin interface will be started
-on this port.
-
To access the admin interface, the ingress settings of the Service Mesh have to
-be configured to allow access to the specified port (see Configuring the
-Proxy). All metrics will be
-exposed under the /stats endpoint. Metrics in Prometheus format can be scraped
-from the /stats/prometheus endpoint.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Secrets & recovery
-
When the Coordinator is configured with the initial manifest, it generates a random secret seed.
-From this seed, it uses an HKDF to derive the CA root key and a signing key for the manifest history.
-This derivation is deterministic, so the seed can be used to bring any Coordinator to this Coordinator's state.
-
The secret seed is returned to the user on the first call to contrast set, encrypted with the user's public seed share owner key.
-If no seed share owner key is provided, a key is generated and stored in the working directory.
The Coordinator runs as a StatefulSet with a dynamically provisioned persistent volume.
-This volume stores the manifest history and the associated runtime policies.
-The manifest isn't considered sensitive information, because it needs to be passed to the untrusted infrastructure in order to start workloads.
-However, the Coordinator must ensure its integrity and that the persisted data corresponds to the manifests set by authorized users.
-Thus, the manifest is stored in plain text, but is signed with a private key derived from the Coordinator's secret seed.
When a Coordinator starts up, it doesn't have access to the signing secret and can thus not verify the integrity of the persisted manifests.
-It needs to be provided with the secret seed, from which it can derive the signing key that verifies the signatures.
-This procedure is called recovery and is initiated by the workload owner.
-The CLI decrypts the secret seed using the private seed share owner key, verifies the Coordinator and sends the seed through the Recover method.
-The Coordinator recovers its key material and verifies the manifest history signature.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Confidential Containers
-
Contrast uses some building blocks from Confidential Containers (CoCo), a CNCF Sandbox project that aims to standardize confidential computing at the pod level.
-The project is under active development and many of the high-level features are still in flux.
-Contrast uses the more stable core primitive provided by CoCo: its Kubernetes runtime.
Kubernetes can be extended to use more than one container runtime with RuntimeClass objects.
-The Container Runtime Interface (CRI) implementation, for example containerd, dispatches pod management API calls to the appropriate RuntimeClass.
-RuntimeClass implementations are usually based on an OCI runtime, such as runc, runsc or crun.
-In CoCo's case, the runtime is Kata Containers with added confidential computing capabilities.
Kata Containers is an OCI runtime that runs pods in VMs.
-The pod VM spawns an agent process that accepts management commands from the Kata runtime running on the host.
-There are two options for creating pod VMs: local to the Kubernetes node, or remote VMs created with cloud provider APIs.
-Using local VMs requires either bare metal servers or VMs with support for nested virtualization.
-Local VMs communicate with the host over a virtual socket.
-For remote VMs, host-to-agent communication is tunnelled through the cloud provider's network.
-
Kata Containers was originally designed to isolate the guest from the host, but it can also run pods in confidential VMs (CVMs) to shield pods from their underlying infrastructure.
-In confidential mode, the guest agent is configured with an Open Policy Agent (OPA) policy to authorize API calls from the host.
-This policy also contains checksums for the expected container images.
-It's derived from Kubernetes resource definitions and its checksum is included in the attestation report.
Azure Kubernetes Service (AKS) provides CoCo-enabled node pools as a preview offering.
-These node pools leverage Azure VM types capable of nested virtualization (CVM-in-VM) and the CoCo stack is pre-installed.
-Contrast can be deployed directly into a CoCo-enabled AKS cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Product features
-
Contrast simplifies the deployment and management of Confidential Containers, offering optimal data security for your workloads while integrating seamlessly with your existing Kubernetes environment.
-
From a security perspective, Contrast employs the Confidential Containers concept and provides security benefits that go beyond individual containers, shielding your entire deployment from the underlying infrastructure.
-
From an operational perspective, Contrast provides the following key features:
-
-
-
Managed Kubernetes compatibility: Initially compatible with Azure Kubernetes Service (AKS), Contrast is designed to support additional platforms such as AWS EKS and Google Cloud GKE as they begin to accommodate confidential containers.
-
-
-
Lightweight installation: Contrast can be integrated as a day-2 operation within existing clusters, adding minimal components to your setup. This facilitates straightforward deployments using your existing YAML configurations, Helm charts, or Kustomization, enabling native Kubernetes orchestration of your applications.
-
-
-
Remote attestation: Contrast generates a concise attestation statement that verifies the identity, authenticity, and integrity of your deployment both internally and to external parties. This architecture ensures that updates or scaling of the application don't compromise the attestation’s validity.
-
-
-
Service mesh: Contrast securely manages a Public Key Infrastructure (PKI) for your deployments, issues workload-specific certificates, and establishes transparent mutual TLS (mTLS) connections across pods. This is done by harnessing the envoy proxy to ensure secure communications within your Kubernetes cluster.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Contrast security overview
-
This document outlines the security measures of Contrast and its capability to counter various threats.
-Contrast is designed to shield entire Kubernetes deployments from the infrastructure, enabling entities to manage sensitive information (such as regulated or personally identifiable information (PII)) in the public cloud, while maintaining data confidentiality and ownership.
-
Contrast is applicable in situations where establishing trust with the workload operator or the underlying infrastructure is challenging.
-This is particularly beneficial for regulated sectors looking to transition sensitive activities to the cloud, without sacrificing security or compliance.
-It allows for cloud adoption by maintaining a hardware-based separation from the cloud service provider.
Leveraging Confidential Computing technology, Contrast provides three defining security properties:
-
-
Encryption of data in use: Contrast ensures that all data processed in memory is encrypted, making it inaccessible to unauthorized users or systems, even if they have physical access to the hardware.
-
Workload isolation: Each pod runs in its isolated runtime environment, preventing any cross-contamination between workloads, which is critical for multi-tenant infrastructures.
-
Remote attestation: This feature allows data owners and workload operators to verify that the Contrast environment executing their workloads hasn't been tampered with and is running in a secure, pre-approved configuration.
-
-
The runtime encryption is transparently provided by the confidential computing hardware during the workload's lifetime.
-The workload isolation and remote attestation involves two phases:
-
-
An attestation process detects modifications to the workload image or its runtime environment during the initialization. This protects the workload's integrity pre-attestation.
-
A protected runtime environment and a policy mechanism prevents the platform operator from accessing or compromising the instance at runtime. This protects a workload's integrity and confidentiality post-attestation.
-
-
For more details on confidential computing see our whitepaper.
-The attestation architecture describes Contrast's attestation process and the resulting chain of trust in detail.
Contrast uses the Kubernetes runtime of the Confidential Containers project.
-Confidential Containers significantly decrease the size of the trusted computing base (TCB) of a Kubernetes deployment, by isolating each pod within its own confidential micro-VM environment.
-The TCB is the totality of elements in a computing environment that must be trusted not to be compromised.
-A smaller TCB results in a smaller attack surface. The following diagram shows how Confidential Containers remove the cloud & datacenter infrastructure and the physical hosts, including the hypervisor, the host OS, the Kubernetes control plane, and other components, from the TCB (red).
-In the confidential context, depicted in green, only the workload containers along with their confidential micro-VM environment are included within the TCB.
-Their integrity is verifiable through remote attestation.
-
Contrast uses hardware-based mechanisms, specifically leveraging CPU features, such as AMD SEV or Intel TDX, to provide the isolation of the workload.
-This implies that both the CPU and its microcode are integral components of the TCB.
-However, it should be noted that the CPU microcode aspects aren't depicted in the accompanying graphic.
-
-
Contrast adds the following components to a deployment that become part of the TCB.
-The components that are part of the TCB are:
-
-
The workload containers: Container images that run the actual application.
The runtime policies: Policies that enforce the runtime environments for the workload containers during their lifetime.
-
The manifest: A manifest file defining the reference values of an entire confidential deployment. It contains the policy hashes for all pods of the deployment and the expected hardware reference values for the Confidential Container runtime.
-
The Coordinator: An attestation service that runs in a Confidential Container in the Kubernetes cluster. The Coordinator is configured with the manifest. User-facing, you can verify this service and the effective manifest using remote attestation, providing you with a concise attestation for the entire deployment. Cluster-facing, it verifies all pods and their policies based on remote attestation procedures and the manifest.
In a Contrast deployment, there are three parties:
-
-
-
The container image provider, who creates the container images that represent the application that has access to the protected data.
-
-
-
The workload operator, who runs the workload in a Kubernetes cluster. The operator typically has full administrative privileges to the deployment. The operator can manage cluster resources such as nodes, volumes, and networking rules, and the operator can interact with any Kubernetes or underlying cloud API.
-
-
-
The data owner, who owns the protected data. A data owner can verify the deployment using the Coordinator attestation service. The verification includes the identity, integrity, and confidentiality of the workloads, the runtime environment and the access permissions.
-
-
-
Contrast supports a trust model where the container image provider, workload operator, and data owner are separate, mutually distrusting parties.
-
The following diagram shows the system components and parties.
Contrast is designed to defend against five possible attacks:
-
-
A malicious cloud insider: malicious employees or third-party contractors of cloud service providers (CSPs) potentially have full access to various layers of the cloud infrastructure. That goes from the physical datacenter up to the hypervisor and Kubernetes layer. For example, they can access the physical memory of the machines, modify the hypervisor, modify disk contents, intercept network communications, and attempt to compromise the confidential container at runtime. A malicious insider can expand the attack surface or restrict the runtime environment. For example, a malicious operator can add a storage device to introduce new attack vectors. As another example, a malicious operator can constrain resources such as limiting a guest's memory size, changing its disk space, or changing firewall rules.
-
A malicious cloud co-tenant: malicious cloud user ("hackers") may break out of their tenancy and access other tenants' data. Advanced attackers may even be able to establish a permanent foothold within the infrastructure and access data over a longer period. The threats are analogous to the cloud insider access scenario, without the physical access.
-
A malicious workload operator: malicious workload operators, for example Kubernetes administrators, have full access to the workload deployment and the underlying Kubernetes platform. The threats are analogously to the cloud insider access scenario, with access to everything that's above the hypervisor level.
-
A malicious attestation client: this attacker connects to the attestation service and sends malformed request.
-
A malicious container image provider: a malicious container image provider has full control over the application development itself. This attacker might release a malicious version of the workload containing harmful operations.
Contrast shields a workload from the aforementioned threats with three main components:
-
-
The runtime environment safeguards against the physical memory and disk attack surface.
-
The runtime policies safeguard against the Kubernetes control plane and container runtime attack surface.
-
The service mesh safeguards against the network attack surface.
-
-
The following tables describe concrete threats and how they're mitigated in Contrast grouped by these categories:
-
-
Attacks on the confidential container environment
-
Attacks on the attestation service
-
Attacks on workloads
-
-
Attacks on the confidential container environment
-
This table describes potential threats and mitigation strategies related to the confidential container environment.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the network connection of the launcher or image repository.
An attacker can change the image URL and control the workload binary. However these actions are reflected in the attestation report. The image repository isn't controlled using an access list, therefore the image is assumed to be viewable by everyone. You must ensure that the workload container image doesn't contain any secrets.
An attacker modifies a container's runtime environment configuration in the Kubernetes control plane.
The attestation process and the runtime policies detects unsafe configurations that load non-authentic images or perform any other modification to the expected runtime environment.
This table describes potential threats and mitigation strategies to the attestation service.
-
Threat
Mitigation
Mitigation implementation
An attacker intercepts the Coordinator deployment and modifies the image or hijacks the runtime environment.
This threat is mitigated by having an attestation procedure and attested, encrypted TLS connections to the Coordinator. The attestation evidence for the Coordinator image is distributed with our releases, protected by supply chain security, and fully reproducible.
An attacker intercepts the network connection between the workload and the Coordinator and reads secret keys from the wire.
This threat is mitigated by having an attested, encrypted TLS connection. This connection helps protect the secrets from passive eavesdropping. The attacker can't create valid workload certificates that would be accepted in Contrast's service mesh. An attacker can't impersonate a valid workload container because the container's identity is guaranteed by the attestation protocol.
Within the network between your workload and the Coordinator.
An attacker exploits parsing discrepancies, which leads to undetected changes in the attestation process.
This risk is mitigated by having a parsing engine written in memory-safe Go that's tested against the attestation specification of the hardware vendor. The runtime policies are available as an attestation artifact for further inspection and audits to verify their effectiveness.
An attacker reads or modifies data written to disk via persistent volumes.
Currently, persistent volumes aren't supported in Contrast. In the future, this threat is mitigated by encrypted and integrity-protected volume mounts.
An attacker publishes a new image version containing malicious code.
The attestation process and the runtime policies require a data owner to accept a specific version of the workload and any update to the workload needs to be explicitly acknowledged.
The following table describes three example use cases and how they map to the defined threat model in this document:
-
Use Case
Example Scenario
Migrate sensitive workloads to the cloud
TechSolve Inc., a software development firm, aimed to enhance its defense-in-depth strategy for its cloud-based development environment, especially for projects involving proprietary algorithms and client data. TechSolve acts as the image provider, workload operator, and data owner, combining all three personas in this scenario. In our attestation terminology, they're the workload operator and relying party in one entity. Their threat model includes a malicious cloud insider and cloud co-tenant.
Make your SaaS more trustworthy
SaaSProviderX, a company offering cloud-based project management tools, sought to enhance its platform's trustworthiness amidst growing concerns about data breaches and privacy. Here, the relying party is the SaaS customer as the data owner. The goal is to achieve a form of operator exclusion and only allow selective operations on the deployment. Hence, their threat model includes a malicious workload operator.
Simplify regulatory compliance
HealthSecure Inc. has been managing a significant volume of sensitive patient data on-premises. With the increasing demand for advanced data analytics and the need for scalable infrastructure, the firm decides to migrate its data analytics operations to the cloud. However, the primary concern is maintaining the confidentiality and security of patient data during and after the migration, in compliance with healthcare regulations. In this compliance scenario, the regulator serves as an additional relying party. HealthSecure must implement a mechanism that ensures the isolation of patient data can be verifiably guaranteed to the regulator.
-
In each scenario, Contrast ensures exclusive data access and processing capabilities are confined to the designated workloads. It achieves this by effectively isolating the workload from the infrastructure and other components of the stack. Data owners are granted the capability to audit and approve the deployment environment before submitting their data, ensuring a secure handover. Meanwhile, workload operators are equipped to manage and operate the application seamlessly, without requiring direct access to either the workload or its associated data.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Components
-
Contrast is composed of several key components that work together to manage and scale confidential containers effectively within Kubernetes environments.
-This page provides an overview of the core components essential for deploying and managing Contrast.
The CLI serves as the primary management tool for Contrast deployments. It's designed to streamline the configuration and operation of Contrast in several ways:
-
-
Installation and setup: The CLI facilitates the installation of the necessary runtime classes required for Contrast to function within a Kubernetes cluster.
-
Policy generation: It allows users to generate runtime policies, adapt the deployment files, and generate the Contrast manifest.
-
Configuration management: Through the CLI, users can configure the Contrast Coordinator with the generated manifest.
-
Verification and attestation: The CLI provides tools to verify the integrity and authenticity of the Coordinator and the entire deployment via remote attestation.
The Contrast Coordinator is the central remote attestation service of a Contrast deployment.
-It runs inside a confidential container inside your cluster.
-The Coordinator can be verified via remote attestation, and a Contrast deployment is self-contained.
-The Coordinator is configured with a manifest, a configuration file containing the reference attestation values of your deployment.
-It ensures that your deployment's topology adheres to your specified manifest by verifying the identity and integrity of all confidential pods inside the deployment.
-The Coordinator is also a certificate authority and issues certificates for your workload pods during the attestation procedure.
-Your workload pods can establish secure, encrypted communication channels between themselves based on these certificates using the Coordinator as the root CA.
-As your app needs to scale, the Coordinator transparently verifies new instances and then provides them with their certificates to join the deployment.
-
To verify your deployment, the Coordinator's remote attestation statement combined with the manifest offers a concise single remote attestation statement for your entire deployment.
-A third party can use this to verify the integrity of your distributed app, making it easy to assure stakeholders of your app's identity and integrity.
The manifest is the configuration file for the Coordinator, defining your confidential deployment.
-It's automatically generated from your deployment by the Contrast CLI.
-It currently consists of the following parts:
-
-
Policies: The identities of your Pods, represented by the hashes of their respective runtime policies.
-
Reference Values: The remote attestation reference values for the Kata confidential micro-VM that's the runtime environment of your Pods.
-
WorkloadOwnerKeyDigest: The workload owner's public key digest. Used for authenticating subsequent manifest updates.
Runtime Policies are a mechanism to enable the use of the untrusted Kubernetes API for orchestration while ensuring the confidentiality and integrity of your confidential containers.
-They allow us to enforce the integrity of your containers' runtime environment as defined in your deployment files.
-The runtime policy mechanism is based on the Open Policy Agent (OPA) and translates the Kubernetes deployment YAML into the Rego policy language of OPA.
-The Kata Agent inside the confidential micro-VM then enforces the policy by only acting on permitted requests.
-The Contrast CLI provides the tooling for automatically translating Kubernetes deployment YAML into the Rego policy language of OPA.
Contrast provides an Initializer that handles the remote attestation on the workload side transparently and
-fetches the workload certificate. The Initializer runs as an init container before your workload is started.
-It provides the workload container and the service mesh sidecar with the workload certificates.
Contrast depends on a Kubernetes runtime class, which is installed
-by the node-installer DaemonSet.
-This runtime consists of a containerd runtime plugin, a virtual machine manager (cloud-hypervisor), and a podvm image (IGVM and rootFS).
-The installer takes care of provisioning every node in the cluster so it provides this runtime class.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Policies
-
Kata runtime policies are an integral part of the Confidential Containers preview on AKS.
-They prescribe how a Kubernetes pod must be configured to launch successfully in a confidential VM.
-In Contrast, policies act as a workload identifier: only pods with a policy registered in the manifest receive workload certificates and may participate in the confidential deployment.
-Verification of the Contrast Coordinator and its manifest transitively guarantees that all workloads meet the owner's expectations.
The Kata agent running in the confidential micro-VM exposes an RPC service AgentService to the Kata runtime.
-This service handles potentially untrustworthy requests from outside the TCB, which need to be checked against a policy.
-
Kata runtime policies are written in the policy language Rego.
-They specify what AgentService methods can be called, and the permissible parameters for each call.
-
Policies consist of two parts: a list of rules and a data section.
-While the list of rules is static, the data section is populated with information from the PodSpec and other sources.
Runtime policies are programmatically generated from Kubernetes manifests by the Contrast CLI.
-The generate subcommand inspects pod definitions and derives rules for validating the pod at the Kata agent.
-There are two important integrity checks: container image checksums and OCI runtime parameters.
-
For each of the container images used in a pod, the CLI downloads all image layers and produces a cryptographic dm-verity checksum.
-These checksums are the basis for the policy's storage data.
-
The CLI combines information from the PodSpec, ConfigMaps, and Secrets in the provided Kubernetes manifests to derive a permissible set of command-line arguments and environment variables.
-These constitute the policy's OCI data.
The generated policy document is annotated to the pod definitions in Base64 encoding.
-This annotation is propagated to the Kata runtime, which calculates the SHA256 checksum for the policy and uses that as SNP HOSTDATA for the confidential micro-VM.
-
After the VM launched, the runtime calls the agent's SetPolicy method with the full policy document.
-If the policy doesn't match the checksum in HOSTDATA, the agent rejects the policy.
-Otherwise, it applies the policy to all future AgentService requests.
Contrast verifies its confidential containers following these steps:
-
-
The Contrast CLI generates a policy and attaches it to the pod definition.
-
Kubernetes schedules the pod on a node with the confidential computing runtime.
-
Containerd invokes the Kata runtime to create the pod sandbox.
-
The Kata runtime starts a CVM with the policy's digest as HOSTDATA.
-
The Kata runtime sets the policy using the SetPolicy method.
-
The Kata agent verifies that the incoming policy's digest matches HOSTDATA.
-
The CLI sets a manifest in the Contrast Coordinator, including a list of permitted policies.
-
The Contrast Initializer sends an attestation report to the Contrast Coordinator, asking for a mesh certificate.
-
The Contrast Coordinator verifies that the started pod has a permitted policy hash in its HOSTDATA field.
-
-
After the last step, we know that the policy hasn't been tampered with and, thus, that the workload matches expectations and may receive mesh certificates.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Contrast Runtime
-
The Contrast runtime is responsible for starting pods as confidential virtual machines.
-This works by specifying the runtime class to be used in a pod spec and by registering the runtime class with the apiserver.
-The RuntimeClass resource defines a name for referencing the class and
-a handler used by the container runtime (containerd) to identify the class.
-
apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: # This name is used by pods in the runtimeClassName field name: contrast-cc-abcdef # This name is used by the # container runtime interface implementation (containerd) handler: contrast-cc-abcdef
-
Confidential pods that are part of a Contrast deployment need to specify the
-same runtime class in the runtimeClassName field, so Kubernetes uses the
-Contrast runtime instead of the default containerd / runc handler.
-
apiVersion: v1 kind: Pod spec: runtimeClassName: contrast-cc-abcdef # ...
The runtime consists of additional software components that need to be installed
-and configured on every SEV-SNP-enabled worker node.
-This installation is performed automatically by the node-installer DaemonSet.
The handler field in the Kubernetes RuntimeClass instructs containerd not to use the default runc implementation.
-Instead, containerd invokes a custom plugin called containerd-shim-contrast-cc-v2.
-This shim is described in more detail in the upstream source repository and in the containerd documentation.
The containerd shim uses cloud-hypervisor to create a confidential virtual machine for every pod.
-This requires the cloud-hypervisor binary to be installed on every node (responsibility of the node-installer).
Contrast uses a special containerd snapshotter (tardev) to provide container images as block devices to the pod-VM. This snapshotter consists of a host component that pulls container images and a guest component (kernel module) used to mount container images.
-The tardev snapshotter uses dm-verity to protect the integrity of container images.
-Expected dm-verity container image hashes are part of Contrast runtime policies and are enforced by the kata-agent.
-This enables workload attestation by specifying the allowed container image as part of the policy. Read the chapter on policies for more information.
Every pod-VM starts with the same guest image. It consists of an IGVM file and a root filesystem.
-The IGVM file describes the initial memory contents of a pod-VM and consists of:
-
-
Linux kernel image
-
initrd
-
kernel commandline
-
-
Additionally, a root filesystem image is used that contains a read-only partition with the user space of the pod-VM and a verity partition to guarantee the integrity of the root filesystem.
-The root filesystem contains systemd as the init system, and the kata agent for managing the pod.
-
This pod-VM image isn't specific to any pod workload. Instead, container images are mounted at runtime.
The RuntimeClass resource above registers the runtime with the Kubernetes api.
-The node-level installation is carried out by the Contrast node-installer
-DaemonSet that ships with every Contrast release.
-
After deploying the installer, it performs the following steps on each node:
-
-
Install the Contrast containerd shim (containerd-shim-contrast-cc-v2)
-
Install cloud-hypervisor as the virtual machine manager (VMM)
-
Install an IGVM file for pod-VMs of this class
-
Install a read only root filesystem disk image for the pod-VMs of this class
-
Reconfigure containerd by adding a runtime plugin that corresponds to the handler field of the Kubernetes RuntimeClass
-
Restart containerd to make it aware of the new plugin
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Service mesh
-
The Contrast service mesh secures the communication of the workload by automatically
-wrapping the network traffic inside mutual TLS (mTLS) connections. The
-verification of the endpoints in the connection establishment is based on
-certificates that are part of the
-PKI of the Coordinator.
-
The service mesh can be enabled on a per-workload basis by adding a service mesh
-configuration to the workload's object annotations. During the contrast generate
-step, the service mesh is added as a sidecar
-container to
-all workloads which have a specified configuration. The service mesh container first
-sets up iptables rules based on its configuration and then starts
-Envoy for TLS origination and termination.
The service mesh container can be configured using the following object annotations:
-
-
contrast.edgeless.systems/servicemesh-ingress to configure ingress.
-
contrast.edgeless.systems/servicemesh-egress to configure egress.
-
contrast.edgeless.systems/servicemesh-admin-interface-port to configure the Envoy
-admin interface. If not specified, no admin interface will be started.
-
-
If you aren't using the automatic service mesh injection and want to configure the
-service mesh manually, set the environment variables CONTRAST_INGRESS_PROXY_CONFIG,
-CONTRAST_EGRESS_PROXY_CONFIG and CONTRAST_ADMIN_PORT in the service mesh sidecar directly.
All TCP ingress traffic is routed over Envoy by default. Since we use
-TPROXY, the destination address
-remains the same throughout the packet handling.
-
Any incoming connection is required to present a client certificate signed by the
-mesh CA certificate.
-Envoy presents a certificate chain of the mesh
-certificate of the workload and the intermediate CA certificate as the server certificate.
-
If the deployment contains workloads which should be reachable from outside the
-Service Mesh, while still handing out the certificate chain, disable client
-authentication by setting the annotation contrast.edgeless.systems/servicemesh-ingress as
-<name>#<port>#false. Separate multiple entries with ##. You can choose any
-descriptive string identifying the service on the given port for the <name> field,
-as it's only informational.
-
Disable redirection and TLS termination altogether by specifying
-<name>#<port>#true. This can be beneficial if the workload itself handles TLS
-on that port or if the information exposed on this port is non-sensitive.
-
The following example workload exposes a web service on port 8080 and metrics on
-port 7890. The web server is exposed to a 3rd party end-user which wants to
-verify the deployment, therefore it's still required that the server hands out
-it certificate chain signed by the mesh CA certificate. The metrics should be
-exposed via TCP without TLS.
Note, that changing the environment variables of the sidecar container directly will
-only have an effect if the workload isn't configured to automatically generate a
-service mesh component on contrast generate. Otherwise, the service mesh sidecar
-container will be regenerated on every invocation of the command.
To be able to route the egress traffic of the workload through Envoy, the remote
-endpoints' IP address and port must be configurable.
-
-
Choose an IP address inside the 127.0.0.0/8 CIDR and a port not yet in use
-by the pod.
-
Configure the workload to connect to this IP address and port.
-
Set <name>#<chosen IP>:<chosen port>#<original-hostname-or-ip>:<original-port>
-as the contrast.edgeless.systems/servicemesh-egress workload annotation. Separate multiple
-entries with ##. Choose any string identifying the service on the given port as
-<name>.
-
-
This redirects the traffic over Envoy. The endpoint must present a valid
-certificate chain which must be verifiable with the
-mesh CA certificate.
-Furthermore, Envoy uses a certificate chain with the mesh certificate of the workload
-and the intermediate CA certificate as the client certificate.
-
The following example workload has no ingress connections and two egress
-connection to different microservices. The microservices are part
-of the confidential deployment. One is reachable under billing-svc:8080 and
-the other under cart-svc:8080.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Workload deployment
-
The following instructions will guide you through the process of making an existing Kubernetes deployment
-confidential and deploying it together with Contrast.
-
A running CoCo-enabled cluster is required for these steps, see the setup guide on how to set it up.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Your Kubernetes resources need some modifications to run as Confidential Containers.
-This section guides you through the process and outlines the necessary changes.
Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files
-unchanged, you can copy the files into a separate local directory.
-You can also generate files from a Helm chart or from a Kustomization.
To specify that a workload (pod, deployment, etc.) should be deployed as confidential containers,
-add runtimeClassName: contrast-cc to the pod spec (pod definition or template).
-This is a placeholder name that will be replaced by a versioned runtimeClassName when generating policies.
In the initialization process, the contrast-tls-certs shared volume is populated with X.509 certificates for your workload.
-These certificates are used by the Contrast Service Mesh, but can also be used by your application directly.
-The following tab group explains the setup for both scenarios.
-
Drop-in service mesh
Go integration
Contrast can be configured to handle TLS in a sidecar container.
-This is useful for workloads that are hard to configure with custom certificates, like Java applications.
Configuration of the sidecar depends heavily on the application.
-The following example is for an application with these properties:
-
The container has a main application at TCP port 8001, which should be TLS-wrapped and doesn't require client authentication.
-
The container has a metrics endpoint at TCP port 8080, which should be accessible in plain text.
-
All other endpoints require client authentication.
-
The app connects to a Kubernetes service backend.default:4001, which requires client authentication.
During the generate step, this configuration will be translated into a Service Mesh sidecar container which handles TLS connections automatically.
-The only change required to the app itself is to let it connect to 127.0.0.2:4001 to reach the backend service.
-You can find more detailed documentation in the Service Mesh chapter.
The mesh certificate contained in certChain.pem authenticates this workload, while the mesh CA certificate mesh-ca.pem authenticates its peers.
-Your app should turn on client authentication to ensure peers are running as confidential containers, too.
-See the Certificate Authority section for detailed information about these certificates.
The following example shows how to configure a Golang app, with error handling omitted for clarity.
Run the generate command to add the necessary components to your deployment files.
-This will add the Contrast Initializer to every workload with the specified contrast-cc runtime class
-and the Contrast Service Mesh to all workloads that have a specified configuration.
-After that, it will generate the execution policies and add them as annotations to your deployment files.
-A manifest.json with the reference values of your deployment will be created.
Please be aware that runtime policies currently have some blind spots. For example, they can't guarantee the starting order of containers. See the current limitations for more details.
-
If you don't want the Contrast Initializer to automatically be added to your
-workloads, there are two ways you can skip the Initializer injection step,
-depending on how you want to customize your deployment.
-
Command-line flag
Per-workload annotation
You can disable the Initializer injection completely by specifying the
---skip-initializer flag in the generate command.
If you want to disable the Initializer injection for a specific workload with
-the contrast-cc runtime class, you can do so by adding an annotation to the workload.
When disabling the automatic Initializer injection, you can manually add the
-Initializer as a sidecar container to your workload before generating the
-policies. Configure the workload to use the certificates written to the
-contrast-tls-certsvolumeMount.
For the next steps, we will need to connect to the Coordinator. The released Coordinator resource
-includes a LoadBalancer definition we can use.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
-
Port-forwarding of Confidential Containers
kubectl port-forward uses a Container Runtime Interface (CRI) method that isn't supported by the Kata shim.
-If you can't use a public load balancer, you can deploy a port-forwarder.
-The port-forwarder relays traffic from a CoCo pod and can be accessed via kubectl port-forward.
This will use the reference values from the manifest file to attest the Coordinator.
-After this step, the Coordinator will start issuing TLS certificates to the workloads. The init container
-will fetch a certificate for the workload and the workload is started.
An end user (data owner) can verify the Contrast deployment using the verify command.
-
contrast verify -c"${coordinator}:1313"
-
The CLI will attest the Coordinator using the reference values from the given manifest file. It will then write the
-service mesh root certificate and the history of manifests into the verify/ directory. In addition, the policies
-referenced in the active manifest are also written to the directory. The verification will fail if the active
-manifest at the Coordinator doesn't match the manifest passed to the CLI.
You can securely connect to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-First, expose the service on a public IP address via a LoadBalancer service:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the SAN field.
-Validation fails since the certificate contains no IP entries as a subject alternative name (SAN).
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
If the Contrast Coordinator restarts, it enters recovery mode and waits for an operator to provide key material.
-For demonstration purposes, you can simulate this scenario by deleting the Coordinator pod.
-
kubectl delete pod -l app.kubernetes.io/name=coordinator
-
Kubernetes schedules a new pod, but that pod doesn't have access to the key material the previous pod held in memory and can't issue certificates for workloads yet.
-You can confirm this by running verify again, or you can restart a workload pod, which should stay in the initialization phase.
-However, the secret seed in your working directory is sufficient to recover the coordinator.
-
contrast recover -c"${coordinator}:1313"
-
Now that the Coordinator is recovered, all workloads should pass initialization and enter the running state.
-You can now verify the Coordinator again, which should return the same manifest you set before.
-
warning
The recovery process invalidates the mesh CA certificate:
-existing workloads won't be able to communicate with workloads newly spawned.
-All workloads should be restarted after the recovery succeeded.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
This is documentation for Contrast 0.9, which is no longer actively maintained.
For up-to-date documentation, see the latest version (1.0).
Version: 0.9
Confidential emoji voting
-
-
This tutorial guides you through deploying emojivoto as a
-confidential Contrast deployment and validating the deployment from a voters perspective.
-
Emojivoto is an example app allowing users to vote for different emojis and view votes
-on a leader board. It has a microservice architecture consisting of a
-web frontend (web), a gRPC backend for listing available emojis (emoji), and a backend for
-the voting and leader board logic (voting). The vote-bot simulates user traffic by submitting
-votes to the frontend.
Using a voting service, users' votes are considered highly sensitive data, as we require
-a secret ballot. Also, users are likely interested in the fairness of the ballot. For
-both requirements, we can use Confidential Computing and, specifically, workload attestation
-to prove to those interested in voting that the app is running in a protected environment
-where their votes are processed without leaking to the platform provider or workload owner.
Contrast depends on a custom Kubernetes RuntimeClass (contrast-cc),
-which needs to be installed in the cluster prior to the Coordinator or any confidential workloads.
-This consists of a RuntimeClass resource and a DaemonSet that performs installation on worker nodes.
-This step is only required once for each version of the runtime.
-It can be shared between Contrast deployments.
Run the generate command to generate the execution policies and add them as
-annotations to your deployment files. A manifest.json file with the reference values
-of your deployment will be created:
The deployment YAML shipped for this demo is already configured to be used with Contrast.
-A runtime classcontrast-cc-<platform>-<runtime-hash>
-was added to the pods to signal they should be run as Confidential Containers. During the generation process,
-the Contrast Initializer will be added as an init container to these
-workloads to facilitate the attestation and certificate pulling before the actual workload is started.
Further, the deployment YAML is also configured with the Contrast service mesh.
-The configured service mesh proxy provides transparent protection for the communication between
-the different components of emojivoto.
Configure the coordinator with a manifest. It might take up to a few minutes
-for the load balancer to be created and the Coordinator being available.
-
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"The user API of your Contrast Coordinator is available at $coordinator:1313" contrast set-c"${coordinator}:1313" deployment/
-
The CLI will use the reference values from the manifest to attest the Coordinator deployment
-during the TLS handshake. If the connection succeeds, it's ensured that the Coordinator
-deployment hasn't been tampered with.
Now that the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload,
-we can deploy the application:
-
kubectl apply -f deployment/
-
Inter-deployment communication
The Contrast Coordinator issues mesh certificates after successfully validating workloads.
-These certificates can be used for secure inter-deployment communication. The Initializer
-sends an attestation report to the Coordinator, retrieves certificates and a private key in return
-and writes them to a volumeMount. The service mesh sidecar is configured to use the credentials
-from the volumeMount when communicating with other parts of the deployment over mTLS.
-The public facing frontend for voting uses the mesh certificate without client authentication.
As voters, we want to verify the fairness and confidentiality of the deployment before
-deciding to vote. Regardless of the scale of our distributed deployment, Contrast only
-needs a single remote attestation step to verify the deployment. By doing remote attestation
-of the Coordinator, we transitively verify those systems the Coordinator has already attested
-or will attest in the future. Successful verification of the Coordinator means that
-we can be sure it will enforce the configured manifest.
The CLI will attest the Coordinator using the reference values from a given manifest. This manifest needs
-to be communicated out of band to everyone wanting to verify the deployment, as the verify command checks
-if the currently active manifest at the Coordinator matches the manifest given to the CLI. If the command succeeds,
-the Coordinator deployment was successfully verified to be running in the expected Confidential
-Computing environment with the expected code version. The Coordinator will then return its
-configuration over the established TLS channel. The CLI will store this information, namely the root
-certificate of the mesh (mesh-ca.pem) and the history of manifests, into the verify/ directory.
-In addition, the policies referenced in the manifest history are also written into the same directory.
In the next step, the Coordinator configuration that was written by the verify command needs to be audited.
-A potential voter should inspect the manifest and the referenced policies. They could delegate
-this task to an entity they trust.
After ensuring the configuration of the Coordinator fits the expectation, you can securely connect
-to the workloads using the Coordinator's mesh-ca.pem as a trusted CA certificate.
-
To access the web frontend, expose the service on a public IP address via a LoadBalancer service:
-
frontendIP=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}') echo"Frontend is available at https://$frontendIP, you can visit it in your browser."
-
Using openssl, the certificate of the service can be validated with the mesh-ca.pem:
By default, mesh certificates are issued with a wildcard DNS entry. The web frontend is accessed
-via load balancer IP in this demo. Tools like curl check the certificate for IP entries in the subject alternative name (SAN) field.
-Validation fails since the certificate contains no IP entries as a SAN.
-For example, a connection attempt using the curl and the mesh CA certificate with throw the following error:
-
$ curl--cacert ./verify/mesh-ca.pem "https://${frontendIP}:443" curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
The Policies section of the manifest maps policy hashes to a list of SANs. To enable certificate verification
-of the web frontend with tools like curl, edit the policy with your favorite editor and add the frontendIP to
-the list that already contains the "web" DNS entry:
Next, set the changed manifest at the coordinator with:
-
contrast set-c"${coordinator}:1313" deployment/
-
The Contrast Coordinator will rotate the mesh ca certificate on the manifest update. Workload certificates issued
-after the manifest update are thus issued by another certificate authority and services receiving the new CA certificate chain
-won't trust parts of the deployment that got their certificate issued before the update. This way, Contrast ensures
-that parts of the deployment that received a security update won't be infected by parts of the deployment at an older
-patch level that may have been compromised. The mesh-ca.pem is updated with the new CA certificate chain.
The Coordinator has the new manifest set, but the different containers of the app are still
-using the older certificate authority. The Contrast Initializer terminates after the initial attestation
-flow and won't pull new certificates on manifest updates.
After the update has been rolled out, connecting to the frontend using curl will successfully validate
-the service certificate and return the HTML document of the voting site:
Platform support: At present, Contrast is exclusively available on Azure AKS, supported by the Confidential Container preview for AKS. Expansion to other cloud platforms is planned, pending the availability of necessary infrastructure enhancements.
-
Bare-metal support: Support for running Contrast on bare-metal Kubernetes will be available soon for AMD SEV and Intel TDX.
Persistent volumes: Contrast only supports volumes with volumeMode: Block. These block devices are provided by the untrusted environment and should be treated accordingly. We plan to provide transparent encryption on top of block devices in a future release.
Resource limits: There is an existing bug on AKS where container memory limits are incorrectly applied. The current workaround involves using only memory requests instead of limits.
Coverage: While the enforcement of workload policies generally functions well, there are scenarios not yet fully covered. It's crucial to review deployments specifically for these edge cases.
-
Order of events: The current policy evaluation mechanism on API requests isn't stateful, so it can't ensure a prescribed order of events. Consequently, there's no guaranteed enforcement that the service mesh sidecar container runs before the workload container. This order ensures that all traffic between pods is securely encapsulated within TLS connections.
-
Absence of events: Policies can't ensure certain events have happened. A container, such as the service mesh sidecar, can be omitted entirely. Environment variables may be missing.
-
Volume integrity checks: While persistent volumes aren't supported yet, integrity checks don't currently cover other objects such as ConfigMaps and Secrets.
-
-
warning
The policy limitations, in particular the missing guarantee that our service mesh sidecar has been started before the workload container affects the service mesh implementation of Contrast. Currently, this requires inspecting the iptables rules on startup or terminating TLS connections in the workload directly.
CLI availability: The CLI tool is currently only available for Linux. This limitation arises because certain upstream dependencies haven't yet been ported to other platforms.
The Contrast Coordinator is a singleton and can't be scaled to more than one instance.
-When this instance's pod is restarted, for example for node maintenance, it needs to be recovered manually.
-In a future release, we plan to support distributed Coordinator instances that can recover automatically.
CoCo on AKS is currently in preview. An extension for the az CLI is needed to create such a cluster.
-Add the extension with the following commands:
-
az extension add\ --name aks-preview \ --allow-preview true az extension update \ --name aks-preview \ --allow-preview true
-
Then register the required feature flags in your subscription to allow access to the public preview:
-
az feature register \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"
-
The registration can take a few minutes. The status of the operation can be checked with the following
-command, which should show the registration state as Registered:
-
az feature show \ --namespace"Microsoft.ContainerService"\ --name"KataCcIsolationPreview"\ --output table
-
Afterward, refresh the registration of the ContainerService provider:
-
az provider register \ --namespace"Microsoft.ContainerService"
First, we need to create an AKS cluster. We can't directly create a CoCo-enabled cluster, so we'll need to create a
-non-CoCo cluster first, and then add a CoCo node pool, optionally replacing the non-CoCo node pool.
-
We'll first start by creating the non-CoCo cluster:
-
# Select the name for your AKS cluster azClusterName="ContrastDemo" az aks create \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"\ --kubernetes-version 1.29\ --os-sku AzureLinux \ --node-vm-size Standard_DC4as_cc_v5 \ --node-count 1\ --generate-ssh-keys
After trying out Contrast, you might want to clean up the cloud resources created in this step.
-In case you've created a new resource group, you can just delete that group with
-
az group delete \ --name"${azResourceGroup:?}"
-
Deleting the resource group will also delete the cluster and all other related resources.
-
To only cleanup the AKS cluster and node pools, run
-
az aks delete \ --resource-group "${azResourceGroup:?}"\ --name"${azClusterName:?}"
Collecting logs can be a good first step to identify problems in your
-deployment. Both the CLI and the Contrast Coordinator as well as the Initializer
-can be configured to emit additional logs.
The CLI logs can be configured with the --log-level command-line flag, which
-can be set to either debug, info, warn or error. The default is info.
-Setting this to debug can get more fine-grained information as to where the
-problem lies.
The logs from the Coordinator and the Initializer can be configured via the
-environment variables CONTRAST_LOG_LEVEL, CONTRAST_LOG_FORMAT and
-CONTRAST_LOG_SUBSYSTEMS.
-
-
CONTRAST_LOG_LEVEL can be set to one of either debug, info, warn, or
-error, similar to the CLI (defaults to info).
-
CONTRAST_LOG_FORMAT can be set to text or json, determining the output
-format (defaults to text).
-
CONTRAST_LOG_SUBSYSTEMS is a comma-seperated list of subsystems that should
-be enabled for logging, which are disabled by default. Subsystems include:
-kds-getter, issuer and validator.
-To enable all subsystems, use * as the value for this environment variable.
-Warnings and error messages from subsystems get printed regardless of whether
-the subsystem is listed in the CONTRAST_LOG_SUBSYSTEMS environment variable.
-
-
To configure debug logging with all subsystems for your Coordinator, add the
-following variables to your container definition.
While the Contrast Coordinator has a policy that allows certain configurations,
-the Initializer and service mesh don't. When changing environment variables of other
-parts than the Coordinator, ensure to rerun contrast generate to update the policy.
-
To access the logs generated by the Coordinator, you can use kubectl with the
-following command:
If the Coordinator or a workload pod fails to even start, it can be helpful to
-look at the events of the pod during the startup process using the describe
-command.
LAST SEEN TYPE REASON OBJECT MESSAGE 32m Warning Failed Pod/coordinator-0 kubelet Error: failed to create containerd task: failed to create shim task: "CreateContainerRequest is blocked by policy: ...
-
A common error, as in this example, is that the container creation was blocked by the
-policy. Potential reasons are a modification of the deployment YAML without updating
-the policies afterward, or a version mismatch between Contrast components.
When generating the policies, Contrast will download the images specified in your deployment
-YAML and include their cryptographic identity. If the image tag is moved to another
-container image after the policy has been generated, the image downloaded at deploy time
-will differ from the one at generation time, and the policy enforcement won't allow the
-container to be started in the pod VM.
-
To ensure the correct image is always used, pin the container image to a fixed sha256:
A version mismatch between Contrast components can cause policy validation or attestation
-to fail. Each Contrast runtime is identifiable based on its (shortened) measurement value
-used to name the runtime class version.
-
First, analyze which runtime class is currently installed in your cluster by running
-
kubectl get runtimeclasses
-
This should give you output similar to the following one.
-
NAME HANDLER AGE contrast-cc-aks-clh-snp-7173acb5 contrast-cc-aks-clh-snp-7173acb5 23h kata-cc-isolation kata-cc 45d
-
The output shows that there are four Contrast runtime classes installed (as well as the runtime class provided
-by the AKS CoCo preview, which isn't used by Contrast).
-
Next, check if the pod that won't start has the correct runtime class configured, and the
-Coordinator uses the exact same runtime:
-
kubectl -n<namespace> get -o=jsonpath='{.spec.runtimeClassName}' pod/<pod-name> kubectl -n<namespace> get -o=jsonpath='{.spec.runtimeClassName}' pod/<coordinator-pod-name>
-
The output should list the runtime class the pod is using:
-
contrast-cc-aks-clh-snp-7173acb5
-
Version information about the currently used CLI can be obtained via the version flag:
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
Version: 1.0
CLI telemetry
-
The Contrast CLI sends telemetry data to Edgeless Systems when you use CLI commands.
-This allows to understand how Contrast is used and to improve it.
-
The CLI sends the following data:
-
-
The CLI version
-
The CLI target OS and architecture (GOOS and GOARCH)
-
The command that was run
-
The kind of error that occurred (if any)
-
-
The CLI doesn't collect sensitive information.
-The implementation is open-source and can be reviewed.
-
IP addresses may be processed or stored for security purposes.
-
The data that the CLI collects adheres to the Edgeless Systems privacy policy.
-
You can disable telemetry by setting the environment variable DO_NOT_TRACK=1 before running the CLI.
-
-
\ No newline at end of file
diff --git a/pr-preview/pr-918/architecture/attestation.html b/pr-preview/pr-918/architecture/attestation.html
deleted file mode 100644
index 5b2cf6c210..0000000000
--- a/pr-preview/pr-918/architecture/attestation.html
+++ /dev/null
@@ -1,104 +0,0 @@
-
-
-
-
-
-Attestation in Contrast | Contrast
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
Version: 1.0
Attestation in Contrast
-
This document describes the attestation architecture of Contrast, adhering to the definitions of Remote ATtestation procedureS (RATS) in RFC 9334.
-The following gives a detailed description of Contrast's attestation architecture.
-At the end of this document, we included an FAQ that answers the most common questions regarding attestation in hindsight of the security benefits.
Contrast integrates with the RATS architecture, leveraging their definition of roles and processes including Attesters, Verifiers, and Relying Parties.
-
-
Figure 1: Conceptual attestation architecture. Taken from RFC 9334.
-
-
Attester: Assigned to entities that are responsible for creating Evidence which is then sent to a Verifier.
-
Verifier: These entities utilize the Evidence, Reference Values, and Endorsements. They assess the trustworthiness of the Attester by applying an Appraisal Policy for Evidence. Following this assessment, Verifiers generate Attestation Results for use by Relying Parties. The Appraisal Policy for Evidence may be provided by the Verifier Owner, programmed into the Verifier, or acquired through other means.
-
Relying Party: Assigned to entities that utilize Attestation Results, applying their own appraisal policies to make specific decisions, such as authorization decisions. This process is referred to as the "appraisal of Attestation Results." The Appraisal Policy for Attestation Results might be sourced from the Relying Party Owner, configured by the owner, embedded in the Relying Party, or obtained through other protocols or mechanisms.
This includes all Pods of the Contrast deployment that run inside Confidential Containers and generate cryptographic evidence reflecting their current configuration and state.
-Their evidence is rooted in the hardware measurements from the CPU and their confidential VM environment.
-The details of this evidence are given below in the section on evidence generation and appraisal.
-
-
Figure 2: Attestation flow of a confidential pod. Based on the layered attester graphic in RFC 9334.
-
Pods run in Contrast's runtime environment (B), effectively within a confidential VM.
-During launch, the CPU (A) measures the initial memory content of the confidential VM that contains Contrast's pod-VM image and generates the corresponding attestation evidence.
-The image is in IGVM format, encapsulating all information required to launch a virtual machine, including the kernel, the initramfs, and kernel cmdline.
-The kernel cmdline contains the root hash for dm-verity that ensures the integrity of the root filesystem.
-The root filesystem contains all components of the container's runtime environment including the guest agent (C).
-
In the userland, the guest agent takes care of enforcing the runtime policy of the pod.
-While the policy is passed in during the initialization procedure via the host, the evidence for the runtime policy is part of the CPU measurements.
-During the deployment the policy is annotated to the Kubernetes Pod resources.
-On AMD SEV-SNP the hash of the policy is then added to the attestation report via the HOSTDATA field by the hypervisor.
-When provided with the policy from the Kata host, the guest agent verifies that the policy's hash matches the one in the HOSTDATA field.
-
In summary a Pod's evidence is the attestation report of the CPU that provides evidence for runtime environment and the runtime policy.
The Coordinator acts as a verifier within the Contrast deployment, configured with a Manifest that defines the reference values and serves as an appraisal policy for all pods in the deployment.
-It also pulls endorsements from hardware vendors to verify the hardware claims.
-The Coordinator operates within the cluster as a confidential container and provides similar evidence as any other Pod when it acts as an attester.
-In RATS terminology, the Coordinator's dual role is defined as a lead attester in a composite device which spans the entire deployment: Coordinator and the workload pods.
-It collects evidence from other attesters and conveys it to a verifier, generating evidence about the layout of the whole composite device based on the Manifest as the appraisal policy.
-
-
Figure 3: Contrast deployment as a composite device. Based on the composite device in RFC 9334.
-
The CLI serves as the verifier for the Coordinator and the entire Contrast deployment, containing the reference values for the Coordinator and the endorsements from hardware vendors.
-These reference values are built into the CLI during our release process and can be reproduced offline via reproducible builds.
A relying party in the Contrast scenario could be, for example, the data owner that interacts with the application.
-The relying party can use the CLI to obtain the attestation results and Contrast's CA certificates bound to these results.
-The CA certificates can then be used by the relying party to authenticate the application, for example through TLS connections.
In Contrast, attestation evidence revolves around a hardware-generated attestation report, which contains several critical pieces of information:
-
-
The hardware attestation report: This report includes details such as the chip identifier, platform information, microcode versions, and comprehensive guest measurements. The entire report is signed by the CPU's private key, ensuring the authenticity and integrity of the data provided.
-
The launch measurements: Included within the hardware attestation report, this is a digest generated by the CPU that represents a hash of all initial guest memory pages. This includes essential components like the kernel, initramfs, and the kernel command line. Notably, it incorporates the root filesystem's dm-verity root hash, verifying the integrity of the root filesystem.
-
The runtime policy hash: Also part of the hardware attestation report, this field contains the hash of the Rego policy which dictates all expected API commands and their values from the host to the Kata guest agent. It encompasses crucial settings such as dm-verity hashes for the container image layers, environment variables, and mount points.
The appraisal of this evidence in Contrast is governed by two main components:
-
-
The Manifest: A JSON file used by the Coordinator to align with reference values. It sets the expectations for runtime policy hashes for each pod and includes what should be reported in the hardware attestation report for each component of the deployment.
-
The CLI's appraisal policy: This policy encompasses expected values of the Coordinator’s guest measurements and its runtime policy. It's embedded into the CLI during the build process and ensures that any discrepancy between the built-in values and those reported by the hardware attestation can be identified and addressed. The integrity of this policy is safeguardable through reproducible builds, allowing verification against the source code reference.
-
-
Frequently asked questions about attestation in Contrast
-
What's the purpose of remote attestation in Contrast?
-
Remote attestation in Contrast ensures that software runs within a secure, isolated confidential computing environment.
-This process certifies that the memory is encrypted and confirms the integrity and authenticity of the software running within the deployment.
-By validating the runtime environment and the policies enforced on it, Contrast ensures that the system operates in a trustworthy state and hasn't been tampered with.
-
How does Contrast ensure the security of the attestation process?
-
Contrast leverages hardware-rooted security features such as AMD SEV-SNP to generate cryptographic evidence of a pod’s current state and configuration.
-This evidence is checked against pre-defined appraisal policies to guarantee that only verified and authorized pods are part of a Contrast deployment.
Attestation confirms the integrity of the runtime environment and the identity of the workloads.
-It plays a critical role in preventing unauthorized changes and detecting potential modifications at runtime.
-The attestation provides integrity and authenticity guarantees, enabling relying parties—such as workload operators or data owners—to confirm the effective protection against potential threats, including malicious cloud insiders, co-tenants, or compromised workload operators.
-More details on the specific security benefits can be found here.
-
How can you verify the authenticity of attestation results?
-
Attestation results in Contrast are tied to cryptographic proofs generated and signed by the hardware itself.
-These proofs are then verified using public keys from trusted hardware vendors, ensuring that the results aren't only accurate but also resistant to tampering.
-For further authenticity verification, all of Contrast's code is reproducibly built, and the attestation evidence can be verified locally from the source code.
-
How are attestation results used by relying parties?
-
Relying parties use attestation results to make informed security decisions, such as allowing access to sensitive data or resources only if the attestation verifies the system's integrity.
-Thereafter, the use of Contrast's CA certificates in TLS connections provides a practical approach to communicate securely with the application.
In summary, Contrast's attestation strategy adheres to the RATS guidelines and consists of robust verification mechanisms that ensure each component of the deployment is secure and trustworthy.
-This comprehensive approach allows Contrast to provide a high level of security assurance to its users.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
Version: 1.0
Certificate authority
-
The Coordinator acts as a certificate authority (CA) for the workloads
-defined in the manifest.
-After a workload pod's attestation has been verified by the Coordinator,
-it receives a mesh certificate and the mesh CA certificate.
-The mesh certificate can be used for example in a TLS connection as the server or
-client certificate to proof to the other party that the workload has been
-verified by the Coordinator. The other party can verify the mesh certificate
-with the mesh CA certificate. While the certificates can be used by the workload
-developer in different ways, they're automatically used in Contrast's service
-mesh to establish mTLS connections between workloads in the same deployment.
The Coordinator establishes a public key infrastructure (PKI) for all workloads
-contained in the manifest. The Coordinator holds three certificates: the root CA
-certificate, the intermediate CA certificate, and the mesh CA certificate.
-The root CA certificate is a long-lasting certificate and its private key signs
-the intermediate CA certificate. The intermediate CA certificate and the mesh CA
-certificate share the same private key. This intermediate private key is used
-to sign the mesh certificates. Moreover, the intermediate private key and
-therefore the intermediate CA certificate and the mesh CA certificate are
-rotated when setting a new manifest.
Depending on the configuration of the first manifest, it allows the workload
-owner to update the manifest and, therefore, the deployment.
-Workload owners and data owners can be mutually untrusted parties.
-To protect against the workload owner silently introducing malicious containers,
-the Coordinator rotates the intermediate private key every time the manifest is
-updated and, therefore, the
-intermediate CA certificate and mesh CA certificate. If the user doesn't
-trust the workload owner, they use the mesh CA certificate obtained when they
-verified the Coordinator and the manifest. This ensures that the user only
-connects to workloads defined in the manifest they verified since only those
-workloads' certificates are signed with this intermediate private key.
-
Similarly, the service mesh also uses the mesh CA certificate obtained when the
-workload was started, so the workload only trusts endpoints that have been
-verified by the Coordinator based on the same manifest. Consequently, a
-manifest update requires a fresh rollout of the services in the service mesh.
The root CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This should only be used if the data owner trusts all future updates to the
-manifest and workloads. This is, for instance, the case when the workload owner is
-the same entity as the data owner.
-
The mesh CA certificate is returned when verifying the Coordinator.
-The data owner can use it to verify the mesh certificates of the workloads.
-This certificate is bound to the manifest set when the Coordinator is verified.
-If the manifest is updated, the mesh CA certificate changes.
-New workloads will receive mesh certificates signed by the new mesh CA certificate.
-The Coordinator with the new manifest needs to be verified to retrieve the new mesh CA certificate.
-The service mesh also uses the mesh CA certificate to verify the mesh certificates.
-
The intermediate CA certificate links the root CA certificate to the
-mesh certificate so that the mesh certificate can be verified with the root CA
-certificate. It's part of the certificate chain handed out by
-endpoints in the service mesh.
-
The mesh certificate is part of the certificate chain handed out by
-endpoints in the service mesh. During the startup of a pod, the Initializer
-requests a certificate from the Coordinator. This mesh certificate will be returned if the Coordinator successfully
-verifies the workload. The mesh certificate
-contains X.509 extensions with information from the workloads attestation
-document.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
Version: 1.0
Observability
-
The Contrast Coordinator can expose metrics in the
-Prometheus format. These can be monitored to quickly
-identify problems in the gRPC layer or attestation errors. Prometheus metrics
-are numerical values associated with a name and additional key/values pairs,
-called labels.
The metrics can be accessed at the Coordinator pod at the port specified in the
-CONTRAST_METRICS_PORT environment variable under the /metrics endpoint. By
-default, this environment variable isn't specified, hence no metrics will be
-exposed.
-
The Coordinator exports gRPC metrics under the prefix contrast_grpc_server_.
-These metrics are labeled with the gRPC service name and method name.
-Metrics of interest include contrast_grpc_server_handled_total, which counts
-the number of requests by return code, and
-contrast_grpc_server_handling_seconds_bucket, which produces a histogram of
-request latency.
The meshapi.MeshAPI service records metrics for the method NewMeshCert, which
-gets called by the Initializer when starting a
-new workload. Attestation failures from workloads to the Coordinator can be
-tracked with the counter contrast_meshapi_attestation_failures_total.
-
The current manifest generation is exposed as a
-gauge with the metric
-name contrast_coordinator_manifest_generation. If no manifest is set at the
-Coordinator, this counter will be zero.
The Service Mesh can be configured to expose
-metrics via its Envoy admin
-interface. Be
-aware that the admin interface can expose private information and allows
-destructive operations to be performed. To enable the admin interface for the
-Service Mesh, set the annotation
-contrast.edgeless.systems/servicemesh-admin-interface-port in the configuration
-of your workload. If this annotation is set, the admin interface will be started
-on this port.
-
To access the admin interface, the ingress settings of the Service Mesh have to
-be configured to allow access to the specified port (see Configuring the
-Proxy). All metrics will be
-exposed under the /stats endpoint. Metrics in Prometheus format can be scraped
-from the /stats/prometheus endpoint.
Contrast is now open source! Check it out on GitHub and leave us a ⭐️ if you like it!
Version: 1.0
Secrets & recovery
-
When the Coordinator is configured with the initial manifest, it generates a random secret seed.
-From this seed, it uses an HKDF to derive the CA root key and a signing key for the manifest history.
-This derivation is deterministic, so the seed can be used to bring any Coordinator to this Coordinator's state.
-
The secret seed is returned to the user on the first call to contrast set, encrypted with the user's public seed share owner key.
-If no seed share owner key is provided, a key is generated and stored in the working directory.
The Coordinator runs as a StatefulSet with a dynamically provisioned persistent volume.
-This volume stores the manifest history and the associated runtime policies.
-The manifest isn't considered sensitive information, because it needs to be passed to the untrusted infrastructure in order to start workloads.
-However, the Coordinator must ensure its integrity and that the persisted data corresponds to the manifests set by authorized users.
-Thus, the manifest is stored in plain text, but is signed with a private key derived from the Coordinator's secret seed.
When a Coordinator starts up, it doesn't have access to the signing secret and can thus not verify the integrity of the persisted manifests.
-It needs to be provided with the secret seed, from which it can derive the signing key that verifies the signatures.
-This procedure is called recovery and is initiated by the workload owner.
-The CLI decrypts the secret seed using the private seed share owner key, verifies the Coordinator and sends the seed through the Recover method.
-The Coordinator recovers its key material and verifies the manifest history signature.
The Coordinator provides each workload a secret seed during attestation. This secret can be used by the workload to derive additional secrets for example to
-encrypt persistent data. Like the workload certificates it's mounted in the shared Kubernetes volume contrast-secrets in the path <mountpoint>/secrets/workload-secret-seed.
-
warning
The workload owner can decrypt data encrypted with secrets derived from the workload secret.
-The workload owner can derive the workload secret themselves, since it's derived from the secret seed known to the workload owner.
-If the data owner and the workload owner is the same entity, then they can safely use the workload secrets.
