Skip to content

Latest commit

 

History

History
177 lines (132 loc) · 8.77 KB

README.md

File metadata and controls

177 lines (132 loc) · 8.77 KB

CCNP Deployment Guide

CCNP is designed for collecting confidential computing primitives in cloud native environments. It's designed to run as DaemonSet (for Kubernetes cluster) or container (for docker container) on confidential virtual machines nodes, such as Intel Trust Domain (TD). Below diagram illustrates CCNP deployment process. In this document, it will use Intel TD as an example of CVM and deploy CCNP on Intel TD nodes.

Deployment diagram

Create TD

You can use cvm image rewriter to prepare a TD enlightened guest image.

NOTE:

  • If your initial guest image already has TDX kernel, it's required to run plugin 08 and 09 to set device access permission.
  • If your initial guest image is a normal Ubuntu guest image, it's required to run plugin 05 to install TDX kernel and then 08, 09 to set device access permission.

Start a TD using qemu-test.sh or start-virt.sh.

  • Use qemu-test.sh, please use -q <vsock/tdvmcall> to make sure get quote works for the TD.

    $ sudo ./qemu-test.sh -i output.qcow2 -t td -p <qemu monitor port> -f <ssh_forward port> -q tdvmcall

  • Use start-virt.sh. The Libvirt XML template is tdx-libvirt-ubuntu-host.xml.template. It uses vsock for getting quote.

    $ sudo ./start-virt.sh -i <guest image>

Prepare a K8S cluster with TD as worker nodes

You can either create a K8S cluster in the TD or let the TD join an existing K8S cluster. Please choose one of the following step to make sure the K8S cluster is prepared with the TD running in it. CCNP will be deployed on the TD later.

Option 1: Create a K8S cluster on the TD

After TDs are started, users need to setup a K8S cluster in the TDs. Please refer to the k8s official documentation.

NOTE: If the cluster has only one node (master node), the taint on the node needs to be removed.

You can also refer to K3S to start a lightweight Kubernetes cluster for experimental purpose.

Option 2: Add the TD to an existing K8S cluster

After TDs are started, users can let the TDs join an existing K8S cluster. Please refer to the k8s official documentation for detailed steps.

Deploy CCNP

The following scripts can help to generate CCNP images and deploy them in the TD nodes.

  • build.sh: The tool will build docker images and push them to remote registry if required.
  • deploy-ccnp.sh: The tool will deploy CCNP services as DaemonSet on TDs in the K8S cluster.
  • deploy-and-exec-ccnp-example.sh: The tool will deploy an example pod and show getting event logs, measurement and perform verification using CCNP in the pod.
  • prerequisite.sh: This tool will complete the prerequisites for deploying CCNP on Ubuntu. For other platforms, you can follow the section below.

Prerequisite

The prerequisite steps are required for CCNP deployment. You can run prerequisite.sh for Ubuntu host.

$ cd script
$ sudo ./prerequisite.sh

You can also complete the steps following below guide manually.

  • Install Helm on the TD nodes. Please refer to the HELM quick start.
  • Install docker on the TD nodes. Please refer to Get Docker.
  • Install python3-pip on the TD nodes. Please refer to pip document.
  • Set access permission to TD device node and ccnp working directory on the TD nodes.
$ sudo mkdir -p /etc/udev/rules.d
$ sudo touch /etc/udev/rules.d/90-tdx.rules
# Check TD device node on TD
$ ls /dev/tdx*

# If above output is "/dev/tdx-guest"
$ sudo bash -c 'echo "SUBSYSTEM==\"misc\",KERNEL==\"tdx-guest\",MODE=\"0666\"">/etc/udev/rules.d/90-tdx.rules'
# If above output is "/dev/tdx_guest"
$ sudo bash -c 'echo "SUBSYSTEM==\"misc\",KERNEL==\"tdx_guest\",MODE=\"0666\"">/etc/udev/rules.d/90-tdx.rules'
# make the udev setup effective
$ sudo udevadm trigger

$ sudo touch /usr/lib/tmpfiles.d/ccnp.conf
$ sudo bash -c 'echo "D /run/ccnp/uds 0757 - - -">/usr/lib/tmpfiles.d/ccnp.conf'
# make the directory setup effective by running below command or restarting the node
$ sudo systemd-tmpfiles --create

Build CCNP images

Run below scripts to generate CCNP images. It will generate 5 images and push them to user specific registry.

NOTE: The scripts need to run on a server with docker installed.

$ cd container
$ sudo ./build.sh -r <remote registry> -g <docker image tag>

e.g.

# Build images with tag 0.3 and push them to remote registry test-registry.intel.com
$ sudo ./build.sh -r test-registry.intel.com/test -g 0.3

# Build images only with tag 0.3
$ sudo ./build.sh -a build -g 0.3

NOTE: please set HTTP_PROXY, HTTPS_PROXY, NO_PROXY in your terminal if they are needed in your environments.

After the script is successful, it's supposed to see below docker images for CCNP.

$ sudo docker images
ccnp-node-measurement-example   <your image tag>
ccnp-eventlog-server            <your image tag>
ccnp-measurement-server         <your image tag>
ccnp-quote-server               <your image tag>
ccnp-device-plugin              <your image tag>

Deploy CCNP services

CCNP deployment tool will deploy TDX device plugin and DaemonSets for CCNP event log, measurement and quote. Run below scripts on each TD node.

# Deploy CCNP with user specified remote registry and image tag
$ sudo ./deploy-ccnp.sh -r <remote registry> -g <tag>
e.g.
$ sudo ./deploy-ccnp.sh -r test-registry.intel.com/test -g 0.3

# Delete existing CCNP and Deploy CCNP with user specified remote registry and image tag
$ sudo ./deploy-ccnp.sh -r <remote registry> -g <tag> -d

After it's successful, you should see helm release ccnp-device-plugin and 3 DaemonSets in namespace ccnp.

$ sudo helm list
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
ccnp-device-plugin      default         1               2023-12-27 08:12:05.814766198 +0000 UTC deployed        ccnp-device-plugin-0.1.0        latest
$ sudo kubectl get ds -n ccnp
NAME                 DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                                        AGE
eventlog-server      1         1         1       1            1           intel.feature.node.kubernetes.io/tdx-guest=enabled   24h
quote-server         1         1         1       1            1           intel.feature.node.kubernetes.io/tdx-guest=enabled   24h
measurement-server   1         1         1       1            1           intel.feature.node.kubernetes.io/tdx-guest=enabled   24h
$ sudo kubectl get pods -n ccnp
NAME                       READY   STATUS    RESTARTS      AGE
eventlog-server-bk2g5      1/1     Running   2 (39s ago)   24h
quote-server-6lhf6         1/1     Running   0             26s
measurement-server-4q9v7   1/1     Running   0             26s

Install CCNP SDK

There are two options to install CCNP SDK.
Option 1: Users can run the following command on each work node to install CCNP client library for Python with PyPI.

pip install ccnp

Option 2: Users can also run the following commands to install CCNP SDK from source code.

cd confidential-cloud-native-primitives/sdk/python3/
pip install -r requirements.txt
pip install -e .

At this step, CCNP has been installed successfully. For more detailed information, including SDK usage, please refer to here.

CCNP example

The script deploy-and-exec-ccnp-example.sh is an example of using CCNP to collect event log, measurement and perform verification in a pod.

$ cd script
# Specify the registry name and tag used in image building
$ sudo ./deploy-and-exec-ccnp-example.sh -r <remote-registry> -g <tag>

# You can also specify which integrity measurement register (RTMR in the case of Intel TD) to verify
# e.g. Show RTMR[1] and RTMR[2] using below command
$ sudo ./deploy-and-exec-ccnp-example.sh -r <remote-registry> -g <tag> -i '1 2'

The example output of verification can be found at sample-output-for-node-measurement-tool-full.txt and sample-output-for-node-measurement-tool-selected.txt