Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attestation driver and proxy (with KBS attestation) #528

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

tylerfanelli
Copy link

@tylerfanelli tylerfanelli commented Nov 21, 2024

This series introduces the attestation module in SVSM as well as the attestation proxy to facilitate communication between SVSM and a remote server for TEE evidence evaluation.

Abstract

To unlock early persistent state in SVSM (for example, persistent vTPM data), an attestation of the launch environment must take place to ensure that SVSM is running on trusted hardware and launched correctly. As SVSM does not have a network stack available, we have opted to introduce a proxy that would run on the host (for now, although this may change as other options are explored) and facilitate attestation between SVSM and a remote verification server such as KBS.

Much of what is implemented here is described in the Early Attestation and Measurement Architecture in COCONUT SVSM document.

Changes

There are three main modules introduced into SVSM with this series:

  • kernel/attest.rs: Attestation driver that communicates with the proxy through the COM3 serial port. This will eventually communicate over another channel (virtio-vsock, etc..) when available.
  • aproxy: Attestation proxy that runs on a host and facilitates communication with a remote attestation server.
  • libaproxy: Shared types between kernel/attest.rs and aproxy for (de)serialization of data.

Attestation

There exists two different phases of attestation:

  • Negotiation: Attestation servers usually require that a client embed metadata into its attestation evidence. The negotiation phase (initiated by SVSM) defines what exactly should be included in TEE attestation evidence. For instance, a server may require that a client embed a nonce hash and public components of the TEE public key (used to encrypt secrets) into the attestation evidence to ensure it is fresh and legitimate. The negotiation phase returns the parameters that SVSM should include in its attestation evidence based on the underlying attestation server and protocol.
  • Attestation: With all relevant data embedded in TEE evidence, SVSM sends its evidence to the remote server for evaluation. Upon successful attestation, the proxy will obtain an encrypted secret (only decryptable by SVSM's TEE key) for SVSM to use. For example, SVSM could use this secret to unlock encrypted persistent storage.

As there exists multiple protocols for TEE attestation, the proxy was built to be configurable to different protocols. As such, SVSM can be completely agnostic of the attestation protocol used. This is done by splitting the proxy up into two main components:

  • Front-end: To keep SVSM agnostic to the underlying attestation protocol, we define a standard front-end interface between SVSM and the proxy, the types of this interface for both negotiation and attestation are defined in the libaproxy crate. In the negotiation phase, SVSM will write a NegotiationRequest and receive a NegotiationResponse from the proxy. In the attestation phase, SVSM will write a AttestationRequest and receive a AttestationResponse from the proxy. This allows SVSM to not be tied to a specific attestation protocol, leaving the possibility for new protocols to be enabled in the future (for example, aTLS).

  • Back-end: This implements the specific attestation protocol that the communicating server implements. It is configurable with the --backend argument when launching the proxy. The initial backend option allowed is KBS, but the proxy can be configured to implement something like aTLS support if desired.

                                          ┌───────────┐
┌────┐               ┌─────┐              │Attestation│
│SVSM│◄─────────────►│Proxy│◄────────────►│Server     │
└────┘               └─────┘              └───────────┘
      │             │       │            │             
      └─────────────┘       └────────────┘             
         FRONT-END             BACK-END                
       (independent          (dependent                
       of attestation        on attestation            
       server)               server and                
                             protocol)        

Try for yourself

To try for yourself, I've set up a quick demo with a "dummy" KBS server that requires no configuration and simply returns a secret: hello, SVSM! that SVSM can then print. This requires a SEV-SNP machine with a SVSM-enabled kernel.

  1. In one window, clone and run the dummy KBS server used for testing:
$ git clone https://github.com/tylerfanelli/kbs-test.git
$ cd kbs-test
$ cargo run

This will run a "KBS server" (at least suitable for testing) at http://0.0.0.0:8080.

  1. Clone and build this branch of SVSM
$ git clone https://github.com/tylerfanelli/svsm.git

... build OVMF, qemu, SVSM IGVM, etc...

$ FW_FILE=... make FEATURES=attest
  1. Run the proxy on the host
$ cd svsm
$ cargo build --target=x86_64-unknown-linux-gnu -p aproxy
$ target/x86_64-unknown-linux-gnu/debug/aproxy --protocol kbs --url http://0.0.0.0:8080 --unix /tmp/svsm-proxy.sock --force

This runs the proxy with the following specified in the arguments:

  • --url http://0.0.0.0:8080: The attestation server is running at http://0.0.0.0:8080.
  • --protocol kbs: The attestation server communicates via the KBS protocol, configure the backend to use the KBS protocol.
  • --unix /tmp/svsm-proxy.sock: Listen for messages from SVSM on a socket created in file /tmp/svsm-proxy-sock.
  • --force: Remove the /tmp/svsm-proxy.sock file (if it already exists) before creating the socket.
  1. Run a guest with SVSM

Initially, SVSM communicates over the COM3 serial port. The attestation proxy socket will need to be available in the correct -serial port argument position to ensure it communicates with the right socket.

$ sudo ./qemu/build/qemu-system-x86_64 \
  -enable-kvm \
  -cpu EPYC-v4 \
  -machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
  -object memory-backend-memfd,id=ram1,size=8G,share=true,prealloc=false,reserve=false \
  -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,igvm-file=$IGVM \
  -smp 8 \
  -no-reboot \
  -netdev user,id=vmnic -device e1000,netdev=vmnic,romfile= \
  -drive file=$QCOW2,if=none,id=disk0,format=qcow2,snapshot=off \
  -device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=on \
  -device scsi-hd,drive=disk0,bootindex=0 \
  -vga std \
  -serial stdio \
  -serial pty	\
  -serial unix:/tmp/svsm-proxy.sock \

NOTE

Upon running, you may think that there is a hang in the proxy, as SVSM seems to freeze:

[SVSM] Validating 0x0000000000800000-0x0000000000809000
[SVSM] Validating 0x000000000080a000-0x0000000000820000
[SVSM] Flash region 0 at 0x00000000ffc00000 size 000000000000400000

This is not a hang, but rather, the creation of a 3072-bit RSA (as well as decryption with this key) is very slow at present (sometimes taking around ~1 minute for creation+decryption in my tests). Eventually, SVSM should begin running again and show the following message:

[SVSM] Decrypted secret from attestation server: hello, SVSM!

And thus we have decrypted secrets from a KBS attestation server (via the proxy).

TODO

The only purpose of b192792 is for others to see a secret decrypted from the server available and printed by SVSM. This must be removed once others have tried for themselves.

Other TODOs before this can be considered for merging:

  • Hide the kernel/attest module behind a feature flag in the kernel module. Perhaps a feature called attest?
  • Make relevant changes in SVSM Makefile to build the kernel module with the attest feature.

Future work:

  • Right now, SVSM communicates with the proxy over a serial port. Work on virtio-vsock support to use that instead.

Co-developed-by: Stefano Garzarella [email protected]
Signed-off-by: Tyler Fanelli [email protected]

cc/ @stefano-garzarella @joergroedel @deeglaze @berrange @fitzthum @stringlytyped @IT302 @Isaac-Matthews

kernel/src/serial.rs Outdated Show resolved Hide resolved
kernel/src/svsm.rs Outdated Show resolved Hide resolved
libaproxy/src/attestation.rs Outdated Show resolved Hide resolved
kernel/src/attest.rs Outdated Show resolved Hide resolved
@berrange
Copy link

This is not a hang, but rather, the creation of a 3072-bit RSA (as well as decryption with this key) is very slow at present (sometimes taking around ~1 minute for creation+decryption in my tests).

Do you have any insight on why this is so slow, or is it remaining for a future debugging effort ?

@tlendacky
Copy link
Contributor

This is not a hang, but rather, the creation of a 3072-bit RSA (as well as decryption with this key) is very slow at present (sometimes taking around ~1 minute for creation+decryption in my tests).

Do you have any insight on why this is so slow, or is it remaining for a future debugging effort ?

Have you tried using RdRand instead of RdSeed?

@berrange
Copy link

There are three main modules introduced into SVSM with this series:

  • kernel/attest.rs: Attestation driver that communicates with the proxy through the COM3 serial port. This will eventually communicate over another channel (virtio-vsock, etc..) when available.
  • aproxy: Attestation proxy that runs on a host and facilitates communication with a remote attestation server.
  • libaproxy: Shared types between kernel/attest.rs and aproxy for (de)serialization of data.

As a general point, I wonder whether these should really all live in the 'svsm' git repo.

Consider that the execution context of the proxy is the host OS, and the context of SVSM's driver is in guest VM. With this we're defining an ABI for a protocol between the host OS and guest VM, that potentially needs to remain stable & backwards compatible more or less indefinitely (forever). ie it is unlikely to be viable to mandate that the deployed host side proxy is upgraded in lock-step with SVSM releases, and thus we need to expect mis-matched releases of the two and ensure things continue to work. Splitting the proxy off as a separate repository, and defining a formal document specifying the interaction between SVSM & the proxy, will re-inforce the message that these two sides are independent and need to retain backwards compatibility as their respective impls evolve over time.

libaproxy/src/attestation.rs Show resolved Hide resolved
libaproxy/src/negotiation.rs Outdated Show resolved Hide resolved
libaproxy/src/negotiation.rs Outdated Show resolved Hide resolved
libaproxy/src/negotiation.rs Outdated Show resolved Hide resolved
kernel/src/attest.rs Outdated Show resolved Hide resolved
aproxy/src/backend/kbs.rs Outdated Show resolved Hide resolved
aproxy/src/backend/kbs.rs Outdated Show resolved Hide resolved
libaproxy/src/negotiation.rs Outdated Show resolved Hide resolved
kernel/src/attest.rs Outdated Show resolved Hide resolved
aproxy/src/backend/mod.rs Outdated Show resolved Hide resolved
@tylerfanelli
Copy link
Author

tylerfanelli commented Nov 26, 2024

This is not a hang, but rather, the creation of a 3072-bit RSA (as well as decryption with this key) is very slow at present (sometimes taking around ~1 minute for creation+decryption in my tests).

Do you have any insight on why this is so slow, or is it remaining for a future debugging effort ?

@berrange I'm not sure yet, I haven't looked in much detail besides seeing some other issues in the rust RSA crate discussing it.

Have you tried using RdRand instead of RdSeed?

@tlendacky I have, and my tests suggest RdRand is about 100% slower than RdSeed (~90 seconds for creation+decryption on a 3072-bit RSA key).

@tylerfanelli tylerfanelli force-pushed the attest branch 2 times, most recently from faf75aa to 39727d1 Compare November 26, 2024 06:25
@berrange
Copy link

This is not a hang, but rather, the creation of a 3072-bit RSA (as well as decryption with this key) is very slow at present (sometimes taking around ~1 minute for creation+decryption in my tests).

Do you have any insight on why this is so slow, or is it remaining for a future debugging effort ?

@berrange I'm not sure yet, I haven't looked in much detail besides seeing some other issues in the rust RSA crate discussing it.

They're saying RustCrypto is x3 slower than native, but that still doesn't feel like it is a root cause here. That bug reports says generating 100 keys takes 18 seconds. If we need 1 key, that's on the order of 0.2 secs to generate. The actual encryption/decryption operations shouldn't be even measurable given the small size of the data we're processing.

So I'm surprised anything crypto related can add up to as much as a 1 minute - that's orders of magnitude slower than it ought to be.

Have you tried using RdRand instead of RdSeed?

@tlendacky I have, and my tests suggest RdRand is about 100% slower than RdSeed (~90 seconds for creation+decryption on a 3072-bit RSA key).

That's surprising, as I wouldn't have thought either of those two should be "hot path" calls.

There's something altogether strange going on here to make this so slow.

@joergroedel joergroedel added the in-review PR is under active review and not yet approved label Dec 5, 2024
@tylerfanelli
Copy link
Author

tylerfanelli commented Dec 11, 2024

From advice during the weekly call, I've dropped support for RSA keys (at least for now) and replaced with EC keys. See 5dc6dac for more details and review. The initial implementation uses ECDH384+SHA256+AES128 (no salt) encryption. The previous performance issues seen from RSA keys have been resolved by this.

To support this, changes were made to the testing attestation server (https://github.com/tylerfanelli/kbs-test.git), so you would need to fetch those changes from the upstream branch to test.

Furthermore, I listed one of the goals before this can be supported upstream as: Ensure that aproxy's KBS backend works with production KBS servers such as Trustee. Although in future plans, none of the servers support EC keys yet. Therefore, this goal will be put on TODO and the proxy can be marked experimental until this support is available.

I've also added an attest feature flag (75245c3) to compile the attestation driver. I've edited the demo instructions above to compile with this flag for those looking to try for themselves.

There still exists 56ccb1b which simply prints a secret for the demo. Once this is approved, I can remove that commit.

cc/ @berrange @stefano-garzarella @joergroedel

@tylerfanelli tylerfanelli marked this pull request as ready for review December 11, 2024 04:36
@stefano-garzarella
Copy link
Member

@joergroedel can you approve the workflow on this PR? so we can eventually fix simple issues highlighted by our CI.

Copy link
Member

@stefano-garzarella stefano-garzarella left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick review and left some comments, thanks for working on this!

kernel/src/svsm.rs Outdated Show resolved Hide resolved
libaproxy/src/negotiation.rs Outdated Show resolved Hide resolved
kernel/src/attest.rs Show resolved Hide resolved
kernel/src/mm/address_space.rs Show resolved Hide resolved
kernel/Cargo.toml Outdated Show resolved Hide resolved
kernel/src/attest.rs Show resolved Hide resolved
@tylerfanelli tylerfanelli force-pushed the attest branch 4 times, most recently from 39e6fef to ce0301b Compare December 17, 2024 06:08
@tylerfanelli
Copy link
Author

@stefano-garzarella Can you re-review? I've resolved all comments.

@stefano-garzarella
Copy link
Member

@stefano-garzarella Can you re-review? I've resolved all comments.

Thanks, I'll do ASAP!

In the meantime, the following patch should fix CI and Makefile (I tested only with make clippy):

diff --git a/Makefile b/Makefile
index 512e7161..0ee07d54 100644
--- a/Makefile
+++ b/Makefile
@@ -173,7 +173,7 @@ bin/svsm-test.bin: bin/svsm-test
        objcopy -O binary $< $@
 
 clippy:
-       cargo clippy --workspace --all-features --exclude packit --exclude svsm-fuzz --exclude igvmbuilder --exclude igvmmeasure --exclude stage1 -- -D warnings
+       cargo clippy --workspace --all-features --exclude packit --exclude svsm-fuzz --exclude igvmbuilder --exclude igvmmeasure --exclude stage1 --exclude aproxy -- -D warnings
        cargo clippy --workspace --all-features --exclude packit --exclude svsm-fuzz --exclude svsm --exclude 'user*' --exclude stage1 --target=x86_64-unknown-linux-gnu -- -D warnings
        cargo clippy -p stage1 --all-features --target=x86_64-unknown-linux-gnu -- -D warnings ${STAGE1_RUSTC_ARGS}
        RUSTFLAGS="--cfg fuzzing" cargo clippy --package svsm-fuzz --all-features --target=x86_64-unknown-linux-gnu -- -D warnings
diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml
index b7c26f69..a42be59b 100644
--- a/.github/workflows/rust.yml
+++ b/.github/workflows/rust.yml
@@ -56,7 +56,7 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: clippy
-          args: --workspace --exclude igvmbuilder --exclude igvmmeasure --exclude svsm-fuzz --exclude packit --exclude stage1 --all-features -- -D warnings
+          args: --workspace --exclude igvmbuilder --exclude igvmmeasure --exclude svsm-fuzz --exclude packit --exclude stage1 --exclude aproxy --all-features -- -D warnings
 
       - name: Clippy on std x86_64-unknown-linux-gnu
         uses: actions-rs/cargo@v1
@@ -151,7 +151,7 @@ jobs:
 
       - name: Clippy with undocumented_unsafe_blocks for PR branch
         run: |
-          cargo clippy --workspace --all-features --exclude packit --exclude stage1 --exclude svsm-fuzz --exclude igvmbuilder --exclude igvmmeasure --quiet -- -W clippy::undocumented_unsafe_blocks 2> clippy_warnings_pr.txt
+          cargo clippy --workspace --all-features --exclude packit --exclude stage1 --exclude svsm-fuzz --exclude igvmbuilder --exclude igvmmeasure --exclude aproxy --quiet -- -W clippy::undocumented_unsafe_blocks 2> clippy_warnings_pr.txt
 
       # Required because after the next checkout everything is removed.
       - name: Upload PR warnings artifact
@@ -171,7 +171,7 @@ jobs:
 
       - name: Clippy with undocumented_unsafe_blocks for base branch
         run: |
-          cargo clippy --workspace --all-features --exclude packit --exclude stage1 --exclude svsm-fuzz --exclude igvmbuilder --exclude igvmmeasure --quiet -- -W clippy::undocumented_unsafe_blocks 2> clippy_warnings_base.txt
+          cargo clippy --workspace --all-features --exclude packit --exclude stage1 --exclude svsm-fuzz --exclude igvmbuilder --exclude igvmmeasure --exclude aproxy --quiet -- -W clippy::undocumented_unsafe_blocks 2> clippy_warnings_base.txt
 
       - name: Download PR warnings artifact
         uses: actions/download-artifact@v3
@@ -189,7 +189,7 @@ jobs:
             echo "ERROR: $(($PR_WARNINGS - $BASE_WARNINGS)) new undocumented unsafe code blocks detected in this PR"
             echo "enabling the clippy::undocumented_unsafe_blocks lint in this way:"
             echo "$ cargo clippy --workspace --all-features --exclude packit --exclude stage1 --exclude svsm-fuzz \\"
-            echo "  --exclude igvmbuilder --exclude igvmmeasure -- -W clippy::undocumented_unsafe_blocks"
+            echo "  --exclude igvmbuilder --exclude igvmmeasure --exclude aproxy -- -W clippy::undocumented_unsafe_blocks"
             echo ""
             diff --color -u clippy_warnings_base.txt clippy_warnings_pr.txt
             exit 1

@tylerfanelli
Copy link
Author

I believe I've fixed the final CI issue. @joergroedel would you mind re-approving the workflow?

@stefano-garzarella
Copy link
Member

I believe I've fixed the final CI issue.

You did, but the CI decided to break xD (see #583 now merged in main) so, I guess another rebase is needed.

kernel/src/attest.rs Show resolved Hide resolved
Makefile Show resolved Hide resolved
aproxy/src/main.rs Outdated Show resolved Hide resolved
aproxy/src/attest.rs Outdated Show resolved Hide resolved
aproxy/src/main.rs Outdated Show resolved Hide resolved
@tylerfanelli
Copy link
Author

@stefano-garzarella I've addressed your latest review. Would you mind re-reviewing?

Cargo.toml Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
libaproxy/src/attestation.rs Outdated Show resolved Hide resolved
aproxy/src/backend/mod.rs Outdated Show resolved Hide resolved
aproxy/src/main.rs Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
Cargo.toml Outdated Show resolved Hide resolved
aproxy/src/backend/mod.rs Show resolved Hide resolved
igvmmeasure/Cargo.toml Outdated Show resolved Hide resolved
For a universal interface when reading or writing multiple bytes over a
transport channel, introduce generic Read/Write traits.

Implement these traits over the serial::SerialPort for use in the
attestation driver.

This is an attempt to mimic the std::io::{Read, Write} traits.

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
The attestation driver requires a method to fetch an attestation report
from the SNP report response message.

Signed-off-by: Tyler Fanelli <[email protected]>
Creating a new EC private/public key pair results in a stack overflow
with a stack size of less than 12 pages.

Signed-off-by: Tyler Fanelli <[email protected]>
The attestation module provides services early attestation and
measurement of the SVSM environment.

This commits initializes the attestation driver and sends an initial
negotiation request to the attestation proxy.

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
To validate the freshness of attestation evidence, KBS servers return a
challenge upon client authentication. This challenge contains a
ephemeral nonce in the form of base64-encoded bytes. This nonce *must*
be hashed into the attestation report along with the components of the
TEE public key.

This commit reads the negotiation request from SVSM, and depending on
the backend, fetches the negotiation parameters accordingly. As KBS is
the backend used in the initial implementation, this translates into a
call to the KBS /auth handler to fetch the nonce, and then a
specification of both the nonce and TEE public components to be included
in the attestation evidence (in the form of negotiation parameters).

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
Given the negotiation parameters from the proxy, SVSM can now hash the
parameters and embed them in the attestation evidence. It can also read
the negotiation key specified and create key accordingly.

As SEV-SNP is the only supported TEE architecture at the moment, there's
also a few changes required to serialize a SEV-SNP attestation report
into bytes.

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
With serialized TEE evidence and key, the proxy can now transfer these
to the attestation server for evaluation.

KBS will first transfer the evidence to the /attest endpoint for
evaluation. Upon successful attestation, the client can then retrieve
resources from the server.

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
Use the TEE key to decrypt the secret received from the proxy.

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
When other attestation protocols are implemented for the backend
attestation server communication, they can register themselves with the
BACKEND registrar as well.

Signed-off-by: Tyler Fanelli <[email protected]>
aproxy can be excluded from non-`target=x86_64-unknown-linux-gnu` clippy
checks.

Co-developed-by: Stefano Garzarella <[email protected]>
Signed-off-by: Tyler Fanelli <[email protected]>
Copy link
Member

@stefano-garzarella stefano-garzarella left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@tylerfanelli
Copy link
Author

@joergroedel Can you re-enable the CI workflows?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in-review PR is under active review and not yet approved
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants