diff --git a/docs/docs/architecture/encrypted-storage.md b/docs/docs/architecture/encrypted-storage.md index 4288ddac27..f047fa4a9d 100644 --- a/docs/docs/architecture/encrypted-storage.md +++ b/docs/docs/architecture/encrypted-storage.md @@ -55,3 +55,8 @@ For encryption Constellation uses AES in XTS-Plain64. The key size is 512 bit. To interact with the dm-integrity kernel module, Constellation uses [libcryptsetup](https://gitlab.com/cryptsetup/cryptsetup/). When enabled, the used data integrity algorithm is [HMAC](https://datatracker.ietf.org/doc/html/rfc2104) with SHA256 as the hash function. The tag size is 32 Bytes. + +## Encrypted S3 object storage + +Constellation comes with a service that you can use to transparently retrofit client-side encryption to existing applications that use S3 (AWS or compatible) for storage. +To learn more, check out the [s3proxy documentation](../workflows/s3proxy.md). diff --git a/docs/docs/getting-started/examples/filstash-s3proxy.md b/docs/docs/getting-started/examples/filstash-s3proxy.md new file mode 100644 index 0000000000..6dd67f28c0 --- /dev/null +++ b/docs/docs/getting-started/examples/filstash-s3proxy.md @@ -0,0 +1,71 @@ + +# Deploying Filestash + +Filestash is a web frontend for different storage backends, including S3. +It's a useful application to showcase s3proxy in action. + +1. Deploy s3proxy as described in [Deployment](../../workflows/s3proxy.md#deployment). +2. Create a deployment file for Filestash with one pod: + +```sh +cat << EOF > "deployment-filestash.yaml" +apiVersion: apps/v1 +kind: Deployment +metadata: + name: filestash +spec: + replicas: 1 + selector: + matchLabels: + app: filestash + template: + metadata: + labels: + app: filestash + spec: + imagePullSecrets: + - name: regcred + hostAliases: + - ip: $(kubectl get svc s3proxy-service -o=jsonpath='{.spec.clusterIP}') + hostnames: + - "s3.eu-west-1.amazonaws.com" + containers: + - name: filestash + image: machines/filestash:latest + ports: + - containerPort: 8334 + volumeMounts: + - name: ca-cert + mountPath: /etc/ssl/certs/kube-ca.crt + subPath: kube-ca.crt + volumes: + - name: ca-cert + secret: + secretName: s3proxy-tls + items: + - key: ca.crt + path: kube-ca.crt +EOF +``` + +The pod spec includes the `hostAliases` key, which adds an entry to the pod's `/etc/hosts`. +The entry forwards all requests for `s3.eu-west-1.amazonaws.com` to the Kubernetes service `s3proxy-service`. +If you followed the s3proxy [Deployment](../../workflows/s3proxy.md#deployment) guide, this service points to a s3proxy pod. + +To use other regions than `eu-west-1`, add more entries to `hostAliases` for all regions you require. +Use the same IP for those entries. For example to add `us-east-1` add: +```yaml +- ip: $(kubectl get svc s3proxy-service -o=jsonpath='{.spec.clusterIP}') + hostnames: + - "s3.us-east-1.amazonaws.com" +``` + +The spec also includes a volume mount for the TLS certificate and adds it to the pod's certificate trust store. +The volume is called `ca-cert`. +The key `ca.crt` of that volume is mounted to `/etc/ssl/certs/kube-ca.crt`, which is the default certificate trust store location for that container's OpenSSL library. +Not adding the CA certificate will result in TLS authentication errors. + +3. Apply the file: `kubectl apply -f deployment-filestash.yaml` + +Afterward, you can use a port forward to access the Filestash pod: +`kubectl port-forward pod/$(kubectl get pod --selector='app=filestash' -o=jsonpath='{.items[*].metadata.name}') 8334:8334` diff --git a/docs/docs/workflows/s3proxy.md b/docs/docs/workflows/s3proxy.md new file mode 100644 index 0000000000..9c228c98e1 --- /dev/null +++ b/docs/docs/workflows/s3proxy.md @@ -0,0 +1,57 @@ +# Install s3proxy + +Constellation includes a transparent client-side encryption proxy for [AWS S3](https://aws.amazon.com/de/s3/) and compatible stores. +s3proxy encrypts objects before sending them to S3 and automatically decrypts them on retrieval, without requiring changes to your application. +With s3proxy, you can use S3 for storage in a confidential way without having to trust the storage provider. + +## Limitations + +Currently, s3proxy has the following limitations: +- Only `PutObject` and `GetObject` requests are encrypted/decrypted by s3proxy. +By default, s3proxy will block requests that may expose unencrypted data to S3 (e.g. UploadPart). +The `allow-multipart` flag disables request blocking for evaluation purposes. +- Using the [Range](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestSyntax) header on `GetObject` is currently not supported and will result in an error. + +These limitations will be removed with future iterations of s3proxy. +If you want to use s3proxy but these limitations stop you from doing so, consider [opening an issue](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&projects=&template=feature_request.yml). + +## Deployment + +You can add the s3proxy to your Constellation cluster as follows: +1. Download the deployment manifest: + ```bash + wget https://raw.githubusercontent.com/edgelesssys/constellation/main/s3proxy/deploy/deployment-s3proxy.yaml + ``` +2. Replace the values named `replaceme` in `deployment-s3proxy.yaml` with valid AWS credentials. These credentials are used by s3proxy to access your S3 buckets. +3. Deploy s3proxy: + ```bash + kubectl apply -f deployment-s3proxy.yaml + ``` + +If you want to run a demo application, check out the [Filestash with s3proxy](../getting-started/examples/filstash-s3proxy.md) example. + + +## Technical details + +### Encryption + +s3proxy relies on Google's [Tink Cryptographic Library](https://developers.google.com/tink) to implement cryptographic operations securely. +The used cryptographic primitives are [NIST SP 800 38f](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-38F.pdf) for key wrapping and [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)-[GCM](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Galois/counter_(GCM)) with 256 bit keys for data encryption. + +s3proxy uses [envelope encryption](https://cloud.google.com/kms/docs/envelope-encryption) to encrypt objects. +This means s3proxy uses a key encryption key (KEK) issued by the [KeyService](../architecture/microservices.md#keyservice) to encrypt data encryption keys (DEKs). +Each S3 object is encrypted with its own DEK. +The encrypted DEK is then saved as metadata of the encrypted object. +This enables key rotation of the KEK without re-encrypting the data in S3. +The approach also allows access to objects from different locations, as long as each location has access to the KEK. + +### Traffic interception + +To use s3proxy, you have to redirect your outbound S3 traffic to s3proxy. +This can either be done by modifying your client application or by changing the deployment of your application. + +The necessary deployment modifications are to add DNS redirection and a trusted TLS certificate to the client's trust store. +DNS redirection can be defined for each pod, allowing you to use s3proxy for one application without changing other applications in the same cluster. +Adding a trusted TLS certificate is necessary as clients communicate with s3proxy via HTTPS. +To have your client application trust s3proxy's TLS certificate, the certificate has to be added to the client's certificate trust store. +The [Filestash with s3proxy](../getting-started/examples/filstash-s3proxy.md) example shows how to do this. diff --git a/docs/sidebars.js b/docs/sidebars.js index 5c206a43d2..203e9007bd 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -119,6 +119,11 @@ const sidebars = { label: 'Horizontal Pod Autoscaling', id: 'getting-started/examples/horizontal-scaling' }, + { + type: 'doc', + label: 'Filestash with s3proxy', + id: 'getting-started/examples/filstash-s3proxy' + }, ] }, ], @@ -165,6 +170,11 @@ const sidebars = { label: 'Install cert-manager', id: 'workflows/cert-manager', }, + { + type: 'doc', + label: 'Install s3proxy', + id: 'workflows/s3proxy', + }, { type: 'doc', label: 'Terminate your cluster', diff --git a/docs/styles/Vocab/constellation/accept.txt b/docs/styles/Vocab/constellation/accept.txt index 1709702844..31b6e437c1 100644 --- a/docs/styles/Vocab/constellation/accept.txt +++ b/docs/styles/Vocab/constellation/accept.txt @@ -22,6 +22,7 @@ Dynatrace [Ee]mojivoto etcd Filebeat +Filestash Filestore Fluentd Fulcio @@ -54,6 +55,7 @@ sigstore [Ss]uperset Syft systemd +Tink [Uu]nencrypted unspoofable updatable diff --git a/s3proxy/deploy/deployment-s3proxy.yaml b/s3proxy/deploy/deployment-s3proxy.yaml index c3a5f38d96..49ca522ac4 100644 --- a/s3proxy/deploy/deployment-s3proxy.yaml +++ b/s3proxy/deploy/deployment-s3proxy.yaml @@ -24,7 +24,7 @@ spec: - "s3.eu-west-1.amazonaws.com" issuerRef: name: selfsigned-issuer - kind: ClusterIssuer + kind: Issuer group: cert-manager.io --- apiVersion: apps/v1 @@ -47,7 +47,7 @@ spec: - name: regcred containers: - name: s3proxy - image: ghcr.io/edgelesssys/constellation/s3proxy:v2.12.0-pre.0.20231009141917-226cb427d0b1 + image: ghcr.io/edgelesssys/constellation/s3proxy:v2.12.0 args: - "--level=-1" ports: