-
Notifications
You must be signed in to change notification settings - Fork 54
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
s3proxy: add new page to documentation (v2.12)
- Loading branch information
Showing
7 changed files
with
146 additions
and
3 deletions.
There are no files selected for viewing
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
71 changes: 71 additions & 0 deletions
71
docs/versioned_docs/version-2.12/getting-started/examples/filestash-s3proxy.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
|
||
# Deploying Filestash | ||
|
||
Filestash is a web frontend for different storage backends, including S3. | ||
It's a useful application to showcase s3proxy in action. | ||
|
||
1. Deploy s3proxy as described in [Deployment](../../workflows/s3proxy.md#deployment). | ||
2. Create a deployment file for Filestash with one pod: | ||
|
||
```sh | ||
cat << EOF > "deployment-filestash.yaml" | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: filestash | ||
spec: | ||
replicas: 1 | ||
selector: | ||
matchLabels: | ||
app: filestash | ||
template: | ||
metadata: | ||
labels: | ||
app: filestash | ||
spec: | ||
imagePullSecrets: | ||
- name: regcred | ||
hostAliases: | ||
- ip: $(kubectl get svc s3proxy-service -o=jsonpath='{.spec.clusterIP}') | ||
hostnames: | ||
- "s3.eu-west-1.amazonaws.com" | ||
containers: | ||
- name: filestash | ||
image: machines/filestash:latest | ||
ports: | ||
- containerPort: 8334 | ||
volumeMounts: | ||
- name: ca-cert | ||
mountPath: /etc/ssl/certs/kube-ca.crt | ||
subPath: kube-ca.crt | ||
volumes: | ||
- name: ca-cert | ||
secret: | ||
secretName: s3proxy-tls | ||
items: | ||
- key: ca.crt | ||
path: kube-ca.crt | ||
EOF | ||
``` | ||
|
||
The pod spec includes the `hostAliases` key, which adds an entry to the pod's `/etc/hosts`. | ||
The entry forwards all requests for `s3.eu-west-1.amazonaws.com` to the Kubernetes service `s3proxy-service`. | ||
If you followed the s3proxy [Deployment](../../workflows/s3proxy.md#deployment) guide, this service points to a s3proxy pod. | ||
|
||
To use other regions than `eu-west-1`, add more entries to `hostAliases` for all regions you require. | ||
Use the same IP for those entries. For example to add `us-east-1` add: | ||
```yaml | ||
- ip: $(kubectl get svc s3proxy-service -o=jsonpath='{.spec.clusterIP}') | ||
hostnames: | ||
- "s3.us-east-1.amazonaws.com" | ||
``` | ||
The spec also includes a volume mount for the TLS certificate and adds it to the pod's certificate trust store. | ||
The volume is called `ca-cert`. | ||
The key `ca.crt` of that volume is mounted to `/etc/ssl/certs/kube-ca.crt`, which is the default certificate trust store location for that container's OpenSSL library. | ||
Not adding the CA certificate will result in TLS authentication errors. | ||
|
||
3. Apply the file: `kubectl apply -f deployment-filestash.yaml` | ||
|
||
Afterward, you can use a port forward to access the Filestash pod: | ||
`kubectl port-forward pod/$(kubectl get pod --selector='app=filestash' -o=jsonpath='{.items[*].metadata.name}') 8334:8334` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
# Install s3proxy | ||
|
||
Constellation includes a transparent client-side encryption proxy for [AWS S3](https://aws.amazon.com/de/s3/) and compatible stores. | ||
s3proxy encrypts objects before sending them to S3 and automatically decrypts them on retrieval, without requiring changes to your application. | ||
With s3proxy, you can use S3 for storage in a confidential way without having to trust the storage provider. | ||
|
||
## Limitations | ||
|
||
Currently, s3proxy has the following limitations: | ||
- Only `PutObject` and `GetObject` requests are encrypted/decrypted by s3proxy. | ||
By default, s3proxy will block requests that may expose unencrypted data to S3 (e.g. UploadPart). | ||
The `allow-multipart` flag disables request blocking for evaluation purposes. | ||
- Using the [Range](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestSyntax) header on `GetObject` is currently not supported and will result in an error. | ||
|
||
These limitations will be removed with future iterations of s3proxy. | ||
If you want to use s3proxy but these limitations stop you from doing so, consider [opening an issue](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&projects=&template=feature_request.yml). | ||
|
||
## Deployment | ||
|
||
You can add the s3proxy to your Constellation cluster as follows: | ||
1. Download the deployment manifest: | ||
```bash | ||
wget https://raw.githubusercontent.com/edgelesssys/constellation/main/s3proxy/deploy/deployment-s3proxy.yaml | ||
``` | ||
2. Replace the values named `replaceme` in `deployment-s3proxy.yaml` with valid AWS credentials. These credentials are used by s3proxy to access your S3 buckets. | ||
3. Deploy s3proxy: | ||
```bash | ||
kubectl apply -f deployment-s3proxy.yaml | ||
``` | ||
|
||
If you want to run a demo application, check out the [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example. | ||
|
||
|
||
## Technical details | ||
|
||
### Encryption | ||
|
||
s3proxy relies on Google's [Tink Cryptographic Library](https://developers.google.com/tink) to implement cryptographic operations securely. | ||
The used cryptographic primitives are [NIST SP 800 38f](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-38F.pdf) for key wrapping and [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)-[GCM](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Galois/counter_(GCM)) with 256 bit keys for data encryption. | ||
|
||
s3proxy uses [envelope encryption](https://cloud.google.com/kms/docs/envelope-encryption) to encrypt objects. | ||
This means s3proxy uses a key encryption key (KEK) issued by the [KeyService](../architecture/microservices.md#keyservice) to encrypt data encryption keys (DEKs). | ||
Each S3 object is encrypted with its own DEK. | ||
The encrypted DEK is then saved as metadata of the encrypted object. | ||
This enables key rotation of the KEK without re-encrypting the data in S3. | ||
The approach also allows access to objects from different locations, as long as each location has access to the KEK. | ||
|
||
### Traffic interception | ||
|
||
To use s3proxy, you have to redirect your outbound S3 traffic to s3proxy. | ||
This can either be done by modifying your client application or by changing the deployment of your application. | ||
|
||
The necessary deployment modifications are to add DNS redirection and a trusted TLS certificate to the client's trust store. | ||
DNS redirection can be defined for each pod, allowing you to use s3proxy for one application without changing other applications in the same cluster. | ||
Adding a trusted TLS certificate is necessary as clients communicate with s3proxy via HTTPS. | ||
To have your client application trust s3proxy's TLS certificate, the certificate has to be added to the client's certificate trust store. | ||
The [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example shows how to do this. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters