Skip to content

Commit

Permalink
docs: explain service mesh injection
Browse files Browse the repository at this point in the history
  • Loading branch information
davidweisse committed Jun 10, 2024
1 parent 6b2d19f commit 6a5574f
Show file tree
Hide file tree
Showing 3 changed files with 68 additions and 76 deletions.
12 changes: 6 additions & 6 deletions docs/docs/architecture/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,13 @@ metrics via its [Envoy admin
interface](https://www.envoyproxy.io/docs/envoy/latest/operations/admin). Be
aware that the admin interface can expose private information and allows
destructive operations to be performed. To enable the admin interface for the
Service Mesh, set the environment variable `EDG_ADMIN_PORT` in the configuration
of the Service Mesh sidecar. If this variable is set, the admin interface will
be started on this port.
Service Mesh, set the annotation
`contrast.edgeless.systems/servicemesh-admin-interface-port` in the configuration
of your workload. If this annotation is set, the admin interface will be started
on this port.

To access the admin interface, the Service Mesh sidecar container needs to have
a corresponding container port and the ingress settings of the Proxy have to be
configured to allow access to the specified port (see [Configuring the
To access the admin interface, the ingress settings of the Service Mesh have to
be configured to allow access to the specified port (see [Configuring the
Proxy](../components/service-mesh#configuring-the-proxy)). All metrics will be
exposed under the `/stats` endpoint. Metrics in Prometheus format can be scraped
from the `/stats/prometheus` endpoint.
97 changes: 48 additions & 49 deletions docs/docs/components/service-mesh.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,25 @@ verification of the endpoints in the connection establishment is based on
certificates that are part of the
[PKI of the Coordinator](../architecture/certificates.md).

The service mesh can be enabled on a per-pod basis by adding the `service-mesh`
container as a [sidecar container](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/).
The service mesh container first sets up `iptables`
rules based on its configuration and then starts [Envoy](https://www.envoyproxy.io/)
for TLS origination and termination.
The service mesh can be enabled on a per-workload basis by adding a service mesh
configuration to the workload's object annotations. During the `contrast generate`
step, the service mesh is added as a [sidecar
container](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/) to
all workloads which have a specified configuration. The service mesh container first
sets up `iptables` rules based on its configuration and then starts
[Envoy](https://www.envoyproxy.io/) for TLS origination and termination.

## Configuring the Proxy

The service mesh container can be configured using the `EDG_INGRESS_PROXY_CONFIG`
and `EDG_EGRESS_PROXY_CONFIG` environment variables.
The service mesh container can be configured using the following object annotations:
- `contrast.edgeless.systems/servicemesh-ingress` to configure ingress.
- `contrast.edgeless.systems/servicemesh-egress` to configure egress.
- `contrast.edgeless.systems/servicemesh-admin-interface-port` to configure the Envoy
admin interface. If not specified, no admin interface will be started.

If you aren't using the automatic service mesh injection and want to configure the
service mesh manually, set the environment variables `EDG_INGRESS_PROXY_CONFIG`,
`EDG_EGRESS_PROXY_CONFIG` and `EDG_ADMIN_PORT` in the service mesh sidecar directly.

### Ingress

Expand All @@ -30,7 +39,7 @@ certificate of the workload and the intermediate CA certificate as the server ce

If the deployment contains workloads which should be reachable from outside the
Service Mesh, while still handing out the certificate chain, disable client
authentication by setting the environment variable `EDG_INGRESS_PROXY_CONFIG` as
authentication by setting the annotation `contrast.edgeless.systems/servicemesh-ingress` as
`<name>#<port>#false`. Separate multiple entries with `##`. You can choose any
descriptive string identifying the service on the given port for the
informational-only field `<name>`.
Expand All @@ -50,26 +59,13 @@ apiVersion: apps/v1
kind: Deployment
metadata:
name: web
annotations:
contrast.edgeless.systems/servicemesh-ingress: "web#8080#false##metrics#7890#true"
spec:
replicas: 1
template:
spec:
runtimeClassName: contrast-cc
initContainers:
- name: sidecar
image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy@sha256:..."
restartPolicy: Always
volumeMounts:
- name: contrast-tls-certs
mountPath: /tls-config
env:
- name: EDG_INGRESS_PROXY_CONFIG
value: "web#8080#false##metrics#7890#true"
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
containers:
- name: web-svc
image: ghcr.io/edgelesssys/frontend:v1.2.3@...
Expand All @@ -78,14 +74,35 @@ spec:
name: web
- containerPort: 7890
name: metrics
```
When invoking `contrast generate`, the resulting deployment will be injected with the
Contrast service mesh as an init container.

```yaml
# ...
initContainers:
- env:
- name: EDG_INGRESS_PROXY_CONFIG
value: "web#8080#false##metrics#7890#true"
image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy@sha256:..."
name: contrast-service-mesh
restartPolicy: Always
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: true
volumeMounts:
- name: contrast-tls-certs
mountPath: /tls-config
volumes:
- name: contrast-tls-certs
emptyDir: {}
```

Note, that changing the environment variables of the sidecar container directly will
only have an effect if the workload isn't configured to automatically generate a
service mesh component on `contrast generate`. Otherwise, the service mesh sidecar
container will be regenerated on every invocation of the command.

### Egress

To be able to route the egress traffic of the workload through Envoy, the remote
Expand All @@ -95,8 +112,9 @@ endpoints' IP address and port must be configurable.
by the pod.
* Configure the workload to connect to this IP address and port.
* Set `<name>#<chosen IP>:<chosen port>#<original-hostname-or-ip>:<original-port>`
as `EDG_EGRESS_PROXY_CONFIG`. Separate multiple entries with `##`. Choose any
string identifying the service on the given port as `<name>`.
as the `contrast.edgeless.systems/servicemesh-egress` workload annotation. Separate multiple
entries with `##`. Choose any string identifying the service on the given port as
`<name>`.

This redirects the traffic over Envoy. The endpoint must present a valid
certificate chain which must be verifiable with the
Expand All @@ -114,33 +132,14 @@ apiVersion: apps/v1
kind: Deployment
metadata:
name: web
annotations:
contrast.edgeless.systems/servicemesh-egress: "billing#127.137.0.1:8081#billing-svc:8080##cart#127.137.0.2:8081#cart-svc:8080"
spec:
replicas: 1
template:
spec:
runtimeClassName: contrast-cc
initContainers:
- name: sidecar
image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy@sha256:..."
restartPolicy: Always
volumeMounts:
- name: contrast-tls-certs
mountPath: /tls-config
env:
- name: EDG_EGRESS_PROXY_CONFIG
value: "billing#127.137.0.1:8081#billing-svc:8080##cart#127.137.0.2:8081#cart-svc:8080"
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
containers:
- name: currency-conversion
image: ghcr.io/edgelesssys/conversion:v1.2.3@...
volumeMounts:
- name: contrast-tls-certs
mountPath: /tls-config
volumes:
- name: contrast-tls-certs
emptyDir: {}
```
35 changes: 14 additions & 21 deletions docs/docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ This section guides you through the process and outlines the necessary changes.

### RuntimeClass

Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files
unchanged, you can copy the files into a separate local directory.
Contrast will add annotations and additional configuration to your Kubernetes YAML files.
If you want to keep the original files unchanged, you can copy the files into a separate local directory.
You can also generate files from a Helm chart or from a Kustomization.

<Tabs groupId="yaml-source">
Expand Down Expand Up @@ -74,7 +74,7 @@ spec: # v1.PodSpec
### Handling TLS
The initializer populates the shared volume with X.509 certificates for your workload.
In the initialization process, the `contrast-tls-certs` shared volume is populated with X.509 certificates for your workload.
These certificates are used by the [Contrast Service Mesh](components/service-mesh.md), but can also be used by your application directly.
The following tab group explains the setup for both scenarios.

Expand All @@ -92,24 +92,16 @@ The following example is for an application with these properties:
* All other endpoints require client authentication.
* The app connects to a Kubernetes service `backend.default:4001`, which requires client authentication.

Add the following sidecar definition to your workload:
Add the following annotations to your workload:

```yaml
spec: # v1.PodSpec
initContainers:
- name: tls-sidecar
image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy:latest"
restartPolicy: Always
env:
- name: EDG_INGRESS_PROXY_CONFIG
value: "main#8001#false##metrics#8080#true"
- name: EDG_EGRESS_PROXY_CONFIG
value: "backend#127.0.0.2:4001#backend.default:4001"
volumeMounts:
- name: contrast-tls-certs
mountPath: /tls-config
metadata: # apps/v1.Deployment, apps/v1.DaemonSet, ...
annotations:
contrast.edgeless.systems/sm-ingress: "main#8001#false##metrics#8080#true"
contrast.edgeless.systems/sm-egress: "backend#127.0.0.2:4001#backend.default:4001"
```

During the `generate` step, this configuration will be translated into a Service Mesh sidecar container which handles TLS connections automatically.
The only change required to the app itself is to let it connect to `127.0.0.2:4001` to reach the backend service.
You can find more detailed documentation in the [Service Mesh chapter](components/service-mesh.md).

Expand Down Expand Up @@ -161,9 +153,10 @@ cfg := &tls.Config{
## Generate policy annotations and manifest

Run the `generate` command to add the necessary components to your deployment files.
This will add the Contrast Initializer to every workload with the specified `contrast-cc` runtime class.
After that, it will generate the execution policies and add them as annotations to your
deployment files. A `manifest.json` with the reference values of your deployment will be created.
This will add the Contrast Initializer to every workload with the specified `contrast-cc` runtime class
and the Contrast Service Mesh to all workloads that have a specified configuration.
After that, it will generate the execution policies and add them as annotations to your deployment files.
A `manifest.json` with the reference values of your deployment will be created.

```sh
contrast generate resources/
Expand All @@ -177,7 +170,7 @@ If you don't want the Contrast Initializer to automatically be added to your
workloads, there are two ways you can skip the Initializer injection step,
depending on how you want to customize your deployment.

<Tabs groupId="initializer-injection">
<Tabs groupId="injection">
<TabItem value="flag" label="Command-line flag">

You can disable the Initializer injection completely by specifying the
Expand Down

0 comments on commit 6a5574f

Please sign in to comment.