diff --git a/docs/docs/architecture/observability.md b/docs/docs/architecture/observability.md index fc71fbffaf..3cb4ced15b 100644 --- a/docs/docs/architecture/observability.md +++ b/docs/docs/architecture/observability.md @@ -43,13 +43,13 @@ metrics via its [Envoy admin interface](https://www.envoyproxy.io/docs/envoy/latest/operations/admin). Be aware that the admin interface can expose private information and allows destructive operations to be performed. To enable the admin interface for the -Service Mesh, set the environment variable `EDG_ADMIN_PORT` in the configuration -of the Service Mesh sidecar. If this variable is set, the admin interface will -be started on this port. +Service Mesh, set the annotation +`contrast.edgeless.systems/servicemesh-admin-interface-port` in the configuration +of your workload. If this annotation is set, the admin interface will be started +on this port. -To access the admin interface, the Service Mesh sidecar container needs to have -a corresponding container port and the ingress settings of the Proxy have to be -configured to allow access to the specified port (see [Configuring the +To access the admin interface, the ingress settings of the Service Mesh have to +be configured to allow access to the specified port (see [Configuring the Proxy](../components/service-mesh#configuring-the-proxy)). All metrics will be exposed under the `/stats` endpoint. Metrics in Prometheus format can be scraped from the `/stats/prometheus` endpoint. diff --git a/docs/docs/components/service-mesh.md b/docs/docs/components/service-mesh.md index 53a1623d79..3e83c22a88 100644 --- a/docs/docs/components/service-mesh.md +++ b/docs/docs/components/service-mesh.md @@ -6,16 +6,25 @@ verification of the endpoints in the connection establishment is based on certificates that are part of the [PKI of the Coordinator](../architecture/certificates.md). -The service mesh can be enabled on a per-pod basis by adding the `service-mesh` -container as a [sidecar container](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/). -The service mesh container first sets up `iptables` -rules based on its configuration and then starts [Envoy](https://www.envoyproxy.io/) -for TLS origination and termination. +The service mesh can be enabled on a per-workload basis by adding a service mesh +configuration to the workload's object annotations. During the `contrast generate` +step, the service mesh is added as a [sidecar +container](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/) to +all workloads which have a specified configuration. The service mesh container first +sets up `iptables` rules based on its configuration and then starts +[Envoy](https://www.envoyproxy.io/) for TLS origination and termination. ## Configuring the Proxy -The service mesh container can be configured using the `EDG_INGRESS_PROXY_CONFIG` -and `EDG_EGRESS_PROXY_CONFIG` environment variables. +The service mesh container can be configured using the following object annotations: +- `contrast.edgeless.systems/servicemesh-ingress` to configure ingress. +- `contrast.edgeless.systems/servicemesh-egress` to configure egress. +- `contrast.edgeless.systems/servicemesh-admin-interface-port` to configure the Envoy + admin interface. If not specified, no admin interface will be started. + +If you aren't using the automatic service mesh injection and want to configure the +service mesh manually, set the environment variables `EDG_INGRESS_PROXY_CONFIG`, +`EDG_EGRESS_PROXY_CONFIG` and `EDG_ADMIN_PORT` in the service mesh sidecar directly. ### Ingress @@ -30,7 +39,7 @@ certificate of the workload and the intermediate CA certificate as the server ce If the deployment contains workloads which should be reachable from outside the Service Mesh, while still handing out the certificate chain, disable client -authentication by setting the environment variable `EDG_INGRESS_PROXY_CONFIG` as +authentication by setting the annotation `contrast.edgeless.systems/servicemesh-ingress` as `##false`. Separate multiple entries with `##`. You can choose any descriptive string identifying the service on the given port for the informational-only field ``. @@ -50,26 +59,13 @@ apiVersion: apps/v1 kind: Deployment metadata: name: web + annotations: + contrast.edgeless.systems/servicemesh-ingress: "web#8080#false##metrics#7890#true" spec: replicas: 1 template: spec: runtimeClassName: contrast-cc - initContainers: - - name: sidecar - image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy@sha256:..." - restartPolicy: Always - volumeMounts: - - name: contrast-tls-certs - mountPath: /tls-config - env: - - name: EDG_INGRESS_PROXY_CONFIG - value: "web#8080#false##metrics#7890#true" - securityContext: - privileged: true - capabilities: - add: - - NET_ADMIN containers: - name: web-svc image: ghcr.io/edgelesssys/frontend:v1.2.3@... @@ -78,14 +74,35 @@ spec: name: web - containerPort: 7890 name: metrics +``` + +When invoking `contrast generate`, the resulting deployment will be injected with the +Contrast service mesh as an init container. + +```yaml +# ... + initContainers: + - env: + - name: EDG_INGRESS_PROXY_CONFIG + value: "web#8080#false##metrics#7890#true" + image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy@sha256:..." + name: contrast-service-mesh + restartPolicy: Always + securityContext: + capabilities: + add: + - NET_ADMIN + privileged: true volumeMounts: - name: contrast-tls-certs mountPath: /tls-config - volumes: - - name: contrast-tls-certs - emptyDir: {} ``` +Note, that changing the environment variables of the sidecar container directly will +only have an effect if the workload isn't configured to automatically generate a +service mesh component on `contrast generate`. Otherwise, the service mesh sidecar +container will be regenerated on every invocation of the command. + ### Egress To be able to route the egress traffic of the workload through Envoy, the remote @@ -95,8 +112,9 @@ endpoints' IP address and port must be configurable. by the pod. * Configure the workload to connect to this IP address and port. * Set `#:#:` -as `EDG_EGRESS_PROXY_CONFIG`. Separate multiple entries with `##`. Choose any -string identifying the service on the given port as ``. +as the `contrast.edgeless.systems/servicemesh-egress` workload annotation. Separate multiple +entries with `##`. Choose any string identifying the service on the given port as +``. This redirects the traffic over Envoy. The endpoint must present a valid certificate chain which must be verifiable with the @@ -114,33 +132,14 @@ apiVersion: apps/v1 kind: Deployment metadata: name: web + annotations: + contrast.edgeless.systems/servicemesh-egress: "billing#127.137.0.1:8081#billing-svc:8080##cart#127.137.0.2:8081#cart-svc:8080" spec: replicas: 1 template: spec: runtimeClassName: contrast-cc - initContainers: - - name: sidecar - image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy@sha256:..." - restartPolicy: Always - volumeMounts: - - name: contrast-tls-certs - mountPath: /tls-config - env: - - name: EDG_EGRESS_PROXY_CONFIG - value: "billing#127.137.0.1:8081#billing-svc:8080##cart#127.137.0.2:8081#cart-svc:8080" - securityContext: - privileged: true - capabilities: - add: - - NET_ADMIN containers: - name: currency-conversion image: ghcr.io/edgelesssys/conversion:v1.2.3@... - volumeMounts: - - name: contrast-tls-certs - mountPath: /tls-config - volumes: - - name: contrast-tls-certs - emptyDir: {} ``` diff --git a/docs/docs/deployment.md b/docs/docs/deployment.md index 2e6fdeaf14..ba0412b479 100644 --- a/docs/docs/deployment.md +++ b/docs/docs/deployment.md @@ -33,8 +33,8 @@ This section guides you through the process and outlines the necessary changes. ### RuntimeClass -Contrast will add annotations to your Kubernetes YAML files. If you want to keep the original files -unchanged, you can copy the files into a separate local directory. +Contrast will add annotations and additional configuration to your Kubernetes YAML files. +If you want to keep the original files unchanged, you can copy the files into a separate local directory. You can also generate files from a Helm chart or from a Kustomization. @@ -74,7 +74,7 @@ spec: # v1.PodSpec ### Handling TLS -The initializer populates the shared volume with X.509 certificates for your workload. +In the initialization process, the `contrast-tls-certs` shared volume is populated with X.509 certificates for your workload. These certificates are used by the [Contrast Service Mesh](components/service-mesh.md), but can also be used by your application directly. The following tab group explains the setup for both scenarios. @@ -92,24 +92,16 @@ The following example is for an application with these properties: * All other endpoints require client authentication. * The app connects to a Kubernetes service `backend.default:4001`, which requires client authentication. -Add the following sidecar definition to your workload: +Add the following annotations to your workload: ```yaml -spec: # v1.PodSpec - initContainers: - - name: tls-sidecar - image: "ghcr.io/edgelesssys/contrast/service-mesh-proxy:latest" - restartPolicy: Always - env: - - name: EDG_INGRESS_PROXY_CONFIG - value: "main#8001#false##metrics#8080#true" - - name: EDG_EGRESS_PROXY_CONFIG - value: "backend#127.0.0.2:4001#backend.default:4001" - volumeMounts: - - name: contrast-tls-certs - mountPath: /tls-config +metadata: # apps/v1.Deployment, apps/v1.DaemonSet, ... + annotations: + contrast.edgeless.systems/sm-ingress: "main#8001#false##metrics#8080#true" + contrast.edgeless.systems/sm-egress: "backend#127.0.0.2:4001#backend.default:4001" ``` +During the `generate` step, this configuration will be translated into a Service Mesh sidecar container which handles TLS connections automatically. The only change required to the app itself is to let it connect to `127.0.0.2:4001` to reach the backend service. You can find more detailed documentation in the [Service Mesh chapter](components/service-mesh.md). @@ -161,9 +153,10 @@ cfg := &tls.Config{ ## Generate policy annotations and manifest Run the `generate` command to add the necessary components to your deployment files. -This will add the Contrast Initializer to every workload with the specified `contrast-cc` runtime class. -After that, it will generate the execution policies and add them as annotations to your -deployment files. A `manifest.json` with the reference values of your deployment will be created. +This will add the Contrast Initializer to every workload with the specified `contrast-cc` runtime class +and the Contrast Service Mesh to all workloads that have a specified configuration. +After that, it will generate the execution policies and add them as annotations to your deployment files. +A `manifest.json` with the reference values of your deployment will be created. ```sh contrast generate resources/ @@ -177,7 +170,7 @@ If you don't want the Contrast Initializer to automatically be added to your workloads, there are two ways you can skip the Initializer injection step, depending on how you want to customize your deployment. - + You can disable the Initializer injection completely by specifying the