-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TLS_PASSTHROUGH document #650
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
# TLSRoute API Reference | ||
|
||
## Introduction | ||
|
||
With integration of the Gateway API, AWS Gateway API Controller supports `TLSRoute`. | ||
This allows you to define and manage end-to-end TLS encrypted traffic routing to your Kubernetes clusters. | ||
|
||
### Considerations | ||
|
||
- `TLSRoute` sectionName must refer to a `TLS` protocol listener with `mode: Passthrough` in the parentRefs `Gateway`. | ||
- `TLSRoute` only supports to have one rule. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. perhaps to be more specific, only supports one default rule There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But I think Kubernetes resource TLSRoute don't have a concept of "default rule" ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ok, i misunderstood. or the wording can be "can only have one rule" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Probably change to: " the Since the k8s gateway api standard supports to have multiple rules, and other gateway api implementation can have multiple rules in TLSRoute for example: envoyproxy/gateway, https://github.com/envoyproxy/gateway But our implementation don't support that because we are base on vpc lattice There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In other docs we have a "Features" and a "Limitations" section. Can we be consistent here as well? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I did consider to add "Features" and "Limitations", but don't think TLSRoute has some fancy features that need one separate section besides some words in the "Introduction" and " Example Configuration" sections. For "Limitations", just use the word "Considerations", same as the word in the vpc lattice doc because that are more likely design decision and not limitation? What do you think? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK I think it's fine to leave as-is |
||
- `TLSRoute` does not support any rule matching condition. | ||
- The `hostnames` field with exactly one host name is required. | ||
|
||
|
||
## Example Configuration | ||
|
||
Here is a sample configuration that demonstrates how to set up a `TLSRoute` resource to route end-to-end TLS encrypted traffic to a nginx service: | ||
|
||
```yaml | ||
apiVersion: gateway.networking.k8s.io/v1alpha2 | ||
kind: TLSRoute | ||
metadata: | ||
name: nginx-tls-route | ||
spec: | ||
hostnames: | ||
- nginx-test.my-test.com | ||
parentRefs: | ||
- name: my-hotel-tls-passthrough | ||
sectionName: tls | ||
rules: | ||
- backendRefs: | ||
- name: nginx-tls | ||
kind: Service | ||
port: 443 | ||
``` | ||
|
||
In this example: | ||
|
||
- The `TLSRoute` is named ` nginx-tls-route` and is associated with a parent gateway named `my-hotel-tls-passthrough` that has | ||
a listener section named `tls`: | ||
``` | ||
- name: tls | ||
protocol: TLS | ||
port: 443 | ||
tls: | ||
mode: Passthrough | ||
``` | ||
- The `TLSRoute` is configured to route traffic to a k8s service named `nginx-tls` on port 443. | ||
- The `hostnames` field is set to `nginx-test.my-test.com`. The customer must use this hostname to send traffic to the nginx service. | ||
|
||
|
||
For the detailed tls passthrough traffic connectivity setup, please refer the user guide [here](../guides/tls-passthrough.md). | ||
|
||
For the detailed Gateway API `TLSRoute` resource specifications, you can refer to the | ||
Kubernetes official [documentation](https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1alpha2.TLSRoute). | ||
|
||
For the VPC Lattice tls passthrough Listener configuration details, you can refer to the VPC Lattice [documentation](https://docs.aws.amazon.com/vpc-lattice/latest/ug/tls-listeners.html). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,222 @@ | ||
# TLS Passthrough Support | ||
|
||
[Kubernetes Gateway API](https://gateway-api.sigs.k8s.io/guides/tls/) lays out the general guidelines on how to configure TLS passthrough. Here are examples on how to use them against AWS Gateway Api controller and VPC Lattice. | ||
|
||
## Install Gateway API TLSRoute CRD | ||
|
||
The TLSRoute CRD is already included in the helm chart and deployment.yaml, if you are using these 2 methods to install the controller no extra steps needed. | ||
If you want to install the TLSRoute CRD manually by yourself: | ||
``` | ||
# Install CRD | ||
kubectl apply -f config/crds/bases/gateway.networking.k8s.io_tlsroutes.yaml | ||
# Verfiy TLSRoute CRD | ||
kubectl get crd tlsroutes.gateway.networking.k8s.io | ||
NAME CREATED AT | ||
tlsroutes.gateway.networking.k8s.io 2024-03-07T23:16:22Z | ||
``` | ||
|
||
## Setup TLS Passthrough Connectivity in a single cluster | ||
|
||
### 1. Configure TLS Passthrough Listener on Gateway | ||
|
||
``` | ||
kubectl apply -f files/examples/my-gateway-tls-passthrough.yaml | ||
``` | ||
|
||
``` | ||
# tls listener config snips: | ||
apiVersion: gateway.networking.k8s.io/v1beta1 | ||
kind: Gateway | ||
metadata: | ||
name: my-hotel-tls-passthrough | ||
spec: | ||
gatewayClassName: amazon-vpc-lattice | ||
listeners: | ||
... | ||
- name: tls | ||
protocol: TLS | ||
port: 443 | ||
tls: | ||
mode: Passthrough | ||
... | ||
``` | ||
|
||
### 2. Configure TLSRoute | ||
|
||
``` | ||
# Suppose in the below example, we use the "parking" service as the client pod to test the TLS passthrough traffic. | ||
kubectl apply -f files/examples/parking.yaml | ||
|
||
# Configure nginx backend service (This nginx image includes a self-signed certificate) | ||
kubectl apply -f files/example/nginx-server-tls-passthrough.yaml | ||
|
||
# configure nginx tls route | ||
kubectl apply -f files/examples/tlsroute-nginx.yaml | ||
|
||
``` | ||
|
||
### 3. Verify the controller has reconciled nginx-tls route | ||
|
||
Make sure the TLSRoute has the `application-networking.k8s.aws/lattice-assigned-domain-name` annotation and status `Accepted: True` | ||
``` | ||
kubectl get tlsroute nginx-tls -o yaml | ||
apiVersion: gateway.networking.k8s.io/v1alpha2 | ||
kind: TLSRoute | ||
metadata: | ||
annotations: | ||
application-networking.k8s.aws/lattice-assigned-domain-name: nginx-tls-default-0af995120af2711bc.7d67968.vpc-lattice-svcs.us-west-2.on.aws | ||
... | ||
name: nginx-tls | ||
namespace: default | ||
... | ||
|
||
status: | ||
parents: | ||
- conditions: | ||
- lastTransitionTime: ..... | ||
message: "" | ||
observedGeneration: 1 | ||
reason: Accepted | ||
status: "True" | ||
type: Accepted | ||
- lastTransitionTime: ..... | ||
message: "" | ||
observedGeneration: 1 | ||
reason: ResolvedRefs | ||
status: "True" | ||
type: ResolvedRefs | ||
controllerName: application-networking.k8s.aws/gateway-api-controller | ||
|
||
``` | ||
|
||
### 4. Verify TLS Passthrough Traffic | ||
|
||
``` | ||
kubectl get deployment nginx-tls | ||
NAME READY UP-TO-DATE AVAILABLE AGE | ||
nginx-tls 2/2 2 2 1d | ||
|
||
# Use the specified TLSRoute hostname to send traffic to the beackend nginx service | ||
kubectl exec deployments/parking -- curl -kv https://nginx-test.my-test.com --resolve nginx-test.my-test.com:443:169.254.171.0 | ||
|
||
* Trying 169.254.171.0:443... | ||
* Connected to nginx-test.my-test.com (169.254.171.0) port 443 (#0) | ||
.... | ||
* TLSv1.2 (OUT), TLS header, Certificate Status (22): | ||
* TLSv1.2 (OUT), TLS handshake, Client hello (1): | ||
* TLSv1.2 (IN), TLS handshake, Server hello (2): | ||
* TLSv1.2 (IN), TLS handshake, Certificate (11): | ||
* TLSv1.2 (IN), TLS handshake, Server key exchange (12): | ||
* TLSv1.2 (IN), TLS handshake, Server finished (14): | ||
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16): | ||
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): | ||
* TLSv1.2 (OUT), TLS handshake, Finished (20): | ||
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): | ||
* TLSv1.2 (IN), TLS handshake, Finished (20): <---------- TLS Handshake from client pod to the backend `nginx-tls` pod successfully, no tls termination in the middle | ||
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 | ||
* ALPN, server accepted to use h2 | ||
.... | ||
<body> | ||
<h1>Welcome to nginx!</h1> | ||
<p>If you see this page, the nginx web server is successfully installed and | ||
working. Further configuration is required.</p> | ||
.... | ||
|
||
``` | ||
|
||
## Setup TLS Passthrough Connectivity spanning multiple clusters | ||
|
||
|
||
![tlsoute multi cluster](../images/tlsroute-multi-cluster.png) | ||
|
||
### 1. In this example we still use the "parking" Kubernetes service as the client pod to test the cross cluster TLS passthrough traffic. | ||
``` | ||
kubectl apply -f files/examples/parking.yaml | ||
``` | ||
|
||
### 2. In cluster-1, create `tls-rate1` Kubernetes Service: | ||
``` | ||
kubectl apply -f files/examples/tls-rate1.yaml | ||
``` | ||
|
||
### 3. Configure ServiceExport with TargetGroupPolicy `protocol:TCP` in cluster-2 | ||
|
||
``` | ||
# Create tls-rate2 Kubernetes Service in cluster-2 | ||
kubectl apply -f files/examples/tls-rate2.yaml | ||
# Create serviceexport in cluster-2 | ||
kubectl apply -f files/examples/tls-rate2-export.yaml | ||
# Create targetgroup policy to configure TCP protocol for tls-rate2 in cluster-2 | ||
kubectl apply -f files/examples/tls-rate2-targetgrouppolicy.yaml | ||
``` | ||
|
||
``` | ||
# Snips of serviceexport config | ||
apiVersion: application-networking.k8s.aws/v1alpha1 | ||
kind: ServiceExport | ||
metadata: | ||
name: tls-rate-2 | ||
annotations: | ||
application-networking.k8s.aws/federation: "amazon-vpc-lattice" | ||
# Snips of targetgroup policy config | ||
apiVersion: application-networking.k8s.aws/v1alpha1 | ||
kind: TargetGroupPolicy | ||
metadata: | ||
name: tls-rate2 | ||
spec: | ||
targetRef: | ||
group: "application-networking.k8s.aws" | ||
kind: ServiceExport | ||
name: tls-rate2 | ||
protocol: TCP | ||
``` | ||
|
||
### 4. Configure ServiceImport in cluster1 | ||
|
||
``` | ||
kubectl apply -f files/examples/tls-rate2-import.yaml | ||
``` | ||
|
||
### 5. Configure TLSRoute for blue/green deployment | ||
|
||
``` | ||
kubectl apply -f files/examples/rate-tlsroute-bluegreen.yaml | ||
|
||
# snips of TLSRoute span multiple Kubernetes Clusters | ||
apiVersion: gateway.networking.k8s.io/v1alpha2 | ||
kind: TLSRoute | ||
metadata: | ||
name: tls-rate | ||
spec: | ||
hostnames: | ||
- tls-rate.my-test.com | ||
parentRefs: | ||
- name: my-hotel-tls | ||
sectionName: tls | ||
rules: | ||
- backendRefs: | ||
- name: tls-rate1 <---------- to Kubernetes Cluster-1 | ||
kind: Service | ||
port: 443 | ||
weight: 10 | ||
- name: tls-rate2 <---------- to Kubernetes Cluster-2 | ||
kind: ServiceImport | ||
port: 443 | ||
weight: 90 | ||
``` | ||
### 6. Verify cross-cluster TLS passthrough traffic | ||
|
||
Expected to receive the weighted traffic route to tls-rate1 service(10%) and tls-rate2 service(90%), if you curl the `tls-rate.my-test.com` from the client pod multiple times: | ||
``` | ||
kubectl exec deploy/parking -- sh -c 'for ((i=1; i<=30; i++)); do curl -k https://tls-rate.my-test.com --resolve tls-rate.my-test.com:443:169.254.171.0 2>/dev/null; done' | ||
|
||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod <----> k8s service in cluster-2 | ||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod | ||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod | ||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod | ||
Requsting to TLS Pod(tls-rate1-98cc7fd87a-642zw): tls-rate1 handler pod <----> k8s service in cluster-1 | ||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod | ||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod | ||
Requsting to TLS Pod(tls-rate2-7f8b9cc97b-fgqk6): tls-rate2 handler pod | ||
Requsting to TLS Pod(tls-rate1-98cc7fd87a-642zw): tls-rate1 handler pod | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
apiVersion: gateway.networking.k8s.io/v1beta1 | ||
kind: Gateway | ||
metadata: | ||
name: my-hotel-tls-passthrough | ||
spec: | ||
gatewayClassName: amazon-vpc-lattice | ||
listeners: | ||
- name: http | ||
protocol: HTTP | ||
port: 80 | ||
- name: tls | ||
protocol: TLS | ||
port: 443 | ||
tls: | ||
mode: Passthrough |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this'll be super helpful