Skip to content

Commit

Permalink
Fixes for relay quickstart and documentation
Browse files Browse the repository at this point in the history
Html link testing will pass once googleforgames#807 has been merged.
  • Loading branch information
markmandel committed Oct 4, 2023
1 parent a6a97fe commit 7a29e25
Show file tree
Hide file tree
Showing 12 changed files with 182 additions and 215 deletions.
1 change: 1 addition & 0 deletions docs/preprocessor.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ cargo run -q --manifest-path ../Cargo.toml &> ../target/quilkin.commands || true
cargo run -q --manifest-path ../Cargo.toml -- proxy --help &> ../target/quilkin.proxy.commands || true
cargo run -q --manifest-path ../Cargo.toml -- manage --help &> ../target/quilkin.manage.commands || true
cargo run -q --manifest-path ../Cargo.toml -- relay --help &> ../target/quilkin.relay.commands || true
cargo run -q --manifest-path ../Cargo.toml -- agent --help &> ../target/quilkin.agent.commands || true

# Credit: https://github.com/rust-lang/mdBook/issues/1462#issuecomment-778650045
jq -M -c .[1] <&0
3 changes: 2 additions & 1 deletion docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- [Netcat](./deployment/quickstarts/netcat.md)
- [Agones + Xonotic (Sidecar)](./deployment/quickstarts/agones-xonotic-sidecar.md)
- [Agones + Xonotic (xDS)](./deployment/quickstarts/agones-xonotic-xds.md)
- [Agones + Xonotic (Relay)](./deployment/quickstarts/agones-xonotic-relay.md)

# Services

Expand Down Expand Up @@ -41,7 +42,7 @@

- [Relay](./services/relay.md)
- [Metrics]()
- [Providers]()
- [Agents](./services/agent.md)

# SDKs

Expand Down
215 changes: 141 additions & 74 deletions docs/src/deployment/quickstarts/agones-xonotic-relay.md

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 7 additions & 4 deletions docs/src/services/agent.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Control Plane Relay
# Control Plane Agents

| services | ports | Protocol |
|----------|-------|-----------|
| QCMP | 7600 | UDP(IPv4 OR IPv6) |
| services | ports | Protocol |
|----------|-------|-------------------|
| QCMP | 7600 | UDP(IPv4 OR IPv6) |

> **Note:** This service is currently in active experimentation and development
so there may be bugs which cause it to be unusable for production, as always
Expand All @@ -12,6 +12,9 @@ For multi-cluster integration, Quilkin provides a `agent` service, that can be
deployed to a cluster to act as a beacon for QCMP pings and forward cluster
configuration information to a `relay` service

Agent configuration and functionality matches that of Control Plane Providers, such as
[Filesystem](./xds/providers/filesystem.md) and [Agones](./xds/providers/agones.md).

To view all options for the `agent` subcommand, run:

```shell
Expand Down
8 changes: 4 additions & 4 deletions docs/src/services/relay.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Control Plane Relay

| services | ports | Protocol |
|----------|-------|-----------|
| ADS | 7800 | gRPC(IPv4) |
| CPDS | 7900 | gRPC(IPv4) |
| services | ports | Protocol |
|----------|-------|--------------------|
| ADS | 7800 | gRPC(IPv4 OR IPv6) |
| CPDS | 7900 | gRPC(IPv4 OR IPv6) |

> **Note:** This service is currently in active experimentation and development
so there may be bugs which cause it to be unusable for production, as always
Expand Down
112 changes: 2 additions & 110 deletions examples/agones-xonotic-relay/README.md
Original file line number Diff line number Diff line change
@@ -1,111 +1,3 @@
# Agones & Xonotic Example
# Agones & Xonotic via Relay Example

An example of running [Xonotic](https://xonotic.org/) with Quilkin on an [Agones](https://agones.dev/) cluster,
utlising the Quilkin xDS Agones provider, with a TokenRouter to provide routing and access control to the
allocated `GameServer` instance.

To interact with the demo, you will need to download the Xonotic client and an existing Agones Kubernetes cluster.

## Installation on the Cluster

To install Quilkin as an Agones integrated xDS control plane, we can create a deployment of Quilkin running in
`manage agones` mode, with the appropriate permissions.

```shell
$ kubectl apply -f ./xds-control-plane.yaml
configmap/quilkin-xds-filter-config created
serviceaccount/quilkin-agones created
clusterrole.rbac.authorization.k8s.io/quilkin-agones created
rolebinding.rbac.authorization.k8s.io/quilkin-agones created
deployment.apps/quilkin-manage-agones created
service/quilkin-manage-agones created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
quilkin-manage-agones-68b47457d4-42fxl 1/1 Running 0 3s
```

This also creates an internal `Service` endpoint for our Quilkin proxy instances to connect to named
`quilkin-manage-agones`.

To install the Quilkin Proxy pool which connects to the above xDS provider, we can split up a Deployment of Quilkin
instances that point to the aforementioned Service, like so:

```shell
$ kubectl apply -f ./proxy-pool.yaml
deployment.apps/quilkin-proxies created
service/quilkin-proxies created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
quilkin-manage-agones-68b47457d4-42fxl 1/1 Running 0 20m
quilkin-proxies-7448987cfb-6kbsz 1/1 Running 0 3s
quilkin-proxies-7448987cfb-p6krc 1/1 Running 0 3s
quilkin-proxies-7448987cfb-s9gxz 1/1 Running 0 3s
```

We can now see that we have 3 proxies running alongside and connected to our xDS provider.

## Create the `Fleet`

Next, create the `Fleet` of Xonotic `GameServer` instances:

```shell
$ kubectl apply -f ./fleet.yaml
fleet.agones.dev/xonotic created
$ kubectl get gameservers # run until they are Ready
NAME STATE ADDRESS PORT NODE AGE
xonotic-d7rfx-55j7q Ready 34.168.170.51 7226 gke-agones-default-534a3f8d-ifpc 34s
xonotic-d7rfx-nx7xr Ready 34.168.170.51 7984 gke-agones-default-534a3f8d-ifpc 34s
xonotic-d7rfx-sn5d6 Ready 34.168.170.51 7036 gke-agones-default-534a3f8d-ifpc 34s
```

To let the Quilkin xDS provider know what token will route to which `GameServer` we need to apply the
`quilkin.dev/tokens` annotation to an allocated `GameServer`, with the token content as its value - so let's create
an allocation, and apply the annotation all in one go!

```shell
$ kubectl create -f ./gameserverallocation.yaml
gameserverallocation.allocation.agones.dev/xonotic-d7rfx-nx7xr created
$ kubectl get gs
NAME STATE ADDRESS PORT NODE AGE
xonotic-d7rfx-55j7q Allocated 34.168.170.51 7226 gke-agones-default-534a3f8d-ifpc 23m
xonotic-d7rfx-nx7xr Ready 34.168.170.51 7984 gke-agones-default-534a3f8d-ifpc 23m
xonotic-d7rfx-sn5d6 Ready 34.168.170.51 7036 gke-agones-default-534a3f8d-ifpc 23m
```

> Don't do this more than once, as then multiple allocated `GameServers` will have the same routing token!
## Connecting Client Side

Instead of connecting to Xonotic or an Agones `GameServer` directly, we'll want to grab the IP and exposed port of
the `Service` that fronts all our Quilkin proxies:

```shell
$ kubectl get service quilkin-proxies
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
quilkin-proxies LoadBalancer 10.109.0.12 35.246.94.14 7000:30174/UDP 3h22m
```

We then take the EXTERNAL-IP and port from the `quilkin-proxies` service, and replace the`${LOADBALANCER_IP}`
with it in `client-token.yaml`.

Run this configuration locally as:

```shell
$ quilkin -c ./client-token.yaml proxy
{"timestamp":"2022-10-07T22:10:47.257635Z","level":"INFO","fields":{"message":"Starting Quilkin","version":"0.4.0-dev","commit":"c77260a2526542c564829a2c66935c60f00adcd2"},"target":"quilkin::cli"}
{"timestamp":"2022-10-07T22:10:47.258273Z","level":"INFO","fields":{"message":"Starting","port":7000,"proxy_id":"markmandel45"},"target":"quilkin::proxy"}
{"timestamp":"2022-10-07T22:10:47.258321Z","level":"INFO","fields":{"message":"Starting admin endpoint","address":"[::]:9092"},"target":"quilkin::admin"}
{"timestamp":"2022-10-07T22:10:47.258812Z","level":"INFO","fields":{"message":"Quilkin is ready"},"target":"quilkin::proxy"}
```

Now connect to the local client proxy on "127.0.0.1:7000" via the "Multiplayer > Address" field in the
Xonotic client, and Quilkin will take care of appending the routing token to all your UDP packets, which the Quilkin
proxies will route to the Allocated GameServer, and you can play a gamee!

...And you didn't have to change the client or the dedicated game server 🤸

## Metrics

The Quilkin instances are also annotated with the
[appropriate Prometheus annotations](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus#scraping-pod-metrics-via-annotations)
to inform Prometheus on how to scrape the Quilkin proxies for metrics.
This is the code example for the "Quickstart: Quilkin with Agones and Xonotic (Relay)" Guide, linked via the homepage.
8 changes: 3 additions & 5 deletions examples/agones-xonotic-relay/client-token.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2022 Google LLC
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -22,7 +22,5 @@ filters:
on_write: DO_NOTHING
bytes: NDU2 # 456
clusters:
default:
localities:
- endpoints:
- address: ${LOADBALANCER_IP}:7777
- endpoints:
- address: ${LOADBALANCER_IP}:7777
12 changes: 7 additions & 5 deletions examples/agones-xonotic-relay/fleet.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2022 Google LLC All Rights Reserved.
# Copyright 2023 Google LLC All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -21,16 +21,18 @@ metadata:
name: xonotic
spec:
replicas: 3
strategy:
type: Recreate
template:
spec:
ports:
- name: default
containerPort: 26000
- name: default
containerPort: 26000
health:
initialDelaySeconds: 30
periodSeconds: 60
template:
spec:
containers:
- name: xonotic
image: gcr.io/agones-images/xonotic-example:0.8
- name: xonotic
image: us-docker.pkg.dev/agones-images/examples/xonotic-example:1.2
2 changes: 1 addition & 1 deletion examples/agones-xonotic-relay/gameserverallocation.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2022 Google LLC All Rights Reserved.
# Copyright 2023 Google LLC All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down
6 changes: 3 additions & 3 deletions examples/agones-xonotic-relay/proxy-pool.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2022 Google LLC
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -16,7 +16,7 @@

#
# Pool of Quilkin proxies, tied to the
# xDS control plane in xds-control-plane.yaml
# Quilkin relay control plane in relay-control-plane.yaml
#

---
Expand All @@ -42,7 +42,7 @@ spec:
spec:
containers:
- name: quilkin
image: us-docker.pkg.dev/quilkin/release/quilkin:0.6.0
image: us-docker.pkg.dev/quilkin-mark-dev/release/quilkin:0.7.0
args: ["proxy", "--management-server", "http://quilkin-relay-agones:7800"]
env:
- name: RUST_LOG
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright 2022 Google LLC
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand All @@ -15,15 +15,15 @@
#

#
# Everything to setup the xDS Agones integrated control plane
# Everything to setup the relay and agent Agones control plane
#

---
# ANCHOR: config-map
apiVersion: v1
kind: ConfigMap
metadata:
name: quilkin-xds-filter-config
name: quilkin-relay-filter-config
labels:
quilkin.dev/configmap: "true"
data:
Expand All @@ -40,7 +40,7 @@ data:

---

# RBAC Setup for Agones xDS Control Plane
# RBAC Setup for Agones Relay Control Plane

apiVersion: v1
kind: ServiceAccount
Expand Down Expand Up @@ -113,7 +113,7 @@ spec:
env:
- name: RUST_LOG
value: info
image: us-docker.pkg.dev/quilkin/release/quilkin:0.6.0
image: us-docker.pkg.dev/quilkin-mark-dev/release/quilkin:0.7.0
livenessProbe:
failureThreshold: 3
httpGet:
Expand Down Expand Up @@ -152,12 +152,13 @@ spec:
- name: quilkin
args:
- agent
- --relay
- http://quilkin-relay-agones:7900
- agones
env:
- name: RUST_LOG
value: info
image: us-docker.pkg.dev/quilkin/release/quilkin:0.6.0
image: us-docker.pkg.dev/quilkin-mark-dev/release/quilkin:0.7.0
livenessProbe:
failureThreshold: 3
httpGet:
Expand All @@ -172,10 +173,12 @@ metadata:
name: quilkin-relay-agones
spec:
ports:
- port: 7800
- name: ads
port: 7800
protocol: TCP
targetPort: 7800
- port: 7900
- name: cpds
port: 7900
protocol: TCP
targetPort: 7900
selector:
Expand Down

0 comments on commit 7a29e25

Please sign in to comment.