Skip to content

Commit

Permalink
Updates to ReadMe and example files for clearer instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
igooch committed Feb 20, 2024
1 parent 95e6039 commit 4e124e3
Show file tree
Hide file tree
Showing 4 changed files with 104 additions and 33 deletions.
71 changes: 48 additions & 23 deletions examples/simple-genai-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,49 +18,74 @@ type GenAIRequest struct {
```

If you need a different request structure, you will need to clone or fork this repository and
modify this in `main.go`. Update `REPOSITORY` in the `agones/examples/simple-genai-server/Makefile`
modify this in [main.go](main.go). Update `REPOSITORY` in the
[agones/examples/simple-genai-server/Makefile](Makefile)
to your own container registry and run `make build && make push` from within the
`agones/examples/simple-genai-server` path. Then modify the `gameserver.yaml` to pull the image from
your container registry. If you are making a series of changes you may also want to add
`imagePullPolicy: Always` to the container image in `gameserver.yaml`.

## Setting up the Game Server
## Setting up Agones

This example uses two separate clusters for the GenAI server and the game server. For the game
server follow the instructions for [creating a cluster](https://agones.dev/site/docs/installation/creating-cluster/)
and [installing Agones](https://agones.dev/site/docs/installation/install-agones/).
To set up the Game Servers on a different cluster than the GenAI server follow the
instructions for [creating a cluster](https://agones.dev/site/docs/installation/creating-cluster/).

Modify the `gameserver.yaml` `GenAiEndpoint` value to your inference server's endpoint. If you want
to manually interact with the GenAI endpoint via netcat, remove the rest of the env variables in the
`gameserver.yaml`. Optionally, include the `GenAiContext` in your `gameserver.yaml`.
To set the Game Servers on the same cluster as your GenAI server
[create the firewall](https://agones.dev/site/docs/installation/creating-cluster/gke/#creating-the-firewall),
[create an Agones controller node pool](https://agones.dev/site/docs/installation/creating-cluster/gke/#optional-creating-a-dedicated-node-pool),
optionally [create a metrics node pool](https://agones.dev/site/docs/installation/creating-cluster/gke/#optional-creating-a-metrics-node-pool),
and finally create a node pool for your game servers:
```bash
gcloud container node-pools create game-servers \
--cluster=[CLUSTER_NAME] \
--zone=[COMPUTE_ZONE] \
--tags=game-server \
--node-taints agones.dev/role=gameserver:NoExecute \
--node-labels agones.dev/role=gameserver \
--num-nodes=4 \
--machine-type=e2-standard-4
```

If you want to have two clients "chat" to each other, modify the `gameserver.yaml` `SimEndpoint`
value to your inference server's endpoint. Alternatively you can create a basic http server that
accepts requests in the structure noted in the above section, and returns a predetermined set of
responses for the chat. The `GenAiContext` is sent to the `GenAiEndpoint` with each request, and the
`SimContext` is sent to the `SimEndpoint` with each request as part of the GenAIRequest structure.
The default values for `GenAiContext` and `SimContext` are empty strings. The `Prompt` is the first
sent to prompt send to the GenAI endpoint to start the chat. The default values for the prompt is an
empty string. `NumChats` is the number of requests made to the `SimEndpoint` and `GenAiEndpoint`.
The default value for is `NumChats` is `1`.
In either case, follow the instructions for [installing Agones](https://agones.dev/site/docs/installation/install-agones/).

## Setting up the Game Server

To manually interact with GenAI endpoint via netcat, modify the
[gameserver_manualchat.yaml](gameserver_manualchat.yaml) `GenAiEndpoint` value to your inference
server's endpoint. Optionally, include the `GenAiContext` in your `gameserver.yaml` that will
include the context with each chat (post request) that you make to the GenAI endpoint.

If you want to have two clients "chat" to each other, modify the
[gameserver_autochat.yaml](gameserver_autochat.yaml) `GenAiEndpoint` value to your inference
server's endpoint. Also modift the `SimEndpoint` value to your inference server's endpoint.
Alternatively you can create a basic http server that accepts requests in the structure noted in the
above section, and returns a predetermined set of responses for the chat. The `GenAiContext` is sent
to the `GenAiEndpoint` with each request, and the `SimContext` is sent to the `SimEndpoint` with
each request as part of the GenAIRequest structure. The default values for `GenAiContext` and
`SimContext` are empty strings. The `Prompt` is the first sent to prompt send to the GenAI endpoint
to start the chat. The default values for the prompt is an empty string. `NumChats` is the number of
requests made to the `SimEndpoint` and `GenAiEndpoint`. The default value for is `NumChats` is `1`.

## Running the Game Server

Once you have modified the game server, apply to your Agones cluster with `kubectl -f gameserver.yaml`.
Once you have modified the `gameserver_autochat.yaml` or `gameserver_manualchat.yaml` to use your
endpoint(s), apply to your Agones cluster with `kubectl apply -f gameserver_autochat.yaml` or
`kubectl apply -f gameserver_manualchat.yaml`

Note that if your inference server is in a different cluster you'll want to make sure you're using
the kubectl context that points to your Agones cluster and not the inference cluster.

If you set up the `SimEndpoint` the chat will be in the game server logs:
If you set up the `gameserver_autochat.yaml` the chat will be in the game server logs:

```bash
kubectl logs -f gen-ai-server -c simple-genai-game-server
kubectl logs -f gen-ai-server-auto -c simple-genai-game-server
```

If you did not set up `SimEndpoint` you can manually send requests to the GenAI endpoint. Retreive
the IP address and port:
If you set up the `gameserver_manualchat.yaml` you can manually send requests to the GenAI endpoint.
Retreive the IP address and port:

```bash
kubectl get gs gen-ai-server -o jsonpath='{.status.address}:{.status.ports[0].port}'
kubectl get gs gen-ai-server-manual -o jsonpath='{.status.address}:{.status.ports[0].port}'
```

You can now send requests to the GenAI endpoint:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
apiVersion: agones.dev/v1
kind: GameServer
metadata:
name: gen-ai-server
name: gen-ai-server-auto
spec:
ports:
- name: default
Expand All @@ -27,9 +27,12 @@ spec:
containers:
- name: simple-genai-game-server
image: us-docker.pkg.dev/agones-images/examples/simple-genai-game-server:0.1
# imagePullPolicy: Always # add for development
env:
- name: GenAiEndpoint
# Replace with your GenAI and Sim inference servers' endpoint addresses.
# Replace with your GenAI and Sim inference servers' endpoint addresses. If the game
# server is in the same cluster as your inference server you can also use the k8s
# service discovery such as value: http://vertex-chat-api.genai.svc.cluster.local:80
value: "http://192.1.1.2/genai/chat"
- name: SimEndpoint
value: "http://192.1.1.2/genai/chat"
Expand Down
46 changes: 46 additions & 0 deletions examples/simple-genai-server/gameserver_manualchat.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
# Copyright 2024 Google LLC All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: agones.dev/v1
kind: GameServer
metadata:
name: gen-ai-server-manual
spec:
ports:
- name: default
portPolicy: Dynamic
containerPort: 7654
protocol: TCP
template:
spec:
containers:
- name: simple-genai-game-server
image: us-docker.pkg.dev/agones-images/examples/simple-genai-game-server:0.1
# imagePullPolicy: Always # add for development
env:
- name: GenAiEndpoint
# Replace with your GenAI and Sim inference servers' endpoint addresses. If the game
# server is in the same cluster as your inference server you can also use the k8s
# service discovery such as value: http://vertex-chat-api.genai.svc.cluster.local:80
value: "http://192.1.1.2/genai/chat"
- name: GenAiContext
# Context is optional, and will be sent along with each post request
value: "You are a car salesperson"
resources:
requests:
memory: 64Mi
cpu: 20m
limits:
memory: 64Mi
cpu: 20m
13 changes: 5 additions & 8 deletions examples/simple-genai-server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,6 @@ func main() {
simContext := flag.String("SimContext", "", "Context for the Sim endpoint")
numChats := flag.Int("NumChats", 1, "Number of back and forth chats between the sim and genAI")

var simConn *connection
var genAiConn *connection

flag.Parse()
if ep := os.Getenv("PORT"); ep != "" {
port = &ep
Expand Down Expand Up @@ -89,6 +86,7 @@ func main() {
log.Fatalf("Could not send ready message")
}

var simConn *connection
if *simEndpoint != "" {
log.Printf("Creating Sim Client at endpoint %s", *simEndpoint)
simConn = initClient(*simEndpoint, *simContext, "Sim")
Expand All @@ -98,7 +96,7 @@ func main() {
log.Fatalf("GenAiEndpoint must be specified")
}
log.Printf("Creating GenAI Client at endpoint %s", *genAiEndpoint)
genAiConn = initClient(*genAiEndpoint, *genAiContext, "GenAI")
genAiConn := initClient(*genAiEndpoint, *genAiContext, "GenAI")

// Start up TCP listener so the user can interact with the GenAI endpoint manually
if simConn == nil {
Expand Down Expand Up @@ -175,12 +173,11 @@ func autonomousChat(prompt string, conn1 *connection, conn2 *connection, numChat
response, err := handleGenAIRequest(prompt, conn1)
if err != nil {
log.Fatalf("could not send request: %v", err)
} else {
log.Printf("%d %s RESPONSE: %s\n", numChats, conn1.name, response)
}
log.Printf("%d %s RESPONSE: %s\n", numChats, conn1.name, response)

numChats -= 1
// Flip between the connection that the response is sent to
// Flip between the connection that the response is sent to.
autonomousChat(response, conn2, conn1, numChats)
}

Expand All @@ -196,7 +193,7 @@ func tcpListener(port string, genAiConn *connection) {
for {
conn, err := ln.Accept()
if err != nil {
log.Printf("Unable to accept incoming TCP connection: %v", err)
log.Fatalf("Unable to accept incoming TCP connection: %v", err)
}
go tcpHandleConnection(conn, genAiConn)
}
Expand Down

0 comments on commit 4e124e3

Please sign in to comment.