Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KUBE-503: adjust Github workflows, adjust for latest Proto definitions and add e2e test #3

Merged
merged 9 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 37 additions & 16 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,43 +57,64 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

# TODO: uncomment after Helm chart is released
# - name: Create k8s Kind Cluster
# uses: helm/kind-action@v1
# with:
# cluster_name: cloud-proxy-e2e

# - uses: azure/[email protected]

# - name: Run e2e tests
# run: |
# echo "$GCP_CREDENTIALS_JSON" > "$GCP_CREDENTIALS"
# ./e2e/run.sh
# env:
# GCP_CREDENTIALS_JSON: ${{ secrets.TEST_GCP_CREDENTIALS }}
# GCP_CREDENTIALS: gcp-credentials.json

- name: Login to Google Artifact Registry
uses: docker/login-action@v3
with:
registry: us-docker.pkg.dev
username: _json_key
password: ${{ secrets.ARTIFACT_BUILDER_JSON_KEY }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v2
if: ${{ github.event_name == 'pull_request' }}
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push pr
if: github.event_name == 'pull_request'
uses: docker/build-push-action@v4
with:
context: .
platforms: linux/arm64,linux/amd64
push: ${{ github.event_name == 'pull_request' }}
tags: ghcr.io/castai/cloud-proxy:${{ github.sha }}

- name: Build and push main
if: github.event_name != 'release'
uses: docker/build-push-action@v3
uses: docker/build-push-action@v4
with:
context: .
push: true
platforms: linux/arm64,linux/amd64
push: ${{ github.event_name != 'pull_request' }}
tags: |
us-docker.pkg.dev/castai-hub/library/cloud-proxy:${{ github.sha }}

- name: Build and push release
if: github.event_name == 'release'
uses: docker/build-push-action@v3
uses: docker/build-push-action@v4
with:
context: .
push: true
platforms: linux/arm64,linux/amd64
push: true
tags: |
us-docker.pkg.dev/castai-hub/library/cloud-proxy:${{ env.RELEASE_TAG }}
us-docker.pkg.dev/castai-hub/library/cloud-proxy:latest

#- name: Docker pull for fossa main
# if: github.event_name == 'release'
# run: docker pull us-docker.pkg.dev/castai-hub/library/cloud-proxy:${{ env.RELEASE_TAG }}
#
#- name: FOSSA scan docker image
# if: github.event_name == 'release'
# continue-on-error: true
# uses: fossas/fossa-action@v1
# with:
# api-key: ${{ secrets.FOSSA_API_KEY }}
# container: us-docker.pkg.dev/castai-hub/library/cloud-proxy:${{ env.RELEASE_TAG }}

7 changes: 4 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# If you prefer the allow list template instead of the deny list, see community template:
# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
#
.terraform
.terraform/*.json
terraform.tfstate*

# Binaries for programs and plugins
*.exe
*.exe~
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM gcr.io/distroless/static-debian11
FROM gcr.io/distroless/static-debian11:nonroot

ARG TARGETARCH="amd64"

Expand Down
11 changes: 8 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,23 +1,28 @@
VERSION ?= poc2
REPO ?= lachezarcast/cloud-proxy
VERSION ?= latest
REPO ?= us-docker.pkg.dev/castai-hub/library/cloud-proxy

build:
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags "-s -w" -o bin/castai-cloud-proxy-amd64 ./cmd/proxy
#docker build -t us-docker.pkg.dev/castai-hub/library/svc:$(VERSION) .
docker build -t $(REPO):$(VERSION) --platform linux/amd64 .
.PHONY: build

push:
docker push $(REPO):$(VERSION)
.PHONY: push

release: build push
.PHONY: release

deploy: build push
# Get the latest digest because it doesn't work for some f. reason and put it in the yaml
@DIGEST=$$(docker inspect --format='{{index .RepoDigests 0}}' $(REPO):$(VERSION) | awk -F@ '{print $$2}'); \
sed "s/{{IMAGE_DIGEST}}/$${DIGEST}/g" dummy_deploy.yaml > tmp.yaml
kubectl apply -f tmp.yaml
rm tmp.yaml

.PHONY: deploy

generate-grpc:
protoc --go_out=./internal/castai/proto --go-grpc_out=./internal/castai/proto ./internal/castai/proto/proxy.proto
.PHONY: generate-grpc

19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,22 @@ gcloud projects add-iam-policy-binding projects/<PROJECT_ID> \
--condition=None
```

## Dev cloud-proxy deployment

You can use this [Terraform module](./hack/terraform/) to create a GKE cluster, onboard to CAST AI and install the castai-cloud-proxy Helm chart.

You might need to tweak the values in the [helm_release](./hack/terraform/cloud-proxy.tf) resource for the castai-cloud-proxy, to use the proper image and Helm chart. For now the Helm chart is not published, to you have to clone the [helm-charts repo](https://github.com/castai/helm-charts) and provide a local path in the `helm_release`.

To deploy the Terraform module execute:

```bash
export TF_VAR_castai_api_url=https://api-...localenv.cast.ai
export TF_VAR_castai_api_token=<your-token>
export TF_VAR_castai_grpc_url=grpc-...localenv.cast.ai:443
export TF_VAR_cluster_name=<cluster_name>

terraform -chdir=hack/terraform apply
```

The castai-cloud-proxy will try to connect the GRPC server provided in the variables.

7 changes: 0 additions & 7 deletions charts/cast-cloud-proxy/Chart.yaml

This file was deleted.

53 changes: 0 additions & 53 deletions charts/cast-cloud-proxy/templates/deployment.yaml

This file was deleted.

13 changes: 0 additions & 13 deletions charts/cast-cloud-proxy/templates/serviceaccount.yaml

This file was deleted.

67 changes: 0 additions & 67 deletions charts/cast-cloud-proxy/values.yaml

This file was deleted.

Loading
Loading