Decouple crib config from cl-cluster helm chart (#12877)

.gitignore

@@ -25,6 +25,7 @@ tools/clroot/db.sqlite3-wal
 .env*
 .dbenv
 !charts/chainlink-cluster/.env.example
+!crib/.env.example
 !.github/actions/setup-postgres/.env
 .direnv
 .idea

charts/chainlink-cluster/README.md

@@ -2,112 +2,9 @@
 
 Example CL nodes cluster for system level tests
 
-Install `kubefwd` (no nixpkg for it yet, planned)
-
-```
-brew install txn2/tap/kubefwd
-```
-
-If you want to build images you need [docker](https://docs.docker.com/engine/install/) service running
-
-Enter the shell (from the root project dir)
-
-```
-nix develop
-```
-
 # Develop
-
-## New cluster
-
-We are using [devspace](https://www.devspace.sh/docs/getting-started/installation?x0=3)
-
-Configure the cluster, see `deployments.app.helm.values` and [values.yaml](./values.yaml) comments for more details
-
-Set up your K8s access
-
-```
-export DEVSPACE_IMAGE="..."
-./setup.sh ${my-personal-namespace-name-crib}
-```
-
-Create a .env file based on the .env.sample file
-
-```sh
-cp .env.sample .env
-# Fill in the required values in .env
-```
-
-Build and deploy the current state of your repository
-
-```
-devspace deploy
-```
-
-Default `ttl` is `72h`, use `ttl` command to update if you need more time
-
-Valid values are `1h`, `2m`, `3s`, etc. Go time format is invalid `1h2m3s`
-
-```
-devspace run ttl ${namespace} 120h
-```
-
-If you want to deploy an image tag that is already available in ECR, use:
-
-```
-devspace deploy --override-image-tag "<image-tag>"
-```
-
-If you want to deploy an image tag from a public ECR repo, use:
-
-```
-export DEVSPACE_IMAGE=public.ecr.aws/chainlink/chainlink
-devspace deploy --override-image-tag 2.9.0
-```
-
-Forward ports to check UI or run tests
-
-```
-devspace run connect ${my-personal-namespace-name-crib}
-```
-
-List ingress hostnames
-
-```
-devspace run ingress-hosts
-```
-
-Destroy the cluster
-
-```
-devspace purge
-```
-
-## Running load tests
-
-Check this [doc](../../integration-tests/load/ocr/README.md)
-
-If you used `devspace dev ...` always use `devspace reset pods` to switch the pods back
-
-# Helm
-
-If you would like to use `helm` directly, please uncomment data in `values.yaml`
-
-## Install from local files
-
-```
-helm install -f values.yaml cl-cluster .
-```
-
-Forward all apps (in another terminal)
-
-```
-sudo kubefwd svc -n cl-cluster
-```
-
-Then you can connect and run your tests
-
 ## Install from release
+Note: The setup below doesn't work at the moment.
 
 Add the repository
 
@@ -130,6 +27,7 @@ helm install -f values.yaml cl-cluster .
 ```
 
 ## Create a new release
+Note: The setup below doesn't work at the moment.
 
 Bump version in `Chart.yml` add your changes and add `helm_release` label to any PR to trigger a release
 
@@ -143,52 +41,4 @@ helm test cl-cluster
 
 ```
 helm uninstall cl-cluster
-```
-
-# Grafana dashboard
-
-We are using [Grabana](https://github.com/K-Phoen/grabana) lib to create dashboards programmatically
-
-You can also select dashboard platform in `INFRA_PLATFORM` either `kubernetes` or `docker`
-
-You can select the dashboard panels with `PANELS_INCLUDED` which is a list of panel names separated by comma
-If you don't specify it will include core panels by default
-
-```
-export LOKI_TENANT_ID=promtail
-export LOKI_URL=...
-export GRAFANA_URL=...
-export GRAFANA_TOKEN=...
-export PROMETHEUS_DATA_SOURCE_NAME=Thanos
-export LOKI_DATA_SOURCE_NAME=Loki
-export INFRA_PLATFORM=kubernetes
-export GRAFANA_FOLDER=DashboardCoreDebug
-export DASHBOARD_NAME=CL-Cluster
-
-devspace run dashboard_deploy
-```
-
-Open Grafana folder `DashboardCoreDebug` and find dashboard `ChainlinkClusterDebug`
-
-# Testing
-
-Deploy your dashboard and run soak/load [tests](../../integration-tests/load/), check [README](../../integration-tests/README.md) for further explanations
-
-```
-devspace run dashboard_deploy
-devspace run workload
-devspace run dashboard_test
-```
-
-# Local Testing
-
-Go to [dashboard-lib](../../dashboard) and link the modules locally
-
-```
-cd dashboard
-pnpm link --global
-cd charts/chainlink-cluster/dashboard/tests
-pnpm link --global dashboard-tests
-```
-
-Then run the tests with commands mentioned above
+```

crib/README.md

@@ -0,0 +1,156 @@
+# Crib Devspace Setup
+
+CRIB is a devspace configuration to launch chainlink cluster for system level tests
+
+Install `kubefwd` (no nixpkg for it yet, planned)
+
+```
+brew install txn2/tap/kubefwd
+```
+
+If you want to build images you need [docker](https://docs.docker.com/engine/install/) service running
+
+Enter the shell (from the root project dir)
+
+```
+nix develop
+```
+
+# Develop
+
+## New cluster
+
+We are using [devspace](https://www.devspace.sh/docs/getting-started/installation?x0=3)
+
+Configure the cluster, see `deployments.app.helm.values` and [values.yaml](../charts/chainlink-cluster/values.yaml) comments for more details
+
+Set up your K8s access
+
+```
+export DEVSPACE_IMAGE="..."
+./setup.sh ${my-personal-namespace-name-crib}
+```
+
+Create a .env file based on the .env.sample file
+
+```sh
+cp .env.sample .env
+# Fill in the required values in .env
+```
+
+Build and deploy the current state of your repository
+
+```
+devspace deploy
+```
+
+Default `ttl` is `72h`, use `ttl` command to update if you need more time
+
+Valid values are `1h`, `2m`, `3s`, etc. Go time format is invalid `1h2m3s`
+
+```
+devspace run ttl ${namespace} 120h
+```
+
+If you want to deploy an image tag that is already available in ECR, use:
+
+```
+devspace deploy --override-image-tag "<image-tag>"
+```
+
+If you want to deploy an image tag from a public ECR repo, use:
+
+```
+export DEVSPACE_IMAGE=public.ecr.aws/chainlink/chainlink
+devspace deploy --override-image-tag 2.9.0
+```
+
+Forward ports to check UI or run tests
+
+```
+devspace run connect ${my-personal-namespace-name-crib}
+```
+
+List ingress hostnames
+
+```
+devspace run ingress-hosts
+```
+
+Destroy the cluster
+
+```
+devspace purge
+```
+
+## Running load tests
+
+Check this [doc](../../integration-tests/load/ocr/README.md)
+
+If you used `devspace dev ...` always use `devspace reset pods` to switch the pods back
+
+# Helm
+
+If you would like to use `helm` directly, please uncomment data in `values.yaml`
+
+## Install from local files
+
+```
+helm install -f values.yaml cl-cluster .
+```
+
+Forward all apps (in another terminal)
+
+```
+sudo kubefwd svc -n cl-cluster
+```
+
+Then you can connect and run your tests
+
+# Grafana dashboard
+
+We are using [Grabana](https://github.com/K-Phoen/grabana) lib to create dashboards programmatically
+
+You can also select dashboard platform in `INFRA_PLATFORM` either `kubernetes` or `docker`
+
+You can select the dashboard panels with `PANELS_INCLUDED` which is a list of panel names separated by comma
+If you don't specify it will include core panels by default
+
+```
+export LOKI_TENANT_ID=promtail
+export LOKI_URL=...
+export GRAFANA_URL=...
+export GRAFANA_TOKEN=...
+export PROMETHEUS_DATA_SOURCE_NAME=Thanos
+export LOKI_DATA_SOURCE_NAME=Loki
+export INFRA_PLATFORM=kubernetes
+export GRAFANA_FOLDER=DashboardCoreDebug
+export DASHBOARD_NAME=CL-Cluster
+
+devspace run dashboard_deploy
+```
+
+Open Grafana folder `DashboardCoreDebug` and find dashboard `ChainlinkClusterDebug`
+
+# Testing
+
+Deploy your dashboard and run soak/load [tests](../../integration-tests/load/), check [README](../../integration-tests/README.md) for further explanations
+
+```
+devspace run dashboard_deploy
+devspace run workload
+devspace run dashboard_test
+```
+
+# Local Testing
+
+Go to [dashboard-lib](../dashboard-lib) and link the modules locally
+
+```
+cd dashboard
+pnpm link --global
+cd crib/dashboard/tests
+pnpm link --global dashboard-tests
+```
+
+Then run the tests with commands mentioned above

charts/chainlink-cluster/devspace.yaml → crib/devspace.yaml

@@ -52,7 +52,7 @@ commands:
   ttl: |-
     kubectl label namespace $1 cleanup.kyverno.io/ttl=$2 --overwrite
   workload: |-
-    cd ../../integration-tests/load/ocr && go test -v -run TestOCRLoad || cd -
+    cd ../integration-tests/load/ocr && go test -v -run TestOCRLoad || cd -
   dashboard_deploy: |-
     go run dashboard/cmd/deploy.go
   dashboard_test: |-
@@ -98,7 +98,7 @@ deployments:
       releaseName: "app"
       chart:
         name: cl-cluster
-        path: .
+        path: ../charts/chainlink-cluster
       # for simplicity, we define all the values here
       # they can be defined the same way in values.yml
       # devspace merges these "values" with the "values.yaml" before deploy

charts/chainlink-cluster/go.mod → crib/go.mod

@@ -34,5 +34,5 @@ replace (
 	github.com/mwitkow/grpc-proxy => github.com/smartcontractkit/grpc-proxy v0.0.0-20230731113816-f1be6620749f
 
 	github.com/sercand/kuberesolver/v4 => github.com/sercand/kuberesolver/v5 v5.1.1
-	github.com/smartcontractkit/chainlink/dashboard-lib => ../../dashboard-lib
+	github.com/smartcontractkit/chainlink/dashboard-lib => ./../dashboard-lib
 )

dashboard-lib/README.md

@@ -16,7 +16,7 @@ dashboard
 
 Each component should contain rows, logic and unique variables in `component.go`
 
-Components should be imported from this module, see [example](../charts/chainlink-cluster/dashboard/cmd/deploy.go)
+Components should be imported from this module, see [example](../crib/dashboard/cmd/deploy.go)
 
 ## How to convert from JSON using Grabana codegen utility
 1. Download Grabana binary [here](https://github.com/K-Phoen/grabana/releases)

integration-tests/load/ocr/README.md

@@ -13,7 +13,7 @@ sudo kubefwd svc -n cl-cluster
 
 Change environment connection configuration [here](../../../charts/chainlink-cluster/connect.toml)
 
-If you haven't changed anything in [devspace.yaml](../../../charts/chainlink-cluster/devspace.yaml) then default connection configuration will work
+If you haven't changed anything in [devspace.yaml](../../crib/devspace.yaml) then default connection configuration will work
 
 ## Usage