SS-1029 running serve in rancher desktop (#41)

Co-authored-by: Hamza <[email protected]>
ScilifelabDataCentre · Sep 25, 2024 · d3b07a5 · d3b07a5
1 parent 4af4743
commit d3b07a5
Show file tree

Hide file tree

Showing 31 changed files with 364 additions and 126 deletions.
diff --git a/README.md b/README.md
@@ -9,30 +9,321 @@ This repository contains Helm charts for SciLifeLab Serve.
 If you are using SciLifeLab Serve and notice a bug or if there is a feature you would like to be added feel free to [create an issue](https://github.com/ScilifelabDataCentre/serve/issues/new/choose) with a bug report or feature request.
 
 ## How to deploy
+
+### Prerequisites
+
+- A Kubernetes cluster version **1.28.6**
+- Helm 3
+- A storage class for dynamic provisioning of persistent volumes
+
+If you are going to run this on a remote cluster, then you probably don't need to think about this
+as these things will be provided by your cloud provider.
+
+But in case of a local deployment, navigate to the next section.
+
+#### Setup for local deployment
+
+If you are going to run this locally, you need to have a Kubernetes cluster running on your machine. 
+You can use [Rancher Desktop](https://rancherdesktop.io/) for this purpose.
+
+Follow their instruction to install Rancher Desktop, and then start it.
+
+Recommended settings for Rancher Desktop:
+- `Preferences > Kubernetes` select kubernetes version `1.28.6`.
+- `Preferences > Container Engine` select `containerd` as the container engine.
+- `Preferences > Virtual Machine > Emulation` select `QEMU`
+  - If you are running on an M3 Mac select `VZ`
+- `Preferences > Virtual Machine > Hardware` select `4 CPUs` and `16 GB` of memory.
+
+##### Serve image
+
+By default, the image is pulled from the public registry. This image is the one we are using in production.
+So you don't need to build the image yourself if you want to just try it out locally.
+
+But if you want to develop, you need to build the image yourself.
+
+**Building image for Rancher Desktop**
+
+Rancher Desktop brings a number of tools when you install it. 
+One of them is `nerdctl` which is a drop-in replacement for `docker` and `docker-compose`.
+
+Rancher Desktop also brings a local registry that you can use to push images to. 
+And this registry can be accessed from your Kubernetes cluster and used as if you were using docker.
+
+See [Serve](https://github.com/ScilifelabDataCentre/serve/) repository for up-to-date instructions on 
+how to build the image for local development.
+
+But this setup expects that you have an image tagged `mystudio` built using `nerdctl` and pushed to the `k8s.io` namespace.
+
+### Deploying
+
+> Using the following you'll make sure that your Rancher Desktop installation is working as expected using the default settings.
+> These instructions are almost the same as the ones you would use for a remote cluster except for the storage class.
+> If it doesn't work you should debug your installation and contact team members for help.
+
+**Outcomes of this section**
+- You'll prepare your environment for the proper local deployment of Serve;
+- Running instance of Serve on your local machine available on [http://studio.127.0.0.1.nip.io/](http://studio.127.0.0.1.nip.io/).
+
+
+
 First, clone this repository
+
+```bash
+$ git clone https://github.com/ScilifelabDataCentre/serve-charts.git
+```
+
+Then navigate to the `serve-charts/serve` folder
+
+```bash
+$ cd serve-charts/serve
 ```
-git clone https://github.com/ScilifelabDataCentre/serve-charts.git
+
+Now you need to create an override file for the `values.yaml` file.
+
+Create a file called `values-local.yaml` and add the following content:
+
+```yaml filename="values-local.yaml"
+# https://helm.sh/docs/chart_template_guide/yaml_techniques/#yaml-anchors
+# for local development
+storageClass: &storage_class local-path
+#storage access mode
+access_mode: &access_mode ReadWriteOnce
+accessmode: *access_mode
+
+global:
+  studio:
+    superuserPassword: "Test@12345"
+    superuserEmail: "[email protected]"
+    storageClass: *storage_class
+  postgresql:
+    storageClass: *storage_class
+
+studio:
+  # Only locally on a debug environment
+  debug: true
+  storage:
+    storageClass: *storage_class
+  media:
+    storage:
+      storageClass: *storage_class
+      accessModes: *access_mode
+
+postgresql:
+  primary:
+    persistence:
+      storageClass: *storage_class
+      accessModes:
+        - *access_mode
 ```
 
-Then navigate to the `serve-charts/serve` folder, and run
+This is necessary because the default values are set for a production environment. Specifically, the storage class 
+has to change because the default storage class is not available in a Rancher Desktop environment.
 
+```bash
+$ helm dependency update
+# The following command will install the chart with the values from values.yaml and values-local.yaml
+# values-local.yaml will override the values from values.yaml
+$ helm install serve . -f values.yaml -f values-local.yaml
 ```
-helm dependency update
-helm install serve .
+
+As a result you should have a running instance of Serve on your local machine available on [http://studio.127.0.0.1.nip.io/](http://studio.127.0.0.1.nip.io/).
+
+#### Swapping default docker image with the one built locally
+
+<details>
+  <summary>TJ;DR Just commands</summary>
+
+  ```bash
+  $ git clone https://github.com/ScilifelabDataCentre/serve-charts.git
+  $ cd serve-charts/serve
+  $ cat <<EOF > values-local.yaml
+environment: "local"
+# Path will be mounted using rancher desktop to the /app path in the container
+source_code_path: "/Users/nikch187/Projects/sll/serve"
+# https://helm.sh/docs/chart_template_guide/yaml_techniques/#yaml-anchors
+# for local development
+storageClass: &storage_class local-path
+#storage access mode
+access_mode: &access_mode ReadWriteOnce
+accessmode: *access_mode
+
+global:
+  studio:
+    superuserPassword: "Test@12345"
+    superuserEmail: "[email protected]"
+    storageClass: *storage_class
+  postgresql:
+    storageClass: *storage_class
+
+studio:
+  # Only locally on a debug environment
+  debug: true
+  storage:
+    storageClass: *storage_class
+  media:
+    storage:
+      storageClass: *storage_class
+      accessModes: *access_mode
+
+  # We use pull policy Never because see the following link:
+  # https://github.com/rancher-sandbox/rancher-desktop/issues/952#issuecomment-993135128
+  static:
+    image: mystudio
+    pullPolicy: Never
+
+  image:
+    repository: mystudio
+    pullPolicy: Never
+
+  securityContext:
+    # Disables security context for local development
+    # Essentially allow the container to run as root
+    enabled: false
+
+  readinessProbe:
+    enabled: false
+
+  livenessProbe:
+    enabled: false
+
+postgresql:
+  primary:
+    persistence:
+      storageClass: *storage_class
+      accessModes:
+        - *access_mode 
+  EOF
+  $ helm upgrade serve . -f values.yaml -f values-local.yaml
+  ```
+</details>
+
+**Outcomes of this section:**
+- Instead of a Django server, you'll have an ssh server running for the [PyCharm setup](https://github.com/ScilifelabDataCentre/serve/?tab=readme-ov-file#deploy-serve-for-local-development-with-rancher-desktop)
+- You'll have a host machine's folder with the [Serve](https://github.com/ScilifelabDataCentre/serve/) code mounted to the container;
+
+Now that everything is running, you can swap the default image with the one you built locally.
+
+> See the [Serve image section](https://github.com/ScilifelabDataCentre/serve/?tab=readme-ov-file#deploy-serve-for-local-development-with-rancher-desktop) for instructions on how to build the image.
+
+Go back to the `values-local.yaml` file update it with the following content:
+
+```yaml filename="values-local.yaml"
+environment: "local"
+
+# Path will be mounted using rancher desktop to the /app path in the container
+source_code_path: "/absolute/path/to/your/serve"
+# https://helm.sh/docs/chart_template_guide/yaml_techniques/#yaml-anchors
+# ...
+studio:
+  # Append the following to the end of the studio section
+  
+  # We use pull policy Never because see the following link:
+  # https://github.com/rancher-sandbox/rancher-desktop/issues/952#issuecomment-993135128
+  static:
+    image: mystudio
+    pullPolicy: Never
+
+  image:
+    repository: mystudio
+    pullPolicy: Never
+
+  securityContext:
+    # Disables security context for local development
+    # Essentially allow the container to run as root
+    enabled: false
+
+  readinessProbe:
+    enabled: false
+
+  livenessProbe:
+    enabled: false 
 ```
 
-Depending on your storageclass, you might have to set this aswell. 
-For instance, if you use `microk8s`, them you run
+<details>
+  <summary>Full content of the values-local.yaml file</summary>
+  
+```yaml
+  environment: "local"
+  # Path will be mounted using rancher desktop to the /app path in the container
+  source_code_path: "/Users/nikch187/Projects/sll/serve"
+  # https://helm.sh/docs/chart_template_guide/yaml_techniques/#yaml-anchors
+  # for local development
+  storageClass: &storage_class local-path
+  #storage access mode
+  access_mode: &access_mode ReadWriteOnce
+  accessmode: *access_mode
+
+  global:
+    studio:
+      superuserPassword: "Test@12345"
+      superuserEmail: "[email protected]"
+      storageClass: *storage_class
+    postgresql:
+      storageClass: *storage_class
+
+  studio:
+    # Only locally on a debug environment
+    debug: true
+    storage:
+      storageClass: *storage_class
+    media:
+      storage:
+        storageClass: *storage_class
+        accessModes: *access_mode
+
+    # We use pull policy Never because see the following link:
+    # https://github.com/rancher-sandbox/rancher-desktop/issues/952#issuecomment-993135128
+    static:
+      image: mystudio
+      pullPolicy: Never
+
+    image:
+      repository: mystudio
+      pullPolicy: Never
+
+    securityContext:
+      # Disables security context for local development
+      # Essentially allow the container to run as root
+      enabled: false
+
+    readinessProbe:
+      enabled: false
+
+    livenessProbe:
+      enabled: false
+
+  postgresql:
+    primary:
+      persistence:
+        storageClass: *storage_class
+        accessModes:
+          - *access_mode
+  ```
+
+</details>
+
+After doing this run the following command to upgrade the deployment:
 
+```bash
+helm upgrade serve . -f values.yaml -f values-local.yaml
 ```
-helm install --set global.postgresql.storageClass=microk8s-hostpath serve .
+
+Now you can proceed to [set up PyCharm](https://github.com/ScilifelabDataCentre/serve?tab=readme-ov-file#pycharm-setup)
+
+If you don't want to set up PyCharm, you can just run Django from the container.
+
+```bash
+$ kubectl get po
+# Get the name of the studio pod
+$ kubectl exec -it <studio-pod-name> -- /bin/bash
+# Now you are inside the container 
+$ sh scripts/run_web.sh
 ```
 
-All resources will by default be created in the default namespace.
-Serve will be avaliable at https://studio.127.0.0.1.nip.io
-Obs that you might have to make changes to your particular ingress controller (nginx is supported in this chart) to connect to the URL.
-If the ingress does not work for any reason, you can try to port-forward the studio service port to your localhost. 
+Please note, that the folder you are in, `/app`, is the folder where the code is mounted.
 
+It means that you can make changes to the code on your host machine and see the changes in the container.
 
 ## Deploy an SSL certificate
 
@@ -84,10 +375,10 @@ studio:
   inactive_users: false #Users that sign-up can be inactive by default if desired
   csrf_trusted_origins: "https://studio.127.0.0.1.nip.io:8082" #extra trusted origin for django server, for example if you port-forward to port 8082
   image: # using a local image registry with hostname k3d-registry
-    repository: k3d-registry:35187/stackn:develop #This image can be built from Dockerfile (https://github.com/scaleoutsystems/stackn)
+    repository: k3d-registry:35187/serve:develop #This image can be built from Dockerfile (https://github.com/scaleoutsystems/serve)
     pullPolicy: Always # used to ensure that each time we redeploy always pull the latest image
   static:
-    image: k3d-registry:35187/stackn-nginx:develop #This image can be built from Dockerfile.nginx (https://github.com/scaleoutsystems/stackn)
+    image: k3d-registry:35187/serve-nginx:develop #This image can be built from Dockerfile.nginx (https://github.com/scaleoutsystems/serve)
   media:
     storage:
       accessModes: ReadWriteOnce

diff --git a/apps/custom-app/templates/deployment.yaml b/apps/custom-app/templates/deployment.yaml
@@ -28,10 +28,6 @@ spec:
         type: app
         pod: {{ .Values.appname }}
     spec:
-      {{- with .Values.imagePullSecrets }}
-      imagePullSecrets:
-        {{- toYaml . | nindent 8 }}
-      {{- end }}
       automountServiceAccountToken: false
       securityContext:
         seccompProfile:

diff --git a/apps/custom-app/values.yaml b/apps/custom-app/values.yaml
@@ -22,9 +22,6 @@ service:
   name: customapp-svc
   port: 80
 
-imagePullSecrets:
-  - name: regcred
-
 ingress:
   secretName: prod-ingress
 

diff --git a/apps/dash/templates/deployment.yaml b/apps/dash/templates/deployment.yaml
@@ -28,10 +28,6 @@ spec:
         type: app
         pod: {{ .Values.appname }}
     spec:
-    {{- with .Values.imagePullSecrets }}
-      imagePullSecrets:
-        {{- toYaml . | nindent 8 }}
-    {{- end }}
       automountServiceAccountToken: false
       securityContext:
         {{- toYaml .Values.podSecurityContext | nindent 8 }}

diff --git a/apps/dash/values.yaml b/apps/dash/values.yaml
@@ -18,9 +18,6 @@ service:
   name: dashapp-svc
   port: 80
 
-imagePullSecrets:
-  - name: regcred
-
 ingress:
   secretName: prod-ingress
 

diff --git a/apps/jupyter-lab/templates/deployment.yaml b/apps/jupyter-lab/templates/deployment.yaml
@@ -81,10 +81,6 @@ spec:
         - name: {{ $key }}
           mountPath: /home/jovyan/work/{{ $key }}
         {{- end }}
-    {{- with .Values.imagePullSecrets }}
-      imagePullSecrets:
-        {{- toYaml . | nindent 8 }}
-    {{- end }}
       terminationGracePeriodSeconds: 30
       dnsPolicy: ClusterFirst
       volumes: