Add local dev env via skaffold

- create new dev overlay - create skaffold yaml - add tasks to create/clean local k8s dev env - update README Signed-off-by: adrianc <[email protected]>
Mellanox · Aug 15, 2023 · 8eb513f · 8eb513f
1 parent 81612a0
commit 8eb513f
Show file tree

Hide file tree

Showing 5 changed files with 86 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -19,6 +19,10 @@
   - [Build and Run Locally](#build-and-run-locally)
     - [Prerequisites](#prerequisites-1)
     - [Build \& Run](#build--run)
+    - [Build \& Run in local K8s cluster](#build--run-in-local-k8s-cluster)
+      - [Install Prerequisites](#install-prerequisites)
+      - [Create local k8s development environment](#create-local-k8s-development-environment)
+      - [Cleanup local k8s development environment](#cleanup-local-k8s-development-environment)
 
 ## Overview
 
@@ -192,3 +196,27 @@ task build
 
 > __Note__: To build container image run `task image:build`. to deploy this image in your k8 cluster, you should re-tag and upload to your
 > own image registry, then create an overlay for exsiting deployment which overrides the image name with your own image path.
+
+
+### Build & Run in local K8s cluster
+
+For local development it is possible to use [skaffold](https://skaffold.dev/) to deploy
+local changes onto a local k8s cluster.
+
+#### Install Prerequisites
+
+1. [install Docker](https://docs.docker.com/engine/install/)
+2. [install minikube](https://minikube.sigs.k8s.io/docs/start/)
+3. [install skaffold](https://skaffold.dev/docs/install/#standalone-binary)
+
+#### Create local k8s development environment
+
+```shell
+task -o interleaved localdev:start
+```
+
+#### Cleanup local k8s development environment
+
+```shell
+task -o interleaved localdev:clean
+```
diff --git a/Taskfile.dist.yaml b/Taskfile.dist.yaml
@@ -30,6 +30,7 @@ includes:
       BIN_DIR: "{{.LOCAL_BIN}}"
   kind: ./taskfiles/Kind.yaml
   image: ./taskfiles/Image.yaml
+  localdev: ./taskfiles/LocalDev.yaml
 
 tasks:
   clean:

diff --git a/deployment/k8s/overlays/dev/kustomization.yaml b/deployment/k8s/overlays/dev/kustomization.yaml
@@ -0,0 +1,18 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: nic-feature-discovery
+
+patches:
+  - target:
+      version: v1
+      group: apps
+      kind: DaemonSet
+      name: nic-feature-discovery-ds
+    patch: |-
+      - op: replace
+        path:  /spec/template/spec/containers/0/args
+        value: ["--v=5", "--logging-format=text"]
+
+resources:
+- ../default
diff --git a/skaffold.yaml b/skaffold.yaml
@@ -0,0 +1,21 @@
+apiVersion: skaffold/v4beta6
+kind: Config
+metadata:
+  name: nic-feature-discovery
+build:
+  artifacts:
+    - image: ghcr.io/mellanox/nic-feature-discovery
+      docker:
+        dockerfile: Dockerfile
+manifests:
+  kustomize:
+    paths:
+      - deployment/k8s/base
+profiles:
+  - name: local-dev
+    manifests:
+      kustomize:
+        paths:
+          - deployment/k8s/overlays/dev
+    activation:
+      - command: dev
diff --git a/taskfiles/LocalDev.yaml b/taskfiles/LocalDev.yaml
@@ -0,0 +1,18 @@
+  version: '3'
+
+  tasks:   
+    start:
+      desc: deploy using skaffold to k8s using current kube context (run with -o interleaved for interactive output)
+      vars:
+        NFD_VERSION: v0.13.3
+      cmd: |
+        minikube --profile nfd-dev start
+        # this one will point docker cli to the same daemon as used by minikube nfd-dev cluster
+        # that way skaffold will build images in the correct local repo.
+        eval $(minikube -p nfd-dev docker-env)
+        kubectl apply -k "https://github.com/kubernetes-sigs/node-feature-discovery/deployment/overlays/default?ref={{.NFD_VERSION}}"
+        skaffold dev
+    
+    clean:
+      desc: clean local k8s dev cluster
+      cmd: minikube --profile nfd-dev delete