Switch from in tree to csi #448

ITViking · 2024-09-06T10:20:01Z

What does this PR do?

These are the scripts we have used to move to the new CSI

Should this be tested by the reviewer and how?

no

Any specific requests for how the PR should be reviewed?

read it

What are the relevant tickets?

https://reload.atlassian.net/jira/software/c/projects/DDFDRIFT/boards/464?search=csi&selectedIssue=DDFDRIFT-189

… to developers and operators

…tches the nginx deployment to use it

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

shellcheck

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 14 in 952de06

    
           kubectl get pvc -n $NAMESPACE $PVC_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${PVC_NAME}.yaml"

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 15 in 952de06

    
           kubectl get pv $VOLUME_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${VOLUME_NAME}"

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 17 in 952de06

kubectl delete pvc -n $NAMESPACE $PVC_NAME --wait=false

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 17 in 952de06

kubectl delete pvc -n $NAMESPACE $PVC_NAME --wait=false

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 18 in 952de06

kubectl patch pvc -n $NAMESPACE $PVC_NAME -p '{"metadata":{"finalizers":null}}'

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 18 in 952de06

kubectl patch pvc -n $NAMESPACE $PVC_NAME -p '{"metadata":{"finalizers":null}}'

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 20 in 952de06

kubectl delete pv $VOLUME_NAME --grace-period=0 --wait=false

📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

dpl-platform/infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

Line 21 in 952de06

kubectl patch pv $VOLUME_NAME -p '{"metadata":{"finalizers":null}}'

github-actions · 2024-09-06T10:20:21Z

infrastructure/dpladm/bin/csi-migration-scripts/make-tmp-pv-and-pvc-for-namespace.sh

+NAMESPACES=$(kubectl get ns -l lagoon.sh/controller=lagoon --no-headers | awk '{print $1}' | grep main)
+VOLUMEHANDLE_PREFIX="/subscriptions/8ac8a259-5bb3-4799-bd1e-455145b12550/resourceGroups/rg-env-dplplat01/providers/Microsoft.Storage/storageAccounts/stdpldplplat01585708af/"
+for ns in $NAMESPACES; do
+  echo $ns


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:21Z

infrastructure/dpladm/bin/csi-migration-scripts/make-tmp-pv-and-pvc-for-namespace.sh

+for ns in $NAMESPACES; do
+  echo $ns
+  # Get pvc variable
+  SHARE_NAME=$(kubectl get pvc -n $ns tmp-nginx | grep pvc | awk '{print $3}' | cut -c 5- )


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:21Z

infrastructure/dpladm/bin/csi-migration-scripts/make-tmp-pv-and-pvc-for-namespace.sh

+  SHARE_NAME=$(kubectl get pvc -n $ns tmp-nginx | grep pvc | awk '{print $3}' | cut -c 5- )
+  # echo $SHARE_NAME
+
+  PROJECT_NAME=$(echo $ns | rev | cut -c 6- | rev)


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:22Z

infrastructure/dpladm/bin/csi-migration-scripts/make-tmp-pv-and-pvc-for-namespace.sh

+  kubectl apply -f pv.yaml
+  kubectl apply -f pvc.yaml
+
+  kubectl patch deployments.apps -n $ns cli -p '{"spec":{"template":{"spec": {"volumes": [{"name": "nginx", "persistentVolumeClaim": { "claimName": "nginx"}}]}}}}'


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:22Z

infrastructure/dpladm/bin/csi-migration-scripts/make-tmp-pv-and-pvc-for-namespace.sh

+  kubectl apply -f pvc.yaml
+
+  kubectl patch deployments.apps -n $ns cli -p '{"spec":{"template":{"spec": {"volumes": [{"name": "nginx", "persistentVolumeClaim": { "claimName": "nginx"}}]}}}}'
+  kubectl patch deployments.apps -n $ns nginx -p '{"spec":{"template":{"spec": {"volumes": [{"name": "nginx", "persistentVolumeClaim": { "claimName": "nginx"}}]}}}}'


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:23Z

infrastructure/dpladm/bin/csi-migration-scripts/create-pv-and-pvc-for-namespace.sh

+# Change it's name slighty so we can recognize it from the old ones
+NEW="new-"
+NEW_VOLUME_NAME=${NEW}${VOLUME_NAME}
+echo $NEW_VOLUME_NAME


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:23Z

infrastructure/dpladm/bin/csi-migration-scripts/create-pv-and-pvc-for-namespace.sh

+kubectl apply -f pvc.yaml
+
+# Switch the nginx deployments nginx volume to use the new PVC
+kubectl patch deployments.apps -n $1 nginx -p '{"spec":{"template":{"spec": {"volumes": [{"name": "nginx", "persistentVolumeClaim": { "claimName": "new-nginx"}}]}}}}'


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:23Z

infrastructure/dpladm/bin/csi-migration-scripts/create-pv-and-pvc-for-namespace.sh

+
+echo "Proceeding to remove the now obsolete PV and PVC from the namespace $1"
+
+backupAndDeleteOldPvAndPvc $1 $VOLUME_NAME "nginx"


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:23Z

infrastructure/dpladm/bin/csi-migration-scripts/create-pv-and-pvc-for-namespace.sh

+
+echo "Proceeding to remove the now obsolete PV and PVC from the namespace $1"
+
+backupAndDeleteOldPvAndPvc $1 $VOLUME_NAME "nginx"


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:24Z

infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

+  local PVC_NAME=$3
+
+  # Backup the the old PVC and PV before deleting them
+  kubectl get pvc -n $NAMESPACE $PVC_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${PVC_NAME}.yaml"


📝 [shellcheck] _{reported by reviewdog 🐶}
Double quote to prevent globbing and word splitting. SC2086

github-actions · 2024-09-06T10:20:29Z

infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

+  kubectl get pvc -n $NAMESPACE $PVC_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${PVC_NAME}.yaml"
+  kubectl get pv $VOLUME_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${VOLUME_NAME}"


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

kubectl get pvc -n $NAMESPACE $PVC_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${PVC_NAME}.yaml"

kubectl get pv $VOLUME_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${VOLUME_NAME}"

kubectl get pvc -n "$NAMESPACE" "$PVC_NAME" -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${PVC_NAME}.yaml"

kubectl get pv "$VOLUME_NAME" -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${VOLUME_NAME}"

github-actions · 2024-09-06T10:20:29Z

infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

+  kubectl delete pvc -n $NAMESPACE $PVC_NAME --wait=false
+  kubectl patch pvc -n $NAMESPACE $PVC_NAME -p '{"metadata":{"finalizers":null}}'


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

kubectl delete pvc -n $NAMESPACE $PVC_NAME --wait=false

kubectl patch pvc -n $NAMESPACE $PVC_NAME -p '{"metadata":{"finalizers":null}}'

kubectl delete pvc -n "$NAMESPACE" "$PVC_NAME" --wait=false

kubectl patch pvc -n "$NAMESPACE" "$PVC_NAME" -p '{"metadata":{"finalizers":null}}'

github-actions · 2024-09-06T10:20:29Z

infrastructure/dpladm/bin/csi-migration-scripts/deleteOldPvAndPvc.sh

+  kubectl delete pv $VOLUME_NAME  --grace-period=0 --wait=false
+  kubectl patch pv $VOLUME_NAME -p '{"metadata":{"finalizers":null}}'


[shellcheck (suggestion)] _{reported by reviewdog 🐶}

Suggested change

kubectl delete pv $VOLUME_NAME --grace-period=0 --wait=false

kubectl patch pv $VOLUME_NAME -p '{"metadata":{"finalizers":null}}'

kubectl delete pv "$VOLUME_NAME" --grace-period=0 --wait=false

kubectl patch pv "$VOLUME_NAME" -p '{"metadata":{"finalizers":null}}'

Philip added 23 commits September 6, 2024 12:17

add a staging site for the DDF people to test on and give canary back…

e9745c8

… to developers and operators

Intermediary SC for the new CSI pluging

4680b1d

new PV for using with CSI

e604855

new PVC for using with CSI

8e044b2

script that creates new PV and PVC, using CSI, for a namespace and pa…

42f5a78

…tches the nginx deployment to use it

patch the deployment to use the new PVC

2e37cab

check that the namespace exists before doing anything

153813a

clean up the old pvc and pv

896a4fd

make a backup before deletion

8e5965d

sharename was wrong

18c10a9

needed to be split as the names are not the same

8f3928b

point to the new one - we're done testing for now

a533fab

backups from sites the script has been run on

c74779b

update name of the new csi, so lagoon can find and use it

58b6a83

point to the new SC as the provider

a985d0d

changes after having run scripts

0677c42

we wont need this anymore

7cc04ab

script to copy files from nginx to cli pod in a namespace

e1ce7c8

script to delete obsolote pv's and pvc's

e7d0973

script to make temporary pv and pvc's for namespace

2fe8ba4

script to patch the nginx's in a namespace so they use the right storage

f659772

script to patch the metadata so lagoon will recognize the pvc's

7f3d807

clean up the infrastruture folder

952de06

ITViking requested a review from ath88 September 6, 2024 10:20

ITViking assigned ath88 Sep 6, 2024

github-actions bot reviewed Sep 6, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch from in tree to csi #448

Switch from in tree to csi #448

ITViking commented Sep 6, 2024

github-actions bot left a comment

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024

github-actions bot Sep 6, 2024


		echo "Proceeding to remove the now obsolete PV and PVC from the namespace $1"

		backupAndDeleteOldPvAndPvc $1 $VOLUME_NAME "nginx"

		kubectl get pvc -n $NAMESPACE $PVC_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${PVC_NAME}.yaml"
		kubectl get pv $VOLUME_NAME -o yaml > "./pvAndPvcBackup/${NAMESPACE}_${VOLUME_NAME}"

Switch from in tree to csi #448

Are you sure you want to change the base?

Switch from in tree to csi #448

Conversation

ITViking commented Sep 6, 2024

What does this PR do?

Should this be tested by the reviewer and how?

Any specific requests for how the PR should be reviewed?

What are the relevant tickets?

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment

github-actions bot Sep 6, 2024

Choose a reason for hiding this comment