kubeadm: enable join dryrun e2e tests #3116

SataQiu · 2024-10-26T08:24:33Z

kubeadm: enable join dryrun e2e tests without a real control plane

Ref: #2653

SataQiu · 2024-10-26T08:29:48Z

/hold for review

pacoxu · 2024-10-28T05:13:45Z

/lgtm

carlory · 2024-10-28T06:21:33Z

/lgtm

neolit123

i had this PR planned but i could not get to it. so thanks for sending it.

i think we should do the following:

rename init-dry -> dry-run-without-cluster
perform the required_version check there
if versions is 1.32, run all these before a cluster is created:

init dry run
join control plane dry run
join worker (too maybe?)
upgrade apply dry run
upgrade node dry run
reset dry run

then continue testing the commands after the init node was created as it is now.

neolit123 · 2024-10-28T06:59:51Z

kinder/ci/tools/update-workflows/templates/workflows/dryrun-tasks.yaml

@@ -60,6 +60,22 @@ tasks:
      docker exec {{ .vars.clusterName }}-control-plane-1 kubeadm init --ignore-preflight-errors={{ .vars.kubeadmIgnorePreflightErrors }} --kubernetes-version={{ .vars.kubernetesVersion }} --dry-run=true --v={{ .vars.kubeadmVerbosity }} || exit 1
      docker exec {{ .vars.clusterName }}-control-plane-1 kubeadm init --ignore-preflight-errors={{ .vars.kubeadmIgnorePreflightErrors }} --kubernetes-version={{ .vars.kubernetesVersion }} --upload-certs=true --dry-run=true --v={{ .vars.kubeadmVerbosity }} || exit 1


i think the upload certs is no longer required? (or we can only call this variant)

neolit123 · 2024-10-28T07:01:05Z

kinder/ci/tools/update-workflows/templates/workflows/dryrun-tasks.yaml

+      required_version="1.32" # join dry-run without real control plane is available since v1.32
+      current_version=$(echo "{{ .vars.kubernetesVersion }}" | grep -oP '\d+\.\d+')
+      if awk "BEGIN {exit !($current_version < $required_version)}"; then
+        echo "Join dry-run without real control plane is not available for Kubernetes version {{ .vars.kubernetesVersion }}. Skipping."
+        exit 0
+      fi


i like this check. it avoids us having to write updates to the kinder config (new params) or adding more workflow files.

SataQiu · 2024-10-30T12:18:35Z

kubeadm upgrade dryrun without cluster is still not supported yet.

# kubeadm upgrade apply -f v1.32.0-alpha.3.47+daef8c2419a638 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables --allow-release-candidate-upgrades=true --allow-experimental-upgrades=true --dry-run --v=6

open /etc/kubernetes/admin.conf: no such file or directory
failed to load kubeconfig
k8s.io/kubernetes/cmd/kubeadm/app/util/apiclient.(*DryRun).WithKubeConfigFile
        k8s.io/kubernetes/cmd/kubeadm/app/util/apiclient/dryrun.go:79
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.getClient
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/common.go:195
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.newApplyData
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/apply.go:226
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.newCmdApply.func2
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/apply.go:146
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).InitData
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:185
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.newCmdApply.func1
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/apply.go:99
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/[email protected]/command.go:985
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/[email protected]/command.go:1117
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/[email protected]/command.go:1041
k8s.io/kubernetes/cmd/kubeadm/app.Run
        k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:47
main.main
        k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
runtime.main
        runtime/proc.go:272
runtime.goexit
        runtime/asm_amd64.s:1700
couldn't create a Kubernetes client from file "/etc/kubernetes/admin.conf"
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.newApplyData
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/apply.go:228
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.newCmdApply.func2
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/apply.go:146
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).InitData
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow/runner.go:185
k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade.newCmdApply.func1
        k8s.io/kubernetes/cmd/kubeadm/app/cmd/upgrade/apply.go:99
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/[email protected]/command.go:985
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/[email protected]/command.go:1117
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/[email protected]/command.go:1041
k8s.io/kubernetes/cmd/kubeadm/app.Run
        k8s.io/kubernetes/cmd/kubeadm/app/kubeadm.go:47
main.main
        k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
runtime.main
        runtime/proc.go:272
runtime.goexit
        runtime/asm_amd64.s:1700

neolit123 · 2024-10-30T12:49:39Z

kubeadm upgrade dryrun without cluster is still not supported yet.

i think it used to work. perhaps we broke it with upgrade apply phases.
checking..

neolit123 · 2024-10-30T13:00:23Z

kubeadm upgrade dryrun without cluster is still not supported yet.

i think it used to work. perhaps we broke it with upgrade apply phases. checking..

no, seems like upgrade does need the admin.conf
https://github.com/kubernetes/kubernetes/pull/126776/files#diff-657870bffd956e3e3553db7dbb5cb725ea53922da959fcf869c2e31880df72c7R195

but i can try to make it work without it.

neolit123 · 2024-10-30T19:01:09Z

kubeadm upgrade dryrun without cluster is still not supported yet.

i think it used to work. perhaps we broke it with upgrade apply phases. checking..

no, seems like upgrade does need the admin.conf https://github.com/kubernetes/kubernetes/pull/126776/files#diff-657870bffd956e3e3553db7dbb5cb725ea53922da959fcf869c2e31880df72c7R195

but i can try to make it work without it

kubeadm: support dryrunning upgrade without a real cluster kubernetes#128457

SataQiu · 2024-11-13T08:53:42Z

this is ready for more review, I think we can merge it now.
@neolit123

neolit123

LGTM, two minor nits.

neolit123 · 2024-11-13T10:17:21Z

kinder/ci/tools/update-workflows/templates/workflows/dryrun-tasks.yaml

+      set -x
+      required_version="1.32" # dryrun without cluster is available since v1.32
+      current_version=$(echo "{{ .vars.kubernetesVersion }}" | grep -oP 'v\K\d+\.\d+')
+      if awk "BEGIN {exit !($current_version < $required_version)}"; then


i think this would work too

Suggested change

if awk "BEGIN {exit !($current_version < $required_version)}"; then

if [ "$current_version" \< "$required_version" ]; then

tested in a local bash.

#!/bin/bash required_version="1.32" current_version=$(echo "v1.34.0" | grep -oP 'v\K\d+\.\d+') if [ "$current_version" \< "$required_version" ]; then echo "less" else echo "supported" fi

neolit123 · 2024-11-13T10:18:25Z

kinder/ci/tools/update-workflows/templates/workflows/dryrun-tasks.yaml

+        echo "Full dryrun support without cluster is not available for Kubernetes version {{ .vars.kubernetesVersion }}, only init-dryrun will be executed for this task."
+        docker exec {{ .vars.clusterName }}-control-plane-1 bash -c "until crictl ps &> /dev/null; do echo 'Waiting for the container runtime to be running ...'; sleep 1; done"
+        docker exec {{ .vars.clusterName }}-control-plane-1 kubeadm init --ignore-preflight-errors={{ .vars.kubeadmIgnorePreflightErrors }} --kubernetes-version={{ .vars.kubernetesVersion }} --dry-run --v={{ .vars.kubeadmVerbosity }} || exit 1
+      else


Suggested change

else

exit 0

fi

just so that we don't indent the else part.
also added one extra line after the fi.

neolit123

/lgtm
/apporove
thanks @SataQiu

k8s-ci-robot · 2024-11-13T12:33:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: carlory, neolit123, pacoxu, SataQiu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [SataQiu,neolit123,pacoxu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

SataQiu · 2024-11-13T14:15:03Z

/hold cancel

Let's see if it works

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 26, 2024

k8s-ci-robot requested a review from carlory October 26, 2024 08:24

k8s-ci-robot added the area/kinder Issues to track work in the kinder tool label Oct 26, 2024

k8s-ci-robot requested a review from neolit123 October 26, 2024 08:24

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 26, 2024

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 26, 2024

k8s-ci-robot assigned pacoxu Oct 28, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 28, 2024

k8s-ci-robot assigned carlory Oct 28, 2024

neolit123 reviewed Oct 28, 2024

View reviewed changes

SataQiu force-pushed the join-dryrun-20241026 branch from 8994f1a to 40b5550 Compare October 30, 2024 11:47

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 30, 2024

SataQiu force-pushed the join-dryrun-20241026 branch from 40b5550 to c6c1db0 Compare October 30, 2024 11:48

SataQiu force-pushed the join-dryrun-20241026 branch 2 times, most recently from ed957ec to b6fd229 Compare October 30, 2024 12:22

SataQiu changed the title ~~kubeadm: enable join dryrun e2e tests~~ [WIP]kubeadm: enable join dryrun e2e tests Oct 30, 2024

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 30, 2024

neolit123 mentioned this pull request Oct 30, 2024

kubeadm: support dryrunning upgrade without a real cluster kubernetes/kubernetes#128457

Merged

SataQiu changed the title ~~[WIP]kubeadm: enable join dryrun e2e tests~~ kubeadm: enable join dryrun e2e tests Nov 13, 2024

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 13, 2024

neolit123 reviewed Nov 13, 2024

View reviewed changes

add dryrun task without real cluster

1d8547f

SataQiu force-pushed the join-dryrun-20241026 branch from b6fd229 to 1d8547f Compare November 13, 2024 12:28

neolit123 reviewed Nov 13, 2024

View reviewed changes

k8s-ci-robot assigned neolit123 Nov 13, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 13, 2024

neolit123 mentioned this pull request Nov 13, 2024

add dry run e2e tests #2653

Closed

3 tasks

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 13, 2024

k8s-ci-robot merged commit 3a2bb82 into kubernetes:main Nov 13, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kubeadm: enable join dryrun e2e tests #3116

kubeadm: enable join dryrun e2e tests #3116

SataQiu commented Oct 26, 2024 •

edited

Loading

SataQiu commented Oct 26, 2024

pacoxu commented Oct 28, 2024

carlory commented Oct 28, 2024

neolit123 left a comment •

edited

Loading

neolit123 Oct 28, 2024 •

edited

Loading

neolit123 Oct 28, 2024

SataQiu commented Oct 30, 2024 •

edited

Loading

neolit123 commented Oct 30, 2024

neolit123 commented Oct 30, 2024

neolit123 commented Oct 30, 2024

SataQiu commented Nov 13, 2024

neolit123 left a comment

neolit123 Nov 13, 2024

neolit123 Nov 13, 2024

neolit123 left a comment

k8s-ci-robot commented Nov 13, 2024

SataQiu commented Nov 13, 2024

		@@ -60,6 +60,22 @@ tasks:
		docker exec {{ .vars.clusterName }}-control-plane-1 kubeadm init --ignore-preflight-errors={{ .vars.kubeadmIgnorePreflightErrors }} --kubernetes-version={{ .vars.kubernetesVersion }} --dry-run=true --v={{ .vars.kubeadmVerbosity }} \|\| exit 1
		docker exec {{ .vars.clusterName }}-control-plane-1 kubeadm init --ignore-preflight-errors={{ .vars.kubeadmIgnorePreflightErrors }} --kubernetes-version={{ .vars.kubernetesVersion }} --upload-certs=true --dry-run=true --v={{ .vars.kubeadmVerbosity }} \|\| exit 1

	if awk "BEGIN {exit !($current_version < $required_version)}"; then
	if [ "$current_version" \< "$required_version" ]; then

-      else
+            exit 0
+      fi

kubeadm: enable join dryrun e2e tests #3116

kubeadm: enable join dryrun e2e tests #3116

Conversation

SataQiu commented Oct 26, 2024 • edited Loading

SataQiu commented Oct 26, 2024

pacoxu commented Oct 28, 2024

carlory commented Oct 28, 2024

neolit123 left a comment • edited Loading

Choose a reason for hiding this comment

neolit123 Oct 28, 2024 • edited Loading

Choose a reason for hiding this comment

neolit123 Oct 28, 2024

Choose a reason for hiding this comment

SataQiu commented Oct 30, 2024 • edited Loading

neolit123 commented Oct 30, 2024

neolit123 commented Oct 30, 2024

neolit123 commented Oct 30, 2024

SataQiu commented Nov 13, 2024

neolit123 left a comment

Choose a reason for hiding this comment

neolit123 Nov 13, 2024

Choose a reason for hiding this comment

neolit123 Nov 13, 2024

Choose a reason for hiding this comment

neolit123 left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Nov 13, 2024

SataQiu commented Nov 13, 2024

SataQiu commented Oct 26, 2024 •

edited

Loading

neolit123 left a comment •

edited

Loading

neolit123 Oct 28, 2024 •

edited

Loading

SataQiu commented Oct 30, 2024 •

edited

Loading