Sample-Setup for using the RKE2-ClusterAPI Provider on Hetzner Cloud
Install the clusterctl
cli. See Install ClusterCTL
curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v1.7.1/clusterctl-linux-amd64 -o clusterctl
sudo install -o root -g root -m 0755 clusterctl /usr/local/bin/clusterctl
clusterctl version
We will use a Kind-Cluster as Managment Cluster
# Create Kind-Cluster for local managment
sudo kind create cluster --image kindest/node:v1.29.2
export CAPH_MGT_CLUSTER_KUBECONFIG=/tmp/mgt-kubeconfig
kind get kubeconfig > $CAPH_MGT_CLUSTER_KUBECONFIG
export KUBECONFIG=$CAPH_MGT_CLUSTER_KUBECONFIG
Setup Environment Variables
export HCLOUD_CONTROL_PLANE_MACHINE_TYPE=cpx31
export HCLOUD_WORKER_MACHINE_TYPE=cpx31
export HCLOUD_SSH_KEY="lraus-cka_sshkey"
export HCLOUD_REGION="nbg1"
We need to prepare our environment variables for cluster template generation. We will be using the Hetzner CAPI-Provder from Syself to provision the inital cluster with kubeadm
.
# Setup syself/hetzner clusterAPI Provider
clusterctl init --core cluster-api --bootstrap kubeadm --control-plane kubeadm --infrastructure hetzner
The following componentes will be installed on the managment cluster
Fetching providers
Installing cert-manager Version="v1.12.2"
Waiting for cert-manager to be available...
Installing Provider="cluster-api" Version="v1.4.4" TargetNamespace="capi-system"
Installing Provider="bootstrap-kubeadm" Version="v1.4.4" TargetNamespace="capi-kubeadm-bootstrap-system"
Installing Provider="control-plane-kubeadm" Version="v1.4.4" TargetNamespace="capi-kubeadm-control-plane-system"
Installing Provider="infrastructure-hetzner" Version="v1.0.0-beta.18" TargetNamespace="caph-system"
Export the configuration variables for the cluster
source ./env-vars/kubeadm.rc
Then generate the cluster with clusterctl
clusterctl generate cluster hetzner-capi-demo --control-plane-machine-count=1 --worker-machine-count=2 > hetzner-capi-kubeadm-demo.yaml
k apply -f kubeadm-cluster.yaml
k get clusters
export CAPH_WORKER_CLUSTER_KUBECONFIG=/tmp/workload-kubeconfig
clusterctl get kubeconfig hetzner-capi-demo > $CAPH_WORKER_CLUSTER_KUBECONFIG
export KUBECONFIG=$CAPH_WORKER_CLUSTER_KUBECONFIG
Now we need to deploy the CNI
helm repo add cilium https://helm.cilium.io/
helm upgrade --install cilium cilium/cilium --namespace kube-system -f templates/cilium.yaml
We will be using the experimental ClusterAPI Provider cluster-api-provider-rke2 from Rancher. We switch our Bootstrap Provider to RKE2 and Control-Plane to RKE2. We still use the SysElf Provider for the infrastructure provisoning. Make sure this is run on the CAPI-Managment-Cluster.
# We need to remove the kubeadm provider befor installing rke2.
clusterctl delete --bootstrap kubeadm --control-plane kubeadm
# Setup syself/hetzner clusterAPI Provider
clusterctl init --bootstrap rke2 --control-plane rke2 --infrastructure hetzner
Setup Environment Variables and create Hetzner API-Key as secret $HCLOUD_TOKEN
should already be exposed!
kubectl create secret generic hcloud --from-literal="token=$HCLOUD_TOKEN"
source ./env-vars/rke2.rc
# Generate the cluster
clusterctl generate cluster hetzner-capi-rke2-demo --from capi-conf-templates/rke2-online.yaml > hetzner-capi-rke2-demo.yaml
k apply -f hetzner-capi-rke2-demo.yaml
The cluster will now be provisoned. After this, you can access the Kubeconfig of our new Workload Cluster via clusterctl
clusterctl get kubeconfig hetzner-capi-rke2-demo > /tmp/workload-kubeconfig
export KUBECONFIG=/tmp/workload-kubeconfig
For the nodes to get into a "Ready"-State, we will need to install the "Hetzner-Cloud-Controller". See HetznerCloud Cloud Controller Manager for more information.
kubectl -n kube-system create secret generic hcloud --from-literal="token=$HCLOUD_TOKEN"
helm repo add hcloud https://charts.hetzner.cloud
helm repo update hcloud
helm install hccm hcloud/hcloud-cloud-controller-manager -n kube-system
First you will need to build an Ubuntu Image with snapd
installed. Our image will be called `microk8s-ubuntu-22.04-2023-07-21-1830``and can then be referenced in the clusterapi-template.
packer build templates/node-image/microk8s-image/image.json
Export the environment variables and generate you're Cluster Template from the configuration
source env-vars/microk8s.rc
clusterctl generate cluster hetzner-microk8s-demo --from ./capi-conf-templates/microk8s.yaml > hetzner-microk8s-demo.yaml
Export Kubernetes Config
export CAPH_WORKER2_CLUSTER_KUBECONFIG=/tmp/workload1-kubeconfig
clusterctl get kubeconfig hetzner-mikrok8s-demo > $CAPH_WORKER2_CLUSTER_KUBECONFIG
export KUBECONFIG=$CAPH_WORKER2_CLUSTER_KUBECONFIG
# Install Ingress Controller
helm upgrade --install ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx --create-namespace
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
kubectl create namespace cattle-system
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.0/cert-manager.yaml
helm install rancher rancher-latest/rancher --namespace cattle-system --set hostname=rancher.my.org --set bootstrapPassword=Leon123!