Skip to content

Latest commit

 

History

History
186 lines (148 loc) · 5.2 KB

install.md

File metadata and controls

186 lines (148 loc) · 5.2 KB
Raw

Install plugin

This plugin allows Kubernetes to use Proxmox VE storage as a persistent storage solution for stateful applications. Supported storage types:

  • Directory
  • LVM
  • LVM-thin
  • ZFS
  • NFS
  • Ceph

Proxmox configuration

Proxmox CSI Plugin requires the correct privileges in order to allocate and attach disks.

Create CSI role in Proxmox:

pveum role add CSI -privs "VM.Audit VM.Config.Disk Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"

Next create a user kubernetes-csi@pve for the CSI plugin and grant it the above role

pveum user add kubernetes-csi@pve
pveum aclmod / -user kubernetes-csi@pve -role CSI
pveum user token add kubernetes-csi@pve csi -privsep 0

All VMs in the cluster must have the SCSI Controller set to VirtIO SCSI single or VirtIO SCSI type to be able to attach disks.

Install CSI Driver

Create a namespace csi-proxmox for the plugin and grant it the privileged permissions

kubectl create ns csi-proxmox
kubectl label ns csi-proxmox pod-security.kubernetes.io/enforce=privileged

Install the plugin by using kubectl

Create a Proxmox cloud config to connect to your cluster with the Proxmox user you just created

# config.yaml
clusters:
  - url: https://cluster-api-1.exmple.com:8006/api2/json
    insecure: false
    token_id: "kubernetes-csi@pve!csi"
    token_secret: "secret"
    region: Region-1
  # Add more clusters if needed
  - url: https://cluster-api-2.exmple.com:8006/api2/json
    insecure: false
    token_id: "kubernetes-csi@pve!csi"
    token_secret: "secret"
    region: Region-2

Upload the configuration to the Kubernetes as a secret

kubectl -n csi-proxmox create secret generic proxmox-csi-plugin --from-file=config.yaml

Install latest release version

kubectl apply -f https://raw.githubusercontent.com/sergelogvinov/proxmox-csi-plugin/main/docs/deploy/proxmox-csi-plugin-release.yml

Or install latest stable version (edge)

kubectl apply -f https://raw.githubusercontent.com/sergelogvinov/proxmox-csi-plugin/main/docs/deploy/proxmox-csi-plugin.yml

Install the plugin by using Helm

Create the helm values file, for more information see values.yaml

# proxmox-csi.yaml
config:
  clusters:
    - url: https://cluster-api-1.exmple.com:8006/api2/json
      insecure: false
      token_id: "kubernetes-csi@pve!csi"
      token_secret: "secret"
      region: Region-1
    # Add more clusters if needed
    - url: https://cluster-api-2.exmple.com:8006/api2/json
      insecure: false
      token_id: "kubernetes-csi@pve!csi"
      token_secret: "secret"
      region: Region-2

# Define the storage classes
storageClass:
  - name: proxmox-data-xfs
    storage: data
    reclaimPolicy: Delete
    fstype: xfs

Install the plugin. You need to prepare the csi-proxmox namespace first, see above

helm upgrade -i -n csi-proxmox -f proxmox-csi.yaml proxmox-csi-plugin oci://ghcr.io/sergelogvinov/charts/proxmox-csi-plugin

Option for k0s

If you're running k0s you need to add extra value to the helm chart

kubeletDir: /var/lib/k0s/kubelet

Option for microk8s

If you're running microk8s you need to add extra value to the helm chart

kubeletDir: /var/snap/microk8s/common/var/lib/kubelet

Install the plugin by using Talos machine config

If you're running Talos you can install Proxmox CSI plugin using the machine config

cluster:
  externalCloudProvider:
    enabled: true
    manifests:
      - https://raw.githubusercontent.com/sergelogvinov/proxmox-csi-plugin/main/docs/deploy/proxmox-csi-plugin.yml

Or all together with the Proxmox Cloud Controller Manager

  • Proxmox CCM will label the nodes
  • Proxmox CSI will use the labeled nodes to define the regions and zones
cluster:
  inlineManifests:
    - name: proxmox-cloud-controller-manager
      contents: |-
        apiVersion: v1
        kind: Secret
        type: Opaque
        metadata:
          name: proxmox-cloud-controller-manager
          namespace: kube-system
        stringData:
          config.yaml: |
            clusters:
              - url: https://cluster-api-1.exmple.com:8006/api2/json
                insecure: false
                token_id: "kubernetes-csi@pve!ccm"
                token_secret: "secret"
                region: Region-1
    - name: proxmox-csi-plugin
      contents: |-
        apiVersion: v1
        kind: Secret
        type: Opaque
        metadata:
          name: proxmox-csi-plugin
          namespace: csi-proxmox
        stringData:
          config.yaml: |
            clusters:
              - url: https://cluster-api-1.exmple.com:8006/api2/json
                insecure: false
                token_id: "kubernetes-csi@pve!csi"
                token_secret: "secret"
                region: Region-1
  externalCloudProvider:
    enabled: true
    manifests:
      - https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/docs/deploy/cloud-controller-manager.yml
      - https://raw.githubusercontent.com/sergelogvinov/proxmox-csi-plugin/main/docs/deploy/proxmox-csi-plugin.yml