Integrate with CAPMOX

[Mattes83]

Feature Request

Description

We are using Cluster api with the proxmox provider CAPMOX (https://github.com/ionos-cloud/cluster-api-provider-proxmox).
CAPMOX sets the providerid on the cluster resources in the management cluster to proxmox://{SystemUUID} and cluster api uses this providerid to find the nodes in the workload cluster. So we definetely need the providerid in the format proxmox://{SystemUUID}.

Can we use this proxmox ccm and csi anyway? Currently I am running a custom patched version with adjusted providerid. Ccm looks for the nodes by using SystemUUID instead of vmid. Is there already another solution for this?

If not I'd be happy to file a pull request for this.

I guess I should then hide this functionality behind a flag so that the original behavior is still in place.

[sergelogvinov]

Hello, very good question.

This project uses the format proxmox://${ProxmoxClusterName}/${VmID} because it was designed for multiple Proxmox clusters. Corosync has a limitation on the maximum number of nodes in a single cluster. The best way to handle this is to split one rack (48 servers) into 4 clusters, for example.

VmID is a unique number for a VM, and you cannot change it through the Proxmox API. It might be possible to change SystemUUID instead.

I am very curious: what does CAPMOX do with the ProviderID string? Is it possible to disable it, for example?

PS. We can also add compatibility for two types of ProviderID strings, but we need to implement this in both projects https://github.com/sergelogvinov/proxmox-csi-plugin, but SystemUUID is not tell the controller on which cluster the node is running unfortunately...

Please feel free to share your thoughts on this. 👍

[Mattes83]

Hi,
Thanks for your fast reply. I will try to explain:

With Cluster API, you have a management cluster in which Cluster API runs. This cluster is used to manage the lifecycle of the workload clusters. In the management cluster, you have some custom resources for each node in the workload cluster. CAPMOX is a kind of plugin for Cluster API which can create Kubernetes clusters based on Proxmox. The provider ID is also present in these custom resources in the management cluster and is used to match the custom resources to the real Kubernetes nodes. That means that Cluster API in the management cluster connects to the workload cluster and looks for a node with the corresponding provider ID. This is needed, for example, to perform health checks and decide if the node has to be rebuilt. So the provider ID cannot be disabled.

I think in the case of Cluster API, the SystemUUID is unlikely to change, as Cluster API treats the machines as immutable. Instead of changing something on the machine, you are replacing the old one with a new one.

I am currently running a version where I have replaced the old provider ID with the other format. I am just using FindVMByUUID which you have added in #142 to find the nodes. This means I iterate through all known clusters, but as Cluster API only supports one Proxmox cluster per Kubernetes cluster anyway, that's okay for me.
I also changed it in the CSI and was able to successfully provision volumes.

Would you be okay if I introduce a new flag like --enable-cluster-api-compatibility and hide the alternative provider ID behind it?

[sergelogvinov]

Oh, I see! It would definitely be great to have CAPI support in this project as well. It's such a popular and well-regarded solution, and I'm really interested in seeing how it could enhance things here. 👍

[Mattes83]

Great to hear this!
I will open a PR for this, but it will take a while as I am on holidays for the next two weeks