Merge pull request k0rdent#295 from a13x5/vsphere-provider-docs

Add vsphere provider docs
eromanova · Sep 16, 2024 · f95fa29 · f95fa29
2 parents 8c91890 + a2bf1f8
commit f95fa29
Show file tree

Hide file tree

Showing 5 changed files with 308 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -37,6 +37,7 @@ Optionally, the following CLIs may be helpful:
 Follow the instruction to configure providers. Currently supported providers:
 * [AWS](docs/aws/main.md#prepare-the-aws-infra-provider)
 * [Azure](docs/azure/main.md)
+* [vSphere](docs/vsphere/main.md)
 
 ### Installation
 

diff --git a/docs/vsphere/cluster-parameters.md b/docs/vsphere/cluster-parameters.md
@@ -0,0 +1,129 @@
+# vSphere cluster parameters
+
+## Prerequisites
+
+- vSphere provider [prerequisites](main.md#prerequisites) are complete.
+
+## Cluster Identity
+
+To provide credentials for CAPI vSphere provider (CAPV) the
+`VSphereClusterIdentity` resource must be created. This should be done before
+provisioning any clusters.
+
+To create cluster identity you'll only need username and password for your
+vSphere instance.
+
+The example of the objects can be found below:
+
+**Secret**:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: vsphere-cluster-identity-secret
+  namespace: hmc-system
+stringData:
+  username: user
+  password: Passw0rd
+```
+
+**VsphereClusterIdentity**:
+
+```yaml
+apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
+kind: VSphereClusterIdentity
+metadata:
+  name: vsphere-cluster-identity
+  namespace: hmc-system
+spec:
+  secretName: vsphere-cluster-identity-secret
+  allowedNamespaces:
+    selector:
+      matchLabels: {}
+```
+
+These objects then should be referenced in the `ManagedCluster` object in the
+`.spec.config.clusterIdentity` field.
+
+
+## ManagedCluster parameters
+
+To deploy managed cluster a number of parameters should be passed to the
+`ManagedCluster` object.
+
+### Parameter list
+
+The following is the list of vSphere specific parameters, which are _required_
+for successful cluster creation.
+
+| Parameter                           | Example                               | Description                        |
+|-------------------------------------|---------------------------------------|------------------------------------|
+| `.spec.config.vsphere.server`       | `vcenter.example.com`                 | Address of the vSphere server      |
+| `.spec.config.vsphere.thumbprint`   | `"00:00:00"`                          | Certificate thumbprint             |
+| `.spec.config.vsphere.datacenter`   | `DC`                                  | Datacenter name                    |
+| `.spec.config.vsphere.datastore`    | `/DC/datastore/DS`                    | Datastore path                     |
+| `.spec.config.vsphere.resourcePool` | `/DC/host/vCluster/Resources/ResPool` | Resource pool path                 |
+| `.spec.config.vsphere.folder`       | `/DC/vm/example`                      | vSphere folder path                |
+| `.spec.config.vsphere.username`     | `user`                                | Username for your vSphere instance |
+| `.spec.config.vsphere.password`     | `password`                            | Password for your vSphere instance |
+
+_You can check [machine parameters](machine-parameters.md) for machine specific
+parameters._
+
+To obtain vSphere certificate thumbprint you can use the following command:
+
+```bash
+curl -sw %{certs} https://vcenter.example.com | openssl x509 -sha256 -fingerprint -noout | awk -F '=' '{print $2}'
+```
+
+Username and password currently must be passed once more in the `ManagedCluster`
+object to provide proper authentication for CCM and CSI driver.
+
+## Example of ManagedCluster CR
+
+With all above parameters provided your `ManagedCluster` can look like this:
+
+```yaml
+apiVersion: hmc.mirantis.com/v1alpha1
+kind: ManagedCluster
+metadata:
+  name: cluster-1
+spec:
+  template: vsphere-standalone-cp
+  config:
+    clusterIdentity:
+      name: vsphere-cluster-identity
+    vsphere:
+      server: vcenter.example.com
+      thumbprint: "00:00:00"
+      datacenter: "DC"
+      datastore: "/DC/datastore/DC"
+      resourcePool: "/DC/host/vCluster/Resources/ResPool"
+      folder: "/DC/vm/example"
+      username: "user"
+      password: "Passw0rd"
+    controlPlaneEndpointIP: "172.16.0.10"
+
+    controlPlane:
+      ssh:
+        user: ubuntu
+        publicKey: |
+          ssh-rsa AAA...
+      rootVolumeSize: 50
+      cpus: 2
+      memory: 4096
+      vmTemplate: "/DC/vm/template"
+      network: "/DC/network/Net"
+
+    worker:
+      ssh:
+        user: ubuntu
+        publicKey: |
+          ssh-rsa AAA...
+      rootVolumeSize: 50
+      cpus: 2
+      memory: 4096
+      vmTemplate: "/DC/vm/template"
+      network: "/DC/network/Net"
+```
diff --git a/docs/vsphere/hosted-control-plane.md b/docs/vsphere/hosted-control-plane.md
@@ -0,0 +1,61 @@
+# Hosted control plane (k0smotron) deployment
+
+## Prerequisites
+
+- Management Kubernetes cluster (v1.28+) deployed on vSphere with HMC installed
+  on it
+
+Keep in mind that all control plane components for all managed clusters will
+reside in the management cluster.
+
+
+## ManagedCluster manifest
+
+Hosted CP template has mostly identical parameters with standalone CP, you can
+check them in the [cluster parameters](cluster-parameters.md) and the
+[machine parameters](machine-parameters.md) sections.
+
+**Important note on control plane endpoint IP**
+
+Since vSphere provider requires that user will provide control plane endpoint IP
+before deploying the cluster you should make sure that this IP will be the same
+that will be assigned to the k0smotron LB service. Thus you must provide control
+plane endpoint IP to the k0smotron service via annotation which is accepted by
+your LB provider (in the following example `kube-vip` annotation is used)
+
+```yaml
+apiVersion: hmc.mirantis.com/v1alpha1
+kind: ManagedCluster
+metadata:
+  name: cluster-1
+spec:
+  template: vsphere-hosted-cp
+  config:
+    clusterIdentity:
+      name: vsphere-cluster-identity
+    vsphere:
+      server: vcenter.example.com
+      thumbprint: "00:00:00"
+      datacenter: "DC"
+      datastore: "/DC/datastore/DC"
+      resourcePool: "/DC/host/vCluster/Resources/ResPool"
+      folder: "/DC/vm/example"
+      username: "user"
+      password: "Passw0rd"
+    controlPlaneEndpointIP: "172.16.0.10"
+
+    ssh:
+      user: ubuntu
+      publicKey: |
+        ssh-rsa AAA...
+    rootVolumeSize: 50
+    cpus: 2
+    memory: 4096
+    vmTemplate: "/DC/vm/template"
+    network: "/DC/network/Net"
+
+    k0smotron:
+      service:
+        annotations:
+          kube-vip.io/loadbalancerIPs: "172.16.0.10"
+```
diff --git a/docs/vsphere/machine-parameters.md b/docs/vsphere/machine-parameters.md
@@ -0,0 +1,55 @@
+# vSphere machine parameters
+
+## SSH
+
+Currently SSH configuration on vSphere expects that user is already created
+during template creation. Because of that you must pass username along with SSH
+public key to configure SSH access.
+
+
+SSH public key can be passed to `.spec.config.ssh.publicKey` (in case of
+hosted CP) parameter or `.spec.config.controlPlane.ssh.publicKey` and
+`.spec.config.worker.ssh.publicKey` parameters (in case of standalone CP) of the
+`ManagedCluster` object.
+
+SSH public key must be passed literally as a string.
+
+Username can be passed to `.spec.config.controlPlane.ssh.user`,
+`.spec.config.worker.ssh.user` or `.spec.config.ssh.user` depending on you
+deployment model.
+
+## VM resources
+
+The following parameters are used to define VM resources:
+
+| Parameter         | Example | Description                                                          |
+|-------------------|---------|----------------------------------------------------------------------|
+| `.rootVolumeSize` | `50`    | Root volume size in GB (can't be less than one defined in the image) |
+| `.cpus`           | `2`     | Number of CPUs                                                       |
+| `.memory`         | `4096`  | Memory size in MB                                                    |
+
+The resource parameters are the same for hosted and standalone CP deployments,
+but they are positioned differently in the spec, which means that they're going to:
+
+- `.spec.config` in case of hosted CP deployment.
+- `.spec.config.controlPlane` in in case of standalone CP for control plane
+  nodes.
+- `.spec.config.worker` in in case of standalone CP for worker nodes.
+
+## VM Image and network
+
+To provide image template path and network path the following parameters must be
+used:
+
+| Parameter     | Example           | Description         |
+|---------------|-------------------|---------------------|
+| `.vmTemplate` | `/DC/vm/template` | Image template path |
+| `.network`    | `/DC/network/Net` | Network path        |
+
+As with resource parameters the position of these parameters in the
+`ManagedCluster` depends on deployment type and these parameters are used in:
+
+- `.spec.config` in case of hosted CP deployment.
+- `.spec.config.controlPlane` in in case of standalone CP for control plane
+  nodes.
+- `.spec.config.worker` in in case of standalone CP for worker nodes.
diff --git a/docs/vsphere/main.md b/docs/vsphere/main.md
@@ -0,0 +1,62 @@
+# vSphere provider
+
+## Prerequisites
+
+1. `kubectl` CLI installed locally.
+2. vSphere instance version `6.7.0` or higher.
+3. vSphere account with appropriate privileges.
+4. Image template.
+5. vSphere network with DHCP enabled.
+
+### Image template
+
+You can use pre-buit image templates from [CAPV project](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/blob/main/README.md#kubernetes-versions-with-published-ovas)
+or build your own.
+
+When building your own image make sure that vmware tools and cloud-init are
+installed and properly configured.
+
+You can follow [official open-vm-tools guide](https://docs.vmware.com/en/VMware-Tools/11.0.0/com.vmware.vsphere.vmwaretools.doc/GUID-C48E1F14-240D-4DD1-8D4C-25B6EBE4BB0F.html)
+on how to correctly install vmware-tools.
+
+When setting up cloud-init you can refer to [official docs](https://cloudinit.readthedocs.io/en/latest/index.html)
+and specifically [vmware datasource docs](https://cloudinit.readthedocs.io/en/latest/reference/datasources/vmware.html)
+for extended information regarding cloud-init on vSphere.
+
+### vSphere network
+
+When creating network make sure that it has DHCP service.
+
+Also make sure that the part of your network is out of DHCP range (e.g. network
+172.16.0.0/24 with DHCP range 172.16.0.100-172.16.0.254). This is needed to make
+sure that LB services will not create any IP conflicts in the network.
+
+### vSphere privileges
+
+To function properly the user assigned to vSphere provider should be able to
+manipulate vSphere resources. The following is the general overview of the
+required privileges:
+
+- `Virtual machine` - full permissions are required
+- `Network` - `Assign network` is sufficient
+- `Datastore` - it should be possible for user to manipulate virtual machine
+  files and metadata
+
+In addition to that specific CSI driver permissions are required see
+[the official doc](https://docs.vmware.com/en/VMware-vSphere-Container-Storage-Plug-in/2.0/vmware-vsphere-csp-getting-started/GUID-0AB6E692-AA47-4B6A-8CEA-38B754E16567.html)
+to get more information on CSI specific permissions.
+
+## vSphere cluster parameters
+
+Follow the [vSphere cluster parameters](cluster-parameters.md) guide to setup
+mandatory parameters for vSphere clusters.
+
+## vSphere machine parameters
+
+Follow the [vSphere machine parameters](machine-parameters.md) guide if you want
+to setup/modify the default machine parameters.
+
+## vSphere hosted control plane
+
+Follow the [Hosted control plane](hosted-control-plane.md) guide to deploy
+hosted control plane cluster on vSphere.