Merge pull request #416 from red-hat-storage/sync_us--devel

Syncing latest changes from upstream devel for ceph-csi
red-hat-storage · Nov 19, 2024 · a508a0a · a508a0a
2 parents 33a1034 + 3e9b438
commit a508a0a
Show file tree

Hide file tree

Showing 14 changed files with 145 additions and 11 deletions.
diff --git a/README.md b/README.md
@@ -35,9 +35,9 @@ attaching them to workloads.
 Independent CSI plugins are provided to support RBD and CephFS backed volumes,
 
 - For details about configuration and deployment of RBD plugin, please refer
-  [rbd doc](https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-rbd.md) and
+  [rbd doc](https://github.com/ceph/ceph-csi/blob/devel/docs/rbd/deploy.md) and
   for CephFS plugin configuration and deployment please
-  refer [cephFS doc](https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-cephfs.md).
+  refer [cephFS doc](https://github.com/ceph/ceph-csi/blob/devel/docs/cephfs/deploy.md).
 - For example usage of the RBD and CephFS CSI plugins, see examples in `examples/`.
 - Stale resource cleanup, please refer [cleanup doc](docs/resource-cleanup.md).
 
@@ -75,7 +75,7 @@ Ceph CSI version, they will be asked to upgrade when requesting support.
 
 ### Ceph-CSI features and available versions
 
-Please refer [rbd nbd mounter](./docs/rbd-nbd.md#support-matrix)
+Please refer [rbd nbd mounter](./docs/design/proposals/rbd-nbd.md#support-matrix)
 for its support details.
 
 | Plugin | Features                                                  | Feature Status | CSI Driver Version | CSI Spec Version | Ceph Cluster Version | Kubernetes Version |

diff --git a/charts/ceph-csi-cephfs/README.md b/charts/ceph-csi-cephfs/README.md
@@ -66,6 +66,33 @@ version.
 We recommend not to use `--reuse-values` in case there are new defaults AND
 compare your currently used values with the new default values.
 
+### Enabling encryption support
+
+To enable FSCrypt support, you will need to include the KMS configuration in
+`encryptionKMSConfig`.
+
+Here is a `values.yaml` example using a Kubernetes secret (`kubernetes` KMS)
+
+```yaml
+encryptionKMSConfig:
+    encryptionKMSType: "metadata"
+    secretName: "cephfs-encryption-passphrase" # This secret needs to contain the passphrase as the key `encryptionPassphrase`
+    secretNamespace: "my-namespace"
+storageClass:
+    encrypted: true
+    encryptionKMSID: kubernetes
+```
+
+#### Least privilege secret access
+
+If you use the `metadata` and let RBAC created by the chart, permissions
+will be given to access **only** the secret referenced in the
+`encryptionKMSConfig`. This is something important to keep in mind, as a
+manual change to the config to point to another secret or add further KMS
+config will not be authorized. If you wish to give CephCSI a global secret
+access to the cluster, you may set `rbac.leastPrivileges` to `false`, and
+permissions will be granted globally via a *ClusterRole*.
+
 #### Known Issues Upgrading
 
 - When upgrading to version >=3.7.0, you might encounter an error that the
@@ -110,11 +137,13 @@ charts and their default values.
 | Parameter                                      | Description                                                                                                                                          | Default                                            |
 | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
 | `rbac.create`                                  | Specifies whether RBAC resources should be created                                                                                                   | `true`                                             |
+| `rbac.leastPrivileges`                         | Specifies whether RBAC resources should be created with a restricted scope when supported (only secrets supported currently)                         | `true`                                             |
 | `serviceAccounts.nodeplugin.create`            | Specifies whether a nodeplugin ServiceAccount should be created                                                                                      | `true`                                             |
 | `serviceAccounts.nodeplugin.name`              | The name of the nodeplugin ServiceAccount to use. If not set and create is true, a name is generated using the fullname                              | ""                                                 |
 | `serviceAccounts.provisioner.create`           | Specifies whether a provisioner ServiceAccount should be created                                                                                     | `true`                                             |
 | `serviceAccounts.provisioner.name`             | The name of the provisioner ServiceAccount of provisioner to use. If not set and create is true, a name is generated using the fullname              | ""                                                 |
 | `csiConfig`                                    | Configuration for the CSI to connect to the cluster                                                                                                  | []                                                 |
+| `encryptionKMSConfig`                          | Configuration for the encryption KMS                                                                                                                 | `{}`                                               |
 | `commonLabels`                                 | Labels to apply to all resources                                                               | `{}`                                                  |
 | `logLevel`                                     | Set logging level for csi containers. Supported values from 0 to 5. 0 for general useful logs, 5 for trace level verbosity.                          | `5`                                                |
 | `sidecarLogLevel`                              | Set logging level for csi sidecar containers. Supported values from 0 to 5. 0 for general useful logs, 5 for trace level verbosity.                  | `1`                                                |
@@ -174,7 +203,7 @@ charts and their default values.
 | `provisionerSocketFile`                        | The filename of the provisioner socket                                                                                                               | `csi-provisioner.sock`                             |
 | `pluginSocketFile`                             | The filename of the plugin socket                                                                                                                    | `csi.sock`                                         |
 | `readAffinity.enabled` | Enable read affinity for CephFS subvolumes. Recommended to set to true if running kernel 5.8 or newer. | `false` |
-| `readAffinity.crushLocationLabels` | Define which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click [here](https://github.com/ceph/ceph-csi/blob/v3.9.0/docs/deploy-cephfs.md#read-affinity-using-crush-locations-for-cephfs-subvolumes)| `[]` |
+| `readAffinity.crushLocationLabels` | Define which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click [here](https://github.com/ceph/ceph-csi/blob/devel/docs/cephfs/deploy.md#read-affinity-using-crush-locations-for-cephfs-subvolumes)| `[]` |
 | `kubeletDir`                                   | Kubelet working directory                                                                                                                            | `/var/lib/kubelet`                                 |
 | `driverName`                                   | Name of the csi-driver                                                                                                                               | `cephfs.csi.ceph.com`                              |
 | `configMapName`                                | Name of the configmap which contains cluster configuration                                                                                           | `ceph-csi-config`                                  |
@@ -184,6 +213,8 @@ charts and their default values.
 | `storageClass.name`                            | Specifies the cephFS StorageClass name                                                                                                               | `csi-cephfs-sc`                                    |
 | `storageClass.annotations`                     | Specifies the annotations for the cephFS storageClass                                                                                                | `[]`                                               |
 | `storageClass.clusterID`                       | String representing a Ceph cluster to provision storage from                                                                                         | `<cluster-ID>`                                     |
+| `storageClass.encrypted`                       | Specifies whether volume should be encrypted. Set it to true if you want to enable encryption                                                        | `""`                                               |
+| `storageClass.encryptionKMSID`                 | Specifies the encryption kms id                                                                                                                      | `""`                                               |
 | `storageClass.fsName`                          | CephFS filesystem name into which the volume shall be created                                                                                        | `myfs`                                             |
 | `storageClass.pool`                            | Ceph pool into which volume data shall be stored                                                                                                     | `""`                                               |
 | `storageClass.fuseMountOptions`                | Comma separated string of Ceph-fuse mount options                                                                                                    | `""`                                               |

diff --git a/charts/ceph-csi-cephfs/templates/encryptionkms-configmap.yaml b/charts/ceph-csi-cephfs/templates/encryptionkms-configmap.yaml
@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ .Values.kmsConfigMapName | quote }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    app: {{ include "ceph-csi-cephfs.name" . }}
+    chart: {{ include "ceph-csi-cephfs.chart" . }}
+    component: {{ .Values.nodeplugin.name }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+    {{- with .Values.commonLabels }}{{ toYaml . | trim | nindent 4 }}{{- end }}
+data:
+  config.json: |-
+{{ toJson .Values.encryptionKMSConfig | indent 4 -}}
diff --git a/charts/ceph-csi-cephfs/templates/nodeplugin-clusterrole.yaml b/charts/ceph-csi-cephfs/templates/nodeplugin-clusterrole.yaml
@@ -14,10 +14,14 @@ rules:
   - apiGroups: [""]
     resources: ["nodes"]
     verbs: ["get"]
+  # allow to read Vault Token and connection options from the Tenants namespace
   - apiGroups: [""]
     resources: ["configmaps"]
     verbs: ["get"]
+{{- if and .Values.encryptionKMSConfig .Values.encryptionKMSConfig.secretNamespace (not .Values.rbac.leastPrivileges) }}
+  # allow to read the encryption key used with the metadata KMS
   - apiGroups: [""]
     resources: ["secrets"]
     verbs: ["get"]
 {{- end -}}
+{{- end -}}
diff --git a/charts/ceph-csi-cephfs/templates/nodeplugin-role.yaml b/charts/ceph-csi-cephfs/templates/nodeplugin-role.yaml
@@ -0,0 +1,22 @@
+{{- if and .Values.rbac.create .Values.rbac.leastPrivileges -}}
+{{- if and .Values.encryptionKMSConfig (eq .Values.encryptionKMSConfig.encryptionKMSType "metadata") .Values.encryptionKMSConfig.secretNamespace .Values.encryptionKMSConfig.secretName -}}
+kind: Role
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: {{ include "ceph-csi-cephfs.nodeplugin.fullname" . }}
+  namespace: {{ .Values.encryptionKMSConfig.secretNamespace }}
+  labels:
+    app: {{ include "ceph-csi-cephfs.name" . }}
+    chart: {{ include "ceph-csi-cephfs.chart" . }}
+    component: {{ .Values.nodeplugin.name }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+    {{- with .Values.commonLabels }}{{ toYaml . | trim | nindent 4 }}{{- end }}
+rules:
+  # allow to read the encryption key used with the metadata KMS
+  - apiGroups: [""]
+    resources: ["secrets"]
+    verbs: ["get"]
+    resourceNames: [{{ .Values.encryptionKMSConfig.secretName | quote }}]
+{{- end -}}
+{{- end -}}
diff --git a/charts/ceph-csi-cephfs/templates/nodeplugin-rolebinding.yaml b/charts/ceph-csi-cephfs/templates/nodeplugin-rolebinding.yaml
@@ -0,0 +1,24 @@
+{{- if and .Values.rbac.create .Values.rbac.leastPrivileges -}}
+{{- if and .Values.encryptionKMSConfig (eq .Values.encryptionKMSConfig.encryptionKMSType "metadata") .Values.encryptionKMSConfig.secretNamespace -}}
+kind: RoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: {{ include "ceph-csi-cephfs.nodeplugin.fullname" . }}
+  namespace: {{ .Values.encryptionKMSConfig.secretNamespace }}
+  labels:
+    app: {{ include "ceph-csi-cephfs.name" . }}
+    chart: {{ include "ceph-csi-cephfs.chart" . }}
+    component: {{ .Values.nodeplugin.name }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+    {{- with .Values.commonLabels }}{{ toYaml . | trim | nindent 4 }}{{- end }}
+subjects:
+  - kind: ServiceAccount
+    name: {{ include "ceph-csi-cephfs.serviceAccountName.nodeplugin" . }}
+    namespace: {{ .Release.Namespace }}
+roleRef:
+  kind: Role
+  name: {{ include "ceph-csi-cephfs.nodeplugin.fullname" . }}
+  apiGroup: rbac.authorization.k8s.io
+{{- end -}}
+{{- end -}}
diff --git a/charts/ceph-csi-cephfs/templates/storageclass.yaml b/charts/ceph-csi-cephfs/templates/storageclass.yaml
@@ -20,6 +20,12 @@ parameters:
 {{- if .Values.storageClass.pool }}
   pool: {{ .Values.storageClass.pool }}
 {{- end }}
+{{- if .Values.storageClass.encrypted }}
+  encrypted: "{{ .Values.storageClass.encrypted }}"
+{{- end }}
+{{- if .Values.storageClass.encryptionKMSID }}
+  encryptionKMSID: {{ .Values.storageClass.encryptionKMSID }}
+{{- end }}
 {{- if .Values.storageClass.fuseMountOptions }}
   fuseMountOptions: "{{ .Values.storageClass.fuseMountOptions }}"
 {{- end }}

diff --git a/charts/ceph-csi-cephfs/values.yaml b/charts/ceph-csi-cephfs/values.yaml
@@ -2,6 +2,9 @@
 rbac:
   # Specifies whether RBAC resources should be created
   create: true
+  # When possible try and reduce the scope of permission to only give
+  # access to resources defined in the config. See the README for more info
+  leastPrivileges: true
 
 serviceAccounts:
   nodeplugin:
@@ -31,6 +34,20 @@ serviceAccounts:
 #       radosNamespace: "csi"
 csiConfig: []
 
+# Configuration for the encryption KMS
+# yamllint disable-line rule:line-length
+# Ref: https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-cephfs.md#cephfs-volume-encryption
+# Example:
+#   encryptionKMSConfig:
+#     encryptionKMSType: vault
+#     vaultAddress: https://vault.example.com
+#     vaultAuthPath: /v1/auth/kubernetes/login
+#     vaultRole: csi-kubernetes
+#     vaultPassphraseRoot: /v1/secret
+#     vaultPassphrasePath: ceph-csi/
+#     vaultCAVerify: "true"
+encryptionKMSConfig: {}
+
 # Labels to apply to all resources
 commonLabels: {}
 
@@ -329,6 +346,18 @@ storageClass:
   # If omitted, defaults to "csi-vol-".
   # volumeNamePrefix: "foo-bar-"
   volumeNamePrefix: ""
+
+  # (optional) Instruct the plugin it has to encrypt the volume
+  # By default it is disabled. Valid values are "true" or "false".
+  # A string is expected here, i.e. "true", not true.
+  # encrypted: "true"
+  encrypted: ""
+
+  # (optional) Use external key management system for encryption passphrases by
+  # specifying a unique ID matching KMS ConfigMap. The ID is only used for
+  # correlation to configmap entry.
+  encryptionKMSID: ""
+
   # The secrets have to contain user and/or Ceph admin credentials.
   provisionerSecret: csi-cephfs-secret
   # If the Namespaces are not specified, the secrets are assumed to
@@ -400,6 +429,8 @@ configMapName: ceph-csi-config
 externallyManagedConfigmap: false
 # Name of the configmap used for ceph.conf
 cephConfConfigMapName: ceph-config
+# Name of the configmap used for encryption kms configuration
+kmsConfigMapName: ceph-csi-encryption-kms-config
 # CephFS RadosNamespace used to store CSI specific objects and keys.
 # radosNamespaceCephFS: csi
 # Unique ID distinguishing this instance of Ceph CSI among other instances,

diff --git a/charts/ceph-csi-rbd/README.md b/charts/ceph-csi-rbd/README.md
@@ -187,7 +187,7 @@ charts and their default values.
 | `topology.enabled`                             | Specifies whether topology based provisioning support should be exposed by CSI                                                                       | `false`                                            |
 | `topology.domainLabels`                        | DomainLabels define which node labels to use as domains for CSI nodeplugins to advertise their domains                                               | `{}`                                               |
 | `readAffinity.enabled` | Enable read affinity for RBD volumes. Recommended to set to true if running kernel 5.8 or newer. | `false` |
-| `readAffinity.crushLocationLabels` | Define which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click [here](https://github.com/ceph/ceph-csi/blob/v3.9.0/docs/deploy-rbd.md#read-affinity-using-crush-locations-for-rbd-volumes)| `[]` |
+| `readAffinity.crushLocationLabels` | Define which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click [here](https://github.com/ceph/ceph-csi/blob/devel/docs/rbd/deploy.md#read-affinity-using-crush-locations-for-rbd-volumes)| `[]` |
 | `provisionerSocketFile`                        | The filename of the provisioner socket                                                                                                               | `csi-provisioner.sock`                             |
 | `pluginSocketFile`                             | The filename of the plugin socket                                                                                                                    | `csi.sock`                                         |
 | `kubeletDir`                                   | kubelet working directory                                                                                                                            | `/var/lib/kubelet`                                 |

diff --git a/charts/ceph-csi-rbd/values.yaml b/charts/ceph-csi-rbd/values.yaml
@@ -46,7 +46,7 @@ csiConfig: []
 csiMapping: []
 
 # Configuration for the encryption KMS
-# Ref: https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-rbd.md
+# Ref: https://github.com/ceph/ceph-csi/blob/devel/docs/rbd/deploy.md
 # Example:
 # encryptionKMSConfig:
 #   vault-unique-id-1:
@@ -415,7 +415,7 @@ storageClass:
   # By default host-path /var/log/ceph of node is bind-mounted into
   # csi-rbdplugin pod at /var/log/ceph mount path. This is to configure
   # target bindmount path used inside container for ceph clients logging.
-  # See docs/rbd-nbd.md for available configuration options.
+  # See docs/design/proposals/rbd-nbd.md for available configuration options.
   # cephLogDir: /var/log/ceph
   cephLogDir: ""
 

diff --git a/docs/resource-cleanup.md b/docs/resource-cleanup.md
@@ -28,7 +28,7 @@ a. get pv_name
 ### 2. Get omap key/value
 
 a. get omapkey (suffix of csi.volumes.default is value used for the CLI option
-   [--instanceid](deploy-rbd.md#configuration) in the provisioner deployment.)
+   [--instanceid](rbd/deploy.md#configuration) in the provisioner deployment.)
 
   ```
   rados listomapkeys csi.volumes.default -p pool_name | grep pv_name
@@ -56,7 +56,7 @@ b. get omapval
 
 ### 3. Delete the RBD image or CephFS subvolume
 
-a. remove rbd image(csi-vol-omapval, the prefix csi-vol is value of [volumeNamePrefix](deploy-rbd.md#configuration))
+a. remove rbd image(csi-vol-omapval, the prefix csi-vol is value of [volumeNamePrefix](rbd/deploy.md#configuration))
 
   ```
   rbd remove rbd_image_name -p pool_name

diff --git a/docs/snap-clone.md b/docs/snap-clone.md
@@ -56,7 +56,7 @@ The snapshot is created on/for an existing PVC. You should
 have a PVC in bound state before creating snapshot from it.
 It is recommended to create a volume snapshot or a PVC clone
 only when the PVC is not in use.
-Please refer pvc creation [doc](https://github.com/ceph/ceph-csi/blob/devel/docs/deploy-cephfs.md)
+Please refer pvc creation [doc](https://github.com/ceph/ceph-csi/blob/devel/docs/cephfs/deploy.md)
 for more information on how to create a PVC.
 
 - Verify if PVC is in Bound state