diff --git a/.gitignore b/.gitignore index 485dee6..5c3bdbb 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1 @@ -.idea +*bundle.yaml diff --git a/README.md b/README.md index 5259ca0..ede2ef9 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ - + # Deploying Redis Enterprise on Kubernetes * [Quickstart Guide](#quickstart-guide) @@ -10,10 +10,15 @@ * [Private Repositories](#private-repositories) * [Pull Secrets](#pull-secrets) * [Advanced Configuration](#advanced-configuration) +* [How to connect to Redis Enterprise Software web console?](#How-to-connect-to-Redis-Enterprise-Software-web-console?) * [Upgrade](#upgrade) * [Supported K8S Distributions](#supported-k8s-distributions) -This page describe how to deploy Redis Enterprise on Kubernetes using the Redis Enterprise Operator. High level architecture and overview of the solution can be found [HERE](https://docs.redislabs.com/latest/platforms/kubernetes/). +This page describes how to deploy Redis Enterprise on Kubernetes using the Redis Enterprise Operator. The Redis Enterprise Operator supports two Custom Resource Definitions (CRDs): +* Redis Enterprise Cluster (REC): an API to create Redis Enterprise clusters. Note that only one cluster is supported per operator deployment. +* Redis Enterprise Database (REDB): an API to create Redis databases running on the Redis Enterprise cluster. +Note that the Redis Enterprise Operator is namespaced. +High level architecture and overview of the solution can be found [HERE](https://docs.redislabs.com/latest/platforms/kubernetes/). ## Quickstart Guide @@ -27,9 +32,9 @@ This page describe how to deploy Redis Enterprise on Kubernetes using the Redis The following are the images and tags for this release: | Component | k8s | Openshift | | --- | --- | --- | -| Redis Enterprise | `redislabs/redis:6.0.20-69` | `redislabs/redis:6.0.20-69.rhel7-openshift` | -| Operator | `redislabs/operator:6.0.20-4` | `redislabs/operator:6.0.20-4` | -| Services Rigger | `redislabs/k8s-controller:6.0.20-4` | `redislabs/k8s-controller:6.0.20-4` | +| Redis Enterprise | `redislabs/redis:6.0.20-97` | `redislabs/redis:6.0.20-97.rhel7-openshift` | +| Operator | `redislabs/operator:6.0.20-12` | `redislabs/operator:6.0.20-12` | +| Services Rigger | `redislabs/k8s-controller:6.0.20-12` | `redislabs/k8s-controller:6.0.20-12` | > * RedHat certified images are available on [Redhat Catalog](https://access.redhat.com/containers/#/product/71f6d1bb3408bd0d)
@@ -65,6 +70,7 @@ This is the fastest way to get up and running with a new Redis Enterprise on Kub kubectl apply -f service_account.yaml kubectl apply -f crds/v1/rec_crd.yaml kubectl apply -f crds/v1alpha1/redb_crd.yaml + kubectl apply -f admission-service.yaml kubectl apply -f operator.yaml ``` @@ -98,10 +104,6 @@ This is the fastest way to get up and running with a new Redis Enterprise on Kub 5. Redis Enterprise Database (REDB) Admission Controller: The Admission Controlller is recommended for use. It uses the Redis Enterprise Cluster to dynamically validate that REDB resources as configured by the operator are valid. Steps to configure the Admission Controller: - * Install the Admission Controller via a bundle: - ```shell script - kubectl create -f admission.bundle.yaml - ``` * Wait for the secret to be created: ```shell script kubectl get secret admission-tls @@ -115,15 +117,14 @@ This is the fastest way to get up and running with a new Redis Enterprise on Kub ```shell script # save cert CERT=`kubectl get secret admission-tls -o jsonpath='{.data.cert}'` - sed 's/NAMESPACE_OF_SERVICE_ACCOUNT/REPLACE_WITH_NAMESPACE/g' admission/webhook.yaml | kubectl create -f - + sed 's/NAMESPACE_OF_SERVICE_ACCOUNT/REPLACE_WITH_NAMESPACE/g' webhook.yaml | kubectl create -f - # create patch file cat > modified-webhook.yaml < Note: An example REDB.yaml file may be found [HERE] (https://github.com/RedisLabs/redis-enterprise-k8s-docs/tree/master/examples/v1alpha1). This is an alternative to copying and pasting the example code below into the CLI - The Redis Enterprise Operator can be instructed to manage databases on the Redis Enterprise Cluster using the REDB custom resource. Example: ```yaml @@ -222,11 +220,7 @@ Other custom configurations are referenced in this repository. 6. Redis Enterprise Database (REDB) Admission Controller: The Admission Controlller is recommended for use. It uses the Redis Enterprise Cluster to dynamically validate that REDB resources as configured by the operator are valid. Steps to configure the Admission Controller: - * Install the Admission Controller via a bundle: - ```shell script - kubectl create -f admission.bundle.yaml - ``` - * Wait for the secret to be created: + * Wait for the secret to be created by the operator bundle deployment ```shell script kubectl get secret admission-tls NAME TYPE DATA AGE @@ -298,8 +292,7 @@ Other custom configurations are referenced in this repository. ### Installation on VMWare Tanzu Instruction on how to deploy the Operator on PKS can be found on the [Redis Labs documentation Website](https://docs.redislabs.com/latest/platforms/kubernetes/getting-started/tanzu/) - - + ## Configuration ### RedisEnterpriseCluster custom resource @@ -310,7 +303,7 @@ The operator deploys a `RedisEnterpriseCluster` with default configurations valu redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: redislabs/redis - versionTag: 6.0.20-69 + versionTag: 6.0.20-97 ``` * Persistence @@ -412,21 +405,21 @@ For example: redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/redis - versionTag: 6.0.20-69 + versionTag: 6.0.20-97 ``` ```yaml redisEnterpriseServicesRiggerImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/k8s-controller - versionTag: 6.0.20-4 + versionTag: 6.0.20-12 ``` ```yaml bootstrapperImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/operator - versionTag: 6.0.20-4 + versionTag: 6.0.20-12 ``` In Operator Deployment spec (operator.yaml): @@ -438,7 +431,7 @@ spec: spec: containers: - name: redis-enterprise-operator - image: harbor.corp.local/redisenterprise/operator:6.0.20-4 + image: harbor.corp.local/redisenterprise/operator:6.0.20-12 ``` Image specification follow the [K8s Container schema](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#container-v1-core). @@ -485,11 +478,82 @@ spec: - Full [Redis Enterprise Database Custom Resource Specification](redis_enterprise_database_api.md)

+ +## How to connect to Redis Enterprise Software web console? + +The username and password for the web console are stored in a secret with the Redis Enterprise Cluster name on the k8s. +in order to connect to the web console the port-forward or load balancer can be used. + +First, please follow the below instructions to extract the username and password from the secret: +1. Switch to the namespace with the Redis Enterprise Cluster via the command below, replace with the relevant namespace: +```bash + kubectl config set-context --current --namespace= +``` +![Alt text](./images/web_console_1.png?raw=true) +2. List the secrets via the command: +```bash + kubectl get secret +``` +![Alt text](./images/web_console_2.png?raw=true) +3. Run the command below to view the secret with the credentials , replace the with the name of your Redis Enterprise Cluster: +```bash + kubectl get secret -o yaml +``` +![Alt text](./images/web_console_3.png?raw=true) +4. Extract the username and password via the commands below, replace the with the name of your Redis Enterprise Cluster: +```bash + kubectl get secret -o jsonpath='{.data.username}' | base64 --decode + kubectl get secret -o jsonpath='{.data.password}' | base64 --decode +``` +![Alt text](./images/web_console_4.png?raw=true) + +In order to connect to the web console please use one of the following methods: + +Method 1: using port-forward +1. Get the port of the cluster UI service via the command below, replace the with the name of your Redis Enterprise Cluster: +```bash + kubectl get service/-ui -o yaml +``` +Note: the default port is 8443. +![Alt text](./images/web_console_5.png?raw=true) +2. Run the command below to set port-forward, replace the with the name of your Redis Enterprise Cluster, use the port of the service for the service port and the port you want to use on the local machine as the local port: +```bash + kubectl port-forward service/-ui : +``` +![Alt text](./images/web_console_6.png?raw=true) +3. In the web browser on the local machine to see the Redis Enterprise web console go to: +https://localhost: +Don't forget to replace the with the one used in the previous command. +![Alt text](./images/web_console_7.png?raw=true) + +Method 2: load balancer +> Note - configuring a load balancer service for the UI will create an external IP address, widely available (when set on cloud providers which support external load balancers). Use with caution. +1. Run the command below to set the UI service type as load balancer, replace the with the name of your Redis Enterprise Cluster: +```bash + kubectl patch rec --type merge --patch "{\"spec\":{\"uiServiceType\":\"LoadBalancer\"}}" +``` +![Alt text](./images/web_console_8.png?raw=true) +2. Get the external IP and service port of the service via the command below: +```bash + kubectl get service/-ui +``` +Note: the default port is 8443. +![Alt text](./images/web_console_9.png?raw=true) +3. In the web browser on the local machine to see the Redis Enterprise web console go to: +https://: +Don't forget to replace the and with the values from the previous step. +![Alt text](./images/web_console_10.png?raw=true) + +Note: in the examples above the Redis Enterprise Cluster name is: 'rec' and the namespace is 'demo'. + +

+ ## Upgrade + The Operator automates and simplifies the upgrade process. The Redis Enterprise Cluster Software, and the Redis Enterprise Operator for Kubernetes versions are tightly coupled and should be upgraded together. It is recommended to use the bundle.yaml to upgrade, as it loads all the relevant CRD documents for this version. If the updated CRDs are not loaded, the operator might fail. -There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within the Redis Enterprise Cluster Spec to instruct the operator to automatically upgrade to the compatible version, or specify the correct Redis Enterprise image manually using the versionTag attribute. The Redis Enterprise Version compatible with this release is 6.0.20-69 +There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within the Redis Enterprise Cluster Spec to instruct the operator to automatically upgrade to the compatible version, or specify the correct Redis Enterprise image manually using the versionTag attribute. The Redis Enterprise Version compatible with this release is 6.0.20-97 ```yaml autoUpgradeRedisEnterprise: true @@ -498,7 +562,7 @@ There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within t Alternatively: ```yaml RedisEnterpriseImageSpec: - versionTag: redislabs/redis:6.0.20-69 + versionTag: redislabs/redis:6.0.20-97 ``` ## Supported K8S Distributions @@ -507,7 +571,6 @@ Supported versions (platforms/versions that are not listed are not supported): | Distribution | Support Status | |---------------------------------|----------------| | Openshift 3.11 (K8s 1.11) | supported | -| Openshift 4.4 (K8s 1.17) | deprecated | | OpenShift 4.5 (K8s 1.18) | supported | | OpenShift 4.6 (K8s 1.19) | supported | | OpenShift 4.7 (K8s 1.20) | supported | @@ -517,9 +580,7 @@ Supported versions (platforms/versions that are not listed are not supported): | KOPS vanilla 1.18 | supported | | KOPS vanilla 1.19 | supported | | KOPS vanilla 1.20 | supported | -| GKE 1.15 | deprecated | -| GKE 1.16 | deprecated | -| GKE 1.17 | supported | +| GKE 1.17* | deprecated | | GKE 1.18 | supported | | GKE 1.19 | supported | | GKE 1.20 | supported | @@ -528,9 +589,11 @@ Supported versions (platforms/versions that are not listed are not supported): | Rancher 2.5 (K8s 1.17) | supported | | Rancher 2.5 (K8s 1.18) | supported | | Rancher 2.5 (K8s 1.19). | supported | -| VMWare TKGIE*** 1.7 (K8s 1.16) | supported | +| VMWare TKGIE** 1.7 (K8s 1.16) | deprecated | +| VMWare TKGIE** 1.8 (K8s 1.17) | deprecated | +| VMWare TKGIE*** 1.10 (K8s 1.19) | supported | | AKS 1.18 | supported | -\* No longer supported by Red Hat -\*\* No longer supported by Google +\* No longer supported by Google +\*\* No longer supported by VMware \*\*\* Tanzu Kubernetes Grid Integrated Edition diff --git a/admission/service.yaml b/admission-service.yaml similarity index 79% rename from admission/service.yaml rename to admission-service.yaml index b19789b..200987f 100644 --- a/admission/service.yaml +++ b/admission-service.yaml @@ -8,4 +8,4 @@ spec: protocol: TCP targetPort: 8443 selector: - app: redb-admission + name: redis-enterprise-operator diff --git a/admission.bundle.yaml b/admission.bundle.yaml deleted file mode 100644 index d95cced..0000000 --- a/admission.bundle.yaml +++ /dev/null @@ -1,87 +0,0 @@ ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: redis-enterprise-admission ---- -kind: Role -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: redis-enterprise-admission -rules: - - apiGroups: [""] - resources: ["secrets"] - verbs: ["*"] - - apiGroups: ["app.redislabs.com"] - resources: ["*"] - verbs: ["*"] - ---- -kind: RoleBinding -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: redis-enterprise-admission -subjects: -- kind: ServiceAccount - name: redis-enterprise-admission -roleRef: - kind: Role - name: redis-enterprise-admission - apiGroup: rbac.authorization.k8s.io ---- -apiVersion: v1 -kind: Service -metadata: - name: admission -spec: - ports: - - port: 443 - protocol: TCP - targetPort: 8443 - selector: - app: redb-admission ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: admission-deploy -spec: - selector: - matchLabels: - app: redb-admission - template: - metadata: - labels: - app: redb-admission - spec: - serviceAccountName: redis-enterprise-admission - containers: - - name: admin - image: redislabs/operator:6.0.20-4 - command: - - /usr/local/bin/admission - imagePullPolicy: Always - ports: - - containerPort: 443 - env: - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - resources: - limits: - cpu: 1000m - memory: 512Mi - requests: - cpu: 250m - memory: 256Mi - readinessProbe: - failureThreshold: 3 - successThreshold: 1 - periodSeconds: 30 - timeoutSeconds: 10 - httpGet: - path: /healthz - port: 8443 - scheme: HTTPS ---- diff --git a/admission/README.md b/admission/README.md index 82a67a8..73a6c2a 100644 --- a/admission/README.md +++ b/admission/README.md @@ -1,54 +1,11 @@ - - # REDB Admission Controller Redis Labs' Redis Enterprise Operator provides an installable admission control that can be used to verify RedisEnterpriseDatabase resources on creation and modification for correctness. This prevents end users from creating syntatically valid but functionally invalid database configurations. The admission control leverages Kubernetes' built in [Dynamic Admission Control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/). **Note:** Redis Labs' Redis Enterprise Operator can also be installed through the [Gesher Admission Proxy](GESHER.md) -## Admission Control via Bundle Installation - -1. Install the Admission Controller via a bundle into the same namespace the REC was installed into. - -```shell script -kubectl create -f admission.bundle.yaml -``` - -## Individual Yaml Installation - -1. namespaced Role that allows creation and reading of Secrets - - ```shell script - kubectl apply -f role.yaml - ``` - -2. ServiceAccount for admission controller to run as - - ```shell script - kubectl apply -f service_account.yaml - ``` - -3. Binding namespaced Role to the service account - - ```shell script - kubectl apply -f role_binding.yaml - ``` - -4. Kubernetes Service that is used to access the Admission Control HTTP Server - - ```shell script - kubectl apply -f service.yaml - ``` - -5. TLS Key generator + Admission Controller HTTP Server - - ```shell script - kubectl apply -f deployment.yaml - ``` - ## Hooking up the Admission controller directly with Kubernetes - -**NOTE**: This only has to be done the first time setting up the admission controller, it can be skipped on update +**NOTE**: This only has to be done the first time setting up the redis enterprise operator, it can be skipped on update 1. Wait for the secret to be created @@ -58,7 +15,7 @@ kubectl create -f admission.bundle.yaml admission-tls Opaque 2 2m43s ``` -2. Enable the Kubernetes webhook using the generated certificate +2. Enable the Kubernetes webhook using the generated certificate stored in a kubernetes secret **NOTE**: One must replace REPLACE_WITH_NAMESPACE in the following command with the namespace the REC was installed into. @@ -70,10 +27,9 @@ kubectl create -f admission.bundle.yaml # create patch file cat > modified-webhook.yaml < Note - other utilities to view secrets are available + +## Changing the Redis Enterprise Cluster Credentials +### Replacing the password +Please follow the following steps: +1. Take note of the current password, see above +2. Exec into a Redis Enterprise Cluster node pod using the following command: + ``` + kubectl exec -it -0 bash + ``` +3. Within the pod console, run a command to add the new password as supported for the existing user, replace with the existing credentials and the new password: + ``` + REC_USER="`cat /opt/redislabs/credentials/username`"; REC_PASSWORD="`cat /opt/redislabs/credentials/password`";curl -k --request POST --url https://localhost:9443/v1/users/password -u "$REC_USER:$REC_PASSWORD" --header 'Content-Type: application/json' --data "{\"username\":\"$REC_USER\",\"old_password\":\"$REC_PASSWORD\", \"new_password\":\"\"}" + ``` +4. Update the cluster credential secret: using the commands ran outside of the Redis Enterprise Cluster node pod: + a. Save the existing username to a text file (replace with actual). + ``` + echo -n "" > username + ``` + b. Save the new password to a text file (replace with actual). + ``` + echo -n "" > password + ``` + c. Update the secret: + ``` + kubectl create secret generic --from-file=./username --from-file=./password --dry-run -o yaml | kubectl apply -f - + ``` +5. Wait 5 minutes to make sure all components have read the new password from the updated secret +6. Exec into a Redis Enterprise Cluster node pod (see above) and run the following command to remove the previous password so only the new one applies. Important: replace OLD PASSWORD with the one being replaced, see step 1 above. + ``` + REC_USER="`cat /opt/redislabs/credentials/username`"; REC_PASSWORD="`cat /opt/redislabs/credentials/password`";curl -k --request DELETE --url https://localhost:9443/v1/users/password -u "$REC_USER:$REC_PASSWORD" --header 'Content-Type: application/json' --data "{\"username\":\"$REC_USER\",\"old_password\":\" Note: the username to be used with the K8s secret is the email displayed on the Redis Enterprise UI + + > Note: this procedure is only supported for version 6.0.20-5 or above +### Replacing the password and the username +Please follow the following steps: +1. Log into the Redis Enterprise Cluster UI using the credentials as explained above. +2. Add another admin user, choose a password +3. Set the new username with the Redis Enterprise Cluster spec (username field) +4. Update the cluster credential secret: + a. Save the new username to a text file (replace with actual). + ``` + echo -n "" > username + ``` + b. Save the new password to a text file (replace with actual). + ``` + echo -n "" > password + ``` + c. Update the secret: + ``` + kubectl create secret generic --from-file=./username --from-file=./password --dry-run -o yaml | kubectl apply -f - + ``` + > Note: the username to be used with the K8s secret is the email displayed on the Redis Enterprise UI +5. Wait 5 minutes to make sure all components have read the new password from the updated secret +6. Delete the previous admin user using the Redis Enterprise Cluster UI + > Note: this procedure is only supported for version 6.0.20-5 or above + > Note: the operator might log errors in the time period between updating the username in the REC spec and the secret update + + diff --git a/crds/README.md b/crds/README.md new file mode 100644 index 0000000..542f90e --- /dev/null +++ b/crds/README.md @@ -0,0 +1,30 @@ +## CRD's Schema
+ + +Purpose: +The schema is meant to ease the use of CRDs by restricting the fields to defined types and providing relevant description. + + +### Work Flow - How add a new field: +* Not all steps are always required + +#### 1. Update Tags: +In the files ++ `pkg/apis/app/v1alpha1/redisenterprisedatabase_types.go`
++ `pkg/apis/app/v1alpha1/redisenterprisecluster_types.go`
+add tags on top of variables to mark allowed types (Enum)
+(see: https://book.kubebuilder.io/reference/markers/crd-validation.html).
+They will be used by the operator sdk when creating the scheme.
+**Default values** do not appear in the schema since the operator has its own default mechanism. + + +#### 2 .Create the schema: +1) Install operator-sdk version 0.17.2.
+See: https://sdk.operatorframework.io/docs/installation/install-operator-sdk/#install-from-github-release +2) Run `operator-sdk generate crds` + +#### 3. Update the schema: +- Edit the current schema in all of the relevant CRD yamls with the new section that was generated
+- Note that fields which are k8s objects might be very long - e.g. `SideContainersSpec` +in this case -
it appears in the schema but with no description in the fields since it extends the schema and is not unique for Redis Enterprise. + diff --git a/crds/v1/rec_crd.yaml b/crds/v1/rec_crd.yaml index 43ee4fb..0e4f2cc 100644 --- a/crds/v1/rec_crd.yaml +++ b/crds/v1/rec_crd.yaml @@ -135,6 +135,21 @@ spec: value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object + clusterCredentialSecretName: + description: Secret Name/Path to use for Cluster Credentials. If left + blank, will use cluster name + type: string + clusterCredentialSecretRole: + description: Used only if ClusterCredentialSecretType is vault, to define + vault role to be used. If blank, defaults to "redis-enterprise-operator" + type: string + clusterCredentialSecretType: + description: Type of Secret to use for ClusterCredential, Vault, Kuberetes,... + If left blank, will default ot kubernetes secrets + enum: + - vault + - kubernetes + type: string clusterRecovery: description: ClusterRecovery initiates cluster recovery when set to true. Note that this field is cleared automatically after the cluster diff --git a/crds/v1alpha1/rec_crd.yaml b/crds/v1alpha1/rec_crd.yaml index 5b6da48..b5f98d4 100644 --- a/crds/v1alpha1/rec_crd.yaml +++ b/crds/v1alpha1/rec_crd.yaml @@ -146,6 +146,9 @@ spec: clusterCredentialSecretType: description: Type of Secret to use for ClusterCredential, Vault, Kuberetes,... If left blank, will default ot kubernetes secrets + enum: + - vault + - kubernetes type: string clusterRecovery: description: ClusterRecovery initiates cluster recovery when set to @@ -181,6 +184,12 @@ spec: license: description: Redis Enterprise License type: string + licenseSecretName: + description: K8s secret or Vault Secret Name/Path to use for Cluster License. When left blank, the license + is read from the "license" field. Note that you can't specify non-empty values in both + "license" and "licenseSecretName", only one of these fields can be used to pass the license string. + The license needs to be stored under the key "license". + type: string nodeSelector: additionalProperties: type: string diff --git a/crds/v1alpha1/redb_crd.yaml b/crds/v1alpha1/redb_crd.yaml index 69a754b..58b4af9 100644 --- a/crds/v1alpha1/redb_crd.yaml +++ b/crds/v1alpha1/redb_crd.yaml @@ -3,6 +3,28 @@ kind: CustomResourceDefinition metadata: name: redisenterprisedatabases.app.redislabs.com spec: + additionalPrinterColumns: + - JSONPath: .status.version + name: Version + type: string + - JSONPath: .status.internalEndpoints[*].port + name: Port + type: string + - JSONPath: .status.redisEnterpriseCluster + name: Cluster + type: string + - JSONPath: .status.shardStatuses.active + name: Shards + type: string + - JSONPath: .status.status + name: Status + type: string + - JSONPath: .status.specStatus + name: Spec Status + type: string + - JSONPath: .metadata.creationTimestamp + name: Age + type: date group: app.redislabs.com names: kind: RedisEnterpriseDatabase diff --git a/images/web_console_1.png b/images/web_console_1.png new file mode 100644 index 0000000..5c7243f Binary files /dev/null and b/images/web_console_1.png differ diff --git a/images/web_console_10.png b/images/web_console_10.png new file mode 100644 index 0000000..7245a66 Binary files /dev/null and b/images/web_console_10.png differ diff --git a/images/web_console_2.png b/images/web_console_2.png new file mode 100644 index 0000000..962887b Binary files /dev/null and b/images/web_console_2.png differ diff --git a/images/web_console_3.png b/images/web_console_3.png new file mode 100644 index 0000000..88f1767 Binary files /dev/null and b/images/web_console_3.png differ diff --git a/images/web_console_4.png b/images/web_console_4.png new file mode 100644 index 0000000..01032d2 Binary files /dev/null and b/images/web_console_4.png differ diff --git a/images/web_console_5.png b/images/web_console_5.png new file mode 100644 index 0000000..8d783d6 Binary files /dev/null and b/images/web_console_5.png differ diff --git a/images/web_console_6.png b/images/web_console_6.png new file mode 100644 index 0000000..2bf876f Binary files /dev/null and b/images/web_console_6.png differ diff --git a/images/web_console_7.png b/images/web_console_7.png new file mode 100644 index 0000000..98cc77b Binary files /dev/null and b/images/web_console_7.png differ diff --git a/images/web_console_8.png b/images/web_console_8.png new file mode 100644 index 0000000..85286b5 Binary files /dev/null and b/images/web_console_8.png differ diff --git a/images/web_console_9.png b/images/web_console_9.png new file mode 100644 index 0000000..5917225 Binary files /dev/null and b/images/web_console_9.png differ diff --git a/log_collector/log_collector.py b/log_collector/log_collector.py index 54d295a..3bdfa07 100755 --- a/log_collector/log_collector.py +++ b/log_collector/log_collector.py @@ -527,7 +527,7 @@ def run_shell_command_timeout(args, cwd=None, shell=True, env=None): """ def get_process_children(parent): - piped_process = subprocess.Popen('ps --no-headers -o pid ' + piped_process = subprocess.Popen('ps --no-headers -o pid ' # pylint: disable=R1732 '--ppid %d' % parent, shell=True, stdout=subprocess.PIPE, @@ -546,7 +546,7 @@ class Alarm(Exception): def alarm_handler(_, __): raise Alarm - piped_process = subprocess.Popen(args, + piped_process = subprocess.Popen(args, # pylint: disable=R1732 shell=shell, cwd=cwd, stdout=subprocess.PIPE, diff --git a/log_collector/pylama.ini b/log_collector/pylama.ini index 768d8e2..4c44e83 100644 --- a/log_collector/pylama.ini +++ b/log_collector/pylama.ini @@ -1,5 +1,6 @@ [pylama:pylint] max_line_length = 120 +persistent = no [pylama:pep8] -max_line_length = 120 \ No newline at end of file +max_line_length = 120 diff --git a/multi-namespace-redb/README.md b/multi-namespace-redb/README.md index cbc0fcd..b1b40ba 100644 --- a/multi-namespace-redb/README.md +++ b/multi-namespace-redb/README.md @@ -58,26 +58,22 @@ roleRef: ``` -### 3. Updating the operator deployment +### 3. Updating the operator configmap -The operator has to be deployed with a comma seperated list of namespaces it will watch for REDB objects. +The operator has to be deployed with a comma separated list of namespaces it will watch for REDB objects. -Specifically, a new environment variable is added to the operator's container (edit the redis-enterprise-deployment deployment within the operator namespace): - -```yaml -env: -... -- name: REDB_NAMESPACES -value: "comma,delimited,list,of,namespaces,to,watch" -... +Specifically, a new environment variable is added to the operator's configmap (edit the operator-environment-config configmap within the operator namespace): +* Patch the configmap by running the following command: +``` +kubectl patch configmap/operator-environment-config \ + -n \ + --type merge \ + -p '{"data":{"REDB_NAMESPACES":"comma,delimited,list,of,namespaces,to,watch"}}' ``` -### 4. Updating the admission control deployment -Admission control for REDB is recommended. If that is used, repeat step 3 to update the admission-deploy deployment (same environment variable). -_Note: make sure to edit the container spec under 'containers' and not 'initContainers'_ - -After these steps have been done, users within the specified namespaces that have permission to create REDB objects will be able to create them. The database will be created within the centrally managed redis enterprise cluster and services corresponding to the databases will be visible within the namespace. +> Note - the admission controller uses the same config map ## Additional areas for consideration * When deploying multiple Redis Enterprise Operators within the same K8s cluster, do not configure more than one of the operators to watch the same namespace. * Only configure the operator to watch a namespace once the namespace is created and configured with the role/role_binding as explained above. If configured to watch a namespace without setting those permissions or a namespace that is not created yet, the operator will fail and not perform normal operations. * The Redis Enterprise Operator creates a service named after the REDB within the REDB namespace. The service is of type "External Name". This service can only be accessed from within the K8s cluster. Configuring Load Balancer/Node Port services automatically is not supported at this time. The service type configuration within the Redis Enterprise Cluster custom resource will be ignored. The External Name service will point to another ClusterIP service created within the operator namespace, exposing the actual Redis endpoint. +* When deploying a RedisEnterpriseCluster, the operator will read environment variables from a configmap named operator-environment-config within the operator namespace. Adding new values to this configmap will reset the operator's pod and propagate the new variables to its containers. Note that this configmap can be created manually before deploying the RedisEnterpriseCluster, or the operator will automatically create it once a RedisEnterpriseCluster is deployed. \ No newline at end of file diff --git a/multi-namespace-redb/operator.yaml b/multi-namespace-redb/operator.yaml index 30b3cc4..1bbf997 100644 --- a/multi-namespace-redb/operator.yaml +++ b/multi-namespace-redb/operator.yaml @@ -15,11 +15,21 @@ spec: serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.20-4 + image: redislabs/operator:6.0.20-12 command: - redis-enterprise-operator imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 8080 env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace - name: WATCH_NAMESPACE valueFrom: fieldRef: @@ -30,8 +40,6 @@ spec: fieldPath: metadata.name - name: OPERATOR_NAME value: "redis-enterprise-operator" - - name: DATABASE_CONTROLLER_ENABLED - value: "true" - name: REDB_NAMESPACES value: "COMMA,DELIMITED,CONSUMER_NAMESPACE_LIST_TO_WATCH" resources: @@ -41,3 +49,44 @@ spec: requests: cpu: 500m memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8080 + scheme: HTTP + - name: admission + image: redislabs/operator:6.0.20-12 + command: + - /usr/local/bin/admission + imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 443 + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 1000m + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8443 + scheme: HTTPS \ No newline at end of file diff --git a/openshift.bundle.yaml b/openshift.bundle.yaml index 6de4496..8116a88 100644 --- a/openshift.bundle.yaml +++ b/openshift.bundle.yaml @@ -54,7 +54,7 @@ rules: # needed rbac rules for services controller - apiGroups: [""] resources: ["pods"] - verbs: ["get", "watch", "list", "update", "patch"] + verbs: ["get", "watch", "list", "update", "patch", "delete"] - apiGroups: [""] resources: ["services"] verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] @@ -226,6 +226,21 @@ spec: value. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/' type: object type: object + clusterCredentialSecretName: + description: Secret Name/Path to use for Cluster Credentials. If left + blank, will use cluster name + type: string + clusterCredentialSecretRole: + description: Used only if ClusterCredentialSecretType is vault, to define + vault role to be used. If blank, defaults to "redis-enterprise-operator" + type: string + clusterCredentialSecretType: + description: Type of Secret to use for ClusterCredential, Vault, Kuberetes,... + If left blank, will default ot kubernetes secrets + enum: + - vault + - kubernetes + type: string clusterRecovery: description: ClusterRecovery initiates cluster recovery when set to true. Note that this field is cleared automatically after the cluster @@ -1810,6 +1825,18 @@ spec: type: object --- +apiVersion: v1 +kind: Service +metadata: + name: admission +spec: + ports: + - port: 443 + protocol: TCP + targetPort: 8443 + selector: + name: redis-enterprise-operator +--- apiVersion: apps/v1 kind: Deployment metadata: @@ -1824,28 +1851,36 @@ spec: labels: name: redis-enterprise-operator spec: - serviceAccount: redis-enterprise-operator + serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.20-4 + image: registry.connect.redhat.com/redislabs/redis-enterprise-operator:6.0.20-12 securityContext: runAsUser: 1001 command: - redis-enterprise-operator imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 8080 env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: DEPLOY_RHEL7 value: "true" - - name: DATABASE_CONTROLLER_ENABLED - value: "true" resources: limits: cpu: 4000m @@ -1853,12 +1888,75 @@ spec: requests: cpu: 500m memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8080 + scheme: HTTP + - name: admission + image: registry.connect.redhat.com/redislabs/redis-enterprise-operator:6.0.20-12 + command: + - /usr/local/bin/admission + imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 443 + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 1000m + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8443 + scheme: HTTPS --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: redisenterprisedatabases.app.redislabs.com spec: + additionalPrinterColumns: + - JSONPath: .status.version + name: Version + type: string + - JSONPath: .status.internalEndpoints[*].port + name: Port + type: string + - JSONPath: .status.redisEnterpriseCluster + name: Cluster + type: string + - JSONPath: .status.shardStatuses.active + name: Shards + type: string + - JSONPath: .status.status + name: Status + type: string + - JSONPath: .status.specStatus + name: Spec Status + type: string + - JSONPath: .metadata.creationTimestamp + name: Age + type: date group: app.redislabs.com names: kind: RedisEnterpriseDatabase diff --git a/openshift/OLM/README.md b/openshift/OLM/README.md new file mode 100644 index 0000000..24b069f --- /dev/null +++ b/openshift/OLM/README.md @@ -0,0 +1,78 @@ +# Installing Redis Enterprise Operator on Openshift's OLM +### Overview +One way to install Redis Enterprise Operator is using Openshift's marketplace for operators - OLM.
+This document is a step-by-step guide for how to do that.
+For users who wish to use Redis Enterprise Operator with Hashicorp Vault this document also contains reference for required steps. + +1. [ Installation ](#Installation) +2. [ Troubleshooting ](#Troubleshooting) + +### Installation: + + +1. Create a project for the Redis Enterprise Operator and apply the SCC (Security Context Constraint):
+ follow steps [1-3](../../README.md#installation-on-openshift). +2. When running with Vault follow these steps: + 1. Verify the [prerequisites](../../vault/README.md#prerequisites). + 1. Configure Vault policy for the operator: [step 1-2](../../vault/README.md#deploying-the-operator:). + 2. Configure the operator's configmap: [step 3](../../vault/README.md#deploying-the-operator:). + 3. Create a Vault CA certificate secret: [step 5](../../vault/README.md#deploying-the-operator). +3. Install the Redis Enterprise Operator:
+ In your project, under the left menu bar select "Operators" -> "OperatorHub" -> select "Redis Enterprise Operator" and click on "Install Operator": + + ![Alt text](images/install_operator.png?raw=true "Title")
+ + Click "Install":
+ ![Alt text](images/install_operator2.png?raw=true "Title")
+ Next configure the Operator Installation: + - Channels: + - For the latest version use "production" Update Channel. When using the production channel, each new operator version will be used for upgrades. + - When using a version specific channel, if a newer minor version is released for that version, it will be used for upgrades. + - "Approval strategy" can be Manual or Automatic as explained here: https://docs.openshift.com/container-platform/4.7/operators/admin/olm-upgrading-operators.html + - "Installed Namespace" - choose your project name.
+ Click "Install" and wait for the Operator to complete installation, this can around 1 minute. + ![Alt text](images/install_operator3.png?raw=true "Title") +4. Create a Redis Enterprise Cluster (REC) instance:
+ 1. When Using Vault:
+ Follow steps [1-3](../../vault/README.md#creating-the-redis-enterprise-cluster) to configure the REC credentials and policy for the REC service account. + 2. Create an REC.
+ - Click "Create instance" under RedisEnterpriseCluster: + ![Alt text](images/create_rec.png?raw=true "Title") + You can then configure it using "Form View" or "YAML view".
+ - If you do not need special configuration just click "Create" to use the default values
+ - Follow the REC status: it should start with *BootstrappingFirstPod* then proceed to *Initializing* and eventually *Running*
+ ![Alt text](images/create_rec2.png?raw=true "Title") + + The relevant K8S resources would now be created - you can view them from "Resources" or navigate the menu to watch specific resources. + +5. Create a Redis Enterprise Database (REDB) instance:
+ - Note you must first have an REC in order to create a database + - Note that an admission controller for REDB objects is automatically deployed when installing the operator + 1. When Running Vault: + Create a secret for the REDB follow [step 1](../../vault/README.md#creating-an-redb). + 2. Create an REDB instance. You can then configure it using "Form View" or "YAML view" then click "Create".
+ ![Alt text](images/redb.png?raw=true "Title") + Verify it's status is "Active":
+ ![Alt text](images/redb2.png?raw=true "Title") + +### Troubleshooting + +1. Operator installation fails.
+ In case the reason is not clear you can examine your operator installation by running:
+ `kubectl get pod -n openshift-marketplace`
+ you should be able to find the relevant pod for Redis Enterprise Operator - you can then run
+ `kubectl describe pod -n openshift-marketplace` or
+ `kubectl logs pod -n openshift-marketplace`
to investigate further. +2. The Operator pod is not starting. + Connect to the Operator pod's containers and view logs.
+ One possible reason is Vault was not configured to use the correct service account, In this case the logs should indicate it. +3. After creating an REC the Stateful set does not start.
+ It is likely you did not apply the SCC as needed. Repeat step 1 and delete the first STS pod that is trying to start. + + + + + + + + diff --git a/openshift/OLM/images/create_rec.png b/openshift/OLM/images/create_rec.png new file mode 100644 index 0000000..8bcfa34 Binary files /dev/null and b/openshift/OLM/images/create_rec.png differ diff --git a/openshift/OLM/images/create_rec2.png b/openshift/OLM/images/create_rec2.png new file mode 100644 index 0000000..ebcbfd8 Binary files /dev/null and b/openshift/OLM/images/create_rec2.png differ diff --git a/openshift/OLM/images/install_operator.png b/openshift/OLM/images/install_operator.png new file mode 100644 index 0000000..8680288 Binary files /dev/null and b/openshift/OLM/images/install_operator.png differ diff --git a/openshift/OLM/images/install_operator2.png b/openshift/OLM/images/install_operator2.png new file mode 100644 index 0000000..9c804d7 Binary files /dev/null and b/openshift/OLM/images/install_operator2.png differ diff --git a/openshift/OLM/images/install_operator3.png b/openshift/OLM/images/install_operator3.png new file mode 100644 index 0000000..dee833c Binary files /dev/null and b/openshift/OLM/images/install_operator3.png differ diff --git a/openshift/OLM/images/redb.png b/openshift/OLM/images/redb.png new file mode 100644 index 0000000..940f8f0 Binary files /dev/null and b/openshift/OLM/images/redb.png differ diff --git a/openshift/OLM/images/redb2.png b/openshift/OLM/images/redb2.png new file mode 100644 index 0000000..350259e Binary files /dev/null and b/openshift/OLM/images/redb2.png differ diff --git a/openshift/operator_rhel.yaml b/openshift/operator_rhel.yaml index 40a1442..fcd4475 100644 --- a/openshift/operator_rhel.yaml +++ b/openshift/operator_rhel.yaml @@ -12,28 +12,36 @@ spec: labels: name: redis-enterprise-operator spec: - serviceAccount: redis-enterprise-operator + serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.20-4 + image: registry.connect.redhat.com/redislabs/redis-enterprise-operator:6.0.20-12 securityContext: runAsUser: 1001 command: - redis-enterprise-operator imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 8080 env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: DEPLOY_RHEL7 value: "true" - - name: DATABASE_CONTROLLER_ENABLED - value: "true" resources: limits: cpu: 4000m @@ -41,3 +49,44 @@ spec: requests: cpu: 500m memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8080 + scheme: HTTP + - name: admission + image: registry.connect.redhat.com/redislabs/redis-enterprise-operator:6.0.20-12 + command: + - /usr/local/bin/admission + imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 443 + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 1000m + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8443 + scheme: HTTPS \ No newline at end of file diff --git a/openshift/rec_rhel.yaml b/openshift/rec_rhel.yaml index 0add19a..2291b88 100644 --- a/openshift/rec_rhel.yaml +++ b/openshift/rec_rhel.yaml @@ -7,7 +7,7 @@ spec: nodes: 3 redisEnterpriseImageSpec: repository: registry.connect.redhat.com/redislabs/redis-enterprise - versionTag: 6.0.20-69.rhel7-openshift + versionTag: 6.0.20-97.rhel7-openshift redisEnterpriseServicesRiggerImageSpec: repository: registry.connect.redhat.com/redislabs/services-manager bootstrapperImageSpec: diff --git a/openshift/role.yaml b/openshift/role.yaml index f082444..7975385 100644 --- a/openshift/role.yaml +++ b/openshift/role.yaml @@ -37,7 +37,7 @@ rules: # needed rbac rules for services controller - apiGroups: [""] resources: ["pods"] - verbs: ["get", "watch", "list", "update", "patch"] + verbs: ["get", "watch", "list", "update", "patch", "delete"] - apiGroups: [""] resources: ["services"] verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] diff --git a/operator.yaml b/operator.yaml index f2b6d86..83ead3d 100644 --- a/operator.yaml +++ b/operator.yaml @@ -15,11 +15,21 @@ spec: serviceAccountName: redis-enterprise-operator containers: - name: redis-enterprise-operator - image: redislabs/operator:6.0.20-4 + image: redislabs/operator:6.0.20-12 command: - redis-enterprise-operator imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 8080 env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace - name: WATCH_NAMESPACE valueFrom: fieldRef: @@ -30,8 +40,6 @@ spec: fieldPath: metadata.name - name: OPERATOR_NAME value: "redis-enterprise-operator" - - name: DATABASE_CONTROLLER_ENABLED - value: "true" resources: limits: cpu: 4000m @@ -39,3 +47,44 @@ spec: requests: cpu: 500m memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8080 + scheme: HTTP + - name: admission + image: redislabs/operator:6.0.20-12 + command: + - /usr/local/bin/admission + imagePullPolicy: Always + envFrom: + - configMapRef: + name: operator-environment-config + optional: true + ports: + - containerPort: 443 + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 1000m + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 5 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8443 + scheme: HTTPS \ No newline at end of file diff --git a/redis_enterprise_cluster_api.md b/redis_enterprise_cluster_api.md index 9447eb8..22b35ca 100644 --- a/redis_enterprise_cluster_api.md +++ b/redis_enterprise_cluster_api.md @@ -154,6 +154,7 @@ RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster | uiAnnotations | Annotations for Redis Enterprise UI service | map[string]string | | false | | servicesRiggerSpec | Specification for service rigger | *[ServicesRiggerConfigurationSpec](#servicesriggerconfigurationspec) | | false | | license | Redis Enterprise License | string | Empty string which is a [Trial Mode licesne](https://docs.redislabs.com/latest/rs/administering/cluster-operations/settings/license-keys/#trial-mode) | false | +| licenseSecretName | K8s secret or Vault Secret Name/Path to use for Cluster License. When left blank, the license is read from the \"license\" field. Note that you can't specify non-empty values in both \"license\" and \"licenseSecretName\", only one of these fields can be used to pass the license string. The license needs to be stored under the key \"license\". | string | Empty string | false | | username | Username for the admin user of Redis Enterprise | string | demo@redislabs.com | false | | nodeSelector | Selector for nodes that could fit Redis Enterprise pod | *map[string]string | | false | | redisEnterpriseImageSpec | Specification for Redis Enterprise container image | *[ImageSpec](#imagespec) | the default Redis Enterprise image for this version | false | @@ -182,8 +183,8 @@ RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster | slaveHA | Slave high availability mechanism configuration. | *[SlaveHA](#slaveha) | | false | | clusterCredentialSecretName | Secret Name/Path to use for Cluster Credentials. If left blank, will use cluster name | string | | false | | clusterCredentialSecretType | Type of Secret to use for ClusterCredential, Vault, Kuberetes,... If left blank, will default ot kubernetes secrets | string | | true | -| clusterCredentialSecretRole | Used only if ClusterCredentialSecretType is vault, to define vault role to be used. If blank, defaults to \"redis-enterprise-operator\" | string | | true | -| vaultCASecret | K8s secret name containing Vault's CA cert - defaults to \"vault-ca-cert\" | string | | true | +| clusterCredentialSecretRole | Used only if ClusterCredentialSecretType is vault, to define vault role to be used. If blank, defaults to \"redis-enterprise-rec\" | string | | true | +| vaultCASecret | K8s secret name containing Vault's CA cert - defaults to \"vault-ca-cert\" | string | | false | | redisEnterpriseServicesConfiguration | RS Cluster optional services settings | *[RedisEnterpriseServicesConfiguration](#redisenterpriseservicesconfiguration) | | false | [Back to Table of Contents](#table-of-contents) diff --git a/redis_enterprise_database_api.md b/redis_enterprise_database_api.md index b57a2f4..65e3025 100644 --- a/redis_enterprise_database_api.md +++ b/redis_enterprise_database_api.md @@ -90,7 +90,7 @@ Redis Enterprise Module: https://redislabs.com/redis-enterprise/modules/ | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| name | The module's name e.g \"search\" for redissearch | string | | true | +| name | The module's name e.g \"ft\" for redissearch | string | | true | | version | Module's semantic version e.g \"1.6.12\" | string | | true | | config | Module command line arguments e.g. VKEY_MAX_ENTITY_COUNT 30 | string | | false | [Back to Table of Contents](#table-of-contents) diff --git a/release_info.yaml b/release_info.yaml index 66f3667..166a991 100644 --- a/release_info.yaml +++ b/release_info.yaml @@ -1,2 +1,2 @@ -operatorVersion: 6.0.20-4 -rsVersion: 6.0.20-69 +operatorVersion: 6.0.20-12 +rsVersion: 6.0.20-97 diff --git a/role.yaml b/role.yaml index f082444..7975385 100644 --- a/role.yaml +++ b/role.yaml @@ -37,7 +37,7 @@ rules: # needed rbac rules for services controller - apiGroups: [""] resources: ["pods"] - verbs: ["get", "watch", "list", "update", "patch"] + verbs: ["get", "watch", "list", "update", "patch", "delete"] - apiGroups: [""] resources: ["services"] verbs: ["get", "watch", "list", "update", "patch", "create", "delete"] diff --git a/vault/README.md b/vault/README.md index c8436f9..390b5e0 100644 --- a/vault/README.md +++ b/vault/README.md @@ -1,6 +1,8 @@ # Integrating the Redis Enterprise Operator with Hashicorp Vault ## Overview -Hashicorp Vault can be used to store secrets as an alternative to K8s secrets. Hashicorp Vault can be configured as the source of secrets used by the Redis Enterprise K8s operator. For now, the following items are supported: +Hashicorp Vault can be used to store secrets as an alternative to K8s secrets.
+Hashicorp Vault can be configured as the source of secrets used by the Redis Enterprise K8s operator. For now,
+the following items are supported: * Redis Enterprise Cluster Credentials * REDB admission TLS cert * REDB secrets: @@ -10,25 +12,36 @@ Hashicorp Vault can be used to store secrets as an alternative to K8s secrets. H * default user secret This document explains how to use Hashicorp Vault as a source for secrets. + +1. [ Prerequisites ](#prerequisites) +2. [ Deployment ](#deployment) +3. [ Deploying the operator ](#deployment_operator) +4. [ Creating the REC ](#deployment_rec) +5. [ Creating an REDB ](#deployment_redb) + > Note: when using Openshift it might be recommended to use oc instead of kubectl + ## Prerequisites * Deploy a Hashicorp Vault instance and make sure there is network access to it from the Kubernetes cluster. The solution has been tested with Hashicorp Vault v1.6.2. The Hashicorp Vault instance must be using TLS. * Configure the Hashicorp Vault Kubernetes authentication for the Kubernetes cluster the operator is being deployed. Refer to the Hashicorp Vault documentation for details. * Deploy the Hashicorp Vault agent sidecar controller on the Kubernetes cluster (https://learn.hashicorp.com/tutorials/vault/kubernetes-sidecar) -* Note that Hashicorp offers a Vault Enterprise product. The Vault Enterprise product supports namespaces. Those namespaces should not be confused with Kubernetes namespaces. This document assumes that the Hashicorp Vault instance used is the Enterprise product and a Vault namespace is used. The namespace is referred to as the below. +* Note that Hashicorp offers a Vault Enterprise product. The Vault Enterprise product supports namespaces. Those namespaces should not be confused with Kubernetes namespaces. This document assumes that the Hashicorp Vault instance used is the Enterprise product, and a Vault namespace is used. The namespace is referred to as the below. * Redis Enterprise will use a kv-v2 secret engine. Make sure it is available on the Hashicorp Vault instance (or create one if needed) and take note of the path it is mounted on, since it will be used later. + ## Deployment ### General considerations Hashicorp Vault and the Redis Enterprise Operator can be deployed in multiple scenarios that might affect the details of the process below. The document assumes the following: -* Hashicorp Vault enterprise is used, and Vault namespaces are used. If that is not the case, it is recommended to remove the namespace parameters, environment variables and annotations from the relevant directions. +* Hashicorp Vault enterprise is used, and Vault namespaces are used. If that is not the case, it is recommended to remove the namespace parameters, environment variables from the relevant directions. * Multiple Redis Enterprise Clusters are configured within the same K8s cluster, configured to authenticate to Hashicorp Vault. * To ensure privacy and avoid duplication, the K8S_NAMESPACE is appended to multiple names of Hashicorp Vault configurations. That might need to be further adjusted in cases multiple K8s clusters are used with the same K8s namespaces. +* The minimum TTL of the Vault token under the policy assigned to redis enterprise should be one hour. (see also https://learn.hashicorp.com/tutorials/vault/token-management#configure-the-token-ttl) + + ### Deploying the operator -1. Deploy the operator by applying the Redis Labs Kubernetes Operator Bundle as explained [here](../README.md) - steps 1,2 (steps 1-4 on OpenShift). Once operator is running, proceed to the steps below. Avoid creating the Redis Enterprise Cluster custom resource. -2. Configure a Hashicorp Vault policy. The policy will be used to grant the operator access to the secrets. +1. Configure a Hashicorp Vault policy. The policy will be used to grant the operator access to the secrets. - Run the following command within the Hashicorp Vault interface (use kubectl exec when Vault is deployed on Kubernetes, replace with the namespace where the operator is deployed into): + Run the following command within the Hashicorp Vault interface (use kubectl exec when Vault is deployed on Kubernetes, replace `` with the namespace where the operator is deployed into): ``` vault policy write -namespace= redisenterprise- - </*" { @@ -39,73 +52,68 @@ Hashicorp Vault and the Redis Enterprise Operator can be deployed in multiple sc } EOF ``` -3. Create a role to bind the Redis Enterprise operator service account to the policy configured in the previous step: +2. Create a role to bind the Redis Enterprise operator service account to the policy configured in the previous step: ``` - vault write -namespace= auth//role/redis-enterprise-operator- \ + vault write -namespace= auth//role/redis-enterprise-operator- \ bound_service_account_names="redis-enterprise-operator" \ bound_service_account_namespaces= \ policies=redisenterprise- ``` - > Note - replace AUTH_PATH with the path kubernetes auth is enabled in Hashicorp Vault. The default is "kubernetes" -4. Create a K8s secret containing the Certificate Authority Certificate (CACert) used to create the Hashicorp Vault instance server certificate. Name the secret vault-ca-cert. Save the ca cert to a file before running the following command: -``` -kubectl create secret generic vault-ca-cert \ - --namespace \ - --from-file=vault.ca= -``` -5. Modify the operator deployment to enable Hashicorp Vault agent sidecar container: - * Determine the Hashicorp Vault server Fully Qualified Domain Name (FQDN). If the Vault server is running with k8s, it would typically be .): - * The path the kv-2 secret engine being used is enabled on should be set as the value of the "VAULT_SECRET_ROOT" environment variable. - * The value of the VAULT_SECRET_PREFIX should be unique to the Redis Enterprise Cluster. Here we use "redisenterprise-". This value has to be consistent with Hashicorp Vault roles and policies. - * The value of the VAULT_SERVER_FQDN environment variable should be set with the Fully Qualified Domain Name of the Hashicorp Vault server. - * Save the following content to a file called operator-deployment-patch.yaml, replacing values as needed: -``` -spec: - template: + > Note - replace `` with the path kubernetes auth is enabled in Hashicorp Vault. The default is "kubernetes" +3. Create the operator's configuration in configmap called with the relevant Vault configuration:
+ edit and save this content in a file called `operator-environment-config.yaml`
+ run `kubectl apply -f operator-environment-config.yaml`
+ see notes on the parameters below. + ``` + apiVersion: v1 + kind: ConfigMap metadata: - annotations: - vault.hashicorp.com/agent-init-first: "true" - vault.hashicorp.com/agent-inject: "true" - vault.hashicorp.com/agent-inject-token: "true" - vault.hashicorp.com/ca-cert: "/vault/tls/vault.ca" - vault.hashicorp.com/tls-secret: "vault-ca-cert" - vault.hashicorp.com/role: "redis-enterprise-operator-" - vault.hashicorp.com/namespace: - vault.hashicorp.com/auth-path: auth/ - spec: - volumes: - - name: vault-ca-cert - secret: - defaultMode: 420 - secretName: vault-ca-cert - containers: - - name: redis-enterprise-operator - env: - - name: VAULT_SERVER_FQDN - value: - - name: VAULT_SERVICE_PORT_HTTPS - value: "8200" - - name: VAULT_SECRET_ROOT - value: "secret" - - name: VAULT_SECRET_PREFIX - value: "redisenterprise-" - - name: VAULT_NAMESPACE - value: - volumeMounts: - - mountPath: /vault/tls - name: vault-ca-cert - readOnly: true + name: operator-environment-config + data: + CREDENTIAL_TYPE: "vault" + VAULT_SERVER_FQDN: + VAULT_SERVICE_PORT_HTTPS: "8200" + VAULT_SECRET_ROOT: "secret" + VAULT_SECRET_PREFIX: "redisenterprise-" + VAULT_AUTH_PATH: + VAULT_NAMESPACE: + VAULT_ROLE: "redis-enterprise-operator-" ``` - > Note - the server certificate of the Hashicorp Vault instance must be signed by the Certificate Authority used within the secret. - > - * Run the following command to update the operator deployment: + * `VAULT_SERVER_FQDN`: Hashicorp Vault server Fully Qualified Domain Name (FQDN). If the Vault server is running with k8s,
+ it would typically be `.)`: + * `VAULT_SECRET_ROOT`: the path the kv-2 secret engine being used is enabled on. + * `VAULT_SECRET_PREFIX`: should be unique to the Redis Enterprise Cluster. Here we use `redisenterprise-`.
+ This value has to be consistent with Hashicorp Vault roles and policies. + * `VAULT_SERVER_FQDN`: the Fully Qualified Domain Name of the Hashicorp Vault server. + * `VAULT_AUTH_PATH`: the path kubernetes auth is enabled in Hashicorp Vault, defaults to `kubernetes` - use no leading/trailing slashes.
+ * `VAULT_NAMESPACE`: should be used when deploying with Hashicorp Vault enterprise.
+ * `VAULT_ROLE`: the Vault role you configured in the previous step, defaults to `redis-enterprise-opertaor`.
+ +4. Deploy the operator by applying the Redis Labs Kubernetes Operator Bundle as explained [here](../README.md) - steps 1,2 (steps 1-4 on OpenShift).
+ Once operator is running, proceed to the steps below. Avoid creating the Redis Enterprise Cluster custom resource. +5. Save the admission controller secret to Vault: + 1. Generate a json file with key/cert pair to be used by admission: + ``` + kubectl exec -it $(kubectl get pod -l name=redis-enterprise-operator -o jsonpath='{.items[0].metadata.name}') -- /usr/local/bin/generate-tls -infer > output.json + ``` + 2. Apply the secret to vault - execute the following within the Hashicorp Vault CLI interface (you will need to copy the file from the previous step): + ``` + vault kv put /redisenterprise-/admission-tls @output.json + ``` + > Note - the output.json file is needed for additional steps below (deployment of admission controller) + +6. Create a K8s secret containing the Certificate Authority Certificate (CACert) used to create the Hashicorp Vault instance server certificate.
+ Name the secret `vault-ca-cert` and the key `vault.ca` . Save the CA cert to a file before running the following command: ``` - kubectl patch deployment redis-enterprise-operator -n --patch "$(cat operator-deployment-patch.yaml)" + kubectl create secret generic vault-ca-cert \ + --namespace \ + --from-file=vault.ca= ``` - > Note - the change configures the sidecar injector to inject a token into the operator pod and configures the TLS settings required for secure communication with the Hashicorp Vault instance. -5. Verify that the sidecar container was created within the operator deployment. The operator pod should have 2 containers running. -### Creating the REC -1. Choose a random password. Unlike the default deployment, the operator is not creating a default password for the Redis Enterprise Cluster credentials and those need to be chosen. It is recommended to use a tool to generate a random password at least 8 characters long. + > Note - the server certificate of the Hashicorp Vault instance must be signed by the Certificate Authority used within the secret.
+ + +### Creating the Redis Enterprise Cluster +1. Choose a random password. Unlike the default deployment, the operator is not creating a default password for the Redis Enterprise Cluster credentials, and those need to be chosen. It is recommended to use a tool to generate a random password at least 8 characters long. 2. Save the password as a secret within the Hashicorp Vault instance, replace values as needed. Execute the following command within the Hashicorp Vault CLI interface: ``` vault kv put -namespace= /redisenterprise-/ username= password= @@ -129,76 +137,18 @@ spec: spec: # Add fields here nodes: 3 - clusterCredentialSecretName: rec + clusterCredentialSecretName: rec clusterCredentialSecretType: vault clusterCredentialSecretRole: redis-enterprise-rec- vaultCASecret: vault-ca-cert podAnnotations: - vault.hashicorp.com/namespace: vault.hashicorp.com/auth-path: auth/ - ``` > Note - the "clusterCredentialSecretName" field as used to query the secret from Hashicorp Vault. See section below for explanation about secret name field values. -### Deploy REDB admission controller +### Deploy REDB admission controller (for OLM this is not needed) It is not recommended to use the admission bundle here if you want to avoid creation of K8s secrets. -Instead, do a step by step installation. -1. Deploy the service - apply the following [yaml](../admission/service.yaml) -2. Generate a json file with key/cert pair to be used by admission: - ``` - kubectl exec -it $(kubectl get pod -l name=redis-enterprise-operator -o jsonpath='{.items[0].metadata.name}') -- /usr/local/bin/generate-tls -infer > output.json - ``` -3. Apply the secret to vault - execute the following within the Hashicorp Vault CLI interface (you will need to copy the file from the previous step): - ``` - vault kv put secret/redisenterprise/admission-tls @output.json - ``` -4. Apply the admission deployment [yaml](../admission/deployment.yaml) -5. Modify the admission deployment to enable Hashicorp Vault agent sidecar container: - * Save the following content to a file called admission-deployment-patch.yaml (set values as needed): - ``` - spec: - template: - metadata: - annotations: - vault.hashicorp.com/agent-init-first: "true" - vault.hashicorp.com/agent-inject: "true" - vault.hashicorp.com/agent-inject-token: "true" - vault.hashicorp.com/ca-cert: "/vault/tls/vault.ca" - vault.hashicorp.com/tls-secret: "vault-ca-cert" - vault.hashicorp.com/role: "redis-enterprise-operator-" - vault.hashicorp.com/namespace: - vault.hashicorp.com/auth-path: auth/ - spec: - serviceAccountName: redis-enterprise-operator - volumes: - - name: vault-ca-cert - secret: - defaultMode: 420 - secretName: vault-ca-cert - containers: - - name: admin - env: - - name: VAULT_SERVER_FQDN - value: - - name: VAULT_SERVICE_PORT_HTTPS - value: "8200" - - name: CREDENTIAL_TYPE - value: "vault" - - name: VAULT_SECRET_ROOT - value: "secret" - - name: VAULT_SECRET_PREFIX - value: "redisenterprise-" - - name: VAULT_NAMESPACE - value: - volumeMounts: - - mountPath: /vault/tls - name: vault-ca-cert - readOnly: true - ``` - * Patch the deployment - ``` - kubectl patch deployment admission-deploy -n --patch "$(cat admission-deployment-patch.yaml)" - ``` -6. Create the Kubernetes Validating Webhook +Instead, do a step by step installation. +1. Create the Kubernetes Validating Webhook (for OLM this is not needed) **NOTE**: One must replace REPLACE_WITH_NAMESPACE in the following command with the namespace the REC was installed into. ```shell script @@ -209,18 +159,19 @@ Instead, do a step by step installation. # create patch file cat > modified-webhook.yaml < Note - use the output.json that was created in the steps above +2. Make sure admission works, see [here](../admission/README.md#verifying-installation) for steps description -### Creating REDB + +### Creating an REDB An REDB has several secrets associate with it: 1. The password for the REDB 2. [Replica Source](../redis_enterprise_database_api.md#replicasource) (optional) @@ -235,6 +186,4 @@ Steps to create an REDB: The REC spec indicted you are running with Vault and no further configuration is required. 3. The other REDB secrets (2 to 4) should be created in this path `redisenterprise-/`. The secrets should comply with the REDB [secrets schema](https://github.com/RedisLabs/redis-enterprise-operator/blob/master/deploy/redis_enterprise_database_api.md). -> Note - when using the Redis Enterprise Vault plugin it it recommended to set defaultUser: false and associate users through ACL bindings to the REDB - - +> Note - when using the Redis Enterprise Vault plugin it recommended to set defaultUser: false and associate users through ACL bindings to the REDB