From 15664159ae3a80064949a263a7d854d1c52997a7 Mon Sep 17 00:00:00 2001 From: Ubuntu Date: Tue, 23 Jun 2020 17:24:18 +0000 Subject: [PATCH] promoting version 6.0.6-6 --- README.md | 458 ++++++++++-------- admission.bundle.yaml | 157 ++++++ admission.openshift.bundle.yaml | 157 ++++++ admission/README.md | 148 ++++++ admission/cluster_role.yaml | 8 + admission/cluster_role_binding.yaml | 12 + admission/deployment.yaml | 75 +++ admission/deployment_openshift.yaml | 75 +++ admission/role.yaml | 8 + admission/role_binding.yaml | 11 + admission/service.yaml | 11 + admission/service_account.yaml | 4 + admission/webhook.yaml | 19 + bundle.yaml | 34 +- ...p_v1alpha1_redisenterprisedatabase_cr.yaml | 4 +- log_collector/log_collector.py | 82 +++- openshift.bundle.yaml | 42 +- openshift/operator_rhel.yaml | 11 +- openshift/redis-enterprise-cluster_rhel.yaml | 2 +- .../{sb_rbac.yaml => sb_cluster_role.yaml} | 0 openshift/sb_cluster_role_binding.yaml | 12 + openshift/with_service_broker_rhel.yaml | 2 +- operator.yaml | 12 +- rack_awareness/rack_aware_rec.yaml | 2 +- ...ator.md => redis_enterprise_cluster_api.md | 72 +-- redis_enterprise_database_api.md | 103 ++++ release_info.yaml | 3 + docs/topics.md => topics.md | 85 +++- 28 files changed, 1314 insertions(+), 295 deletions(-) create mode 100644 admission.bundle.yaml create mode 100644 admission.openshift.bundle.yaml create mode 100644 admission/README.md create mode 100644 admission/cluster_role.yaml create mode 100644 admission/cluster_role_binding.yaml create mode 100644 admission/deployment.yaml create mode 100644 admission/deployment_openshift.yaml create mode 100644 admission/role.yaml create mode 100644 admission/role_binding.yaml create mode 100644 admission/service.yaml create mode 100644 admission/service_account.yaml create mode 100644 admission/webhook.yaml rename openshift/{sb_rbac.yaml => sb_cluster_role.yaml} (100%) create mode 100644 openshift/sb_cluster_role_binding.yaml rename docs/operator.md => redis_enterprise_cluster_api.md (87%) create mode 100644 redis_enterprise_database_api.md create mode 100644 release_info.yaml rename docs/topics.md => topics.md (67%) diff --git a/README.md b/README.md index 40c5269..abed597 100644 --- a/README.md +++ b/README.md @@ -1,117 +1,131 @@ -# Deploying Redis Enterprise K8s using an operator (custom controller) +# Deploying Redis Enterprise on Kubernetes -* [Documentation](#documentation) * [Quickstart Guide](#quickstart-guide) -* [Prerequisites](#prerequisites) -* [Basic installation](#basic-installation) -* [OpenShift](#openshift) -* [Configuration Options](#configuration) -* [Private Repositories](#private-repositories) -* [Pull Secrets](#pull-secrets) -* [IPV4 enforcement](#ipv4-enforcement) + * [Prerequisites](#prerequisites) + * [Installation](#installation) + * [Installation on OpenShift](#installation-on-openshift) +* [Configuration](#configuration) + * [RedisEnterpriseCluster custom resource](#redisenterprisecluster-custom-resource) + * [Private Repositories](#private-repositories) + * [Pull Secrets](#pull-secrets) + * [Advanced Configuration](#advanced-configuration) * [Upgrade](#upgrade) ->Note: Please see the release notes for what's new in the latest release. - -## Additional Documentation - -- [Advanced Topics](docs/topics.md) -- [Resource Specification Reference](docs/operator.md) +This page describe how to deploy Redis Enterprise on Kubernetes using the Redis Enterprise Operator. High level architecture and overview of the solution can be found [HERE](https://docs.redislabs.com/latest/platforms/kubernetes/). ## Quickstart Guide ### Prerequisites -- A minimum of 3 nodes which support the following requirements -- A Kubernetes cluster (server) version of 1.9 or higher +- A Kubernetes cluster version of 1.11 or higher, with a minimum of 3 worker nodes. - A Kubernetes client (kubectl) with a matching version. For OpenShift, an OpenShift client (oc). -- For service broker - a k8s distribution that supports service catalog (see also: service-catalog) -- Access to DockerHub, RedHat Container Catalog or a private repository that can serve the required images +- Access to DockerHub, RedHat Container Catalog or a private repository that can serve the required images. -For Service Broker, please see openshift/with_service_broker_rhel.yaml. - -RedHat certified images are available on: https://access.redhat.com/containers/#/product/71f6d1bb3408bd0d The following are the images and tags for this release: +| Component | k8s | Openshift | +| --- | --- | --- | +| Redis Enterprise | `redislabs/redis:6.0.6-35` | `redislabs/redis:6.0.6-35.rhel7-openshift` | +| Operator | `redislabs/operator:6.0.6-6` | `redislabs/operator:6.0.6-6.rhel7` | +| Services Rigger | `redislabs/k8s-controller:6.0.6-6` | `redislabs/k8s-controller:6.0.6-6.rhel7` | +> * RedHat certified images are available on [Redhat Catalog](https://access.redhat.com/containers/#/product/71f6d1bb3408bd0d)
-Redis Enterprise - `redislabs/redis:5.4.14-31` or `redislabs/redis:5.4.14-31.rhel7-openshift` -Operator - `redislabs/operator:5.4.14-7` or `redislabs/operator:5.4.14-7.rhel7` +### Installation +The "Basic" installation deploys the operator (from the current release) with the default Ubuntu/Alpine base OS images from DockerHub and default settings. +This is the fastest way to get up and running with a new Redis Enterprise on Kubernetes. -Services Rigger - `redislabs/k8s-controller:5.4.14-7` or `redislabs/k8s-controller:5.4.14-7.rhel7` +1. Create a new namespace: -Service Broker - `redislabs/service-broker:78_4b9b17f` or `redislabs/service-broker:78_4b9b17f.rhel7` + ```bash + kubectl create namespace demo + ``` -## Basic installation -The basic installations deploys the operator from the current release with the default Ubuntu/Alpine base OS images from DockerHub and default settings. -This is the fastest way to get up and running with a new cluster in most environments. -Other Kubernetes distributions setup process as well as other custom configurations are referenced in this repository. + Switch context to the newly created namespace: -Note: The v1 version of the crd is the one recommended for use and referenced in yaml file names below. However, the v1alpha1 version is the only supported version for running on K8s 1.9 and 1.10. For those versions, use the relevant yamls for v1alpha1. + ```bash + kubectl config set-context --current --namespace=demo + ``` -1. Clone this repo: -```bash -git clone https://github.com/RedisLabs/redis-enterprise-k8s-docs -``` +2. Deploy the operator bundle -2. Create a new namespace: + To deploy the default installation with `kubectl`, the following command will deploy a bundle of all the yaml declarations required for the operator: -```bash -kubectl create namespace demo -``` + ```bash + kubectl apply -f bundle.yaml + ``` -Switch context to the newly created namespace: + Alternatively, to run each of the declarations of the bundle individually, run the following commands *instead* of the bundle: -```bash -kubectl config set-context --current --namespace=demo -``` + ```bash + kubectl apply -f role.yaml + kubectl apply -f role_binding.yaml + kubectl apply -f service_account.yaml + kubectl apply -f crds/app_v1_redisenterprisecluster_crd.yaml + kubectl apply -f crds/app_v1alpha1_redisenterprisedatabase_crd.yaml + kubectl apply -f operator.yaml + ``` + > Note: The rbac.yaml file used in previous releases has been broken down into three distinct files: + `role.yaml`, `role_binding.yaml` and `service_account.yaml`. + The `crd.yaml` file was renamed to `redisenterprisecluster_crd.yaml`, with the API version prepended to the filename. + Apply the `crds/app_v1alpha1_redisenterprisedatabase_crd.yaml` if managing database instances through Kubernetes API and commands is desired. -3. To deploy the default installation with `kubectl`, the following command will deploy a bundle of all the yaml declarations required for the operator: +3. Run `kubectl get deployment` and verify redis-enterprise-operator deployment is running. -```bash -kubectl apply -f bundle.yaml -``` + A typical response may look like this: -Alternatively, to run each of the declarations of the bundle individually, run the following commands *instead* of the bundle: + ```bash + NAME READY UP-TO-DATE AVAILABLE AGE + redis-enterprise-operator 1/1 1 1 2m + ``` -```bash -kubectl apply -f role.yaml -kubectl apply -f role_binding.yaml -kubectl apply -f service_account.yaml -kubectl apply -f crds/app_v1_redisenterprisecluster_crd.yaml -kubectl apply -f operator.yaml -``` -> Note: The rbac.yaml file used in previous releases has been broken down into three distinct files: -role.yaml, role_binding.yaml and service_account.yaml. -The crd.yaml file was renamed to redisenterprisecluster_crd.yaml, with the API version prepended to the filename. +4. Redis Enterprise Cluster custom resource - `RedisEnterpriseCluster` -4. Run `kubectl get deployment` and verify redis-enterprise-operator deployment is running. + Create a `RedisEnterpriseCluster`(REC) using the default configuration, which is suitable for development type deployments and works in typical scenarios. For more advanced deployment options you may choose the configuration relevant for you - see the index at the top for documentation references that cover many scenarios and the examples in the example folder. -A typical response may look like this: + ```bash + kubectl apply -f crds/app_v1_redisenterprisecluster_cr.yaml + ``` -```bash -|NAME |DESIRED | CURRENT | UP-TO-DATE | AVAILABLE | AGE| -|-------------------------|-------------------------------------------------| -|redis-enterprise-operator|1 | 1 | 1 | 1 | 2m | -``` + > Note: The redis-enterprise-cluster.yaml file was renamed to redisenterprisecluster_cr.yaml, with the API version prepended to the filename. -5. Create A Redis Enterprise Cluster using the default configuration, which is suitable for development type deployments and works in typical scenarios. For more advanced deployment options you may choose the configuration relevant for you - see the index at the top for documentation references that cover many scenarios. +5. Run ```kubectl get rec``` and verify creation was successful. `rec` is a shortcut for RedisEnterpriseCluster. + A typical response may look like this: + ``` + NAME AGE + redis-enterprise 5m + ``` -```bash -kubectl apply -f crds/app_v1_redisenterprisecluster_cr.yaml -``` +6. Redis Enterprise Database custom resource - `RedisEnterpriseDatabase` (Alpha feature) -> Note: The redis-enterprise-cluster.yaml file was renamed to redisenterprisecluster_cr.yaml, with the API version prepended to the filename. + Create a `RedisEnterpriseDatabase` (REDB) by using Custom Resource. + The Redis Enterprise Operator can be instructed to manage databases on the Redis Enterprise Cluster using the REDB custom resource. + Example: + ```yaml + cat << EOF > /tmp/redis-enterprise-database.yml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: redis-enterprise-database + spec: + redisEnterpriseCluster: + name: redis-enterprise + memorySize: 100MB + EOF + kubectl apply -f /tmp/redis-enterprise-database.yml + ``` + Replace the name of the cluster with the one used on the current namespace. + All REDB configuration options are documented [here](redis_enterprise_database_api.md). + + + > Optional: REDB admission controller (Alpha feature) + > + > When using the REDB Custom Resource Definition (Redis Enterprise Database) it is recommended to set up admission control to improve input validation and catch configuration errors before they reach the cluster. The procedure is documented [here](admission/README.md) -6. Run ```kubectl get rec``` and verify creation was successful. "rec" is a shortcut for RedisEnterpriseCluster. -A typical response may look like this: -```bash -|NAME |AGE -|redis-enterprise |5m -``` -### OpenShift + +### Installation on OpenShift The "OpenShift" installations deploys the operator from the current release with the RHEL image from DockerHub and default OpenShift settings. This is the fastest way to get up and running with a new cluster on OpenShift 3.x. @@ -121,177 +135,192 @@ Other custom configurations are referenced in this repository. 1. Create a new project: -```bash -oc new-project my-project -``` + ```bash + oc new-project my-project + ``` 2. Perform the following commands (you need cluster admin permissions for your Kubernetes cluster): -```bash -oc apply -f openshift/scc.yaml -``` + ```bash + oc apply -f openshift/scc.yaml + ``` -You should receive the following response: -`securitycontextconstraints.security.openshift.io "redis-enterprise-scc" configured` + You should receive the following response: + `securitycontextconstraints.security.openshift.io "redis-enterprise-scc" configured` 3. Provide the operator permissions for pods (substitute your project for "my-project"): -```bash -oc adm policy add-scc-to-group redis-enterprise-scc system:serviceaccounts:my-project -``` + ```bash + oc adm policy add-scc-to-group redis-enterprise-scc system:serviceaccounts:my-project + ``` 4. Deploy the OpenShift operator bundle: + > NOTE: Update the `storageClassName` setting in `openshift.bundle.yaml` (by default its set to `gp2`). -```bash -oc apply -f openshift.bundle.yaml -``` + ```bash + oc apply -f openshift.bundle.yaml + ``` -Apply the `RedisEnterpriseCluster` resource with RHEL7 based images +5. Redis Enterprise Cluster custom resource - `RedisEnterpriseCluster` -```bash -kubectl apply -f openshift/redis-enterprise-cluster_rhel.yaml -``` + Apply the `RedisEnterpriseCluster` resource with RHEL7 based images: -### Configuration: -The operator deploys with default configurations values, but those can be customized: + ```bash + kubectl apply -f openshift/redis-enterprise-cluster_rhel.yaml + ``` -Redis Image -```yaml - redisEnterpriseImageSpec: - imagePullPolicy: IfNotPresent - repository: redislabs/redis - versionTag: 5.4.14-31 -``` +6. Redis Enterprise Database custom resource - `RedisEnterpriseDatabase` (Alpha feature) -Persistence -```yaml - persistentSpec: - enabled: true - volumeSize: "10Gi" # if you don't provide default is 5 times RAM size - storageClassName: "standard" #on AWS common storage class is gp2 -``` + Create a `RedisEnterpriseDatabase` (REDB) by using Custom Resource. + The Redis Enterprise Operator can be instructed to manage databases on the Redis Enterprise Cluster using the REDB custom resource. + Example: + ```yaml + cat << EOF > /tmp/redis-enterprise-database.yml + apiVersion: app.redislabs.com/v1alpha1 + kind: RedisEnterpriseDatabase + metadata: + name: redis-enterprise-database + spec: + redisEnterpriseCluster: + name: redis-enterprise + memorySize: 100MB + EOF + kubectl apply -f /tmp/redis-enterprise-database.yml + ``` + Replace the name of the cluster with the one used on the current namespace. + All REDB configuration options are documented [here](redis_enterprise_database_api.md). -Redis Enterprise Nodes (pods) -```yaml - redisEnterpriseNodeResources: - limits: - cpu: "4000m" - memory: 4Gi - requests: - cpu: "4000m" - memory: 4Gi -``` -User Name to be used for accessing the cluster. Default is demo@redislabs.com -```yaml -username: "admin@acme.com" -``` + > Optional: REDB admission controller (Alpha feature) + > + > When using the REDB Custom Resource Definition (Redis Enterprise Database) it is recommended to set up admission controller to improve input validation and catch configuration errors before they reach the cluster. The procedure is documented [here](admission/README.md). -UI service type: Load Balancer or cluster IP (default) -```yaml -uiServiceType: LoadBalancer -``` -Extra Labels: additional labels to tag the k8s resources created during deployment -```yaml - extraLabels: - example1: "some-value" - example2: "some-value" -``` -UI annotations - add custom annotation to the UI service -```yaml - uiAnnotations: - uiAnnotation1: 'UI-annotation1' - uiAnnotation2: 'UI-Annotation2' -``` +## Configuration +### RedisEnterpriseCluster custom resource +The operator deploys a `RedisEnterpriseCluster` with default configurations values, but those can be customized in the `RedisEnterpriseCluster` spec as follow: -SideCar containers- images that will run along side the redis enterprise containers -```yaml - sideContainersSpec: - - name: sidecar - image: dockerhub_repo/repo:tag - imagePullPolicy: IfNotPresent -``` +* Redis Enterprise Image + ```yaml + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: redislabs/redis + versionTag: 6.0.6-35 + ``` -Service Broker (only for supported clusters) -```yaml - serviceBrokerSpec: - enabled: true +* Persistence + ```yaml persistentSpec: - storageClassName: "gp2" #adjust according to infrastructure -``` - -CRDB (Active Active): -*Currently supported for OpenShift* - -```yaml -activeActive: # edit values according to your cluster - apiIngressUrl: my-cluster1-api.myopenshiftcluster1.com - dbIngressSuffix: -dbsuffix1.myopenshiftcluster1.com - method: openShiftRoute -``` - -With Service Broker support (add this in addition to serviceBrokerSpec section): -```yaml -activeActive: # edit values according to your cluster - apiIngressUrl: my-cluster1-api.myopenshiftcluster1.com - dbIngressSuffix: -dbsuffix1.myopenshiftcluster1.com - method: openShiftRoute - peerClusters: - - apiIngressUrl: my-cluster2-api.myopenshiftcluster2.com - authSecret: cluster2_secret - dbIngressSuffix: -dbsuffix2.myopenshiftcluster2.com - fqdn: ..svc.cluster.local - - apiIngressUrl: my-cluster3-api.myopenshiftcluster3.com - authSecret: cluster3_secret - dbIngressSuffix: -dbsuffix3.myopenshiftcluster3.com - fqdn: ..svc.cluster.local -``` - -#### Private Repositories - -Whenever images are not pulled from DockerHub, the following configuration options must be specified: + enabled: true + volumeSize: "10Gi" # if you don't provide default is 5 times RAM size + storageClassName: "standard" #on AWS common storage class is gp2 + ``` + +* Redis Enterprise Nodes(pods) resources + ```yaml + redisEnterpriseNodeResources: + limits: + cpu: "4000m" + memory: 4Gi + requests: + cpu: "4000m" + memory: 4Gi + ``` + +* Cluster username (Default is demo@redislabs.com) + ```yaml + username: "admin@acme.com" + ``` + +* Extra Labels: Additional labels to tag the k8s resources created during deployment + ```yaml + extraLabels: + example1: "some-value" + example2: "some-value" + ``` + +* UI service type: Load Balancer or cluster IP (default) + ```yaml + uiServiceType: LoadBalancer + ``` + +* UI annotations: Add custom annotation to the UI service + ```yaml + uiAnnotations: + uiAnnotation1: 'UI-annotation1' + uiAnnotation2: 'UI-Annotation2' + ``` + +* SideCar containers: images that will run along side the redis enterprise containers + ```yaml + sideContainersSpec: + - name: sidecar + image: dockerhub_repo/repo:tag + imagePullPolicy: IfNotPresent + ``` + +* CRDB (Active Active): + > Currently supported for OpenShift* + ```yaml + activeActive: # edit values according to your cluster + apiIngressUrl: my-cluster1-api.myopenshiftcluster1.com + dbIngressSuffix: -dbsuffix1.myopenshiftcluster1.com + method: openShiftRoute + ``` + + +* IPV4 enforcement + + You might not have IPV6 support in your K8S cluster. + In this case, you could enforce the use of IPV4, by adding the following attribute to the REC spec: + ```yaml + enforceIPv4: true + ``` + Note: Setting 'enforceIPv4' to 'true' is a requirement for running REC on PKS. + + [requirements]: https://redislabs.com/redis-enterprise-documentation/administering/designing-production/hardware-requirements/ + [service-catalog]: https://kubernetes.io/docs/concepts/extend-kubernetes/service-catalog/ + +* Full detail can be found in [Redis Enterprise Cluster Custom Resource Specification](redis_enterprise_cluster_api.md). + + +### Private Repositories + +Whenever images are not pulled from DockerHub, the following configuration must be specified: In *RedisEnterpriseClusterSpec* (redis_enterprise_cluster.yaml): - *redisEnterpriseImageSpec* - *redisEnterpriseServicesRiggerImageSpec* -- *serviceBrokerSpec - imageSpec* (if deploying the Service Broker) - *bootstrapperImageSpec* Image specifications in *RedisEnterpriseClusterSpec* follow the same schema: - -| Field | Description | Scheme | Default Value | Required | -| ----- | ----------- | ------ | -------- | -------- | -| repository | Repository | string | | true | -| versionTag | | string | | true | -| imagePullPolicy | | v1.PullPolicy | | true | +>see [ImageSpec](redis_enterprise_cluster_api.md#imagespec) for full reference For example: ```yaml redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/redis - versionTag: 5.4.14-31 + versionTag: 6.0.6-35 ``` ```yaml redisEnterpriseServicesRiggerImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/k8s-controller - versionTag: 5.4.14-7 + versionTag: 6.0.6-6 ``` ```yaml bootstrapperImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/operator - versionTag: 5.4.14-7 + versionTag: 6.0.6-6 ``` In Operator Deployment spec (operator.yaml): -- containers - image For example: ```yaml @@ -300,12 +329,12 @@ spec: spec: containers: - name: redis-enterprise-operator - image: harbor.corp.local/redisenterprise/operator:5.4.14-7 + image: harbor.corp.local/redisenterprise/operator:6.0.6-6 ``` Image specification follow the [K8s Container schema](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#container-v1-core). -#### Pull secrets +### Pull secrets Private repositories which require login can be accessed by creating a pull secret and declaring it in both the *RedisEnterpriseClusterSpec* and in the Operator Deployment spec. @@ -316,7 +345,7 @@ kubectl create secret docker-registry regcred --docker-server=` is your Private Docker Registry FQDN. ([https://index.docker.io/v1/](https://index.docker.io/v1/) for DockerHub) +- `` is your Private repository FQDN. ([https://index.docker.io/v1/](https://index.docker.io/v1/) for DockerHub) - `` is your Docker username. - `` is your Docker password. - `` is your Docker email. @@ -339,20 +368,25 @@ spec: -name: regcred ``` -#### IPV4 enforcement -You might not have IPV6 support in your K8S cluster. -In this case, you could enforce the use of IPV4, by adding the following attribute to the REC spec: +### Advanced Configuration +- To configure Priority Class, Node Pool, Eviction Thresholds and other advances configuration see [topics.md](topics.md) file. +- Full [Redis Enterprise Cluster Custom Resource Specification](redis_enterprise_cluster_api.md) +- Full [Redis Enterprise Database Custom Resource Specification](redis_enterprise_database_api.md) + +

+## Upgrade +The Operator automates and simplifies the upgrade process. +The Redis Enterprise Cluster Software, and the Redis Enterprise Operator for Kubernetes versions are tightly coupled and should be upgraded together. +It is recommended to use the bundle.yaml to upgrade, as it loads all the relevant CRD documents for this version. If the updated CRDs are not loaded, the operator might fail. +There are two ways to upgrade - either set 'autoUpgradeRedisEnterprise' within the Redis Enterprise Cluster Spec to instruct the operator to automatically upgrade to the compatible version, or specify the correct Redis Enterprise image manually using the versionTag attribute. The Redis Enterprise Version compatible with this release is 6.0.6-35 + ```yaml - enforceIPv4: true + autoUpgradeRedisEnterprise: true ``` -Note: Setting 'enforceIPv4' to 'true' is a requirement for running REC on PKS. -[requirements]: https://redislabs.com/redis-enterprise-documentation/administering/designing-production/hardware-requirements/ -[service-catalog]: https://kubernetes.io/docs/concepts/extend-kubernetes/service-catalog/ +Alternatively: +```yaml + RedisEnterpriseImageSpec: + versionTag: redislabs/redis:6.0.6-35 +``` -#### Upgrade -The Operator automates and simplifies the upgrade process.
-The Redis Enterprise Cluster Software and the Redis Enterprise Operator for Kubernetes versions are tightly coupled and should be upgraded together.
-It is recommended to use the bundle.yaml to upgrade.
-The Redis Enterprise Cluster spec 'autoUpgradeRedisEnterprise' impacts the upgrade process. If set to true, the Operator will upgrade Redis Enterprise as soon as possible. If this attribute is not set, the Operator upgrades Redis Enterprise only when explicitly set by the user using the 'versionTag' attribute of the Redis Enterprise Cluster spec.
-Note: When updating the version of the operator, the StatefulSet gets updated even before the container of the Redis Enterprise Cluster gets updated, due to the Bootstrapper component. That behavior might be revisited in future versions. diff --git a/admission.bundle.yaml b/admission.bundle.yaml new file mode 100644 index 0000000..7eca5e6 --- /dev/null +++ b/admission.bundle.yaml @@ -0,0 +1,157 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: redis-enterprise-admission +--- +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +rules: + - apiGroups: ["certificates.k8s.io"] + resources: ["certificatesigningrequests"] + verbs: ["create", "watch"] +--- +kind: Role +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["*"] +--- +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +subjects: +- kind: ServiceAccount + name: redis-enterprise-admission +roleRef: + kind: Role + name: redis-enterprise-admission + apiGroup: rbac.authorization.k8s.io +--- +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +subjects: +- kind: ServiceAccount + namespace: NAMESPACE_OF_SERVICE_ACCOUNT + name: redis-enterprise-admission +roleRef: + kind: ClusterRole + name: redis-enterprise-admission + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: v1 +kind: Service +metadata: + name: admission +spec: + ports: + - port: 443 + protocol: TCP + targetPort: 8443 + selector: + app: redb-admission +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: admission-deploy +spec: + selector: + matchLabels: + app: redb-admission + template: + metadata: + labels: + app: redb-admission + spec: + serviceAccountName: redis-enterprise-admission + containers: + - name: admin + image: redislabs/operator-internal:6.0.6-6 + command: + - /usr/local/bin/admission + args: + - '-v=0' + imagePullPolicy: Always + ports: + - containerPort: 443 + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 100m + memory: 512Mi + requests: + cpu: 25m + memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 30 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8443 + scheme: HTTPS + initContainers: + - name: admin-init + image: redislabs/operator-internal:6.0.6-6 + command: + - /usr/local/bin/admission + args: + - '-generate-tls' + - '-v=1' + imagePullPolicy: Always + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + resources: + limits: + cpu: 2000m + memory: 256Mi + requests: + cpu: 100m + memory: 256Mi + +--- +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: ValidatingWebhookConfiguration +metadata: + name: redb-admission +webhooks: +- name: redb.admission.redislabs + failurePolicy: Fail + rules: + - apiGroups: ["app.redislabs.com"] + apiVersions: ["v1alpha1"] + operations: ["*"] + resources: ["redisenterprisedatabases"] + clientConfig: + service: + namespace: NAMESPACE_OF_SERVICE_ACCOUNT + name: admission + path: /admission + caBundle: "" # Fill in with BASE64 encoded signed cert + admissionReviewVersions: ["v1beta1"] +--- diff --git a/admission.openshift.bundle.yaml b/admission.openshift.bundle.yaml new file mode 100644 index 0000000..3a84ee0 --- /dev/null +++ b/admission.openshift.bundle.yaml @@ -0,0 +1,157 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: redis-enterprise-admission +--- +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +rules: + - apiGroups: ["certificates.k8s.io"] + resources: ["certificatesigningrequests"] + verbs: ["create", "watch"] +--- +kind: Role +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["*"] +--- +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +subjects: +- kind: ServiceAccount + name: redis-enterprise-admission +roleRef: + kind: Role + name: redis-enterprise-admission + apiGroup: rbac.authorization.k8s.io +--- +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: redis-enterprise-admission +subjects: +- kind: ServiceAccount + namespace: NAMESPACE_OF_SERVICE_ACCOUNT + name: redis-enterprise-admission +roleRef: + kind: ClusterRole + name: redis-enterprise-admission + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: v1 +kind: Service +metadata: + name: admission +spec: + ports: + - port: 443 + protocol: TCP + targetPort: 8443 + selector: + app: redb-admission +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: admission-deploy +spec: + selector: + matchLabels: + app: redb-admission + template: + metadata: + labels: + app: redb-admission + spec: + serviceAccountName: redis-enterprise-admission + containers: + - name: admin + image: redislabs/operator-internal:6.0.6-6.rhel7 + command: + - /usr/local/bin/admission + args: + - '-v=0' + imagePullPolicy: Always + ports: + - containerPort: 443 + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 100m + memory: 512Mi + requests: + cpu: 25m + memory: 256Mi + readinessProbe: + failureThreshold: 3 + successThreshold: 1 + periodSeconds: 30 + timeoutSeconds: 10 + httpGet: + path: /healthz + port: 8443 + scheme: HTTPS + initContainers: + - name: admin-init + image: redislabs/operator-internal:6.0.6-6.rhel7 + command: + - /usr/local/bin/admission + args: + - '-generate-tls' + - '-v=1' + imagePullPolicy: Always + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + resources: + limits: + cpu: 2000m + memory: 256Mi + requests: + cpu: 100m + memory: 256Mi + +--- +apiVersion: admissionregistration.k8s.io/v1beta1 +kind: ValidatingWebhookConfiguration +metadata: + name: redb-admission +webhooks: +- name: redb.admission.redislabs + failurePolicy: Fail + rules: + - apiGroups: ["app.redislabs.com"] + apiVersions: ["v1alpha1"] + operations: ["*"] + resources: ["redisenterprisedatabases"] + clientConfig: + service: + namespace: NAMESPACE_OF_SERVICE_ACCOUNT + name: admission + path: /admission + caBundle: "" # Fill in with BASE64 encoded signed cert + admissionReviewVersions: ["v1beta1"] +--- diff --git a/admission/README.md b/admission/README.md new file mode 100644 index 0000000..a7eec5f --- /dev/null +++ b/admission/README.md @@ -0,0 +1,148 @@ +## REDB Admission Controller Setup + +In order to enable the REDB admission controller one has to deploy multiple Kubernetes resource. + +One can either install them via the provided yaml bundle, or individually. + +##### Bundle Installation + +1. one installs them via a bundle after editing it to use the correct namespace. + +**NOTE**: One must replace REPLACE_WITH_NAMESPACE in the following command with the proper namespace + +```shell script +sed 's/NAMESPACE_OF_SERVICE_ACCOUNT/REPLACE_WITH_NAMESPACE/g' admission.bundle.yaml | kubectl create -f - +``` + +If this is the first time one is deploying the admission controller, one has to approve the CSR and setup the webhook to enable resource validation. If one has already set these up, and one is just updating the admission controller, one skips steps 2 and 3 as they are already configured correctly + +2. and waits for the CSR to ready and approves it + +wait for it to be ready to be approved + +```shell script +kubectl get csr admission-tls +``` + +and approve it once it's pending approval + +```shell script +kubectl certificate approve admission-tls +``` +or on openshift +```shell script +oc adm certificate approve admission-tls +``` + +3. and modifies the webhook to use the certificate generated + +```shell script +# save cert +CERT=`kubectl get csr admission-tls -o jsonpath='{.status.certificate}'` +# create patch file +cat > modified-webhook.yaml < modified-webhook.yaml < Note this document is auto-generated from code comments. To contribute a change please change the code comments. ## Table of Contents * [Objects](#objects) @@ -7,16 +7,13 @@ This document describes the parameters for the Redis Enterprise CRD * [ImageSpec](#imagespec) * [PeerCluster](#peercluster) * [PersistentConfigurationSpec](#persistentconfigurationspec) - * [RedisDatabase](#redisdatabase) - * [RedisDatabaseList](#redisdatabaselist) - * [RedisDatabaseSpec](#redisdatabasespec) - * [RedisDatabaseStatus](#redisdatabasestatus) * [RedisEnterpriseCluster](#redisenterprisecluster) * [RedisEnterpriseClusterList](#redisenterpriseclusterlist) * [RedisEnterpriseClusterSpec](#redisenterpriseclusterspec) * [RedisEnterpriseClusterStatus](#redisenterpriseclusterstatus) * [ServiceBrokerSpec](#servicebrokerspec) * [ServicesRiggerConfigurationSpec](#servicesriggerconfigurationspec) + * [SlaveHA](#slaveha) * [UpgradeSpec](#upgradespec) * [Enums](#enums) * [ActiveActiveMethod](#activeactivemethod) @@ -68,45 +65,12 @@ Specification for Redis Enterprise Cluster persistence | volumeSize | | resource.Quantity | | true | [Back to Table of Contents](#table-of-contents) -### RedisDatabase -TBD - -| Field | Description | Scheme | Default Value | Required | -| ----- | ----------- | ------ | -------- | -------- | -| metadata | | [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#objectmeta-v1-meta) | | true | -| spec | | [RedisDatabaseSpec](#redisdatabasespec) | | true | -| status | | [RedisDatabaseStatus](#redisdatabasestatus) | | false | -[Back to Table of Contents](#table-of-contents) - -### RedisDatabaseList -TBD - -| Field | Description | Scheme | Default Value | Required | -| ----- | ----------- | ------ | -------- | -------- | -| metadata | | [metav1.ListMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#listmeta-v1-meta) | | true | -| items | | [][RedisDatabase](#redisdatabase) | | true | -[Back to Table of Contents](#table-of-contents) - -### RedisDatabaseSpec -not implemented yet - -| Field | Description | Scheme | Default Value | Required | -| ----- | ----------- | ------ | -------- | -------- | -[Back to Table of Contents](#table-of-contents) - -### RedisDatabaseStatus -not implemented yet - -| Field | Description | Scheme | Default Value | Required | -| ----- | ----------- | ------ | -------- | -------- | -[Back to Table of Contents](#table-of-contents) - ### RedisEnterpriseCluster RedisEnterpriseCluster is the Schema for the redisenterpriseclusters API | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| metadata | | [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#objectmeta-v1-meta) | | false | +| metadata | | [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta) | | false | | spec | | [RedisEnterpriseClusterSpec](#redisenterpriseclusterspec) | | false | | status | | [RedisEnterpriseClusterStatus](#redisenterpriseclusterstatus) | | false | [Back to Table of Contents](#table-of-contents) @@ -116,7 +80,7 @@ RedisEnterpriseClusterList contains a list of RedisEnterpriseCluster | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| metadata | | [metav1.ListMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#listmeta-v1-meta) | | false | +| metadata | | [metav1.ListMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#listmeta-v1-meta) | | false | | items | | [][RedisEnterpriseCluster](#redisenterprisecluster) | | true | [Back to Table of Contents](#table-of-contents) @@ -137,10 +101,10 @@ RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster | redisEnterpriseImageSpec | Specification for Redis Enterprise container image | *[ImageSpec](#imagespec) | the default Redis Enterprise image for this version | false | | redisEnterpriseServicesRiggerImageSpec | Specification for Services Rigger container image | *[ImageSpec](#imagespec) | the default Services Rigger image for this version | false | | bootstrapperImageSpec | Specification for Bootstrapper container image | *[ImageSpec](#imagespec) | the default Bootstrapper image for this version | false | -| redisEnterpriseNodeResources | Compute resource requirements for Redis Enterprise containers | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#resourcerequirements-v1-core) | 2 CPUs and 4GB memory | false | -| bootstrapperResources | Compute resource requirements for bootstrapper containers | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#resourcerequirements-v1-core) | 0.1 CPUs and 128Mi memory | false | -| redisEnterpriseServicesRiggerResources | Compute resource requirements for Services Rigger pod | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#resourcerequirements-v1-core) | 0.5 CPU and 0.5GB memory | false | -| pullSecrets | PullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images. If specified, these secrets will be passed to individual puller implementations for them to use. More info: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ | [][v1.LocalObjectReference](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#localobjectreference-v1-core) | empty | false | +| redisEnterpriseNodeResources | Compute resource requirements for Redis Enterprise containers | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#resourcerequirements-v1-core) | 2 CPUs and 4GB memory | false | +| bootstrapperResources | Compute resource requirements for bootstrapper containers | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#resourcerequirements-v1-core) | 0.1 CPUs and 128Mi memory | false | +| redisEnterpriseServicesRiggerResources | Compute resource requirements for Services Rigger pod | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#resourcerequirements-v1-core) | 0.5 CPU and 0.5GB memory | false | +| pullSecrets | PullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images. If specified, these secrets will be passed to individual puller implementations for them to use. More info: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ | [][v1.LocalObjectReference](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#localobjectreference-v1-core) | empty | false | | persistentSpec | Specification for Redis Enterprise Cluster persistence | [PersistentConfigurationSpec](#persistentconfigurationspec) | disabled | false | | serviceBrokerSpec | Specification for Service Broker | [ServiceBrokerSpec](#servicebrokerspec) | disabled | false | | sideContainersSpec | Specification for a side container that will be added to each Redis Enterprise pod | []v1.Container | empty | false | @@ -154,9 +118,11 @@ RedisEnterpriseClusterSpec defines the desired state of RedisEnterpriseCluster | clusterRecovery | ClusterRecovery initiates cluster recovery when set to true. Note that this field is cleared automatically after the cluster is recovered | *bool | | false | | rackAwarenessNodeLabel | Node label that specifies rack ID - if specified, will create rack aware cluster. Rack awareness requires node label must exist on all nodes. Additionally, operator needs a special cluster role with permission to list nodes. | string | | false | | priorityClassName | Adds the priority class to pods managed by the operator | string | | false | -| volumes | additional volumes | []v1.Volume | | false | -| redisEnterpriseVolumeMounts | additional volume mounts within the redis enterprise containers | []v1.VolumeMount | | false | +| volumes | additional volumes | [][v1.Volume](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#volume-v1-core) | | false | +| redisEnterpriseVolumeMounts | additional volume mounts within the redis enterprise containers | [][v1.VolumeMount](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#volumemount-v1-core) | | false | | podAnnotations | pod annotations | map[string]string | | false | +| podTolerations | Tolerations that are added to all managed pods. for more information: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ | [][v1.Toleration](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#toleration-v1-core) | empty | false | +| slaveHA | Slave high availability mechanism configuration. | *[SlaveHA](#slaveha) | | false | [Back to Table of Contents](#table-of-contents) ### RedisEnterpriseClusterStatus @@ -176,7 +142,7 @@ Specification for Service Broker | enabled | Whether to deploy Service Broker | bool | | true | | persistentSpec | Persistence specification for Service Broker | [PersistentConfigurationSpec](#persistentconfigurationspec) | | false | | imageSpec | Image specification for Service Broker | *[ImageSpec](#imagespec) | | false | -| resources | Compute resource requirements for Service Broker | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#resourcerequirements-v1-core) | | false | +| resources | Compute resource requirements for Service Broker | *[v1.ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#resourcerequirements-v1-core) | | false | [Back to Table of Contents](#table-of-contents) ### ServicesRiggerConfigurationSpec @@ -184,11 +150,19 @@ Specification for service rigger | Field | Description | Scheme | Default Value | Required | | ----- | ----------- | ------ | -------- | -------- | -| databaseServiceType | Service type for access to databases | string | | true | +| databaseServiceType | Service types for access to databases. should be a comma separated list. The possible values are cluster_ip, headless and load_balancer. | string | cluster_ip,headless | true | | serviceNaming | | string | | true | | extraEnvVars | | []v1.EnvVar | | false | [Back to Table of Contents](#table-of-contents) +### SlaveHA + + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| slaveHAGracePeriod | Time in seconds between when a node fails, and when slave high availability mechanism starts relocating shards. If set to 0, will not affect cluster configuration. | *uint32 | 1800 | true | +[Back to Table of Contents](#table-of-contents) + ### UpgradeSpec Specification for upgrades of Redis Enterprise diff --git a/redis_enterprise_database_api.md b/redis_enterprise_database_api.md new file mode 100644 index 0000000..1d0512b --- /dev/null +++ b/redis_enterprise_database_api.md @@ -0,0 +1,103 @@ +# Redis Enterprise Database API +This document describes the parameters for the Redis Enterprise Database custom resource +> Note this document is auto-generated from code comments. To contribute a change please change the code comments. +## Table of Contents +* [Objects](#objects) + * [RedisEnterpriseConnection](#redisenterpriseconnection) + * [RedisEnterpriseDatabase](#redisenterprisedatabase) + * [RedisEnterpriseDatabaseList](#redisenterprisedatabaselist) + * [RedisEnterpriseDatabaseSpec](#redisenterprisedatabasespec) + * [RedisEnterpriseDatabaseStatus](#redisenterprisedatabasestatus) +* [Enums](#enums) + * [DatabasePersistence](#databasepersistence) + * [DatabaseStatus](#databasestatus) +## Objects + +### RedisEnterpriseConnection +Connection between a database, and Its Redis Enterprise Cluster + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| name | The name of the Redis Enterprise Cluster where the database should be stored. | string | | true | +[Back to Table of Contents](#table-of-contents) + +### RedisEnterpriseDatabase +RedisEnterpriseDatabase is the Schema for the redisenterprisedatabases API + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| metadata | | [metav1.ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#objectmeta-v1-meta) | | false | +| spec | | [RedisEnterpriseDatabaseSpec](#redisenterprisedatabasespec) | | false | +| status | | [RedisEnterpriseDatabaseStatus](#redisenterprisedatabasestatus) | | false | +[Back to Table of Contents](#table-of-contents) + +### RedisEnterpriseDatabaseList +RedisEnterpriseDatabaseList contains a list of RedisEnterpriseDatabase + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| metadata | | [metav1.ListMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#listmeta-v1-meta) | | false | +| items | | [][RedisEnterpriseDatabase](#redisenterprisedatabase) | | true | +[Back to Table of Contents](#table-of-contents) + +### RedisEnterpriseDatabaseSpec +RedisEnterpriseDatabaseSpec defines the desired state of RedisEnterpriseDatabase + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| redisEnterpriseCluster | Connection to Redis Enterprise Cluster | [RedisEnterpriseConnection](#redisenterpriseconnection) | | true | +| memorySize | memory size of database. use formats like 100MB, 0.1GB. minimum value in 100MB. | string | 100MB | false | +| rackAware | Whether database should be rack aware. This improves availability - more information: https://docs.redislabs.com/latest/rs/concepts/high-availability/rack-zone-awareness/ | *bool | | false | +| shardCount | Number of database server-side shards | uint16 | 1 | false | +| replication | In-memory database replication. When enabled, database will have replica shard for every master - leading to higher availability. | *bool | false | false | +| persistence | Database on-disk persistence policy | *[DatabasePersistence](#databasepersistence) | disabled | false | +| databaseSecretName | The name of the K8s secret that holds the password to the database. | string | | false | +| evictionPolicy | Database eviction policy. see more https://docs.redislabs.com/latest/rs/administering/database-operations/eviction-policy/ | string | volatile-lru | false | +| tlsMode | Require SSL authenticated and encrypted connections to the database. enabled - all incoming connections to the Database must use SSL. disabled - no incoming connection to the Database should use SSL. replica_ssl - databases that replicate from this one need to use SSL. | string | disabled | false | +| enforceClientAuthentication | Require authentication of client certificates for SSL connections to the database. | *bool | true | false | +[Back to Table of Contents](#table-of-contents) + +### RedisEnterpriseDatabaseStatus +RedisEnterpriseDatabaseStatus defines the observed state of RedisEnterpriseDatabase + +| Field | Description | Scheme | Default Value | Required | +| ----- | ----------- | ------ | -------- | -------- | +| databaseUID | Database UID provided by redis enterprise | string | | false | +| specStatus | Whether the desired specification is valid | [SpecStatusName](#specstatusname) | | false | +| status | The status of the database | [DatabaseStatus](#databasestatus) | | false | +| createdTime | Time when the database was created | string | | false | +| lastUpdated | Time when the database was last updated | string | | false | +| shardStatuses | Aggregated statuses of shards | map[string]uint16 | | false | +| lastActionUid | UID of the last action done by operator on this database | string | | false | +| lastActionStatus | Status of the last action done by operator on this database | string | | false | +| version | Database compatibility version | string | | false | +[Back to Table of Contents](#table-of-contents) +## Enums + +### DatabasePersistence +Database persistence policy. see https://docs.redislabs.com/latest/rs/concepts/data-access/persistence/ + +| Value | Description | +| ----- | ----------- | +| DatabasePersistenceDisabled | Data is not persisted | +| DatabasePersistenceAofEverySecond | Data is synced to disk every second | +| DatabasePersistenceAofAlways | Data is synced to disk with every write. | +| DatabasePersistenceSnapshotEveryHour | A snapshot of the database is created every hour | +| DatabasePersistenceSnapshotEvery6Hour | A snapshot of the database is created every 6 hours. | +| DatabasePersistenceSnapshotEvery12Hour | A snapshot of the database is created every 12 hours. | +[Back to Table of Contents](#table-of-contents) + +### DatabaseStatus +State of the Redis Enterprise Database + +| Value | Description | +| ----- | ----------- | +| DatabaseStatusPending | Database is pending creation | +| DatabaseStatusActive | Database is ready to be used | +| DatabaseStatusActiveChangePending | Database is ready to be used, but a change is pending | +| DatabaseStatusDeletePending | Database will be deleted soon | +| DatabaseStatusImportPending | Database will be imported soon | +| DatabaseStatusCreationFailed | Database creation has failed | +| DatabaseStatusRecovery | Database creation has failed | +| DatabaseStatusUnknown | Database status unknown | +[Back to Table of Contents](#table-of-contents) diff --git a/release_info.yaml b/release_info.yaml new file mode 100644 index 0000000..20a04e4 --- /dev/null +++ b/release_info.yaml @@ -0,0 +1,3 @@ +operatorVersion: 6.0.6-6 +rsVersion: 6.0.6-35 +serviceBrokerVersion: 78_4b9b17f \ No newline at end of file diff --git a/docs/topics.md b/topics.md similarity index 67% rename from docs/topics.md rename to topics.md index 5fe79ff..c8bc086 100644 --- a/docs/topics.md +++ b/topics.md @@ -10,10 +10,12 @@ - [Pod Security Policy (PSP)](#pod-security-policy-psp) - [Service Broker](#service-broker) - [Private Repositories](#private-repositories) -- [Pull secrets](#pull-secrets) +- [Pull Secrets](#pull-secrets) - [IPV4 enforcement](#ipv4-enforcement) - [Side Cars](#side-cars) - [Extra Labels](#extra-labels) +- [Resource Limits and Quotas](#resource-limits-and-quotas) +- [Custom Resource Deletion](#custom-resource-deletion) ## Guaranteed Quality of Service @@ -67,7 +69,7 @@ kind: RedisEnterpriseCluster metadata: name: example-redisenterprisecluster spec: - size: 3 + nodes: 3 nodeSelector: cloud.google.com/gke-nodepool: pool1 ``` @@ -116,7 +118,7 @@ If you use this option, you should add the policy name to REC configuration, in podSecurityPolicyName: "redis-enterprise-psp" ``` ->see [RedisEnterpriseClusterSpec](./operator.md#redisenterpriseclusterspec) for full reference +>see [RedisEnterpriseClusterSpec](operator.md#redisenterpriseclusterspec) for full reference ## Service Broker @@ -149,7 +151,7 @@ Add the `serviceBrokerSpec` Service Broker in the RedisEntepteriseCluster Spec ( storageClassName: "gp2" ``` ->see [ServiceBrokerSpec](./operator.md#servicebrokerspec) for full reference +>see [ServiceBrokerSpec](operator.md#servicebrokerspec) for full reference ## Private Repositories @@ -163,7 +165,7 @@ In *RedisEnterpriseClusterSpec* (redis_enterprise_cluster.yaml): - *bootstrapperImageSpec* Image specifications in *RedisEnterpriseClusterSpec* follow the same schema: ->see [ImageSpec](./operator.md#imagespec) for full reference +>see [ImageSpec](operator.md#imagespec) for full reference For example: @@ -171,21 +173,21 @@ For example: redisEnterpriseImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/redis - versionTag: 5.4.14-31 + versionTag: 6.0.6-35 ``` ```yaml redisEnterpriseServicesRiggerImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/k8s-controller - versionTag: 5.4.14-7 + versionTag: 6.0.6-6 ``` ```yaml bootstrapperImageSpec: imagePullPolicy: IfNotPresent repository: harbor.corp.local/redisenterprise/operator - versionTag: 5.4.14-7 + versionTag: 6.0.6-6 ``` In Operator Deployment spec (operator.yaml): @@ -198,7 +200,7 @@ spec: spec: containers: - name: redis-enterprise-operator - image: harbor.corp.local/redisenterprise/operator:5.4.14-7 + image: harbor.corp.local/redisenterprise/operator:6.0.6-6 ``` Image specification follow the [K8s Container schema](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#container-v1-core). @@ -274,3 +276,68 @@ additional labels to tag the k8s resources created during deployment example1: "some-value" example2: "some-value" ``` + +## Resource Limits and Quotas + +All the pods created by the operator are set with with a resources section to their spec, so it is possible to apply a ResourceQuota on the namespace of the Redis Enterprise Cluster. The operator itself is set with resources limit and request. +The recommended settings are set in the operator.yaml file and the bundles. The operator was tested and proved to be working in minimal workloads with the following settings in operator.yaml: + + +```yaml + resources: + limits: + cpu: 0.5 + memory: 256Mi + requests: + cpu: 0.5 + memory: 256Mi +``` + +When creating ResourceQuota, be careful when applying quotas on ConfigMaps. When testing the operator the limit was found to be met even when one ConfigMap was used, perhaps due to enforcement logic of some sort. The following ResourceQuota worked on internal testing, but might need tweaking according to the deployment scenario: +```yaml + hard: + secrets: "40" + persistentvolumeclaims: "20" + replicationcontrollers: "40" + pods: "40" + requests.storage: "120400Mi" + services: "20" + requests.memory: "43344Mi" + limits.memory: "57792Mi" + limits.cpu: "64" + requests.cpu: "48" +``` + +## Custom Resource Deletion +### REDB Deletion +The Redis Enterprise Database (REDB) object has a finalizer, to make sure the database is deleted before the REDB custom resource is removed from k8s. +The finalizer name is `finalizer.redisenterprisedatabases.app.redislabs.com`. +When a user requests the deletion of REDB (for example by running `kubectl delete redb `), the following happens: +1. K8s API adds `DeletionTimestamp` to the REDB resource. +2. The Operator notices the `DeletionTimestamp`, and sends delete request to the RS API. +3. When RS API approves delete request, the operator removes the REDB finalizer. +4. K8s cleans up the REDB resource, now that it has no finalizers. + +If for some reason the user ends up with a REDB resource that can't be deleted, because the finalizer can't be removed, they can remove the finalizer manually by editing the REDB resource. +For example, if the REDB name is `redis-enterprise-database`, here is a command to remove its finalizer manually: +```shell script +kubectl patch redb redis-enterprise-database --type=json -p '[{"op":"remove","path":"/metadata/finalizers","value":"finalizer.redisenterprisedatabases.app.redislabs.com"}]' +``` +note: In this case the database may still exist in the Redis Enterprise cluster, and should be deleted via RS GUI, or API. + +### REC Deletion +The Redis Enterprise Cluster (REC) object has a finalizer, to make sure all REDBs on that cluster are deleted before the REC custom resource is removed from k8s. +The finalizer name is `redbfinalizer.redisenterpriseclusters.app.redislabs.com`. +When a user requests the deletion of REC (for example by running `kubectl delete rec `), the following happens: +1. K8s API adds `DeletionTimestamp` to the REC resource. +2. The Operator notices the `DeletionTimestamp`, and checks if this REC has REDBs attached to it. +3. If there are such REDBs, the operator will not delete the REC, and will log the error: `Cannot delete REC, as REDBs that were stored in the cluster still exist.` +4. When there are no more REDBs attached to that REC, the operator will remove the finalizer from the REC resource. +5. K8s cleans up the REC resource, including deployments and stateful sets, now that it has no finalizers. + +If for some reason the user ends up with a REC resource that can't be deleted, because the finalizer can't be removed, they can remove the finalizer manually by editing the REC resource. +For example, if the REC name is `redis-enterprise`, here is a command to remove its finalizer manually: +```shell script +kubectl patch rec redis-enterprise --type=json -p '[{"op":"remove","path":"/metadata/finalizers","value":"redbfinalizer.redisenterpriseclusters.app.redislabs.com"}]' +``` +note: In this case the REDB resources that were attached to the REC may still exist. see [REDB Deletion](#redb-deletion) for details on how to delete these REDBs.