Configure number of Live CAE replicas

By setting `with.delivery.minCae`, the number of replicas of Live CAEs is configured. There is no autoscaling.

To make the configuration more resilient, the CRD has been updated to use either integers or strings for the `minCae`, `maxCae`, and `rls` values.

Closes #14.

README.md

@@ -10,29 +10,30 @@
 
 The operator:
 * manages the creation and updating of all the Kubernetes resources required to run CoreMedia Content Cloud. Care has been taken to have sensible defaults for all parameters wherever possible.
-* can create a fresh installation from scratch, creating MariaDB and MongoDB database servers, and initialize the necessary database schemas and secrets.
+* can create a fresh installation from scratch, creating MariaDB and MongoDB database servers, and initialize the necessary database schemas and secrets, suitable for a development environment. For production, persistence should be provided, for example by using cloud services, or using other operators to provision databases.
 * can deploy against existing databases, using pre-existing secrets provided.
-* can use a custom resource definition or a config map to supply the values.
+* can use a custom resource definition or a config map to supply the values. This makes it possible to use the operator even on clusters where you cannot install cluster-wide resources.
 * deploys the CoreMedia Content Cloud components step by step. This ensures that components that require other components are only started when the dependencies have been initialized successfully.
 * imports test users and contents initially.
-* builds ingresses automatically from CAE site mappings.
-* creates random passwords for all components and configures them to use them (MariaDB, MongoDB, and UAPI/Corba).
 * run additional jobs, for example to re-import content into a running installation.
+* configures the live CAE deployment with the desired number of replicas.
+* builds ingresses automatically from CAE site mappings.
+* creates random passwords for all components and configures the components to use them (MariaDB, MongoDB, and UAPI/Corba).
+* configures Solr clustering by specifying the number of replicas to create.
 
 Planned features include:
-* Creating a scalable delivery stage automatically by simply providing the number of Replication Live Servers and minimum and maximum number of Content Application Engines.
-* Configure Solr clustering by specifying the number of replicas to create.
+* Creating a scalable delivery stage automatically by simply providing the number of Replication Live Servers.
 * Support for Traefik ingress controller and its resource types (in addition to the [kubernetes/ingress-nginx](https://github.com/kubernetes/ingress-nginx)).
 * Admission webhook that verifies consistency of the custom resource, and can migrate between CRD versions.
 
 ## Quick Links
 
 * [Helm chart cmcc-operator](charts/cmcc-operator) to install the operator
 * [Helm chart cmcc](charts/cmcc) to create a CoreMedia Content Cloud deployment with the operator
-* [CoreMediaContentClouds custom resource documentation](docs/custom-resource.md)
+* [Configuring provisioning through the CoreMediaContentClouds custom resource](docs/custom-resource.md): complete description of all options
 * [Installing the Operator](#preparing-your-cluster-and-installing-the-operator)
-* [Using the Operator to create a CoreMedia installation](#using-the-operator)
-* [Customizing the CMCC Operator](docs/customizing-the-operator.md) information for developers
+* [Using the Operator to create a CoreMedia installation](#using-the-operator): quick start
+* [Customizing the CMCC Operator](docs/customizing-the-operator.md): information for developers
 * [ghcr.io/t-systems-mms/cmcc-operator/cmcc-operator](https://github.com/T-Systems-MMS/cmcc-operator/pkgs/container/cmcc-operator%2Fcmcc-operator) Docker Image
 
 ## Preparing Your Cluster and Installing the Operator
@@ -103,7 +104,7 @@ kubectl apply -f https://raw.githubusercontent.com/T-Systems-MMS/cmcc-operator/m
 
 ### Using the Custom Resource Definition
 
-You need to add the [Custom Resource Definition](k8s/cmcc-crd.yaml) `CoreMediaContentClouds` (or `cmcc` for short) to the cluster, and create a number of object for the operator: a ClusterRole, a ClusterRoleMapping, a ServiceAccount, and a Deployment for the operator. An example can be found in [`k8s/cmcc-operator.yaml`](k8s/cmcc-operator.yaml).
+You need to add the [Custom Resource Definition](k8s/cmcc-crd.yaml) `CoreMediaContentClouds` (or `cmcc` for short) to the cluster, and create a number of objects for the operator: a ClusterRole, a ClusterRoleMapping, a ServiceAccount, and a Deployment for the operator. An example can be found in [`k8s/cmcc-operator.yaml`](k8s/cmcc-operator.yaml).
 
 
 ### Using a Config Map

build.gradle

@@ -27,7 +27,7 @@ compileJava {
 dependencies {
     implementation 'org.springframework.boot:spring-boot-starter-actuator'
     implementation 'org.springframework.boot:spring-boot-starter-web'
-    implementation 'io.javaoperatorsdk:operator-framework-spring-boot-starter:2.0.1'
+    implementation 'io.javaoperatorsdk:operator-framework-spring-boot-starter:2.2.0'
     implementation 'org.bouncycastle:bcpkix-jdk15on:1.70'
     // required for the fabric8 k8s client to grok k3d certificates
     implementation 'org.springframework.boot:spring-boot-starter'
@@ -38,7 +38,7 @@ dependencies {
     annotationProcessor 'org.projectlombok:lombok'
     annotationProcessor 'io.fabric8:crd-generator-apt:5.12.1'
     testImplementation 'org.springframework.boot:spring-boot-starter-test'
-    testImplementation 'io.javaoperatorsdk:operator-framework-spring-boot-starter-test:2.0.1'
+    testImplementation 'io.javaoperatorsdk:operator-framework-spring-boot-starter-test:2.2.0'
 }
 
 tasks.named('test') {

charts/cmcc-operator/templates/crd.yaml

@@ -315,14 +315,23 @@ spec:
                     description: Create default components for the delivery stage
                     properties:
                       rls:
+                        anyOf:
+                        - type: integer
+                        - type: string
                         description: Number of RLS to create
-                        type: integer
+                        x-kubernetes-int-or-string: true
                       minCae:
+                        anyOf:
+                        - type: integer
+                        - type: string
                         description: Minimum number of CAEs per RLS
-                        type: integer
+                        x-kubernetes-int-or-string: true
                       maxCae:
+                        anyOf:
+                        - type: integer
+                        - type: string
                         description: Maximum number of CAEs per RLS
-                        type: integer
+                        x-kubernetes-int-or-string: true
                     type: object
                   management:
                     description: Create default components for the management stage

k8s/cmcc-crd.yaml

@@ -314,14 +314,23 @@ spec:
                     description: Create default components for the delivery stage
                     properties:
                       rls:
+                        anyOf:
+                        - type: integer
+                        - type: string
                         description: Number of RLS to create
-                        type: integer
+                        x-kubernetes-int-or-string: true
                       minCae:
+                        anyOf:
+                        - type: integer
+                        - type: string
                         description: Minimum number of CAEs per RLS
-                        type: integer
+                        x-kubernetes-int-or-string: true
                       maxCae:
+                        anyOf:
+                        - type: integer
+                        - type: string
                         description: Maximum number of CAEs per RLS
-                        type: integer
+                        x-kubernetes-int-or-string: true
                     type: object
                   management:
                     description: Create default components for the management stage

src/main/java/com/tsystemsmms/cmcc/cmccoperator/components/AbstractComponent.java

@@ -10,15 +10,12 @@
 
 package com.tsystemsmms.cmcc.cmccoperator.components;
 
-import com.tsystemsmms.cmcc.cmccoperator.customresource.ConfigMapCustomResource;
 import com.tsystemsmms.cmcc.cmccoperator.crds.*;
 import com.tsystemsmms.cmcc.cmccoperator.customresource.CustomResource;
 import com.tsystemsmms.cmcc.cmccoperator.targetstate.TargetState;
 import com.tsystemsmms.cmcc.cmccoperator.utils.EnvVarSet;
 import io.fabric8.kubernetes.api.model.*;
-import io.fabric8.kubernetes.api.model.apps.StatefulSet;
-import io.fabric8.kubernetes.api.model.apps.StatefulSetBuilder;
-import io.fabric8.kubernetes.api.model.apps.StatefulSetSpecBuilder;
+import io.fabric8.kubernetes.api.model.apps.*;
 import io.fabric8.kubernetes.client.KubernetesClient;
 import lombok.Getter;
 import lombok.Setter;
@@ -236,6 +233,35 @@ public StatefulSet buildStatefulSet() {
                 .build();
     }
 
+    /**
+     * Create the StatefulSet for reconciliation.
+     *
+     * @return the created StatefulSet.
+     */
+    public Deployment buildDeployment(int replicas) {
+        return new DeploymentBuilder()
+                .withMetadata(getResourceMetadata())
+                .withSpec(new DeploymentSpecBuilder()
+                        .withReplicas(replicas)
+                        .withSelector(new LabelSelectorBuilder()
+                                .withMatchLabels(getSelectorLabels())
+                                .build())
+                        .withTemplate(new PodTemplateSpecBuilder()
+                                .withMetadata(new ObjectMetaBuilder()
+                                        .withLabels(getSelectorLabels())
+                                        .build())
+                                .withSpec(new PodSpecBuilder()
+                                        .withContainers(buildContainers())
+                                        .withInitContainers(getInitContainers())
+                                        .withSecurityContext(getPodSecurityContext())
+                                        .withTerminationGracePeriodSeconds(getTerminationGracePeriodSeconds())
+                                        .withVolumes(getVolumes())
+                                        .build())
+                                .build())
+                        .build())
+                .build();
+    }
+
     public long getTerminationGracePeriodSeconds() {
         return 5L;
     }

src/main/java/com/tsystemsmms/cmcc/cmccoperator/components/corba/CAEComponent.java

@@ -10,23 +10,25 @@
 
 package com.tsystemsmms.cmcc.cmccoperator.components.corba;
 
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.tsystemsmms.cmcc.cmccoperator.components.Component;
 import com.tsystemsmms.cmcc.cmccoperator.components.HasMongoDBClient;
 import com.tsystemsmms.cmcc.cmccoperator.components.HasService;
 import com.tsystemsmms.cmcc.cmccoperator.components.HasSolrClient;
 import com.tsystemsmms.cmcc.cmccoperator.crds.ClientSecretRef;
 import com.tsystemsmms.cmcc.cmccoperator.crds.ComponentSpec;
+import com.tsystemsmms.cmcc.cmccoperator.crds.Milestone;
 import com.tsystemsmms.cmcc.cmccoperator.crds.SiteMapping;
 import com.tsystemsmms.cmcc.cmccoperator.targetstate.CustomResourceConfigError;
 import com.tsystemsmms.cmcc.cmccoperator.targetstate.TargetState;
 import com.tsystemsmms.cmcc.cmccoperator.utils.EnvVarSet;
 import io.fabric8.kubernetes.api.model.*;
+import io.fabric8.kubernetes.api.model.apps.Deployment;
 import io.fabric8.kubernetes.client.KubernetesClient;
+import io.fabric8.kubernetes.client.dsl.RollableScalableResource;
 import lombok.extern.slf4j.Slf4j;
 
-import java.util.HashMap;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Map;
+import java.util.*;
 
 import static com.tsystemsmms.cmcc.cmccoperator.utils.Utils.concatOptional;
 
@@ -37,9 +39,12 @@ public class CAEComponent extends CorbaComponent implements HasMongoDBClient, Ha
     public static final String KIND_PREVIEW = "preview";
     public static final String SOLR_COLLECTION_LIVE = "live";
     public static final String SOLR_COLLECTION_PREVIEW = "preview";
+    public static final String EXTRA_REPLICAS = "replicas";
 
     String servletPathPattern;
 
+    private int replicas = 1;
+
     public CAEComponent(KubernetesClient kubernetesClient, TargetState targetState, ComponentSpec componentSpec) {
         super(kubernetesClient, targetState, componentSpec, "cae-preview");
 
@@ -64,8 +69,19 @@ public CAEComponent(KubernetesClient kubernetesClient, TargetState targetState,
                 UAPI_CLIENT_SECRET_REF_KIND, "webserver"
         ));
         servletPathPattern = String.join("|", getDefaults().getServletNames());
+        if (componentSpec.getExtra().containsKey(EXTRA_REPLICAS))
+            replicas = Integer.parseInt(componentSpec.getExtra().get(EXTRA_REPLICAS));
     }
 
+    @Override
+    public Component updateComponentSpec(ComponentSpec newCs) {
+        super.updateComponentSpec(newCs);
+        if (newCs.getExtra().containsKey(EXTRA_REPLICAS))
+            replicas = Integer.parseInt(newCs.getExtra().get(EXTRA_REPLICAS));
+        return this;
+    }
+
+
     @Override
     public void requestRequiredResources() {
         super.requestRequiredResources();
@@ -76,7 +92,7 @@ public void requestRequiredResources() {
     @Override
     public List<HasMetadata> buildResources() {
         List<HasMetadata> resources = new LinkedList<>();
-        resources.add(buildStatefulSet());
+        resources.add(buildDeployment(replicas));
         resources.add(buildService());
         resources.add(buildPvc());
         return resources;
@@ -191,4 +207,14 @@ public List<VolumeMount> getVolumeMounts() {
 
         return volumeMounts;
     }
+
+    @Override
+    public Optional<Boolean> isReady() {
+        if (Milestone.compareTo(getCmcc().getStatus().getMilestone(), getComponentSpec().getMilestone()) < 0)
+            return Optional.empty();
+        RollableScalableResource<Deployment> deployment =
+                getKubernetesClient().apps().deployments().inNamespace(getNamespace()).withName(getTargetState().getResourceNameFor(this));
+        return Optional.of(deployment.isReady());
+    }
+
 }

src/main/java/com/tsystemsmms/cmcc/cmccoperator/crds/WithOptions.java

@@ -11,6 +11,7 @@
 package com.tsystemsmms.cmcc.cmccoperator.crds;
 
 import com.fasterxml.jackson.annotation.JsonPropertyDescription;
+import io.fabric8.kubernetes.api.model.IntOrString;
 import lombok.Data;
 
 @Data
@@ -30,10 +31,10 @@ public class WithOptions {
     @Data
     public static class WithDelivery {
         @JsonPropertyDescription("Number of RLS to create")
-        int rls = 0;
+        IntOrString rls = new IntOrString(0);
         @JsonPropertyDescription("Minimum number of CAEs per RLS")
-        int minCae = 0;
+        IntOrString minCae = new IntOrString(0);
         @JsonPropertyDescription("Maximum number of CAEs per RLS")
-        int maxCae = 0;
+        IntOrString maxCae = new IntOrString(0);
     }
 }

src/main/java/com/tsystemsmms/cmcc/cmccoperator/targetstate/DefaultTargetState.java

@@ -10,6 +10,8 @@
 
 package com.tsystemsmms.cmcc.cmccoperator.targetstate;
 
+import com.tsystemsmms.cmcc.cmccoperator.components.corba.CAEComponent;
+import com.tsystemsmms.cmcc.cmccoperator.crds.WithOptions;
 import com.tsystemsmms.cmcc.cmccoperator.customresource.CustomResource;
 import com.tsystemsmms.cmcc.cmccoperator.components.ComponentSpecBuilder;
 import com.tsystemsmms.cmcc.cmccoperator.components.generic.MongoDBComponent;
@@ -25,8 +27,11 @@
 import org.springframework.beans.factory.BeanFactory;
 
 import java.util.List;
+import java.util.Map;
 import java.util.Optional;
 
+import static com.tsystemsmms.cmcc.cmccoperator.utils.Utils.getInt;
+
 /**
  * Create the runtime config based on the CRD data.
  */
@@ -85,14 +90,17 @@ public void convergeDefaultComponents() {
             ));
         }
 
-        if (cmcc.getSpec().getWith().getDelivery().getRls() != 0
-                || cmcc.getSpec().getWith().getDelivery().getMinCae() > 1
-                || cmcc.getSpec().getWith().getDelivery().getMaxCae() > cmcc.getSpec().getWith().getDelivery().getMinCae()) {
+        WithOptions.WithDelivery delivery = cmcc.getSpec().getWith().getDelivery();
+        if (getInt(delivery.getRls()) != 0
+                || getInt(delivery.getMaxCae()) > getInt(delivery.getMinCae())) {
             throw new RuntimeException("Unable to configure RLS and HPA, not implemented yet");
         }
-        if (cmcc.getSpec().getWith().getDelivery().getMinCae() == 1) {
+        if (getInt(delivery.getMinCae()) > 0) {
+            Map<String, String> liveCaeExtra = Map.of(
+                    CAEComponent.EXTRA_REPLICAS, String.valueOf(getInt(delivery.getMinCae()))
+            );
             componentCollection.addAll(List.of(
-                    ComponentSpecBuilder.ofType("cae").withKind("live").build()
+                    ComponentSpecBuilder.ofType("cae").withKind("live").withExtra(liveCaeExtra).build()
             ));
         }
     }

src/main/java/com/tsystemsmms/cmcc/cmccoperator/utils/Utils.java

@@ -13,6 +13,7 @@
 import com.fasterxml.jackson.databind.ObjectMapper;
 import io.fabric8.kubernetes.api.model.EnvVar;
 import io.fabric8.kubernetes.api.model.EnvVarSource;
+import io.fabric8.kubernetes.api.model.IntOrString;
 import io.fabric8.kubernetes.api.model.SecretKeySelector;
 import lombok.SneakyThrows;
 
@@ -172,4 +173,16 @@ public static boolean booleanOf(Object o, boolean def) {
         }
         return l != 0 || s.equals("on") || s.equals("true") || s.equals("yes");
     }
+
+    /**
+     * Returns the integer value from a IntOrString, converting a string to integer if necessary.
+     *
+     * @param v value
+     * @return integer value
+     */
+    public static int getInt(IntOrString v) {
+        if (v.getIntVal() != null)
+            return v.getIntVal();
+        return Integer.parseInt(v.getStrVal());
+    }
 }