[BUG] Cannot create service of type LoadBalancer for redis and redis-sentinel components during the initial creation of Cluster

[tejaboppana]

Describe the bug
I tried creating a redis cluster with replication topology and 2 components - redis and redis-sentinel. I modified the service for both the components using this option to create a service of type LoadBalancer with annotations (AWS NLB in private subnets). The redis-sentinel component is created successfully but the creation of redis components never starts and I do not see any error in Cluster or Component status.

To Reproduce
Steps to reproduce the behavior:

1. Create a Redis Cluster with the following yaml file :

apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: test-redis-use-topology
  namespace: default
spec:
  clusterDefinitionRef: redis
  topology: replication
  terminationPolicy: Delete
  componentSpecs:
  - name: redis
    replicas: 2
    disableExporter: true
    resources:
      limits:
        cpu: '0.5'
        memory: 0.5Gi
      requests:
        cpu: '0.5'
        memory: 0.5Gi
   services:
   - name: redis
      serviceType: LoadBalancer
      annotations:
         service.beta.kubernetes.io/aws-loadbalancer-backend-protocol: tcp
         service.beta.kubernetes.io/aws-loadbalancer-nlb-target-type: instance
         service.beta.kubernetes.io/aws-loadbalancer-subnets: <SubentID comma separated>
         service.beta.kubernetes.io/aws-loadbalancer-target-group-attributes: preserve_client_ip
         service.beta.kubernetes.io/aws-loadbalancer-target-node-labels: karpenter.sh/registered=true
         service.beta.kubernetes.io/aws-loadbalancer-type: external
    volumeClaimTemplates:
    - name: data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
  - name: redis-sentinel
    replicas: 3
    resources:
      limits:
        cpu: '0.5'
        memory: 0.5Gi
      requests:
        cpu: '0.5'
        memory: 0.5Gi
    services:
    - name: redis-sentinel
      serviceType: LoadBalancer
      annotations:
         service.beta.kubernetes.io/aws-loadbalancer-backend-protocol: tcp
         service.beta.kubernetes.io/aws-loadbalancer-nlb-target-type: instance
         service.beta.kubernetes.io/aws-loadbalancer-subnets: <SubentID comma separated>
         service.beta.kubernetes.io/aws-loadbalancer-target-group-attributes: preserve_client_ip
         service.beta.kubernetes.io/aws-loadbalancer-target-node-labels: karpenter.sh/registered=true
         service.beta.kubernetes.io/aws-loadbalancer-type: external
    volumeClaimTemplates:
      - name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 10Gi

2. After I apply this yaml file , I see that only the redis-sentinel component is created and running. The redis component creation never starts.

However, if I first create the Cluster without modifying the services

apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: test-redis-use-topology
  namespace: default
spec:
  clusterDefinitionRef: redis
  topology: replication
  terminationPolicy: Delete
  componentSpecs:
  - name: redis
    replicas: 2
    disableExporter: true
    resources:
      limits:
        cpu: '0.5'
        memory: 0.5Gi
      requests:
        cpu: '0.5'
        memory: 0.5Gi
    volumeClaimTemplates:
    - name: data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
  - name: redis-sentinel
    replicas: 3
    resources:
      limits:
        cpu: '0.5'
        memory: 0.5Gi
      requests:
        cpu: '0.5'
        memory: 0.5Gi
    volumeClaimTemplates:
      - name: data
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 10Gi

Both the components are created successfully. After this if I first try to update the redis component's service and then update the redis-sentinel component's service loadbalancers are created as expected. If I update both the components at the same time, loadbalancer is created only for the redis-sentinel.

Expected behavior
When I modify the service for both the components to Loadbalancer with annotations, both the components are created successfully.

Desktop (please complete the following information):

• OS: The EC2 instances in kubernetes cluster use Bottlerocket OS.

[shanshanying]

Hi @tejaboppana

Please clarify the version of KubeBlocks first. Seems you are running Redis on KB 0.9.x ?

[tejaboppana]

Yes I am running Redis on KB 0.9.2

[shanshanying]

Yes I am running Redis on KB 0.9.2

Got it. @Y-Rookie PTAL

[Y-Rookie]

@tejaboppana
Hello, please note that if you intend to connect to the cluster using a Redis Sentinel client, Redis currently does not support LoadBalancer service type for internal cluster networking (specified in cluster.Spec.componentSpecs[x].services). In this case, you must use serviceType=NodePort when creating the cluster.

However, if you plan to connect directly to Redis without using a Redis Sentinel client, you have the option to create a LoadBalancer service after cluster creation. This can be done through cluster.Spec.service (distinct from cluster.Spec.componentSpecs[x].services), where you can set the selector to kubeblocks.io/role: primary to target the primary node.

[Y-Rookie]

@tejaboppana Hello, please note that if you intend to connect to the cluster using a Redis Sentinel client, Redis currently does not support LoadBalancer service type for internal cluster networking (specified in cluster.Spec.componentSpecs[x].services). In this case, you must use serviceType=NodePort when creating the cluster.

However, if you plan to connect directly to Redis without using a Redis Sentinel client, you have the option to create a LoadBalancer service after cluster creation. This can be done through cluster.Spec.service (distinct from cluster.Spec.componentSpecs[x].services), where you can set the selector to kubeblocks.io/role: primary to target the primary node.

And I will further investigate the issue regarding the Redis component not being created according to specifications.

[shanshanying]

@Y-Rookie another isssue we should pay attention is: why the redis cmp is not created but cluster is running. I reproduced the case on my test env: