Merge remote-tracking branch 'origin/main' into feat/custom-security-…

…context

README.md

@@ -20,7 +20,7 @@ KubeBlocks add-ons.
 | ---- | ---- | ----------- | ----------- |
 | apecloud-mysql | apecloud-mysql-8.0.30<br>wescale-0.2.7 | ApeCloud MySQL is a database that is compatible with MySQL syntax and achieves high availability through the utilization of the RAFT consensus protocol. | xuriwuyun |
 | apecloud-postgresql | apecloud-postgresql-14.11.0 | ApeCloud PostgreSQL is a database that is compatible with PostgreSQL syntax and achieves high availability through the utilization of the RAFT consensus protocol. | ldming |
-| clickhouse | clickhouse-24.8.3 | ClickHouse is an open-source column-oriented OLAP database management system. Use it to boost your database performance while providing linear scalability and hardware efficiency. | sophon-zt |
+| clickhouse | clickhouse-22.9.4<br>clickhouse-24.8.3 | ClickHouse is an open-source column-oriented OLAP database management system. Use it to boost your database performance while providing linear scalability and hardware efficiency. | sophon-zt |
 | elasticsearch | elasticsearch-7.10.1<br>elasticsearch-7.7.1<br>elasticsearch-7.8.1<br>elasticsearch-8.1.3<br>elasticsearch-8.8.2 | Elasticsearch is a distributed, RESTful search engine optimized for speed and relevance on production-scale workloads. | iziang |
 | etcd | etcd-3.5.15<br>etcd-3.5.6 | Etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. | free6om |
 | greptimedb | greptimedb-0.3.2 | An open-source, cloud-native, distributed time-series database with PromQL/SQL/Python supported. | GreptimeTeam sh2 |
@@ -44,11 +44,11 @@ KubeBlocks add-ons.
 | pulsar | pulsar-bookies-recovery-2.11.2<br>pulsar-bookies-recovery-3.0.2<br>pulsar-bookkeeper-2.11.2<br>pulsar-bookkeeper-3.0.2<br>pulsar-broker-2.11.2<br>pulsar-broker-3.0.2<br>pulsar-proxy-2.11.2<br>pulsar-proxy-3.0.2<br>pulsar-zookeeper-2.11.2<br>pulsar-zookeeper-3.0.2 | Apache Pulsar is an open-source, distributed messaging and streaming platform built for the cloud. | caiq1nyu |
 | qdrant | qdrant-1.10.0<br>qdrant-1.5.0<br>qdrant-1.7.3<br>qdrant-1.8.1<br>qdrant-1.8.4 | High-performance, massive-scale Vector Database for the next generation of AI. | iziang |
 | rabbitmq | rabbitmq-3.10.25<br>rabbitmq-3.11.28<br>rabbitmq-3.12.14<br>rabbitmq-3.13.2<br>rabbitmq-3.13.7<br>rabbitmq-3.8.14<br>rabbitmq-3.9.29 | RabbitMQ is a reliable and mature messaging and streaming broker. | xuriwuyun |
-| redis | redis-7.0.6<br>redis-7.2.4<br>redis-cluster-7.0.6<br>redis-cluster-7.2.4<br>redis-sentinel-7.0.6<br>redis-sentinel-7.2.4<br>redis-twemproxy-0.5.0 | Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps. | Y-Rookie |
+| redis | redis-7.0.6<br>redis-7.2.4<br>redis-7.2.7<br>redis-cluster-7.0.6<br>redis-cluster-7.2.4<br>redis-cluster-7.2.7<br>redis-sentinel-7.0.6<br>redis-sentinel-7.2.4<br>redis-sentinel-7.2.7<br>redis-twemproxy-0.5.0 | Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps. | Y-Rookie |
 | risingwave | risingwave-compactor-v1.0.0<br>risingwave-compute-v1.0.0<br>risingwave-connector-v1.0.0<br>risingwave-frontend-v1.0.0<br>risingwave-meta-v1.0.0 | RisingWave is a distributed SQL streaming database that enables cost-efficient and reliable processing of streaming data. | RisingWave Labs |
 | starrocks-ce | starrocks-ce-be-1.0.0-alpha.0-3.2.2<br>starrocks-ce-be-1.0.0-alpha.0-3.3.0<br>starrocks-ce-fe-1.0.0-alpha.0-3.2.2<br>starrocks-ce-fe-1.0.0-alpha.0-3.3.0 | A Linux Foundation project, is the next-generation data platform designed to make data-intensive real-time analytics fast and easy. | yandongxiao iziang |
 | tdengine | tdengine-3.0.5 | TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps. | leon-inf |
-| tidb | tidb-6.5.10<br>tidb-7.1.5<br>tidb-7.5.2<br>tidb-pd-6.5.10<br>tidb-pd-7.1.5<br>tidb-pd-7.5.2<br>tikv-6.5.10<br>tikv-7.1.5<br>tikv-7.5.2 | TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. | csuzhangxc cjc7373 |
+| tidb | tidb-6.5.10<br>tidb-7.1.5<br>tidb-7.5.2<br>tidb-8.4.0<br>tidb-pd-6.5.10<br>tidb-pd-7.1.5<br>tidb-pd-7.5.2<br>tidb-pd-8.4.0<br>tikv-6.5.10<br>tikv-7.1.5<br>tikv-7.5.2<br>tikv-8.4.0 | TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. | csuzhangxc cjc7373 |
 | vanilla-postgresql | anilla-postgresql-12.15.0<br>anilla-postgresql-14.7.0<br>anilla-postgresql-15.6.1-138<br>anilla-postgresql-15.7.0 | Vanilla-PostgreSQL is compatible with the native PostgreSQL kernel, enabling it to quickly provide HA solutions for various variants based on the native PostgreSQL kernel. | kizuna-lek |
 | victoria-metrics | ictoria-metrics-1.0.0 | VictoriaMetrics is a fast, cost-effective and scalable monitoring solution and time series database. | sophon-zt ButterBright |
 | weaviate | weaviate-1.19.6 | Weaviate is an open-source vector database. It allows you to store data objects and vector embeddings from your favorite ML-models, and scale seamlessly into billions of data objects. | iziang |

addons-cluster/apecloud-mysql/templates/_helpers.tpl

@@ -13,7 +13,10 @@ The minimum proxy cpu cores is 0.5 and the maximum cpu cores is 64.
 {{- $proxyCPU = 64 }}
 {{- end }}
 - name: wescale-ctrl
-  {{- if not .Values.localEtcdEnabled }}
+  env:
+    - name: ETCDCTL_API
+      value: "{{ .Values.etcd.local.etcdctlApi }}"
+  {{- if eq .Values.etcd.mode "serviceRef" }}
   serviceRefs:
     {{ include "apecloud-mysql-cluster.serviceRef" . | indent 4 }}
   {{- end }}
@@ -32,7 +35,10 @@ The minimum proxy cpu cores is 0.5 and the maximum cpu cores is 64.
       cpu: 500m
       memory: 128Mi
 - name: wescale
-  {{- if not .Values.localEtcdEnabled }}
+  env:
+    - name: ETCDCTL_API
+      value: "{{ .Values.etcd.local.etcdctlApi }}"
+  {{- if eq .Values.etcd.mode "serviceRef" }}
   serviceRefs:
     {{ include "apecloud-mysql-cluster.serviceRef" . | indent 4 }}
   {{- end }}
@@ -61,7 +67,7 @@ raftGroup mode: max(replicas, 3)
 
 {{- define "apecloud-mysql-cluster.topology" }}
 {{- if and (eq .Values.mode "raftGroup") .Values.proxyEnabled }}
-  {{- if .Values.localEtcdEnabled }}
+  {{- if  eq .Values.etcd.mode "local" }}
     {{- "apecloud-mysql-proxy-etcd" }}
   {{- else }}
     {{- "apecloud-mysql-proxy" }}
@@ -73,12 +79,34 @@ raftGroup mode: max(replicas, 3)
 
 {{- define "apecloud-mysql-cluster.serviceRef" }}
 - name: etcd
-  namespace: {{ .Release.Namespace }}
-  serviceDescriptor: {{ include "kblib.clusterName" . }}-etcd-descriptor
+  namespace: {{ .Values.etcd.serviceRef.namespace }}
+  {{- if .Values.etcd.serviceRef.cluster }}
+  clusterServiceSelector:
+    cluster: {{ .Values.etcd.serviceRef.cluster.name }}
+    service:
+      component: {{ .Values.etcd.serviceRef.cluster.component }}
+      service: {{ .Values.etcd.serviceRef.cluster.service }}
+      port: {{ .Values.etcd.serviceRef.cluster.port }}
+    {{- if .Values.etcd.serviceRef.cluster.credential }}
+    credential:
+      component: {{ .Values.etcd.serviceRef.cluster.component }}
+      name: {{ .Values.etcd.serviceRef.cluster.credential }}
+    {{- end }}
+  {{- end }}
+  serviceDescriptor: {{ .Values.etcd.serviceRef.serviceDescriptor }}
 {{- end -}}
 
 {{- define "apecloud-mysql-cluster.etcdComponents" }}
 - name: etcd
+  serviceVersion: {{ .Values.etcd.local.serviceVersion | default "3.5.15" | quote }}
+  replicas: {{ .Values.etcd.local.replicas }}
+  resources:
+    requests:
+      cpu: 500m
+      memory: 500Mi
+    limits:
+      cpu: 500m
+      memory: 500Mi
   volumeClaimTemplates:
     - name: data
       spec:
@@ -87,15 +115,7 @@ raftGroup mode: max(replicas, 3)
           - ReadWriteOnce
         resources:
           requests:
-            storage: {{ .Values.etcd.resources.storage }}
-  replicas: {{ .Values.etcd.replicas }}
-  resources:
-    requests:
-      cpu: 500m
-      memory: 500Mi
-    limits:
-      cpu: 500m
-      memory: 500Mi
+            storage: {{ .Values.etcd.local.resources.storage }}
 {{- end -}}
 
 {{- define "apecloud-mysql-cluster.schedulingPolicy" }}
@@ -116,4 +136,4 @@ schedulingPolicy:
             app.kubernetes.io/instance: {{ include "kblib.clusterName" . }}
             apps.kubeblocks.io/component-name: mysql
         topologyKey: kubernetes.io/hostname
-{{- end -}}
+{{- end -}}

addons-cluster/apecloud-mysql/templates/cluster.yaml

@@ -5,30 +5,31 @@ metadata:
   namespace: {{ .Release.Namespace }}
   labels:
     {{ include "kblib.clusterLabels" . | nindent 4 }}
-  {{- if and (eq .Values.mode "raftGroup") .Values.proxyEnabled }}
-  annotations:
-    kubeblocks.io/extra-env: '{"KB_PROXY_ENABLED":"on","ETCDCTL_API": "{{ .Values.etcd.etcdctlApi}}"}'
-  {{- end }}
 spec:
   terminationPolicy: {{ .Values.extra.terminationPolicy }}
   clusterDef: apecloud-mysql # ref clusterdefinition.name
   topology: {{ include "apecloud-mysql-cluster.topology" . }}
   componentSpecs:
     - name: mysql
+      serviceVersion: {{ .Values.version }}
+      {{- if and (eq .Values.mode "raftGroup") .Values.proxyEnabled }}
+      env:
+        - name: KB_PROXY_ENABLED
+          value: "on"
+      {{- end }}
       {{- include "kblib.componentMonitor" . | indent 6 }}
       {{- include "apecloud-mysql-cluster.schedulingPolicy" . | indent 6 }}
-      {{- if and (eq .Values.mode "raftGroup") .Values.proxyEnabled (not .Values.localEtcdEnabled) }}
-      serviceVersion: {{ .Values.version }}
+      {{- if and (eq .Values.mode "raftGroup") .Values.proxyEnabled (eq .Values.etcd.mode "serviceRef") }}
       serviceRefs:
         {{ include "apecloud-mysql-cluster.serviceRef" . | indent 8 }}
       {{- end }}
       replicas: {{ include "apecloud-mysql-cluster.replicas" . }}
       {{- include "kblib.componentResources" . | indent 6 }}
       {{- include "kblib.componentStorages" . | indent 6 }}
       {{- if and (eq .Values.mode "raftGroup") .Values.proxyEnabled }}
-      {{- include "apecloud-mysql-cluster.proxyComponents" . | indent 4 }}
-        {{- if .Values.localEtcdEnabled }}
-      {{- include "apecloud-mysql-cluster.etcdComponents" . | indent 4 }}
+        {{- include "apecloud-mysql-cluster.proxyComponents" . | indent 4 }}
+        {{- if eq .Values.etcd.mode "local" }}
+          {{- include "apecloud-mysql-cluster.etcdComponents" . | indent 4 }}
         {{- end }}
       {{- end }}
 

addons-cluster/apecloud-mysql/values.schema.json

@@ -70,6 +70,79 @@
       "description": "Enable proxy or not.",
       "type": "boolean",
       "default": false
+    },
+    "etcd": {
+      "type": "object",
+      "properties": {
+        "mode": {
+          "type": "string",
+          "enum": ["serviceRef", "local"],
+          "default": "local"
+        },
+        "local": {
+          "type": "object",
+          "properties": {
+            "serviceVersion": {
+              "type": "string",
+              "default": "3.5.6"
+            },
+            "etcdctlApi": {
+              "type": "string",
+              "default": "3"
+            },
+            "replicas": {
+              "type": "integer",
+              "default": 3,
+              "minimum": 1,
+              "maximum": 3
+            },
+            "resources": {
+              "type": "object",
+              "properties": {
+                "storage": {
+                  "type": "string",
+                  "default": "20Gi"
+                }
+              }
+            }
+          }
+        },
+        "serviceRef": {
+          "type": "object",
+          "properties": {
+            "namespace": {
+              "type": "string",
+              "default": "default"
+            },
+            "cluster": {
+              "type": "object",
+              "properties": {
+                "name": {
+                  "type": "string"
+                },
+                "component": {
+                  "type": "string",
+                  "default": "etcd"
+                },
+                "service": {
+                  "type": "string",
+                  "default": "headless"
+                },
+                "port": {
+                  "type": "string",
+                  "default": "client"
+                },
+                "credential": {
+                  "type": "string"
+                }
+              }
+            },
+            "serviceDescriptor": {
+              "type": "string"
+            }
+          }
+        }
+      }
     }
   }
 }

addons-cluster/apecloud-mysql/values.yaml

@@ -39,26 +39,46 @@ storage: 20
 ##
 proxyEnabled: false
 
-## @param localEtcdEnabled if true, use the local etcd
-## if mode is standalone, localEtcdEnabled is always false
-## if mode is raftGroup, localEtcdEnabled can be true
-localEtcdEnabled: false
-
 proxy:
   storageClassName: ""
 
 ## customized default values to override kblib chart's values
 extra:
   disableExporter: true
-  rbacEnabled: true
   terminationPolicy: Delete
 
 
 etcd:
-  serviceReference:
-    endpoint: "" # ex: etcd-cluster-etcd-headless.default.svc.cluster.local:2379
-  serviceVersion: 3.5.6
-  etcdctlApi: "3" # the etcd tool etcdctl api version
-  replicas: 3 # no support other num
-  resources:
-    storage: 20Gi
+  # etcd mode
+  #
+  # @param mode The deployment mode of etcd
+  # @valid serviceRef, local
+  # @default local
+  mode: local
+  local:
+    serviceVersion: 3.5.6
+    etcdctlApi: "3" # the etcd tool etcdctl api version
+    replicas: 3 # no support other num
+    resources:
+      storage: 20Gi
+  serviceRef:
+    # -- Kubernetes namespace for etcd
+    # @default default
+    namespace: default
+    cluster:
+      # -- Name of the etcd cluster
+      # @required
+      name: etcdm-cluster
+      # -- Component name for etcd
+      # @default etcd
+      component: etcd
+      # -- Service type for etcd
+      # @default headless
+      service: headless
+      # -- Port name for etcd client
+      # @default client
+      port: client
+      # -- Credentials for etcd
+      # @optional
+      credential: ""
+    serviceDescriptor: ""

addons-cluster/clickhouse/templates/_helpers.tpl

@@ -85,8 +85,9 @@ Define clickhouse componentSpec with ComponentDefinition.
 {{- define "clickhouse-component" -}}
 - name: clickhouse
   componentDef: clickhouse-24
-  replicas: {{ $.Values.replicaCount | default 2 }}
+  replicas: {{ $.Values.replicas | default 2 }}
   disableExporter: {{ $.Values.disableExporter | default "false" }}
+  serviceVersion: {{ $.Values.version }}
   serviceAccountName: {{ include "clickhouse-cluster.serviceAccountName" $ }}
   systemAccounts:
     - name: admin
@@ -110,8 +111,9 @@ Define clickhouse keeper componentSpec with ComponentDefinition.
 {{- define "clickhouse-keeper-component" -}}
 - name: ch-keeper
   componentDef: clickhouse-keeper-24
-  replicas: {{ .Values.keeper.replicaCount }}
+  replicas: {{ .Values.keeper.replicas }}
   disableExporter: {{ $.Values.disableExporter | default "false" }}
+  serviceVersion: {{ $.Values.version }}
   {{- with .Values.keeper.tolerations }}
   tolerations: {{ .| toYaml | nindent 4 }}
   {{- end }}
@@ -149,12 +151,13 @@ Define clickhouse shardingComponentSpec with ComponentDefinition.
 */}}
 {{- define "clickhouse-sharding-component" -}}
 - name: shard
-  shards: {{ .Values.shardCount }}
+  shards: {{ .Values.shards }}
   template:
     name: clickhouse
     componentDef: clickhouse-24
-    replicas: {{ $.Values.replicaCount | default 2 }}
+    replicas: {{ $.Values.replicas | default 2 }}
     disableExporter: {{ $.Values.disableExporter | default "false" }}
+    serviceVersion: {{ $.Values.version }}
     serviceAccountName: {{ include "clickhouse-cluster.serviceAccountName" $ }}
     systemAccounts:
     - name: admin
@@ -176,11 +179,12 @@ Define clickhouse shardingComponentSpec with ComponentDefinition.
 Define clickhouse componentSpec with compatible ComponentDefinition API
 */}}
 {{- define "clickhouse-nosharding-component" -}}
-{{- range $i := until (.Values.shardCount | int) }}
+{{- range $i := until (.Values.shards | int) }}
 - name: shard-{{ $i }}
   componentDef: clickhouse-24
-  replicas: {{ $.Values.replicaCount | default 2 }}
+  replicas: {{ $.Values.replicas | default 2 }}
   disableExporter: {{ $.Values.disableExporter | default "false" }}
+  serviceVersion: {{ $.Values.version }}
   serviceAccountName: {{ include "clickhouse-cluster.serviceAccountName" $ }}
   {{- with $.Values.tolerations }}
   tolerations: {{ .| toYaml | nindent 4 }}

addons-cluster/clickhouse/templates/cluster.yaml

@@ -1,17 +1,17 @@
 {{- include "kblib.clusterCommon" . }}
-{{- if eq (.Values.shardCount | int) 1 }}
+{{- if eq (.Values.shards | int) 1 }}
   clusterDef: clickhouse
   topology: {{ .Values.mode }}
 {{- end }}
   componentSpecs:
   {{- if eq .Values.mode "cluster" }}
   {{- include "clickhouse-keeper-component" . | nindent 4 }}
   {{- end }}
-  {{- if eq (.Values.shardCount | int) 1 }}
+  {{- if eq (.Values.shards | int) 1 }}
   {{- include "clickhouse-component" . | nindent 4 }}
   {{- end }}
-  {{- if gt (.Values.shardCount | int) 1 }}
-  {{- if not .Values.sharding.enabled }}
+  {{- if gt (.Values.shards | int) 1 }}
+  {{- if not .Values.sharding }}
   {{- include "clickhouse-nosharding-component" . | nindent 4 }}
   {{- else }}
   shardings:

addons-cluster/clickhouse/templates/validate.yaml

@@ -1,5 +1,5 @@
-{{- if .Values.keeper.replicaCount }}
-  {{- if ne (mod (int .Values.keeper.replicaCount) 2) 1 }}
-    {{ fail "Zookeeper cluster does not support running with even number replicas." }}
+{{- if .Values.keeper.replicas }}
+  {{- if ne (mod (int .Values.keeper.replicas) 2) 1 }}
+    {{ fail "Clickhouse keeper does not support running with even number replicas." }}
   {{- end }}
 {{- end }}