Skip to content

Commit

Permalink
feat: support running stream-processors as statefulsets with persiste…
Browse files Browse the repository at this point in the history 
…nce (#324)

* feat: support running stream-processors as statefulsets with persistence

* fix: added serviceName
  • Loading branch information
@chgl
chgl authored Apr 29, 2024
1 parent 3653fb8 commit def759f
Show file tree
Hide file tree
Showing 7 changed files with 161 additions and 19 deletions.
6 changes: 6 additions & 0 deletions charts/stream-processors/Chart.lock
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
dependencies:
- name: common
repository: oci://registry-1.docker.io/bitnamicharts
version: 2.19.1
digest: sha256:5f77c3af0e0885af1871f79c1f3283bfb25f5ee44fb33d7cbb645f3fbb43de5d
generated: "2024-04-27T21:08:50.334425347+02:00"
13 changes: 10 additions & 3 deletions charts/stream-processors/Chart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,20 @@ keywords:
- kafka
- strimzi kafka operator
- stream processing
version: 1.6.1
version: 1.7.0-beta.1
dependencies:
- name: common
version: 2.19.1
repository: oci://registry-1.docker.io/bitnamicharts
annotations:
artifacthub.io/recommendations: |
- url: https://artifacthub.io/packages/helm/strimzi/strimzi-kafka-operator
- url: https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack
# When using the list of objects option the valid supported kinds are
# added, changed, deprecated, removed, fixed and security.
artifacthub.io/changes: |
- kind: fixed
description: "values-test security context"
- kind: added
description: "option for persistence and running as a StatefulSet"
- kind: added
description: "mount /tmp as an emtpyDir by default"
description: "changed default ContainerSecurityContext to use default seccompProfile"
46 changes: 32 additions & 14 deletions charts/stream-processors/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,17 @@ This chart deploys the Kafka Stream Processors on a [Kubernetes](http://kubernet
- Helm v3
- Strimzi Kafka Operator installed

To install a simple Kafka cluster after the Strimzi Kafka Operator is installed, run:
To install a simple Kafka cluster using the Strimzi Operator, run:

```sh
helm upgrade --install strimzi-operator oci://quay.io/strimzi-helm/strimzi-kafka-operator
kubectl apply -f https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/main/examples/kafka/kafka-ephemeral-single.yaml

# Optional: install the Kube Prometheus stack to test the ServiceMonitor integration:
helm upgrade --install kube-prom-stack oci://ghcr.io/prometheus-community/charts/kube-prometheus-stack \
--set=kubeStateMetrics.enabled=false \
--set=nodeExporter.enabled=false \
--set=grafana.enabled=false
```

## Installing the Chart
Expand Down Expand Up @@ -50,19 +57,30 @@ The command removes all the Kubernetes components associated with the chart and

The following table lists the configurable parameters of the `stream-processors` chart and their default values.

| Parameter | Description | Default |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| strimziClusterName | name of the strimzi cluster. Used to construct the bootstrap server URL. | <code>my-cluster</code> |
| securityProtocol | The Kafka security protocol to use. See <https://kafka.apache.org/26/javadoc/org/apache/kafka/common/security/auth/SecurityProtocol.html> for a list of supported values. | <code>SSL</code> |
| nameOverride | | <code>""</code> |
| fullnameOverride | | <code>""</code> |
| securityContext.allowPrivilegeEscalation | | <code>false</code> |
| securityContext.privileged | | <code>false</code> |
| securityContext.runAsNonRoot | | <code>true</code> |
| securityContext.runAsUser | | <code>11111</code> |
| securityContext.runAsGroup | | <code>11111</code> |
| defaultReplicaCount | sets the replicas value for all processor deployments unless overridden on a per-processor level as `.replicaCount` | <code>1</code> |
| defaultRevisionHistoryLimit | sets the revisionHistoryLimit value for all processor deployments unless overridden on a per-processor level as `.revisionHistoryLimit` | <code>10</code> |
| Parameter | Description | Default |
| -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------- |
| strimziClusterName | name of the strimzi cluster. Used to construct the bootstrap server URL. | <code>my-cluster</code> |
| securityProtocol | The Kafka security protocol to use. See <https://kafka.apache.org/26/javadoc/org/apache/kafka/common/security/auth/SecurityProtocol.html> for a list of supported values. | <code>SSL</code> |
| nameOverride | | <code>""</code> |
| fullnameOverride | | <code>""</code> |
| securityContext.allowPrivilegeEscalation | | <code>false</code> |
| securityContext.privileged | | <code>false</code> |
| securityContext.readOnlyRootFilesystem | | <code>false</code> |
| securityContext.runAsNonRoot | | <code>true</code> |
| securityContext.runAsUser | | <code>11111</code> |
| securityContext.runAsGroup | | <code>11111</code> |
| securityContext.seccompProfile.type | | <code>RuntimeDefault</code> |
| defaultReplicaCount | sets the replicas value for all processor deployments unless overridden on a per-processor level as `.replicaCount` | <code>1</code> |
| defaultRevisionHistoryLimit | sets the revisionHistoryLimit value for all processor deployments unless overridden on a per-processor level as `.revisionHistoryLimit` | <code>10</code> |
| default.persistence.enabled | Enable persistence. This changes the processor's type from Deployment to a StatefulSet | <code>false</code> |
| default.persistence.volumeName | Name to assign the volume | <code>"data"</code> |
| default.persistence.existingClaim | Name of an existing PVC to use | <code>""</code> |
| default.persistence.mountPath | The path the volume will be mounted at. Also sets the `SPRING_KAFKA_STREAMS_STATE_DIR` environment variable to this value. | <code>/opt/kafka/streams/state</code> |
| default.persistence.storageClass | PVC Storage Class for the data volume If defined, storageClassName: <storageClass> If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. (gp2 on AWS, standard on GKE, AWS & OpenStack) | <code>""</code> |
| default.persistence.size | PVC Storage Request for volume | <code>8Gi</code> |
| default.persistence.persistentVolumeClaimRetentionPolicy.enabled | Enable Persistent volume retention policy for Statefulset | <code>false</code> |
| default.persistence.persistentVolumeClaimRetentionPolicy.whenScaled | Volume retention behavior when the replica count of the StatefulSet is reduced | <code>Retain</code> |
| default.persistence.persistentVolumeClaimRetentionPolicy.whenDeleted | Volume retention behavior that applies when the StatefulSet is deleted | <code>Retain</code> |

Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example:

Expand Down
9 changes: 8 additions & 1 deletion charts/stream-processors/README.tpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,17 @@ This chart deploys {{ .Project.App }} on a [Kubernetes](http://kubernetes.io) cl
- {{ . }}
{{- end }}

To install a simple Kafka cluster after the Strimzi Kafka Operator is installed, run:
To install a simple Kafka cluster using the Strimzi Operator, run:

```sh
helm upgrade --install strimzi-operator oci://quay.io/strimzi-helm/strimzi-kafka-operator
kubectl apply -f https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/main/examples/kafka/kafka-ephemeral-single.yaml
# Optional: install the Kube Prometheus stack to test the ServiceMonitor integration:
helm upgrade --install kube-prom-stack oci://ghcr.io/prometheus-community/charts/kube-prometheus-stack \
--set=kubeStateMetrics.enabled=false \
--set=nodeExporter.enabled=false \
--set=grafana.enabled=false
```
## Installing the Chart
Expand Down
Loading

0 comments on commit def759f

Please sign in to comment.