Refactor opensearch-cluster helm chart (#754)

### Description
We found that the current `opensearch-cluster` helm chart supports only
`OpenSearchCluster` configuration, also configuration itself is very
limited because not all possible options were defined in the template.
For our needs, we have decided to improve this helm chart and think that
our changes could be useful for the community.

The main differences in the new helm chart:
- Supports all existing CRDs
- Includes auto-generated README.md file with description for all
possible configuration values
- Includes `Ingress` configuration for `Opensearch` and `Dashboards`
- Values.yaml has examples of operator-specific configurations, making
it easier for users to understand what to set
- Easy to maintain and extend

We were trying to build this chart by following the next logic. We
didn't try to template all possible configurations for CRDs, instead, we
relied on official parameters supported by specific sections of the CRD,
e.g.

Instead of doing this:
```yaml
  {{- if .Values.opensearchCluster.bootstrap }}
  bootstrap:
    {{- if .Values.opensearchCluster.bootstrap.additionalConfig }}
    additionalConfig:
      {{ toYaml .Values.opensearchCluster.bootstrap.additionalConfig | nindent 6 }}
    {{- end }}
  {{- end }}
  {{- if .Values.opensearchCluster.initHelper }}
```
We did:
```yaml
  {{- with .Values.cluster.bootstrap }}
  bootstrap: {{ . | toYaml | nindent 4 }}
  {{- end }}
```

And defined all possible configuration options inside the
`.Values.cluster.bootstrap`.

Pros of such logic:
- it helps simplify the maintenance of the chart, when a new config
option is added you just need to add its default value to `values.yaml`
file. And even if for some reason it wasn't added, users still could use
it;
- all existing and new configuration options will be available for
users;
- configuration options have the same format and naming as it is defined
in the operator doc, so users will be not confused by different naming
in helm and off doc.
- it allows to automatically check all defined options, so when users
make a typo or put invalid configuration they will receive an error
during deployment, helping them to understand what they did wrong.

If you apply it, I could help with maintaining this helm chart by fixing
bugs, adding new features, etc.
### Issues Resolved
Closes #699
Closes #9
Closes #855
Closes #866
Closes #902
Closes #667
Closes #904 

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and
signing off your commits, please check
[here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).

---------

Signed-off-by: Yevhenii Tiutiunnyk <[email protected]>

charts/opensearch-cluster/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

charts/opensearch-cluster/CHANGELOG.md

@@ -14,6 +14,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Security
 ---
 
+## [3.0.0]
+### Added
+- Now it is possible to define any configuration that is supported by corresponding CRD by using exactly the same format
+as it is defined in the CRD
+- Support for all existing CRDs
+- Ingress configuration for Opensearch and Dashboards
+- Auto-generated README.md file with description for all possible configuration values
+### Changed
+- `opensearchCluster` variable was replaced by `cluster`. The configuration structure of each custom resource (OpenSearchCluster, OpensearchIndexTemplate, etc) follows the corresponding CRD documentation
+### Deprecated
+- opensearch-cluster helm chart is a fully refactored chart. Before upgrading to v3 check that [default chart values](../../charts/opensearch-cluster/values.yaml)
+  matches with your configuration.
+### Removed
+### Fixed
+### Security
+
 ## [2.6.1]
 ### Added
 ### Changed

charts/opensearch-cluster/Chart.yaml

@@ -4,7 +4,7 @@ description: A Helm chart for OpenSearch Cluster
 type: application
 
 ## The opensearch-cluster Helm Chart version
-version: 2.7.0
+version: 3.0.0
 
 ## The operator version
 appVersion: 2.7.0

charts/opensearch-cluster/README.md

@@ -0,0 +1,109 @@
+# opensearch-cluster
+
+![Version: 3.0.0](https://img.shields.io/badge/Version-3.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.7.0](https://img.shields.io/badge/AppVersion-2.7.0-informational?style=flat-square)
+
+A Helm chart for OpenSearch Cluster
+
+## Values
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| actionGroups | list | `[]` | List of OpensearchActionGroup. Check values.yaml file for examples. |
+| cluster.annotations | object | `{}` | OpenSearchCluster annotations |
+| cluster.bootstrap.additionalConfig | object | `{}` | bootstrap additional configuration, key-value pairs that will be added to the opensearch.yml configuration |
+| cluster.bootstrap.affinity | object | `{}` | bootstrap pod affinity rules |
+| cluster.bootstrap.jvm | string | `""` | bootstrap pod jvm options. If jvm is not provided then the java heap size will be set to half of resources.requests.memory which is the recommend value for data nodes. If jvm is not provided and resources.requests.memory does not exist then value will be -Xmx512M -Xms512M |
+| cluster.bootstrap.nodeSelector | object | `{}` | bootstrap pod node selectors |
+| cluster.bootstrap.resources | object | `{}` | bootstrap pod cpu and memory resources |
+| cluster.bootstrap.tolerations | list | `[]` | bootstrap pod tolerations |
+| cluster.confMgmt.smartScaler | bool | `false` | Enable nodes to be safely removed from the cluster |
+| cluster.dashboards.additionalConfig | object | `{}` | Additional properties for opensearch_dashboards.yaml |
+| cluster.dashboards.affinity | object | `{}` | dashboards pod affinity rules |
+| cluster.dashboards.annotations | object | `{}` | dashboards annotations |
+| cluster.dashboards.basePath | string | `""` | dashboards Base Path for Opensearch Clusters running behind a reverse proxy |
+| cluster.dashboards.enable | bool | `true` | Enable dashboards deployment |
+| cluster.dashboards.env | list | `[]` | dashboards pod env variables |
+| cluster.dashboards.image | string | `"docker.io/opensearchproject/opensearch-dashboards"` | dashboards image |
+| cluster.dashboards.imagePullPolicy | string | `"IfNotPresent"` | dashboards image pull policy |
+| cluster.dashboards.imagePullSecrets | list | `[]` | dashboards image pull secrets |
+| cluster.dashboards.labels | object | `{}` | dashboards labels |
+| cluster.dashboards.nodeSelector | object | `{}` | dashboards pod node selectors |
+| cluster.dashboards.opensearchCredentialsSecret | object | `{}` | Secret that contains fields username and password for dashboards to use to login to opensearch, must only be supplied if a custom securityconfig is provided |
+| cluster.dashboards.pluginsList | list | `[]` | List of dashboards plugins to install |
+| cluster.dashboards.podSecurityContext | object | `{}` | dasboards pod security context configuration |
+| cluster.dashboards.replicas | int | `1` | number of dashboards replicas |
+| cluster.dashboards.resources | object | `{}` | dashboards pod cpu and memory resources |
+| cluster.dashboards.securityContext | object | `{}` | dashboards security context configuration |
+| cluster.dashboards.service.loadBalancerSourceRanges | list | `[]` | source ranges for a loadbalancer |
+| cluster.dashboards.service.type | string | `"ClusterIP"` | dashboards service type |
+| cluster.dashboards.tls.caSecret | object | `{}` | Secret that contains the ca certificate as ca.crt. If this and generate=true is set the existing CA cert from that secret is used to generate the node certs. In this case must contain ca.crt and ca.key fields |
+| cluster.dashboards.tls.enable | bool | `false` | Enable HTTPS for dashboards |
+| cluster.dashboards.tls.generate | bool | `true` | generate certificate, if false secret must be provided |
+| cluster.dashboards.tls.secret | string | `nil` | Optional, name of a TLS secret that contains ca.crt, tls.key and tls.crt data. If ca.crt is in a different secret provide it via the caSecret field |
+| cluster.dashboards.tolerations | list | `[]` | dashboards pod tolerations |
+| cluster.dashboards.version | string | `"2.3.0"` | dashboards version |
+| cluster.general.additionalConfig | object | `{}` | Extra items to add to the opensearch.yml |
+| cluster.general.additionalVolumes | list | `[]` | Additional volumes to mount to all pods in the cluster. Supported volume types configMap, emptyDir, secret (with default Kubernetes configuration schema) |
+| cluster.general.drainDataNodes | bool | `true` | Controls whether to drain data notes on rolling restart operations |
+| cluster.general.httpPort | int | `9200` | Opensearch service http port |
+| cluster.general.image | string | `"docker.io/opensearchproject/opensearch"` | Opensearch image |
+| cluster.general.imagePullPolicy | string | `"IfNotPresent"` | Default image pull policy |
+| cluster.general.keystore | list | `[]` | Populate opensearch keystore before startup |
+| cluster.general.monitoring.enable | bool | `false` | Enable cluster monitoring |
+| cluster.general.monitoring.monitoringUserSecret | string | `""` | Secret with 'username' and 'password' keys for monitoring user. You could also use OpenSearchUser CRD instead of setting it. |
+| cluster.general.monitoring.pluginUrl | string | `""` | Custom URL for the monitoring plugin |
+| cluster.general.monitoring.scrapeInterval | string | `"30s"` | How often to scrape metrics |
+| cluster.general.monitoring.tlsConfig | object | `{}` | Override the tlsConfig of the generated ServiceMonitor |
+| cluster.general.pluginsList | list | `[]` | List of Opensearch plugins to install |
+| cluster.general.podSecurityContext | object | `{}` | Opensearch pod security context configuration |
+| cluster.general.securityContext | object | `{}` | Opensearch securityContext |
+| cluster.general.serviceAccount | string | `""` | Opensearch serviceAccount name. If Service Account doesn't exist it could be created by setting `serviceAccount.create` and `serviceAccount.name` |
+| cluster.general.serviceName | string | `""` | Opensearch service name |
+| cluster.general.setVMMaxMapCount | bool | `true` | Enable setVMMaxMapCount. OpenSearch requires the Linux kernel vm.max_map_count option to be set to at least 262144 |
+| cluster.general.snapshotRepositories | list | `[]` | Opensearch snapshot repositories configuration |
+| cluster.general.vendor | string | `"Opensearch"` |  |
+| cluster.general.version | string | `"2.3.0"` | Opensearch version |
+| cluster.ingress.dashboards.annotations | object | `{}` | dashboards ingress annotations |
+| cluster.ingress.dashboards.className | string | `""` | Ingress class name |
+| cluster.ingress.dashboards.enabled | bool | `false` | Enable ingress for dashboards service |
+| cluster.ingress.dashboards.hosts | list | `[]` | Ingress hostnames |
+| cluster.ingress.dashboards.tls | list | `[]` | Ingress tls configuration |
+| cluster.ingress.opensearch.annotations | object | `{}` | Opensearch ingress annotations |
+| cluster.ingress.opensearch.className | string | `""` | Opensearch Ingress class name |
+| cluster.ingress.opensearch.enabled | bool | `false` | Enable ingress for Opensearch service |
+| cluster.ingress.opensearch.hosts | list | `[]` | Opensearch Ingress hostnames |
+| cluster.ingress.opensearch.tls | list | `[]` | Opensearch tls configuration |
+| cluster.initHelper.imagePullPolicy | string | `"IfNotPresent"` | initHelper image pull policy |
+| cluster.initHelper.imagePullSecrets | list | `[]` | initHelper image pull secret |
+| cluster.initHelper.resources | object | `{}` | initHelper pod cpu and memory resources |
+| cluster.initHelper.version | string | `"1.36"` | initHelper version |
+| cluster.labels | object | `{}` | OpenSearchCluster labels |
+| cluster.name | string | `""` | OpenSearchCluster name, by default release name is used |
+| cluster.nodePools | list | `[{"component":"masters","diskSize":"30Gi","replicas":3,"resources":{"limits":{"cpu":"500m","memory":"2Gi"},"requests":{"cpu":"500m","memory":"2Gi"}},"roles":["master","data"]}]` | Opensearch nodes configuration |
+| cluster.security.config.adminCredentialsSecret | object | `{}` | Secret that contains fields username and password to be used by the operator to access the opensearch cluster for node draining. Must be set if custom securityconfig is provided. |
+| cluster.security.config.adminSecret | object | `{}` | TLS Secret that contains a client certificate (tls.key, tls.crt, ca.crt) with admin rights in the opensearch cluster. Must be set if transport certificates are provided by user and not generated |
+| cluster.security.config.securityConfigSecret | object | `{}` | Secret that contains the differnt yml files of the opensearch-security config (config.yml, internal_users.yml, etc) |
+| cluster.security.tls.http.caSecret | object | `{}` | Optional, secret that contains the ca certificate as ca.crt. If this and generate=true is set the existing CA cert from that secret is used to generate the node certs. In this case must contain ca.crt and ca.key fields |
+| cluster.security.tls.http.generate | bool | `true` | If set to true the operator will generate a CA and certificates for the cluster to use, if false - secrets with existing certificates must be supplied |
+| cluster.security.tls.http.secret | object | `{}` | Optional, name of a TLS secret that contains ca.crt, tls.key and tls.crt data. If ca.crt is in a different secret provide it via the caSecret field |
+| cluster.security.tls.transport.adminDn | list | `[]` | DNs of certificates that should have admin access, mainly used for securityconfig updates via securityadmin.sh, only used when existing certificates are provided |
+| cluster.security.tls.transport.caSecret | object | `{}` | Optional, secret that contains the ca certificate as ca.crt. If this and generate=true is set the existing CA cert from that secret is used to generate the node certs. In this case must contain ca.crt and ca.key fields |
+| cluster.security.tls.transport.generate | bool | `true` | If set to true the operator will generate a CA and certificates for the cluster to use, if false secrets with existing certificates must be supplied |
+| cluster.security.tls.transport.nodesDn | list | `[]` | Allowed Certificate DNs for nodes, only used when existing certificates are provided |
+| cluster.security.tls.transport.perNode | bool | `true` | Separate certificate per node |
+| cluster.security.tls.transport.secret | object | `{}` | Optional, name of a TLS secret that contains ca.crt, tls.key and tls.crt data. If ca.crt is in a different secret provide it via the caSecret field |
+| componentTemplates | list | `[]` | List of OpensearchComponentTemplate. Check values.yaml file for examples. |
+| fullnameOverride | string | `""` |  |
+| indexTemplates | list | `[]` | List of OpensearchIndexTemplate. Check values.yaml file for examples. |
+| ismPolicies | list | `[]` | List of OpenSearchISMPolicy. Check values.yaml file for examples. |
+| nameOverride | string | `""` |  |
+| roles | list | `[]` | List of OpensearchRole. Check values.yaml file for examples. |
+| serviceAccount.annotations | object | `{}` | Service Account annotations |
+| serviceAccount.create | bool | `false` | Create Service Account |
+| serviceAccount.name | string | `""` | Service Account name. Set `general.serviceAccount` to use this Service Account for the Opensearch cluster |
+| tenants | list | `[]` | List of additional tenants. Check values.yaml file for examples. |
+| users | list | `[]` | List of OpensearchUser. Check values.yaml file for examples. |
+| usersRoleBinding | list | `[]` | Allows to link any number of users, backend roles and roles with a OpensearchUserRoleBinding. Each user in the binding will be granted each role Check values.yaml file for examples. |
+
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.11.0](https://github.com/norwoodj/helm-docs/releases/v1.11.0)

charts/opensearch-cluster/templates/NOTES.txt

@@ -0,0 +1,38 @@
+1. Get the Opensearch application URL by running these commands:
+{{- if .Values.cluster.ingress.opensearch.enabled }}
+{{- range $host := .Values.cluster.ingress.opensearch.hosts }}
+  {{- range .paths }}
+  http{{ if $.Values.cluster.ingress.opensearch.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
+  {{- end }}
+{{- end }}
+{{- else }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "opster.io/opensearch-cluster={{ .Values.cluster.name | default (include "opensearch-cluster.name" .) }}" -o jsonpath="{.items[0].metadata.name}")
+  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
+  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 9200:$CONTAINER_PORT
+  curl https://127.0.0.1:9200 -k
+{{- end }}
+
+{{ if .Values.cluster.dashboards.enable }}
+2. Get the Dashboards application URL by running these commands:
+{{- if .Values.cluster.ingress.dashboards.enabled }}
+{{- range $host := .Values.cluster.ingress.dashboards.hosts }}
+  {{- range .paths }}
+  http{{ if $.Values.cluster.ingress.dashboards.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
+  {{- end }}
+{{- end }}
+{{- else if contains "NodePort" .Values.cluster.dashboards.service.type }}
+  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "opensearch-cluster.fullname" . }})
+  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo http://$NODE_IP:$NODE_PORT
+{{- else if contains "LoadBalancer" .Values.cluster.dashboards.service.type }}
+     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
+           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "opensearch-cluster.fullname" . }}'
+  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "opensearch-cluster.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
+  echo http://$SERVICE_IP:{{ .Values.service.port }}
+{{- else if contains "ClusterIP" .Values.cluster.dashboards.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "opensearch.cluster.dashboards={{ include "opensearch-cluster.name" . }}" -o jsonpath="{.items[0].metadata.name}")
+  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
+  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
+  Visit http://127.0.0.1:8080 to use your application
+{{- end }}
+{{- end }}

charts/opensearch-cluster/templates/_helpers.tpl

@@ -1,63 +1,60 @@
-{{/* vim: set filetype=mustache: */}}
 {{/*
 Expand the name of the chart.
 */}}
 {{- define "opensearch-cluster.name" -}}
-{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{- define "opensearch-cluster.cluster-name" -}}
+{{- default .Values.cluster.name .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- end }}
 
 {{/*
 Create a default fully qualified app name.
 We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
 If release name contains chart name it will be used as a full name.
 */}}
 {{- define "opensearch-cluster.fullname" -}}
-{{- if .Values.fullnameOverride -}}
-{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
-{{- else -}}
-{{- $name := default .Chart.Name .Values.nameOverride -}}
-{{- if contains $name .Release.Name -}}
-{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
-{{- else -}}
-{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-{{- end -}}
-{{- end -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
 
 {{/*
 Create chart name and version as used by the chart label.
 */}}
 {{- define "opensearch-cluster.chart" -}}
-{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
 
 {{/*
 Common labels
 */}}
 {{- define "opensearch-cluster.labels" -}}
 helm.sh/chart: {{ include "opensearch-cluster.chart" . }}
-{{ include "opensearch-cluster.selectorLabels" . }}
 {{- if .Chart.AppVersion }}
 app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
 {{- end }}
 app.kubernetes.io/managed-by: {{ .Release.Service }}
-{{- end -}}
-
-{{/*
-Selector labels
-*/}}
-{{- define "opensearch-cluster.selectorLabels" -}}
-app.kubernetes.io/name: {{ include "opensearch-cluster.name" . }}
-app.kubernetes.io/instance: {{ .Release.Name }}
-{{- end -}}
+{{- with .Values.cluster.labels }}
+{{ . | toYaml }}
+{{- end }}
+{{- end }}
 
 {{/*
 Create the name of the service account to use
 */}}
 {{- define "opensearch-cluster.serviceAccountName" -}}
-{{- if .Values.serviceAccount.create -}}
-    {{ default (include "opensearch-cluster.fullname" .) .Values.serviceAccount.name }}
-{{- else -}}
-    {{ default "default" .Values.serviceAccount.name }}
-{{- end -}}
-{{- end -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "opensearch-cluster.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}

charts/opensearch-cluster/templates/actiongroups.yaml

@@ -0,0 +1,25 @@
+{{- $labels := include "opensearch-cluster.labels" . }}
+{{- $clusterName := include "opensearch-cluster.cluster-name" . }}
+{{- range .Values.actionGroups }}
+---
+apiVersion: opensearch.opster.io/v1
+kind: OpensearchActionGroup
+metadata:
+  name: {{ .name }}
+  {{- with $.Values.cluster.annotations }}
+  annotations: {{ . | toYaml | nindent 4 }}
+  {{- end }}
+  labels: {{ $labels | nindent 4 }}
+spec:
+  opensearchCluster:
+    name: {{ $clusterName }}
+  {{- with .description }}
+  description: {{ . | toString }}
+  {{- end }}
+  {{- with .allowedActions }}
+  allowedActions: {{ . | toYaml | nindent 4 }}
+  {{- end }}
+  {{- with .type }}
+  type: {{ . | toString }}
+  {{- end }}
+{{- end }}