From b81f60f4af6c6abea4f40a60738cc32143885ac0 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 09:17:17 -0600 Subject: [PATCH 01/63] more deployment docs --- docs/deployment.md | 43 +++++++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index 87d932f..ac0754a 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -4,12 +4,25 @@ You should have at least two clusters to take full advantage of Atlas. One to ac Atlas should only be installed to the **observability** cluster. All downstream clusters will need an envoy instance deployed, Atlas will provide the necessary helm values to configure the downstream clusters. -## Step 0. Deploy Prometheus with Thanos Sidecar +!!! important + It is **HIGHLY** recommend using the same namespace for your observability components, it makes deployment management much easier. The default for atlas is `monitoring`. + +## Requirements + +- 1 Cluster to act as the Observability Cluster +- 1 Cluster to act as a Downstream Cluster +- Ability to install helm charts +- The envoy helm chart must be installed to an edge node (typically where an ingress instance would be deployed) + +### Deploy Prometheus with Thanos Sidecar It is recommended you use the same namespace like `monitoring` for the deployment of Prometheus and Atlas. How you deploy Prometheus with the Thanos Sidecar is up to you, however I would recommend simply using the [kube-prometheus-stack]() helm chart as it makes this process very simple and takes care of the more complicated bits for you. If you want Thanos persisting to S3 you can pass your S3 credentials along as well. +!!! note + When using `kube-prometheus-stack` ensure `servicePerReplica` is enabled for both prometheus and alertmanager sections, this will allow proper routing to each individual instance. + Once you have your Prometheus instances deployed, please make sure to note the service name as it will be necessary for configuring Atlas properly. If you are use `kube-prometheus-stack` most of the defaults will work out of the box. If you are using something non-standard, please make sure that the Prometheus Port and Thanos Sidecar ports are on the service. ## Step 1. Deploying Atlas @@ -139,18 +152,28 @@ Once you have the values, install helm on your downstream cluster. Make sure you helm install envoy --values downstream1.yaml chart/ ``` -Once complete, this envoy proxy will come online and configure itself automatically. +Please note that ### Step 5. Repeat If you have more than one downstream cluster, repeast steps 3 and 4 until you've added all your clusters. -1. Deploy Atlas with Helm -2. Modify `kube-system/coredns` configmap (ideally with giops) to forward altas TLD to atlas coredns server -3. Generate Downstream Envoy Helm Values -4. Deploy Downstream Envoy Helm Chart -5. Deploy Downstream Prometheus -6. Create Service for Downstream Cluster in Observability Cluster -7. Sit back and enjoy the metrics flowing in! +### Step 6. Configure Downstream Prometheus for Observability Alertmanagers + +To take full advantage of what Atlas offers, you can configure your downstream prometheus instances to talk to the alertmanagers in the observability cluster. + +You'll need to add an alertmanager entry per the number of alertmanagr instances that are on the observability cluster to the downstream prometheus instance. If you are using the prometheus operator then you can simple add an additional alert managers configuration like the following. -**Note:** when using `kube-prometheus-stack` ensure `servicePerReplica` is enabled for both prometheus and alertmanager sections. +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: additional-alertmanager-configs + namespace: monitoring +data: + config.yaml: | + - scheme: http + static_configs: + - targets: + - %s +``` From 980cf75b4f6611759e87581da632ea23cade6d94 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:17:29 -0600 Subject: [PATCH 02/63] chart work and publishing --- .github/renovate.json | 12 + .github/workflows/release-charts.yml | 21 ++ chart/Chart.lock | 6 - chart/Chart.yaml | 9 - charts/atlas/Chart.yaml | 9 + .../atlas}/templates/_helpers.yaml | 0 .../atlas}/templates/cm-coredns.yaml | 0 .../templates/deployment-controller.yaml | 2 + .../atlas}/templates/deployment-coredns.yaml | 0 .../templates/deployment-envoy-ads.yaml | 2 + .../atlas}/templates/ingress-envoy-ads.yaml | 0 .../atlas}/templates/rbac-bindings.yaml | 0 .../atlas}/templates/rbac-roles.yaml | 0 .../atlas}/templates/rbac-sa.yaml | 0 .../atlas}/templates/service-coredns.yaml | 0 .../atlas}/templates/service-envoy-ads.yaml | 0 {chart => charts/atlas}/values.yaml | 9 +- charts/envoy/Chart.yaml | 12 + charts/envoy/templates/NOTES.txt | 0 charts/envoy/templates/_helpers.tpl | 32 ++ charts/envoy/templates/atlas-am.yaml | 19 + charts/envoy/templates/configmap.yaml | 19 + charts/envoy/templates/deployment.yaml | 122 ++++++ .../envoy/templates/poddisruptionbudget.yaml | 15 + charts/envoy/templates/service.yaml | 32 ++ charts/envoy/templates/servicemonitor.yaml | 38 ++ charts/envoy/templates/xds.configmap.yaml | 15 + charts/envoy/values.yaml | 351 ++++++++++++++++++ 28 files changed, 706 insertions(+), 19 deletions(-) create mode 100644 .github/workflows/release-charts.yml delete mode 100644 chart/Chart.lock delete mode 100644 chart/Chart.yaml create mode 100644 charts/atlas/Chart.yaml rename {chart => charts/atlas}/templates/_helpers.yaml (100%) rename {chart => charts/atlas}/templates/cm-coredns.yaml (100%) rename {chart => charts/atlas}/templates/deployment-controller.yaml (96%) rename {chart => charts/atlas}/templates/deployment-coredns.yaml (100%) rename {chart => charts/atlas}/templates/deployment-envoy-ads.yaml (95%) rename {chart => charts/atlas}/templates/ingress-envoy-ads.yaml (100%) rename {chart => charts/atlas}/templates/rbac-bindings.yaml (100%) rename {chart => charts/atlas}/templates/rbac-roles.yaml (100%) rename {chart => charts/atlas}/templates/rbac-sa.yaml (100%) rename {chart => charts/atlas}/templates/service-coredns.yaml (100%) rename {chart => charts/atlas}/templates/service-envoy-ads.yaml (100%) rename {chart => charts/atlas}/values.yaml (94%) create mode 100755 charts/envoy/Chart.yaml create mode 100755 charts/envoy/templates/NOTES.txt create mode 100755 charts/envoy/templates/_helpers.tpl create mode 100644 charts/envoy/templates/atlas-am.yaml create mode 100755 charts/envoy/templates/configmap.yaml create mode 100755 charts/envoy/templates/deployment.yaml create mode 100755 charts/envoy/templates/poddisruptionbudget.yaml create mode 100755 charts/envoy/templates/service.yaml create mode 100755 charts/envoy/templates/servicemonitor.yaml create mode 100755 charts/envoy/templates/xds.configmap.yaml create mode 100755 charts/envoy/values.yaml diff --git a/.github/renovate.json b/.github/renovate.json index 8459fe2..d70e10a 100644 --- a/.github/renovate.json +++ b/.github/renovate.json @@ -21,5 +21,17 @@ "^github.com/aws/aws-sdk-go-v2/.*" ] } + ], + "regexManagers": [ + { + "fileMatch": [ + "charts/.+.yaml$" + ], + "matchStrings": [ + "appVersion:\\s(?.*)\\s?", + "image:\\srepository:\\s(?.*)\\stag:\\s(?.*)\\s?" + ], + "datasourceTemplate": "github-releases" + } ] } \ No newline at end of file diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml new file mode 100644 index 0000000..f508c2a --- /dev/null +++ b/.github/workflows/release-charts.yml @@ -0,0 +1,21 @@ +name: release-charts + +on: + push: + branches: + - main + +jobs: + charts: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - name: Publish Helm charts + uses: stefanprodan/helm-gh-pages@master + with: + token: ${{ secrets.GITHUB_TOKEN }} + charts_url: https://charts.goatlas.io/ + owner: goatlas-io + repository: charts + branch: gh-pages + target_dir: charts diff --git a/chart/Chart.lock b/chart/Chart.lock deleted file mode 100644 index c3446ee..0000000 --- a/chart/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: envoy - repository: https://charts.helm.sh/stable/ - version: 1.9.4 -digest: sha256:6308d164e7da4f068dfe73583eb5c99def7e2dc505d1e9e12d55e42aeaeb1bc0 -generated: "2021-09-06T12:01:14.803655-06:00" diff --git a/chart/Chart.yaml b/chart/Chart.yaml deleted file mode 100644 index 2bbab83..0000000 --- a/chart/Chart.yaml +++ /dev/null @@ -1,9 +0,0 @@ -apiVersion: v2 -name: atlas -version: 1.0.0 -appVersion: 1.0.0 -description: Atlas, forced by Zeus to support the heavens and the skies on his shoulders. -dependencies: -- name: envoy - version: 1.9.4 - repository: https://charts.helm.sh/stable/ \ No newline at end of file diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml new file mode 100644 index 0000000..60fc6b0 --- /dev/null +++ b/charts/atlas/Chart.yaml @@ -0,0 +1,9 @@ +apiVersion: v2 +name: atlas +version: 1.0.0 +appVersion: 0.1.0 +description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. +dependencies: +- name: envoy + version: 1.0.0 + repository: https://ekristen.github.com/atlas/ \ No newline at end of file diff --git a/chart/templates/_helpers.yaml b/charts/atlas/templates/_helpers.yaml similarity index 100% rename from chart/templates/_helpers.yaml rename to charts/atlas/templates/_helpers.yaml diff --git a/chart/templates/cm-coredns.yaml b/charts/atlas/templates/cm-coredns.yaml similarity index 100% rename from chart/templates/cm-coredns.yaml rename to charts/atlas/templates/cm-coredns.yaml diff --git a/chart/templates/deployment-controller.yaml b/charts/atlas/templates/deployment-controller.yaml similarity index 96% rename from chart/templates/deployment-controller.yaml rename to charts/atlas/templates/deployment-controller.yaml index 9a7857b..abbd545 100644 --- a/chart/templates/deployment-controller.yaml +++ b/charts/atlas/templates/deployment-controller.yaml @@ -77,6 +77,8 @@ spec: value: {{ .Values.envoyads.host }} - name: ATLAS_ENVOY_ADDRESS value: {{ .Values.controller.envoy.host }} + - name: ATLAS_ALERTMANAGER_SELECTOR + value: {{ .Values.atlas.alertmanagerSelector }} {{- if .Values.resources }} resources: {{ toYaml .Values.resources | indent 10 }} diff --git a/chart/templates/deployment-coredns.yaml b/charts/atlas/templates/deployment-coredns.yaml similarity index 100% rename from chart/templates/deployment-coredns.yaml rename to charts/atlas/templates/deployment-coredns.yaml diff --git a/chart/templates/deployment-envoy-ads.yaml b/charts/atlas/templates/deployment-envoy-ads.yaml similarity index 95% rename from chart/templates/deployment-envoy-ads.yaml rename to charts/atlas/templates/deployment-envoy-ads.yaml index 8b5e2e5..5db7ca2 100644 --- a/chart/templates/deployment-envoy-ads.yaml +++ b/charts/atlas/templates/deployment-envoy-ads.yaml @@ -74,6 +74,8 @@ spec: env: - name: ATLAS_ENVOY_ADDRESS value: {{ .Values.controller.envoy.host }} + - name: ATLAS_ALERTMANAGER_SELECTOR + value: {{ .Values.atlas.alertmanagerSelector }} {{- if .Values.envoyads.resources }} resources: {{ toYaml .Values.envoyads.resources | indent 10 }} diff --git a/chart/templates/ingress-envoy-ads.yaml b/charts/atlas/templates/ingress-envoy-ads.yaml similarity index 100% rename from chart/templates/ingress-envoy-ads.yaml rename to charts/atlas/templates/ingress-envoy-ads.yaml diff --git a/chart/templates/rbac-bindings.yaml b/charts/atlas/templates/rbac-bindings.yaml similarity index 100% rename from chart/templates/rbac-bindings.yaml rename to charts/atlas/templates/rbac-bindings.yaml diff --git a/chart/templates/rbac-roles.yaml b/charts/atlas/templates/rbac-roles.yaml similarity index 100% rename from chart/templates/rbac-roles.yaml rename to charts/atlas/templates/rbac-roles.yaml diff --git a/chart/templates/rbac-sa.yaml b/charts/atlas/templates/rbac-sa.yaml similarity index 100% rename from chart/templates/rbac-sa.yaml rename to charts/atlas/templates/rbac-sa.yaml diff --git a/chart/templates/service-coredns.yaml b/charts/atlas/templates/service-coredns.yaml similarity index 100% rename from chart/templates/service-coredns.yaml rename to charts/atlas/templates/service-coredns.yaml diff --git a/chart/templates/service-envoy-ads.yaml b/charts/atlas/templates/service-envoy-ads.yaml similarity index 100% rename from chart/templates/service-envoy-ads.yaml rename to charts/atlas/templates/service-envoy-ads.yaml diff --git a/chart/values.yaml b/charts/atlas/values.yaml similarity index 94% rename from chart/values.yaml rename to charts/atlas/values.yaml index ea77e8a..26231e5 100644 --- a/chart/values.yaml +++ b/charts/atlas/values.yaml @@ -12,6 +12,11 @@ rbac: metrics: enabled: true +# This is a label selector for the service that represents the +# alertmanager on the observability cluster. +atlas: + alertmanagerSelector: "app=kube-prometheus-stack-alertmanager" + controller: ports: grpc: 6305 @@ -29,10 +34,6 @@ coredns: envoyads: enabled: true - # This is a label selector for the service that represents the - # alertmanager on the observability cluster. - alertmanager: - selector: "app=kube-prometheus-stack-alertmanager" ingress: enabled: true host: envoyads.atlas.local diff --git a/charts/envoy/Chart.yaml b/charts/envoy/Chart.yaml new file mode 100755 index 0000000..6db158a --- /dev/null +++ b/charts/envoy/Chart.yaml @@ -0,0 +1,12 @@ +apiVersion: v1 +appVersion: 1.18.3 +description: Envoy is an open source edge and service proxy, designed for cloud-native applications. +home: https://goatlas.io/ +keywords: +- envoy +- proxy +- atlas +name: envoy +sources: +- https://github.com/ekristen/atlas +version: 1.0.0 diff --git a/charts/envoy/templates/NOTES.txt b/charts/envoy/templates/NOTES.txt new file mode 100755 index 0000000..e69de29 diff --git a/charts/envoy/templates/_helpers.tpl b/charts/envoy/templates/_helpers.tpl new file mode 100755 index 0000000..d141d3a --- /dev/null +++ b/charts/envoy/templates/_helpers.tpl @@ -0,0 +1,32 @@ +{{/* vim: set filetype=mustache: */}} +{{/* +Expand the name of the chart. +*/}} +{{- define "envoy.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "envoy.fullname" -}} +{{- if .Values.fullnameOverride -}} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- $name := default .Chart.Name .Values.nameOverride -}} +{{- if contains $name .Release.Name -}} +{{- .Release.Name | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} +{{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "envoy.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} +{{- end -}} diff --git a/charts/envoy/templates/atlas-am.yaml b/charts/envoy/templates/atlas-am.yaml new file mode 100644 index 0000000..734af92 --- /dev/null +++ b/charts/envoy/templates/atlas-am.yaml @@ -0,0 +1,19 @@ +{{- range $i, $e := until (int .Values.atlas.alertmanagerCount) }} +--- +apiVersion: v1 +kind: Service +metadata: + name: alertmanager{{ $i }} +spec: + ports: + - name: http + port: 11903 + protocol: TCP + targetPort: 11903 + selector: + app: envoy + release: {{ $.Release.Name }} + type: ClusterIP + clusterIP: None +{{- end }} + diff --git a/charts/envoy/templates/configmap.yaml b/charts/envoy/templates/configmap.yaml new file mode 100755 index 0000000..f55a4ed --- /dev/null +++ b/charts/envoy/templates/configmap.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "envoy.fullname" . }} + labels: + app: {{ template "envoy.name" . }} + chart: {{ template "envoy.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +data: +{{- range $key, $value := .Values.files }} + {{ $key }}: |- +{{ $value | default "" | indent 4 }} +{{- end -}} +{{- range $key, $value := .Values.templates }} + {{ $key }}: |- +{{ $valueWithDefault := default "" $value -}} +{{ tpl $valueWithDefault $ | indent 4 }} +{{- end -}} diff --git a/charts/envoy/templates/deployment.yaml b/charts/envoy/templates/deployment.yaml new file mode 100755 index 0000000..b1f92b7 --- /dev/null +++ b/charts/envoy/templates/deployment.yaml @@ -0,0 +1,122 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "envoy.fullname" . }} + labels: + app: {{ template "envoy.name" . }} + chart: {{ template "envoy.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + replicas: {{ .Values.replicaCount }} + selector: + matchLabels: + app: {{ template "envoy.name" . }} + release: {{ .Release.Name }} + strategy: + {{ .Values.strategy | nindent 4 }} + template: + metadata: + labels: + app: {{ template "envoy.name" . }} + release: {{ .Release.Name }} + component: controller + {{- if .Values.podLabels }} + ## Custom pod labels + {{- range $key, $value := .Values.podLabels }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + annotations: + checksum/config: {{ include (print .Template.BasePath "/configmap.yaml") . | sha256sum }} + {{- if .Values.podAnnotations }} + ## Custom pod annotations + {{- range $key, $value := .Values.podAnnotations }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + spec: + securityContext: + {{ toYaml .Values.securityContext | nindent 8 }} + {{- if .Values.priorityClassName }} + priorityClassName: "{{ .Values.priorityClassName }}" + {{- end }} + terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }} + {{- if .Values.initContainersTemplate }} + initContainers: + {{ tpl .Values.initContainersTemplate $ | nindent 8 }} + {{- end }} + containers: + + - name: envoy + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + command: + {{ toYaml .Values.command | nindent 12 }} + args: + {{- if $.Values.argsTemplate }} + {{ tpl $.Values.argsTemplate $ | nindent 12}} + {{- else }} + {{ toYaml .Values.args | nindent 12 }} + {{- end }} + ports: + {{- with .Values.ports }} + {{- range $key, $port := . }} + - name: {{ $key }} + {{ toYaml $port | nindent 14 }} + {{- end }} + {{- end }} + + livenessProbe: + {{ toYaml .Values.livenessProbe | nindent 12 }} + readinessProbe: + {{ toYaml .Values.readinessProbe | nindent 12 }} + env: + {{- range $key, $value := .Values.env }} + - name: {{ $key | upper | replace "." "_" }} + value: {{ $value | quote }} + {{- end }} + resources: + {{ toYaml .Values.resources | nindent 12 }} + volumeMounts: + - name: config + mountPath: /config + {{- if .Values.volumeMounts }} + {{ toYaml .Values.volumeMounts | nindent 12 }} + {{- end }} + {{- range $key, $value := .Values.secretMounts }} + - name: {{ $key }} + mountPath: {{ $value.mountPath }} + {{- end }} + lifecycle: + {{ toYaml .Values.lifecycle | nindent 12 }} + + {{- if .Values.sidecarContainersTemplate }} + {{ tpl .Values.sidecarContainersTemplate $ | nindent 8 }} + {{- end }} + + {{- with .Values.nodeSelector }} + nodeSelector: + {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: + {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: + {{ toYaml . | nindent 8 }} + {{- end }} + volumes: + - name: config + configMap: + name: {{ template "envoy.fullname" . }} + {{- if .Values.volumes }} + {{ toYaml .Values.volumes | nindent 8 }} + {{- end }} + {{- range $key, $value := .Values.secretMounts }} + - name: {{ $key }} + secret: + secretName: {{ $value.secretName }} + defaultMode: {{ $value.defaultMode }} + {{- end }} diff --git a/charts/envoy/templates/poddisruptionbudget.yaml b/charts/envoy/templates/poddisruptionbudget.yaml new file mode 100755 index 0000000..ece46fd --- /dev/null +++ b/charts/envoy/templates/poddisruptionbudget.yaml @@ -0,0 +1,15 @@ +apiVersion: policy/v1beta1 +kind: PodDisruptionBudget +metadata: + name: {{ template "envoy.fullname" . }} + labels: + app: {{ template "envoy.name" . }} + chart: {{ template "envoy.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + selector: + matchLabels: + app: {{ template "envoy.name" . }} + release: {{ .Release.Name }} +{{ .Values.podDisruptionBudget | indent 2 }} diff --git a/charts/envoy/templates/service.yaml b/charts/envoy/templates/service.yaml new file mode 100755 index 0000000..4cdee01 --- /dev/null +++ b/charts/envoy/templates/service.yaml @@ -0,0 +1,32 @@ +{{- if .Values.service.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ .Values.service.name }} + labels: + app: {{ template "envoy.name" . }} + chart: {{ template "envoy.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} + annotations: +{{- with .Values.service.annotations }} +{{ toYaml . | indent 4 }} +{{- end }} +spec: + type: {{ .Values.service.type }} + {{- if ne .Values.service.loadBalancerIP "" }} + loadBalancerIP: {{ .Values.service.loadBalancerIP }} + {{- end }} + ports: + {{- range $key, $value := .Values.service.ports }} + - name: {{ $key }} +{{ toYaml $value | indent 6 }} + {{- end }} + {{- if .Values.service.loadBalancerSourceRanges }} + loadBalancerSourceRanges: + {{- toYaml .Values.service.loadBalancerSourceRanges | nindent 6 }} + {{- end }} + selector: + app: {{ template "envoy.name" . }} + release: {{ .Release.Name }} +{{- end }} diff --git a/charts/envoy/templates/servicemonitor.yaml b/charts/envoy/templates/servicemonitor.yaml new file mode 100755 index 0000000..4e0f4da --- /dev/null +++ b/charts/envoy/templates/servicemonitor.yaml @@ -0,0 +1,38 @@ +{{- if and ( .Capabilities.APIVersions.Has "monitoring.coreos.com/v1" ) ( .Values.serviceMonitor.enabled ) }} +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + labels: + app: {{ template "envoy.name" . }} + chart: {{ template "envoy.chart" . }} + heritage: {{ .Release.Service }} + release: {{ .Release.Name }} +{{- if .Values.serviceMonitor.additionalLabels }} +{{ toYaml .Values.serviceMonitor.additionalLabels | indent 4}} +{{- end }} + name: {{ template "envoy.fullname" . }} +{{- if .Values.serviceMonitor.namespace }} + namespace: {{ .Values.serviceMonitor.namespace }} +{{- end }} +spec: + endpoints: + - targetPort: {{ .Values.ports.admin.containerPort }} + interval: {{ .Values.serviceMonitor.interval }} + path: "/stats/prometheus" + jobLabel: {{ template "envoy.fullname" . }} + namespaceSelector: + matchNames: + - {{ .Release.Namespace }} + selector: + matchLabels: + app: {{ template "envoy.name" . }} + release: {{ .Release.Name }} + {{- with .Values.serviceMonitor.targetLabels }} + targetLabels: +{{ toYaml . | trim | indent 4 -}} + {{- end }} + {{- with .Values.serviceMonitor.podTargetLabels }} + podTargetLabels: +{{ toYaml . | trim | indent 4 -}} + {{- end }} +{{- end }} diff --git a/charts/envoy/templates/xds.configmap.yaml b/charts/envoy/templates/xds.configmap.yaml new file mode 100755 index 0000000..97d1689 --- /dev/null +++ b/charts/envoy/templates/xds.configmap.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "envoy.fullname" . }}-xds + labels: + app: {{ template "envoy.name" . }} + chart: {{ template "envoy.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +data: +{{- range $filename, $content := .Values.xds }} + {{ tpl $filename $ }}: |- +{{ $valueWithDefault := default "" $content -}} +{{ tpl $valueWithDefault $ | indent 4 }} +{{- end -}} diff --git a/charts/envoy/values.yaml b/charts/envoy/values.yaml new file mode 100755 index 0000000..bd7c2f9 --- /dev/null +++ b/charts/envoy/values.yaml @@ -0,0 +1,351 @@ +replicaCount: 2 + +podDisruptionBudget: | + maxUnavailable: 1 + +## ref: https://pracucci.com/graceful-shutdown-of-kubernetes-pods.html +terminationGracePeriodSeconds: 30 + +strategy: | + type: RollingUpdate + rollingUpdate: + maxSurge: 2 + maxUnavailable: 1 + +image: + repository: envoyproxy/envoy + tag: v1.18.3 + pullPolicy: IfNotPresent + +command: + - /usr/local/bin/envoy +args: + - -l + - $loglevel + - -c + - /config/envoy.yaml + +## Args template allows you to use Chart template expressions to dynamically generate args +# argsTemplate: |- +# - -c +# - /docker-entrypoint.sh envoy --service-node ${POD_NAME} --service-cluster {{ template "envoy.fullname" . }} -l debug -c /config/envoy.yaml + +## Client service. +service: + enabled: true + ## Service name is user-configurable for maximum service discovery flexibility. + name: envoy + type: ClusterIP + ## Ignored if the type is not LoadBalancer or if the IP is empty string + loadBalancerIP: "" + annotations: + {} + ## AWS example for use with LoadBalancer service type. + # external-dns.alpha.kubernetes.io/hostname: envoy.cluster.local + # service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true" + # service.beta.kubernetes.io/aws-load-balancer-internal: "true" + ports: + n0: + port: 10000 + targetPort: n0 + protocol: TCP + ## Used to whitelist certain source CIDRs + # loadBalancerSourceRanges: + # - 0.0.0.0/0 + +ports: + admin: + containerPort: 9901 + protocol: TCP + n0: + containerPort: 10000 + protocol: TCP + +resources: + {} + # We usually recommend not to specify default resources and to leave this as a conscious + # choice for the user. This also increases chances charts run on environments with little + # resources, such as Minikube. If you do want to specify resources, uncomment the following + # lines, adjust them as necessary, and remove the curly braces after 'resources:'. + # limits: + # cpu: 100m + # memory: 128Mi + # requests: + # cpu: 100m + # memory: 128Mi + +priorityClassName: "" + +nodeSelector: {} + +tolerations: [] + +affinity: + {} + # podAntiAffinity: + # preferredDuringSchedulingIgnoredDuringExecution: + # - weight: 50 + # podAffinityTerm: + # topologyKey: failure-domain.beta.kubernetes.io/zone + # labelSelector: + # matchLabels: + # release: envoy + # requiredDuringSchedulingIgnoredDuringExecution: + # - weight: 40 + # topologyKey: "kubernetes.io/hostname" + # labelSelector: + # matchLabels: + # release: envoy + +## ref: https://github.com/envoyproxy/envoy/pull/2896 +podAnnotations: + {} + # prometheus.io/scrape: "true" + # prometheus.io/path: "/stats/prometheus" + # prometheus.io/port: "9901" + +podLabels: + {} + # team: "developers" + # service: "envoy" + +livenessProbe: + tcpSocket: + port: admin + initialDelaySeconds: 30 + # periodSeconds: 10 + # timeoutSeconds: 5 + # failureThreshold: 3 + # successThreshold: 1 + +readinessProbe: + tcpSocket: + port: admin + initialDelaySeconds: 30 + # periodSeconds: 10 + # timeoutSeconds: 5 + # failureThreshold: 3 + # successThreshold: 1 + +securityContext: {} + +env: {} + +## Create secrets out-of-band from Helm like this: +## +## $ kubectl create secret generic envoy --from-file=./some-secret.txt +## +secretMounts: + {} + # secret: + # secretName: envoy + # mountPath: /secret + # defaultMode: 256 # 256 in base10 == 0400 in octal + +files: + envoy.yaml: |- + ## refs: + ## - https://www.envoyproxy.io/docs/envoy/latest/start/start#quick-start-to-run-simple-example + ## - https://raw.githubusercontent.com/envoyproxy/envoy/master/configs/google_com_proxy.v2.yaml + admin: + access_log_path: /dev/stdout + address: + socket_address: + address: 0.0.0.0 + port_value: 9901 + + static_resources: + listeners: + - name: listener_0 + address: + socket_address: + address: 0.0.0.0 + port_value: 10000 + filter_chains: + - filters: + - name: envoy.http_connection_manager + config: + access_log: + - name: envoy.file_access_log + config: + path: /dev/stdout + stat_prefix: ingress_http + route_config: + name: local_route + virtual_hosts: + - name: local_service + domains: ["*"] + routes: + - match: + prefix: "/" + route: + host_rewrite: www.google.com + cluster: service_google + http_filters: + - name: envoy.router + clusters: + - name: service_google + connect_timeout: 0.25s + type: LOGICAL_DNS + dns_lookup_family: V4_ONLY + lb_policy: ROUND_ROBIN + hosts: + - socket_address: + address: google.com + port_value: 443 + tls_context: + sni: www.google.com + +## Uncomment this section to use helm values to dynamically generate enovy.yaml +# templates: +# envoy.yaml: |- +# ## refs: +# ## - https://www.envoyproxy.io/docs/envoy/latest/start/start#quick-start-to-run-simple-example +# ## - https://raw.githubusercontent.com/envoyproxy/envoy/master/configs/google_com_proxy.v2.yaml +# admin: +# access_log_path: /dev/stdout +# address: +# socket_address: +# address: 0.0.0.0 +# port_value: {{ .Values.ports.admin.containerPort }} + +# static_resources: +# listeners: +# - name: listener_0 +# address: +# socket_address: +# address: 0.0.0.0 +# port_value: {{ .Values.ports.n0.containerPort }} +# filter_chains: +# - filters: +# - name: envoy.http_connection_manager +# config: +# access_log: +# - name: envoy.file_access_log +# config: +# path: /dev/stdout +# stat_prefix: ingress_http +# route_config: +# name: local_route +# virtual_hosts: +# - name: local_service +# domains: ["*"] +# routes: +# - match: +# prefix: "/" +# route: +# host_rewrite: www.google.com +# cluster: service_google +# http_filters: +# - name: envoy.router +# clusters: +# - name: service_google +# connect_timeout: 0.25s +# type: LOGICAL_DNS +# dns_lookup_family: V4_ONLY +# lb_policy: ROUND_ROBIN +# hosts: +# - socket_address: +# address: google.com +# port_value: 443 +# tls_context: +# sni: www.google.com + +## Additional volumes to be added to Envoy pods +# volumes: +# - name: xds +# emptyDir: {} + +## Additional volume mounts to be added to Envoy containers(Primary containers of Envoy pods) +# volumeMounts: +# - name: xds +# mountPath: /srv/runtime + +## Init containers +# initContainersTemplate: |- +# - name: xds-init +# image: mumoshu/envoy-xds-configmap-loader:canary-6090275 +# command: +# - envoy-xds-configmap-loader +# args: +# - --configmap={{ template "envoy.fullname" . }}-xds +# - --onetime +# - --insecure +# env: +# - name: POD_NAMESPACE +# valueFrom: +# fieldRef: +# fieldPath: metadata.namespace +# volumeMounts: +# - name: xds +# mountPath: /srv/runtime + +## Sidecar containers +# sidecarContainersTemplate: |- +# - name: xds-update +# image: mumoshu/envoy-xds-configmap-loader:canary-6090275 +# command: +# - envoy-xds-configmap-loader +# args: +# - --configmap={{ template "envoy.fullname" . }}-xds +# - --sync-interval=5s +# - --insecure +# env: +# - name: POD_NAMESPACE +# valueFrom: +# fieldRef: +# fieldPath: metadata.namespace +# volumeMounts: +# - name: xds +# mountPath: /srv/runtime + +## ServiceMonitor consumed by prometheus-operator +serviceMonitor: + ## If the operator is installed in your cluster, set to true to create a Service Monitor Entry + enabled: false + interval: "15s" + targetLabels: [] + podTargetLabels: [] + ## Namespace in which the service monitor is created + # namespace: monitoring + # Added to the ServiceMonitor object so that prometheus-operator is able to discover it + ## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#prometheusspec + additionalLabels: {} + +### Lifecycle Events +lifecycle: {} +# preStop: +# exec: +# command: +# - sh +# - -c +# - "sleep 60" + +## PrometheusRule consumed by prometheus-operator +prometheusRule: + enabled: false + ## Namespace in which the prometheus rule is created + # namespace: monitoring + ## Define individual alerting rules as required + ## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#rulegroup + ## https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/ + groups: + upstream-rules: + enabled: true + rules: + high4xxRate: + enabled: true + alert: High4xxRate + expr: sum(rate(envoy_cluster_upstream_rq_xx{response_code_class="4"}[1m])) / sum(rate(envoy_cluster_upstream_rq_xx[1m])) * 100 > 1 + for: 1m + labels: + severity: page + annotations: + summary: "4xx response rate above 1%" + description: "The 4xx error response rate for envoy cluster {{ $labels.envoy_cluster_name }} reported a service replication success rate of {{ $value }}% for more than 1 minute." + ## Added to the PrometheusRule object so that prometheus-operator is able to discover it + ## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#prometheusspec + additionalLabels: {} + +atlas: + alertmanagerCount: 1 From 9157fe081913eb0ebfb4695ad8a71be575d9ddad Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:18:00 -0600 Subject: [PATCH 03/63] removing old workflows --- .github/workflows/docker.yml | 103 -------------------------- .github/workflows/release-drafter.yml | 16 ---- 2 files changed, 119 deletions(-) delete mode 100644 .github/workflows/docker.yml delete mode 100644 .github/workflows/release-drafter.yml diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml deleted file mode 100644 index 9677d79..0000000 --- a/.github/workflows/docker.yml +++ /dev/null @@ -1,103 +0,0 @@ -name: docker - -on: - workflow_dispatch: - push: - branches: - - master - tags: - - "v*.*.*" - pull_request: - branches: - - master - -jobs: - version: - runs-on: ubuntu-latest - if: startsWith(github.head_ref, 'renovate') == false - steps: - - name: Checkout - uses: actions/checkout@v2 - with: - fetch-depth: 0 - - name: Detect Version for Docker - id: docker-version - run: echo ::set-output name=version::$(SEP="-" .ci/version) - - name: Detect Version - id: version - run: echo ::set-output name=version::$(.ci/version) - outputs: - docker-version: ${{ steps.docker-version.outputs.version }} - version: ${{ steps.version.outputs.version }} - - image: - runs-on: ubuntu-latest - needs: [version] - steps: - - name: Get current date - id: date - run: echo "::set-output name=date::$(date +'%s')" - - name: Checkout - uses: actions/checkout@v2 - with: - fetch-depth: 0 - - name: Get branch names - id: branch-name - uses: tj-actions/branch-names@v4.9 - - name: Setup Docker Metadata - id: docker_metadata - uses: crazy-max/ghaction-docker-meta@v3 - with: - images: | - ghcr.io/ekristen/atlas - tags: | - type=sha,prefix=${{ steps.branch-name.outputs.current_branch }}- - type=sha,prefix=${{ steps.branch-name.outputs.current_branch }}-,suffix=-${{ steps.date.outputs.date }} - type=ref,event=tag - type=ref,event=pr - type=raw,value=v${{ needs.version.outputs.docker-version }} - type=raw,value=${{ steps.branch-name.outputs.current_branch }} - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v1 - - name: Login to GitHub Container Registry - uses: docker/login-action@v1 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - name: Build to Registry - uses: docker/build-push-action@v2 - with: - context: . - tags: ${{ steps.docker_metadata.outputs.tags }} - labels: ${{ steps.docker_metadata.outputs.labels }} - build-args: | - BRANCH=${{ steps.branch-name.outputs.current_branch }} - cache-from: | - type=registry,ref=ghcr.io/ekristen/atlas:master - type=registry,ref=ghcr.io/ekristen/atlas:${{ steps.branch-name.outputs.current_branch }} - cache-to: type=inline - outputs: type=registry - - name: Build to Docker - uses: docker/build-push-action@v2 - with: - context: . - tags: ${{ steps.docker_metadata.outputs.tags }} - labels: ${{ steps.docker_metadata.outputs.labels }} - build-args: | - BRANCH=${{ steps.branch-name.outputs.current_branch }} - cache-from: | - type=registry,ref=ghcr.io/ekristen/atlas:master - type=registry,ref=ghcr.io/ekristen/atlas:${{ steps.branch-name.outputs.current_branch }} - cache-to: type=inline - outputs: type=docker - - name: Run Trivy vulnerability scanner in docker mode - uses: aquasecurity/trivy-action@master - if: ${{ github.event_name == 'pull_request' }} - with: - image-ref: ghcr.io/ekristen/atlas:${{ steps.docker_metadata.outputs.version }} - format: "table" - exit-code: "1" - ignore-unfixed: true - vuln-type: "os,library" - severity: "CRITICAL,HIGH" diff --git a/.github/workflows/release-drafter.yml b/.github/workflows/release-drafter.yml deleted file mode 100644 index ecba06c..0000000 --- a/.github/workflows/release-drafter.yml +++ /dev/null @@ -1,16 +0,0 @@ -name: Release Drafter - -on: - push: - branches: - - master - pull_request: - types: [opened, reopened, synchronize] - -jobs: - update_release_draft: - runs-on: ubuntu-latest - steps: - - uses: release-drafter/release-drafter@v5 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} From e8df0ca701bd0840f9587cd8b0b3477fae1df498 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:18:47 -0600 Subject: [PATCH 04/63] update mkdocs meta, adding workflow to release docs --- .github/workflows/release-docs.yml | 29 +++++++++++++++++++++++++++++ mkdocs.yml | 6 +++--- 2 files changed, 32 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/release-docs.yml diff --git a/.github/workflows/release-docs.yml b/.github/workflows/release-docs.yml new file mode 100644 index 0000000..6195580 --- /dev/null +++ b/.github/workflows/release-docs.yml @@ -0,0 +1,29 @@ +name: release-docs + +on: + workflow_dispatch: + push: + tags: + - "v*" + +jobs: + publish: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + with: + python-version: 3.x + - run: pip install mkdocs-material + - run: pip install mkdocs-material mkdocs-awesome-pages-plugin mkdocs-minify-plugin mkdocs-redirects + - run: mkdocs build + - name: Deploy to GitHub Pages + if: success() + uses: crazy-max/ghaction-github-pages@v2 + with: + repo: goatlas-io/docs + target_branch: gh-pages + build_dir: public + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/mkdocs.yml b/mkdocs.yml index d276601..a97156f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,7 +1,7 @@ site_name: Atlas -site_url: https://ekristen.github.io/atlas/ -repo_name: ekristen/atlas -repo_url: https://github.com/ekristen/atlas +site_url: https://goatlas.io/docs/ +repo_name: goatlas-io/atlas +repo_url: https://github.com/goatlas-io/atlas edit_uri: "" site_dir: public From c5ddcacb61ec64fefb1166572ae868cc9d2001f7 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:19:11 -0600 Subject: [PATCH 05/63] organize and add to .gitignore --- .gitignore | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index 04d0932..8c90d15 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,8 @@ vendor hack/deployment/**/*.json hack/deployment/**/*.yaml +release +archive # Files .envrc @@ -10,5 +12,4 @@ test-*.yaml chart/charts/*.tgz droplet.json envoy-values.yaml -archive -helm-values.yaml +helm-values.yaml \ No newline at end of file From a7ba1248044ec3d7241ad7335f32bce43b4aa835 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:19:31 -0600 Subject: [PATCH 06/63] switch to main branch --- .github/workflows/tests.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 261b77f..30abd87 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -4,10 +4,10 @@ on: workflow_dispatch: pull_request: branches: - - master + - main push: branches: - - master + - main jobs: tests: From 7ae6764eda97133ebfc9f1c6e46dade105d8dd43 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:20:52 -0600 Subject: [PATCH 07/63] allow for namespace to be configurable --- Makefile | 17 ++-- pkg/commands/envoy-ads.go | 2 +- pkg/controllers/atlas/controller.go | 88 ++++++++++++------- .../atlas/templates/envoy-downstream.tmpl | 2 + pkg/envoy/helpers.go | 2 +- pkg/envoy/server.go | 27 +++--- 6 files changed, 80 insertions(+), 58 deletions(-) diff --git a/Makefile b/Makefile index 4fcc325..badde15 100644 --- a/Makefile +++ b/Makefile @@ -9,17 +9,14 @@ MODULE := $(shell head -n1 go.mod | cut -f2 -d' ') vendor: go mod vendor -build: - go build -ldflags "-X $(MODULE)/pkg/common.SUMMARY=$(SUMMARY) -X $(MODULE)/pkg/common.BRANCH=$(BRANCH) -X $(MODULE)/pkg/common.VERSION=$(VERSION)" -o $(NAME) - -release: vendor - go build -mod=vendor -ldflags "-X $(MODULE)/pkg/common.SUMMARY=$(SUMMARY) -X $(MODULE)/pkg/common.BRANCH=$(BRANCH) -X $(MODULE)/pkg/common.VERSION=$(VERSION)" -o $(NAME) . - -run-%: - go run -mod=vendor -ldflags "-X $(MODULE)/pkg/common.SUMMARY=$(SUMMARY) -X $(MODULE)/pkg/common.BRANCH=$(BRANCH) -X $(MODULE)/pkg/common.VERSION=$(VERSION)" main.go $* - docs-build: docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material build docs-serve: - docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material + docke/r run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material + +build: + SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser build + +snapshot: + SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser release --snapshot --skip-publish --rm-dist \ No newline at end of file diff --git a/pkg/commands/envoy-ads.go b/pkg/commands/envoy-ads.go index 0a4bbc3..2451a71 100644 --- a/pkg/commands/envoy-ads.go +++ b/pkg/commands/envoy-ads.go @@ -100,7 +100,7 @@ func init() { &cli.StringFlag{ Name: "alertmanager-selector", Usage: "Label Selector for AlertManager", - EnvVars: []string{"ALERTMANAGER_SELECTOR"}, + EnvVars: []string{"ATLAS_ALERTMANAGER_SELECTOR"}, Value: common.ObservabilityAlertManagerServiceLabel, }, &cli.StringFlag{ diff --git a/pkg/controllers/atlas/controller.go b/pkg/controllers/atlas/controller.go index dedb3a8..4d61572 100644 --- a/pkg/controllers/atlas/controller.go +++ b/pkg/controllers/atlas/controller.go @@ -28,6 +28,7 @@ import ( corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/labels" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/selection" @@ -38,6 +39,7 @@ import ( "github.com/ekristen/atlas/pkg/common" "github.com/ekristen/atlas/pkg/config" + "github.com/ekristen/atlas/pkg/envoy" ) //go:embed templates/* @@ -64,6 +66,8 @@ type Controller struct { dnsUpdateLock sync.Mutex dnsLastHash string + + namespace string } func Register( @@ -79,7 +83,7 @@ func Register( c := Controller{ ctx: ctx, config: config, - log: log.WithField("component-type", "controller").WithField("component", common.MonitoringNamespace), + log: log.WithField("component-type", "controller").WithField("component", cli.String("namespace")), cli: cli, apply: apply, secrets: secrets, @@ -87,6 +91,7 @@ func Register( configmaps: configmaps, services: services, servicesCache: services.Cache(), + namespace: cli.String("namespace"), } c.secrets.OnChange(ctx, common.NAME, c.handleSecretChange) @@ -171,7 +176,7 @@ func (c *Controller) createObservabilityValues() error { s := &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: common.ObservabilityEnvoyValuesSecretName, - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, }, StringData: map[string]string{ "values.yaml": string(buf.Bytes()), @@ -244,7 +249,7 @@ func (c *Controller) configureCA() error { var currentCASecret *corev1.Secret - caSecret, err := c.secrets.Get(common.MonitoringNamespace, common.CASecretName, metav1.GetOptions{}) + caSecret, err := c.secrets.Get(c.namespace, common.CASecretName, metav1.GetOptions{}) if err != nil { if apierrors.IsNotFound(err) { isNew = true @@ -295,7 +300,7 @@ func (c *Controller) configureCA() error { caSecret = &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: common.CASecretName, - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, Annotations: map[string]string{}, Labels: map[string]string{ common.IsCALabel: "true", @@ -370,7 +375,7 @@ func (c *Controller) configureCA() error { func (c *Controller) setupPKI() error { doGenerate := false - ingressTLSSecret, err := c.secrets.Get(common.MonitoringNamespace, common.IngressTLSSecretName, metav1.GetOptions{}) + ingressTLSSecret, err := c.secrets.Get(c.namespace, common.IngressTLSSecretName, metav1.GetOptions{}) if err != nil && !apierrors.IsNotFound(err) { return err } else if err != nil && apierrors.IsNotFound(err) { @@ -382,7 +387,7 @@ func (c *Controller) setupPKI() error { } } - mtlsClientSecret, err := c.secrets.Get(common.MonitoringNamespace, common.ClientSecretName, metav1.GetOptions{}) + mtlsClientSecret, err := c.secrets.Get(c.namespace, common.ClientSecretName, metav1.GetOptions{}) if err != nil && !apierrors.IsNotFound(err) { return err } else if err != nil && apierrors.IsNotFound(err) { @@ -394,7 +399,7 @@ func (c *Controller) setupPKI() error { } } - mtlsServerSecret, err := c.secrets.Get(common.MonitoringNamespace, common.ServerSecretName, metav1.GetOptions{}) + mtlsServerSecret, err := c.secrets.Get(c.namespace, common.ServerSecretName, metav1.GetOptions{}) if err != nil && !apierrors.IsNotFound(err) { return err } else if err != nil && apierrors.IsNotFound(err) { @@ -440,7 +445,7 @@ func (c *Controller) setupPKI() error { ingressTLSSecret = &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: common.IngressTLSSecretName, - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, Labels: map[string]string{ common.IsCertLabel: "true", common.CASerialLabel: fmt.Sprintf("%d", ingressSerial), @@ -458,7 +463,7 @@ func (c *Controller) setupPKI() error { mtlsClientSecret = &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: common.ClientSecretName, - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, Labels: map[string]string{ common.IsCertLabel: "true", common.CASerialLabel: fmt.Sprintf("%d", clientSerial), @@ -477,7 +482,7 @@ func (c *Controller) setupPKI() error { mtlsServerSecret = &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: common.ServerSecretName, - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, Labels: map[string]string{ common.IsCertLabel: "true", common.CASerialLabel: fmt.Sprintf("%d", serverSerial), @@ -505,7 +510,7 @@ func (c *Controller) generateCert(extKeyUsage []x509.ExtKeyUsage, commonName str serial := big.NewInt(time.Now().UTC().Unix()) subject := pkix.Name{ - Organization: []string{"ekristen.github.io"}, + Organization: []string{"goatlas.io"}, OrganizationalUnit: []string{"Atlas"}, Country: []string{"US"}, Province: []string{"DC"}, @@ -562,7 +567,7 @@ func (c *Controller) generateCA() (*big.Int, *bytes.Buffer, *bytes.Buffer, error ca := &x509.Certificate{ SerialNumber: big.NewInt(time.Now().UTC().Unix()), Subject: pkix.Name{ - Organization: []string{"ekristen.github.io"}, + Organization: []string{"goatlas.io"}, OrganizationalUnit: []string{"Atlas"}, Country: []string{"US"}, Province: []string{"DC"}, @@ -696,7 +701,7 @@ func (c *Controller) handleServiceChangeforDNS(key string, service *corev1.Servi c.dnsUpdateLock.Lock() defer c.dnsUpdateLock.Unlock() - monitoringNamespace := common.MonitoringNamespace + monitoringNamespace := c.namespace requirement, err := labels.NewRequirement(common.SidecarLabel, selection.Exists, []string{}) if err != nil { @@ -773,7 +778,7 @@ func (c *Controller) handleServiceChangeforDNS(key string, service *corev1.Servi cm := &corev1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ Name: c.cli.String("dns-config-map-name"), - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, }, Data: map[string]string{ "atlas.zone": string(buf.Bytes()), @@ -789,39 +794,54 @@ func (c *Controller) handleServiceChangeforDNS(key string, service *corev1.Servi } func (c *Controller) generateEnvoyValuesSecret(service *corev1.Service) (*corev1.Secret, error) { - ca, err := c.secretsCache.Get(common.MonitoringNamespace, common.CASecretName) + ca, err := c.secretsCache.Get(c.namespace, common.CASecretName) if err != nil { return nil, err } - server, err := c.secretsCache.Get(common.MonitoringNamespace, common.ServerSecretName) + server, err := c.secretsCache.Get(c.namespace, common.ServerSecretName) if err != nil { return nil, err } - client, err := c.secretsCache.Get(common.MonitoringNamespace, common.ClientSecretName) + client, err := c.secretsCache.Get(c.namespace, common.ClientSecretName) if err != nil { return nil, err } + actualAMServices := []*corev1.Service{} + amServices, err := c.services.List(c.namespace, v1.ListOptions{ + LabelSelector: c.cli.String("alertmanager-selector"), + }) + if err != nil { + return nil, err + } + for _, service := range amServices.Items { + if _, ok := service.Spec.Selector["statefulset.kubernetes.io/pod-name"]; ok { + actualAMServices = append(actualAMServices, &service) + } + } + data := struct { - CA string - ServerCert string - ServerKey string - ClientCert string - ClientKey string - ClusterID string - EnvoyADSAddress string - EnvoyADSPort int64 + CA string + ServerCert string + ServerKey string + ClientCert string + ClientKey string + ClusterID string + EnvoyADSAddress string + EnvoyADSPort int64 + AlertmanagerCount int }{ - CA: string(ca.Data["ca.pem"]), - ServerCert: string(server.Data["tls.crt"]), - ServerKey: string(server.Data["tls.key"]), - ClientCert: string(client.Data["tls.crt"]), - ClientKey: string(client.Data["tls.key"]), - ClusterID: service.Name, - EnvoyADSAddress: c.config.ADSAddress, - EnvoyADSPort: c.config.ADSPort, + CA: string(envoy.CombineCAs(ca)), + ServerCert: string(server.Data["tls.crt"]), + ServerKey: string(server.Data["tls.key"]), + ClientCert: string(client.Data["tls.crt"]), + ClientKey: string(client.Data["tls.key"]), + ClusterID: service.Name, + EnvoyADSAddress: c.config.ADSAddress, + EnvoyADSPort: c.config.ADSPort, + AlertmanagerCount: len(actualAMServices), } d, err := templates.ReadFile("templates/envoy-downstream.tmpl") @@ -847,7 +867,7 @@ func (c *Controller) generateEnvoyValuesSecret(service *corev1.Service) (*corev1 s := &corev1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: secretName, - Namespace: common.MonitoringNamespace, + Namespace: c.namespace, }, StringData: map[string]string{ "values.yaml": string(buf.Bytes()), diff --git a/pkg/controllers/atlas/templates/envoy-downstream.tmpl b/pkg/controllers/atlas/templates/envoy-downstream.tmpl index 54347dc..2219ff3 100644 --- a/pkg/controllers/atlas/templates/envoy-downstream.tmpl +++ b/pkg/controllers/atlas/templates/envoy-downstream.tmpl @@ -1,3 +1,5 @@ +atlas: + alertmanagerCount: {{ .AlertmanagerCount }} replicaCount: 1 args: - -l diff --git a/pkg/envoy/helpers.go b/pkg/envoy/helpers.go index b135a5e..89fd7e2 100644 --- a/pkg/envoy/helpers.go +++ b/pkg/envoy/helpers.go @@ -334,7 +334,7 @@ func buildSecretTLSValidation(name string, ca []byte) *tls.Secret { } } -func combineCAs(ca *k8scorev1.Secret) []byte { +func CombineCAs(ca *k8scorev1.Secret) []byte { cas := [][]byte{} sep := []byte("\n") for k, v := range ca.Data { diff --git a/pkg/envoy/server.go b/pkg/envoy/server.go index ffb139c..81910f5 100644 --- a/pkg/envoy/server.go +++ b/pkg/envoy/server.go @@ -73,6 +73,8 @@ type EnvoyADS struct { servicesCache wranglercorev1.ServiceCache secrets wranglercorev1.SecretController secretsCache wranglercorev1.SecretCache + + namespace string } func Register( @@ -95,6 +97,7 @@ func Register( apply: apply, cli: cliCtx, debugEnvoy: false, + namespace: cliCtx.String("namespace"), } return ads @@ -156,7 +159,7 @@ func (e *EnvoyADS) secretOnChange(key string, secret *k8scorev1.Secret) (*k8scor func (e *EnvoyADS) serviceOnChange(key string, service *k8scorev1.Service) (*k8scorev1.Service, error) { if service == nil { - if strings.Contains(key, common.MonitoringNamespace) { + if strings.Contains(key, e.namespace) { if err := e.Sync(); err != nil { e.log.WithError(err).Error("unable to sync") } @@ -200,22 +203,22 @@ func (e *EnvoyADS) Sync() error { func (e *EnvoyADS) SyncClusters(versionID string, clusters []*atlasCluster) error { actualAMServices := []*k8scorev1.Service{} - ca, err := e.secretsCache.Get(common.MonitoringNamespace, common.CASecretName) + ca, err := e.secretsCache.Get(e.namespace, common.CASecretName) if err != nil { return err } - server, err := e.secretsCache.Get(common.MonitoringNamespace, common.ServerSecretName) + server, err := e.secretsCache.Get(e.namespace, common.ServerSecretName) if err != nil { return err } - client, err := e.secretsCache.Get(common.MonitoringNamespace, common.ClientSecretName) + client, err := e.secretsCache.Get(e.namespace, common.ClientSecretName) if err != nil { return err } - amServices, err := e.services.List(common.MonitoringNamespace, v1.ListOptions{ + amServices, err := e.services.List(e.namespace, v1.ListOptions{ LabelSelector: e.cli.String("alertmanager-selector"), }) if err != nil { @@ -258,7 +261,7 @@ func (e *EnvoyADS) SyncClusters(versionID string, clusters []*atlasCluster) erro // Note: we do not send the client cert, because the is controlled by the // static cluster definition for the xds_cluster for dynamic discovery. dsclusterSecretResources := []types.Resource{ - buildSecretTLSValidation("validation", combineCAs(ca)), + buildSecretTLSValidation("validation", CombineCAs(ca)), buildSecretTLSCertificate("server", server.Data["tls.crt"], server.Data["tls.key"]), } @@ -288,7 +291,7 @@ func (e *EnvoyADS) SyncClusters(versionID string, clusters []*atlasCluster) erro } dsclusterSnapshot := cache.NewSnapshot( - versionID, + versionID, // version of snapshot []types.Resource{}, // endpoints dsclusterClusters, // clusters dsclusterRoutes, // routes @@ -322,23 +325,23 @@ func (e *EnvoyADS) SyncObservability(versionID string, clusters []*atlasCluster) addClientSecret = true } - ca, err := e.secretsCache.Get(common.MonitoringNamespace, common.CASecretName) + ca, err := e.secretsCache.Get(e.namespace, common.CASecretName) if err != nil { return err } - server, err := e.secretsCache.Get(common.MonitoringNamespace, common.ServerSecretName) + server, err := e.secretsCache.Get(e.namespace, common.ServerSecretName) if err != nil { return err } - client, err := e.secretsCache.Get(common.MonitoringNamespace, common.ClientSecretName) + client, err := e.secretsCache.Get(e.namespace, common.ClientSecretName) if err != nil { return err } secretResources := []types.Resource{ - buildSecretTLSValidation("validation", combineCAs(ca)), + buildSecretTLSValidation("validation", CombineCAs(ca)), buildSecretTLSCertificate("server", server.Data["tls.crt"], server.Data["tls.key"]), } @@ -526,7 +529,7 @@ func (e *EnvoyADS) getClusters() ([]*atlasCluster, error) { } selector := labels.NewSelector().Add(*requirement) - services, err := e.servicesCache.List(common.MonitoringNamespace, selector) + services, err := e.servicesCache.List(e.namespace, selector) if err != nil { return nil, err } From f5acf60a204c457796eba8d91dd013058ed2b734 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:21:04 -0600 Subject: [PATCH 08/63] switch to using goreleaser --- .github/workflows/release.yml | 94 +++++++++++++++++++++++++++++++++++ .goreleaser.yml | 54 ++++++++++++++++++++ Dockerfile.gorelease | 5 ++ 3 files changed, 153 insertions(+) create mode 100644 .github/workflows/release.yml create mode 100644 .goreleaser.yml create mode 100644 Dockerfile.gorelease diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..3e30452 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,94 @@ +name: release + +on: + workflow_dispatch: + push: + tags: + - "v*" + +jobs: + charts: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - name: Publish Helm charts + uses: stefanprodan/helm-gh-pages@master + with: + token: ${{ secrets.GITHUB_TOKEN }} + + version: + runs-on: ubuntu-latest + if: startsWith(github.head_ref, 'renovate') == false + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + - name: Detect Version for Docker + id: docker-version + run: echo ::set-output name=VERSION::$(SEP="-" .ci/version) + - name: Detect Version + id: version + run: echo ::set-output name=VERSION::$(.ci/version) + outputs: + docker-version: ${{ steps.docker-version.outputs.VERSION }} + version: ${{ steps.version.outputs.VERSION }} + + goreleaser: + runs-on: ubuntu-latest + needs: + - version + env: + SUMMARY: ${{ needs.version.outputs.version }} + VERSION: ${{ needs.version.outputs.version }} + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + - uses: actions/setup-node@v2 + with: + node-version: 12 + - name: Set up Go + uses: actions/setup-go@v2 + with: + go-version: 1.16 + - name: Login to GitHub Container Registry + uses: docker/login-action@v1 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Extract branch name + shell: bash + run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})" + id: extract_branch + - name: Run GoReleaser for Tags + uses: goreleaser/goreleaser-action@v2 + if: startsWith(github.ref, 'refs/tags/v') + with: + distribution: goreleaser + version: latest + args: release --rm-dist + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + BRANCH: ${{ steps.extract_branch.outputs.branch }} + - name: Run GoReleaser for Non-Tags + uses: goreleaser/goreleaser-action@v2 + if: startsWith(github.ref, 'refs/tags/v') == false + with: + distribution: goreleaser + version: latest + args: release --rm-dist --snapshot --skip-publish + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + BRANCH: ${{ steps.extract_branch.outputs.branch }} + - name: Artifact Upload + uses: actions/upload-artifact@v2 + if: startsWith(github.ref , 'refs/tags/v') == false + with: + name: release + path: | + release/*.tar.gz + release/config.yaml + release/checksums.txt diff --git a/.goreleaser.yml b/.goreleaser.yml new file mode 100644 index 0000000..81e9bde --- /dev/null +++ b/.goreleaser.yml @@ -0,0 +1,54 @@ +dist: release +env: + - PACKAGE_NAME=github.com/ekristen/atlas + - SUMMARY={{ .Env.SUMMARY }} + - VERSION={{ .Env.VERSION }} + - BRANCH={{ .Env.BRANCH }} +before: + hooks: + - go mod vendor + - go mod tidy +release: + github: + owner: ekristen + name: atlas +builds: + - id: linux + goos: + - linux + goarch: + - amd64 + ignore: + - goarch: 386 + ldflags: + - -X {{.Env.PACKAGE_NAME}}/pkg/common.SUMMARY={{.Env.SUMMARY}} -X {{.Env.PACKAGE_NAME}}/pkg/common.BRANCH={{.Env.BRANCH}} -X {{.Env.PACKAGE_NAME}}/pkg/common.VERSION={{.Env.VERSION}} +archives: + - replacements: + 386: i386 + amd64: x86_64 +dockers: + - use: buildx + goos: linux + goarch: amd64 + dockerfile: Dockerfile.gorelease + image_templates: + - ghcr.io/ekristen/atlas:latest + - ghcr.io/ekristen/atlas:{{ .Tag }} + build_flag_templates: + - "--pull" + - "--label=org.opencontainers.image.created={{.Date}}" + - "--label=org.opencontainers.image.title={{.ProjectName}}" + - "--label=org.opencontainers.image.revision={{.FullCommit}}" + - "--label=org.opencontainers.image.version={{.Version}}" + - "--label=org.opencontainers.image.source=https://github.com/ekristen/atlas" + - "--platform=linux/amd64" +checksum: + name_template: "checksums.txt" +snapshot: + name_template: "{{ .Tag }}-next" +changelog: + sort: asc + filters: + exclude: + - "^docs:" + - "^test:" diff --git a/Dockerfile.gorelease b/Dockerfile.gorelease new file mode 100644 index 0000000..bf67a19 --- /dev/null +++ b/Dockerfile.gorelease @@ -0,0 +1,5 @@ + +FROM debian +ENTRYPOINT ["/usr/local/bin/atlas"] +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* +COPY atlas /usr/local/bin/atlas From 045a0492c294f85980a1d9616b7138a43d6a3720 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:21:56 -0600 Subject: [PATCH 09/63] switching to goatlas-io --- .goreleaser.yml | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/.goreleaser.yml b/.goreleaser.yml index 81e9bde..dbdb366 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -1,6 +1,6 @@ dist: release env: - - PACKAGE_NAME=github.com/ekristen/atlas + - PACKAGE_NAME=github.com/goatlas-io/atlas - SUMMARY={{ .Env.SUMMARY }} - VERSION={{ .Env.VERSION }} - BRANCH={{ .Env.BRANCH }} @@ -10,7 +10,7 @@ before: - go mod tidy release: github: - owner: ekristen + owner: goatlas-io name: atlas builds: - id: linux @@ -32,15 +32,14 @@ dockers: goarch: amd64 dockerfile: Dockerfile.gorelease image_templates: - - ghcr.io/ekristen/atlas:latest - - ghcr.io/ekristen/atlas:{{ .Tag }} + - ghcr.io/goatlas-io/atlas:{{ .Tag }} build_flag_templates: - "--pull" - "--label=org.opencontainers.image.created={{.Date}}" - "--label=org.opencontainers.image.title={{.ProjectName}}" - "--label=org.opencontainers.image.revision={{.FullCommit}}" - "--label=org.opencontainers.image.version={{.Version}}" - - "--label=org.opencontainers.image.source=https://github.com/ekristen/atlas" + - "--label=org.opencontainers.image.source=https://github.com/goatlas-io/atlas" - "--platform=linux/amd64" checksum: name_template: "checksums.txt" From de5721b93102f00a0785743714de394f79e314ac Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 20:23:16 -0600 Subject: [PATCH 10/63] change module name to github.com/goatlas-io/atlas --- charts/envoy/Chart.yaml | 2 +- go.mod | 2 +- main.go | 4 ++-- pkg/commands/cluster-add.go | 2 +- pkg/commands/cluster-values.go | 2 +- pkg/commands/controller.go | 8 ++++---- pkg/commands/envoy-ads.go | 8 ++++---- pkg/controllers/atlas/controller.go | 6 +++--- pkg/envoy/metrics.go | 2 +- pkg/envoy/server.go | 4 ++-- 10 files changed, 20 insertions(+), 20 deletions(-) diff --git a/charts/envoy/Chart.yaml b/charts/envoy/Chart.yaml index 6db158a..111ecf8 100755 --- a/charts/envoy/Chart.yaml +++ b/charts/envoy/Chart.yaml @@ -8,5 +8,5 @@ keywords: - atlas name: envoy sources: -- https://github.com/ekristen/atlas +- https://github.com/goatlas-io/atlas version: 1.0.0 diff --git a/go.mod b/go.mod index c704081..a7f2378 100644 --- a/go.mod +++ b/go.mod @@ -1,4 +1,4 @@ -module github.com/ekristen/atlas +module github.com/goatlas-io/atlas go 1.16 diff --git a/main.go b/main.go index 696a1d6..862e911 100644 --- a/main.go +++ b/main.go @@ -7,8 +7,8 @@ import ( "github.com/sirupsen/logrus" "github.com/urfave/cli/v2" - _ "github.com/ekristen/atlas/pkg/commands" - "github.com/ekristen/atlas/pkg/common" + _ "github.com/goatlas-io/atlas/pkg/commands" + "github.com/goatlas-io/atlas/pkg/common" ) func main() { diff --git a/pkg/commands/cluster-add.go b/pkg/commands/cluster-add.go index 90e5e52..aef797b 100644 --- a/pkg/commands/cluster-add.go +++ b/pkg/commands/cluster-add.go @@ -15,7 +15,7 @@ import ( "k8s.io/apimachinery/pkg/util/intstr" "k8s.io/client-go/kubernetes" - "github.com/ekristen/atlas/pkg/common" + "github.com/goatlas-io/atlas/pkg/common" ) type clusterAddCommand struct { diff --git a/pkg/commands/cluster-values.go b/pkg/commands/cluster-values.go index 6fd6dc2..73c3f30 100644 --- a/pkg/commands/cluster-values.go +++ b/pkg/commands/cluster-values.go @@ -17,7 +17,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/client-go/kubernetes" - "github.com/ekristen/atlas/pkg/common" + "github.com/goatlas-io/atlas/pkg/common" ) //go:embed templates/* diff --git a/pkg/commands/controller.go b/pkg/commands/controller.go index 9e8b5f1..7369725 100644 --- a/pkg/commands/controller.go +++ b/pkg/commands/controller.go @@ -14,10 +14,10 @@ import ( "k8s.io/apimachinery/pkg/util/runtime" "k8s.io/client-go/kubernetes" - "github.com/ekristen/atlas/pkg/common" - "github.com/ekristen/atlas/pkg/config" - "github.com/ekristen/atlas/pkg/controllers/atlas" - "github.com/ekristen/atlas/pkg/metrics" + "github.com/goatlas-io/atlas/pkg/common" + "github.com/goatlas-io/atlas/pkg/config" + "github.com/goatlas-io/atlas/pkg/controllers/atlas" + "github.com/goatlas-io/atlas/pkg/metrics" ) type controlCommand struct{} diff --git a/pkg/commands/envoy-ads.go b/pkg/commands/envoy-ads.go index 2451a71..3ec81ee 100644 --- a/pkg/commands/envoy-ads.go +++ b/pkg/commands/envoy-ads.go @@ -15,10 +15,10 @@ import ( "k8s.io/apimachinery/pkg/util/runtime" "k8s.io/client-go/kubernetes" - "github.com/ekristen/atlas/pkg/common" - "github.com/ekristen/atlas/pkg/config" - "github.com/ekristen/atlas/pkg/envoy" - "github.com/ekristen/atlas/pkg/metrics" + "github.com/goatlas-io/atlas/pkg/common" + "github.com/goatlas-io/atlas/pkg/config" + "github.com/goatlas-io/atlas/pkg/envoy" + "github.com/goatlas-io/atlas/pkg/metrics" ) type envoyADSCommand struct{} diff --git a/pkg/controllers/atlas/controller.go b/pkg/controllers/atlas/controller.go index 4d61572..aba07f2 100644 --- a/pkg/controllers/atlas/controller.go +++ b/pkg/controllers/atlas/controller.go @@ -37,9 +37,9 @@ import ( core "github.com/rancher/wrangler/pkg/generated/controllers/core/v1" "github.com/rancher/wrangler/pkg/relatedresource" - "github.com/ekristen/atlas/pkg/common" - "github.com/ekristen/atlas/pkg/config" - "github.com/ekristen/atlas/pkg/envoy" + "github.com/goatlas-io/atlas/pkg/common" + "github.com/goatlas-io/atlas/pkg/config" + "github.com/goatlas-io/atlas/pkg/envoy" ) //go:embed templates/* diff --git a/pkg/envoy/metrics.go b/pkg/envoy/metrics.go index 366cb1d..826016b 100644 --- a/pkg/envoy/metrics.go +++ b/pkg/envoy/metrics.go @@ -1,7 +1,7 @@ package envoy import ( - "github.com/ekristen/atlas/pkg/metrics" + "github.com/goatlas-io/atlas/pkg/metrics" "github.com/prometheus/client_golang/prometheus" ) diff --git a/pkg/envoy/server.go b/pkg/envoy/server.go index 81910f5..8b0f854 100644 --- a/pkg/envoy/server.go +++ b/pkg/envoy/server.go @@ -13,8 +13,8 @@ import ( "github.com/urfave/cli/v2" "google.golang.org/grpc" - "github.com/ekristen/atlas/pkg/common" - "github.com/ekristen/atlas/pkg/config" + "github.com/goatlas-io/atlas/pkg/common" + "github.com/goatlas-io/atlas/pkg/config" "github.com/rancher/wrangler/pkg/apply" wranglercorev1 "github.com/rancher/wrangler/pkg/generated/controllers/core/v1" From 790b0e2c17a2daa198a391c77c7adaf24533ed91 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 21:10:36 -0600 Subject: [PATCH 11/63] allow for manual run of release-charts --- .github/workflows/release-charts.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml index f508c2a..6309667 100644 --- a/.github/workflows/release-charts.yml +++ b/.github/workflows/release-charts.yml @@ -1,6 +1,7 @@ name: release-charts on: + workflow_dispatch: push: branches: - main From 5c9a4d51eb904de5f3bbe97f29243ba82e2ffc12 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 21:23:20 -0600 Subject: [PATCH 12/63] fix: github action for chart release --- .github/workflows/release-charts.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml index 6309667..4f0bd2e 100644 --- a/.github/workflows/release-charts.yml +++ b/.github/workflows/release-charts.yml @@ -14,9 +14,11 @@ jobs: - name: Publish Helm charts uses: stefanprodan/helm-gh-pages@master with: - token: ${{ secrets.GITHUB_TOKEN }} + token: ${{ secrets.BOT_GITHUB_TOKEN }} charts_url: https://charts.goatlas.io/ owner: goatlas-io repository: charts branch: gh-pages target_dir: charts + commit_username: ekristen + commit_email: erik@erikkristensen.com From c969eea00f2bdef62053bfa147382d6401ed7d7a Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 21:25:52 -0600 Subject: [PATCH 13/63] pin action version --- .github/workflows/release-charts.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml index 4f0bd2e..1c35aa9 100644 --- a/.github/workflows/release-charts.yml +++ b/.github/workflows/release-charts.yml @@ -12,7 +12,7 @@ jobs: steps: - uses: actions/checkout@v2 - name: Publish Helm charts - uses: stefanprodan/helm-gh-pages@master + uses: stefanprodan/helm-gh-pages@v1.4.1 with: token: ${{ secrets.BOT_GITHUB_TOKEN }} charts_url: https://charts.goatlas.io/ From 1772c03971c0efbae369867302e654a3013aa5a8 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 21:31:14 -0600 Subject: [PATCH 14/63] fix release-charts --- .github/workflows/release-charts.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml index 1c35aa9..bb38011 100644 --- a/.github/workflows/release-charts.yml +++ b/.github/workflows/release-charts.yml @@ -16,9 +16,9 @@ jobs: with: token: ${{ secrets.BOT_GITHUB_TOKEN }} charts_url: https://charts.goatlas.io/ + charts_dir: charts owner: goatlas-io repository: charts branch: gh-pages - target_dir: charts commit_username: ekristen commit_email: erik@erikkristensen.com From 4cca625953f6df3111f2f992c04b6b41c346f304 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 21:34:02 -0600 Subject: [PATCH 15/63] attempt to fix chart release --- .github/workflows/release-charts.yml | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml index bb38011..fb6e72e 100644 --- a/.github/workflows/release-charts.yml +++ b/.github/workflows/release-charts.yml @@ -14,11 +14,9 @@ jobs: - name: Publish Helm charts uses: stefanprodan/helm-gh-pages@v1.4.1 with: - token: ${{ secrets.BOT_GITHUB_TOKEN }} - charts_url: https://charts.goatlas.io/ charts_dir: charts + charts_url: https://charts.goatlas.io owner: goatlas-io repository: charts branch: gh-pages - commit_username: ekristen - commit_email: erik@erikkristensen.com + linting: off From af57deaaef89f5a2a5aed427abbc49d7aec4f7a2 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sun, 10 Oct 2021 21:35:45 -0600 Subject: [PATCH 16/63] attempt to fix chart release --- .github/workflows/release-charts.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/release-charts.yml b/.github/workflows/release-charts.yml index fb6e72e..b19bb37 100644 --- a/.github/workflows/release-charts.yml +++ b/.github/workflows/release-charts.yml @@ -12,8 +12,9 @@ jobs: steps: - uses: actions/checkout@v2 - name: Publish Helm charts - uses: stefanprodan/helm-gh-pages@v1.4.1 + uses: stefanprodan/helm-gh-pages@master with: + token: ${{ secrets.BOT_GITHUB_TOKEN }} charts_dir: charts charts_url: https://charts.goatlas.io owner: goatlas-io From 040224564c9ff97e867d4ab02d745512d8581b28 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Mon, 11 Oct 2021 17:34:52 -0600 Subject: [PATCH 17/63] fixing helm chart deps and deployment --- .gitignore | 1 + charts/atlas/Chart.lock | 6 ++++++ charts/atlas/Chart.yaml | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 charts/atlas/Chart.lock diff --git a/.gitignore b/.gitignore index 8c90d15..eb9d066 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,7 @@ vendor hack/deployment/**/*.json hack/deployment/**/*.yaml +charts/**/*.tgz release archive diff --git a/charts/atlas/Chart.lock b/charts/atlas/Chart.lock new file mode 100644 index 0000000..d11cb8f --- /dev/null +++ b/charts/atlas/Chart.lock @@ -0,0 +1,6 @@ +dependencies: +- name: envoy + repository: https://charts.goatlas.io + version: 1.0.0 +digest: sha256:bfcdfc90c3124bee3bb7d6e732bd265e509d0ceecefa7b7cd34e5ead56fee657 +generated: "2021-10-11T17:34:17.140789-06:00" diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index 60fc6b0..a395eae 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -6,4 +6,4 @@ description: Atlas delivers automated secure communications between Prometheus a dependencies: - name: envoy version: 1.0.0 - repository: https://ekristen.github.com/atlas/ \ No newline at end of file + repository: https://charts.goatlas.io \ No newline at end of file From d7e698b228c8b919692d750857ae5e8777721673 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Mon, 11 Oct 2021 20:17:30 -0600 Subject: [PATCH 18/63] publish docs on main push for now --- .github/workflows/release-docs.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/release-docs.yml b/.github/workflows/release-docs.yml index 6195580..afbf09d 100644 --- a/.github/workflows/release-docs.yml +++ b/.github/workflows/release-docs.yml @@ -3,8 +3,8 @@ name: release-docs on: workflow_dispatch: push: - tags: - - "v*" + branches: + - main jobs: publish: From 2434a445563c420a0c947ffe49d1f9c92bb369f0 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Mon, 11 Oct 2021 20:18:44 -0600 Subject: [PATCH 19/63] use bot token --- .github/workflows/release-docs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release-docs.yml b/.github/workflows/release-docs.yml index afbf09d..5e4b79d 100644 --- a/.github/workflows/release-docs.yml +++ b/.github/workflows/release-docs.yml @@ -26,4 +26,4 @@ jobs: target_branch: gh-pages build_dir: public env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITHUB_TOKEN: ${{ secrets.BOT_GITHUB_TOKEN }} From 44ec7f3a66159d5928275cce77a41a981b8c5f37 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 16:49:06 -0600 Subject: [PATCH 20/63] tweak atlas envoy chart and renovate --- .github/renovate.json | 20 ++++++++++++++++++++ charts/envoy/Chart.yaml | 2 +- charts/envoy/values.yaml | 10 +++++----- pkg/commands/templates/helm-chart.tmpl | 2 +- pkg/commands/templates/helm-release.tmpl | 12 ++++++------ 5 files changed, 33 insertions(+), 13 deletions(-) diff --git a/.github/renovate.json b/.github/renovate.json index d70e10a..80c6381 100644 --- a/.github/renovate.json +++ b/.github/renovate.json @@ -20,6 +20,16 @@ "matchPackagePatterns": [ "^github.com/aws/aws-sdk-go-v2/.*" ] + }, + { + "groupName": "atlas-envoy", + "groupSlug": "atlas-envoy", + "matchPackagePatterns": [ + "envoy" + ], + "matchDatasources": [ + "helm" + ] } ], "regexManagers": [ @@ -32,6 +42,16 @@ "image:\\srepository:\\s(?.*)\\stag:\\s(?.*)\\s?" ], "datasourceTemplate": "github-releases" + }, + { + "fileMatch": [ + ".*.tmpl$" + ], + "matchStrings": [ + "chart:\\shttps://charts.goatlas.io/envoy-(?.*).tgz\\s?", + "spec:\\s+chart:\\s(?.*)\\s+version:\\s(?.*)\\s+?" + ], + "datasourceTemplate": "helm" } ] } \ No newline at end of file diff --git a/charts/envoy/Chart.yaml b/charts/envoy/Chart.yaml index 111ecf8..5b389be 100755 --- a/charts/envoy/Chart.yaml +++ b/charts/envoy/Chart.yaml @@ -9,4 +9,4 @@ keywords: name: envoy sources: - https://github.com/goatlas-io/atlas -version: 1.0.0 +version: 1.1.0 diff --git a/charts/envoy/values.yaml b/charts/envoy/values.yaml index bd7c2f9..75d7624 100755 --- a/charts/envoy/values.yaml +++ b/charts/envoy/values.yaml @@ -1,4 +1,7 @@ -replicaCount: 2 +atlas: + alertmanagerCount: 0 + +replicaCount: 1 podDisruptionBudget: | maxUnavailable: 1 @@ -9,7 +12,7 @@ terminationGracePeriodSeconds: 30 strategy: | type: RollingUpdate rollingUpdate: - maxSurge: 2 + maxSurge: 1 maxUnavailable: 1 image: @@ -346,6 +349,3 @@ prometheusRule: ## Added to the PrometheusRule object so that prometheus-operator is able to discover it ## ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#prometheusspec additionalLabels: {} - -atlas: - alertmanagerCount: 1 diff --git a/pkg/commands/templates/helm-chart.tmpl b/pkg/commands/templates/helm-chart.tmpl index 9c66cd4..b89ec7e 100644 --- a/pkg/commands/templates/helm-chart.tmpl +++ b/pkg/commands/templates/helm-chart.tmpl @@ -5,6 +5,6 @@ metadata: name: atlas-envoy namespace: {{ .Namespace }} spec: - chart: https://charts.helm.sh/stable/packages/envoy-1.9.4.tgz + chart: https://charts.goatlas.io/envoy-1.0.1.tgz valuesContent: | {{ .Values | indent 4 }} \ No newline at end of file diff --git a/pkg/commands/templates/helm-release.tmpl b/pkg/commands/templates/helm-release.tmpl index 196c017..e9a1fb7 100644 --- a/pkg/commands/templates/helm-release.tmpl +++ b/pkg/commands/templates/helm-release.tmpl @@ -2,26 +2,26 @@ apiVersion: source.toolkit.fluxcd.io/v1beta1 kind: HelmRepository metadata: - name: atlas-helm-stable + name: atlas namespace: {{ .Namespace }} spec: - url: https://charts.helm.sh/stable + url: https://charts.goatlas.io interval: 24h --- apiVersion: helm.toolkit.fluxcd.io/v2beta1 kind: HelmRelease metadata: - name: atlas-thanos + name: atlas-envoy namespace: {{ .Namespace }} spec: - releaseName: atlas-thanos + releaseName: atlas-envoy interval: 2m chart: spec: chart: envoy - version: 1.9.4 + version: 1.0.0 sourceRef: kind: HelmRepository - name: atlas-helm-stable + name: atlas values: {{ .Values | indent 4 }} \ No newline at end of file From 1e727fcb1515e6b2401ec433f1377bca239157fe Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 16:54:04 -0600 Subject: [PATCH 21/63] ensure CNAME is copied over for docs --- docs/CNAME | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/CNAME diff --git a/docs/CNAME b/docs/CNAME new file mode 100644 index 0000000..02486b0 --- /dev/null +++ b/docs/CNAME @@ -0,0 +1 @@ +docs.goatlas.io \ No newline at end of file From 33af4faa77ac208abcce8b56e21c7b1bc8d7a9fa Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 16:55:33 -0600 Subject: [PATCH 22/63] fixing metadata for mkdocs --- mkdocs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs.yml b/mkdocs.yml index a97156f..2f92fba 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ site_name: Atlas -site_url: https://goatlas.io/docs/ +site_url: https://docs.goatlas.io/ repo_name: goatlas-io/atlas repo_url: https://github.com/goatlas-io/atlas edit_uri: "" From 511a74de1f97c4d8e334bd1460a8512db92c31f7 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 16:59:53 -0600 Subject: [PATCH 23/63] fix renovate configuration for helm updates --- .github/renovate.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/renovate.json b/.github/renovate.json index 80c6381..5451f56 100644 --- a/.github/renovate.json +++ b/.github/renovate.json @@ -51,7 +51,8 @@ "chart:\\shttps://charts.goatlas.io/envoy-(?.*).tgz\\s?", "spec:\\s+chart:\\s(?.*)\\s+version:\\s(?.*)\\s+?" ], - "datasourceTemplate": "helm" + "datasourceTemplate": "helm", + "registryUrlTemplate": "https://charts.goatlas.io" } ] } \ No newline at end of file From 889fb3775eae517d37741e297f9780698b6bd48c Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 17:34:57 -0600 Subject: [PATCH 24/63] documentation update --- .github/workflows/release.yml | 3 --- docs/deployment.md | 8 ++++++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 3e30452..5a96ac4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -46,9 +46,6 @@ jobs: uses: actions/checkout@v2 with: fetch-depth: 0 - - uses: actions/setup-node@v2 - with: - node-version: 12 - name: Set up Go uses: actions/setup-go@v2 with: diff --git a/docs/deployment.md b/docs/deployment.md index ac0754a..4548f11 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -140,6 +140,12 @@ Retrieve the downstream's helm values with the `atlas` or `kubectl` atlas cluster-values --name "downstream1" > downstream1.yaml ``` +**Note:** This command has `--format` option, the default is `raw` which is just values for helm. The other options are `helm-chart` and `helm-release` + +- `helm-chart` -- this is a feature from Rancher on K3S clusters +- `helm-release` -- this is for Flux V2 +- `raw` -- just values for helm install/upgrade commands + OR ```bash @@ -152,8 +158,6 @@ Once you have the values, install helm on your downstream cluster. Make sure you helm install envoy --values downstream1.yaml chart/ ``` -Please note that - ### Step 5. Repeat If you have more than one downstream cluster, repeast steps 3 and 4 until you've added all your clusters. From aab57d8932456e4b64d9de73ff2ab4d9d22140da Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 19:46:34 -0600 Subject: [PATCH 25/63] build on push to main branch --- .github/workflows/release.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5a96ac4..b125193 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -3,6 +3,8 @@ name: release on: workflow_dispatch: push: + branches: + - main tags: - "v*" From f62b9c497ef53d3fd86efa7b1bbb1fda559927c9 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 19:50:08 -0600 Subject: [PATCH 26/63] remove charts release from the workflow file --- .github/workflows/release.yml | 9 --------- 1 file changed, 9 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b125193..b392e74 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -9,15 +9,6 @@ on: - "v*" jobs: - charts: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Publish Helm charts - uses: stefanprodan/helm-gh-pages@master - with: - token: ${{ secrets.GITHUB_TOKEN }} - version: runs-on: ubuntu-latest if: startsWith(github.head_ref, 'renovate') == false From c31f64443d8a5bfdad98175e497eaaf9427a0a61 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 19:55:01 -0600 Subject: [PATCH 27/63] working on building atlas on commits --- .github/workflows/release.yml | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b392e74..c9535da 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -55,7 +55,6 @@ jobs: id: extract_branch - name: Run GoReleaser for Tags uses: goreleaser/goreleaser-action@v2 - if: startsWith(github.ref, 'refs/tags/v') with: distribution: goreleaser version: latest @@ -63,16 +62,6 @@ jobs: env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} BRANCH: ${{ steps.extract_branch.outputs.branch }} - - name: Run GoReleaser for Non-Tags - uses: goreleaser/goreleaser-action@v2 - if: startsWith(github.ref, 'refs/tags/v') == false - with: - distribution: goreleaser - version: latest - args: release --rm-dist --snapshot --skip-publish - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - BRANCH: ${{ steps.extract_branch.outputs.branch }} - name: Artifact Upload uses: actions/upload-artifact@v2 if: startsWith(github.ref , 'refs/tags/v') == false From 7fa4985b4a2db5f2245e59df33b02fcc9d4f5647 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 20:16:25 -0600 Subject: [PATCH 28/63] improving docker image tags --- .goreleaser.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.goreleaser.yml b/.goreleaser.yml index dbdb366..818ffe4 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -33,6 +33,7 @@ dockers: dockerfile: Dockerfile.gorelease image_templates: - ghcr.io/goatlas-io/atlas:{{ .Tag }} + - ghcr.io/goatlas-io/atlas:{{ .Branch }}-{{ .ShortCommit }}-{{ .Timestamp }} build_flag_templates: - "--pull" - "--label=org.opencontainers.image.created={{.Date}}" From 7df3d87d40df2aeffcfd468666afecdfb804506b Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 20:30:26 -0600 Subject: [PATCH 29/63] working docker tags --- .github/workflows/release.yml | 2 +- .goreleaser.yml | 4 ++-- Makefile | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c9535da..87f4054 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -32,7 +32,7 @@ jobs: needs: - version env: - SUMMARY: ${{ needs.version.outputs.version }} + SUMMARY: ${{ needs.version.outputs.docker-version }} VERSION: ${{ needs.version.outputs.version }} steps: - name: Checkout diff --git a/.goreleaser.yml b/.goreleaser.yml index 818ffe4..ec51315 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -32,7 +32,7 @@ dockers: goarch: amd64 dockerfile: Dockerfile.gorelease image_templates: - - ghcr.io/goatlas-io/atlas:{{ .Tag }} + - ghcr.io/goatlas-io/atlas:v{{ .Version }} - ghcr.io/goatlas-io/atlas:{{ .Branch }}-{{ .ShortCommit }}-{{ .Timestamp }} build_flag_templates: - "--pull" @@ -45,7 +45,7 @@ dockers: checksum: name_template: "checksums.txt" snapshot: - name_template: "{{ .Tag }}-next" + name_template: "{{ .Tag }}" changelog: sort: asc filters: diff --git a/Makefile b/Makefile index badde15..51cd645 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ BRANCH := $(shell bash .ci/branch) -SUMMARY := $(shell bash .ci/version) +SUMMARY := $(shell SEP="-" bash .ci/version) VERSION := $(shell cat VERSION) NAME := $(shell basename `pwd`) MODULE := $(shell head -n1 go.mod | cut -f2 -d' ') @@ -19,4 +19,4 @@ build: SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser build snapshot: - SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser release --snapshot --skip-publish --rm-dist \ No newline at end of file + GORELEASER_CURRENT_TAG=$(SUMMARY) SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser release --snapshot --skip-publish --rm-dist \ No newline at end of file From 173b7b284b468b36fd0e4692855839df3bb7c2cd Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Wed, 13 Oct 2021 20:39:49 -0600 Subject: [PATCH 30/63] build for darwin too --- .goreleaser.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.goreleaser.yml b/.goreleaser.yml index ec51315..177426d 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -22,6 +22,15 @@ builds: - goarch: 386 ldflags: - -X {{.Env.PACKAGE_NAME}}/pkg/common.SUMMARY={{.Env.SUMMARY}} -X {{.Env.PACKAGE_NAME}}/pkg/common.BRANCH={{.Env.BRANCH}} -X {{.Env.PACKAGE_NAME}}/pkg/common.VERSION={{.Env.VERSION}} + - id: darwin + goos: + - darwin + goarch: + - amd64 + ignore: + - goarch: 386 + ldflags: + - -X {{.Env.PACKAGE_NAME}}/pkg/common.SUMMARY={{.Env.SUMMARY}} -X {{.Env.PACKAGE_NAME}}/pkg/common.BRANCH={{.Env.BRANCH}} -X {{.Env.PACKAGE_NAME}}/pkg/common.VERSION={{.Env.VERSION}} archives: - replacements: 386: i386 From ea2adcd5e30628806a8a98df3a8e8df8bb273d0c Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 12:58:26 -0600 Subject: [PATCH 31/63] fix docker images --- .github/workflows/release.yml | 11 +++++++++++ .goreleaser.yml | 2 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 87f4054..c040b4e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -53,8 +53,19 @@ jobs: shell: bash run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})" id: extract_branch + - name: Run GoReleaser for Non-Tags + uses: goreleaser/goreleaser-action@v2 + if: startsWith(github.ref , 'refs/tags/v') == false + with: + distribution: goreleaser + version: latest + args: release --rm-dist --snapshot + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + BRANCH: ${{ steps.extract_branch.outputs.branch }} - name: Run GoReleaser for Tags uses: goreleaser/goreleaser-action@v2 + if: startsWith(github.ref , 'refs/tags/v') == true with: distribution: goreleaser version: latest diff --git a/.goreleaser.yml b/.goreleaser.yml index 177426d..812b89a 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -41,7 +41,7 @@ dockers: goarch: amd64 dockerfile: Dockerfile.gorelease image_templates: - - ghcr.io/goatlas-io/atlas:v{{ .Version }} + - ghcr.io/goatlas-io/atlas:{{ .Version }} - ghcr.io/goatlas-io/atlas:{{ .Branch }}-{{ .ShortCommit }}-{{ .Timestamp }} build_flag_templates: - "--pull" From 12a626a2ea914160b4160453a8e5a3c6ee2b3894 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 13:12:22 -0600 Subject: [PATCH 32/63] switch to main --- .ci/tag-version | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/.ci/tag-version b/.ci/tag-version index 0545f29..09e5fb8 100755 --- a/.ci/tag-version +++ b/.ci/tag-version @@ -2,7 +2,7 @@ TAG=${1} VALID_TAG=false -MASTER_ONLY=true +MAIN_ONLY=true if [ "x${TAG}" == "x" ]; then echo "You must specify a tag" @@ -23,16 +23,16 @@ fi if echo "${TAG}" | grep -qE "^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$"; then VALID_TAG=true - MASTER_ONLY=false + MAIN_ONLY=false fi if echo "${TAG}" | grep -qE "^[0-9]+\.[0-9]+\.[0-9]+-rc[0-9]+$"; then VALID_TAG=true - MASTER_ONLY=false + MAIN_ONLY=false fi -if [ "${BRANCH}" != "master" ] && [ "${MASTER_ONLY}" == "true" ]; then - echo "You must only be on the master branch" +if [ "${BRANCH}" != "main" ] && [ "${MAIN_ONLY}" == "true" ]; then + echo "You must only be on the main branch" exit 12 fi From 45cce91eb11181cb98d960a15ee86e9e0d46dab2 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 13:12:26 -0600 Subject: [PATCH 33/63] version: 0.1.1 --- VERSION | 2 +- hack/deployment/deploy.sh | 8 ++------ pkg/common/version.go | 2 +- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/VERSION b/VERSION index 6c6aa7c..17e51c3 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.0 \ No newline at end of file +0.1.1 diff --git a/hack/deployment/deploy.sh b/hack/deployment/deploy.sh index 47dd7b4..f9bf63e 100644 --- a/hack/deployment/deploy.sh +++ b/hack/deployment/deploy.sh @@ -476,7 +476,7 @@ function setup_observability { setup_atlas_values "$IP_ADDRESS" - helm upgrade --kubeconfig observability/kubeconfig.yaml -i -n $NAMESPACE --values observability/atlas-values.yaml atlas ../../chart + helm upgrade --kubeconfig observability/kubeconfig.yaml -i -n $NAMESPACE --values observability/atlas-values.yaml atlas ../../charts/atlas } function setup_downstream { @@ -515,10 +515,6 @@ function setup_downstream { fi IP_ADDRESS=$(jq -r '.[0].networks.v4[0].ip_address' < "$name/droplet.json") - - setup_am "$name" - - KUBECONFIG="$name/kubeconfig.yaml" kubectl apply -f "$name/am.yaml" setup_downstream_prometheus "$name" "$IP_ADDRESS" "$(jq -r '.[0].networks.v4[0].ip_address' < observability/droplet.json)" @@ -582,7 +578,7 @@ function config_cluster_envoy() { rm -f "$name/envoy-values.yaml" KUBECONFIG="observability/kubeconfig.yaml" kubectl get secret -n $NAMESPACE "$name-envoy-values" -o json | jq -r '.data["values.yaml"]' | base64 -D > "$name/envoy-values.yaml" - helm upgrade --kubeconfig "$name/kubeconfig.yaml" -i -n $NAMESPACE --values "$name/envoy-values.yaml" atlas-envoy stable/envoy + helm upgrade --kubeconfig "$name/kubeconfig.yaml" -i -n $NAMESPACE --values "$name/envoy-values.yaml" atlas-envoy ../../charts/envoy } function build_downstream_cluster() { diff --git a/pkg/common/version.go b/pkg/common/version.go index 1ce4871..9db7770 100644 --- a/pkg/common/version.go +++ b/pkg/common/version.go @@ -10,7 +10,7 @@ var SUMMARY = "0.1.0-dev" var BRANCH = "dev" // VERSION of Release -var VERSION = "0.1.0" +var VERSION = "0.1.1" // AppVersion -- var AppVersion AppVersionInfo From 8c87b1c9f14e09cee712abadb7d2d6166c1a19e6 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 17:54:14 -0600 Subject: [PATCH 34/63] build docker images for branch separately --- .github/workflows/docker.yml | 51 +++++++++++++++++++++++++++++++++++ .github/workflows/release.yml | 10 ------- .goreleaser.yml | 3 +-- 3 files changed, 52 insertions(+), 12 deletions(-) create mode 100644 .github/workflows/docker.yml diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml new file mode 100644 index 0000000..d70daaa --- /dev/null +++ b/.github/workflows/docker.yml @@ -0,0 +1,51 @@ +name: docker + +on: + workflow_dispatch: + push: + branches: + - master + pull_request: + branches: + - master + +jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Get current date + id: date + run: echo "::set-output name=date::$(date +'%s')" + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + - name: Get branch names + id: branch-name + uses: tj-actions/branch-names@v4.9 + - name: Setup Docker Metadata + id: docker_metadata + uses: crazy-max/ghaction-docker-meta@v3 + with: + images: | + ghcr.io/goatlas-io/atlas + tags: | + type=sha,prefix=${{ steps.branch-name.outputs.current_branch }}-,suffix=-${{ steps.date.outputs.date }} + - name: Setup Docker Buildx + uses: docker/setup-buildx-action@v1 + - name: Login to GitHub Container Registry + uses: docker/login-action@v1 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Build to Registry + uses: docker/build-push-action@v2 + with: + context: . + tags: ${{ steps.docker_metadata.outputs.tags }} + labels: ${{ steps.docker_metadata.outputs.labels }} + build-args: | + BRANCH=${{ steps.branch-name.outputs.current_branch }} + cache-to: type=inline + outputs: type=registry diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c040b4e..e4a0f39 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -53,16 +53,6 @@ jobs: shell: bash run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})" id: extract_branch - - name: Run GoReleaser for Non-Tags - uses: goreleaser/goreleaser-action@v2 - if: startsWith(github.ref , 'refs/tags/v') == false - with: - distribution: goreleaser - version: latest - args: release --rm-dist --snapshot - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - BRANCH: ${{ steps.extract_branch.outputs.branch }} - name: Run GoReleaser for Tags uses: goreleaser/goreleaser-action@v2 if: startsWith(github.ref , 'refs/tags/v') == true diff --git a/.goreleaser.yml b/.goreleaser.yml index 812b89a..6ebfe7e 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -41,8 +41,7 @@ dockers: goarch: amd64 dockerfile: Dockerfile.gorelease image_templates: - - ghcr.io/goatlas-io/atlas:{{ .Version }} - - ghcr.io/goatlas-io/atlas:{{ .Branch }}-{{ .ShortCommit }}-{{ .Timestamp }} + - ghcr.io/goatlas-io/atlas:v{{ .Version }} build_flag_templates: - "--pull" - "--label=org.opencontainers.image.created={{.Date}}" From 4a369229168759711aa9cb2b505c2d8acb35b849 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 17:54:52 -0600 Subject: [PATCH 35/63] update envoy chart to create altas alertmanager config for use by prometheus operator --- charts/envoy/Chart.yaml | 2 +- charts/envoy/templates/atlas-am.yaml | 16 +++++++++++++++- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/charts/envoy/Chart.yaml b/charts/envoy/Chart.yaml index 5b389be..ffc697d 100755 --- a/charts/envoy/Chart.yaml +++ b/charts/envoy/Chart.yaml @@ -9,4 +9,4 @@ keywords: name: envoy sources: - https://github.com/goatlas-io/atlas -version: 1.1.0 +version: 1.2.0 diff --git a/charts/envoy/templates/atlas-am.yaml b/charts/envoy/templates/atlas-am.yaml index 734af92..93f45e3 100644 --- a/charts/envoy/templates/atlas-am.yaml +++ b/charts/envoy/templates/atlas-am.yaml @@ -16,4 +16,18 @@ spec: type: ClusterIP clusterIP: None {{- end }} - +{{- if gt (int .Values.atlas.alertmanagerCount) 0 }} +--- +apiVersion: v1 +kind: Secret +metadata: + name: atlas-alertmanager-configs + namespace: $NAMESPACE +stringData: + config.yaml: | + - static_configs: + - targets: +{{- range $i, $e := until (int .Values.atlas.alertmanagerCount) }} + - alertmanager{{ $i }}.$NAMESPACE.svc.cluster.local:11903 +{{- end }} +{{- end }} From 88e1ec02884ac07729a867e4c23fa3f21ca897f5 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 17:56:08 -0600 Subject: [PATCH 36/63] update .ci script for new file format --- .ci/tag-version | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.ci/tag-version b/.ci/tag-version index 09e5fb8..afbe6c6 100755 --- a/.ci/tag-version +++ b/.ci/tag-version @@ -46,9 +46,9 @@ MSG="version: ${TAG}" echo "$TAG" > VERSION -if [ -f chart/Chart.yaml ]; then - sed -i.bak "s/appVersion: .*/appVersion: ${TAG}/g" chart/Chart.yaml - sed -i.bak "s/tag: .*/tag: ${VTAG}/g" chart/values.yaml +if [ -f charts/atlas/Chart.yaml ]; then + sed -i.bak "s/appVersion: .*/appVersion: ${TAG}/g" charts/atlas/Chart.yaml + sed -i.bak "s/tag: .*/tag: ${VTAG}/g" charts/atlas/values.yaml rm chart/*.bak fi From cb37543e239923ceee33be4f2696fb7ca53d72d8 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 17:57:01 -0600 Subject: [PATCH 37/63] updating image ref on chart --- charts/atlas/Chart.yaml | 2 +- charts/atlas/values.yaml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index a395eae..3a5e8d6 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v2 name: atlas version: 1.0.0 -appVersion: 0.1.0 +appVersion: 0.1.1 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy diff --git a/charts/atlas/values.yaml b/charts/atlas/values.yaml index 26231e5..5027dfa 100644 --- a/charts/atlas/values.yaml +++ b/charts/atlas/values.yaml @@ -1,6 +1,6 @@ image: - repository: ghcr.io/ekristen/atlas - tag: v1.0.0 + repository: ghcr.io/goatlas-io/atlas + tag: 0.1.1 pullPolicy: IfNotPresent pullSecret: "" From b33f05611cc9a2e223647a30a2d06c4b1b79e94a Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 17:57:56 -0600 Subject: [PATCH 38/63] update script to use latest chart changes --- hack/deployment/deploy.sh | 39 +-------------------------------------- 1 file changed, 1 insertion(+), 38 deletions(-) diff --git a/hack/deployment/deploy.sh b/hack/deployment/deploy.sh index f9bf63e..b327424 100644 --- a/hack/deployment/deploy.sh +++ b/hack/deployment/deploy.sh @@ -94,43 +94,6 @@ controller: EOF } -function setup_am { - local name=$1 - - cat > "$name/am.yaml" < Date: Thu, 14 Oct 2021 17:59:52 -0600 Subject: [PATCH 39/63] Update Helm release envoy to v1.2.0 (#13) Co-authored-by: Renovate Bot --- charts/atlas/Chart.lock | 6 +++--- charts/atlas/Chart.yaml | 2 +- pkg/commands/templates/helm-release.tmpl | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/charts/atlas/Chart.lock b/charts/atlas/Chart.lock index d11cb8f..5c6bdb6 100644 --- a/charts/atlas/Chart.lock +++ b/charts/atlas/Chart.lock @@ -1,6 +1,6 @@ dependencies: - name: envoy repository: https://charts.goatlas.io - version: 1.0.0 -digest: sha256:bfcdfc90c3124bee3bb7d6e732bd265e509d0ceecefa7b7cd34e5ead56fee657 -generated: "2021-10-11T17:34:17.140789-06:00" + version: 1.2.0 +digest: sha256:2ae4d6224eba1230766db020bb2706dd47fab37a4a51cbab490c8f645b8319f8 +generated: "2021-10-14T23:59:34.761721263Z" diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index 3a5e8d6..d663efa 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -5,5 +5,5 @@ appVersion: 0.1.1 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy - version: 1.0.0 + version: 1.2.0 repository: https://charts.goatlas.io \ No newline at end of file diff --git a/pkg/commands/templates/helm-release.tmpl b/pkg/commands/templates/helm-release.tmpl index e9a1fb7..175bbbf 100644 --- a/pkg/commands/templates/helm-release.tmpl +++ b/pkg/commands/templates/helm-release.tmpl @@ -19,7 +19,7 @@ spec: chart: spec: chart: envoy - version: 1.0.0 + version: 1.2.0 sourceRef: kind: HelmRepository name: atlas From 1af750e11d968b7c047b81cae417c6a31427cf3b Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 18:00:08 -0600 Subject: [PATCH 40/63] version: 0.2.0 --- VERSION | 2 +- charts/atlas/Chart.yaml | 2 +- charts/atlas/values.yaml | 4 ++-- pkg/common/version.go | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/VERSION b/VERSION index 17e51c3..0ea3a94 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.1.1 +0.2.0 diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index d663efa..9ee0284 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v2 name: atlas version: 1.0.0 -appVersion: 0.1.1 +appVersion: 0.2.0 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy diff --git a/charts/atlas/values.yaml b/charts/atlas/values.yaml index 5027dfa..a34163a 100644 --- a/charts/atlas/values.yaml +++ b/charts/atlas/values.yaml @@ -1,6 +1,6 @@ image: repository: ghcr.io/goatlas-io/atlas - tag: 0.1.1 + tag: v0.2.0 pullPolicy: IfNotPresent pullSecret: "" @@ -56,7 +56,7 @@ envoy: - /config/envoy.yaml image: repository: envoyproxy/envoy - tag: v1.18.3 + tag: v0.2.0 pullPolicy: IfNotPresent service: enabled: true diff --git a/pkg/common/version.go b/pkg/common/version.go index 9db7770..bcab1c6 100644 --- a/pkg/common/version.go +++ b/pkg/common/version.go @@ -10,7 +10,7 @@ var SUMMARY = "0.1.0-dev" var BRANCH = "dev" // VERSION of Release -var VERSION = "0.1.1" +var VERSION = "0.2.0" // AppVersion -- var AppVersion AppVersionInfo From 6bb8e73791beb1c5296feca5b805a5e7b8a46b27 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 18:00:45 -0600 Subject: [PATCH 41/63] fix ci script, ignore .bak files --- .ci/tag-version | 2 +- .gitignore | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.ci/tag-version b/.ci/tag-version index afbe6c6..eb29aac 100755 --- a/.ci/tag-version +++ b/.ci/tag-version @@ -49,7 +49,7 @@ echo "$TAG" > VERSION if [ -f charts/atlas/Chart.yaml ]; then sed -i.bak "s/appVersion: .*/appVersion: ${TAG}/g" charts/atlas/Chart.yaml sed -i.bak "s/tag: .*/tag: ${VTAG}/g" charts/atlas/values.yaml - rm chart/*.bak + rm charts/atlas/*.bak fi if [ -f pkg/common/version.go ]; then diff --git a/.gitignore b/.gitignore index eb9d066..c4a174a 100644 --- a/.gitignore +++ b/.gitignore @@ -13,4 +13,5 @@ test-*.yaml chart/charts/*.tgz droplet.json envoy-values.yaml -helm-values.yaml \ No newline at end of file +helm-values.yaml +*.bak \ No newline at end of file From 98be8b7b8af33887dcfd2052ebd2fd73ed3fee49 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 18:00:49 -0600 Subject: [PATCH 42/63] version: 0.2.1 --- VERSION | 2 +- charts/atlas/Chart.yaml | 2 +- charts/atlas/values.yaml | 4 ++-- pkg/common/version.go | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/VERSION b/VERSION index 0ea3a94..0c62199 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.2.0 +0.2.1 diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index 9ee0284..9e432d5 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v2 name: atlas version: 1.0.0 -appVersion: 0.2.0 +appVersion: 0.2.1 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy diff --git a/charts/atlas/values.yaml b/charts/atlas/values.yaml index a34163a..9b6d027 100644 --- a/charts/atlas/values.yaml +++ b/charts/atlas/values.yaml @@ -1,6 +1,6 @@ image: repository: ghcr.io/goatlas-io/atlas - tag: v0.2.0 + tag: v0.2.1 pullPolicy: IfNotPresent pullSecret: "" @@ -56,7 +56,7 @@ envoy: - /config/envoy.yaml image: repository: envoyproxy/envoy - tag: v0.2.0 + tag: v0.2.1 pullPolicy: IfNotPresent service: enabled: true diff --git a/pkg/common/version.go b/pkg/common/version.go index bcab1c6..5158ef6 100644 --- a/pkg/common/version.go +++ b/pkg/common/version.go @@ -10,7 +10,7 @@ var SUMMARY = "0.1.0-dev" var BRANCH = "dev" // VERSION of Release -var VERSION = "0.2.0" +var VERSION = "0.2.1" // AppVersion -- var AppVersion AppVersionInfo From 0d0f0e2da4082b92f46d1ed66e68b9d28334b5d8 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 18:30:22 -0600 Subject: [PATCH 43/63] fixing envoy version, and rely on template for envoy version --- charts/atlas/values.yaml | 4 ---- pkg/controllers/atlas/templates/envoy-downstream.tmpl | 4 ---- 2 files changed, 8 deletions(-) diff --git a/charts/atlas/values.yaml b/charts/atlas/values.yaml index 9b6d027..0590b80 100644 --- a/charts/atlas/values.yaml +++ b/charts/atlas/values.yaml @@ -54,10 +54,6 @@ envoy: - info - -c - /config/envoy.yaml - image: - repository: envoyproxy/envoy - tag: v0.2.1 - pullPolicy: IfNotPresent service: enabled: true ports: diff --git a/pkg/controllers/atlas/templates/envoy-downstream.tmpl b/pkg/controllers/atlas/templates/envoy-downstream.tmpl index 2219ff3..e606b12 100644 --- a/pkg/controllers/atlas/templates/envoy-downstream.tmpl +++ b/pkg/controllers/atlas/templates/envoy-downstream.tmpl @@ -6,10 +6,6 @@ args: - info - -c - /config/envoy.yaml -image: - repository: envoyproxy/envoy - tag: v1.18.3 - pullPolicy: IfNotPresent service: enabled: true ports: From 7c626209e4f8b414d5d93f827f16baf4cfd8765a Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Thu, 14 Oct 2021 18:31:14 -0600 Subject: [PATCH 44/63] fixing docker image on --- .github/workflows/docker.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index d70daaa..a1de6c4 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -4,10 +4,10 @@ on: workflow_dispatch: push: branches: - - master + - main pull_request: branches: - - master + - main jobs: build: From 19c0d100b6484a72bf77aaa6c561f5e5e866efb0 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 08:28:35 -0600 Subject: [PATCH 45/63] reorganize examples --- docs/quick-start.md | 12 +++++++----- {hack/deployment => examples/demo-do}/README.md | 0 {hack/deployment => examples/demo-do}/deploy.sh | 0 examples/{ => resources}/README.md | 0 examples/{ => resources}/downstream-cluster.yaml | 0 .../{ => resources}/internal-downstream-sidecar.yaml | 0 6 files changed, 7 insertions(+), 5 deletions(-) rename {hack/deployment => examples/demo-do}/README.md (100%) rename {hack/deployment => examples/demo-do}/deploy.sh (100%) rename examples/{ => resources}/README.md (100%) rename examples/{ => resources}/downstream-cluster.yaml (100%) rename examples/{ => resources}/internal-downstream-sidecar.yaml (100%) diff --git a/docs/quick-start.md b/docs/quick-start.md index 4ace35a..7a4873e 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -1,11 +1,11 @@ -# Quick Start +# Quick Start (aka Demo using Digital Ocean) -The easiest way to get started is to take the [deploy script](hack/deployment/deploy.sh) for a spin. It requires a Digital Ocean account. +The easiest way to get started is to take the [deploy script](examples/demo-do/deploy.sh) for a spin. It requires a Digital Ocean account. To use this script you'll need a Digital Ocean API token. Once you have acquired the token, simply export `DIGITALOCEAN_ACCESS_TOKEN` to your shell and then from the root of the Atlas project run the following ... ```bash -bash hack/deployment/deploy.sh up +bash examples/demo-do/deploy.sh up ``` This script will deploy four clusters: @@ -15,6 +15,8 @@ This script will deploy four clusters: - downstream2 - downstream3 -Once the script is done running a set of details will be printed to the screen. If you want to see the details again simply re-run the script with `down` instead of `up`. +Once the script is done running a set of details will be printed to the screen. If you want to see the details again simply re-run the script with `details` instead of `up`. -When you are all done, `bash hack/deployment/depoy.sh down` to tear it all down. +The details output will give you all the urls to the various components that can be interacted with on the observability cluster and the downstream clusters. + +When you are all done, `bash examples/demo-do/depoy.sh down` to tear it all down. diff --git a/hack/deployment/README.md b/examples/demo-do/README.md similarity index 100% rename from hack/deployment/README.md rename to examples/demo-do/README.md diff --git a/hack/deployment/deploy.sh b/examples/demo-do/deploy.sh similarity index 100% rename from hack/deployment/deploy.sh rename to examples/demo-do/deploy.sh diff --git a/examples/README.md b/examples/resources/README.md similarity index 100% rename from examples/README.md rename to examples/resources/README.md diff --git a/examples/downstream-cluster.yaml b/examples/resources/downstream-cluster.yaml similarity index 100% rename from examples/downstream-cluster.yaml rename to examples/resources/downstream-cluster.yaml diff --git a/examples/internal-downstream-sidecar.yaml b/examples/resources/internal-downstream-sidecar.yaml similarity index 100% rename from examples/internal-downstream-sidecar.yaml rename to examples/resources/internal-downstream-sidecar.yaml From 25fd9f66ee0a508818cc43b78029664c3b89aa1f Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 08:28:52 -0600 Subject: [PATCH 46/63] another attempt at building and pushing docker image when not a tag --- .github/workflows/docker.yml | 51 ----------------------------------- .github/workflows/release.yml | 19 ++++++++++++- Dockerfile | 2 +- 3 files changed, 19 insertions(+), 53 deletions(-) delete mode 100644 .github/workflows/docker.yml diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml deleted file mode 100644 index a1de6c4..0000000 --- a/.github/workflows/docker.yml +++ /dev/null @@ -1,51 +0,0 @@ -name: docker - -on: - workflow_dispatch: - push: - branches: - - main - pull_request: - branches: - - main - -jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Get current date - id: date - run: echo "::set-output name=date::$(date +'%s')" - - name: Checkout - uses: actions/checkout@v2 - with: - fetch-depth: 0 - - name: Get branch names - id: branch-name - uses: tj-actions/branch-names@v4.9 - - name: Setup Docker Metadata - id: docker_metadata - uses: crazy-max/ghaction-docker-meta@v3 - with: - images: | - ghcr.io/goatlas-io/atlas - tags: | - type=sha,prefix=${{ steps.branch-name.outputs.current_branch }}-,suffix=-${{ steps.date.outputs.date }} - - name: Setup Docker Buildx - uses: docker/setup-buildx-action@v1 - - name: Login to GitHub Container Registry - uses: docker/login-action@v1 - with: - registry: ghcr.io - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - name: Build to Registry - uses: docker/build-push-action@v2 - with: - context: . - tags: ${{ steps.docker_metadata.outputs.tags }} - labels: ${{ steps.docker_metadata.outputs.labels }} - build-args: | - BRANCH=${{ steps.branch-name.outputs.current_branch }} - cache-to: type=inline - outputs: type=registry diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e4a0f39..dfa4686 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -7,6 +7,9 @@ on: - main tags: - "v*" + pull_request: + branches: + - main jobs: version: @@ -53,7 +56,7 @@ jobs: shell: bash run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})" id: extract_branch - - name: Run GoReleaser for Tags + - name: Run GoReleaser uses: goreleaser/goreleaser-action@v2 if: startsWith(github.ref , 'refs/tags/v') == true with: @@ -63,6 +66,20 @@ jobs: env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} BRANCH: ${{ steps.extract_branch.outputs.branch }} + - name: Run GoReleaser in Snapshot Mode + uses: goreleaser/goreleaser-action@v2 + if: startsWith(github.ref , 'refs/tags/v') == false + with: + distribution: goreleaser + version: latest + args: release --rm-dist --snapshot + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + BRANCH: ${{ steps.extract_branch.outputs.branch }} + - name: Push Snapshot Docker Image + if: startsWith(github.ref , 'refs/tags/v') == false + run: | + docker push $(docker images -a | grep goatlas | awk '{ print $1 ":" $2 }') - name: Artifact Upload uses: actions/upload-artifact@v2 if: startsWith(github.ref , 'refs/tags/v') == false diff --git a/Dockerfile b/Dockerfile index 743caaa..168104e 100644 --- a/Dockerfile +++ b/Dockerfile @@ -16,7 +16,7 @@ RUN curl --fail -sLo /tini https://github.com/krallin/tini/releases/download/${T FROM debian:stretch-slim ENTRYPOINT ["/usr/bin/tini", "--", "/usr/bin/atlas"] -RUN apt-get update && apt-get install -y ca-certificates liblz4-1 && rm -rf /var/lib/apt/lists/* +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* RUN useradd -r -u 999 -d /home/atlas atlas COPY --from=binaries /tini /usr/bin/tini From fc8173f4b3d9a3921d0d14b86834289dde122a21 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 08:33:51 -0600 Subject: [PATCH 47/63] try again on current tag --- .github/workflows/release.yml | 1 + .goreleaser.yml | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index dfa4686..d47cdf2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -76,6 +76,7 @@ jobs: env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} BRANCH: ${{ steps.extract_branch.outputs.branch }} + GORELEASER_CURRENT_TAG: ${{ needs.version.outputs.docker-version }} - name: Push Snapshot Docker Image if: startsWith(github.ref , 'refs/tags/v') == false run: | diff --git a/.goreleaser.yml b/.goreleaser.yml index 6ebfe7e..abe0fb2 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -41,7 +41,7 @@ dockers: goarch: amd64 dockerfile: Dockerfile.gorelease image_templates: - - ghcr.io/goatlas-io/atlas:v{{ .Version }} + - ghcr.io/goatlas-io/atlas:{{ .Version }} build_flag_templates: - "--pull" - "--label=org.opencontainers.image.created={{.Date}}" From c1c60448629709efd9702dad1a68140c05efa4b4 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 08:36:01 -0600 Subject: [PATCH 48/63] fix link in docs --- docs/deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/deployment.md b/docs/deployment.md index 4548f11..d5de0eb 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -18,7 +18,7 @@ Atlas should only be installed to the **observability** cluster. All downstream It is recommended you use the same namespace like `monitoring` for the deployment of Prometheus and Atlas. -How you deploy Prometheus with the Thanos Sidecar is up to you, however I would recommend simply using the [kube-prometheus-stack]() helm chart as it makes this process very simple and takes care of the more complicated bits for you. If you want Thanos persisting to S3 you can pass your S3 credentials along as well. +How you deploy Prometheus with the Thanos Sidecar is up to you, however I would recommend simply using the [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) helm chart as it makes this process very simple and takes care of the more complicated bits for you. If you want Thanos persisting to S3 you can pass your S3 credentials along as well. !!! note When using `kube-prometheus-stack` ensure `servicePerReplica` is enabled for both prometheus and alertmanager sections, this will allow proper routing to each individual instance. From e9fe6880a3e9dc8ed9a564deb6d4bc0b51566313 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 08:40:00 -0600 Subject: [PATCH 49/63] switch to branch-short-timestamp for snapshots --- .goreleaser.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.goreleaser.yml b/.goreleaser.yml index abe0fb2..f5ad96a 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -53,7 +53,7 @@ dockers: checksum: name_template: "checksums.txt" snapshot: - name_template: "{{ .Tag }}" + name_template: "{{ .Env.BRANCH }}-{{ .ShortCommit }}-{{ .Timestamp }}" changelog: sort: asc filters: From cc804e39124b70bb55b012dd18df9a1506b227af Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 09:43:58 -0600 Subject: [PATCH 50/63] fix bad format in helm chart --- charts/envoy/Chart.yaml | 2 +- charts/envoy/README.md | 8 ++++++++ charts/envoy/templates/atlas-am.yaml | 3 +-- 3 files changed, 10 insertions(+), 3 deletions(-) create mode 100644 charts/envoy/README.md diff --git a/charts/envoy/Chart.yaml b/charts/envoy/Chart.yaml index ffc697d..9b1ce14 100755 --- a/charts/envoy/Chart.yaml +++ b/charts/envoy/Chart.yaml @@ -9,4 +9,4 @@ keywords: name: envoy sources: - https://github.com/goatlas-io/atlas -version: 1.2.0 +version: 1.2.1 diff --git a/charts/envoy/README.md b/charts/envoy/README.md new file mode 100644 index 0000000..4c7ed53 --- /dev/null +++ b/charts/envoy/README.md @@ -0,0 +1,8 @@ +# Envoy Chart for Atlas + +This chart was originally forked from the stable/envoy chart, but has been slightly modified to add some specific Atlas related configurations that compliment the Envoy deployment. + +## Modifications + +- Atlas Additional Alertmanager Configuration -- based on the number of alertmanagers deployed in the observability cluster, this is automatically configured. +- Atlas Alertmanager Services -- based on the number of alertmanagers deployed in the observability cluster, this is automatically configured. diff --git a/charts/envoy/templates/atlas-am.yaml b/charts/envoy/templates/atlas-am.yaml index 93f45e3..a6e9ba8 100644 --- a/charts/envoy/templates/atlas-am.yaml +++ b/charts/envoy/templates/atlas-am.yaml @@ -22,12 +22,11 @@ apiVersion: v1 kind: Secret metadata: name: atlas-alertmanager-configs - namespace: $NAMESPACE stringData: config.yaml: | - static_configs: - targets: {{- range $i, $e := until (int .Values.atlas.alertmanagerCount) }} - - alertmanager{{ $i }}.$NAMESPACE.svc.cluster.local:11903 + - alertmanager{{ $i }}.{{ $.Release.Namespace }}.svc.cluster.local:11903 {{- end }} {{- end }} From 8558e34cb77fa3c113d30c1c3672d3abed32c2e4 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 09:44:21 -0600 Subject: [PATCH 51/63] change script to work with public repo! --- .gitignore | 2 ++ docs/quick-start.md | 43 +++++++++++++++++++++++++++++++++++++- examples/demo-do/deploy.sh | 24 --------------------- 3 files changed, 44 insertions(+), 25 deletions(-) diff --git a/.gitignore b/.gitignore index c4a174a..a4cbf87 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,8 @@ vendor hack/deployment/**/*.json hack/deployment/**/*.yaml +examples/demo*/**/*.json +examples/demo*/**/*.yaml charts/**/*.tgz release archive diff --git a/docs/quick-start.md b/docs/quick-start.md index 7a4873e..83478d7 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -4,6 +4,8 @@ The easiest way to get started is to take the [deploy script](examples/demo-do/d To use this script you'll need a Digital Ocean API token. Once you have acquired the token, simply export `DIGITALOCEAN_ACCESS_TOKEN` to your shell and then from the root of the Atlas project run the following ... +## Go Time + ```bash bash examples/demo-do/deploy.sh up ``` @@ -17,6 +19,45 @@ This script will deploy four clusters: Once the script is done running a set of details will be printed to the screen. If you want to see the details again simply re-run the script with `details` instead of `up`. -The details output will give you all the urls to the various components that can be interacted with on the observability cluster and the downstream clusters. +The details output will give you all the urls to the various components that can be interacted with on the observability cluster and the downstream clusters, see below for more details. + +Generally speaking by the time the details page shows up downstream1 and downstream2 will already be connected. Downstream3 will still be in the process of coming online, but should only take another minute or two at most. + +## Details + +In general your details will look something like the following ... + +```text +IP Addresses +----------------------------------------- +Observability: 143.198.182.161 + Downstream1: 198.211.117.92 + Downstream2: 143.244.174.92 + Downstream3: 137.184.97.135 + +Observability Cluster +------------------------------------------ +thanos-query: http://thanos-query.143.198.182.161.nip.io + prometheus: http://prometheus.143.198.182.161.nip.io + alerts: http://alerts.143.198.182.161.nip.io + +Accessing Downstream through Observability Cluster: + +Note: these use the Envoy Proxy Network provided by Atlas to allow secure +communications to downstream cluster components. + + downstream1: http://thanos-query.143.198.182.161.nip.io/prom/downstream1/graph + downstream2: http://thanos-query.143.198.182.161.nip.io/prom/downstream2/graph + downstream3: http://thanos-query.143.198.182.161.nip.io/prom/downstream3/graph + +Important: In a real-world scenario you'd gate access to thanos-query via an oauth2 proxy +or it would only be accessible on an internal network! +``` + +The link to thanos-query in the observability cluster is how you can see your thanos query connected to the sidecars. + +The downstream1-3 links all use the thanos-query and the ingress path prefix that allows accessing of the downstream clusters from the observability cluster. You can confirm this by going to each link and pulling up the prometheus configuration, you'll see the external labels differ for each one. + +## Cleanup When you are all done, `bash examples/demo-do/depoy.sh down` to tear it all down. diff --git a/examples/demo-do/deploy.sh b/examples/demo-do/deploy.sh index b327424..21bb26c 100644 --- a/examples/demo-do/deploy.sh +++ b/examples/demo-do/deploy.sh @@ -11,12 +11,6 @@ DO_SIZE=${DO_SIZE:="s-2vcpu-4gb"} DO_IMAGE=${DO_IMAGE:="ubuntu-20-04-x64"} DIGITALOCEAN_SSH_KEYS=${DIGITALOCEAN_SSH_KEYS:-""} -HELM_TAG=${HELM_TAG:="master-5f87f6a"} -HELM_PULLSECRET=${HELM_PULLSECRET:=""} - -GITHUB_USERNAME=${GITHUB_USERNAME:-""} -GITHUB_TOKEN=${GITHUB_TOKEN:-""} - NAMESPACE=${NAMESPACE:="monitoring"} THANOS_VERSION=${THANOS_VERSION:="v0.23.1"} @@ -78,12 +72,6 @@ function setup_atlas_values { local ip_address=$1 cat > "observability/atlas-values.yaml" < Date: Sat, 16 Oct 2021 09:45:37 -0600 Subject: [PATCH 52/63] Update module github.com/rancher/wrangler to v0.8.7 (#14) Co-authored-by: Renovate Bot --- go.mod | 2 +- go.sum | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/go.mod b/go.mod index a7f2378..3dc7e22 100644 --- a/go.mod +++ b/go.mod @@ -16,7 +16,7 @@ require ( github.com/mitchellh/copystructure v1.2.0 // indirect github.com/mitchellh/hashstructure/v2 v2.0.2 github.com/prometheus/client_golang v1.11.0 - github.com/rancher/wrangler v0.8.6 + github.com/rancher/wrangler v0.8.7 github.com/sirupsen/logrus v1.8.1 github.com/urfave/cli/v2 v2.3.0 google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a // indirect diff --git a/go.sum b/go.sum index a6dbf16..fe4f644 100644 --- a/go.sum +++ b/go.sum @@ -433,6 +433,8 @@ github.com/rancher/lasso v0.0.0-20210616224652-fc3ebd901c08 h1:NxR8Fh0eE7/5/5Zvl github.com/rancher/lasso v0.0.0-20210616224652-fc3ebd901c08/go.mod h1:9qZd/S8DqWzfKtjKGgSoHqGEByYmUE3qRaBaaAHwfEM= github.com/rancher/wrangler v0.8.6 h1:z0PYRySnwEEPtybjSdcUbHlgznIuxQpseVP7OEsKOb0= github.com/rancher/wrangler v0.8.6/go.mod h1:dKEaHNB4izxmPUtpq1Hvr3z3Oh+9k5pCZyFO9sUhlaY= +github.com/rancher/wrangler v0.8.7 h1:WN9EWycceZ9gP5hEqIRJMrwi7cprxETMyKk/qXl+9ZU= +github.com/rancher/wrangler v0.8.7/go.mod h1:dKEaHNB4izxmPUtpq1Hvr3z3Oh+9k5pCZyFO9sUhlaY= github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446/go.mod h1:uYEyJGbgTkfkS4+E/PavXkNJcbFIpEtjt2B0KDQ5+9M= github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= From 828ed24b38f30e3aaaba1994808865f0d249da4e Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 09:48:57 -0600 Subject: [PATCH 53/63] remove bad renovate config entry --- .github/renovate.json | 8 -------- 1 file changed, 8 deletions(-) diff --git a/.github/renovate.json b/.github/renovate.json index 5451f56..c68705c 100644 --- a/.github/renovate.json +++ b/.github/renovate.json @@ -13,14 +13,6 @@ "^k8s.io/" ] }, - { - "matchDatasources": [ - "go" - ], - "matchPackagePatterns": [ - "^github.com/aws/aws-sdk-go-v2/.*" - ] - }, { "groupName": "atlas-envoy", "groupSlug": "atlas-envoy", From 558334c313d5a3674e40b1c4101cc18a04f762a6 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Sat, 16 Oct 2021 10:02:38 -0600 Subject: [PATCH 54/63] Update Helm release envoy to v1.2.1 (#16) Co-authored-by: Renovate Bot --- charts/atlas/Chart.lock | 6 +++--- charts/atlas/Chart.yaml | 2 +- pkg/commands/templates/helm-release.tmpl | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/charts/atlas/Chart.lock b/charts/atlas/Chart.lock index 5c6bdb6..c755e83 100644 --- a/charts/atlas/Chart.lock +++ b/charts/atlas/Chart.lock @@ -1,6 +1,6 @@ dependencies: - name: envoy repository: https://charts.goatlas.io - version: 1.2.0 -digest: sha256:2ae4d6224eba1230766db020bb2706dd47fab37a4a51cbab490c8f645b8319f8 -generated: "2021-10-14T23:59:34.761721263Z" + version: 1.2.1 +digest: sha256:7e0364788c934c016a1884099f6a8c84d111bf70b49be4dd8a877fcbcc42a25d +generated: "2021-10-16T16:01:12.337906289Z" diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index 9e432d5..d2491a9 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -5,5 +5,5 @@ appVersion: 0.2.1 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy - version: 1.2.0 + version: 1.2.1 repository: https://charts.goatlas.io \ No newline at end of file diff --git a/pkg/commands/templates/helm-release.tmpl b/pkg/commands/templates/helm-release.tmpl index 175bbbf..7ea2874 100644 --- a/pkg/commands/templates/helm-release.tmpl +++ b/pkg/commands/templates/helm-release.tmpl @@ -19,7 +19,7 @@ spec: chart: spec: chart: envoy - version: 1.2.0 + version: 1.2.1 sourceRef: kind: HelmRepository name: atlas From 999b8cbbb8d8b47b137092ebb9f9d5c076c27963 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 10:03:19 -0600 Subject: [PATCH 55/63] version: 0.3.0 --- VERSION | 2 +- charts/atlas/Chart.yaml | 2 +- charts/atlas/values.yaml | 2 +- pkg/common/version.go | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/VERSION b/VERSION index 0c62199..0d91a54 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.2.1 +0.3.0 diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index d2491a9..8ca49de 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v2 name: atlas version: 1.0.0 -appVersion: 0.2.1 +appVersion: 0.3.0 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy diff --git a/charts/atlas/values.yaml b/charts/atlas/values.yaml index 0590b80..d32cdc7 100644 --- a/charts/atlas/values.yaml +++ b/charts/atlas/values.yaml @@ -1,6 +1,6 @@ image: repository: ghcr.io/goatlas-io/atlas - tag: v0.2.1 + tag: v0.3.0 pullPolicy: IfNotPresent pullSecret: "" diff --git a/pkg/common/version.go b/pkg/common/version.go index 5158ef6..4bc4b4a 100644 --- a/pkg/common/version.go +++ b/pkg/common/version.go @@ -10,7 +10,7 @@ var SUMMARY = "0.1.0-dev" var BRANCH = "dev" // VERSION of Release -var VERSION = "0.2.1" +var VERSION = "0.3.0" // AppVersion -- var AppVersion AppVersionInfo From 831e659695181a29901d1e306f9d64447e3d844c Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 10:18:12 -0600 Subject: [PATCH 56/63] fixing goreleaser --- .goreleaser.yml | 2 +- Makefile | 5 ++++- go.sum | 2 -- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/.goreleaser.yml b/.goreleaser.yml index f5ad96a..72b2019 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -41,7 +41,7 @@ dockers: goarch: amd64 dockerfile: Dockerfile.gorelease image_templates: - - ghcr.io/goatlas-io/atlas:{{ .Version }} + - ghcr.io/goatlas-io/atlas:{{- if .IsSnapshot -}}{{ .Version }}{{- else -}}{{ .Tag }}{{- end -}} build_flag_templates: - "--pull" - "--label=org.opencontainers.image.created={{.Date}}" diff --git a/Makefile b/Makefile index 51cd645..f305224 100644 --- a/Makefile +++ b/Makefile @@ -4,7 +4,7 @@ VERSION := $(shell cat VERSION) NAME := $(shell basename `pwd`) MODULE := $(shell head -n1 go.mod | cut -f2 -d' ') -.PHONY: docs-build docs-serve +.PHONY: docs-build docs-serve build release snapshot vendor: go mod vendor @@ -18,5 +18,8 @@ docs-serve: build: SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser build +release: + SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser release --skip-publish --rm-dist --skip-validate + snapshot: GORELEASER_CURRENT_TAG=$(SUMMARY) SUMMARY=$(SUMMARY) VERSION=$(VERSION) BRANCH=$(BRANCH) goreleaser release --snapshot --skip-publish --rm-dist \ No newline at end of file diff --git a/go.sum b/go.sum index fe4f644..6080b02 100644 --- a/go.sum +++ b/go.sum @@ -431,8 +431,6 @@ github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40T github.com/qri-io/starlib v0.4.2-0.20200213133954-ff2e8cd5ef8d/go.mod h1:7DPO4domFU579Ga6E61sB9VFNaniPVwJP5C4bBCu3wA= github.com/rancher/lasso v0.0.0-20210616224652-fc3ebd901c08 h1:NxR8Fh0eE7/5/5Zvlog9B5NVjWKqBSb1WYMUF7/IE5c= github.com/rancher/lasso v0.0.0-20210616224652-fc3ebd901c08/go.mod h1:9qZd/S8DqWzfKtjKGgSoHqGEByYmUE3qRaBaaAHwfEM= -github.com/rancher/wrangler v0.8.6 h1:z0PYRySnwEEPtybjSdcUbHlgznIuxQpseVP7OEsKOb0= -github.com/rancher/wrangler v0.8.6/go.mod h1:dKEaHNB4izxmPUtpq1Hvr3z3Oh+9k5pCZyFO9sUhlaY= github.com/rancher/wrangler v0.8.7 h1:WN9EWycceZ9gP5hEqIRJMrwi7cprxETMyKk/qXl+9ZU= github.com/rancher/wrangler v0.8.7/go.mod h1:dKEaHNB4izxmPUtpq1Hvr3z3Oh+9k5pCZyFO9sUhlaY= github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446/go.mod h1:uYEyJGbgTkfkS4+E/PavXkNJcbFIpEtjt2B0KDQ5+9M= From c6dfafaad3d6d076388bb90f79b289c8d42d7bbd Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 10:18:18 -0600 Subject: [PATCH 57/63] version: 0.3.1 --- VERSION | 2 +- charts/atlas/Chart.yaml | 2 +- charts/atlas/values.yaml | 2 +- pkg/common/version.go | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/VERSION b/VERSION index 0d91a54..9e11b32 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.3.0 +0.3.1 diff --git a/charts/atlas/Chart.yaml b/charts/atlas/Chart.yaml index 8ca49de..dd92cac 100644 --- a/charts/atlas/Chart.yaml +++ b/charts/atlas/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v2 name: atlas version: 1.0.0 -appVersion: 0.3.0 +appVersion: 0.3.1 description: Atlas delivers automated secure communications between Prometheus and Thanos Sidecars. dependencies: - name: envoy diff --git a/charts/atlas/values.yaml b/charts/atlas/values.yaml index d32cdc7..a7c733b 100644 --- a/charts/atlas/values.yaml +++ b/charts/atlas/values.yaml @@ -1,6 +1,6 @@ image: repository: ghcr.io/goatlas-io/atlas - tag: v0.3.0 + tag: v0.3.1 pullPolicy: IfNotPresent pullSecret: "" diff --git a/pkg/common/version.go b/pkg/common/version.go index 4bc4b4a..4e9da2b 100644 --- a/pkg/common/version.go +++ b/pkg/common/version.go @@ -10,7 +10,7 @@ var SUMMARY = "0.1.0-dev" var BRANCH = "dev" // VERSION of Release -var VERSION = "0.3.0" +var VERSION = "0.3.1" // AppVersion -- var AppVersion AppVersionInfo From 0e2b4043ab2fafd42205ddab7eb11d838b35b5c8 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 11:09:53 -0600 Subject: [PATCH 58/63] docs: add preflight instructions to quick-start --- docs/quick-start.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/quick-start.md b/docs/quick-start.md index 83478d7..bdf5907 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -4,12 +4,25 @@ The easiest way to get started is to take the [deploy script](examples/demo-do/d To use this script you'll need a Digital Ocean API token. Once you have acquired the token, simply export `DIGITALOCEAN_ACCESS_TOKEN` to your shell and then from the root of the Atlas project run the following ... -## Go Time +## Prelight + +1. Obtain a Digital Ocean Access Token +2. Obtain the SSH Key ID of your SSH key in Digital Ocean +3. `export DIGITALOCEAN_ACCESS_TOKEN=` +4. `export DIGITALOCEAN_SSH_KEYS=` + +!!! note + I'd highly recommend the use of [direnv](https://direnv.net) for managing environment variables throughout directories. + +## Deploy Time ```bash bash examples/demo-do/deploy.sh up ``` +!!! note + This script takes approximately 5-7 minutes to run, depending on how fast Digital Ocean is. It's spinning up a total of 4 servers and installing [k3s](https://k3s.io), then using helm to install the necessary components like prometheus, thanos, envoy and atlas on the various servers. + This script will deploy four clusters: - observability From 05bfd417b4c95c21783b8b19394164c9b5ed68fd Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 11:11:08 -0600 Subject: [PATCH 59/63] docs: fix typo --- docs/quick-start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/quick-start.md b/docs/quick-start.md index bdf5907..5526bcf 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -4,7 +4,7 @@ The easiest way to get started is to take the [deploy script](examples/demo-do/d To use this script you'll need a Digital Ocean API token. Once you have acquired the token, simply export `DIGITALOCEAN_ACCESS_TOKEN` to your shell and then from the root of the Atlas project run the following ... -## Prelight +## Preflight 1. Obtain a Digital Ocean Access Token 2. Obtain the SSH Key ID of your SSH key in Digital Ocean From f92912c802afbe740f8283401ebcf4161e0f22ee Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 11:11:52 -0600 Subject: [PATCH 60/63] docs: tweak verbiage --- docs/quick-start.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/quick-start.md b/docs/quick-start.md index 5526bcf..38278ef 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -14,7 +14,7 @@ To use this script you'll need a Digital Ocean API token. Once you have acquired !!! note I'd highly recommend the use of [direnv](https://direnv.net) for managing environment variables throughout directories. -## Deploy Time +## Deploy ```bash bash examples/demo-do/deploy.sh up @@ -71,6 +71,6 @@ The link to thanos-query in the observability cluster is how you can see your th The downstream1-3 links all use the thanos-query and the ingress path prefix that allows accessing of the downstream clusters from the observability cluster. You can confirm this by going to each link and pulling up the prometheus configuration, you'll see the external labels differ for each one. -## Cleanup +## Teardown When you are all done, `bash examples/demo-do/depoy.sh down` to tear it all down. From b086b7e09cc45847428fc17efef8c42fb4936279 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 16 Oct 2021 11:13:39 -0600 Subject: [PATCH 61/63] docs: improving quick-start --- docs/quick-start.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/quick-start.md b/docs/quick-start.md index 38278ef..c431509 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -1,8 +1,11 @@ # Quick Start (aka Demo using Digital Ocean) -The easiest way to get started is to take the [deploy script](examples/demo-do/deploy.sh) for a spin. It requires a Digital Ocean account. +!!! important + This script requires a Digital Ocean account. -To use this script you'll need a Digital Ocean API token. Once you have acquired the token, simply export `DIGITALOCEAN_ACCESS_TOKEN` to your shell and then from the root of the Atlas project run the following ... +The easiest way to get started is to take the [deploy script](examples/demo-do/deploy.sh) for a spin. + +This script spins up a fully working Atlas deployment with Prometheus and Thanos on several downstream clusters and automatically configures the downstream clusters for secure communication. ## Preflight From 13d210a268eb7c8db00269ac000f9339d4edea63 Mon Sep 17 00:00:00 2001 From: Erik Kristensen Date: Sat, 30 Oct 2021 10:26:19 -0600 Subject: [PATCH 62/63] docs: fix header formatting --- docs/deployment.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index d5de0eb..047e0ec 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -130,7 +130,7 @@ spec: atlas cluster-add --name "downstream1" --replicas 1 --external-ip "1.1.1.1" ``` -### Step 4. Deploy Envoy on Downstream Cluster +## Step 4. Deploy Envoy on Downstream Cluster Atlas generates helm values for the Atlas Envoy Helm Chart for every downstream cluster added. These values come with the necessary seed values to allow initial secure connections to be established. Once comms are established the Envoy Aggreggated Discovery capabilites take over ensuring the downstream envoy instance stays configure properly. @@ -158,11 +158,11 @@ Once you have the values, install helm on your downstream cluster. Make sure you helm install envoy --values downstream1.yaml chart/ ``` -### Step 5. Repeat +## Step 5. Repeat If you have more than one downstream cluster, repeast steps 3 and 4 until you've added all your clusters. -### Step 6. Configure Downstream Prometheus for Observability Alertmanagers +## Step 6. Configure Downstream Prometheus for Observability Alertmanagers To take full advantage of what Atlas offers, you can configure your downstream prometheus instances to talk to the alertmanagers in the observability cluster. From 9f05ee4db982e5cea6faf16fa7e4e276f71d0889 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Sun, 28 May 2023 09:14:22 +0000 Subject: [PATCH 63/63] chore(deps): update golang docker tag to v1.20.4 --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 168104e..9308bac 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,5 +1,5 @@ # Stage 1 - Go, Build the Binary -FROM golang:1.16.6 as go-builder +FROM golang:1.20.4 as go-builder WORKDIR /src/atlas ENV GO111MODULE=on COPY . /src/atlas