Ready to review

README.md

@@ -0,0 +1,16 @@
+[![Loadgenerator CI/CD](https://github.com/northpowered/k8s-final-cource/actions/workflows/loadgenerator-cicd.yml/badge.svg?branch=main)](https://github.com/northpowered/k8s-final-cource/actions/workflows/loadgenerator-cicd.yml)
+[![MkDocs CD](https://github.com/northpowered/k8s-final-cource/actions/workflows/mkdocs-cd.yml/badge.svg)](https://github.com/northpowered/k8s-final-cource/actions/workflows/mkdocs-cd.yml)
+---
+# K8S graduate work
+
+Yandex cloud deployment
+
+All documentation are placed [here](https://docs.k8s-test.northpowered.space/)
+
+URLs:
+
+ * [Frontend example](https://frontend.k8s-test.northpowered.space/)
+ * [Grafana](https://grafana.k8s-test.northpowered.space)
+
+Grafana credentials:
+> viewer:P@ssw0rd

docs/cicd.md

@@ -0,0 +1,121 @@
+# GitHub CI/CD
+
+CI/CD pipelines were made with GitHub actions
+
+## Loadgenerator manifest
+
+```yaml
+name: Loadgenerator CI/CD
+
+on:
+  push:
+    branches: [ "main" ]
+    tags:
+    - 'v*.*.*'
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+  setup:
+    runs-on: ubuntu-latest
+    outputs:
+      repo: ${{ steps.vars.outputs.repo }}
+      tag: ${{ steps.vars.outputs.tag }}
+    steps:
+      - name: Use repository
+        uses: actions/checkout@v2
+      - name: Build variables
+        id: vars
+        run: |
+          echo "::set-output name=repo::$GITHUB_REPOSITORY"
+          echo "::set-output name=tag::$(git rev-parse --short "$GITHUB_SHA")"
+      - name: Upload repository
+        uses: actions/upload-artifact@v2
+        with:
+          name: repository
+          path: |
+            ${{ github.workspace }}/src
+
+
+  build:
+    needs: setup
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout
+      uses: actions/checkout@v3
+    - name: Docker meta
+      id: meta
+      uses: docker/metadata-action@v4
+      with:
+        images: |
+          ghcr.io/${{ github.repository_owner }}/loadgenerator
+        tags: |
+          type=schedule
+          type=ref,event=branch
+          type=ref,event=pr
+          type=semver,pattern={{version}}
+          type=semver,pattern={{major}}.{{minor}}
+          type=semver,pattern={{major}}
+          type=sha
+    - name: Login to GitHub Container Registry
+      uses: docker/login-action@v1
+      with:
+        registry: ghcr.io
+        username: ${{ github.repository_owner }}
+        password: ${{ secrets.CR_PAT }}
+    - name: Build and push
+      uses: docker/build-push-action@v3
+      with:
+        context: ./src/loadgenerator/
+        push: true
+        tags: ${{ steps.meta.outputs.tags }}
+        labels: ${{ steps.meta.outputs.labels }}
+
+  deployment:
+    needs: [setup, build]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Download repository
+        uses: actions/download-artifact@v2
+        with:
+          name: repository
+      - name: Create kube config
+        run: |
+          mkdir -p $HOME/.kube/
+          echo "${{ secrets.KUBECONFIG }}" | base64 -d > $HOME/.kube/config
+          chmod 600 $HOME/.kube/config
+      - name: Install helm
+        run: |
+          curl -LO https://get.helm.sh/helm-v3.8.0-linux-amd64.tar.gz
+          tar -zxvf helm-v3.8.0-linux-amd64.tar.gz
+          mv linux-amd64/helm /usr/local/bin/helm
+          helm version
+      - name: Lint helm charts
+        run: helm lint ./loadgenerator/helm/
+      - name: Deploy
+        run: |
+          helm upgrade loadgenerator ./loadgenerator/helm/
+```
+
+## MKDocs Github pages deployment
+
+```yaml
+name: MkDocs CD
+on:
+  push:
+    branches:
+      - main
+permissions:
+  contents: write
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.x
+      - run: pip install mkdocs-material 
+      - run: mkdocs gh-deploy --force
+```

docs/index.md

@@ -1,3 +1,27 @@
-# K8S course
+# Getting started
 
 Documentation for graduate work of [Skillbox Kubernetes cource](https://skillbox.ru/course/devops-kubernetes/)
+
+Short description:
+
+ * Cloud provider - Yandex cloud
+
+ * Cloud service - Managed kubernetes
+
+ * Infrastructure was managed by - yactl
+
+ * Ingress - nginx
+
+ * Load balancer - by cloud provider
+
+ * Domain - custom, k8s-test.northpowered.space
+
+URLs:
+
+ * [Frontend service](https://frontend.k8s-test.northpowered.space/)
+ * [Grafana](https://grafana.k8s-test.northpowered.space)
+ * [Documentation](https://docs.k8s-test.northpowered.space/)
+
+Grafana credentials:
+
+> viewer:P@ssw0rd

docs/monitoring/grafana.md

@@ -0,0 +1,19 @@
+# Grafana
+
+## Nodes Info
+
+[Dashboard link](https://grafana.k8s-test.northpowered.space/d/ZDyXJgoVk/nodes-info?orgId=1)
+
+## Ingress dashboard
+
+With GeoMap panel for requests geo location
+
+[Dashboard link](https://grafana.k8s-test.northpowered.space/d/BZ1BLRoVk/ingress?orgId=1&refresh=30s)
+
+## Pods info
+
+[Dashboard link](https://grafana.k8s-test.northpowered.space/d/22EaBgoVz/pods?orgId=1)
+
+## Pods logs
+
+[Dashboard link](https://grafana.k8s-test.northpowered.space/d/MPZzoHTVk/pod-logs?orgId=1)

docs/monitoring/logs.md

@@ -0,0 +1,12 @@
+# Logs
+
+## Ingress logging
+
+Nginx log format. Custom format to get coords for **GeoLite2 Database**
+```
+log-format-upstream: "local_ts=$time_local remote_addr=$remote_addr remote_user=$remote_user request=[$request] status=$status body_bytes=$body_bytes_sent user_agent=[$http_user_agent] request_length=$request_length request_time=$request_time proxy_upstream=$proxy_upstream_name upstream_response_time=$upstream_response_time $req_id geo_country_name=$geoip_country_name geo_country_code=$geoip_country_code geo_city=$geoip_city latitude=$geoip_latitude longitude=$geoip_longitude"
+```
+
+## Logs dashboard
+
+[Dashboard link](https://grafana.k8s-test.northpowered.space/d/MPZzoHTVk/pod-logs?orgId=1)

docs/tasks/autoscaling.md

@@ -0,0 +1,79 @@
+# HPA
+
+Load testing instrument: Yandex load testing
+
+
+Frontend HPA manifest:
+
+```yaml
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata:
+  name: frontend-hpa
+spec:
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: deployment
+    name: frontend
+  minReplicas: 1
+  maxReplicas: 3
+  metrics:
+    - type: Resource
+      resource:
+        name: cpu
+        target:
+          type: Utilization
+          averageUtilization: 50
+```
+
+Yandex load testing config:
+
+```yaml
+uploader:
+  enabled: true
+  package: yandextank.plugins.DataUploader
+  job_name: frontend-k8s-test
+  job_dsc: ''
+  ver: ''
+  api_address: loadtesting.api.cloud.yandex.net:443
+pandora:
+  enabled: true
+  package: yandextank.plugins.Pandora
+  config_content:
+    pools:
+      - id: HTTP
+        gun:
+          type: http
+          target: frontend.k8s-test.northpowered.space:443
+          ssl: true
+        ammo:
+          type: uri
+          uris:
+            - /
+            - /cart
+          headers:
+            - '[Host: frontend.k8s-test.northpowered.space]'
+            - '[Connection: Keep-Alive]'
+        result:
+          type: phout
+          destination: ./phout.log
+        startup:
+          type: once
+          times: 1000
+        rps:
+          - type: line
+            duration: 60s
+            from: 1
+            to: 100
+          - type: const
+            duration: 300s
+            ops: 100
+    log:
+      level: error
+    monitoring:
+      expvar:
+        enabled: true
+        port: 1234
+core: {}
+
+```

docs/tasks/cluster_deployment.md

@@ -0,0 +1,33 @@
+# Cluster deployment
+
+Cluster was deployed on Yandex cloud platform, using [managed k8s](https://cloud.yandex.ru/docs/managed-kubernetes/) service
+
+## Configuration
+
+Master node region configuration:
+ * ru-central1-a
+
+ * ru-central1-b
+
+ * ru-central1-c
+
+
+
+| Key      | Value           |
+|-------------------------|----------------|
+| Kubernetes version      | 1.23           |
+| Cluster CIDR            | 172.16.0.0/16  |
+| Services CIDR           | 192.168.0.0/16 |
+| Nodes mask (CIDR)       | 24             |
+| Nodes per cluster limit | 128            |
+| Pods per node limit     | 110            |
+| Cluster public IPv4     | 51.250.70.4    |
+| Cluster private IPv4    | 10.130.0.9     |
+
+## Artifacts
+
+![get_nodes_pods](../static/step_1/get-no_get-po.png)
+
+![get_ingress](../../static/step_1/get-ingress.png)
+
+![ingress_404](../../static/step_1/ingress-404.png)

docs/tasks/future.md

@@ -0,0 +1,33 @@
+# Future plan
+
+## VPN channel
+
+Using VPN for accessing cluster
+
+Preffered product: wireguard
+
+Priority: high
+
+## IDS and SIEM integration
+
+IDS and SIEM for SecOPS department
+
+Preffered products: SELK stack
+
+Priority: medium
+
+## Load balancing
+
+Migrating to Istio with sidecars for flexible loadbalancing
+
+Preferred products: Istio+Envoy
+
+Priority: medium
+
+## Tracing integration
+
+Opentelemetry collector for applications and service-to-service requests
+
+Preferred products: Tempo, Jaeger, Kali
+
+Priority: low

docs/tasks/logs_configuration.md

@@ -0,0 +1,3 @@
+# Logs
+
+Logs configuration placed [here](../monitoring/logs.md)

docs/tasks/monitoring_configuration.md

@@ -0,0 +1,20 @@
+# Monitoring
+
+## Monitoring stack
+
+Products for cluster and service moinitoring:
+
+ * Prometheus
+ * node_exporter
+ * Alertmanager
+ * Loki
+
+## Artifacts
+
+Grafana credentials placed [here](../index.md)
+
+List of metrics placed [here](../monitoring/metrics.md)
+
+Logs information placed [here](../monitoring/logs.md)
+
+Grafana information placed [here](../monitoring/grafana.md)