Doc Contributions and Docker Workflows

.cloudbees/workflows/bp-agent-ecr.yaml

@@ -8,7 +8,7 @@ name: agent
 on:
   push:
     branches:
-      - '**'
+      - 'main'
 
 env:
   REGISTRY: public.ecr.aws/r1n1q0e5

.cloudbees/workflows/bp-tf-ci.yaml

@@ -15,7 +15,7 @@ env:
   BUCKET_NAME_TF_STATE: "cbci-eks-addon-bp-tf-state"
   AWS_ROLE_TO_ASSUME: "infra-admin-ci"
   CLEAN_UP: "true"
-  REPLAY_STR: "0"
+  REPLAY_STR: "1"
   TF_AUTO_VARS_FILE: |
     tags = {
       "cb-owner" : "professional-services"
@@ -25,7 +25,7 @@ env:
     trial_license = {
       first_name  = "CloudBees.io"
       last_name  = "Platform"
-      email = "platform.ci@cloudbees.com"
+      email = "ci.user@cloudbees.io"
       company = "CloudBees Inc."
     }
     suffix = "ci-v3"

.cloudbees/workflows/ldap-gh.yaml

@@ -8,7 +8,7 @@ name: agent
 on:
   push:
     branches:
-      - '**'
+      - 'main'
 
 env:
   REGISTRY: ghcr.io

CONTRIBUTING.md

@@ -4,7 +4,7 @@ This document provides guidelines for contributing to the CloudBees CI add-on fo
 
 ## Design principles
 
-It follows the same approach as the [Terraform AWS EKS Blueprints for Terraform Patterns](https://aws-ia.github.io/terraform-aws-eks-blueprints/)
+It follows the same approach as the [Terraform AWS EKS Blueprints for Terraform Patterns](https://aws-ia.github.io/terraform-aws-eks-blueprints/).
 
 ## Report bugs and feature requests
 
@@ -37,13 +37,26 @@ To submit a pull request:
 3. **Ensure that local tests pass**.
 4. Make commits to your fork using clear commit messages.
 5. Submit a pull request against the `develop` branch and answer any default questions in the pull request interface.
+
+> [!IMPORTANT]
+> You must validate that the CasC bundle source control management (SCM) tool is pointing to the `develop` branch.
+
 6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
 
 >[!TIP]
 > GitHub provides additional documentation on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
 [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
 
-## CI pipeline
+### Pre-commits: Linting, formatting and secrets scanning
+
+Many of the files in the repository can be linted or formatted to maintain a standard of quality. Additionally, secret leaks are watched via [gitleaks](https://github.com/zricethezav/gitleaks#pre-commit) and [git-secrets](https://github.com/awslabs/git-secrets).
+
+When working with the repository for the first time, you must run `pre-commit`:
+
+1. Run `pre-commit install`.
+2. Run `pre-commit run --all-files`.
+
+## Blueprint Terraform CI pipeline
 
 Validate your pull request changes inside the blueprint agent described in the [Dockerfile](.docker/agent). It is the same agent used for the CI pipeline [bp-agent-ecr.yaml](.cloudbees/workflows/bp-agent-ecr.yaml).
 
@@ -59,19 +72,19 @@ The [bp-tf-ci.yaml](.cloudbees/workflows/bp-tf-ci.yaml) blueprints are orchestra
 - AWS Route 53 zone name, to create DNS records.
 
 > [!IMPORTANT]
-> CloudBees Platform currently only supports push events. Therefore, pull requests are sent to the `dev` branch for integration.
+> CloudBees Platform currently only supports push events. Therefore, pull requests are sent to the `develop` branch for integration.
 
-## Pre-commits: Linting, formatting and secrets scanning
+## Release
 
-Many of the files in the repository can be lined or formatted to maintain a standard of quality. Additionally, secret leaks are watched via [gitleaks](https://github.com/zricethezav/gitleaks#pre-commit) and [git-secrets](https://github.com/awslabs/git-secrets).
+Prerequisites:
 
-When working with the repository for the first time, you must run `pre-commit`:
+1. Project administrators must create a pull request from the `develop` branch to the `main` branch.
 
-1. Run `pre-commit install`.
-2. Run `pre-commit run --all-files`.
+> [!IMPORTANT]
+> You must validate that the CasC bundle SCM tool is pointing to the `main` branch.
 
-## Release
+2. Once the pull request is merged, verify that the `main` branch successfully passes the [Terraform CI build](#blueprint-terraform-ci-pipeline).
 
 This project uses [Release Drafter](https://github.com/release-drafter/release-drafter); pull request labels should be set accordingly.
+
 Kubernetes' environment versions are managed centrally in the [blueprints/.k8.env](blueprints/.k8s.env) file.
-For a new release, the latest commits in the `main` branch should successfully pass the CI build.

Makefile

@@ -37,7 +37,7 @@ bpAgent-dRun: ## Build (if not locally present) and Run the Blueprint Agent usin
 deploy: ## Deploy Terraform Blueprint passed as parameter. Example: ROOT=02-at-scale make deploy
 deploy: tfChecks agentCheck
 	terraform -chdir=$(MKFILEDIR)/blueprints/$(ROOT) init
-	terraform -chdir=$(MKFILEDIR)/blueprints/$(ROOT) plan -no-color >> $(MKFILEDIR)/blueprints/$(ROOT)/tfplan.txt
+	terraform -chdir=$(MKFILEDIR)/blueprints/$(ROOT) plan -no-color > $(MKFILEDIR)/blueprints/$(ROOT)/tfplan.txt
 ifeq ($(CI),false)
 	@$(call helpers,ask-confirmation "Deploy $(ROOT). Check plan at blueprints/$(ROOT)/tfplan.txt")
 endif
@@ -92,7 +92,7 @@ set-kube-env: agentCheck
 	@$(call helpers,INFO "Setting Kube environment finished succesfully.")
 
 .PHONY: set-casc-branch
-set-casc-branch: ## Update Casc bundle repository to the branch passed as parameter. Example: ROOT=main make set-casc-branch
+set-casc-branch: ## Update Casc bundle repository to the branch passed as parameter. Example: BRANCH=main make set-casc-branch
 set-casc-branch: agentCheck guard-BRANCH
 	@$(call helpers,set-casc-branch $(BRANCH))
 	@$(call helpers,INFO "Setting Casc Branch finished succesfully.")

README.md

@@ -44,17 +44,14 @@ module "eks_blueprints_addon_cbci" {
 
 By default, it uses a minimum required configuration described in the Helm chart [values.yaml](values.yml) file. If you need to override any default settings with the chart, you can do so by passing the `helm_config` variable.
 
-> [!TIP]
-> The blueprints lifecycle (`deploy` > `validate` > `destroy`) can be orchestrated via the companion [Makefile](Makefile).
-
 ## Prerequisites
 
 ### Tools
 
 The blueprint `deploy` and `destroy` phases use the same requirements provided in the [AWS EKS Blueprints for Terraform - Prerequisites](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#prerequisites). However, the blueprint `validate` phase may require additional tooling, such as `jq` and `velero`.
 
 > [!NOTE]
-> There is a companion [Dockerfile](.docker) to run the blueprints in a containerized development environment, ensuring all dependencies are met. It can be built using the [Makefile](Makefile) target `make dRun`.
+> There is a companion [Dockerfile](.docker/agent/agent.rootless.Dockerfile) to run the blueprints in a containerized development environment, ensuring all dependencies are met. It can be built using the [Makefile](Makefile) target `make bpAgent-dRun`.
 
 ### AWS authentication
 
@@ -97,7 +94,7 @@ The CloudBees CI add-on uses `helms release` for its resources definition, makin
 | create_k8s_secrets | Create the Kubernetes secret cbci-secrets. It can be consumed by CasC. | `bool` | `false` | no |
 | helm_config | CloudBees CI Helm chart configuration. | `any` | <pre>{<br>  "values": [<br>    ""<br>  ]<br>}</pre> | no |
 | k8s_secrets_file | Secrets file .yml path containing the secrets names:values for cbci-secrets. | `string` | `"secrets-values.yml"` | no |
-| prometheus_target | Create resources to be uses CloudBees CI as Prometheus Target. | `bool` | `false` | no |
+| prometheus_target | Create Service Monitor to discover CloudBees CI Apps Prometheus Target dinamically. It is designed to be enabled with AWS EKS Terraform Addon Kube Prometheus Stack. | `bool` | `false` | no |
 
 ### Outputs
 
@@ -110,6 +107,7 @@ The CloudBees CI add-on uses `helms release` for its resources definition, makin
 | cbci_oc_ing | Operations center Ingress for the CloudBees CI add-on. |
 | cbci_oc_pod | Operations center pod for the CloudBees CI add-on. |
 | cbci_oc_url | Operations center URL for the CloudBees CI add-on using a subdomain and certificates. |
+| cbci_secrets | Kubernetes secrets name for CloudBees CI. Optional. |
 | merged_helm_config | (merged) Helm configuration for CloudBees CI. |
 <!-- END_TF_DOCS -->
 

blueprints/01-getting-started/README.md

@@ -63,6 +63,9 @@ When preparing to deploy, you must complete the following steps:
 
 For more information, refer to [The Core Terraform Workflow](https://www.terraform.io/intro/core-workflow) documentation.
 
+> [!TIP]
+> The `deploy` phase can be orchestrated via the companion [Makefile](../../Makefile).
+
 ## Validate
 
 Once the blueprint has been deployed, you can validate it.
@@ -126,3 +129,6 @@ Once you can access the Kubernetes API from your terminal, complete the followin
 ## Destroy
 
 To tear down and remove the resources created in the blueprint, complete the steps for [Amazon EKS Blueprints for Terraform - Destroy](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#destroy).
+
+> [!TIP]
+> The `destroy` phase can be orchestrated via the companion [Makefile](../../Makefile).

blueprints/01-getting-started/main.tf

@@ -238,10 +238,9 @@ resource "null_resource" "create_kubeconfig" {
 
   depends_on = [module.eks]
 
-  # Remove comment block to force the update of the kubeconfig file
-  # triggers = {
-  #   always_run = timestamp()
-  # }
+  triggers = {
+    always_run = timestamp()
+  }
 
   provisioner "local-exec" {
     command = "aws eks update-kubeconfig --name ${module.eks.cluster_name} --region ${local.region} --kubeconfig ${local.kubeconfig_file_path}"

blueprints/02-at-scale/README.md

@@ -57,16 +57,16 @@ Once you have familiarized yourself with [CloudBees CI blueprint add-on: Get sta
 | acm_certificate_arn | AWS Certificate Manager (ACM) certificate for Amazon Resource Names (ARN). |
 | aws_backup_efs_protected_resource | AWS description for the Amazon EFS drive used to back up protected resources. |
 | aws_logstreams_fluentbit | AWS CloudWatch log streams from Fluent Bit. |
-| cbci_agents_pods | Retrieves a list of agent pods running in the cbci-agents namespace. |
+| cbci_agents_pods | Retrieves a list of agent pods running in the agents namespace. |
 | cbci_controller_b_hibernation_post_queue_ws_cache | team-b hibernation monitor endpoint to the build workspace cache. It expects CBCI_ADMIN_TOKEN as the environment variable. |
 | cbci_controller_c_hpa | team-c horizontal pod autoscaling. |
 | cbci_controllers_pods | Operations center pod for the CloudBees CI add-on. |
 | cbci_helm | Helm configuration for the CloudBees CI add-on. It is accessible only via state files. |
 | cbci_liveness_probe_ext | Operations center service external liveness probe for the CloudBees CI add-on. |
 | cbci_liveness_probe_int | Operations center service internal liveness probe for the CloudBees CI add-on. |
 | cbci_namespace | Namespace for the CloudBees CI add-on. |
-| cbci_oc_export_admin_api_token | Exports the operations center admin_cbci_a API token to access the REST API when CSRF is enabled. It expects CBCI_ADMIN_CRUMB as the environment variable. |
-| cbci_oc_export_admin_crumb | Exports the operations center admin_cbci_a crumb, to access the REST API when CSRF is enabled. |
+| cbci_oc_export_admin_api_token | Exports the operations center cbci_admin_user API token to access the REST API when CSRF is enabled. It expects CBCI_ADMIN_CRUMB as the environment variable. |
+| cbci_oc_export_admin_crumb | Exports the operations center cbci_admin_user crumb, to access the REST API when CSRF is enabled. |
 | cbci_oc_ing | Operations center Ingress for the CloudBees CI add-on. |
 | cbci_oc_pod | Operations center pod for the CloudBees CI add-on. |
 | cbci_oc_take_backups | Operations center cluster operations build for the on-demand back up. It expects CBCI_ADMIN_TOKEN as the environment variable. |
@@ -77,7 +77,7 @@ Once you have familiarized yourself with [CloudBees CI blueprint add-on: Get sta
 | grafana_dashboard | Provides access to Grafana dashboards. |
 | kubeconfig_add | Add kubeconfig to the local configuration to access the Kubernetes API. |
 | kubeconfig_export | Export the KUBECONFIG environment variable to access the Kubernetes API. |
-| ldap_admin_password | LDAP password for admin_cbci_a user for the CloudBees CI add-on. Check .docker/ldap/data.ldif. |
+| ldap_admin_password | LDAP password for cbci_admin_user user for the CloudBees CI add-on. Check .docker/ldap/data.ldif. |
 | prometheus_active_targets | Checks active Prometheus targets from the operations center. |
 | prometheus_dashboard | Provides access to Prometheus dashboards. |
 | s3_cbci_arn | CloudBees CI Amazon S3 bucket ARN. |
@@ -90,18 +90,18 @@ Once you have familiarized yourself with [CloudBees CI blueprint add-on: Get sta
 
 ## Deploy
 
-When preparing to deploy, you must [customize the secrets file](#customize-secrets-file) and [update Amazon S3 bucket settings](#update-amazon-s3-bucket-settings).
+In addition to the minimum required settings explained in [Get started - Deploy](../01-getting-started/README.md#deploy), when preparing to deploy, you must [create the secrets file](#create-the-secrets-file) and [update Amazon S3 bucket settings](#update-amazon-s3-bucket-settings)
 
 > [!TIP]
-> To understand the minimum required settings, refer to [Get started - Deploy](../01-getting-started/README.md#deploy).
+> The `deploy` phase can be orchestrated via the companion [Makefile](../../Makefile).
 
-### Secrets file
+### Create the secrets file
 
 You must create your secrets file by copying the contents of [secrets-values.yml.example](k8s/secrets-values.yml.example) to `secrets-values.yml`. This provides [Kubernetes secrets](https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/docs/features/secrets.adoc#kubernetes-secrets) that can be consumed by CasC.
 
 ### Update Amazon S3 bucket settings
 
-Since the Terraform variable `suffix` is used for this blueprint, you must update the Amazon S3 bucket name for CloudBees CI controllers and the Amazon S3 bucket for the backup controller cluster operations. To update the Amazon S3 bucket name, you have the following options:
+Since the optional Terraform variable `suffix` is used for this blueprint, you must update the Amazon S3 bucket name for CloudBees CI controllers and the Amazon S3 bucket for the backup controller cluster operations. To update the Amazon S3 bucket name, you have the following options:
 
  - [Option 1: Update Amazon S3 bucket name using CasC](#option-1-update-amazon-s3-bucket-name-using-casc)
  - [Option 2: Update Amazon S3 bucket name using the CloudBees CI UI](#option-2-update-amazon-s3-bucket-name-using-the-cloudbees-ci-ui)
@@ -266,7 +266,7 @@ The [CloudBees Prometheus Metrics plugin](https://docs.cloudbees.com/docs/cloudb
 1. Issue the following command to verify that the CloudBees CI targets are connected to Prometheus:
 
    ```sh
-   eval $(terraform output --raw prometheus_active_targets) | jq '.data.activeTargets[] | select(.labels.container=="jenkins" or .labels.job=="cjoc") | {job: .labels.job, instance: .labels.instance, status: .health}'
+   eval $(terraform output --raw prometheus_active_targets) | jq '.data.activeTargets[] | select(.labels.container=="jenkins") | {job: .labels.job, instance: .labels.instance, status: .health}'
    ```
 
 2. Issue the following command to access Kube Prometheus Stack dashboards from your web browser and verify that [Jenkins metrics](https://plugins.jenkins.io/metrics/) are available.
@@ -306,14 +306,5 @@ For CloudBees CI build logs:
 
 To tear down and remove the resources created in the blueprint, refer to [Amazon EKS Blueprints for Terraform - Destroy](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/#destroy).
 
-## Additional resources
-
-The following videos provide more insights regarding the capabilities presented in this blueprint:
-
-[![Getting Started with CloudBees CI High Availability](https://img.youtube.com/vi/Qkf9HaA2wio/0.jpg)](https://www.youtube.com/watch?v=Qkf9HaA2wio)
-
-[![Troubleshooting Pipelines With CloudBees Pipeline Explorer](https://img.youtube.com/vi/OMXm6eYd1EQ/0.jpg)](https://www.youtube.com/watch?v=OMXm6eYd1EQ)
-
-[![Getting Started with CloudBees Workspace Caching](https://img.youtube.com/vi/ESU9oN9JUCw/0.jpg)](https://www.youtube.com/watch?v=ESU9oN9JUCw)
-
-[![How to Monitor Jenkins With Grafana and Prometheus](https://img.youtube.com/vi/3H9eNIf9KZs/0.jpg)](https://www.youtube.com/watch?v=3H9eNIf9KZs)
+> [!TIP]
+> The `destroy` phase can be orchestrated via the companion [Makefile](../../Makefile).

blueprints/02-at-scale/casc/oc/items/items-root.yaml

@@ -37,10 +37,10 @@ items:
           kind: "Service"
           metadata:
             labels:
-              "cloudbees.prometheus" : "true"
+              "cloudbees.prometheus": "true"
     properties:
       - configurationAsCode:
-          bundle: "main/none-ha"
+          bundle: "develop/none-ha"
   # Casc, HA
   - kind: managedController
     name: team-c-ha
@@ -71,7 +71,7 @@ items:
           kind: "Service"
           metadata:
             labels:
-              "cloudbees.prometheus" : "true"
+              "cloudbees.prometheus": "true"
     properties:
       - configurationAsCode:
-          bundle: "main/ha"
+          bundle: "develop/ha"

blueprints/02-at-scale/casc/oc/jcasc/main.yaml

@@ -8,15 +8,15 @@ unclassified:
       name: "casc-mm-store"
       retriever:
         SCM:
-          defaultVersion: "main"
+          defaultVersion: ${casc_branch}
           scmSource:
             git:
               credentialsId: "GH-token"
               remote: ${scm_casc_mm_store}
               traits:
               - "gitBranchDiscovery"
               - headWildcardFilter:
-                  includes: "develop"
+                  includes: ${casc_branch}
               - sparseCheckoutPaths:
                   extension:
                     sparseCheckoutPaths:

blueprints/02-at-scale/casc/oc/variables/variables.yaml

@@ -1,6 +1,6 @@
 variables:
   - message: "Welcome to the CloudBees CI blueprint add-on: At scale!"
   - scm_casc_mm_store: "https://github.com/cloudbees/terraform-aws-cloudbees-ci-eks-addon.git"
+  - casc_branch: develop
   #Issue #70
   #- cbci_s3: "cbci-bp02-s3"
-  #- casc_branch: "develop"

blueprints/02-at-scale/k8s/cbci-values.yml

@@ -8,27 +8,34 @@ OperationsCenter:
     Annotations:
       alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true
   NodeSelector:
-    ci_type: cb-apps
+    ${cbciAppsSelector}
   Tolerations:
-    - key: dedicated
+    - key: ${cbciAppsTolerationKey}
       operator: Equal
-      value: cb-apps
+      value: ${cbciAppsTolerationValue}
       effect: NoSchedule
   CasC:
     Enabled: true
     Retriever:
       Enabled: true
       scmRepo: https://github.com/cloudbees/terraform-aws-cloudbees-ci-eks-addon.git
-      scmBranch: main
+      scmBranch: develop
       scmBundlePath: blueprints/02-at-scale/casc/oc
       scmPollingInterval: PT20M
 Persistence:
   StorageClass: efs
 Hibernation:
   Enabled: true
+  NodeSelector:
+    ${cbciAppsSelector}
+  Tolerations:
+    - key: ${cbciAppsTolerationKey}
+      operator: Equal
+      value: ${cbciAppsTolerationValue}
+      effect: NoSchedule
 # BEE-47031
 # Agents:
 #   SeparateNamespace:
 #     Enabled: true
 #     Create: true
-#     Name: cbci-agents
+#     Name: ${cbciAgentsNamespace}