From 76a5b1ed3e751350eb62a2912ae6cb21fab8cffb Mon Sep 17 00:00:00 2001 From: Angela Tran Date: Wed, 12 Jun 2024 17:59:10 +0000 Subject: [PATCH 1/6] docs(infra): move Environments section up, move some sections under it no content edit, just moving things around --- docs/deployment/infrastructure.md | 161 +++++++++++++++--------------- 1 file changed, 81 insertions(+), 80 deletions(-) diff --git a/docs/deployment/infrastructure.md b/docs/deployment/infrastructure.md index e16212a24..36025f12b 100644 --- a/docs/deployment/infrastructure.md +++ b/docs/deployment/infrastructure.md @@ -2,9 +2,68 @@ The infrastructure is configured as code via [Terraform](https://www.terraform.io/), for [various reasons](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/the-benefits-of-infrastructure-as-code/ba-p/2069350). -## Architecture +## Environments + +Within the `CDT Digital CA` directory ([how to switch](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure)), there are two [Subscriptions](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions), with Resource Groups under each. Each environment corresponds to a single Resource Group, [Terraform Workspace](https://developer.hashicorp.com/terraform/language/state/workspaces), and branch. + +| Environment | Subscription | Resource Group | Workspace | Branch | +| ----------- | --------------------- | ----------------------------- | --------- | ------ | +| Dev | `CDT/ODI Development` | `RG-CDT-PUB-VIP-CALITP-D-001` | `dev` | `dev` | +| Test | `CDT/ODI Development` | `RG-CDT-PUB-VIP-CALITP-T-001` | `test` | `test` | +| Prod | `CDT/ODI Production` | `RG-CDT-PUB-VIP-CALITP-P-001` | `default` | `prod` | + +All resources in these Resource Groups should be reflected in Terraform in this repository. The exceptions are: + +- Secrets, such as values under [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/). [`prevent_destroy`](https://developer.hashicorp.com/terraform/tutorials/state/resource-lifecycle#prevent-resource-deletion) is used on these Resources. +- [Things managed by DevSecOps](#ownership) + +You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources). + +For browsing the [Azure portal](https://portal.azure.com), you can [switch your `Default subscription filter`](https://docs.microsoft.com/en-us/azure/azure-portal/set-preferences). -### System interconnections +### Ownership + +The following things in Azure are managed by the California Department of Technology (CDT)'s DevSecOps (OET) team: + +- Subcriptions +- [Resource Groups](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) +- Networking +- Front Door + - Web Application Firewall (WAF) + - Distributed denial-of-service (DDoS) protection +- IAM +- Service connections + +### Architecture + +#### Benefits application + +```mermaid +flowchart LR + internet[Public internet] + frontdoor[Front Door] + django[Django application] + interconnections[Other system interconnections] + + internet --> Cloudflare + Cloudflare --> frontdoor + django <--> interconnections + + subgraph Azure + frontdoor --> NGINX + + subgraph App Service + subgraph Custom container + direction TB + NGINX --> django + end + end + end +``` + +[Front Door](https://docs.microsoft.com/en-us/azure/frontdoor/front-door-overview) also includes the [Web Application Firewall (WAF)](https://docs.microsoft.com/en-us/azure/web-application-firewall/afds/afds-overview) and handles TLS termination. Front Door is managed by the DevSecOps team. + +#### System interconnections ```mermaid flowchart LR @@ -42,64 +101,35 @@ flowchart LR idg -->|User attributes| benefits ``` -### Benefits application - -```mermaid -flowchart LR - internet[Public internet] - frontdoor[Front Door] - django[Django application] - interconnections[Other system interconnections] - - internet --> Cloudflare - Cloudflare --> frontdoor - django <--> interconnections +### Naming conventions - subgraph Azure - frontdoor --> NGINX +The DevSecOps team sets the following naming convention for Resources: - subgraph App Service - subgraph Custom container - direction TB - NGINX --> django - end - end - end +``` +<>-<>-<>-<>-<>-<><>-<>-<> ``` -[Front Door](https://docs.microsoft.com/en-us/azure/frontdoor/front-door-overview) also includes the [Web Application Firewall (WAF)](https://docs.microsoft.com/en-us/azure/web-application-firewall/afds/afds-overview) and handles TLS termination. Front Door is managed by the DevSecOps team. - -## Ownership - -The following things in Azure are managed by the California Department of Technology (CDT)'s DevSecOps (OET) team: - -- Subcriptions -- [Resource Groups](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) -- Networking -- Front Door - - Web Application Firewall (WAF) - - Distributed denial-of-service (DDoS) protection -- IAM -- Service connections - -## Environments - -Within the `CDT Digital CA` directory ([how to switch](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure)), there are two [Subscriptions](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions), with Resource Groups under each. Each environment corresponds to a single Resource Group, [Terraform Workspace](https://developer.hashicorp.com/terraform/language/state/workspaces), and branch. +#### Sample Names -| Environment | Subscription | Resource Group | Workspace | Branch | -| ----------- | --------------------- | ----------------------------- | --------- | ------ | -| Dev | `CDT/ODI Development` | `RG-CDT-PUB-VIP-CALITP-D-001` | `dev` | `dev` | -| Test | `CDT/ODI Development` | `RG-CDT-PUB-VIP-CALITP-T-001` | `test` | `test` | -| Prod | `CDT/ODI Production` | `RG-CDT-PUB-VIP-CALITP-P-001` | `default` | `prod` | +- `RG-CDT-PUB-VIP-BNSCN-E-D-001` +- `ASP-CDT-PUB-VIP-BNSCN-EL-P-001` +- `AS-CDT-PUB-VIP-BNSCN-EL-D-001` -All resources in these Resource Groups should be reflected in Terraform in this repository. The exceptions are: +#### Resource Types -- Secrets, such as values under [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/). [`prevent_destroy`](https://developer.hashicorp.com/terraform/tutorials/state/resource-lifecycle#prevent-resource-deletion) is used on these Resources. -- [Things managed by DevSecOps](#ownership) +Use the following shorthand for conveying the Resource Type as part of the Resource Name: -You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources). +| Resource | Convention | +| ---------------- | ---------- | +| App Service | `AS` | +| App Service Plan | `ASP` | +| Virtual Network | `VNET` | +| Resource Group | `RG` | +| Virtual Machine | `VM` | +| Database | `DB` | +| Subnet | `SNET` | +| Front Door | `FD` | -For browsing the [Azure portal](https://portal.azure.com), you can [switch your `Default subscription filter`](https://docs.microsoft.com/en-us/azure/azure-portal/set-preferences). ## Making changes @@ -149,35 +179,6 @@ lifecycle { } ``` -### Naming conventions - -The DevSecOps team sets the following naming convention for Resources: - -``` -<>-<>-<>-<>-<>-<><>-<>-<> -``` - -#### Sample Names - -- `RG-CDT-PUB-VIP-BNSCN-E-D-001` -- `ASP-CDT-PUB-VIP-BNSCN-EL-P-001` -- `AS-CDT-PUB-VIP-BNSCN-EL-D-001` - -#### Resource Types - -Use the following shorthand for conveying the Resource Type as part of the Resource Name: - -| Resource | Convention | -| ---------------- | ---------- | -| App Service | `AS` | -| App Service Plan | `ASP` | -| Virtual Network | `VNET` | -| Resource Group | `RG` | -| Virtual Machine | `VM` | -| Database | `DB` | -| Subnet | `SNET` | -| Front Door | `FD` | - ## Azure environment setup The following steps are required to set up the environment: From 7c863a8264bfce3bcb9a249617b60ffdf3eef2f1 Mon Sep 17 00:00:00 2001 From: Angela Tran Date: Wed, 12 Jun 2024 18:04:14 +0000 Subject: [PATCH 2/6] docs(infra): content edits to sections under Environments --- docs/deployment/infrastructure.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/docs/deployment/infrastructure.md b/docs/deployment/infrastructure.md index 36025f12b..c1c74dd57 100644 --- a/docs/deployment/infrastructure.md +++ b/docs/deployment/infrastructure.md @@ -4,7 +4,9 @@ The infrastructure is configured as code via [Terraform](https://www.terraform.i ## Environments -Within the `CDT Digital CA` directory ([how to switch](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure)), there are two [Subscriptions](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions), with Resource Groups under each. Each environment corresponds to a single Resource Group, [Terraform Workspace](https://developer.hashicorp.com/terraform/language/state/workspaces), and branch. +Within the `CDT Digital CA` directory, there are two [Subscriptions](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions), with Resource Groups under each. (Refer to Azure's documentation for [switching directories](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure).) + +Each of our environments corresponds to a single Resource Group, [Terraform Workspace](https://developer.hashicorp.com/terraform/language/state/workspaces), and branch. | Environment | Subscription | Resource Group | Workspace | Branch | | ----------- | --------------------- | ----------------------------- | --------- | ------ | @@ -17,10 +19,6 @@ All resources in these Resource Groups should be reflected in Terraform in this - Secrets, such as values under [Key Vault](https://azure.microsoft.com/en-us/services/key-vault/). [`prevent_destroy`](https://developer.hashicorp.com/terraform/tutorials/state/resource-lifecycle#prevent-resource-deletion) is used on these Resources. - [Things managed by DevSecOps](#ownership) -You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources). - -For browsing the [Azure portal](https://portal.azure.com), you can [switch your `Default subscription filter`](https://docs.microsoft.com/en-us/azure/azure-portal/set-preferences). - ### Ownership The following things in Azure are managed by the California Department of Technology (CDT)'s DevSecOps (OET) team: @@ -34,8 +32,12 @@ The following things in Azure are managed by the California Department of Techno - IAM - Service connections +You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources), meaning they are managed outside of Terraform. + ### Architecture +These diagrams show a high-level view of the architecture per environment, including some external systems (e.g. analytics, error monitoring, eligibility servers). + #### Benefits application ```mermaid @@ -44,14 +46,11 @@ flowchart LR frontdoor[Front Door] django[Django application] interconnections[Other system interconnections] - internet --> Cloudflare Cloudflare --> frontdoor django <--> interconnections - subgraph Azure frontdoor --> NGINX - subgraph App Service subgraph Custom container direction TB From e0cfacabe76f2df3ec90fbe7da72db4e0445f0a6 Mon Sep 17 00:00:00 2001 From: Angela Tran Date: Wed, 12 Jun 2024 19:02:58 +0000 Subject: [PATCH 3/6] docs(infra): add introductory section --- docs/deployment/infrastructure.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/deployment/infrastructure.md b/docs/deployment/infrastructure.md index c1c74dd57..f16e1468d 100644 --- a/docs/deployment/infrastructure.md +++ b/docs/deployment/infrastructure.md @@ -2,6 +2,12 @@ The infrastructure is configured as code via [Terraform](https://www.terraform.io/), for [various reasons](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/the-benefits-of-infrastructure-as-code/ba-p/2069350). +## Getting started + +Since the Benefits app is deployed into a Microsoft Azure account provided by the California Department of Technology (CDT)'s Office of Enterprise Technology (OET) team, you'll need to request access from them to the `CDT Digital CA` directory so you can get into the [Azure portal](https://portal.azure.com), and to the `California Department of Technology` directory so you can access [Azure DevOps](https://calenterprise.visualstudio.com/CDT.OET.CAL-ITP). + +The Azure portal is where you can view the infrastructure resources for Benefits. Azure DevOps is where our [infrastructure pipeline](https://github.com/cal-itp/benefits/blob/dev/terraform/azure-pipelines.yml) is run to build and deploy those infrastructure resources. + ## Environments Within the `CDT Digital CA` directory, there are two [Subscriptions](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions), with Resource Groups under each. (Refer to Azure's documentation for [switching directories](https://learn.microsoft.com/en-us/azure/devtest/offer/how-to-change-directory-tenants-visual-studio-azure).) From 5234ef861afa6bfe45af06cfa6c3f7d55518619d Mon Sep 17 00:00:00 2001 From: Angela Tran Date: Wed, 12 Jun 2024 19:05:54 +0000 Subject: [PATCH 4/6] docs(infra): content edits to sections about making changes --- docs/deployment/infrastructure.md | 45 +++++++++++++++++++------------ 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/docs/deployment/infrastructure.md b/docs/deployment/infrastructure.md index f16e1468d..fdcea51bc 100644 --- a/docs/deployment/infrastructure.md +++ b/docs/deployment/infrastructure.md @@ -138,16 +138,9 @@ Use the following shorthand for conveying the Resource Type as part of the Resou ## Making changes -[![Build Status](https://calenterprise.visualstudio.com/CDT.OET.CAL-ITP/_apis/build/status/cal-itp.benefits%20Infra?branchName=dev)](https://calenterprise.visualstudio.com/CDT.OET.CAL-ITP/_build/latest?definitionId=828&branchName=dev) - -Terraform is [`plan`](https://www.terraform.io/cli/commands/plan)'d when code is pushed to any branch on GitHub, then [`apply`](https://www.terraform.io/cli/commands/apply)'d when merged to `dev`. While other automation for this project is done through GitHub Actions, we use an Azure Pipeline (above) for a couple of reasons: - -- Easier authentication with the Azure API using a service connnection -- Log output is hidden, avoiding accidentally leaking secrets +### Set up for local development -### Local development - -1. Get access to the Azure account through the DevSecOps team. +1. [Get access to the Azure account through the DevSecOps team.](#getting-started) 1. Install dependencies: - [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) @@ -167,6 +160,11 @@ Terraform is [`plan`](https://www.terraform.io/cli/commands/plan)'d when code is ``` 1. Create a local `terraform.tfvars` file (ignored by git) from the sample; fill in the `*_OBJECT_ID` variables with values from the Azure Pipeline definition. + +### Development process + +When configuration changes to infrastructure resources are needed, they should be made to the resource definitions in Terraform and submitted via pull request. + 1. Make changes to Terraform files. 1. Preview the changes, as necessary. @@ -174,20 +172,33 @@ Terraform is [`plan`](https://www.terraform.io/cli/commands/plan)'d when code is terraform plan ``` -1. [Submit the changes via pull request.](../development/commits-branches-merging/) +1. [Submit the changes via pull request.](../../development/commits-branches-merging) -For Azure resources, you need to [ignore changes](https://www.terraform.io/language/meta-arguments/lifecycle#ignore_changes) to tags, since they are [automatically created by Azure Policy](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/tag-policies). +!!! info "Azure tags" + For Azure resources, you need to [ignore changes](https://www.terraform.io/language/meta-arguments/lifecycle#ignore_changes) to tags, since they are [automatically created by an Azure Policy managed by CDT](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/tag-policies). -```hcl -lifecycle { - ignore_changes = [tags] -} -``` + ```hcl + lifecycle { + ignore_changes = [tags] + } + ``` + +### Infrastructure pipeline + +[![Build Status](https://calenterprise.visualstudio.com/CDT.OET.CAL-ITP/_apis/build/status/cal-itp.benefits%20Infra?branchName=dev)](https://calenterprise.visualstudio.com/CDT.OET.CAL-ITP/_build/latest?definitionId=828&branchName=dev) + +When code is pushed to any branch on GitHub, our infrastructure pipeline in Azure DevOps runs [`terraform plan`](https://www.terraform.io/cli/commands/plan). When the pull request is merged into `dev`, the pipeline runs [`terraform apply`](https://www.terraform.io/cli/commands/apply). + +While other automation for this project is done through GitHub Actions, we use an Azure Pipeline for a couple of reasons: + +- Easier authentication with the Azure API using a service connnection +- Log output is hidden, avoiding accidentally leaking secrets ## Azure environment setup -The following steps are required to set up the environment: +These steps were followed when setting up our Azure deployment for the first time: +- CDT team creates the [resources that they own](#ownership) - `terraform apply` - Set up Slack notifications by [creating a Slack email](https://slack.com/help/articles/206819278-Send-emails-to-Slack) for the [#notify-benefits](https://cal-itp.slack.com/archives/C022HHSEE3F) channel, then [setting it as a Secret in the Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/secrets/quick-create-portal#add-a-secret-to-key-vault) named `slack-benefits-notify-email` - Set required [App Service configuration](../configuration/environment-variables.md) and [configuration](../configuration/data.md) by setting values in Key Vault (the mapping is defined in [app_service.tf](https://github.com/cal-itp/benefits/blob/dev/terraform/app_service.tf)) From d5c8d1c7669ba67964ee09ebc50e3b7a96ad4597 Mon Sep 17 00:00:00 2001 From: Angela Tran Date: Wed, 12 Jun 2024 19:17:45 +0000 Subject: [PATCH 5/6] chore: whitespace from formatter --- docs/deployment/infrastructure.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/deployment/infrastructure.md b/docs/deployment/infrastructure.md index fdcea51bc..1cb0196fd 100644 --- a/docs/deployment/infrastructure.md +++ b/docs/deployment/infrastructure.md @@ -52,11 +52,14 @@ flowchart LR frontdoor[Front Door] django[Django application] interconnections[Other system interconnections] + internet --> Cloudflare Cloudflare --> frontdoor django <--> interconnections + subgraph Azure frontdoor --> NGINX + subgraph App Service subgraph Custom container direction TB @@ -135,7 +138,6 @@ Use the following shorthand for conveying the Resource Type as part of the Resou | Subnet | `SNET` | | Front Door | `FD` | - ## Making changes ### Set up for local development From 232a7f91b44d6f019cb42f8d6c9f9b6ba0dc5ca3 Mon Sep 17 00:00:00 2001 From: Angela Tran Date: Wed, 12 Jun 2024 22:53:51 +0000 Subject: [PATCH 6/6] docs(infra): move some sections up one level --- docs/deployment/infrastructure.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/deployment/infrastructure.md b/docs/deployment/infrastructure.md index 1cb0196fd..e6512a8a0 100644 --- a/docs/deployment/infrastructure.md +++ b/docs/deployment/infrastructure.md @@ -40,11 +40,11 @@ The following things in Azure are managed by the California Department of Techno You'll see these referenced in Terraform as [data sources](https://developer.hashicorp.com/terraform/language/data-sources), meaning they are managed outside of Terraform. -### Architecture +## Architecture These diagrams show a high-level view of the architecture per environment, including some external systems (e.g. analytics, error monitoring, eligibility servers). -#### Benefits application +### Benefits application ```mermaid flowchart LR @@ -71,7 +71,7 @@ flowchart LR [Front Door](https://docs.microsoft.com/en-us/azure/frontdoor/front-door-overview) also includes the [Web Application Firewall (WAF)](https://docs.microsoft.com/en-us/azure/web-application-firewall/afds/afds-overview) and handles TLS termination. Front Door is managed by the DevSecOps team. -#### System interconnections +### System interconnections ```mermaid flowchart LR @@ -109,7 +109,7 @@ flowchart LR idg -->|User attributes| benefits ``` -### Naming conventions +## Naming conventions The DevSecOps team sets the following naming convention for Resources: @@ -117,13 +117,13 @@ The DevSecOps team sets the following naming convention for Resources: <>-<>-<>-<>-<>-<><>-<>-<> ``` -#### Sample Names +### Sample Names - `RG-CDT-PUB-VIP-BNSCN-E-D-001` - `ASP-CDT-PUB-VIP-BNSCN-EL-P-001` - `AS-CDT-PUB-VIP-BNSCN-EL-D-001` -#### Resource Types +### Resource Types Use the following shorthand for conveying the Resource Type as part of the Resource Name: