diff --git a/terraform/cloud-functions/distributed/README.md b/terraform/cloud-functions/distributed/README.md index 38866623..b7b0ff7a 100644 --- a/terraform/cloud-functions/distributed/README.md +++ b/terraform/cloud-functions/distributed/README.md @@ -161,33 +161,20 @@ Autoscaler infrastructure, with the exception of Cloud Scheduler, lives. cloudresourcemanager.googleapis.com ``` -6. Create a Google App Engine app, to enable the APIs for Cloud Scheduler and Firestore. +6. The Autoscaler state can be stored in either Firestore or Cloud Spanner. - ```sh - gcloud app create --region="${AUTOSCALER_APP_ENGINE_LOCATION}" - ``` - -7. The Autoscaler state can be stored in either Firestore or Cloud Spanner. - - In case you want to use Firestore, update the database created with the - Google App Engine app to use [Firestore native mode][firestore-native]. - - ```sh - gcloud firestore databases update --type=firestore-native - ``` - - In case you want to use Cloud Spanner, no action is needed at this point. + Firestore database will be created by default, In case you want to use Cloud + Spanner, no action is needed at this point. ### Deploying the Autoscaler -1. Set the project ID, region, zone and App Engine location in the +1. Set the project ID, region and zone in the corresponding Terraform environment variables ```sh export TF_VAR_project_id="${AUTOSCALER_PROJECT_ID}" export TF_VAR_region="${AUTOSCALER_REGION}" export TF_VAR_zone="${AUTOSCALER_ZONE}" - export TF_VAR_location="${AUTOSCALER_APP_ENGINE_LOCATION}" ``` 2. Change directory into the Terraform scaler-project directory and initialize @@ -229,14 +216,12 @@ topic and function in the project where the Spanner instances live. gcloud config set project "${APP_PROJECT_ID}" ``` -4. Choose the [region and zone][region-and-zone] and - [App Engine Location][app-engine-location] where the Application project +4. Choose the [region and zone][region-and-zone] where the Application project will be located. ```sh export APP_REGION=us-central1 export APP_ZONE=us-central1-c - export APP_APP_ENGINE_LOCATION=us-central ``` 5. Use the following command to enable the Cloud APIs: @@ -252,13 +237,6 @@ topic and function in the project where the Spanner instances live. cloudbuild.googleapis.com ``` -6. Create an App to enable Cloud Scheduler, but do not create a Firestore - database: - - ```sh - gcloud app create --region="${APP_APP_ENGINE_LOCATION}" - ``` - ### Deploy the Application infrastructure 1. Set the project ID, region, zone and App Engine location in the @@ -268,7 +246,6 @@ topic and function in the project where the Spanner instances live. export TF_VAR_project_id="${APP_PROJECT_ID}" export TF_VAR_region="${APP_REGION}" export TF_VAR_zone="${APP_ZONE}" - export TF_VAR_location="${APP_APP_ENGINE_LOCATION}" ``` 2. If you want to create a new Spanner instance for testing the Autoscaler, set diff --git a/terraform/cloud-functions/per-project/README.md b/terraform/cloud-functions/per-project/README.md index a411b3ec..be6c74bf 100644 --- a/terraform/cloud-functions/per-project/README.md +++ b/terraform/cloud-functions/per-project/README.md @@ -130,14 +130,12 @@ In this section you prepare your project for deployment. gcloud config set project "${PROJECT_ID}" ``` -4. Choose the [region and zone][region-and-zone] and - [App Engine Location][app-engine-location] where the Autoscaler +4. Choose the [region and zone][region-and-zone] where the Autoscaler infrastructure will be located. ```sh export REGION=us-central1 export ZONE=us-central1-c - export APP_ENGINE_LOCATION=us-central ``` 5. Enable the required Cloud APIs @@ -155,24 +153,6 @@ In this section you prepare your project for deployment. cloudresourcemanager.googleapis.com ``` -6. Create a Google App Engine app, to enable the APIs - for Cloud Scheduler and Firestore - - ```sh - gcloud app create --region="${APP_ENGINE_LOCATION}" - ``` - -7. The Autoscaler state can be stored in either Firestore or Cloud Spanner. - - In case you want to use Firestore, update the database created with the - Google App Engine app to use [Firestore native mode][firestore-native]. - - ```sh - gcloud firestore databases update --type=firestore-native - ``` - - In case you want to use Cloud Spanner, no action is needed at this point. - ## Deploying the Autoscaler 1. Set the project ID, region and zone in the corresponding Terraform @@ -203,10 +183,16 @@ In this section you prepare your project for deployment. For more information on how to make your Spanner instance to be managed by Terraform, see [Importing your Spanner instances](#importing-your-spanner-instances) -3. If you chose to store the state in Firestore, skip this step. If you want - to store the state in Cloud Spanner and you don't have a Spanner - instance yet for that, then set the following variable so that Terraform - creates an instance for you named `autoscale-test-state`: +3. If you chose to store the state in Firestore, set the value of the following + variable to true and continue to step 5. + + ```sh + export TF_VAR_terraform_firestore_create=true + ``` + +4. Alternatively, if you want to store the state in Cloud Spanner and you don't + have a Spanner instance yet for that, then set the following variable so that + Terraform creates an instance for you named `autoscale-test-state`. ```sh export TF_VAR_terraform_spanner_state=true @@ -223,7 +209,7 @@ In this section you prepare your project for deployment. ``` If you already have a Spanner instance where state must be stored, - only set the the name of your instance: + only set the the name of your instance. ```sh export TF_VAR_spanner_state_name= @@ -244,20 +230,20 @@ In this section you prepare your project for deployment. For more information on how to make your existing Spanner instance to be managed by Terraform, see [Importing your Spanner instances](../per-project/README.md#importing-your-spanner-instances) -4. Change directory into the Terraform per-project directory and initialize it. +5. Change directory into the Terraform per-project directory and initialize it. ```sh cd "${AUTOSCALER_DIR}" terraform init ``` -5. Import the existing AppEngine application into Terraform state +6. Import the existing AppEngine application into Terraform state ```sh terraform import module.scheduler.google_app_engine_application.app "${PROJECT_ID}" ``` -6. Create the Autoscaler infrastructure. Answer `yes` when prompted, after +7. Create the Autoscaler infrastructure. Answer `yes` when prompted, after reviewing the resources that Terraform intends to create. ```sh diff --git a/terraform/gke/README.md b/terraform/gke/README.md index ff56e06f..847a98d3 100644 --- a/terraform/gke/README.md +++ b/terraform/gke/README.md @@ -184,9 +184,9 @@ In this section you prepare your project for deployment. export TF_VAR_terraform_spanner_test=true ``` - On the other hand, if you do not want to create a new Spanner instance - because you already have an instance for the Autoscaler to monitor, set the - name name of your instance in the following variable + On the other hand, if you do not want to create a new Spanner instance because + you already have an instance for the Autoscaler to monitor, set the name of your + instance in the following variable ```sh export TF_VAR_spanner_name= @@ -207,38 +207,23 @@ In this section you prepare your project for deployment. ## Using Firestore for Autoscaler state -1. To use Firestore for the Autoscaler state, choose the - [App Engine Location][app-engine-location] where the Autoscaler - infrastructure will be created, for example: +### Setting up Firestore for storing Autoscaler state using terraform - ```sh - export APP_ENGINE_LOCATION=us-central - ``` +1. A Firestore database will be created by default for storing the state of the + Autoscaler in the [Firestore native mode][firestore-native]. -2. Enable the additional APIs: +2. Enable the additional APIs necessary for using Firestore: ```sh - gcloud services enable \ + gcloud services enable \ appengine.googleapis.com \ firestore.googleapis.com ``` -3. Create a Google App Engine app to enable the API for Firestore: - - ```sh - gcloud app create --region="${APP_ENGINE_LOCATION}" - ``` - -4. To store the state of the Autoscaler, update the database created with the - Google App Engine app to use [Firestore native mode][firestore-native]. - - ```sh - gcloud firestore databases update --type=firestore-native - ``` +3. Next, continue to [Deploying the Autoscaler](#deploying-the-autoscaler) - You will also need to make a minor modification to the Autoscaler - configuration. The required steps to do this are later in these - instructions. +4. You will also need to make a minor modification to the Autoscaler + configuration. The required steps to do this are later in these instructions. 5. Next, continue to [Deploying the Autoscaler](#deploying-the-autoscaler) @@ -441,7 +426,7 @@ similar process. the schema of the configuration, see the [Poller configuration] [autoscaler-config-params] section. -9. If you have chosen to use Firestore to hold the Autoscaler state as described +9. If you wish to use Firestore to hold the Autoscaler state as described above, edit the above files, and remove the following lines: ```yaml @@ -501,8 +486,8 @@ following the instructions above. ### If the Poller fails to run successfully -1. If you have chosen to use Firestore for Autoscaler state and you see the - following error in the logs: +1. If you have chosen to use Firestore (default) for storing Autoscaler state + and you see the following error in the logs: ```sh Error: 5 NOT_FOUND: Database not found: projects//instances/autoscale-test/databases/spanner-autoscaler-state diff --git a/terraform/gke/main.tf b/terraform/gke/main.tf index 8a52753e..ab29cc86 100644 --- a/terraform/gke/main.tf +++ b/terraform/gke/main.tf @@ -61,9 +61,10 @@ module "autoscaler-cluster" { module "firestore" { source = "../modules/firestore" - project_id = var.project_id - poller_sa_email = module.autoscaler-base.poller_sa_email - scaler_sa_email = module.autoscaler-base.scaler_sa_email + project_id = var.project_id + terraform_firestore_create = var.terraform_firestore_create + poller_sa_email = module.autoscaler-base.poller_sa_email + scaler_sa_email = module.autoscaler-base.scaler_sa_email } module "spanner" { diff --git a/terraform/gke/variables.tf b/terraform/gke/variables.tf index 8d1a633d..5413a0c5 100644 --- a/terraform/gke/variables.tf +++ b/terraform/gke/variables.tf @@ -71,3 +71,8 @@ variable "terraform_dashboard" { type = bool default = true } + +variable "terraform_firestore_create" { + type = bool + default = false +} \ No newline at end of file diff --git a/terraform/modules/firestore/main.tf b/terraform/modules/firestore/main.tf index 895c597e..c1c15583 100644 --- a/terraform/modules/firestore/main.tf +++ b/terraform/modules/firestore/main.tf @@ -15,8 +15,8 @@ */ /* - * While the Firestore database is created using the gcloud CLI, the - * Terraform-created service account used by the Scaler needs read/write + * The Firestore database is created using this terraform module and the + * terraform-created service account used by the Scaler needs read/write * permissions to the instance in the appropriate project if Spanner * is not being used to hold state. */ @@ -27,3 +27,16 @@ resource "google_project_iam_member" "scaler_sa_firestore" { role = "roles/datastore.user" member = "serviceAccount:${var.scaler_sa_email}" } + +resource "google_project_service" "firestore" { + project = var.project_id + service = "firestore.googleapis.com" +} + +resource "google_firestore_database" "database" { + project = var.project_id + name = "(default)" + location_id = var.location + type = "FIRESTORE_NATIVE" + depends_on = [google_project_service.firestore] +} diff --git a/terraform/modules/firestore/variables.tf b/terraform/modules/firestore/variables.tf index c3cb4c19..eb674d01 100644 --- a/terraform/modules/firestore/variables.tf +++ b/terraform/modules/firestore/variables.tf @@ -18,6 +18,18 @@ variable "project_id" { type = string } +variable "region" { + description = "The region where the infrastructure will be deployed." + type = string + default = "us-central1" +} + +variable "location" { + description = "The location within the region where the infrastructure will be deployed." + type = string + default = "nam5" +} + variable "poller_sa_email" { type = string } @@ -26,7 +38,7 @@ variable "scaler_sa_email" { type = string } -variable "region" { - type = string - default = "us-central1" -} +variable "terraform_firestore_create" { + type = bool + default = false +} \ No newline at end of file