project-factory

Project Factory

This is a working example of how to manage project creation at scale, by wrapping the project module and driving it via external data, either directly provided or parsed via YAML files.

The wrapping layer around the project module is intentionally thin, so that

• all the features of the project module are available
• no "magic" or hidden side effects are implemented in code
• debugging and integration of new features is simple

The code is meant to be executed by a high level service accounts with powerful permissions:

• Shared VPC connection if service project attachment is desired
• project creation on the nodes (folder or org) where projects will be defined

The module also supports optional creation of specific resources that are usually part of the project creation flow:

• service accounts used for VM instances, and associated basic roles
• KMS key encrypt/decrypt permissions for service identities in the project
• membership in VPC SC standard or bridge perimeters

Leveraging data defaults, merges, optionals

In addition to the yaml files describing projects, the project factory accepts three additional sets of inputs:

• the data_defaults variable allows specifying defaults for specific project attributes, which are only used if the attributes are not present in a project yaml
• the data_overrides variable works similarly to defaults, but the values specified here take precedence over those in yaml files
• the data_merges variable allows specifying additional values that are merged to sets of maps present in the yaml file, which are preserved

Some examples on where to use each of the three sets are provided below.

Example

module "project-factory" {
  source = "./fabric/blueprints/factories/project-factory"
  # use a default billing account if none is specified via yaml
  data_defaults = {
    billing_account = "012345-67890A-ABCDEF"
  }
  # make sure the environment label and stackdriver service are always added
  data_merges = {
    labels = {
      environment = "test"
    }
    services = [
      "stackdriver.googleapis.com"
    ]
  }
  # always use this contaxt and prefix, regardless of what is in the yaml file
  data_overrides = {
    contacts = {
      "[email protected]" = ["ALL"]
    }
    prefix = "test-pf"
  }
  # location where the yaml files are read from
  factory_data_path = "data"
}
# tftest modules=7 resources=26 files=prj-app-1,prj-app-2,prj-app-3 inventory=example.yaml

billing_account: 012345-67890A-BCDEF0
labels:
 app: app-1
 team: foo
parent: folders/12345678
service_encryption_key_ids:
 compute:
 - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce
services:
  - container.googleapis.com
  - storage.googleapis.com
service_accounts:
  app-1-be:
    iam_project_roles:
    - roles/logging.logWriter
    - roles/monitoring.metricWriter
  app-1-fe:
    display_name: "Test app 1 frontend."

# tftest-file id=prj-app-1 path=data/prj-app-1.yaml

labels:
 app: app-2
 team: foo
parent: folders/12345678
service_accounts:
  app-2-be: {}
services:
- compute.googleapis.com
- run.googleapis.com
- storage.googleapis.com
shared_vpc_service_config:
  host_project: foo-host

# tftest-file id=prj-app-2 path=data/prj-app-2.yaml

parent: folders/12345678
services:
- run.googleapis.com
- storage.googleapis.com

# tftest-file id=prj-app-3 path=data/prj-app-3.yaml

Variables

name	description	type	required	default
factory_data_path	Path to folder with YAML project description data files.	string	✓
data_defaults	Optional default values used when corresponding project data from files are missing.	object({…})		{}
data_merges	Optional values that will be merged with corresponding data from files. Combines with data_defaults, file data, and data_overrides.	object({…})		{}
data_overrides	Optional values that override corresponding data from files. Takes precedence over file data and data_defaults.	object({…})		{}

Outputs

name	description	sensitive
projects	Project module outputs.
service_accounts	Service account emails.

Tests

These tests validate fixes to the project factory.

module "project-factory" {
  source = "./fabric/blueprints/factories/project-factory"
  data_defaults = {
    billing_account = "012345-67890A-ABCDEF"
  }
  data_merges = {
    labels = {
      owner = "foo"
    }
    services = [
      "compute.googleapis.com"
    ]
  }
  data_overrides = {
    prefix = "foo"
  }
  factory_data_path = "data"
}
# tftest modules=4 resources=14 files=test-0,test-1,test-2

parent: folders/1234567890
services:
  - iam.googleapis.com
  - contactcenteraiplatform.googleapis.com
  - container.googleapis.com
# tftest-file id=test-0 path=data/test-0.yaml

parent: folders/1234567890
services:
  - iam.googleapis.com
  - contactcenteraiplatform.googleapis.com
# tftest-file id=test-1 path=data/test-1.yaml

parent: folders/1234567890
services:
  - iam.googleapis.com
  - storage.googleapis.com
# tftest-file id=test-2 path=data/test-2.yaml