GCP AutoMQ BYOC Environment Terraform module

This module is designed for deploying the AutoMQ BYOC (Bring Your Own Cloud) environment using the GCP Provider within a GCP cloud environment.

Upon completion of the installation, the module will output the endpoint of the AutoMQ BYOC environment along with the initial username and password. Users can manage the resources within the environment through the following two methods:

• Using the Web UI to manage resources: This method allows users to manage instances, topics, ACLs, and other resources through a web-ui.
• Using Terraform to manage resources: This method requires users to access the AutoMQ BYOC environment via a web browser for the first time to create a Service Account. Subsequently, users can manage resources within the environment using the Service Account's Access Key and the AutoMQ Terraform Provider.

For managing instances, topics, and other resources within the AutoMQ BYOC environment using the AutoMQ Terraform Provider, please refer to the documentation.

Module Usage

Use this module to install the AutoMQ BYOC environment, supporting two modes:

• Create a new VPC: Recommended only for POC or other testing scenarios. In this mode, the user only needs to specify the region, and resources including VPC, Endpoint, Security Group, GCS Bucket, etc., will be created. After testing, all resources can be destroyed with one click.
• Using an existing VPC: Recommended for production environments. In this mode, the user needs to provide a VPC, subnet, and GCS Bucket that meet the requirements. AutoMQ will deploy the BYOC environment console to the user-specified subnet.

Quick Start

1. Install Terraform

   Ensure Terraform is installed on your system. You can download it from the Terraform website.

2. Configure GCP Credentials

   Make sure your GCP CLI is configured with the necessary credentials. You can configure it using the following command:

   gcloud auth application-default login

3. Create Terraform Configuration File

   Create a file named main.tf in your working directory and add the following content:

Create a new VPC

module "automq-byoc" {
  source = "AutoMQ/automq-byoc-environment/google"

  # Set the identifier for the environment to be installed.
  automq_byoc_env_id                       = "example" 

  cloud_provider_region                    = "asia-southeast1"  
  cloud_provider_zone                      = "asia-southeast1-a"
  cloud_project_id                         = "xxxxxxxx"
}

# Necessary outputs
output "automq_byoc_env_id" {
  value = module.automq-byoc.automq_byoc_env_id
}

output "automq_byoc_endpoint" {
  value = module.automq-byoc.automq_byoc_endpoint
}

output "automq_byoc_initial_username" {
  value = module.automq-byoc.automq_byoc_initial_username
}

output "automq_byoc_initial_password" {
  value = module.automq-byoc.automq_byoc_initial_password
}

output "automq_byoc_vpc_id" {
  value = module.automq-byoc.automq_byoc_vpc_id
}

output "automq_byoc_instance_id" {
  value = module.automq-byoc.automq_byoc_instance_id
}

output "automq_byoc_google_service_account" {
  value = module.automq-byoc.automq_byoc_google_service_account
}

Using an existing VPC

To install the AutoMQ BYOC environment using an existing VPC, ensure your existing VPC meets the necessary requirements. You can find the detailed requirements in the Prepare VPC Documents.

module "automq-byoc" {
  source = "AutoMQ/automq-byoc-environment/google"

  # Set the identifier for the environment to be installed.
  automq_byoc_env_id                       = "example" 

  # Set the target regionId of gcp
  cloud_provider_region                    = "asia-southeast1"  
  cloud_provider_zone                      = "asia-southeast1-a"
  cloud_project_id                         = "xxxxx"

  create_new_vpc                           = false   
  existing_vpc_name                        = "xxxxx-network"
  existing_subnet_name                     = "xxxxx-subnetwork"

  automq_byoc_data_bucket_name             = "bucker-data-xxxx"
  automq_byoc_ops_bucket_name              = "bucker-ops-xxxx"
  automq_byoc_machine_type                 = "e2-standard-2"
  automq_byoc_default_deploy_type          = "k8s"
}

# Necessary outputs
output "automq_byoc_env_id" {
  value = module.automq-byoc.automq_byoc_env_id
}

output "automq_byoc_endpoint" {
  value = module.automq-byoc.automq_byoc_endpoint
}

output "automq_byoc_initial_username" {
  value = module.automq-byoc.automq_byoc_initial_username
}

output "automq_byoc_initial_password" {
  value = module.automq-byoc.automq_byoc_initial_password
}

output "automq_byoc_vpc_id" {
  value = module.automq-byoc.automq_byoc_vpc_id
}

output "automq_byoc_instance_id" {
  value = module.automq-byoc.automq_byoc_instance_id
}

output "automq_byoc_google_service_account" {
  value = module.automq-byoc.automq_byoc_google_service_account
}

4. Initialize Terraform

   Run the following command to initialize Terraform:

   terraform init

5. Apply Terraform Configuration

   Run the following command to apply the Terraform configuration and create the resources:

   terraform apply

   Confirm the action by typing yes when prompted.

6. Retrieve Outputs

   After the deployment is complete, run the following command to retrieve the outputs:

   terraform output

   This will display the AutoMQ environment console endpoint, initial username, and initial password.

7. Access AutoMQ Environment Console

   Use the automq_byoc_endpoint, automq_byoc_initial_username, and automq_byoc_initial_password to access the AutoMQ environment console via a web browser.

8. Manage Resources

   You can manage resources within the AutoMQ BYOC environment using the Web UI or Terraform. For more details, refer to the documentation.

9. Clean Up Resources

   If you no longer need the resources, you can destroy them by running:

   terraform destroy

   Confirm the action by typing yes when prompted.

Helpful Links/Information

• Report Bugs

• AutoMQ Cloud Documents

• Request Features

Requirements

Name	Version
terraform	>= 1.0
google	>= 5, < 7

Providers

Name	Version
google	6.11.0

Modules

No modules.

Resources

Name	Type
google_compute_address.web_ip	resource
google_compute_attached_disk.data_volume_attachment	resource
google_compute_disk.data_volume	resource
google_compute_firewall.allow_googleapis_ipv4	resource
google_compute_firewall.automq_byoc_console_sg	resource
google_compute_instance.automq_byoc_console	resource
google_compute_network.automq_network	resource
google_compute_route.route_ipv4_googleapi	resource
google_compute_route.route_ipv4_googleapi_additional	resource
google_compute_subnetwork.automq_subnetwork	resource
google_compute_subnetwork.gke_subnetwork	resource
google_dns_managed_zone.private_dns_zone	resource
google_dns_managed_zone.private_googleapis	resource
google_dns_record_set.private_googleapis_ipv4	resource
google_dns_record_set.wildcard_googleapis_cname	resource
google_project_iam_binding.automq_byoc_compute_sa_binding	resource
google_project_iam_binding.automq_byoc_dns_sa_binding	resource
google_project_iam_binding.automq_byoc_gke_sa_binding	resource
google_project_iam_binding.automq_byoc_resource_sa_binding	resource
google_project_iam_binding.gke_permission_binding0	resource
google_project_iam_binding.gke_permission_binding1	resource
google_project_iam_binding.gke_permission_binding2	resource
google_project_iam_binding.gke_permission_binding3	resource
google_project_iam_binding.gke_permission_binding4	resource
google_project_iam_binding.gke_permission_binding5	resource
google_project_iam_binding.gke_permission_binding6	resource
google_project_iam_custom_role.automq_byoc_compute_role	resource
google_project_iam_custom_role.automq_byoc_dns_role	resource
google_project_iam_custom_role.automq_byoc_gke_role	resource
google_project_iam_custom_role.automq_byoc_resource_role	resource
google_project_iam_custom_role.automq_byoc_storage_role	resource
google_service_account.automq_byoc_sa	resource
google_storage_bucket.automq_byoc_data_bucket	resource
google_storage_bucket.automq_byoc_ops_bucket	resource
google_storage_bucket_iam_binding.automq_data_storage_permission_binding	resource
google_storage_bucket_iam_binding.automq_ops_storage_permission_binding	resource
google_tags_location_tag_binding.compute_instance_env_tag_binding	resource
google_tags_location_tag_binding.compute_instance_vendor_tag_binding	resource
google_tags_tag_key.automqEnvKey	resource
google_tags_tag_key.automqVendorKey	resource
google_tags_tag_value.automqEnvValue	resource
google_tags_tag_value.automqVendorValue	resource
google_compute_image.console_image	data source
google_compute_network.vpc	data source
google_project.project	data source
google_storage_bucket.data_bucket	data source
google_storage_bucket.ops_bucket	data source

Inputs

Name	Description	Type	Default	Required
cloud_project_id	The Google Cloud Project ID where resources will be created	string	n/a	yes
automq_byoc_env_id	The unique identifier of the AutoMQ environment. This parameter is used to create resources within the environment. Additionally, all cloud resource names will incorporate this parameter as part of their names. This parameter supports only numbers, uppercase and lowercase English letters, and hyphens. It must start with a letter and is limited to a length of 32 characters.	string	n/a	yes
cloud_provider_region	Set the Google Cloud region. AutoMQ will deploy to this region.	string	n/a	yes
cloud_provider_zone	Set the Google Cloud zone. AutoMQ will deploy to this zone.	string	n/a	yes
create_new_vpc	This setting determines whether to create a new VPC. If set to true, a new VPC will be automatically created, which is recommended only for POC scenarios. For production scenarios using AutoMQ, you should provide the VPC where the current Kafka application resides.	bool	true	no
existing_vpc_name	When the create_new_vpc parameter is set to false, specify an existing VPC name where AutoMQ will be deployed.	string	""	no
existing_subnet_name	When the create_new_vpc parameter is set to false, specify an existing subnet name for deploying the AutoMQ BYOC environment console.	string	""	no
automq_byoc_env_console_cidr	Set CIDR block to restrict the source IP address range for accessing the AutoMQ environment console. If not set, the default is 0.0.0.0/0.	string	"0.0.0.0/0"	no
automq_byoc_data_bucket_name	Set the existed GCS bucket used to store message data generated by applications. If this parameter is not set, a new GCS bucket will be automatically created. The message data Bucket must be separate from the Ops Bucket.	string	""	no
automq_byoc_ops_bucket_name	Set the existed GCS bucket used to store AutoMQ system logs and metrics data for system monitoring and alerts. If this parameter is not set, a new GCS bucket will be automatically created. This Bucket does not contain any application business data.	string	""	no
automq_byoc_machine_type	Set the Compute Engine machine type; this parameter is used only for deploying the AutoMQ environment console. You need to provide a machine type with at least 2 cores and 8 GB of memory.	string	"e2-standard-2"	no
automq_byoc_env_version	Set the version for the AutoMQ BYOC environment console. It is recommended to keep the default value, which is the latest version.	string	"1.4.0"	no
use_custom_image	The parameter defaults to false, which means a specific custom image is not specified. If you wish to use a custom image, set this parameter to true and specify the automq_byoc_env_console_image parameter.	bool	false	no
automq_byoc_env_console_image	When the use_custom_image parameter is set to true, this parameter must be set with a custom image name to deploy the AutoMQ console.	string	""	no
automq_byoc_default_deploy_type	Set the default deployment type for the AutoMQ BYOC environment. Currently, only 'k8s' is supported.	string	"k8s"	no

Outputs

Name	Description
automq_byoc_env_id	This parameter is used to create resources within the environment. Additionally, all cloud resource names will incorporate this parameter as part of their names. This parameter supports only numbers, uppercase and lowercase English letters, and hyphens. It must start with a letter and is limited to a length of 32 characters.
automq_byoc_endpoint	The endpoint for the AutoMQ environment console. Users can set this endpoint to the AutoMQ Terraform Provider to manage resources through Terraform. Additionally, users can access this endpoint via web browser, log in, and manage resources within the environment using the WebUI.
automq_byoc_initial_username	The initial username for the AutoMQ environment console. It has the EnvironmentAdmin role permissions. This account is used to log in to the environment, create ServiceAccounts, and manage other resources. For detailed information about environment members, please refer to the documentation.
automq_byoc_initial_password	The initial password for the AutoMQ environment console. This account is used to log in to the environment, create ServiceAccounts, and manage other resources. For detailed information about environment members, please refer to the documentation.
automq_byoc_vpc_id	The VPC ID for the AutoMQ environment deployment.
automq_byoc_instance_id	The EC2 instance id for AutoMQ Console.
automq_byoc_google_service_account	The Google Service Account for the AutoMQ environment deployment.