Terraform Provider for ZenML

This Terraform provider allows you to manage ZenML resources using Infrastructure as Code. It provides the ability to manage:

• ZenML Stacks
• Stack Components
• Service Connectors

Requirements

• Terraform >= 1.0
• Go >= 1.20
• ZenML Server >= 0.70.0

Building The Provider

1. Clone the repository

git clone [email protected]:zenml-io/terraform-provider-zenml.git

2. Build the provider

make build

Installing The Provider

To use the provider in your Terraform configuration:

terraform {
  required_providers {
    zenml = {
      source = "zenml-io/zenml"
    }
  }
}

Using the Provider

Authentication

Service Account API Key

Configure the provider with your ZenML server URL and API key:

provider "zenml" {
  server_url = "https://your-zenml-server.com"
  api_key    = "your-api-key"
}

You can also use environment variables:

export ZENML_SERVER_URL="https://your-zenml-server.com"
export ZENML_API_KEY="your-api-key"

To generate a ZENML_API_KEY, follow these steps:

1. Install ZenML:

pip install zenml

2. Connect to your ZenML server:

zenml connect --url <API_URL>

3. Create a service account and get the API key:

zenml service-account create <MYSERVICEACCOUNTNAME>

This command will print out the ZENML_API_KEY that you can use with this provider.

API Token

Alternatively, you can use an API token for authentication:

provider "zenml" {
  server_url = "https://your-zenml-server.com"
  api_token  = "your-api-token"
}

You can also use environment variables:

export ZENML_SERVER_URL="https://your-zenml-server.com"
export ZENML_API_TOKEN="your-api-token"

Example Usage

Hint: The ZenML Terraform provider is being heavily used in all our Terraform modules. Their code is available on GitHub and can be used as a reference:

• zenml-stack/aws
• zenml-stack/gcp
• zenml-stack/azure

Here's a basic example of creating a stack with components:

# Create a service connector for GCP
resource "zenml_service_connector" "gcp" {
  name        = "gcp-connector"
  type        = "gcp"
  auth_method = "service-account"
  # workspace defaults to "default" if not specified
  
  configuration = {
    project_id = "my-project"
    location   = "us-central1"
    service_account_json = file("service-account.json")
  }
  
  labels = {
    environment = "production"
  }
}

# Create an artifact store component
resource "zenml_stack_component" "artifact_store" {
  name   = "gcs-store"
  type   = "artifact_store"
  flavor = "gcp"
  # workspace defaults to "default" if not specified
  
  configuration = {
    path = "gs://my-bucket/artifacts"
  }
  
  connector_id = zenml_service_connector.gcp.id
  
  labels = {
    environment = "production"
  }
}

# Create a stack using the components
resource "zenml_stack" "ml_stack" {
  name = "production-stack"
  # workspace defaults to "default" if not specified
  
  components = {
    artifact_store = zenml_stack_component.artifact_store.id
  }
  
  labels = {
    environment = "production"
  }
}

Note: All resources support an optional workspace parameter that defaults to "default" if not specified. You can override this by setting workspace = "your-workspace-name" in any resource.

See the examples directory for more complete examples.

Development

Requirements

• Go >= 1.20
• GNU Make
• Terraform >= 1.0

Building

1. Clone the repository

git clone [email protected]:zenml-io/terraform-provider-zenml.git
cd terraform-provider-zenml

2. Build the provider

make build

Testing

Run unit tests:

make test

Run acceptance tests (requires a running ZenML server):

make testacc

Local Development Installation

To install the provider locally for testing:

make install

This will build and install the provider to your local Terraform plugins directory.

Documentation

Generate provider documentation:

make docs

Contributing

See CONTRIBUTING.md for guidelines on contributing to this provider.

Resource Documentation

Stacks

• Stack Resource
• Stack Data Source

Stack Components

• Stack Component Resource
• Stack Component Data Source

Service Connectors

• Service Connector Resource
• Service Connector Data Source

License

Apache License 2.0