Skip to content

Commit

Permalink
deployer: add aws cluster.yaml file generation for parity with gcp
Browse files Browse the repository at this point in the history
  • Loading branch information
consideRatio committed Sep 9, 2024
1 parent 3784377 commit 5f36335
Show file tree
Hide file tree
Showing 3 changed files with 53 additions and 24 deletions.
26 changes: 26 additions & 0 deletions config/clusters/templates/aws/cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: {{ cluster_name }}
provider: aws # {{ sign_in_url }}
aws:
key: enc-deployer-credentials.secret.json
clusterType: eks
clusterName: {{ cluster_name }}
region: {{ cluster_region }}
billing:
paid_by_us: {{ paid_by_us }}
support:
helm_chart_values_files:
- support.values.yaml
- enc-support.secret.values.yaml
hubs: []
# Uncomment the lines below once the support infrastructure was deployed and
# you are ready to add the first cluster

# - name: {{ hub_name }}
# # Tip: consider changing this to something more human friendly
# display_name: "{{ cluster_name }} - {{ hub_name }}"
# domain: {{ hub_name }}.{{ cluster_name }}.2i2c.cloud
# helm_chart: {{ hub_type }}
# helm_chart_values_files:
# - common.values.yaml
# - {{ hub_name }}.values.yaml
# - enc-{{ hub_name }}.secret.values.yaml
22 changes: 22 additions & 0 deletions deployer/commands/generate/dedicated_cluster/aws.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

from .common import (
check_before_continuing_with_generate_command,
generate_cluster_config_file,
generate_config_directory,
generate_support_files,
)
Expand Down Expand Up @@ -105,6 +106,10 @@ def aws(
cluster_region: str = typer.Option(
..., prompt="The region where to deploy the cluster"
),
account_id: str = typer.Option(
...,
prompt="The AWS account id or alias. Declare 2i2c for 2i2c's SSO based accounts and paid_by_us=true",
),
force: bool = typer.Option(
False,
"--force",
Expand All @@ -115,6 +120,14 @@ def aws(
Automatically generate the files required to setup a new cluster on AWS if they don't exist.
Use --force to force existing configuration files to be overwritten by this command.
"""

if account_id == "2i2c":
sign_in_url = "https://2i2c.awsapps.com/start#/"
paid_by_us = True
else:
sign_in_url = f"https://{account_id}.signin.aws.amazon.com/console"
paid_by_us = False

# These are the variables needed by the templates used to generate the cluster config file
# and support files
vars = {
Expand All @@ -124,6 +137,8 @@ def aws(
"hub_type": "basehub",
"cluster_name": cluster_name,
"cluster_region": cluster_region,
"sign_in_url": sign_in_url,
"paid_by_us": str(paid_by_us).lower(),
}

if not check_before_continuing_with_generate_command(
Expand All @@ -133,8 +148,15 @@ def aws(

# If we are here, then either no existing infrastructure files for this cluster have been found
# or the `--force` flag was provided and we can override existing files.

# Automatically generate eksctl and terraform files
generate_infra_files(vars)

# Automatically generate the config directory
cluster_config_directory = generate_config_directory(vars)

# Create the cluster config directory and initial `cluster.yaml` file
generate_cluster_config_file(cluster_config_directory, "aws", vars)

# Generate the support files
generate_support_files(cluster_config_directory, vars)
29 changes: 5 additions & 24 deletions docs/hub-deployment-guide/new-cluster/new-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ We automatically generate the files required to setup a new cluster:
- A ssh public key used by `eksctl` to grant access to the private key.
- A `.tfvars` terraform variables file that will setup most of the non EKS infrastructure.
- The cluster config directory in `./config/cluster/<new-cluster>`
- The `cluster.yaml` config file
- The support values file `support.values.yaml`
- The the support credentials encrypted file `enc-support.values.yaml`
````
Expand Down Expand Up @@ -135,12 +136,13 @@ You can generate these with:
:sync: aws-key
```bash
export CLUSTER_NAME=<cluster-name>;
export CLUSTER_NAME=<cluster-name>
export CLUSTER_REGION=<cluster-region-like ca-central-1>
export ACCOUNT_ID=<declare 2i2c for clusters under 2i2c SSO, otherwise an account id or alias>
```
```bash
deployer generate dedicated-cluster aws --cluster-name=$CLUSTER_NAME --cluster-region=$CLUSTER_REGION
deployer generate dedicated-cluster aws --cluster-name=$CLUSTER_NAME --cluster-region=$CLUSTER_REGION --account-id=$ACCOUNT_ID
```
After running this command, you will be asked to provide the type of hub that will be deployed in the cluster, i.e. `basehub` or `daskhub`.
Expand Down Expand Up @@ -547,29 +549,8 @@ Create a `cluster.yaml` file under the `config/cluster/$CLUSTER_NAME>` folder an
````{tab-item} AWS
:sync: aws-key
```yaml
name: <your-cluster-name>
provider: aws # <copy paste link to sign in url here>
aws:
key: enc-deployer-credentials.secret.json
clusterType: eks
clusterName: $CLUSTER_NAME
region: $CLUSTER_REGION
billing:
# For an AWS account explicitly configured to have the cloud bill
# paid directly by the community and not through 2i2c, declare
# paid_by_us to false
paid_by_us: true
support:
helm_chart_values_files:
- support.values.yaml
- enc-support.secret.values.yaml
hubs: []
```
```{note}
The `aws.key` file is defined _relative_ to the location of the `cluster.yaml` file.
```
A `cluster.yaml` file should already have been generated as part of [](new-cluster:generate-cluster-files).
````
````{tab-item} Google Cloud
Expand Down

0 comments on commit 5f36335

Please sign in to comment.