Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openscapes: update EKS cluster config templates from k8s 1.21 to 1.24 #2139

Merged
merged 2 commits into from
Feb 6, 2023
Merged

openscapes: update EKS cluster config templates from k8s 1.21 to 1.24 #2139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ba496f1
aws, eksctl template's inline comments refined
consideRatio Feb 5, 2023
bb15478
openscapes: add aws eks addon for ebs driver
consideRatio Feb 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ docs/reference/terraform.md

# Don't check-in YAML files for eksctl config generated from jsonnet files
**.eksctl.yaml
eksctl-config.yaml

# Don't version control any terraform state
**.terraform.lock.hcl
Expand Down
53 changes: 43 additions & 10 deletions eksctl/openscapes.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,18 @@
// Exports an eksctl config file for carbonplan cluster
/*
This file is a jsonnet template of a eksctl's cluster configuration file,
that is used with the eksctl CLI to both update and initialize an AWS EKS
based cluster.

This file has in turn been generated from eksctl/template.jsonnet which is
relevant to compare with for changes over time.

To use jsonnet to generate an eksctl configuration file from this, do:

jsonnet openscapes.jsonnet > eksctl-config.yaml

References:
- https://eksctl.io/usage/schema/
*/
local ng = import "./libsonnet/nodegroup.jsonnet";

// place all cluster nodes here
Expand All @@ -21,27 +35,27 @@ local notebookNodes = [
{ instanceType: "m5.2xlarge" },
{ instanceType: "m5.8xlarge" },
];

// Node definitions for dask worker nodes. Config here is merged
// with our dask worker node definition, which uses spot instances.
// A `node.kubernetes.io/instance-type label is set to the name of the
// *first* item in instanceDistribution.instanceTypes, to match
// what we do with notebook nodes. Pods can request a particular
// kind of node with a nodeSelector
local daskNodes = [
// Node definitions for dask worker nodes. Config here is merged
// with our dask worker node definition, which uses spot instances.
// A `node.kubernetes.io/instance-type label is set to the name of the
// *first* item in instanceDistribution.instanceTypes, to match
// what we do with notebook nodes. Pods can request a particular
// kind of node with a nodeSelector
Comment on lines -24 to +44
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a relocation of these comments, matching how they now generate from the template.jsonnet file as changed in another unrelated PR.

{ instancesDistribution+: { instanceTypes: ["m5.large"] }},
{ instancesDistribution+: { instanceTypes: ["m5.xlarge"] }},
{ instancesDistribution+: { instanceTypes: ["m5.2xlarge"] }},
{ instancesDistribution+: { instanceTypes: ["m5.8xlarge"] }},
];


{
apiVersion: 'eksctl.io/v1alpha5',
kind: 'ClusterConfig',
metadata+: {
name: "openscapeshub",
region: clusterRegion,
version: '1.21'
version: '1.24'
},
availabilityZones: masterAzs,
iam: {
Expand All @@ -57,9 +71,28 @@ local daskNodes = [
],
} for namespace in namespaces],
},
// If you add an addon to this config, run the create addon command.
//
// eksctl create addon --config-file=eksctl-config.yaml
//
addons: [
{
// aws-ebs-csi-driver ensures that our PVCs are bound to PVs that
// couple to AWS EBS based storage, without it expect to see pods
// mounting a PVC failing to schedule and PVC resources that are
// unbound.
//
// Related docs: https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html
//
name: 'aws-ebs-csi-driver',
wellKnownPolicies: {
ebsCSIController: true,
},
},
],
nodeGroups: [n + {clusterName:: $.metadata.name} for n in [
ng {
name: 'core-b',
name: 'core-a',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During this upgrade, I deleted the core-b nodepool and added the core-a nodepool

availabilityZones: [nodeAz],
ssh: {
publicKeyPath: 'ssh-keys/openscapes.key.pub'
Expand Down
41 changes: 30 additions & 11 deletions eksctl/template.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,29 @@
// This file is a jinja2 template of a jsonnet template of a eksctl's cluster
// configuration file, which is in turn can be used with the `eksctl` CLI to both
// update and initialize a AWS EKS based cluster.
//
// This jinja2 template is only used by the deployer script as part of creating
// new clusters. If a relevant change is made here or the dependent file
// libsonnet/nodegroup.jsonnet, one may consider if we should manually update
// already generated jsonnet files in this folder.
//
// Configuration reference: https://eksctl.io/usage/schema/
//
{#-
This file is a jinja2 template of a jsonnet template of a eksctl's cluster
configuration file, which in turn is to be used with the eksctl CLI to both
update and initialize an AWS EKS based cluster.

This jinja2 template is used by the deployer script's generate-aws-cluster
command as part of creating new clusters.

References:
- https://infrastructure.2i2c.org/en/latest/hub-deployment-guide/new-cluster/aws.html#generate-cluster-files
-#}
/*
This file is a jsonnet template of a eksctl's cluster configuration file,
that is used with the eksctl CLI to both update and initialize an AWS EKS
based cluster.

This file has in turn been generated from eksctl/template.jsonnet which is
relevant to compare with for changes over time.

To use jsonnet to generate an eksctl configuration file from this, do:

jsonnet << cluster_name >>.jsonnet > eksctl-config.yaml

References:
- https://eksctl.io/usage/schema/
*/
Comment on lines +12 to +26
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a jsonnet comment to be retained after being generated from the jinja2 template, while the {# parts above are jinja2 comments just for this template.jsonnet file.

local ng = import "./libsonnet/nodegroup.jsonnet";

// place all cluster nodes here
Expand Down Expand Up @@ -56,6 +71,10 @@ local daskNodes = [];
iam: {
withOIDC: true,
},
// If you add an addon to this config, run the create addon command.
//
// eksctl create addon --config-file=eksctl-config.yaml
//
addons: [
{
// aws-ebs-csi-driver ensures that our PVCs are bound to PVs that
Expand Down