Volume separation

[pregnor]

Q	A
Bug fix?	no
New feature?	yes
API breaks?	no
Deprecations?	yes
Related tickets	#3566
License	Apache 2.0

What's in this PR?

Introducing a new volume configuration possibility in create / update node pool, create / update cluster API requests.

 {
      "instanceRoot": {
          "encryption": {
              "enabled": true,
              "encryptionKeyARN": "arn:aws:kms:aws-region:012345678901:key/01234567-0123-0123-0123-012345678901"
          }
          "size": 50
          "storage": "ebs"
          "type": "gp3"
      },
      "kubeletRoot": {
          "storage": "instance-store"
      }
  }

Implementation notes:

• validation of storage types: in case of instanceRoot.storage=[ebs, instance-store], kubeletRoot.storage=[ebs, instance-store, none].
• EBS related fields (size, type, encryption) are ignored in case of storage=instance-store/none
• to remove kubeletRoot volume in an update request set kubeletRoot.storage=none
• EBS volume parameters (size, type, encryption) are only passed to CF template in case storage=ebs
• if a some parameter is not specified in an update request it's values is red from actual CF template param if exists
• EBS size defaults to 50, type = gp3 if not specified
• kubeletRoot EBS data volume is tagged same way as instanceRoot EBS
• in case of instanceRoot=instance-store, a compatible image should be specified which supports booting from instance storage, which is available only for C3, D2, G2, I2, M3, and R3 instance types. See more details here
• it's not possible to choose instanceRoot=instance-store and kubeletRoot.storage=ebs because EBS block device mappings not supported for instance-store AMIs.

Why?

Pipeline should support a more fine grained volume configuration for EKS node pools, where users are able to specify volume storage type and ebs volume related parameters separately for instance root and kubelet root volume.

Test cases

• create nodePool:
  - set instanceRoot=ebs / instance-store, kubeletRoot=ebs/instance-store/none
  - use deprecated volume properties
• update nodePool:
  - set instanceRoot=ebs / instance-store, kubeletRoot=ebs/instance-store/none
  - set only image property
  - use deprecated volume properties
• update a nodePool created with earlier template version (<2.5.0):
  - set new image and / or set kubeletRoot=ebs
• create cluster:
  - set instanceRoot=ebs / instance-store, kubeletRoot=ebs/instance-store/none
  - use deprecated volume properties
• update cluster add / update nodePool(s):
  - set instanceRoot=ebs / instance-store, kubeletRoot=ebs/instance-store/none
  - use deprecated volume properties

Checklist

• Implementation tested (with at least one cloud provider)
• Code meets the Developer Guide
• OpenAPI and Postman files updated (if needed)
• User guide and development docs updated (if needed)
• Related Helm chart(s) updated (if needed)