Discover EFS fileSystemId / volumeHandle instead of specifying it

[nitrocode]

Is your feature request related to a problem? Please describe.
I create EFS volumes in terraform, grab the ID, and then add it to the values file to use the EFS

Describe the solution you'd like in detail
I'd like to add a set of unique tags on my EFS and then specify a way of discovering the EFS ID without having to hard code the EFS ID

# specs/pv.yaml using static
-    volumeHandle: fs-582a03f3
+    volumeHandleDiscovery:
+      tags:
+        service: titan
+        env: dev

# specs/sc.yaml using dynamic
-  fileSystemId: fs-92107410
+  fileSystemDiscovery:
+    tags:
+      service: titan
+      env: dev

Describe alternatives you've considered

1. terraform

Create the entire k8s release in terraform so the EFS is created and passed in to a helm_resource so we do not have to hard code this value.

2. initContainer ?

As a workaround, is it possible to use some kind of initContainer using the aws-cli container to retrieve the efs ID on-the-fly, save it as an env var, and make it accessible to the pv/pvc/sc ?

Additional context

• There is SSL discovery in the aws-load-balancer-controlelr so why not EFS discovery in the aws-efs-csi-driver
• https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html

[RyanStan]

/kind feature

Thank you @nitrocode for the feature request. This is an interesting idea, but since tags aren't guaranteed to be unique, we're concerned that the wrong filesystem could be mounted which would be a security concern.

[nitrocode]

Hi Ryan! Thanks for commenting. You are correct, the tags would need to be unique.

In terraform, if you use a singular data source with tags and it matches more than one EFS, it throws an error. That's something that could be implemented here too.

Example of aws_efs_file_system data source

# this fails if an efs doesn't contain both tags
# this fails if multiple matching efs are discovered
# this only succeeds if a single matching efs is discovered
data "aws_efs_file_system" "selected" {
  tags = {
    env     = "dev"
    service = "titan"
  }
}

In my opinion, if the user chooses to discover the EFS volume (which should be optional) then it should be on the user to assign the EFS a high cardinality attribute (such as a tag or multiple tags ANDed together) to correctly identify a single EFS volume, right?

So if this feature was implemented similar to terraform and no EFS volume was discovered or multiple were discovered, then I would imagine the driver would throw an error. I'd imagine the driver would succeed only if a single EFS was discovered.

[RyanStan]

Interesting, I didn't realize Terraform supported that. I can definitely see this being useful. Let's use this issue to track whether others would like this feature as well.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[nitrocode]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[nitrocode]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[nitrocode]

/remove-lifecycle stale

[avanish23]

Hi @nitrocode this would be really interesting. There was another issue #1453 where the ask was to be able to create the EFS when the FileSystemId was not present in the SC definition file; the problem here being the discovery of the EFS after creation. But if we are able to do this I think what we can also achieve is to create a new EFS with these tags the if EFS is not already present and also be able to discover it later on.
Any thoughts on the same?

[avanish23]

/assign

[nitrocode]

Discovering and reusing a pre-existing efs is what we'd prefer since we have data on the existing efs. We hard code the efs id at the moment which works but is cumbersome if we needed to rotate it.

While creating a new efs, if one isnt provided, would also provide a working efs for a pod that needed it, it wouldn't address this particular issue since it wouldn't have all the same data that is on the other efs.

Edit: oh youre saying if it doesn't discover it, it should create a new efs matching those tags? I don't know about that because the intention would be to reuse an efs, and if that ws my misconfigured, I personally wouldn't want it to create an efs unless I configured it to do so. Perhaps you could introduce a new key such as create_if_not_discovered which could default to false if unspecified.