Feature - Make credentials definable per Persistent Volume

[bartdenotte]

/feature

Is your feature request related to a problem? Please describe.
The mountpoint CSI driver handles all mountpoint s3 PV's within a EKS cluster.
When data is segmented due to security concerns using different buckets for multi tenancy or cross account buckets (e.g. BYOB), the current setup requires the CSI driver to be able to access all locations. This requires the IAM policy to global access to all resources and allows the CSI to operate with more access than required (cfr. Least Privilege principle) when handling specific PV requests.

Describe the solution you'd like in detail
Allow credential definition (including temp AWS credentials) per PV or reference to namespace/secret per PV.
This allows to define a IAM policy per PV to minimise the authorisations aligned with the Least privilege principle.
These credentials will be read and propagated to the mount-s3 command.

Describe alternatives you've considered
None

Additional context
I did a small POC where I modified the CSI driver.

• Added additional attributes (awsAccesKeyId, awsSecretAccessKey, awsSessionToken) in spec.csi.volumeAttributes on the PV definition
• Read those additional attributes in pkg/driver/node.go and propagate those the mount function (cfr. d.Mounter.Mount) defined in pkg/driver/mount.go
• When defined use these creds and add those to the env []string. Due to order of credential usage by the AWS client, the creds from the PV take precedence above the IRSA role defined on the csi DaemonSet via the Service account or on the Env vars defined on the CSI DaemonSet

[jjkr]

Thank you for the request. Finer grained access control is something we are looking at, and we have another similar feature request open here: #111. We will share any updates here as we have them.

[psavva]

I would like to make a suggestion...

What if you have multiple daemonsets defined, once per bucket.
ie: instead of hardcoding the secret to aws-secret, use aws-secret- or similar, and deploy multiple daemonsets, one for each bucket.

I think this may work, however, I have not tested it. @jjkr option on this option or workaround ?

[jjkr]

Currently we just use the global secret name to pull the short lived credentials. Code changes would be needed to define a different secret name for each mount. I'm not seeing a potential workaround with multiple daemonsets, but please let us know if you have something viable there.

We hope to implement this feature soon, but don't have specific timelines to share. Please give the issue a thumbs up if it is important to you- we do take that into account for prioritization.

[psavva]

Secrets are bound per namespace. Perhaps a workaround would be one bucket per namespace, not practical, but I think it could work...

[tonedefdev]

This is a much needed feature for centralized eks deployments to mount decentralized s3 buckets as pod volumes. Is there any work being done on this side yet?

[equusit]

This is a much needed feature for centralized eks deployments to mount decentralized s3 buckets as pod volumes. Is there any work being done on this side yet?

The same here - we manage many apps within the same clusters, with many s3 buckets and having to grant permissions to all the buckets to a central role rather than using the applications own app-role is a hard sell

[markleary]

This would be very useful to us for use with on-prem non-aws s3 implementations. @bartdenotte any chance you could make a PR based on the POC you already made?

[muddyfish]

So if I understand the use case here, as compared to pod-level identity in #111, you want to be able to grant volumes permissions rather than pods because it's more granular.

Is it also important that the credentials are defined by cluster administrators as compared to the pod level approach?

Am I missing any other use cases here?

[unexge]

Mountpoint CSI Driver v1.9.0 added support for Pod-level credentials.

Please let us know if there are use-cases that isn't covered with Pod-level credentials and still requires Volume-level credentials.