Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fsspec for opening files on s3 etc #282

Open
Cadair opened this issue Dec 5, 2024 · 2 comments
Open

Use fsspec for opening files on s3 etc #282

Cadair opened this issue Dec 5, 2024 · 2 comments

Comments

@Cadair
Copy link
Contributor

Cadair commented Dec 5, 2024

Rather than hard coding s3 support it would be more flexible and easier to write code to use fsspec to transparently handle files on disk or remote.

@Cadair
Copy link
Contributor Author

Cadair commented Dec 5, 2024

I just had more of a poke about, refactoring to use fsspec would allow you to remove a significant amount of code you have for handling s3 files, while also adding support for the very large list of other providers with no added effort.

I am happy to take a first pass at this if you have the time to review it and integrate it?

@bryan-harter
Copy link
Member

Interesting! I can certainly take a look at it. I should have more time this year to look at cdflib

@bryan-harter bryan-harter moved this from To triage to Backlog in cdflib development Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog
Development

No branches or pull requests

2 participants