Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File missing on S3 #64

Open
ChrisBarker-NOAA opened this issue Oct 19, 2024 · 2 comments
Open

File missing on S3 #64

ChrisBarker-NOAA opened this issue Oct 19, 2024 · 2 comments

Comments

@ChrisBarker-NOAA
Copy link
Collaborator

The CI is failing, 'cause there's a file on the CI that is no longer there:

See #32b70f3

s3://nextgen-dmac-cloud-ingest/nos/wcofs/nos.wcofs.2ds.best.nc.zarr

@jonmjoyce : any ideas? is it gone, or renamed?

@ChrisBarker-NOAA ChrisBarker-NOAA changed the title Files missing on S3 File missing on S3 Oct 19, 2024
@jonmjoyce
Copy link
Collaborator

The bucket s3://nextgen-dmac-cloud-ingest/ should not be used for development. Ideally you should access the experimental bucket hosted by NODD for kerchunk aggregations: https://registry.opendata.aws/noaa-nodd-kerchunk/

For WCOFS, use s3://noaa-nodd-kerchunk-pds/nos/wcofs/wcofs.fields.best.nc.zarr

We recently experienced some data changes that corrupted the ingest that was running for s3://nextgen-dmac-cloud-ingest/. This was an internally used RPS development bucket and we will no longer be supporting it.

@ChrisBarker-NOAA
Copy link
Collaborator Author

thanks!

s3://noaa-nodd-kerchunk-pds/nos/wcofs/wcofs.fields.best.nc.zarr

Is now working. We really shouldn't be doing that in the tests -- except maybe tests that specific to the NODD stuff, but at least I got it running again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants