Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation request: Getting started on catalog builder #87

Open
aradhakrishnanGFDL opened this issue Nov 20, 2024 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@aradhakrishnanGFDL
Copy link
Collaborator

Documenting what I shared with a colleague. @Ciheim for the doc updates, please refer to these notes as needed to help a new user get started.

Project code 

https://github.com/NOAA-GFDL/CatalogBuilder

Sphinx documentation 

https://noaa-gfdl.github.io/CatalogBuilder/

Quick look at how the catalog generation calls look like: 

1- https://github.com/NOAA-GFDL/CatalogBuilder/blob/main/catalogbuilder/scripts/gen_intake_gfdl_runner_config.py
2- This is a "testing" code used as part of the continuous integration tests. It creates sample data (empty)files. Here is where it's called in the CI pipeline. See example csv generated as one of the artifacts from the CI workflow under workflow-artifacts if the project allows access. 

Youtube training

intake-esm concepts and such https://www.youtube.com/watch?v=H3AwVVBOfRE&t=1s
The catalog builder tool relatedhttps://www.youtube.com/watch?v=84qmRI5BJJw&t=1s

Notebook examples

Preliminar notebooks examples to build catalog from a jupyter notebook, and to use them for analyzinghttps://github.com/aradhakrishnanGFDL/canopy-cats
e.g. https://github.com/aradhakrishnanGFDL/canopy-cats/blob/main/notebooks/cm4_sample.ipynb

Notebooks from Pangeo cmip6 hackathon

https://gallery.pangeo.io/repos/pangeo-gallery/cmip6/the hackathon was held with cloud resources made available, which. no longer exists. 

Meetings:

We have (hybrid) analysis meetings held bi-weekly on Wednesdays at 11AM. The meetings also discuss catalog integration into FRE. 

Getting started:

While the docs show you multiple ways to get started, and the conda package is likely the cleanest..or the fre-cli if you're using it at GFDL, just describing how  I run to quickly get things started in developer and reviewed mode. (fre-cli docs need to be improved, so users to keep an eye out on the sphinx doc updates)

clone the reposet up the conda environment using the environment.yml file 

Follow the notebook example here once you set up your configuration file (example below) and your ipykernel. 

https://github.com/NOAA-GFDL/CatalogBuilder/blob/main/catalogbuilder/scripts/configs/config-example.yml

@aradhakrishnanGFDL aradhakrishnanGFDL added the enhancement New feature or request label Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant