Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-208: new configuration format #215

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

gh-208: new configuration format #215

wants to merge 3 commits into from

Conversation

ntessore
Copy link
Contributor

Implements a new "configuration" format using YAML, which is really more of a serialisation of the data that Heracles needs to run.

Quoting from the provided example file:

# Although we call this `configuration', it could be more correctly
# characterised as `serialization' of the data that Heracles needs to run, in
# the form of a YAML file.  The entire configuration is free-form and returned
# as a namespace by `heracles.load_config()`.  However, certain sections such
# as `catalogs` and `fields` are understood and used by the Heracles CLI.

The motivation behind this design is that we have struggled for a while to condense Heracles' structure into a true "descriptive" configuration format. This solution here goes fully the other way, providing a 1:1 serialisation of the data that would be used in a notebook or script. In other words, this configuration format is basically equivalent to Python code snippets with the same content.

The result has advantages and disadvantages, which we are surely going to discover going forward. One advantage is that it becomes very easy and flexible to load configuration from a file and use it in notebooks or scripts:

# load configuration
config = heracles.load_config("experimenting.yaml")

# use catalogues and fields as configured
catalogs = config.catalogs
fields = config.fields

@ntessore ntessore linked an issue Nov 24, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Configuration format
1 participant