Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disallow optional values in quantile output_type specifications #102

Closed
nickreich opened this issue Sep 26, 2024 · 0 comments · Fixed by #103
Closed

disallow optional values in quantile output_type specifications #102

nickreich opened this issue Sep 26, 2024 · 0 comments · Fixed by #103
Assignees

Comments

@nickreich
Copy link
Contributor

Per https://github.com/orgs/hubverse-org/discussions/24,
the thinking here is that it does not make sense for a hub to allow submissions where some quantile output_type_id fields are optional.

As noted here, this is done in a few places already, but conceptually this is not that clean.

Observations with allowing optional quantile values:

  • it will make scoring and ensembling not as straight-forward, as it requires subsetting to just models that have a shared set of quantiles. Complicates downstream analyses.
  • several hubs have optional "0" and "1" quantiles (indicating a "min" and "max") of the distribution. Not all distributions will have these (e.g. many distributions are unbounded), so it's not a concept that is general for all probabilistic representations. If we allow 0/1 quantiles (some might argue we should not), then they definitely should be optional, as not all distributions will have finite min/max.
  • There is a "sorting" issue with pmf and cdf category names that is not a problem with quantiles (because they can always be alphanumerically sorted), so having a list broken up into required and optional sections would not hamper sorting.
@annakrystalli annakrystalli self-assigned this Sep 27, 2024
@annakrystalli annakrystalli mentioned this issue Oct 9, 2024
@annakrystalli annakrystalli linked a pull request Oct 9, 2024 that will close this issue
@annakrystalli annakrystalli moved this from In Progress to Ready for Review in hubverse Development overview Oct 9, 2024
@annakrystalli annakrystalli moved this from Ready for Review to In Progress in hubverse Development overview Oct 9, 2024
@annakrystalli annakrystalli moved this from In Progress to Ready for Review in hubverse Development overview Oct 9, 2024
@annakrystalli annakrystalli moved this from Ready for Review to Reviewed/Ready to Merge in hubverse Development overview Oct 16, 2024
@github-project-automation github-project-automation bot moved this from Reviewed/Ready to Merge to Done in hubverse Development overview Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging a pull request may close this issue.

2 participants