Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Support a validation comparison wrapper to compare different predictions #30

Closed
4 tasks done
Martin-Jung opened this issue May 30, 2022 · 1 comment
Closed
4 tasks done
Assignees
Labels
enhancement New feature or request low priority Quality of life Quality of life improvement

Comments

@Martin-Jung
Copy link
Collaborator

Martin-Jung commented May 30, 2022

Currently validate can be run on individual raster and fitted model objects, but there is no way to compare two different spatial predictions in terms of their metrics. The idea is thus to add a handy function called compare() that acts as a wrapper on validation() outputs (or directly on model objects?).
Steps for implementation:

  • Ensure that each validate() output has an attribute that identifies the tibble as such.

  • Implement a C3 call for compare() on >= 2 outputs of these. The compare function then calculates the differential between the best and the next models and sorts the output (default parameter for sorting provided).

  • When called on a list the function should also work.

  • (Optional): Evaluate whether it is possible to run compare() directly on DistributionModels instead of validation outputs. This requires that each model is of the same family and/or can return some sort of parsimony criterion (AIC, BIC, WAIC, LOO, etc...). This likely requires considerably more work...

@Martin-Jung Martin-Jung added enhancement New feature or request Quality of life Quality of life improvement low priority labels May 30, 2022
@Martin-Jung Martin-Jung self-assigned this May 30, 2022
@Martin-Jung
Copy link
Collaborator Author

Largely been superseded by a different issue ( #72 ) which should also support validation as part of the comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request low priority Quality of life Quality of life improvement
Projects
None yet
Development

No branches or pull requests

1 participant