Rework add coverage to work with raw forecasts #426
Conversation
…ame of the coverage columns
…eviously failing due to `add_coverage()`
Codecov Report
@@ Coverage Diff @@
## expand-tests2 #426 +/- ##
================================================
Coverage ? 80.25%
================================================
Files ? 20
Lines ? 1702
Branches ? 0
================================================
Hits ? 1366
Misses ? 336
Partials ? 0 📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today! |
nikosbosse
changed the title
| data[, quantile_coverage_deviation := quantile_coverage - quantile] | ||
|
|
||
| # reset column order | ||
| new_metrics <- c("interval_coverage", "interval_coverage_deviation", |
There was a problem hiding this comment.
can you use the metrices functions vs duplicating here to make the code cleaner?
There was a problem hiding this comment.
At the moment not really - the thing is that the metrics functions do a many-to-one mapping (i.e. you get one coverage value per forecast (which comprises several quantiles)), whereas this is one-to-one, i.e. one value per quantile / interval.
We need to think again about how to handle one-to-one functions that exist currently and what to do with them: #451
| @@ -91,8 +91,8 @@ Forecasts can be easily and quickly scored using the `score()` function. `score( | |||
| example_quantile %>% | |||
| set_forecast_unit(c("location", "target_end_date", "target_type", "horizon", "model")) %>% | |||
| validate() %>% | |||
| add_coverage() %>% | |||
There was a problem hiding this comment.
if its in the default metrics list what does this actually do? Is it going to be clear to users why it isn't part of score?
There was a problem hiding this comment.
Updating the readme is on the list: #439
This function updates the
add_coverage()function yet again. This timeadd_coverage()now adds rowwise coverage values for interval coverage and quantile coverage