Skip to content

Commit

Permalink
Automatic readme update [ci skip]
Browse files Browse the repository at this point in the history
  • Loading branch information
@actions-user
actions-user committed Feb 28, 2024
1 parent 83d6cfa commit 04e5c80
Showing 1 changed file with 55 additions and 23 deletions.
78 changes: 55 additions & 23 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -165,33 +165,50 @@ example_quantile %>%
digits = 2
) %>%
kable()
#> ! Column "sample_id" not found in data.
#> ! Column "sample_id" not found in data.
#> Some rows containing NA values may be removed. This is fine if not unexpected.
#>
#> ! Column "sample_id" not found in data.
#> ! Column "sample_id" not found in data.
#> Some rows containing NA values may be removed. This is fine if not unexpected.
#>
#> ! Column "sample_id" not found in data.
#> Some rows containing NA values may be removed. This is fine if not unexpected.
#> Warning in (function (..., fun) : Function execution failed, returning NULL.
#> Error: comparison (==) is possible only for atomic and list types
#> Warning in `[.data.table`(data, , `:=`((metric_name), do.call(run_safely, :
#> Column 'ae_median' does not exist to remove
#> Warning in get_score_names(scores, error = TRUE): The following scores have
#> been previously computed, but are no longer column names of the data:
#> `interval_coverage, quantile_coverage, quantile_coverage_deviation`. See
#> `?get_score_names` for further information.
#> `interval_coverage, quantile_coverage, quantile_coverage_deviation, ae_median`.
#> See `?get_score_names` for further information.

#> Warning in get_score_names(scores, error = TRUE): The following scores have
#> been previously computed, but are no longer column names of the data:
#> `interval_coverage, quantile_coverage, quantile_coverage_deviation`. See
#> `?get_score_names` for further information.
#> `interval_coverage, quantile_coverage, quantile_coverage_deviation, ae_median`.
#> See `?get_score_names` for further information.
#> Warning in get_score_names(scores): The following scores have been previously
#> computed, but are no longer column names of the data: `interval_coverage,
#> quantile_coverage, quantile_coverage_deviation`. See `?get_score_names` for
#> further information.
#> quantile_coverage, quantile_coverage_deviation, ae_median`. See
#> `?get_score_names` for further information.
#> Warning in summarise_scores(., by = c("model", "target_type")): The names of
#> the scores previously computed do not match the names of the columns in
#> `scores`. This may lead to unexpected results.
#> Warning in summarise_scores(., fun = signif, digits = 2): The names of the
#> scores previously computed do not match the names of the columns in `scores`.
#> This may lead to unexpected results.
```

| model | target_type | wis | overprediction | underprediction | dispersion | bias | interval_coverage_50 | interval_coverage_90 | interval_coverage_deviation | ae_median | wis_relative_skill | wis_scaled_relative_skill |
|:----------------------|:------------|------:|---------------:|----------------:|-----------:|--------:|---------------------:|---------------------:|----------------------------:|----------:|-------------------:|--------------------------:|
| EuroCOVIDhub-baseline | Cases | 28000 | 14000.0 | 10000.0 | 4100 | 0.0980 | 0.33 | 0.82 | -0.120 | 38000 | 1.30 | 1.6 |
| EuroCOVIDhub-baseline | Deaths | 160 | 66.0 | 2.1 | 91 | 0.3400 | 0.66 | 1.00 | 0.120 | 230 | 2.30 | 3.8 |
| EuroCOVIDhub-ensemble | Cases | 18000 | 10000.0 | 4200.0 | 3700 | -0.0560 | 0.39 | 0.80 | -0.100 | 24000 | 0.82 | 1.0 |
| EuroCOVIDhub-ensemble | Deaths | 41 | 7.1 | 4.1 | 30 | 0.0730 | 0.88 | 1.00 | 0.200 | 53 | 0.60 | 1.0 |
| UMass-MechBayes | Deaths | 53 | 9.0 | 17.0 | 27 | -0.0220 | 0.46 | 0.88 | -0.025 | 78 | 0.75 | 1.3 |
| epiforecasts-EpiNow2 | Cases | 21000 | 12000.0 | 3300.0 | 5700 | -0.0790 | 0.47 | 0.79 | -0.070 | 28000 | 0.95 | 1.2 |
| epiforecasts-EpiNow2 | Deaths | 67 | 19.0 | 16.0 | 32 | -0.0051 | 0.42 | 0.91 | -0.045 | 100 | 0.98 | 1.6 |
| model | wis | overprediction | underprediction | dispersion | bias | interval_coverage_50 | interval_coverage_90 | interval_coverage_deviation | wis_relative_skill | wis_scaled_relative_skill |
|:----------------------|------:|---------------:|----------------:|-----------:|--------:|---------------------:|---------------------:|----------------------------:|-------------------:|--------------------------:|
| EuroCOVIDhub-baseline | 28000 | 14000.0 | 10000.0 | 4100 | 0.0980 | 0.33 | 0.82 | -0.120 | 1.30 | 1.6 |
| EuroCOVIDhub-baseline | 160 | 66.0 | 2.1 | 91 | 0.3400 | 0.66 | 1.00 | 0.120 | 2.30 | 3.8 |
| EuroCOVIDhub-ensemble | 18000 | 10000.0 | 4200.0 | 3700 | -0.0560 | 0.39 | 0.80 | -0.100 | 0.82 | 1.0 |
| EuroCOVIDhub-ensemble | 41 | 7.1 | 4.1 | 30 | 0.0730 | 0.88 | 1.00 | 0.200 | 0.60 | 1.0 |
| UMass-MechBayes | 53 | 9.0 | 17.0 | 27 | -0.0220 | 0.46 | 0.88 | -0.025 | 0.75 | 1.3 |
| epiforecasts-EpiNow2 | 21000 | 12000.0 | 3300.0 | 5700 | -0.0790 | 0.47 | 0.79 | -0.070 | 0.95 | 1.2 |
| epiforecasts-EpiNow2 | 67 | 19.0 | 16.0 | 32 | -0.0051 | 0.42 | 0.91 | -0.045 | 0.98 | 1.6 |

`scoringutils` contains additional functionality to transform forecasts,
to summarise scores at different levels, to visualise them, and to
Expand All @@ -213,8 +230,23 @@ example_quantile %>%
score %>%
summarise_scores(by = c("model", "target_type", "scale")) %>%
head()
#> ! Column "sample_id" not found in data.
#> ! Column "sample_id" not found in data.
#> Some rows containing NA values may be removed. This is fine if not unexpected.
#>
#> ! Column "sample_id" not found in data.
#> ! Column "sample_id" not found in data.
#> Some rows containing NA values may be removed. This is fine if not unexpected.
#> Warning in (function (..., fun) : Function execution failed, returning NULL.
#> Error: comparison (==) is possible only for atomic and list types
#> Warning in `[.data.table`(data, , `:=`((metric_name), do.call(run_safely, :
#> Column 'ae_median' does not exist to remove
#> Warning in get_score_names(scores, error = TRUE): The following scores have
#> been previously computed, but are no longer column names of the data:
#> `ae_median`. See `?get_score_names` for further information.
#> Warning in summarise_scores(., by = c("model", "target_type", "scale")): The
#> names of the scores previously computed do not match the names of the columns
#> in `scores`. This may lead to unexpected results.
#> model target_type scale wis overprediction
#> <char> <char> <char> <num> <num>
#> 1: EuroCOVIDhub-ensemble Cases natural 11550.70664 3650.004755
Expand All @@ -231,14 +263,14 @@ example_quantile %>%
#> 4: 4.103261 30.18099 0.07265625 0.8750000
#> 5: 2.098505 91.40625 0.33906250 0.6640625
#> 6: 16.800951 26.87239 -0.02234375 0.4609375
#> interval_coverage_90 interval_coverage_deviation ae_median
#> <num> <num> <num>
#> 1: 0.8046875 -0.10230114 17707.95312
#> 2: 0.8203125 -0.11437500 32080.48438
#> 3: 0.7890625 -0.06963068 21530.69531
#> 4: 1.0000000 0.20380682 53.13281
#> 5: 1.0000000 0.12142045 233.25781
#> 6: 0.8750000 -0.02488636 78.47656
#> interval_coverage_90 interval_coverage_deviation
#> <num> <num>
#> 1: 0.8046875 -0.10230114
#> 2: 0.8203125 -0.11437500
#> 3: 0.7890625 -0.06963068
#> 4: 1.0000000 0.20380682
#> 5: 1.0000000 0.12142045
#> 6: 0.8750000 -0.02488636
```

## Citation
Expand Down

0 comments on commit 04e5c80

Please sign in to comment.