diff --git a/DESCRIPTION b/DESCRIPTION index 4bd770f5c..cbf8e4e1c 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -41,8 +41,8 @@ Description: Scoring metrics can be used either through a convenient data.frame format, or can be applied as individual functions in a vector / matrix format. All functionality has been implemented with a focus on performance and is - robustly tested. Find more information about scoringutils in the - accompanying paper (Bosse et al., 2022) . + robustly tested. Find more information about the package in the + accompanying paper (). License: MIT + file LICENSE Encoding: UTF-8 LazyData: true diff --git a/R/data.R b/R/data.R index b0f305421..056581ada 100644 --- a/R/data.R +++ b/R/data.R @@ -19,7 +19,7 @@ #' \item{model}{name of the model that generated the forecasts} #' \item{horizon}{forecast horizon in weeks} #' } -#' @source \url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint "example_quantile" @@ -45,7 +45,7 @@ #' \item{model}{name of the model that generated the forecasts} #' \item{horizon}{forecast horizon in weeks} #' } -#' @source \url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint "example_point" @@ -70,7 +70,7 @@ #' \item{prediction}{predicted value} #' \item{sample}{id for the corresponding sample} #' } -#' @source \url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint "example_continuous" @@ -125,7 +125,7 @@ #' \item{horizon}{forecast horizon in weeks} #' \item{prediction}{predicted value} #' } -#' @source \url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint "example_binary" @@ -148,7 +148,7 @@ #' \item{model}{name of the model that generated the forecasts} #' \item{horizon}{forecast horizon in weeks} #' } -#' @source \url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint "example_quantile_forecasts_only" @@ -168,7 +168,7 @@ #' \item{true_value}{true observed values} #' \item{location_name}{name of the country for which a prediction was made} #' } -#' @source \url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +#' @source \url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint "example_truth_only" #' Summary information for selected metrics diff --git a/R/input-check-helpers.R b/R/input-check-helpers.R index 07e80550d..2d7e0e51b 100644 --- a/R/input-check-helpers.R +++ b/R/input-check-helpers.R @@ -219,7 +219,7 @@ check_metrics <- function(metrics) { #' Quantiles must be in the range specified, increase monotonically, #' and contain no duplicates. #' -#' This is used in [bias_range()]() and [bias_quantile()]() to +#' This is used in [bias_range()] and [bias_quantile()] to #' provide informative errors to users. #' #' @param quantiles Numeric vector of quantiles to check diff --git a/inst/WORDLIST b/inst/WORDLIST index bf7cca877..61d2d80b9 100644 --- a/inst/WORDLIST +++ b/inst/WORDLIST @@ -1,16 +1,18 @@ AJ +al +Bosse Bracher CMD COVID CRPS Camacho -Comput Cori DSS Dawid ECDC Eggo EpiNow +et EuroCOVIDhub Gneiting Höhle @@ -44,9 +46,7 @@ facetted facetting frac ggplot -implict jss -matriced medRxiv metacran miscalibrated @@ -58,6 +58,7 @@ pval pvalues rel scoringRules +scoringutils standalone u underprediction diff --git a/man/check_quantiles.Rd b/man/check_quantiles.Rd index e702cd139..b0a2edb0b 100644 --- a/man/check_quantiles.Rd +++ b/man/check_quantiles.Rd @@ -21,7 +21,7 @@ Helper function to check that input quantiles are valid. Quantiles must be in the range specified, increase monotonically, and contain no duplicates. -This is used in \url{bias_range()} and \url{bias_quantile()} to +This is used in \code{\link[=bias_range]{bias_range()}} and \code{\link[=bias_quantile]{bias_quantile()}} to provide informative errors to users. } \keyword{internal} diff --git a/man/example_binary.Rd b/man/example_binary.Rd index 188361c0a..7cdfcfb3c 100644 --- a/man/example_binary.Rd +++ b/man/example_binary.Rd @@ -19,7 +19,7 @@ A data frame with 346 rows and 10 columns: } } \source{ -\url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +\url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint } \usage{ example_binary diff --git a/man/example_continuous.Rd b/man/example_continuous.Rd index a82df3bb3..7c5de286d 100644 --- a/man/example_continuous.Rd +++ b/man/example_continuous.Rd @@ -20,7 +20,7 @@ A data frame with 13,429 rows and 10 columns: } } \source{ -\url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +\url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint } \usage{ example_continuous diff --git a/man/example_point.Rd b/man/example_point.Rd index 77a893df0..1a3036617 100644 --- a/man/example_point.Rd +++ b/man/example_point.Rd @@ -20,7 +20,7 @@ A data frame with } } \source{ -\url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +\url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint } \usage{ example_point diff --git a/man/example_quantile.Rd b/man/example_quantile.Rd index 5e3ac4c9f..0e996eef3 100644 --- a/man/example_quantile.Rd +++ b/man/example_quantile.Rd @@ -20,7 +20,7 @@ A data frame with } } \source{ -\url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +\url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint } \usage{ example_quantile diff --git a/man/example_quantile_forecasts_only.Rd b/man/example_quantile_forecasts_only.Rd index 4ca76a5c8..00520c8cd 100644 --- a/man/example_quantile_forecasts_only.Rd +++ b/man/example_quantile_forecasts_only.Rd @@ -18,7 +18,7 @@ A data frame with 7,581 rows and 9 columns: } } \source{ -\url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +\url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint } \usage{ example_quantile_forecasts_only diff --git a/man/example_truth_only.Rd b/man/example_truth_only.Rd index 9958a0e24..8cfa41498 100644 --- a/man/example_truth_only.Rd +++ b/man/example_truth_only.Rd @@ -15,7 +15,7 @@ A data frame with 140 rows and 5 columns: } } \source{ -\url{https://github.com/covid19-forecast-hub-europe/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint +\url{https://github.com/european-modelling-hubs/covid19-forecast-hub-europe/commit/a42867b1ea152c57e25b04f9faa26cfd4bfd8fa6/} # nolint } \usage{ example_truth_only diff --git a/man/scoringutils-package.Rd b/man/scoringutils-package.Rd index c6efc6415..a44ad90ef 100644 --- a/man/scoringutils-package.Rd +++ b/man/scoringutils-package.Rd @@ -6,9 +6,9 @@ \alias{scoringutils-package} \title{scoringutils: Utilities for Scoring and Assessing Predictions} \description{ -Provides a collection of metrics and proper scoring rules (Tilmann Gneiting & Adrian E Raftery (2007) \doi{10.1198/016214506000001437}, Jordan, A., Krüger, F., & Lerch, S. (2019) \doi{10.18637/jss.v090.i12}) within a consistent framework for evaluation, comparison and visualisation of forecasts. In addition to proper scoring rules, functions are provided to assess bias, sharpness and calibration (Sebastian Funk, Anton Camacho, Adam J. Kucharski, Rachel Lowe, Rosalind M. Eggo, W. John Edmunds (2019) \doi{10.1371/journal.pcbi.1006785}) of forecasts. Several types of predictions (e.g. binary, discrete, continuous) which may come in different formats (e.g. forecasts represented by predictive samples or by quantiles of the predictive distribution) can be evaluated. Scoring metrics can be used either through a convenient data.frame format, or can be applied as individual functions in a vector / matrix format. All functionality has been implemented with a focus on performance and is robustly tested. Find more information about scoringutils in the accompanying paper (Bosse et al., 2022) \href{https://arxiv.org/abs/2205.07090v1}{arXiv:2205.07090v1}. +Provides a collection of metrics and proper scoring rules (Tilmann Gneiting & Adrian E Raftery (2007) \doi{10.1198/016214506000001437}, Jordan, A., Krüger, F., & Lerch, S. (2019) \doi{10.18637/jss.v090.i12}) within a consistent framework for evaluation, comparison and visualisation of forecasts. In addition to proper scoring rules, functions are provided to assess bias, sharpness and calibration (Sebastian Funk, Anton Camacho, Adam J. Kucharski, Rachel Lowe, Rosalind M. Eggo, W. John Edmunds (2019) \doi{10.1371/journal.pcbi.1006785}) of forecasts. Several types of predictions (e.g. binary, discrete, continuous) which may come in different formats (e.g. forecasts represented by predictive samples or by quantiles of the predictive distribution) can be evaluated. Scoring metrics can be used either through a convenient data.frame format, or can be applied as individual functions in a vector / matrix format. All functionality has been implemented with a focus on performance and is robustly tested. Find more information about the package in the accompanying paper (\doi{10.48550/arXiv.2205.07090}). -Provides a collection of metrics and proper scoring rules (Tilmann Gneiting & Adrian E Raftery (2007) \doi{10.1198/016214506000001437}, Jordan, A., Krüger, F., & Lerch, S. (2019) \doi{10.18637/jss.v090.i12}) within a consistent framework for evaluation, comparison and visualisation of forecasts. In addition to proper scoring rules, functions are provided to assess bias, sharpness and calibration (Sebastian Funk, Anton Camacho, Adam J. Kucharski, Rachel Lowe, Rosalind M. Eggo, W. John Edmunds (2019) \doi{10.1371/journal.pcbi.1006785}) of forecasts. Several types of predictions (e.g. binary, discrete, continuous) which may come in different formats (e.g. forecasts represented by predictive samples or by quantiles of the predictive distribution) can be evaluated. Scoring metrics can be used either through a convenient data.frame format, or can be applied as individual functions in a vector / matrix format. All functionality has been implemented with a focus on performance and is robustly tested. Find more information about scoringutils in the accompanying paper (Bosse et al., 2022) \href{https://arxiv.org/abs/2205.07090v1}{arXiv:2205.07090v1}. +Provides a collection of metrics and proper scoring rules (Tilmann Gneiting & Adrian E Raftery (2007) \doi{10.1198/016214506000001437}, Jordan, A., Krüger, F., & Lerch, S. (2019) \doi{10.18637/jss.v090.i12}) within a consistent framework for evaluation, comparison and visualisation of forecasts. In addition to proper scoring rules, functions are provided to assess bias, sharpness and calibration (Sebastian Funk, Anton Camacho, Adam J. Kucharski, Rachel Lowe, Rosalind M. Eggo, W. John Edmunds (2019) \doi{10.1371/journal.pcbi.1006785}) of forecasts. Several types of predictions (e.g. binary, discrete, continuous) which may come in different formats (e.g. forecasts represented by predictive samples or by quantiles of the predictive distribution) can be evaluated. Scoring metrics can be used either through a convenient data.frame format, or can be applied as individual functions in a vector / matrix format. All functionality has been implemented with a focus on performance and is robustly tested. Find more information about the package in the accompanying paper (\doi{10.48550/arXiv.2205.07090}). } \seealso{ Useful links: