light evaluation edits

sessions/forecast-evaluation-of-multiple-models.qmd

@@ -353,7 +353,7 @@ As in the [forecasting concepts session](forecasting-concepts), we will start by
 ::: {.callout-tip}
 ## Reminder: Key things to note about the CRPS
   - Small values are better
-  - As it is an absolute scoring rule it can be difficult to use to compare forecasts across scales.
+  - When scoring absolute values (e.g. number of cases) it can be difficult to use to compare forecasts across scales (i.e., when case numbers are different, for example between locations or at different times).
 :::
 
 First by forecast horizon.
@@ -376,19 +376,19 @@ sc_scores |>
 
 ::: {.callout-tip}
 ## Take 5 minutes 
-How do the CRPS scores change based on forecast date?
-How do the CRPS scores change with forecast horizon?
+How do the CRPS values change based on forecast date?
+How do the CRPS values change with forecast horizon?
 :::
 
 ::: {.callout-note collapse="true"}
 ## Solution
-How do the CRPS scores change based on forecast horizon?
+How do the CRPS values change based on forecast horizon?
 
 - All models have increasing CRPS with forecast horizon.
 - The more mechanistic model has the lowest CRPS at all forecast horizon.
 - The more stastical model starts to outperform the random walk model at longer time horizons.
 
-How do the CRPS scores change with forecast date?
+How do the CRPS values change with forecast date?
 
 - The more statistical model does particularly poorly around the peak of the outbreak but outperforms the random walk model.
 - The more mechanistic model does particularly well around the peak of the outbreak versus all other models
@@ -477,7 +477,7 @@ log_sc_scores <- log_sc_forecasts |>
 
 
 ::: {.callout-tip}
-Reminder: For more on scoring on the log scale see [this paper on scoring forecasts on transformed scales](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011393).
+Reminder: For more on scoring on the log scale see the paper by @bosse2023scorin.
 :::
 
 ### At a glance

sessions/forecast-evaluation.qmd

@@ -366,7 +366,7 @@ log_scores <- log_sc_forecasts |>
   score()
 ```
 
-For more on scoring on the log scale see [this paper on scoring forecasts on transformed scales](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011393).
+For more on scoring on the log scale see the paper by @bosse2023scorin.
 
 ### At a glance
 

sessions/ueifid.bib

@@ -42,5 +42,23 @@ @Article{gneiting2007
   issn            = {1537-274X},
   doi             = {10.1198/016214506000001437},
   url             = {http://dx.doi.org/10.1198/016214506000001437},
-  publisher       = {Informa UK Limited}
+  publisher       = {Informa UK Limited},
+  note            = {PDF available at https://sites.stat.washington.edu/raftery/Research/PDF/Gneiting2007jasa.pdf}
+}
+
+@Article{bosse2023scorin,
+  author          = {Bosse, Nikos I. and Abbott, Sam and Cori, Anne and van
+                  Leeuwen, Edwin and Bracher, Johannes and Funk, Sebastian},
+  title           = {Scoring epidemiological forecasts on transformed scales},
+  journal         = {PLOS Computational Biology},
+  year            = 2023,
+  editor          = {McCaw, James M},
+  volume          = 19,
+  number          = 8,
+  month           = aug,
+  pages           = {e1011393},
+  issn            = {1553-7358},
+  doi             = {10.1371/journal.pcbi.1011393},
+  url             = {http://dx.doi.org/10.1371/journal.pcbi.1011393},
+  publisher       = {Public Library of Science (PLoS)}
 }