From ce8e9d2604131212ae1587a6cc6b6a57a064b4e1 Mon Sep 17 00:00:00 2001 From: Sam Abbott Date: Mon, 4 Nov 2024 15:03:56 +0700 Subject: [PATCH 1/6] reorg sessions --- sessions.qmd | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/sessions.qmd b/sessions.qmd index 12306df..7020be4 100644 --- a/sessions.qmd +++ b/sessions.qmd @@ -18,14 +18,21 @@ comments: false ## Session 2: Forecast evaluation -14:10-15:20 +14:10-15:00 -- An overview of forecasting models (10 mins) -- Practice session: Evaluating forecasts from a range of models (60 mins) +- An introduction to forecast evaluation (5 mins) +- Practice session: Evaluating forecasts from a range of models (40 mins) +- Wrap up (5 mins) -15:30-15:50 +## Session 3: Evaluating forecasts from multiple models -- Wrap up and discussion (20 mins) +15:10-15:50 + +- Why might we want to evaluate forecasts from multiple models? (5 mins) +- Practice session: Evaluating forecasts from multiple models (40 mins) +- Wrap up (5 mins) + +*There is a coffee break at 15:30-15:40 you are welcome to attend this or keep working through the course material.* ## Session 3: Forecast ensembles From 1adc3d2c563e78e76820a6c109d910946c4815e1 Mon Sep 17 00:00:00 2001 From: Sebastian Funk Date: Mon, 4 Nov 2024 15:19:15 +0700 Subject: [PATCH 2/6] update package name to qrensemble --- sessions/forecast-ensembles.qmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sessions/forecast-ensembles.qmd b/sessions/forecast-ensembles.qmd index 9fac523..ca9fc07 100644 --- a/sessions/forecast-ensembles.qmd +++ b/sessions/forecast-ensembles.qmd @@ -46,7 +46,7 @@ The source file of this session is located at `sessions/forecast-ensembles.qmd`. ## Libraries used In this session we will use the `nfidd` package to load a data set of infection times and access stan models and helper functions, the `dplyr` and `tidyr` packages for data wrangling, `ggplot2` library for plotting, the `tidybayes` package for extracting results of the inference and the `scoringutils` package for evaluating forecasts. -We will also use `qra` for quantile regression averaging in the weighted ensemble section. +We will also use `qrensemble` for quantile regression averaging in the weighted ensemble section. ```{r libraries, message = FALSE} library("nfidd") @@ -54,7 +54,7 @@ library("dplyr") library("tidyr") library("ggplot2") library("scoringutils") -library("qra") +library("qrensemble") ``` ::: {.callout-tip} From 8c2ac81d7b02265f99692b5d8359a55ac34984df Mon Sep 17 00:00:00 2001 From: Sebastian Funk Date: Mon, 4 Nov 2024 15:32:27 +0700 Subject: [PATCH 3/6] update actions --- .github/workflows/deploy.yaml | 4 +++- .github/workflows/render-readme.yaml | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/workflows/deploy.yaml b/.github/workflows/deploy.yaml index c1f7a96..094b531 100644 --- a/.github/workflows/deploy.yaml +++ b/.github/workflows/deploy.yaml @@ -26,7 +26,9 @@ jobs: - uses: r-lib/actions/setup-r@v2 with: use-public-rspm: true - extra-repositories: 'https://stan-dev.r-universe.dev' + extra-repositories: + https://stan-dev.r-universe.dev, + https://epiforecasts.r-universe.dev - name: Install dependencies uses: r-lib/actions/setup-r-dependencies@v2 diff --git a/.github/workflows/render-readme.yaml b/.github/workflows/render-readme.yaml index e00d225..b06abde 100644 --- a/.github/workflows/render-readme.yaml +++ b/.github/workflows/render-readme.yaml @@ -23,7 +23,9 @@ jobs: uses: r-lib/actions/setup-r@v2 with: use-public-rspm: true - extra-repositories: 'https://stan-dev.r-universe.dev' + extra-repositories: + https://stan-dev.r-universe.dev, + https://epiforecasts.r-universe.dev - name: Setup pandoc uses: r-lib/actions/setup-pandoc@v2 From 8fb998b28e140d1ad1ce265105c29dcc6428c8c7 Mon Sep 17 00:00:00 2001 From: Sam Abbott Date: Mon, 4 Nov 2024 15:43:27 +0700 Subject: [PATCH 4/6] Update sessions.qmd --- sessions.qmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sessions.qmd b/sessions.qmd index 7020be4..da90949 100644 --- a/sessions.qmd +++ b/sessions.qmd @@ -34,7 +34,7 @@ comments: false *There is a coffee break at 15:30-15:40 you are welcome to attend this or keep working through the course material.* -## Session 3: Forecast ensembles +## Session 4: Forecast ensembles 15:50-17:10 @@ -42,7 +42,7 @@ comments: false - Practice session: Creating forecast ensembles (60 mins) - Wrap up (10 mins) -## Session 4: End of course summary +## Session 5: End of course summary 17:10-17:30 From efe4a0a1a12cc6d316e5b77c69298f9cfec93bd2 Mon Sep 17 00:00:00 2001 From: Sebastian Funk Date: Mon, 4 Nov 2024 15:54:02 +0700 Subject: [PATCH 5/6] fix typo --- sessions/forecast-ensembles.qmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sessions/forecast-ensembles.qmd b/sessions/forecast-ensembles.qmd index 5c3f317..8a01525 100644 --- a/sessions/forecast-ensembles.qmd +++ b/sessions/forecast-ensembles.qmd @@ -5,7 +5,7 @@ order: 8 # Introduction -As we saw in the [forecast evaluation session](forecast-evaluation-of-multiple-models), different modelling approaches have different strength and weaknesses, and it is not clear a prior which one produces the best forecast in any given situation. +As we saw in the [forecast evaluation session](forecast-evaluation-of-multiple-models), different modelling approaches have different strength and weaknesses, and it is not clear a priori which one produces the best forecast in any given situation. One way to attempt to draw strength from a diversity of approaches is the creation of so-called *forecast ensembles* from the forecasts produced by different models. In this session, we'll build ensembles using forecasts from models of different levels of mechanism vs. statistical complexity. From b0572dcf110118d7b19494e0362dc75f96ef6c02 Mon Sep 17 00:00:00 2001 From: Sebastian Funk Date: Mon, 4 Nov 2024 15:54:11 +0700 Subject: [PATCH 6/6] clean trailing whitespace --- sessions/forecast-evaluation-of-multiple-models.qmd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/sessions/forecast-evaluation-of-multiple-models.qmd b/sessions/forecast-evaluation-of-multiple-models.qmd index 03d2928..7a898a3 100644 --- a/sessions/forecast-evaluation-of-multiple-models.qmd +++ b/sessions/forecast-evaluation-of-multiple-models.qmd @@ -9,9 +9,9 @@ We can classify models along a spectrum by how much they include an understandin These different approaches all have different strength and weaknesses, and it is not clear a prior which one produces the best forecast in any given situation. One way to attempt to draw strength from a diversity of approaches is the creation of so-called *forecast ensembles* from the forecasts produced by different models. -In this session, we'll start with forecasts from models of different levels of mechanism vs. statistical complexity and build ensembles of these models. +In this session, we'll start with forecasts from models of different levels of mechanism vs. statistical complexity and build ensembles of these models. We will then compare the performance of these ensembles to the individual models and to each other. -Rather than using the forecast samples we have been using we will instead now use quantile-based forecasts. +Rather than using the forecast samples we have been using we will instead now use quantile-based forecasts. ::: {.callout-note collapse="true"} ## Representations of probabilistic forecasts @@ -63,7 +63,7 @@ The best way to interact with the material is via the [Visual Editor](https://do ## Initialisation -We set a random seed for reproducibility. +We set a random seed for reproducibility. Setting this ensures that you should get exactly the same results on your computer as we do. ```{r} @@ -132,7 +132,7 @@ ggplot(data, aes(x = t, y = Rt)) + ``` ::: -The key assumptions we are making here are: +The key assumptions we are making here are: - The population is constant and we roughly know the size of the population. - The reproduction number only changes due to susceptible depletion @@ -152,7 +152,7 @@ A statistical term that can be used to describe a time series with a trend is sa More specifically, a _stationary_ time series is defined as one whose statistical properties, such as mean and variance, do not change over time. In infectious disease epidemiology, this would only be expected for endemic diseases without external seasonal influence. -The random walk model we used in the [forecasting visualisation session](forecast-visualisation) is a special case of a more general class of models called _autoregressive (AR) models_. +The random walk model we used in the [forecasting visualisation session](forecast-visualisation) is a special case of a more general class of models called _autoregressive (AR) models_. AR models are a class of models which predict the next value in a time series as a linear combination of the previous values in the time series. The random walk model is specifically a special case of an AR(1) model where the next value in the time series is predicted as the previous value, multiplied by a value between 1 and -1 , plus some noise. This becomes a random walk when the multipled value is 0.