Learning outcomes

This workshop explores what an infectious disease forecast is, how to tell whether it is any good, and how to combine forecasts into ensembles. The methods apply broadly across infectious disease epidemiology, from outbreak response to routine surveillance. While examples often use outbreak scenarios for clarity, participants should consider how these approaches apply to their own epidemiological contexts.

Assumed background

  • familiarity with R
  • basic familiarity with probability distributions and the idea of a probabilistic (rather than point) prediction
  • no prior experience of forecasting is required

Forecasting concepts

  • understanding of forecasting as an epidemiological problem
  • ability to use a simple forecasting model on an epidemiological time series in R
  • ability to visualise a probabilistic forecast and compare it to what was later observed
  • familiarity with the forecasting paradigm of maximising sharpness subject to calibration

Forecasting models

  • awareness of forecasting models as a spectrum from mechanistic to statistical
  • understanding of how mechanistic structure (e.g. susceptible depletion) and statistical structure (e.g. autoregressive and differenced models) change a forecast
  • ability to visually compare forecasts from several models

Evaluating forecasts

  • ability to visually assess forecasts
  • familiarity with metrics for evaluating probabilistic forecasts and their properties
  • understanding of proper scoring rules and the components of the (weighted) interval score and CRPS (sharpness, over- and under-prediction)
  • ability to assess calibration and bias using PIT histograms and interval coverage
  • awareness of the effect of scale (e.g. natural vs log) on forecast scores
  • ability to score probabilistic forecasts in R and to compare different models by their scores

Ensemble models

  • understanding of predictive ensembles and their properties
  • ability to create a predictive ensemble of forecasts in R
  • familiarity with different ensembling approaches, including quantile (Vincent) averaging and the linear opinion pool, and with weighted ensembles