Modelling complex reporting processes

Nowcasting and forecasting of infectious disease dynamics

The model we have so far

In the joint nowcasting session we modelled each cell of the reporting triangle as

\[ n_{t,d} \mid \lambda_{t}, p_{t,d} \sim \text{Poisson}\left(\lambda_{t} \times p_{t,d}\right) \]

• \(\lambda_{t}\): expected onsets on day \(t\) (a daily random walk)
• \(p_{t,d}\): probability of an onset on day \(t\) being reported with delay \(d\)
• a single delay distribution, the same on every day

Ideal for learning, and fine when the reporting process is simple.

Each cell \(n_{t,d}\) of the reporting triangle (by Johannes Bracher).

Real reporting is messier

A single, fixed delay cannot capture:

• Day-of-week effects (fewer onsets or reports at weekends, or delays that differ by weekday)
• Time-varying delays (delays drift as a system gains or loses capacity)
• Weekly / batched reporting (data released weekly, not daily)
• Strata (delays differ by age, region, …)
• Missing reference dates

What is the downside of using Stan directly?

To add a day-of-week effect and a time-varying delay by hand:

• Turn the delay vector into a matrix \(p_{t,d}\), times a report-date factor \(w_{(t+d)\bmod 7}\)
• A random walk on the delay parameters with priors
• Re-impose the per-date sum-to-one constraint every day
• More parameters, bigger transformed parameters, slower gradients, more to test

epinowcast: one flexible tool

epinowcast is a Bayesian framework for real-time surveillance.

• A direct generalisation of our joint model: triangle in, counts + reporting out
• Each component is a module, specified with an R formula
• One of several tools

Tip

If you understand the joint nowcasting session, you already understand what epinowcast is doing, just with more flexible components.

The same model, as modules

The expectation, reference and observation modules mirror what we built by hand. The report date module is new and handles effects on the report date, such as day-of-week reporting. The missing data module is also new and handles reports with an unknown reference date.

Mapping back to the bespoke model

Bespoke model	epinowcast module
\(\lambda_{t}\) random walk	enw_expectation(~ 0 + (1 | day))
\(p_{t,d}\) delay distribution	enw_reference(~ 1)
report-date effects	enw_report(~ ...)
Poisson likelihood	enw_obs(family = "poisson")

Note

The enw_expectation() formula acts on the daily growth rate, so a random effect by day gives a geometric random walk on the expected counts.

Day-of-week effects

A weekly pattern can enter in three distinct places:

• Reference date, on the counts: events truly happen at different rates by day of week → goes in the expectation module
• Reference date, on the delay: the delay distribution itself differs by the onset day of week → goes in the reference module
• Report date: the system processes reports at different rates by day of week → goes in the report module

Time-varying delay

Delays may lengthen under strain and shorten with spare capacity.

enw_reference(~ rw(week))   # delay drifts week to week

Weekly (batched) reporting

Data are often reported weekly even though events happen daily.

weekly <- enw_aggregate_cumulative(enw_long, timestep = "week")
enw_preprocess_data(weekly, max_delay = 3, timestep = "week")

• The reporting schedule is a property of the data, not the model
• The same expectation and delay modules apply, just on a weekly timestep

Others, same idea

• Strata (age, region): enw_reference(~ (1 | age_group)), own delay per stratum, partial pooling across strata
• Missing reference dates: a dedicated enw_missing() module

Note

epinowcast is one tool among several. Models written using Stan directly, baselinenowcast benchmarks, and other frameworks all have their place. The right choice depends on the problem.

Your Turn

1. Reproduce the joint nowcasting model in epinowcast
2. Add day-of-week effects (on onsets, on the delay, and on reporting)
3. Let the delay vary over time
4. Fit weekly (batched) reporting with a reporting schedule

Return to the session