Monitor a model: actual against expected by period — actual_vs

For a stored model's predictions on new data, the actual and expected totals of each period, A = sum(w * y) and E = sum(w * mu), with the z-score (A - E) / sqrt(dispersion * sum(w * V(mu))) under the model's variance function V: about standard normal while the model holds. The trend is the slope of A / E - 1 per period step (periods in sorted order), weighted by each period's precision, with its standard error; trend_z beyond about 2 suggests drift.

Usage

actual_vs_expected(
  periods,
  actual,
  expected,
  family,
  weights = NULL,
  dispersion = 1,
  theta = NULL,
  power = NULL
)

Arguments

periods: One period label per row (numbers or strings).
actual: Observed responses.
expected: The model's predicted means for the same rows, e.g. predict(model, newdata).
family: As in glm_fit().
weights: Optional prior weights as fitted (exposure for a rate); leave out for counts with exposure in the offset.
dispersion: The model's dispersion, e.g. model@dispersion.
theta, power: Negative binomial theta, Tweedie power.

Value

A data frame with one row per period: period, n, weight, actual, expected, ratio, std_dev and z; attributes total (the same for all periods) and trend (slope, std_error, z).

Examples

actual_vs_expected(c(2023, 2023, 2024, 2024), c(1, 3, 2, 6), rep(2, 4), "poisson")
#>   period n weight actual expected ratio std_dev z
#> 1   2023 2      2      4        4     1       2 0
#> 2   2024 2      2      8        4     2       2 2