Model weights for blending: stacking and pseudo-BMA

Weights from pointwise held-out log predictive densities (Yao, Vehtari, Simpson and Gelman, 2018), for any model: PSIS-LOO pointwise values from elpd_loo() for a Bayesian fit, or cross-validated log densities for a GLM, GAM or neural network. stacking_weights() maximizes the log score of the mixture of the models' predictive distributions; a model that adds nothing gets weight exactly 0. pseudo_bma_weights() is proportional to exp(elpd); with bootstrap = TRUE (pseudo-BMA+) it is averaged over Bayesian-bootstrap replicates, which keeps a model that is only slightly better from taking all the weight. Blend the models' simulations with blend_predictive().

Usage

stacking_weights(lpd)

pseudo_bma_weights(lpd, bootstrap = TRUE, n_draws = 1000, seed = 0)

Arguments

lpd

A matrix (or data frame) of log densities, one row per observation and one column per model.

bootstrap

Use the Bayesian bootstrap (pseudo-BMA+).

n_draws

Bootstrap replicates.

seed

Seed; replicate b uses stream b.

Value

A named vector of weights summing to 1, named by the columns of lpd.

Details

Stacking matches an SLSQP optimum polished by Newton's method to 1e-10 (validation/scripts/stacking_weights.py); loo::stacking_weights() stops earlier and agrees to about 1e-3.

Examples

lpd <- cbind(a = c(-0.1, -0.1, -3, -3), b = c(-3, -3, -0.1, -0.1))
stacking_weights(lpd)
#>   a   b 
#> 0.5 0.5 
pseudo_bma_weights(lpd, bootstrap = FALSE)
#>   a   b 
#> 0.5 0.5