gallery.leaderboard()

The board. One row per model, ordered by model name. No sort key.

Usage

Source

gallery.leaderboard(panel)

There is no sort_by= argument and no default sort. Held-out ELPD and held-out CRPS are published side by side and the reader picks (decided 2026-07-25). A sort parameter would move that choice from the reader into this function, and whichever value people typed first would become the house ranking by habit. Rows come back in model-name order so a published CSV is byte-reproducible; DataFrame.sort_values is one call away, and SCORE_DIRECTION says which way each column runs so a renderer cannot guess wrong.

Columns:

The two score columns can rest on different cells, because a density and draws are independent capabilities with independent panels. Each therefore carries its own n_cells, member list and fingerprint, and there is deliberately no bare n_cells column for a reader to attach to the wrong one.

task, as_of the protocol and the training cutoff model entry name n_cohorts cohorts in the union of the two panels elpd_members the density-eligible models the ELPD panel was intersected over. The ELPD column is a function of this set, so it travels with it, the same discipline mart_publish_id gets elpd_fingerprint sha256 of the sorted ELPD panel keys n_cells_elpd the ELPD panel - identical on every row elpd SUM of the pointwise log predictive density over the ELPD panel. pd.NA, never 0, when unavailable elpd_per_cell elpd / n_cells_elpd, for reading. It cannot change a ranking (n is the same for every row) and it is NOT a repair for a coverage difference - it is here so a reader can tell 0.05 from 0.5 on a total near -14,000 elpd_status scored, or na: <reason> n_cells_zero_density cells whose pointwise ELPD is -inf. A model with elpd = -inf must show WHY on the same row, or the total reads as a bug rather than a verdict min_ess_kish smallest effective draw count behind any cell on the ELPD panel. Every per-score readout is taken from that score’s own slice, never from the union - a cell on the CRPS panel only would otherwise describe a column it is not in crps_members, crps_fingerprint, n_cells_crps the same three for the CRPS panel, which is a DIFFERENT set of cells crps SUM of the pointwise CRPS over the CRPS panel. Lower is better - the opposite direction to elpd, which is why SCORE_DIRECTION is machine-readable crps_per_cell, crps_status as above n_draws_density_min, n_draws_sample_min smallest draw count behind any of this model’s cohorts, per capability. logmeanexp is biased downward by about cv2/(2S), always in the same direction, and sample CRPS is biased by O(1/S) too, so a model that never escalated through the harness’s sampler stages scores worse for a reason that is not the model n_cells_elpd_own, n_cells_elpd_dropped, n_cells_crps_own, n_cells_crps_dropped this model’s own coverage per score, and what the intersection cost it

The totals are reduced in Python from each model’s own array, not with a pandas aggregation. groupby().sum() over an all-missing group returns 0.0 - measured here on pandas 2.3.3, and a nullable Float64 dtype does not change it, only min_count=1 does. On this board 0.0 is the best ELPD there is and the best CRPS there is, simultaneously. Making the aggregation unreachable beats guarding it.

There is no standard-error column, and no pairwise elpd_diff. The SE needs a decision this module must not make silently: the honest clustering unit is the FIT, and the fit is not the cohort for every entry - sur and copula_glm fit once per company across its lines, and the NN entries fit once over the whole panel, so a cohort-clustered SE would be wrong for four of the candidate entries in the same direction (too small). panel.by_cohort is the input any such estimator will take.

There is no KS / PIT column. The critical value 1.36/sqrt(n) assumes independence: nine cells of one cohort share one posterior and one calendar year, so at cell level that value is about three times too tight and models would be flagged as miscalibrated when they are not. It is also a different estimand from the published milestone-3 KS, which is one PIT per cohort on the run-off-to-ultimate predictive.