## gallery.CohortForecast One model, one cohort, one cutoff, one field, at the held-out cells. Usage ``` python gallery.CohortForecast( model, task, cells, field, log_density=None, draws=None, density_absence=None, draws_absence=None ) ``` This granularity is forced from both sides. Below it, every layer is single-cohort and refuses to be otherwise: [next_diagonal](gallery.next_diagonal.md#ibnr.gallery.next_diagonal) raises on more than one segment combination (measured before the guard: 20 January-origin cohorts plus one July-origin cohort returned 2 cells instead of 62), and all three contract builders raise on a multi-cohort triangle. Above it, every layer loops per cohort, and `kernels.harness` runs those loops in **spawned processes** - so whatever a worker returns must be picklable, which a live entry or an `InferenceData` is not. Plain numpy plus a `HoldoutCells` is. `cells` is carried whole rather than copied piecemeal. That is provenance: only [next_diagonal](gallery.next_diagonal.md#ibnr.gallery.next_diagonal) can build one, so a [CohortForecast](gallery.CohortForecast.md#ibnr.gallery.CohortForecast) cannot describe cells that were never shown to be held out, and [as_of](Triangle.as_of.md#ibnr.Triangle.as_of), `eval_date`, the segment schema, the triangle basis, `train_origins` and the upstream exclusion census all arrive consistent with each other by construction rather than by five parallel arguments that can drift. `log_density` is `(n_draws, n_cells)` on Lebesgue-on-the-loss-amount - the scale `ScoresHeldout.log_lik_at` guarantees and the only scale this module accepts. Either it or `absence` is set, never both and never neither. A pooled fit (the NN entries fit once globally, then predict per cohort) produces one [CohortForecast](gallery.CohortForecast.md#ibnr.gallery.CohortForecast) per scored cohort from a single fit. That is still held out - the pooled model trained on the [as_of](Triangle.as_of.md#ibnr.Triangle.as_of) slice of every cohort - but `model` names the entry, not the fit, and any future clustered standard error must cluster on the FIT, which for those entries is the whole panel and not the cohort. Refused at construction, each because the alternative computes: - an array with no [Absence](gallery.Absence.md#ibnr.gallery.Absence), or an [Absence](gallery.Absence.md#ibnr.gallery.Absence) with an array; - a width that disagrees with the field-narrowed cell count - the check that pins the arrays to their keys; - fewer than 2 draws (see `logmeanexp()`); - zero cells: a cohort whose whole diagonal was excluded upstream has nothing to score, and a zero-width forecast would pass alignment invisibly while making the model look like it covered the cohort. Read `cells.exclusion_counts()` instead; - any NaN; - zero variance across draws in every cell, which means someone handed a repeated point estimate rather than posterior draws. `logmeanexp` then degenerates to the plug-in log density, which is systematically overconfident and looks completely normal. ## Parameter Attributes `model: str` `task: str` `cells: HoldoutCells` `field: str` `log_density: np.ndarray | None = None` `draws: np.ndarray | None = None` `density_absence: Absence | None = None` `draws_absence: Absence | None = None` ## Attributes | Name | Description | |----|----| | [cohort](#cohort) | The cohort's segment VALUES. `()` for an unsegmented triangle. | | [key_frame](#key_frame) | One row per covered cell: the panel key, then the payload. | | [keys](#keys) | Panel keys, in array column order. Tuples, so they set-operate. | | [measure](#measure) | The TRIANGLE's basis, cumulative or incremental - not the density's. | | [n_draws](#n_draws) | Draw count behind whichever arrays are present, or `None`. | ------------------------------------------------------------------------ #### cohort The cohort's segment VALUES. `()` for an unsegmented triangle. `cohort: tuple` ------------------------------------------------------------------------ #### key_frame One row per covered cell: the panel key, then the payload. `key_frame: pd.DataFrame` In the same order as `log_density`'s columns, which is what makes [pointwise_elpd()](gallery.CohortForecast.md#ibnr.gallery.CohortForecast.pointwise_elpd) joinable. ------------------------------------------------------------------------ #### keys Panel keys, in array column order. Tuples, so they set-operate. `keys: list[tuple]` ------------------------------------------------------------------------ #### measure The TRIANGLE's basis, cumulative or incremental - not the density's. `measure: str` ------------------------------------------------------------------------ #### n_draws Draw count behind whichever arrays are present, or `None`. `n_draws: int | None` The two capabilities may carry different draw counts - a density is one value per posterior draw while a sampler may thin - so this reports the density's when there is one and the draws' otherwise, and `n_draws_for()` answers per axis. ## Methods | Name | Description | |----|----| | [ess_kish()](#ess_kish) | `(n_cells,)` effective draws behind each cell. See `_ess_kish()`. | | [pointwise_crps()](#pointwise_crps) | `(n_cells,)` CRPS per cell, against the realized outcomes. Lower is better. | | [pointwise_elpd()](#pointwise_elpd) | `(n_cells,)` log predictive density per cell, `logmeanexp` over draws. | | [unavailable()](#unavailable) | A forecast that offers NEITHER capability, naming a reason for each. | ------------------------------------------------------------------------ #### ess_kish() `(n_cells,)` effective draws behind each cell. See `_ess_kish()`. Usage ``` python ess_kish() ``` ------------------------------------------------------------------------ #### pointwise_crps() `(n_cells,)` CRPS per cell, against the realized outcomes. Lower is better. Usage ``` python pointwise_crps() ``` The draws arrive on the TRIANGLE's basis - that is `PredictsHeldout.predict_at`'s guarantee - so they are scored against `cells.values` directly and no scale question survives to this layer. CRPS is translation-equivariant, so an entry that models increments and one that models cumulatives give comparable numbers once each is on its triangle's basis; it is scale-*dependent* in the other sense, though, so a CRPS in dollars and one in thousands are not comparable. That is the same units question the panel's `value` agreement check already catches, one layer up. ------------------------------------------------------------------------ #### pointwise_elpd() `(n_cells,)` log predictive density per cell, `logmeanexp` over draws. Usage ``` python pointwise_elpd() ``` Per cell, THEN summed - never summed inside the exponential. Summing the log densities of a cohort's cells before the reduction gives the log density of the whole diagonal *jointly* under each draw, which is a legitimate but different quantity: the cells share one posterior, so the joint is not the product of the marginals and it is not additive across cohorts in the way a leaderboard and a stacking objective both need. ------------------------------------------------------------------------ #### unavailable() A forecast that offers NEITHER capability, naming a reason for each. Usage ``` python unavailable( *, model, task, cells, density_reason, draws_reason, detail="", field=None ) ``` Both reasons are REQUIRED and neither has a default. "Does not subclass `ScoresHeldout`" is not by itself evidence of which reason applies, and a default would put the wrong one on a board row: only four registered entries have no density on principle while five more simply have no scorer yet, and printing `no_predictive_density` next to `sur` or `copula_glm` would be a false statement about the model. Two reasons rather than one because the axes are independent and usually differ. `deterministic/mack` is the clearest case: `no_predictive_density` on the density axis is permanent (its bootstrap states no observation model), while its draws axis is real - it subclasses `PredictsHeldout` and belongs on the CRPS board. A single shared reason would libel an entry shaped like that on one axis or the other.