Cat bond backtest forensics, anonymized

This is an anonymized design-partner scenario; the institution is a (re)insurer, the figures are illustrative, and no actor has agreed to anything. We are pre-revenue, building the POC. With that label on it, here is what a cat-bond backtest forensic looks like when the substrate is a versioned graph and a calibration loop, not a slide deck.

The season that already happened

A cat bond is a security whose payout depends on a loss trigger — modeled hurricane activity, typically. The model that priced the bond produced a distribution over the season before it began. The season then happened, and the realized activity is now ground truth. The forensic is not "was the model right"; it is "against the realized season, what did the model say, what would it have said under different NodeVersions, and is the gap a calibration gap or a substrate gap."

Shadow run against ground truth

The calibration harness scores a shadow run against ground truth via Brier and log-loss (deino-eval). The shadow run is the model rerun over the same NodeVersions the live forecast read; the ground truth is the realized season. The gap between them is a number, not a vibe — and the number is what earns or denies the word "probability." Gate 1b requires a shadow run to beat a disclosed baseline on one forecastable series before any forecast side is positioned as accurate. In this scenario, that gate has not passed; the forensic is the loop in motion, not a result.

The forensic is not "was the model right." It is "is the gap a calibration gap or a substrate gap" — and only one of those is fixable by tuning.

Why the substrate matters for the gap

A non-versioned model cannot tell the two gaps apart. If the input state changed between forecast and realization and the model did not record which version it read, the gap could be the model being wrong or the world having moved under it — and the examiner cannot tell. The GNL pins every NodeVersion the forecast read, so the shadow run replays against the same substrate. The gap that remains is a calibration gap, which is the kind you can close with a loop. The gap that was a substrate gap is the one that used to hide, and now cannot.

• Shadow run versus ground truth, scored with Brier and log-loss — a number, not a narrative.
• NodeVersions are pinned in the manifest, so the shadow run replays against the same substrate.
• A calibration gap is fixable by the loop; a substrate gap is a reproducibility failure. The versioned graph tells them apart.

What the (re)insurer does not get

The (re)insurer does not get a certificate that the cat bond was correctly priced. The manifest attests provenance and reproducibility, not truth — the payout trigger is what it is, and the model is a forecast, not an oracle of reality. What the (re)insurer gets is a forensic that an examiner can replay: which wind-field data, which vulnerability curves, which NodeVersions, which model version produced the pre-season cone. The cycle is honest about the gap and honest about what would close it. Calibration is the loop; Gate 1b is the gate; and neither has passed.

A cat-bond model is a forecast of a season. The forensic is the gap, scored honestly. The honest part is the part the regulator reads, and the versioned substrate is what makes it honest.

— design-partner forensic note