Gate 1b: the calibration gate

Gate 1b is the calibration gate. It takes the output of the shadow-run loop — shadow runs scored against ground truth with Brier and log-loss via deino-eval — and asks one question: does the shadow run beat a disclosed baseline on one forecastable series? The baseline is disclosed, meaning it is written down and the reader knows what it is. The series is forecastable, meaning it is a thing that can be forecast (not noise, not a one-off). The score is measured, meaning it came from the loop, not from an assertion. If the shadow run beats the baseline on those terms, Gate 1b passes. If it does not, the forecast side does not scale.

A disclosed baseline, not a straw man

The baseline is disclosed, which is the discipline that makes the gate honest. A baseline you do not disclose is a baseline you can choose to lose — pick a weak comparator, beat it, claim accuracy. A disclosed baseline is one the reader can check: what was the comparator, what was the series, what was the score. The baseline is named in advance, the series is named in advance, and the comparison is on the record. Gate 1b is not we did better than something; it is we did better than this, on this, measured this way.

A baseline you do not disclose is a baseline you can choose to lose. Gate 1b is not we did better than something; it is we did better than this, on this, measured this way.

One forecastable series is enough to start

The gate is one series, not many. That is deliberate. Calibration is hard, and requiring the platform to beat a baseline on every series before scaling anything would mean scaling nothing for years. One forecastable series — one domain where the shadow-run loop has accumulated enough scored runs to beat a disclosed baseline — is enough to position the forecast side as accurate in that domain, and only in that domain. The gate is per-series because calibration is per-series. A platform that is calibrated on sovereign credit is not thereby calibrated on commodity shocks. Gate 1b is honest about scope: pass it on one series, claim accuracy on one series.

Why the gate gates scale

The gate is what prevents an uncalibrated forecast from being positioned as calibrated. Without Gate 1b, a platform can ship cones, call them probabilities, and let the buyer discover whether they mean anything. With Gate 1b, the platform cannot call a cone a probability until the shadow-run loop has measured it against a disclosed baseline and won. The gate protects the word. Probability is a word that implies a measurement; Gate 1b is the measurement that earns the word. We are pre-revenue, building the POC; Gate 1b is research-gated, meaning the loop has not yet filled with enough scored runs to beat a baseline on any series. The gate is the gate; it is not passed yet.

• Disclosed baseline: named in advance, checkable by the reader. Not a straw man chosen to lose.
• One forecastable series: pass the gate on one domain, claim accuracy on one domain. Scope is honest.
• Measured score: Brier and log-loss from the shadow-run loop, not an assertion.
• The gate gates scale: no forecast side is positioned as accurate until the gate passes on its series.

The gate that protects the word

Calibration earns the word probability. A probability without a calibration loop is a decoration; a probability with a calibration loop that has not beaten a baseline is a hypothesis. Gate 1b is the gate that turns the hypothesis into a measurement — the point where the shadow-run loop has scored enough runs, on a disclosed baseline, on one forecastable series, to say: this cone is calibrated, in this domain, to this degree. That is a lot of qualification, and the qualification is the honesty. An unqualified it is accurate is the claim the gate exists to prevent.

Probability is a word that implies a measurement. Gate 1b is the measurement that earns the word, and until the gate passes, the word is withheld — on purpose, by construction.

— gate design note