What an examiner actually looks for
Provenance, reproducibility, boundaries, oversight — the four-part question behind every review.
Every model-risk review, regardless of jurisdiction, reduces to four questions. Learn to hear them, and every submission reads as if someone understood the room.
Every model-risk review, regardless of jurisdiction — EU AI Act, GDPR Art. 22, SR 11-7, BCRA expectations — reduces to four questions. The wording differs; the questions do not. Provenance: which data, which versions, which model produced the output. Reproducibility: can a third party rerun the computation and get the identical bytes. Boundaries: what does the system not do. Oversight: who can override, when, and how is the override recorded. A submission that answers all four is a submission that passes; a submission that answers only the accuracy question — "is the model right?" — is a submission that fails, because the examiner was not asking that.
We are pre-revenue and building the POC. This post maps the four questions to the manifest (BR-842) fields, because the manifest is the artifact designed to answer each one without narration.
The four questions, mapped
- Provenance — which model, which version, which inputs. Manifest fields: model_version (hash-pinned), input_nodeversions (L1 to L4 refs, hash-linked).
- Reproducibility — can a third party rerun and get identical bytes. Manifest field: replay_hash (the replay anchor).
- Boundaries — what the system does not do. Manifest fields: scope and the stated limit; no examiner trusts a model that claims everything.
- Oversight — who can override, when, how recorded. Manifest field: axis_override (typed, hash-linked, non-repudiable).
Why boundaries is the one teams skip
Most submissions answer provenance and reproducibility (or try to) and skip boundaries, because stating what the system does not do feels like admitting a weakness. It is the opposite. An examiner trusts a model with a stated scope because the stated scope is the thing the examiner can rely on; a model that claims to do everything is a model the examiner cannot rely on for anything. The GNL is public signals only at L4 by design — the strictest governance layer is a stated boundary, not a default. Bounded awakening keeps per-query cost flat as the graph grows — a stated limit, not an aspiration. The stated boundary is the credibility; the unstated universal claim is the liability.
No examiner trusts a model that claims everything. The stated boundary is the thing the examiner believes; the unstated universal claim is the thing they do not.
Oversight as a recorded act
Oversight is not a paragraph in a policy document; it is a recorded act. The AxisOverride is a typed, hash-linked value that seals who changed what, when, into the chain. An override is a first-class value, not a comment in a Slack thread. When an examiner asks "who can override, and how is it recorded?" the answer is a field, not a narrative: the override field, the chain link, the timestamp. A human in the loop whose intervention is not in the chain is, for audit purposes, a human who was not there. The manifest makes oversight evidentiary, not aspirational.
What the four questions are not asking
The four questions are not asking "is the model accurate?" Accuracy is a calibration question, and it is answered by a loop, not by a submission: shadow runs against ground truth, scored with Brier and log-loss via deino-eval, gated by Gate 1b before any forecast side is positioned as accurate. The examiner asks the four questions because they are the questions a record can answer; accuracy is the question a loop answers, and the loop is separate. A submission that leads with accuracy has answered the question no one asked; a submission that answers the four has answered the ones everyone does. The manifest attests provenance and reproducibility — never truth — and that is exactly what the four questions are reaching for.
The examiner asks the four questions because they are the questions a record can answer. Accuracy is the question a loop answers — and the loop is separate, on purpose.
— model-risk review note
The honest scope
We are building the POC, pre-revenue. The manifest fields exist; the calibration loop is inert until it is filled. What the manifest gives a firm is the ability to answer the four questions by pointing, not by narrating — and pointing is what an examiner reads as competence. Narration invites argument; a record answers. The four questions are the same in every jurisdiction because they are the questions behind every review; the manifest is the artifact designed to answer them in one place.