How to defend an AI decision to your regulator
What examiners actually look for during a model-risk review, and the documentation to hand them — field-mapped, not vague.
The examiner is not asking "is your model accurate." They are asking "can you prove you know what your model did." Different question, different answer.
The most common mistake in a model-risk review is to bring evidence of accuracy. The examiner is not there to be convinced your model is right; they are there to be convinced you can tell, after the fact, what your model did and why. Those are different meetings. One ends with a number; the other ends with a process you can show, a record you can hand over, and a computation you can replay. The second one is the one that passes.
What the examiner is actually looking for
- Provenance — which data, which versions, which model produced the output. Not "the system used relevant data"; the exact NodeVersions and the model hash.
- Reproducibility — can a third party rerun the computation and get the identical bytes. A log you cannot replay is not evidence.
- Boundaries — what the system does not do. No examiner trusts a model that claims to do everything; they trust one with a stated scope.
- Oversight — who can override, when, and how that override is recorded.
The documentation to hand them
Hand them the manifest. Not a slide deck, not a whitepaper — the per-output, hash-chained audit manifest (BR-842) for the specific decision under review. It carries the model version, the input NodeVersions, the agent trace, the output, and a replay anchor. The examiner’s questions map onto its fields; you are not narrating, you are pointing. "Which model? This field. Which inputs? These versions. Can I replay it? Here is the hash."
Do not narrate the decision to an examiner. Hand them the manifest and point. Narration invites argument; a record answers.
Separate the law from the guidance
A defensible posture names the force behind each requirement. The EU AI Act and GDPR Art. 22 are binding law; SR 11-7 is supervisory guidance; NIST AI RMF and ISO/IEC 42001 are voluntary. An examiner reads "we comply with SR 11-7" differently from "we meet the EU AI Act logging obligation." Lumping them reads as someone who does not know the difference; separating them reads as someone who does. The manifest is built to satisfy the binding tier; the voluntary tier is documentation on top.
The defensible answer to "is your model accurate?" is not "yes." It is "we measure accuracy against a disclosed baseline, and here is the calibration loop that produced the number." A "yes" is a promise; a loop is evidence.
— review prep note
What you do not claim
You do not claim the decision was correct — you claim it is reconstructable. You do not certify truth; you certify provenance and reproducibility. You do not say the model is "in production" if it is not; you say what runs, what is in integration, and what is gated. We are pre-revenue and building the POC, and the most defensible thing about the manifest is the one thing it refuses to assert: that the output was right. That refusal is what makes everything else it asserts credible.