The design review for a governed AI system always arrives at the same four questions. I know because I’ve answered them from both sides of the table — as the person proposing the enforcement layer and as the person deciding whether to trust one.
They are: Where does enforcement live? What is validation allowed to cost? What happens when the guardrail itself fails? And why can’t we just have a model check the model? Each is a genuine trade-off, which means each has a wrong answer that sounds reasonable in the meeting. This article is the version I wish someone had handed me before my first such review — with the regulated-industry stakes from Part 1 kept in frame, because regulation converts several of these from taste into obligation.
Central versus edge: where enforcement lives
Central enforcement puts every model interaction through one gateway — one choke point that authenticates, validates, budgets, and logs. Edge enforcement pushes checks into each application, next to the context that application uniquely has.
The central gateway’s virtues are consistency and evidence. One implementation of the policy means one place to test it, one place to version it, and — the regulated-room bonus — one log that provably covers everything, because unregistered systems can’t authenticate at all. The inventory argument lands here: a control point that everything must pass through is also a census.
The edge’s virtue is knowledge. The application knows this user’s session, this workflow’s state, this tool’s blast radius. A central gateway checking “is this SQL query read-only” can’t know whether the requesting workflow should be reading that table for this customer. Push that check to the edge and it can.
The resolution isn’t a compromise, it’s a layering: the center enforces what must be universally true; the edge enforces what only it can know. Identity, model access, budgets, logging, data-classification limits — central, because a policy with per-team implementations is a policy with per-team bugs. Parameter binding to the authenticated session, per-tool authorization, workflow-state checks — edge, at the policy enforcement point where the action happens. The failure mode to refuse: letting a team argue their app should skip the center because they “already check at the edge.” The moment the center is optional it’s an inventory with holes in it, and you’re back to governing what you can’t see.
The latency budget: what validation may cost
Every validation layer costs milliseconds, and someone in the review will present that cost as decisive. Schema validation on the intent, an authorization-matrix lookup, a denylist parse of generated query language, output scanning — tens of milliseconds each, and they add up in the tail.
Two things are true at once. First: the costs are real, and a governance design that ignores them gets quietly bypassed by teams with latency SLOs — the goat path wins when the paved road is slow. Second: the costs are wildly asymmetric to what they insure. The user cannot perceive forty milliseconds inside a two-second generation. They will very much perceive the model refunding a stranger’s account.
So treat validation latency like any other performance budget: measure it, assign it, and spend it where the risk is. Cheap deterministic checks — schema, identity binding, allowlists — run on everything; they’re microseconds to low milliseconds and they carry most of the enforcement weight. Expensive checks — semantic output scanning, secondary review — are tiered by the risk classification of the tool being called: a T0 read-only lookup gets the cheap path, a T3 money-moving intent gets everything and can afford to, because actions that consequential are rare and nobody balks at a heavier gate in front of them. What you may not do is the thing latency pressure always suggests: sample. Validating ninety percent of interactions is enforcing none of them, because the attacker chooses which interaction they’re in.
Fail closed, fail open — and who signs
The guardrail is software. Software goes down. When the validation service is unreachable, the system must do one of two things: refuse to act until it’s back — fail closed — or proceed unguarded — fail open. There is no third option, and not deciding is deciding: whatever your code does on a timeout today is your actual policy.
Engineering instinct says fail closed, obviously — a security control that evaporates under load isn’t a control. And for anything consequential that’s right: in a regulated deployment, “the compliance checks were down so we kept issuing answers about loan terms” is a sentence you do not want to hear read back to you, slowly, by an examiner.
But fail-closed has a price someone must own: it converts a guardrail outage into a product outage. The chat goes dark; the agent stops mid-workflow; revenue-adjacent flows halt because a validation pod is crash-looping. That price lands on the business, which is exactly why the fail posture is a business decision, and engineering should refuse to make it alone. The right artifact is a short decision record, tiered like everything else: T2/T3 actions fail closed, full stop, with the revenue owner’s signature acknowledging the outage trade; T0 read-only paths may fail open with degraded-mode logging, with the risk owner’s signature acknowledging that trade. Then — because a control nobody tests is a belief — you test the failure: kill the validator in staging and verify the system actually does what the signatures say it does. The outage drill is the control.
Why deterministic checks beat a judge model
The seductive proposal, raised in every review since mid-2023: have a second LLM screen the first one’s output. It demos well. It scales linguistically. It’s the wrong tool for anything that matters, for one architectural reason: the judge fails the same way the defendant does. A judge model is probabilistic, prompt-injectable, and version-drifting — the exact failure classes you’re trying to contain. Stacking a second sample from the same distribution buys you correlation, not assurance. And in the regulated room it buys you nothing at all, because “a model checked it” is precisely the unexplainable, unattestable assurance the examiner won’t accept.
The discipline: anything security-relevant or compliance-relevant gets a deterministic check — parseable, versionable, testable, with a yes/no answer that two runs agree on. Schema conformance, identity binding, authorization, query-language safety, numeric-disclosure exactness: all deterministic, all cheap, all provable. The judge model has a real but junior role — ranking tone, flagging possible issues for humans, scoring qualities that have no crisp definition — advisory work, feeding dashboards and review queues. The rule that keeps the review honest is one sentence: a model’s opinion may inform a decision; it may never be the decision. The model proposes. It never disposes — not even about another model.
The pattern under all four
Look back and the four answers are one answer. Layer the universal center against the knowing edge. Spend latency where risk lives. Choose your failure posture per tier, on paper, with the owner’s name on it. Reserve judgment-by-model for judgments that don’t matter. Every one of these is the same move: decide deliberately, in advance, in writing, at the layer that can actually hold the decision — instead of letting defaults, timeouts, and demo-pressure decide for you.
Which is, in the end, what governance is. Not the PDF. The decided system.
Next in this series: the adverse-action problem — why fair-lending law quietly forbids certain agent architectures, and what “specific and accurate reasons” means when a statute is reading your stack trace.