A Control Nobody Tests Is a Belief

Every control in this series shares a failure mode: it can quietly stop working, and nobody would know.

The session-identity check gets refactored and a code path skips it. A prompt edit reopens an injection the suite used to catch. A key gets minted by hand, outside the flow that reads the inventory. None of these throws an error. None of them shows up in a dashboard by default. The control still exists, in the sense that the code is still there — it just no longer does anything. A control nobody tests is a belief. And a belief, in security, is a liability wearing the costume of a safeguard.

The first three articles built the enforcement layer. This one is about keeping it true over time, which turns out to be a different and harder problem than building it once.

Guardrails are code, and code is tested

The reframe that makes this tractable: prompts, tool contracts, and curation rules are code. Not configuration, not content — code, with all the discipline that word carries. They’re versioned, they’re hashed, they’re reviewed, and — the part that matters here — they’re tested, automatically, on every change.

So the artifacts from the earlier articles all live in the repo as versioned code:

• Prompts and templates — hashed; a change is a diff someone reviews.
• Tool contracts — the JSON schemas and authorization mappings; a changed contract re-enters its review gate.
• Curation rules — what’s allowed into a knowledge corpus, deterministic and inspectable.
• The risk-tier control matrix — not prose in a policy, a MATRIX{} structure in the code that generates the bill-of-materials.

When any of these changes, the same thing happens that happens when you change a function: the test suite runs, and if it regresses, the merge is blocked.

The evaluation suite runs like unit tests

The heart of it is a security evaluation suite that runs daily and also on any change to a governance-relevant artifact. On our platform it’s nine cases, seven of them blocking — meaning a failure doesn’t warn, it stops the pipeline.

And the cases are negative tests. This is the detail people skip and it’s the one that matters. It’s not enough to check that the system does the right thing on good input; you have to prove it refuses the bad. So the suite asks the system to do forbidden things and asserts that it can’t:

• A text-to-query path is asked, through an injected instruction, to escape into a write verb — and the guard must refuse it. The model, handed a “delete everything” instruction, will happily generate the destructive query; the deterministic guard downstream rejects it, and the test asserts the rejection.
• A key-scoping violation — a system reaching for a model it isn’t scoped to — must fail.
• The injection variants from the previous article — the cross-account refund, the encoded payload, the schema-exfiltration jailbreak — must all be denied, with the denial traced and alerted.

The passing criterion, every time, is that the control held — not that the model chose well. A regression is a previously-denied intent now executing, or evaluation accuracy dropping below baseline. Either one auto-blocks the merge, exactly like a failing unit test, because that’s exactly what it is.

The rest of CI: supply chain and secrets

The AI-specific evals sit on top of the ordinary hardening every pipeline should have, and these are lifted straight from what we run:

• Secret scanning on every push and pull request, and locally as a pre-commit hook. Any verified secret fails the pipeline and blocks the merge — no override in the pipeline itself. The only path past is a security-approved, time-boxed exception recorded with the risk function. Detected-and-rotated is the remediation; detected-and-merged is not a permitted outcome.
• Dependency scanning that blocks on critical, actively-exploited vulnerabilities in direct dependencies.
• Container scanning that blocks on critical image findings — images pinned by digest, no mutable tags.
• Static analysis that blocks on new high-severity findings against a baseline.

And the bill-of-materials earns its keep downstream: it’s the precondition for a production key, it’s what a nightly job matches against fresh model and CVE advisories to open tickets with tier-based SLAs, and it’s the provenance record that answers audit questions from history instead of memory.

Enforcement as cron: the jobs that fail loudly

Testing catches regressions at change time. But governance also drifts between changes — a key minted out-of-band, weights that appear on disk, a trace that can’t be attributed to a system. So enforcement also runs on a schedule, and every scheduled check is built to fail loudly:

• Weekly — regenerate the bill-of-materials against the live platform and flag inventory drift, shadow credentials, and unattributed traces. Check container-to-provider egress paths. Verify model-weight integrity and hunt unregistered weights.
• Daily — harvest the audit record before its upstream retention window expires (traces are kept 13 months as hashes, well past the tracer’s own 14-day TTL). Run the evaluation suite against production configuration.

Each alert flows to on-call with a dead-man’s check behind it — so the absence of a job running is itself an alert. A monitoring job that silently dies is the most dangerous kind of failure, because it looks exactly like “all clear.” The dead-man check is what makes “no news” mean something.

The honest scorecard

Here’s the part I’m most sure about, from thirty-one years of watching security programs succeed and fail: the governance document has to mark its own unfinished work, honestly, in the same place it claims its wins.

Ours does. It carries a live posture that separates what’s fully operating from what’s partial or deliberately deferred. Fully operating: accountability, inventory and lifecycle, executable risk classification, corpus-manifest data governance, the attributable audit trail, fifteen AI-specific alert rules. Partial or closed-by-decision, stated as such: there’s no input-side injection classifier yet (deferred on purpose — the deterministic downstream controls are the real defense); harm-category refusal testing is still open and tracked by ticket number; and shadow-AI detection runs while blocking is deferred, because IP-blocking providers who share anycast ranges would break legitimate traffic.

A scorecard that only lists wins is marketing. A governance model earns trust by being inspectable including where it falls short — because the alternative, a document that claims completion it hasn’t earned, is the most dangerous artifact in the whole program. It’s the one that makes everyone stop looking.

What didn’t work

We wrote evals that only tested the happy path. The first suite confirmed the system gave good answers to good questions, went green, and felt like coverage. It tested nothing that mattered. An eval that never asks the system to do something forbidden cannot tell you whether it would. The rewrite made every security-relevant case a negative test — prove the refusal — and only then did the suite start catching real regressions.

We let one scheduled check fail silently for longer than I’ll admit. The job errored, stopped producing output, and because “no output” looked identical to “nothing wrong,” nobody noticed. That’s the origin of the dead-man’s check on every scheduled job now. If a control runs on a timer, you have to monitor the timer, or you’re one silent cron failure away from believing you’re protected when you stopped being protected weeks ago.

Takeaways you can use

1. Treat prompts, tool contracts, and curation rules as versioned code — reviewed, hashed, and tested on every change.
2. Make your evals negative tests. Prove the system refuses the forbidden thing. Happy-path evals are a comfort, not a control.
3. Block the merge on a governance regression, exactly like a failing unit test — a previously-denied action now executing is a build break.
4. Run enforcement on a schedule, and put a dead-man’s check on every job. A monitoring job that silently dies looks exactly like all-clear.
5. Keep an honest scorecard. Mark the deferred and unfinished items in the same document that claims the wins. Inspectability including the gaps is what earns trust.
6. Remember that controls decay. The one you built and never tested again is, functionally, a control you don’t have.

The end of the series, and the through-line

Four articles, one assumption underneath all of them: the model will eventually do the wrong thing — through injection, jailbreak, poisoning, or plain stochastic error. Governance that survives that moment isn’t a policy; it’s a set of deterministic mechanisms that hold when the model fails, plus the discipline to keep proving they still hold. Approval is the credential. You can’t govern what you can’t see. The model proposes and the workflow disposes. And a control nobody tests is a belief.

We publish the whole model in the open — the governance standard and the platform that runs it — because a governance claim you can inspect is worth more than one you can only read, and because most of the hard-won parts here compose from tools you already have.

If you take one thing from the series, take the test at the top of every control: what breaks if someone violates this right now? Run it against your own AI systems this week. The gaps you find are the actual work — and if you’d rather not do that work alone, that’s what we do.

Which of your AI controls would fail the “prove it refuses” test — the one where passing requires the model to misbehave and the control to hold anyway? That’s the honest place to start. I’d like to hear what you find.

---

Paul Vilevac is the founder of Bleenq, with 31 years building secure, scalable production systems, now applied to AI/ML platforms and the way they’re governed. CISSP, CISA, AWS Solutions Architect. This is the fourth and final article in Governance That Bites, a series on making AI governance technically enforced rather than merely advisory — drawn from the governance model we run in the open on ai-homelab. If you want this built against your real systems, that’s what Bleenq does.