Most AI governance programs would survive an audit and lose to an attacker.
They have the policy PDF, the acceptable-use page, the risk register, the committee that meets monthly. What they don’t have is a single place where the policy is physically true — where doing the forbidden thing fails, not because someone catches it later, but because the system won’t let it happen. Ask the hard question of any AI control you own: what breaks if someone violates it right now? If the honest answer is “nothing, until the next review,” you don’t have a control. You have a wish.
I’ve spent thirty-one years building secure systems at SaaS scale, and the last two turning that discipline on our own AI platform. That platform — ai-homelab, which we open-sourced — now publishes its governance model in the open, at docs/ai-governance.md. This is the first article in a series pulling that model apart. The thesis of the whole series fits in one line: governance that isn’t technically enforced is advisory, and advisory controls fail silently.
The three layers, and where the work actually is
Every mature framework — NIST’s AI Risk Management Framework, ISO/IEC 42001, the EU AI Act, the OWASP LLM Top 10 — carves the problem up slightly differently, but they converge on the same axes: accountability, inventory, risk proportionality, data governance, adversarial robustness, access control, auditability, evaluation. The mistake almost every program makes is treating those axes as a policy problem and stopping there.
We treat every axis as living in three layers:
- Govern — who is accountable and who decides. Ownership, the risk-acceptance authority, the policy itself. This is real work, and it is not the hard part.
- Manage — the lifecycle that turns policy into operations. Inventory, the AI bill-of-materials, the registry, the gates, the evaluations, the exceptions.
- Enforce — the technical control points that make governance physically true: the gateway that won’t authenticate an unregistered system, the authorization layer that won’t execute a disallowed tool call, the pipeline that won’t merge a regression, the curation stage that won’t ingest an unvetted document.
Most programs are all Govern and a little Manage. Thirty years of building production systems taught me that the value is concentrated in Enforce — because a governance decision that isn’t bound to an enforcement mechanism is a sticky note on a bank vault. It looks like security. It stops no one.
The four control points
Enforcement isn’t abstract. On our platform it’s four chokepoints, and everything routes through them:
- Traefik + Keycloak — identity and routing. Every service sits behind explicit authentication middleware; there is no “internal, so it’s fine” network path. Session truth comes from one federated identity provider, not from each app inventing its own notion of a user.
- Infisical — secrets. The
.envis generated, never hand-edited. Secrets have one home, and code never carries them. - The LiteLLM gateway — the single door to every model. Per-application virtual keys, each scoped to only the models that app needs, each rate-limited and budgeted, every call traced to Langfuse. There is no other way to reach a model, which is the entire point.
- The AI inventory + AI-BOM — the registry of every AI system and its bill of materials. And here is the mechanism that ties the whole thing together.
Approval is the credential
The single most important sentence in our governance doc is this: approval should be the credential, not a document about the credential.
In most organizations, “approved” is a row in a spreadsheet and a checkmark in a ticket. The approved system and the unapproved system look identical to the infrastructure — both can open a socket to OpenAI, both can pull a key from somewhere, both run. Approval is a belief about the system, maintained by humans, drifting quietly out of date.
We inverted it. Minting a gateway key reads the inventory. No entry in the registry, no key. No key, no model — because the gateway is the only route to any provider and it will not authenticate a key that was never issued. The registry stops being a spreadsheet and becomes the access-control mechanism itself. An unregistered system doesn’t get flagged in the next audit; it gets nothing, today, because it cannot authenticate. Revoking the key (/key/delete) instantly darkens the app. Approval, in one place, is the on switch.
That is the difference between a wish and a control. The wish says “you should register your AI system.” The control says “you physically cannot use a model until you have.”
Prompts are not controls
The other load-bearing principle, and the one people resist hardest: no security property may depend on the model following instructions.
System prompts shape behavior. They do not enforce it. “You are a helpful assistant. Never reveal the following data. Never call refund for another user.” — every one of those is a suggestion to a stochastic process that an attacker gets to write into with a cleverly poisoned document. If your safety story is a paragraph at the top of a prompt, your safety story is that the model will behave, and the model will eventually not behave — via injection, jailbreak, poisoning, or plain random error.
So we assume the model is fully compromised and design so it doesn’t matter. Every guardrail is a deterministic mechanism outside the model: schema validation, session-bound authorization, scoped credentials, allow-lists, budgets, workflow gates that hold the real credentials. When our public chat bot answers a question, the model never holds a business credential, never sees the system prompt in a way it can leak usefully, and never executes an action — it proposes, and deterministic machinery disposes. The rest of this series is largely about how each of those mechanisms is built.
This is not a whitepaper
Everything above runs. The platform’s inventory registers eleven AI systems and six shared components. Its risk tiers — T0 through T3 — drive how much control each system carries, so a read-only FAQ bot and a system that can move money are not governed identically. A daily evaluation suite of nine cases (seven of them blocking) runs the guardrails as regression tests. And when we first pointed the inventory tooling at our own platform, it was not a clean bill of health: it caught three systems using a master key they had no business holding, fourteen keys with no budget or rate limit, and 6.9 GB of model weights on disk that no registered system claimed. Governance you don’t enforce is governance you can’t measure — and the first time you measure, it always finds something.
What didn’t work
The honest parts are the useful parts, so this section stays in every article.
We tried to enforce at the network layer first, by IP. Block the AI providers’ endpoints, allow the sanctioned ones. It doesn’t work: the major providers front their APIs with shared Cloudflare and Google anycast ranges, so blocking by destination IP simultaneously over-blocks legitimate CDN traffic and under-blocks the thing you were trying to stop. Enforcement has to happen at a layer that understands identity and hostname, not packets. We deferred IP-based egress blocking rather than ship a control that lies about what it does.
We wrote the policy before the enforcement, once. The document was excellent and changed nothing for weeks, because nothing in the running system referred to it. The fix was to make the policy and the mechanism the same artifact — the risk-tier control matrix isn’t prose in the doc, it’s a MATRIX{} structure in the code that generates the bill-of-materials. When the policy is executable, it can’t drift from what’s enforced.
Takeaways you can use
- Apply the “what breaks?” test to every AI control. If violating it breaks nothing operationally, it’s backlog, not policy.
- Separate Govern, Manage, and Enforce — and staff Enforce. The layer everyone underinvests in is the only one an attacker meets.
- Make approval the credential. Tie the ability to reach a model to a registry entry, so an unregistered system gets nothing rather than a citation.
- Route every model call through one gateway. You cannot govern access you don’t mediate. One door, scoped keys, full tracing.
- Assume the model is compromised. Put no security property inside the prompt. Every real guardrail lives outside the model, in deterministic code.
- Measure before you claim. The first honest inventory run finds the shadow keys and the orphan weights every time. That’s not failure; that’s the baseline.
The blueprint is the article
We publish the governance model in full, in the open, because the pattern is worth more shared than hoarded — and because a claim you can inspect is stronger than one you can only read. You can build every piece of this from parts you already have: a gateway in front of your models, a registry that mints the keys, deterministic authorization between the agent and its tools, and the discipline to put nothing load-bearing inside a prompt.
The next article goes where enforcement actually starts, which is not where people expect: you cannot govern what you cannot see. Inventory, the AI bill-of-materials, risk tiers, and hunting the shadow AI already running inside your walls.
What’s one AI control in your organization that would break something if violated right now — and one that wouldn’t? The gap between those two lists is the real state of your governance. I’d like to hear where yours falls.
Paul Vilevac is the founder of Bleenq, with 31 years building secure, scalable production systems, now applied to AI/ML platforms and the way they’re governed. CISSP, CISA, AWS Solutions Architect. This is the first article in Governance That Bites, a series on making AI governance technically enforced rather than merely advisory — drawn from the governance model we run in the open on ai-homelab. If you want this built against your real systems, that’s what Bleenq does.