A Hallucinated APR Is Not a Quality Bug

In consumer finance, the difference between an AI quality issue and a federal violation is which sentence the model got wrong. This opens a new series on governance where the regulator reads your architecture: the consumer-lending regulatory stack translated into engineering requirements, why a nonbank lender answers to forty-four states plus the CFPB, and how the enforcement layer you already built becomes the compliance evidence you were about to write by hand.

There is a class of company where an AI hallucination is not a quality bug. It is a federal violation with a docket number.

Say a consumer-lending assistant tells a customer their loan has no origination fee when it does. In most industries that’s an embarrassing error, a support ticket, a retraining item. In consumer finance it has a name: an unfair, deceptive, or abusive act or practice — UDAAP, under the Dodd-Frank Act — and the Consumer Financial Protection Bureau has collected nine-figure penalties for exactly that category of misstatement, made by humans, at scale. The regulator does not have a lower standard for software. A model that misdescribes a product term is deceptive marketing, no matter how apologetically it was generated.

This series is about what governance looks like when the room you’re presenting in contains an examiner. The first series built the argument that policy must be enforced in infrastructure or it’s a wish. That argument gets sharper in a regulated industry, because a second audience shows up: not just the attacker probing your enforcement, but the examiner reading your evidence. The good news — and the thesis of this article — is that they want the same thing. The architecture that stops the attacker is the evidence that satisfies the examiner, if you build it to emit that evidence as a byproduct.

Six consumer-finance statutes — UDAAP, ECOA/Reg B, TILA/Reg Z, FCRA, GLBA/Safeguards, and state law — mapped to three engineering artifacts: blocking evals, enforcement-layer rules, and pipeline/data rules, under the principle 'evidence as exhaust'.
The whole argument at a glance — save or share it.

The stack nobody shows you in the demo

Take a nonbank consumer lender as the worked example — an installment lender operating in forty-plus states. Not a bank: no OCC, no Federal Reserve as prudential supervisor. People hear “nonbank” and assume lighter regulation. What it actually means is different regulation: the CFPB has full supervisory authority, and every state where the lender holds a license examines it too. One AI feature, forty-five examiners.

Here is the stack, translated from legal citation into engineering requirement:

UDAAP (Dodd-Frank §§1031/1036) — the accuracy regime. Anything the AI says about products, terms, pricing, or process must be true against a source of record. Engineering translation: every customer-facing generation about product terms needs grounding in a governed source of truth, and your evaluation suite needs blocking test cases for term accuracy — not sampled, not spot-checked, blocking. A hallucinated APR fails the build.

ECOA / Regulation B — the fairness regime. Credit decisions can’t discriminate on protected bases, and adverse-action notices must give specific, accurate reasons. The CFPB has said in writing that complex models get no black-box exemption: “the algorithm decided” is not a legal reason for denial. Engineering translation: anything in the credit path needs bias testing across protected classes as a release gate, and needs to produce specific, accurate, individually correct reasons — a requirement that quietly rules out certain architectures entirely. More on that in a later part, because it deserves its own article.

TILA / Regulation Z — the arithmetic regime. Cost-of-credit disclosures — APR, finance charge, payment schedule — must be exact. Engineering translation: the model must never be the thing computing or restating these numbers. The calculation lives in tested, deterministic code; the model may present the result verbatim or point to it, never derive it. The LLM interprets; the code computes.

FCRA — the provenance regime. Consumer-report data has permissible-purpose rules, and decisions based on it trigger notice obligations. Engineering translation: bureau data inside your retrieval corpus, your eval datasets, and your traces is regulated cargo — it needs governed access, retention limits, and a provenance trail.

GLBA and the FTC Safeguards Rule — the data-security regime. This is the one engineers underestimate, because it reaches the tooling. Your evaluation datasets contain customer data. Your labeling workflow shows customer data to reviewers. Your traces store customer conversations. All of it is inside a written-information-security-program obligation. The eval infrastructure is not exempt from the controls just because it’s “internal.”

And the states. Rate caps, licensing conduct rules, mini-FTC acts, and — increasingly — state AI statutes with their own risk-assessment and disclosure demands, on staggered timelines, amended mid-flight. The same AI answer can be compliant in one state and a violation in the next. Jurisdiction is a runtime parameter, not a footnote.

One row per regulation, one gate per row

Written as prose, the stack is overwhelming. Written as an engineer would write it, it collapses into something familiar: a requirements table. Each regulation contributes rows to exactly three artifacts you already know how to build.

Blocking scenarios in the eval suite. UDAAP contributes term-accuracy cases. TILA contributes disclosure-exactness cases. The Military Lending Act contributes covered-borrower cases where the tolerance is zero. These are not aspirational metrics on a dashboard — they run like unit tests and a regression blocks the merge, exactly as the fourth article of the last series argued for security controls. The compliance version is the same machinery pointed at a different failure class.

Enforcement-layer rules. ECOA and TILA both reduce to the same architectural command: take the consequential computation out of the model. The policy-enforcement-point pattern — model proposes, deterministic layer disposes — turns out to be not just the secure design but the compliant one, because a deterministic layer can be tested, versioned, and explained to an examiner in a way a sampling distribution cannot.

Data-handling rules in the pipeline. GLBA and FCRA reduce to corpus governance: what may enter a knowledge base, who may see a trace, when an eval dataset must expire. If your ingestion refuses to violate the data manifest — refuses, in code — then your Safeguards Rule story writes itself.

Evidence as exhaust

Here is the part that changes the economics. Most organizations treat compliance evidence as a parallel work stream: engineers build the system, then someone assembles a binder describing it. The binder is expensive, it drifts from reality the day it’s printed, and examiners have learned to distrust it.

But if governance is enforced in infrastructure, the evidence already exists — as logs, as eval results, as version history. The gateway that authenticates every model call is the access-control evidence. The eval suite that blocks on a disclosure regression is the testing evidence, with timestamps. The append-only audit trail is the monitoring evidence. The manifest your build refuses to violate is the data-governance evidence. You don’t write the binder; you query for it.

This is the same inversion the whole first series argued, wearing a suit. A policy that lives in a PDF loses to an attacker and bores an examiner. A policy bound to a control point defeats the attacker and — properly logged — delights the examiner, because examiners spend their lives reading claims and almost never get handed proof.

What this costs, honestly

None of this is free, and the vendors selling “compliance AI” as a checkbox are lying to you in a way that is itself arguably deceptive. The eval suite for a regulated deployment is severalfold larger than for an unregulated one, because every regulation contributes scenario classes and the blocking ones need maintenance forever. Jurisdiction-awareness contaminates your whole retrieval design. And the deterministic-layer requirement means some architectures that demo beautifully — end-to-end agentic flows where the model touches the money — are simply not buildable here, no matter how well they benchmark.

That’s the honest trade. In exchange, you get the only deployment posture that survives contact with both of your adversarial readers. The attacker doesn’t care what your policy says; the examiner doesn’t care what your model intended. Both are defeated by the same thing: a system where the safe behavior is enforced, tested, and logged — and where a hallucinated APR is caught by a blocking eval, in CI, before it ever becomes someone’s docket number.

Next in this series: the trade-offs you’ll be asked to defend in the design review — central versus edge enforcement, the latency price of validation, and why fail-closed is a business decision someone other than you should sign.

Originally written by Paul Vilevac for Bleenq. bleenq.com is the canonical home for this piece.

← All articles