The Choice Tax

Weeks into an engagement, I found undocumented AI tooling running inside the ops function — no access controls, the team pleased with the results, and not one person had asked whether the systems were authorized. That is downstream governance, which is to say no governance at all. The part that should arrest any operator: nobody was negligent. Everyone behaved reasonably. That is the condition under which the exposure compounds, because no one is hunting for a problem that has not announced itself — and this is exactly the kind that announces itself later, in a board risk review or the first technical audit after a deal closes.

That is the AI governance gap in its natural habitat: not a policy failure, not a vendor problem, but a structural absence so ordinary the company had mistaken it for the way things are. The obvious answer to it is wrong, the sophisticated answer is also wrong, and the right one is what almost nobody funds.

Causality, not choice

Most leaders treat AI governance as a stream of choices. Should we approve this output. Should a human review this decision. Each question is a gate, and governance becomes the discipline of standing at each one and choosing well. Call this the Choice model. For anyone walking into a company with deployed systems, it is the most expensive model available, and the default.

The distinction I want to draw is between Choice and Causality — mine, since I have not seen it named cleanly for what it is. Governance is not a stream of decisions made at the moment of output. It is causal architecture: the access a system has, the data it can reach, the decision rights it carries, the way it is built before it ever runs. You do not govern an AI by choosing better at the gate; you govern it by engineering the causes, so that by the time any output exists, the set of possible outputs was already bounded by structure. A caveat I will not bury, because a sharp diligence team will reach for it: upstream architecture bounds the space of possible outputs — it does not determine which one the model produces inside an authorized envelope. That residual behavioral risk is real and second-order; the first-order risk is the envelope, and almost every company I encounter has left it undrawn.

The first wall: stand at the gate

Picture the AI’s output as a point on a river. The intuitive answer is to govern at that point: let the system produce a result, then inspect it, filter it, approve it. This is the gate model, the first answer almost everyone reaches for, because it feels like control.

It holds until load arrives. Every gate is a recurring cost that scales with output, and where these systems run at volume — high-throughput automated decisioning, agents chaining many actions per second — the human gate has only two failure modes. Either it becomes the bottleneck that throttles the machine to human pace, surrendering the reason you deployed it, or the system routes around it and the gate becomes theater. The first wall falls.

The second wall: tell the system to behave

The more sophisticated answer is to stop inspecting outputs one by one and govern behavior instead. Write the policy, set the guidelines, instruct the system on what responsible conduct looks like. This scales where a human reviewer cannot, and it still fails, for a structural reason rather than a matter of better wording: a behavioral instruction cannot intercept a system that is behaving correctly. Telling a system to behave responsibly is telling a financial system to be honest rather than building the controls that make dishonesty impossible.

A hallucinated answer is a performance problem. A correct answer delivered to the wrong person is a control failure, and no policy intercepts it, because the system did nothing wrong — it answered accurately, for someone who should never have been able to ask. The question stops being “Is the AI smart” and becomes “Is the AI authorized.” The policy wall falls quietly, while everyone admires the policy.

The third answer: build the structure

Now move the control upstream, to the source, so that only acceptable outputs are possible. The model physically cannot query the compensation table, because access is denied at the data layer — it never sees the data, so it can never leak it. No gate to bottleneck, no policy to route around, because the control acts on the cause. And unlike a behavioral instruction, engineered structure can be proved: a policy asks to be trusted, a structure can be shown.

One more boundary, stated before anyone has to ask for it: control is a gradient, not a clean binary, sitting anywhere from design-time access configuration to post-hoc audit. Downstream control is not worthless; governance value simply concentrates upstream, and almost every company I encounter is spending at the wrong end of the river. Operators who have run large integrations know the shape on sight — you pay once to build the structure, and it pays back on every transaction afterward. The pattern is older than AI; AI only raised the stakes.

A policy asks to be trusted, a structure can be shown.

One architecture, two ledgers

Here is the part that matters to whoever signs the check: the same upstream architecture that contains the risk also releases the value. One structure, two ledgers, depending on whose P&L reads it.

Start with the ledger that buys relief, because for most large institutions that is the one that keeps people awake. In a Fortune 500 environment, the board is not underwriting an exit; it is carrying continuity, regulatory exposure, and the standing question of whether the institution can prove control of its own systems. Picture the meeting every CISO has rehearsed: an auditor or a regulator asks how the institution knows its AI cannot reach what it should not. “We have a policy” fails in that room — it asks the room to trust the goodwill of every employee with a prompt window. “Access is denied at the data layer, here is the log of what each system was authorized to reach” holds. The upstream work that contained the leak is what lets the accountable executive sit through the audit instead of bracing for it. That is the relief — a record about control, not a story about hope.

The same architecture, read in the other ledger, is upside rather than containment. For a PE-backed or mid-market company the question is what a buyer pays, and the architecture works on two things worth keeping distinct, because conflating them is how this argument gets waved through. The first is the earnings base: a downstream oversight organization is a cost that scales with usage, re-coupling your cost curve to output volume forever, while upstream architecture is one-time capital reused across every interaction the system will ever have — an EBITDA argument, the integration economics again. The second is the multiple, and the mechanism is not cost; it is the removal of a diligence discount. Buyers already price the absence of documented access controls and decision rights — a controls gap is a standard finding in any technical diligence, AI or not, and it gets priced down. “Governed by architecture” removes that line item: an auditable record sitting in the data room where the buy-side team would otherwise log an exposure it cannot underwrite. One claim here is forward thesis, not settled fact, and I will mark it: machine speed is widening the gap between architected and oversight-based governance faster than most investment committees are modeling. The direction is legible, the magnitude still arriving, and the present-tense case stands without it.

One architecture, two payoffs — and in both ledgers, the company that spent downstream is the one holding the wall when it falls.

Accountability does not delegate

The machine has causality. It does not have agency: it produces outputs through structure, but it does not choose in any sense a court will recognize. No regulator drafting current AI law treats “the model did it” as a defense — the direction of travel relocates accountability to the deploying organization. Under the EU AI Act, the deployer who runs a high-risk system carries the competent-human-oversight obligation: the provider must build the system to enable oversight, but the organization that deploys it cannot delegate the oversight away.

For the accountable executive in an enterprise, the shape is plain. The board and the officer who answers for the system are the controlling position, and “the model did it” survives no better in front of their regulator than in front of anyone else’s.

For private equity the shape is sharper. When a GP takes control, the oversight obligation comes with it, and the standard containment instruments — indemnities, D&O, reps and warranties — do not reach the exposure that actually bites, which is timing and trust. A governance failure that surfaces in the data room at exit discounts the very asset the fund is selling, on the fund’s own clock; one that surfaces post-close in a flagship portfolio company is an LP-confidence event no indemnity reverses. The accountable act is upstream: the decision to deploy a system into a position you cannot oversee. You are not absolved because the output was opaque — you are answerable for having chosen the blindness. Even accountability relocates to the cause.

What this actually requires

In practice the upstream work comes down to three questions any operator should be able to answer on inheriting a company with live models.

First, a data-access audit before any AI capability scales: what can each deployed system reach, who authorized that access, by what mechanism. This is not an AI question; it is a controls question applied to AI.

Second, decision-right mapping. Which decisions has the company delegated to a model, and which has it reserved for a human. Most organizations never drew the line — the AI started deciding, the boundary was never set, the ambiguity got normalized. Drawing it, with documented rationale, is the governance act; everything else is consequence management.

Third, provable architecture rather than policy documents. The test is not whether you have an AI governance policy; it is whether you can demonstrate, to a regulator or a diligence team, that the system was constrained by structure and not by the model’s willingness to restrain itself. Policy is theater. Engineered control is evidence. The data room knows the difference.

Policy is theater. Engineered control is evidence. The data room knows the difference.

The choice tax

Most companies are paying it now. They spend downstream — gates, reviewers, policy binders — on governance that caps the return and still leaves the exposure intact, and they call the expense prudence. It is a tax on governing at the wrong end of the river.

The value was never at the gate. It was upstream, in the architecture, where the same structure that removes the risk releases the velocity — provable for the board, defensible against the regulator, priced into the data room, built once rather than paid for forever. The audit you would rather not fail sits downstream of it; so does the discount you would rather the buyer not find. The choices, it turns out, were never where the value lived.

So the question for anyone running an estate of live models: are you governing at the gate, or at the source? The honest answer is often “we have not looked yet” — which is itself the finding.

I work with operating teams to move AI governance upstream, into the architecture, before an audit or a data room forces the question. If that one is live in your portfolio, message me.