Human-in-the-loop checkpoints are one of the most commonly promised controls in AI governance, and one of the most commonly hollow. The promise is simple: before a model's output does anything consequential, a person looks at it and decides. In practice the checkpoint often exists as an interface element rather than a control. There is a button, someone clicks it, and the audit log records a human decision where very little deciding took place.
We have seen this shape before, long before anyone attached it to AI. Identity and access work is full of approval gates built with good intentions and little else. An access request routes to a manager who approves everything because the request form tells them nothing useful. A quarterly access review lands as a spreadsheet with thousands of rows, and it comes back fully certified in an afternoon. A privileged session gets recorded, and nobody ever watches the recording. Each of these passes an audit. None of them would stop a bad decision, because none of them were designed around the question of whether the reviewer could realistically say no.
Why checkpoints go hollow
The failure is rarely the reviewer's fault. It comes from treating human review as a governance feature you add at the end rather than a workload you have to budget for. A checkpoint needs three things to be real: enough context on the screen for the reviewer to judge the specific action, enough time relative to the volume of requests, and a meaningful consequence if they approve something they should have caught. Remove any one of those and the checkpoint degrades into acknowledgment. AI systems make this worse in a predictable way, since they can generate requests faster than any person can evaluate them, and the output usually looks plausible enough to approve on sight.
Borrowing what PAM already learned
Privileged access management has a useful answer here, because it faced the same problem with human administrators. The mature pattern is not to review everything equally. It is to scope the standing permissions down so most actions need no approval at all, then put a genuine gate in front of the small set of actions that are irreversible or high-impact. The gate shows the reviewer exactly what will happen, is time-boxed, and is logged with the reason as well as the result. Applied to an AI agent, the same logic means deciding up front which tool calls, writes, or deployments require a person, and making those rare enough that each one gets real attention.
It also means measuring the checkpoint itself, not just its existence. An approval step with a near-perfect approval rate is worth a closer look, in the same way an access review with zero revocations usually signals a rubber stamp rather than a clean environment. Rejections, escalations, and requests sent back for more context are evidence that the reviewer is actually exercising judgment. Their total absence is a warning sign worth investigating before an auditor or an incident finds it first.
None of this argues against human oversight of AI. It argues for designing the oversight with the same care as the system it governs. We treat AI governance as an access-control problem because the actor on the other side of the gate, whether a person, a service account, or an AI agent, needs a checkpoint someone can realistically refuse. A human in the loop who can only say yes is not a control. It is a signature.