Shipping in companies

Where to put a human in a Claude agent loop

Human review belongs on irreversible writes and ambiguous intent, not on every token. Design the queue like a product, not a guilt valve.

Human-in-the-loop means a person approves, edits or rejects an agent action before it lands. It is not a synonym for "we read the Slack dump." If the queue is unstructured, people rubber-stamp and you have theatre.

Put humans where the expected cost of a mistake is high: money movement, customer messages, record updates in a system of record, anything regulated. Let the agent run freely on drafts, classifications and lookups that are cheap to undo.

A useful review queue

  • Shows the proposed tool call, not just the chat
  • Shows the evidence chunks or record ids
  • Has approve, edit, reject, and escalate
  • Writes the decision back into the trace

Without those, reviewers argue about tone. With them, they argue about the write. That is the job.

Ambiguous intent is a stop, not a guess

If the user could mean two tickets, two accounts or two legal entities, the agent should ask or queue, not pick. Guessing is how you get a fluent disaster. Pair this with narrow tool schemas.

Evals still apply

Reviewer behaviour needs cases too: "must queue," "must refuse," "may auto-send." Put them in the eval harness. Otherwise a prompt change will bypass the queue and nobody will notice until a customer does.

This is standard on our production builds and on regulated work. A checkpoint is a product surface. Design it, staff it, measure it.

Related notes