Human-in-the-Loop Agents

Let the agent run alone for the safe parts and freeze before the risky call. A person reads the plan, then approves, denies, or fixes it.

Safe searches run at once while 12 risky calls freeze for review, so 10 approvals and 2 denials cost 18 minutes and stop one data loss.

Why Does This Exist?

An agent that can read is useful. An agent that can delete, pay, or publish is dangerous. One misread row can drop a table, one wrong address can email a client, one guessed parameter can spend a budget. Full autonomy hands those buttons to a model that still misreads.

Humans stay in the loop to hold those buttons. The agent does the dull work alone: searching, drafting, and formatting. It freezes only at the edge of harm and waits. Chain-of-thought agents write the Thought the person reviews, code execution agents supply the riskiest gated tool, agentic workflow evaluation scores whether the gates help, and emergent behavior is what those gates keep from running wild.

This page covers the freeze, review, and resume cycle. It does not set your risk policy for you. That choice depends on what irreversible means in your own system.

Basics first: a policy boundary is a list of tools or situations that need a person. If that is new, think of it as a locked drawer the agent may point at but never open alone. Everything below assumes safe calls run free and only listed calls freeze.

Think of It Like This

The assistant who may not sign

A capable assistant sorts expenses, drafts the budget, and looks up tax rules alone. The bank password stays with you.

When money must move, the assistant fills the transfer form, lays it on your desk, and waits. You sign, refuse, or cross out a line and hand it back. The assistant never signs for you.

Where it stops: paper waits patiently. Live systems do not. Prices shift, files move, and rows change while the form sits on your desk, so an approval must recheck the world before it runs.

How It Actually Works

Call the proposed call $a$ , which means the tool name plus its arguments. Call the saved state $s$ , which means the history and scratchpad frozen at the pause. Call the human reply $h$ , which means approve, deny, or edit. The cycle is $a$ checked, $s$ saved, $h$ pasted back, then resume.

Freeze only at the edge

The agent runs its normal loop until it emits an $a$ on the boundary list, such as deleting rows or sending mail. The framework saves $s$ , stops the run, and shows the Thought plus $a$ to a person. Safe searches never freeze, so the person sees few requests and reads each one.

What the person can do

Approve runs $a$ as written and pastes the tool result back. Deny blocks $a$ and pastes a refusal the agent must work around. Edit changes the arguments or adds a note, which lands in context as a correction to the trace. To the model it looks like a slow tool reply.

Worked example

One day: 200 tool calls, 12 flagged as risky.

Flag rate is $12 / 200 = 0.06$ , which is 6 percent. The person approves 10 and denies 2. One of the denied calls would have wiped recent rows, so the day stops one data loss.

Review cost is $12 \times 90 = 1080$ seconds, which is $1080 / 60 = 18$ minutes. Eighteen minutes buys back the one loss plus ten clean runs. If the boundary flagged 100 calls instead, the same 90 seconds each would cost 150 minutes, and approvals would turn into rubber stamps. That trade is why workflow evaluation tracks gate counts beside scores.

Use gates for irreversible, costly, or low-confidence calls. Leave the rest free, because every freeze spends attention you cannot get back.

Code

This gate uses only the standard library. Risky names freeze, the rest run, and the counts match the worked example:

1RISKY: set[str] = {"delete_rows", "send_email", "refund_charge"}2
3
4def needs_review(tool: str) -> bool:5    return tool in RISKY6
7
8calls: list[str] = ["search"] * 188 + ["delete_rows"] * 7 + ["send_email"] * 59flagged: list[str] = [c for c in calls if needs_review(c)]10print(len(calls))  # -> 20011print(len(flagged))  # -> 1212print(round(len(flagged) / len(calls), 3))  # -> 0.0613print(12 * 90 // 60)  # -> 18

Four prints, one policy. Two hundred calls, 12 frozen, a 0.06 flag rate, and 18 minutes of review. Approving 10 and denying 2 is then a human choice on top of these counts.

Watch Out For

Too many gates trains blind approval

The boundary flags half the calls, so the person faces a queue of 100 near-identical requests.

The symptom is fast approvals with no real reading, often in under five seconds each. One of them carries a wrong table name and sails through. Flag only irreversible or costly tools, batch low-risk reads to run free, and track median review time. When it falls, the gate is theatre and the list must shrink.

The world moves while you wait

The agent proposes deleting rows older than 30 days, the person approves the next morning, and the table has changed overnight.

The symptom is a correct approval that fails or harms on resume, because $s$ no longer matches reality. Revalidate arguments at resume time, re-fetch the rows the call touches, and fail closed with a fresh freeze when anything shifted. Never replay a day-old $a$ blind.

The Quick Version

Human-in-the-loop agents run safe calls free and freeze before risky tool calls for a person to judge.
A freeze saves state $s$ , shows the Thought plus the proposed call $a$ , and resumes on the human reply $h$ .
Approve runs it, deny forces a new plan, edit fixes the arguments in place.
Twelve flagged calls in 200 is a 0.06 rate, costs $12 \times 90 = 1080$ seconds or 18 minutes, and can stop one data loss.
Flagging too much buys rubber stamps, so keep gates for irreversible, costly, or unsure calls.
Recheck the world at resume, because tables and prices shift while approval waits.