Key takeaways
- An interrupted agent can be given 4 different answers, not one: approve the call as proposed, edit its arguments, reject it with feedback, or respond in place of the tool. Each resumes the run differently.
- An interrupt without persistence is a crash. LangChain's documentation is explicit that a checkpointer is mandatory, and that production deployments need a durable store rather than the in-memory one used in tests.
- Reliability, not capability, is the case for a gate: on τ-bench retail, agents that solved under 50% of tasks once fell below 25% at pass^8, the share that succeed on all 8 identical attempts.
- Editing arguments is not free. LangChain warns that significant modifications can make the model re-evaluate its approach, so an edit can change the plan rather than just the call.
- Our recommendation: gate on irreversibility, not on confidence, which puts a gate on roughly 4 of the 6 action classes below and none on reads or reversible writes. A gate on every call trains reviewers to approve without reading.
Most human-in-the-loop implementations answer one question: may this tool call proceed. LangChain’s documentation describes four answers, and the other three are where the design actually lives.
The reason to care is reliability rather than capability. On τ-bench’s retail domain, agents solving under half of tasks on a single attempt fall below 25% at pass^8, the share of tasks that succeed on all eight identical tries. A gate is how you stop paying for that variance on the steps where variance is expensive.
What can a person actually return to a paused agent?
Four things, and they resume the run in four different states.
| Decision | What executes | What the agent learns | Use it when |
|---|---|---|---|
| Approve | The tool, with the original arguments | Nothing new | The call is right and the action is irreversible |
| Edit | The tool, with modified arguments | That its arguments were changed | The intent is right and a value is wrong |
| Reject | Nothing | Rejection feedback, and it replans | The intent itself is wrong |
| Respond | Nothing; your text becomes the tool result | An answer it could not look up | The tool was an "ask the user" tool |
The distinction between reject and respond is the one most often collapsed. Reject sends the agent back to planning with feedback; respond hands it a fact and leaves the plan intact. Collapsing them means every missing piece of information triggers a replan, which is the expensive path.
Why does an interrupt need a checkpoint behind it?
Because a pause that does not survive the process is a crash with better manners.
LangChain’s documentation is unambiguous here: a checkpointer must be configured to persist graph state across interrupts, and a thread_id must be passed to associate the execution with a conversation. The docs name durable stores such as AsyncPostgresSaver and MongoDBSaver for production, and the in-memory saver for testing only.
This is the part that fails quietly. The in-memory store works in every demo, because the demo resumes within seconds inside the same process. It fails the first time a reviewer takes 20 minutes to look at the queue and the container scales to zero in between. What has to be in the state to survive that is the same problem as deciding what belongs in an agent checkpoint, and the answer does not change because a human caused the pause rather than a deploy.
The side effect that already happened
Only approved or edited calls execute on resume. That is a statement about what runs after the pause, not about what ran before it.
If the agent called three tools and the gate sits on the fourth, the first three have already taken effect. A gate placed after the model has committed to an action but before the action executes is a gate. A gate placed after the action is an audit log.
Which actions deserve a gate?
The irreversible ones, on the same test that governs whether a failed step should be retried or escalated: what does being wrong cost, and can it be undone.
| Action class | Gate | Decision types worth offering | Unattended attempts | Target decision latency |
|---|---|---|---|---|
| Read, query, search | No | None | 3 | n/a |
| Reversible write, draft, staging | No | None | 2 | n/a |
| External send, message, email | Yes | Approve, edit, reject | 0 | 30 min |
| Money movement | Yes | Approve, reject | 0 | 15 min |
| Deletion, publication | Yes | Approve, reject | 0 | 15 min |
| Missing fact the agent cannot look up | Yes | Respond | 0 | 60 min |
Note that money movement gets approve and reject but not edit. An amount a reviewer had to correct is evidence the agent misread the situation, and the reviewer who fixes the number is carrying a plan they did not make.
The failure mode of gating everything is not cost, it is habituation. A reviewer who approves 200 calls a day and has never seen a wrong one stops reading at around the point the gate would first have earned its keep.
What does editing a tool call do to the plan?
More than changing a value, sometimes.
LangChain’s documentation carries an explicit warning: significant modifications to the original arguments may cause the model to re-evaluate its approach, and even to run the tool more than once. The edited call executes, but the agent sees arguments it did not choose, and a model that notices the discrepancy may conclude its plan was wrong.
That makes edit the highest-variance of the four decisions. A one-character correction behaves like approve. A rewritten argument behaves somewhere between approve and reject, and which one you get is not under your control. For anything where the plan matters more than the single call, reject with feedback is the more predictable instrument, because replanning is what you asked for rather than what you triggered.
How would you know the gate is working?
Three counters, and none of them is approval rate.
- Rejection rate per action class. A class gated for six months with zero rejections is a class that does not need a gate. A class rejected on a fifth of calls needs a better agent, not a faster reviewer.
- Time from interrupt to decision, at the 95th percentile. This is the number that tells you whether your checkpoint store is being asked to hold state for seconds or for hours, and the in-memory saver only survives the first case.
- Edits that were followed by a replan. If editing regularly changes the agent’s approach, the team is using edit where it means reject, and the τ-bench reliability numbers say that difference compounds across repeated runs.
The useful question for a review is not how many calls were approved. Ask: which gated action class has never once been rejected? That gate is costing latency and reviewer attention and returning nothing.
Frequently asked questions
What is human-in-the-loop in an AI agent?
A deliberate pause at a chosen point in the run where a person decides what happens next. In LangChain's implementation the person can approve the proposed tool call, edit its arguments, reject it with feedback, or respond directly in place of the tool.
Do you need a checkpointer for human-in-the-loop agents?
Yes. LangChain's documentation states a checkpointer must be configured to persist graph state across interrupts, with a thread_id tying the execution to a conversation. Without it the pause is indistinguishable from a crash.
Which agent actions should require human approval?
The irreversible ones: money movement, deletion, external sends and publishing. Gating reversible steps adds latency and review fatigue without reducing the cost of being wrong, because a reversible mistake can simply be undone.
What happens to side effects when an agent is interrupted mid-run?
Only approved or edited calls execute on resume, but any tool that already ran before the interrupt has already had its effect. This is why the gate must sit before the side effect, not after the model has decided to take it.
Method and sources
- LangChain, Human-in-the-loop documentation, consulted for the four decision types, the mandatory checkpointer and thread_id, the named durable stores, and the warning that significant argument edits can make the model re-evaluate its approach.
- τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains, Yao, Shinn, Razavi and Narasimhan, arXiv:2406.12045. Source of pass^k as published, of the under-50% single-attempt result and of pass^8 below 25% in the retail domain.
- Anthropic, Effective context engineering for AI agents, consulted for how a paused and resumed agent reconstructs state from summaries and notes rather than from the original trace.
- The gating policy recommended here is our editorial judgement, not a measured result. We report no approval rates, review times or error rates of our own, because we have not measured them.