Key takeaways
- A tool call that times out after dispatch has often already happened. In a simulated environment, a retry-only agent duplicated the side effect in 20%, 44% and 72% of runs at low, medium and high fault levels on a customer-activation task.
- Checking whether the action completed before retrying cut those duplicates to 0%, 16% and 20% on the same task, with task success at 100% across all three fault levels.
- An idempotency key only helps if the agent reuses the same key for the same logical operation. A key regenerated on every retry deduplicates nothing.
- Cap retries per logical operation. The study allowed 1 retry, issued only when a postcondition check reported that the action had not completed.
- Verification did most of the work. In the ablation, verify-only cut duplicates from about 42% to 20%, and the full wrapper reached about 28%, so do not treat the key as a substitute for checking state.
In a simulation submitted to arXiv on 31 July 2026, an agent told to retry any failed tool call repeated a customer-activation action in 72% of runs at the highest fault level. At the lowest level it still happened in 20%.
The cause is not the model. A timeout says nothing about whether the call landed, and the loop that retries it has no way to find out. This is the point at which a card gets charged twice or a customer gets two welcome messages.
Why does a timeout make an agent do the same thing twice?
Because “no response” is not “no effect”.
Mansoor, Phadke and Rana’s paper (arXiv:2608.02645) describes non-atomic behaviour: a call times out after dispatch, becomes visible late, succeeds partly, or conflicts with another write. Agent frameworks usually treat a tool call as all-or-nothing and return success or failure. A timeout becomes “failure”, the model sees it, and the model does the obvious thing and calls again.
Nothing in that loop asks the system of record whether the first call went through. The retry is correct when the first call truly failed and a duplicate when it did not, and the loop cannot tell which.
This is a different failure from a malformed call. A schema error is visible in the response and can be repaired. A timeout after dispatch has no response to inspect.
How often does a retry-only agent duplicate the action?
Often, and more often as the environment gets worse.
The study ran 25 episodes per method for each of 2 tasks at 3 fault levels, 300 agent runs in total, against a simulated tool environment with a fixed seed of 42. The baseline prompt told the agent to retry failed calls directly and to omit idempotency keys. The wrapped agent checked the postcondition before retrying.
| Task and fault level | Baseline duplicates | Wrapped duplicates | Baseline success | Wrapped success |
|---|---|---|---|---|
| activate_customer, low | 20% | 0% | 92% | 100% |
| activate_customer, medium | 44% | 16% | not reported here | 100% |
| activate_customer, high | 72% | 20% | 64% | 100% |
| record_invoice, low | 32% | 0% | 100% | 100% |
| record_invoice, medium | 52% | 16% | 96% | 96% |
| record_invoice, high | 76% | 20% | 96% | 100% |
Read the invoice rows carefully. The baseline succeeded in 96% to 100% of invoice runs and still duplicated the action in a third to three quarters of them. Task success did not reveal the problem, because a duplicated write still satisfies “an invoice exists”.
That is why per-run success rate alone cannot catch this class of failure.
What does an idempotency key actually guarantee?
That a repeated request with the same key produces one effect, provided the server implements it and the client reuses the key.
Stripe documents the mechanism clearly. It saves the status code and body of the first request for a key, whether it succeeded or failed, and returns that same result to later requests with the key, including 500 errors. Keys can be up to 255 characters, may be pruned once they are at least 24 hours old, and a reused key with different parameters returns an error. All POST requests accept keys; GET and DELETE do not need them.
Three consequences for an agent:
- The server must support it. If the downstream API has no idempotency support, a key is a string nobody reads.
- The key must be stable across retries. A key generated by the model on each attempt is a new key each time.
- Validation failures are not saved. Stripe does not store a result when parameters fail validation, so those requests can be retried freely.
The paper makes the same point from the other side. When the API has no key support, the verifier still guards against duplicates, but a small window remains where concurrent retries can slip through.
Is the key enough, or does the agent also need to check state?
The key is not enough on its own. Checking state did most of the work in the ablation.
The authors compared three variants on the activate_customer task at medium faults, in a separate configuration whose absolute rates they say are not comparable with the main table. Relative to each other:
| Method | What it adds | Task success | Duplicate actions |
|---|---|---|---|
| Retry-only baseline | Nothing: retries directly, no key | about 58% | about 42% |
| Verify-only | Postcondition check before retry | about 80% | about 20% |
| Full method | Check, verify-before-retry and idempotency keys | about 72% | about 28% |
The full method did not beat verify-only in this ablation, and the authors attribute most of the improvement over the baseline to verification. We read that as a caution against shipping a key and calling the problem closed. It is a single task, a single model and 25 episodes, so it is a direction and not a ranking.
What should the tool layer do on a timeout?
Take the retry decision away from the model and give it to the tool wrapper. The paper resolves ambiguous states inside the wrapper and does not surface them to the model. The order below is our editorial recommendation, built on that design.
- Classify the tool. Reads can be retried. Anything that writes, sends, charges or creates needs the next steps.
- Derive the key in code. Build it from run ID, step ID and target, so the same logical operation always produces the same string. This also survives a restart if the checkpoint stores the key.
- On timeout, query the system of record for the postcondition before anything else: does the customer exist, is the invoice recorded.
- Retry once, and only if the check says it did not happen. The study capped retries at 1 per logical operation.
- Return a clear state to the model: completed, not completed, or unknown. Escalate unknown to a person instead of guessing.
This sits inside the same budget logic as capping what an orchestrator may spend: a retry is spend, and a duplicated charge is spend you cannot recover.
The question to take into your next design review is: for each tool that writes, what does the wrapper do on a timeout, and where is the key derived? If the honest answer is “the model retries”, the 20% to 76% range above is what you should expect to see in a bad hour.
Frequently asked questions
Why does my AI agent do the same thing twice after a timeout?
Because a timeout means no response arrived, not that no action happened. The agent loop sees a failure, the model retries, and nothing in the loop knows whether the first call took effect.
What is an idempotency key for an agent tool call?
A client-generated string that lets the server recognise a repeat of the same request and return the first result instead of acting again. Stripe, for example, accepts keys up to 255 characters on POST requests.
Should the model generate the idempotency key?
No. Derive it in the tool layer from the logical operation, such as the run ID plus step ID plus target. A model-generated key can change between retries, and then it deduplicates nothing.
How many times should an agent retry a tool call?
Once per logical operation is a defensible default for side-effecting calls, and only after checking that the first attempt did not complete. Reads can be retried more freely because repeating them changes nothing.
Method and sources
- Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures, Mansoor, Phadke and Rana, arXiv:2608.02645, submitted 31 July 2026. Source of every duplicate-rate and success-rate figure in this text. Preprint, simulated environment, 2 tasks, 3 fault levels, 25 episodes per method and setting (300 runs in the main comparison); one model, temperature 0.0.
- Stripe API reference, Idempotent requests, read 7 October 2026. Source of the 255-character key limit, the 24-hour minimum before pruning, the saved-result behaviour including 500 responses, and the parameter-mismatch error.
- The retry ceiling, the key derivation recipe and the order of checks in the last section are our editorial judgement. We ran no experiment of our own and publish no duplicate rates of our own.