Key takeaways
- Agent memory lives in 4 places: the in-context summary, a note file, a retrieval index, and the prompt cache. Only the note file and the retrieval index survive a process restart.
- Re-reading a cached prefix costs 0.1x the base input price on most Claude models (0.05x on Opus 5.5) against 1x to resend it, but the cache expires after 5 minutes or 1 hour, so it is an accelerator and never a memory.
- Compaction is lossy by construction. Claude Code's approach keeps the 5 most recently accessed files after a compaction, which is a rule about recency, not about importance.
- A sub-agent returning a summary typically hands back 1,000 to 2,000 tokens instead of its whole trace. That is the compression ratio that makes delegation cheaper than doing the work in one context.
- Our rule: if losing it would force the run to start over, it does not belong in the context window. 0 durable facts should live only in a summary.
Every agent accumulates things it needs to remember, and there are only four places to put them. Most teams pick one, usually whichever their framework made easy, and then use it for facts it was never shaped to hold.
The distinction that matters is not recall quality. It is what survives when the process dies, because a container that scales to zero takes the context window with it and leaves the note file untouched.
What are the four places, and what does each survive?
| Where it lives | Survives a restart | Cost to read again | Lossy | What belongs here |
|---|---|---|---|---|
| In-context summary | No | 1x input | Yes | The current plan and recent turns |
| Note file outside the window | Yes | 1x input | No | Decisions, constraints, progress |
| Retrieval index | Yes | 1x input | No | Facts from prior runs, named at query time |
| Prompt cache | No, expires in 5 min or 1 h | 0.1x input | No | A stable prefix re-read many times |
The third column is the one that misleads. A cache hit at 0.1x the base input price is by far the cheapest way to put a token back in front of the model, which makes the prompt cache look like the obvious home for everything.
It is not memory. It is a discount on repetition, and a steeper one on the newest models, where Anthropic prices a hit at 0.05x on Claude Opus 5.5 and 0.025x on Claude Fable 5.1. Anthropic’s published pricing puts a 5-minute or 1-hour lifetime on it with a 1.25x or 2x write premium to establish it in the first place. Anything you would be upset to lose at minute six belongs in a file.
When is a summary the right answer?
When the thing being remembered is the shape of the work rather than its facts.
Anthropic describes compaction as taking a conversation nearing the context window limit, summarising its contents, and reinitiating a new window with the summary. The point of that is coherence: after compaction the agent still knows what it was doing and why, which is what lets a long-horizon task continue at all.
What compaction cannot promise is completeness. A summary is a lossy encoding chosen by a model under a token budget, and the recovery rules around it are heuristics rather than guarantees. Claude Code’s approach keeps the five most recently accessed files after a compaction. That is a sensible default and it is explicitly a recency rule, so a file that mattered at step 3 and was not touched since is gone at step 40.
The sub-agent is a summary with a boundary
Delegation is the same compression applied deliberately. A sub-agent works in its own clean context and returns a condensed result, typically 1,000 to 2,000 tokens, rather than its entire trace.
That ratio is the whole economic case for the pattern, and it is also its risk: the parent never sees what the child discarded. When the parent later needs a detail the child dropped, the only recovery is to run the child again, which is why multi-agent designs cost several times more tokens than a single agent before they earn anything back.
When should the agent write a file instead?
When losing the fact would force the run to start over.
That is the entire test, and it produces a short list: decisions already made, constraints discovered mid-run, external side effects already taken, and the position in a long sequence. Anthropic’s name for this is structured note-taking, with the agent maintaining a file outside the context window and reloading it on later turns.
A file has three properties a summary cannot have. It is exact, so the fact reads back as written rather than as a model’s paraphrase. It is inspectable, so a person debugging a failed run can read what the agent believed. And it is durable, so the same content that survives a compaction also survives the container being replaced, which makes it the natural companion to whatever the orchestrator persists in its checkpoint.
The cost is that the agent must decide what to write, and models under-write by default. A note file that only ever gains entries when the agent feels reflective is not a memory system.
What does a retrieval index actually buy?
Recall across runs, which is the only thing the other three cannot do.
The distinction between a note file and a retrieval index is whether the agent can name what it wants. A note file is read whole, or by a path the agent already knows. An index answers a question the agent poses without knowing what is in there, which is what you need when the relevant fact came from a different run last month.
Anthropic frames the alternative as just-in-time retrieval: rather than pre-loading data, the agent keeps lightweight identifiers and loads content at runtime through a tool. That is worth stating plainly because it inverts the usual instinct. The identifier is the memory. The content is fetched, used, and allowed to fall out of the window again, which keeps the accumulated context small and avoids the accuracy decay that comes with long contexts.
What should the placement rule be?
Four rules, one per place, applied when the fact is created rather than when the window fills.
- Context summary: nothing durable. If losing it would restart the run, it is in the wrong place. Our target here is 0 facts that exist only in a summary.
- Note file: everything the next attempt needs. Decisions, constraints, and completed side effects. Write on the event, not at the end.
- Retrieval index: facts from other runs. If it was produced by this run, a file is cheaper and exactly reproducible.
- Prompt cache: stable prefixes only. The cache pays back after 1 read at the 5-minute write price and 2 reads at the 1-hour price, so it rewards prefixes that do not change, and any edit above the breakpoint throws the saving away.
| Kind of fact | Cost if lost | Correct place | Rewrite frequency |
|---|---|---|---|
| Current plan and last few turns | Nothing, it regenerates | In-context summary | Every turn |
| A decision already made | The run repeats the decision | Note file | Once, on the event |
| A side effect already taken | A duplicate send or payment | Note file | Once, before the call |
| A constraint discovered mid-run | The run rediscovers it, or breaks it | Note file | Once, on discovery |
| A fact from an earlier run | The agent cannot know it at all | Retrieval index | Once per run |
| A large stable instruction prefix | 1x input instead of 0.1x | Prompt cache | Every 5 min or 1 h |
The review question is not how much context the agent uses. Ask: if this run’s container died right now, what would the next attempt have to rediscover? Every item on that list is currently in the wrong one of the four places.
Frequently asked questions
What is context compaction in an AI agent?
Summarising a context window that is nearing its limit and restarting with the summary in place of the full history. Anthropic describes it as reinitiating a new context window with the summary so the agent can continue on long-horizon work.
Should an agent use a vector database for memory?
Only for recall across many past runs where the agent cannot name what it needs in advance. For state within a single run a written file is cheaper, exactly reproducible, and readable by a person debugging the run.
Does prompt caching count as agent memory?
No. A cache makes re-reading the same prefix cheaper, but it expires on a timer measured in minutes or hours and can be evicted. Anything the agent must still know tomorrow has to be written somewhere durable.
What should an agent write to a note file rather than keep in context?
Decisions, constraints discovered mid-run, and anything whose loss would force the work to restart. Anthropic calls this structured note-taking, with the agent maintaining a file outside the context window and reloading it later.
Method and sources
- Anthropic, Effective context engineering for AI agents, consulted for the four named techniques, for the description of compaction, for the 1,000 to 2,000 token sub-agent return, and for the rule that Claude Code preserves the five most recently accessed files after compaction.
- Anthropic, Claude pricing documentation, read 24 September 2026. Source of the published cache multipliers of 1.25x for a 5-minute write, 2x for a 1-hour write and 0.1x for a cache hit (0.05x on Claude Opus 5.5, 0.025x on Claude Fable 5.1 and Claude Mythos 5.1), and of the 5-minute and 1-hour cache durations.
- Token Economics for LLM Agents: A Dual-View Study from Computing and Economics, arXiv:2605.09104, consulted for the framing of context as a budgeted production factor rather than free storage.
- The four-way classification and the placement rules here are our editorial judgement. We report no measured recall rates, retrieval hit rates or cost savings of our own, because we have not measured them.