Agent Memory Compaction: Keep, Summarise, Drop (2026)
When an interviewer asks how a long-running agent manages its memory, they are testing compaction: under a fixed context cap, each past turn can be kept whole, summarised (fewer tokens, less signal), or dropped. The naive answer — just keep the most recent turns — quietly throws away the important older ones.
Concrete, verifiable example. In the puzzle below, 7 turns compete for a 610-token budget. Choosing keep / summarise / drop per turn to maximise retained importance scores 70. Keeping only the most recent turns whole — the recency default — scores 53, leaving 24% of the achievable importance on the floor. Summarising the bulky-but-useful turns is what closes the gap.
That gap is the interview. Compaction is a knapsack with three options per item: keep (full tokens, full signal), summarise (few tokens, partial signal), or drop (nothing). A senior answer summarises a turn that is too bulky to keep but too important to lose. This guide gives you the model, and a free interactive scored against the exact optimum.
Why this question shows up in AI-engineering interviews
Every long-context agent hits the cap eventually, so memory management is a staple — disguised as:
- “How does your agent handle a conversation longer than its context window?”
- “What do you keep, summarise, or forget as history grows?”
- “How do you compact memory without losing the thread?”
- “Why did the agent forget something important from earlier?”
Each is the same budgeted choice: which turns earn their tokens at full fidelity, which are worth a cheaper summary, and which to drop. Interviewers ask because getting it wrong is why agents ‘forget’ the one detail that mattered — they spent the budget on recent-but-trivial turns.
The mental model that gets you to optimal
Treat it as a knapsack where each turn offers three (tokens, importance) options. Four ideas get you to the best retained importance:
- Summarise the bulky-but-useful. A turn too large to keep whole but too important to drop is the perfect summary candidate — partial signal at a fraction of the tokens often beats all-or-nothing.
- Recency is a heuristic, not the goal. The most recent turns aren’t always the most important; keeping them whole just because they’re newest starves older, higher-value context.
- Value per token, again. Keep a turn whole only when its importance justifies its full token cost; otherwise summarise or drop. You’re maximising importance under a token budget.
- Dropping is a real move. A low-importance turn that resists summarisation should go entirely — its tokens buy more importance elsewhere.
Put together: rank turns by importance per token across their keep/summarise options, and fill the budget — summarising where a summary beats keeping whole, dropping where neither pays. That is the compaction the puzzle scores.
The trap: “just keep the recent turns”
The common wrong answer is a sliding window — keep the last N turns whole, drop the rest. It is simple and it loses information: a bulky important turn just outside the window vanishes, while a trivial recent one keeps its full tokens. In the puzzle, recency scores 53 against the optimal 70, forfeiting 24% of the achievable importance. Naming summarisation as the middle option is the senior answer.
Practice it: compact the memory under the cap
Describing the tradeoff is not the same as making the keep/summarise/drop calls under a budget. Below is the live Memory Compaction puzzle from The Arena. Choose per turn; The Arena computes the compaction that retains the most importance and scores yours as a percentage of it. No login required.
What “good” looks like
When you can compact to the most importance under a cap, you can answer “how does your agent manage memory?” with keep/summarise/drop reasoning instead of “I keep the last few turns.” Beating the recency default’s 53 toward the optimal 70 means you’ve internalised that summarisation is the move that saves the important-but-bulky turn.
Frequently asked
What is memory compaction in AI agents?
Memory compaction is deciding, under a fixed context budget, which past turns to keep whole, which to summarise (fewer tokens, partial signal), and which to drop — to retain the most useful information. It is a knapsack with three options per turn; in the worked example the optimum retains 70 importance versus 53 for keeping only recent turns.
Should an agent just keep the most recent turns?
Not necessarily. Recency is a cheap heuristic but it discards important older context while spending full tokens on trivial recent turns. Summarising bulky-but-important turns and dropping low-value ones usually retains more useful information under the same cap.
How do you summarise conversation history for an LLM agent?
Summarise the turns that are too large to keep whole but too important to drop — paying a few tokens for partial signal. Keep small high-value turns verbatim, and drop low-importance turns entirely. The goal is maximum retained importance per token under the context cap.
Why does an AI agent forget important earlier context?
Usually because the compaction policy is recency-based: it keeps the last N turns whole and drops everything older, so an important earlier turn falls out of the window. A value-aware policy summarises that turn instead of dropping it.
New scored, provably-optimal interview drills, now and then — no spam, unsubscribe anytime.
More interview guides
41 puzzles — RAG tuning, tool routing, agent orchestration, prompt/KV caching, memory compaction — each scored against a provably optimal answer.
Explore the puzzles →