5 Comments
User's avatar
Tirth Patel's avatar

Amazing article on context compaction

Reasoning Under Uncertainty's avatar

A model call produces the summary that becomes the agent's record of its earlier turns once compaction fires. Compaction isn't truncation, it's generation, and whatever the summarising model under-attends to, drops or restates wrong gets written into context as settled fact, with nothing marking which parts drifted.

The harnesses named here, OpenCode, Pi and Claude Code, each run their own implementation, and the token-budget problem gets solved either way; a faithful-summarisation problem takes its place, and it's not clear that one gets measured anywhere. Does any of them check the summary against the session it replaces?

David Parker's avatar

The bit about compaction silently dropping turns is the part most guides gloss over. We ran 44 agents across e-commerce workflows and the ones that failed weren't the ones that hit context limits — they were the ones where compaction quietly collapsed the session into generic boilerplate and the agent kept going as if nothing was wrong. A log entry saying "compacted 12k tokens" tells you almost nothing about what just got thrown away.

Abhishek Pan's avatar

Compacting right before the 1-hour prompt cache expires is a smart default. If you roll your own and drop tool calls before summarizing, keep the IDs of any writes the agent already made outside the summary. Otherwise after compaction it can retry a POST it no longer remembers sending.

Hellospark's avatar

Plain-English version for anyone who hasn't hit this yet: when a session gets too long, most agent tools compact it with another model call that summarizes it, and that summary is where details get lost. The fix we'd copy first is the recall one. Dump the important tool results to files and keep a table of contents to the old conversation, so a detail that missed the summary can still be looked up. Has anyone compared the running-notebook approach against the summary call on their own evals?