What is Agentic Context Management?

What is agentic context management?

Agentic context management (ACM) is the discipline of deciding what an AI agent keeps in its working memory at every step, for how long, and at what cost, instead of just letting it re-send its entire history on every call. It is the difference between an agent that gets more expensive and more confused with every turn, and one that stays sharp and affordable no matter how long the task runs.

The term comes from a straightforward observation: most agent failures and most agent cost blowouts trace back to the same root cause. The model isn't bad at reasoning. It's drowning in its own accumulated history, tool outputs, and stale conversation, and you are paying by the token for the privilege.

Why do agent bills spiral in the first place?

Here is the mechanic nobody explains clearly enough: most hand-built agents re-send the entire conversation history with every single call. If each turn adds roughly 500 tokens and a conversation runs 100 turns, the naive "full-append" pattern doesn't cost you 100 times as much as one turn. It costs you roughly 50 times as much, and the ratio keeps getting worse the longer the conversation runs, because cumulative token cost grows with the square of the conversation length, not linearly with it.

At 100 turns, unmanaged full-append context costs about 6 times more than a properly bounded context. At 200 turns, it's about 13 times more. That is not a rounding error on your API bill. That is the difference between a feature that ships and a feature finance kills.

This is also exactly the failure mode that shows up in production agents built by non-engineers using no-code tools. An employee builds a support bot or a research assistant, it works great in the demo, and three weeks later the token bill has quietly quadrupled because nobody designed for what happens on turn 50. Comet's research on context windows makes the point with a live example: a 50-step agentic workflow at 20,000 tokens per call adds up to a million tokens total, and the failures are invisible. The agent keeps running with incomplete information and produces confident, wrong answers instead of erroring out cleanly.

Is context management only a cost problem, or an accuracy problem too?

The obvious fix, just cap the context and summarize the overflow, has its own trap. Crude summarization bounds your token cost, but it can be catastrophically lossy. In one documented case, compressing an 18,282-token context down to 122 tokens in a single unvalidated step dropped task accuracy from 66.7% to 57.1%, which is worse than giving the agent no context at all.

So the real design problem has two axes, not one: cost and fidelity. Full-append gives you full fidelity until the model gets overwhelmed, at quadratic cost. Crude summarization gives you linear cost but an accuracy cliff. The only combination that actually works is validated compaction: reducing context to a budget while checking that nothing critical was thrown away, which achieves linear cost with preserved accuracy.

This is why researchers building production memory systems now describe agentic context management as five distinct jobs, not one storage bucket: architecting what categories of information matter for a given agent, ingesting raw signal into structured memory, scoping what's relevant to this specific turn versus the whole organization, anticipating what the agent will need next, and compacting the overflow without losing what matters. A system that does only one of these well is a memory tool. A system that does all five is a context-management platform.

What does it look like when context management goes wrong?

Long-context research backs up the anecdotal complaints. A widely cited study on "lost in the middle" behavior found that language models are much better at using information at the very start or end of a long context than information buried in the middle, even when that information is technically present. So a bigger context window doesn't just cost more. It doesn't even guarantee the agent will use what you paid to include.

Agent-building teams have their own name for the sharpest version of this failure: an agent making one perfectly reasonable tool call, like reading a file or listing records, that happens to return a response large enough to blow past its context window entirely. The task doesn't fail gracefully. It just dies, and the agent never understands why. Common triggers include pulling too many documents into context before the user even asks a question, loading hundreds of verbose tool definitions upfront, and letting sequential tool calls quietly accumulate tens of thousands of tokens of intermediate results that nobody is watching.

Which fixes actually work?

Production teams tend to converge on a handful of patterns:

• Sub-agent isolation. Delegate risky, unpredictable operations (file reads, broad searches) to a sub-agent with its own context window. If it dies, the orchestrator survives. A sub-agent can explore 50,000 tokens of material and hand back a 2,000-token summary.
• Just-in-time loading instead of upfront dumping. Load tool definitions and reference material only when needed rather than stuffing everything in at the start. Claude Code's tool lazy-loading, for example, cuts context by roughly 95% by not loading tool schemas until they're actually called.
• Validated compaction. Periodically compress the conversation, but check that the compression didn't drop anything load-bearing before moving on.
• Prompt caching and model routing. Cache repeated context blocks instead of paying full price for the same tokens on every call, and route simple tasks to cheaper models instead of running everything through a frontier model by default. Anthropic's prompt caching cuts cached input token costs by roughly 90%.

None of this is exotic. It's closer to database indexing discipline than machine learning research. Which is exactly the point: agentic context management is an engineering and operations problem, not a model problem.

Why does this matter more now, not less?

This isn't an abstract concern for AI labs. McKinsey's 2026 survey of AI use across enterprises found that about one in five organizations report that AI operating costs, including token costs, are already constraining how much they use the technology. At the same time, nearly a third of respondents said their organizations decided against buying a software product because employees could build the equivalent themselves with agentic coding tools, and that share is even higher among the highest-performing AI adopters. Cost constraints hit those same coding-agent builders about three times as often as everyone else.

Put those two facts together and the shape of the problem is clear. More employees are building their own agents and internal tools. Those tools run longer, call more APIs, and accumulate more context than anyone designed for. And the bill for that accumulation lands on a budget line nobody was tracking. This is the same dynamic covered in "The 'Free Claude' Problem": when anyone can spin up an agent, the token spend follows the same unmanaged path as the memory does, quietly, until someone notices the invoice.

If you're serious about owning the software your team builds rather than just tolerating what shows up, this is one reason platforms like Remy build context and cost controls into the app itself rather than leaving memory management as an afterthought bolted on after the bill arrives.