What is Agentic Context Management?

The plain answer

Agentic context management (ACM) is the discipline of deciding what an AI agent keeps in its working memory at every step, for how long, and at what cost, instead of just letting it re-send its entire history on every call. It is the difference between an agent that gets more expensive and more confused with every turn, and one that stays sharp and affordable no matter how long the task runs.¹

The term comes from a straightforward observation: most agent failures and most agent cost blowouts trace back to the same root cause. The model isn't bad at reasoning. It's drowning in its own accumulated history, tool outputs, and stale conversation, and you are paying by the token for the privilege.¹

Why agent bills spiral in the first place

Here is the mechanic nobody explains clearly enough: most hand-built agents re-send the entire conversation history with every single call. If each turn adds roughly 500 tokens and a conversation runs 100 turns, the naive "full-append" pattern doesn't cost you 100 times as much as one turn. It costs you roughly 50 times as much, and the ratio keeps getting worse the longer the conversation runs, because cumulative token cost grows with the square of the conversation length, not linearly with it.¹

At 100 turns, unmanaged full-append context costs about 6 times more than a properly bounded context. At 200 turns, it's about 13 times more.¹ That is not a rounding error on your API bill. That is the difference between a feature that ships and a feature finance kills.

This is also exactly the failure mode that shows up in production agents built by non-engineers using no-code tools. An employee builds a support bot or a research assistant, it works great in the demo, and three weeks later the token bill has quietly quadrupled because nobody designed for what happens on turn 50. Comet's research on context windows makes the point with a live example: a 50-step agentic workflow at 20,000 tokens per call adds up to a million tokens total, and the failures are invisible. The agent keeps running with incomplete information and produces confident, wrong answers instead of erroring out cleanly.²

It's not just a cost problem. It's an accuracy problem too

The obvious fix, just cap the context and summarize the overflow, has its own trap. Crude summarization bounds your token cost, but it can be catastrophically lossy. In one documented case, compressing an 18,282-token context down to 122 tokens in a single unvalidated step dropped task accuracy from 66.7% to 57.1%, which is worse than giving the agent no context at all.¹

So the real design problem has two axes, not one: cost and fidelity. Full-append gives you full fidelity until the model gets overwhelmed, at quadratic cost. Crude summarization gives you linear cost but an accuracy cliff. The only combination that actually works is validated compaction: reducing context to a budget while checking that nothing critical was thrown away, which achieves linear cost with preserved accuracy.¹

This is why researchers building production memory systems now describe agentic context management as five distinct jobs, not one storage bucket: architecting what categories of information matter for a given agent, ingesting raw signal into structured memory, scoping what's relevant to this specific turn versus the whole organization, anticipating what the agent will need next, and compacting the overflow without losing what matters.¹ A system that does only one of these well is a memory tool. A system that does all five is a context-management platform.

What this looks like when it goes wrong

Long-context research backs up the anecdotal complaints. A widely cited study on "lost in the middle" behavior found that language models are much better at using information at the very start or end of a long context than information buried in the middle, even when that information is technically present.² So a bigger context window doesn't just cost more. It doesn't even guarantee the agent will use what you paid to include.

Agent-building teams have their own name for the sharpest version of this failure: an agent making one perfectly reasonable tool call, like reading a file or listing records, that happens to return a response large enough to blow past its context window entirely. The task doesn't fail gracefully. It just dies, and the agent never understands why.³ Common triggers include pulling too many documents into context before the user even asks a question, loading hundreds of verbose tool definitions upfront, and letting sequential tool calls quietly accumulate tens of thousands of tokens of intermediate results that nobody is watching.³

The fixes that actually work

Production teams tend to converge on a handful of patterns:

• Sub-agent isolation. Delegate risky, unpredictable operations (file reads, broad searches) to a sub-agent with its own context window. If it dies, the orchestrator survives. A sub-agent can explore 50,000 tokens of material and hand back a 2,000-token summary.³
• Just-in-time loading instead of upfront dumping. Load tool definitions and reference material only when needed rather than stuffing everything in at the start. Claude Code's tool lazy-loading, for example, cuts context by roughly 95% by not loading tool schemas until they're actually called.⁴
• Validated compaction. Periodically compress the conversation, but check that the compression didn't drop anything load-bearing before moving on.¹
• Prompt caching and model routing. Cache repeated context blocks instead of paying full price for the same tokens on every call, and route simple tasks to cheaper models instead of running everything through a frontier model by default. Anthropic's prompt caching cuts cached input token costs by roughly 90%.⁵

None of this is exotic. It's closer to database indexing discipline than machine learning research. Which is exactly the point: agentic context management is an engineering and operations problem, not a model problem.

Why this matters more now, not less

This isn't an abstract concern for AI labs. McKinsey's 2026 survey of AI use across enterprises found that about one in five organizations report that AI operating costs, including token costs, are already constraining how much they use the technology.⁶ At the same time, nearly a third of respondents said their organizations decided against buying a software product because employees could build the equivalent themselves with agentic coding tools, and that share is even higher among the highest-performing AI adopters.⁶ Cost constraints hit those same coding-agent builders about three times as often as everyone else.⁶

Put those two facts together and the shape of the problem is clear. More employees are building their own agents and internal tools. Those tools run longer, call more APIs, and accumulate more context than anyone designed for. And the bill for that accumulation lands on a budget line nobody was tracking. This is the same dynamic covered in "The 'Free Claude' Problem": when anyone can spin up an agent, the token spend follows the same unmanaged path as the memory does, quietly, until someone notices the invoice.

If you're serious about owning the software your team builds rather than just tolerating what shows up, this is one reason platforms like Remy build context and cost controls into the app itself rather than leaving memory management as an afterthought bolted on after the bill arrives.

Frequently asked questions

Is agentic context management the same thing as prompt engineering? No. Prompt engineering is about how you phrase a single instruction. Context management is about the entire information lifecycle across a multi-step task: what enters the model's memory, what gets compressed, what gets isolated, and what never gets loaded at all.³

Does a bigger context window solve this? Not by itself. Larger windows raise the ceiling but don't fix attention problems, and models still favor information at the start or end of a long context over material buried in the middle.² A bigger window can also mean a bigger bill for the same task if nothing is managing what actually goes into it.

How much can better context management actually save? It depends on the workload, but the reported ranges are large. Prompt caching alone can cut cached token costs by roughly 40% to 90%, and routing simple tasks to smaller models instead of a frontier model by default commonly cuts spend by 50% to 80% on that traffic.⁵

Who should own this inside a company, not just at the model layer? Whoever owns the budget for the tool. If an employee builds an internal agent with a no-code platform, the context-management choices baked into that platform, or the lack of them, directly determine whether the tool stays cheap or turns into a line item finance has to chase down every quarter.

Is this only a concern for engineering teams? No. As more non-engineers build their own agents and workflows, context management becomes a governance question as much as a technical one. It shows up in the same conversations covered in "The Hidden Costs of AI Agents" and "The 'Claudish' Code Base": who is watching what these tools cost once they're live.