Key Takeaway
Context engineering is the discipline of curating exactly which tokens an LLM agent sees at each step: system prompt, tools, retrieved data, memory, and history. It superseded prompt engineering for production agents in 2025 because most agent failures are context failures, not model failures, and overloading the window degrades accuracy.
For two years the craft was called prompt engineering, and it mostly meant polishing one clever instruction until a chatbot behaved. That framing broke the moment I started shipping agents that run for hundreds of turns, call tools, and read documents. The wording of the prompt turned out to be a tiny fraction of what determined success. Everything else in the window, the retrieved chunks, the tool outputs, the conversation history, the memory, mattered far more.
In this post I walk through what context engineering actually is, why the term crystallised in 2025, the specific ways a context window fails, the four strategies I use to keep it healthy, and the published numbers that show the impact. Every figure here is drawn from a primary source linked at the end.
What context engineering is, and why the term arrived in 2025
The label went mainstream in June 2025. On 19 June, Shopify CEO Tobi Lutke wrote that he preferred context engineering over prompt engineering because it describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the model. Six days later, on 25 June, Andrej Karpathy endorsed it, calling context engineering the delicate art and science of filling the context window with just the right information for the next step. In September 2025 Anthropic published an engineering guide defining it as the strategies for curating and maintaining the optimal set of tokens during model inference.
The key mental shift is that prompt engineering is now a subset of context engineering. Writing a good instruction still matters, but in a real agent it is one input among many. Anthropic frames context engineering as the natural progression of prompt engineering: instead of crafting a single prompt, you manage the entire state of information the model sees across every turn.
| Dimension | Prompt engineering | Context engineering |
|---|---|---|
| Scope | One hand-written prompt | The whole token set across every turn |
| Unit of work | The wording of a single instruction | System prompt, tools, retrieved data, memory, history |
| When it runs | Once, before the call | Continuously, rebuilt on every step |
| Main failure risk | A vague or clumsy instruction | Overloading the window until recall degrades |
How the context window fails
Longer context is not automatically better. Anthropic describes context rot: as the number of tokens grows, the model's ability to accurately recall information from that context decreases, because attention is stretched across more pairwise relationships. Drew Breunig catalogued four concrete ways contexts fail, each with evidence from public studies.
- Poisoning: a hallucination or error enters the context and is then referenced repeatedly. A Gemini agent playing Pokemon kept pursuing impossible goals because a wrong game state stayed pinned in its context.
- Distraction: the context grows so long the model over-focuses on it and neglects what it learned in training. A Databricks study found correctness for Llama 3.1 405b began declining around 32k tokens, well before the window was full.
- Confusion: superfluous tools or content drag quality down. In one benchmark a quantized Llama 3.1 8b failed with 46 tools loaded but succeeded when the set was trimmed to 19.
- Clash: newly added information conflicts with what is already there. A Microsoft and Salesforce study found models performed 39 percent worse when information arrived across several turns, with OpenAI's o3 dropping from 98.1 to 64.1.
More context is not a free upgrade. A model with a million-token window can still lose the thread at a fraction of that limit, so treat every token you add as a cost, not a feature. If a chunk does not help the next step, leaving it out usually improves the answer.
Write, select, compress, isolate: the four strategies
LangChain groups the working techniques into four categories, using a useful analogy: the context window is like RAM, a finite resource an operating system curates. Almost everything I do in production maps onto one of these four verbs.
- Write: save information outside the window so it survives, using a scratchpad for the current task or persistent memory across sessions, then read it back only when needed.
- Select: pull in just the relevant tokens on demand. RAG over documents and even over tool descriptions, plus lightweight identifiers loaded just in time rather than everything upfront.
- Compress: keep only the essential tokens. Summarise older turns, trim stale messages, and post-process bulky tool outputs. Claude Code auto-compacts near 95 percent of window utilisation.
- Isolate: split context across sub-agents or sandboxes. Anthropic's sub-agents each work in their own window and return a condensed summary of roughly 1,000 to 2,000 tokens to the lead agent.
A context budget, in code
Here is the shape of a single agent turn when you treat context as a budget. It writes a small stable core, selects only the tools and documents that fit the goal, and compresses history the moment it nears the limit. The point is that context assembly is code you own, run on every step, not a one-off prompt.
# One agent turn: assemble context under a fixed token budget
MAX_TOKENS = 180_000
COMPACT_AT = 0.95 # Claude Code auto-compacts near 95% utilization
def build_context(state):
# WRITE once, keep it small: system prompt + a few canonical examples
ctx = [state.system_prompt, *state.canonical_examples]
# SELECT: RAG over tool specs and docs — pull the few that fit the goal,
# not all 40 tools (more tools measurably lowers tool-call accuracy)
ctx += select_relevant(state.tools, state.goal, top_k=5)
# SELECT just-in-time: keep lightweight IDs, load full docs only on demand
ctx += retrieve(state.query, top_k=5)
ctx += state.recent_messages
# COMPRESS: if we near the budget, summarize the oldest turns and
# keep decisions, open bugs, and file paths — then drop the raw history
if count_tokens(ctx) > MAX_TOKENS * COMPACT_AT:
summary = summarize(state.older_messages,
keep=["decisions", "open_bugs", "file_paths"])
ctx = replace(ctx, state.older_messages, summary)
return ctx # ISOLATE heavy sub-tasks in sub-agents that return ~1-2k tokens
Anthropic's practical advice: start with the smallest prompt on the best model you have, then add instructions only where you observe real failures. Prefer a few diverse, canonical examples over an exhaustive list of edge cases. A lean context you grow deliberately beats a bloated one you trim later.
The measurable payoff
The reason this stopped being optional is that the numbers are large and repeatable. A 39 percent multi-turn drop, an o3 score collapsing from 98.1 to 64.1, tool accuracy recovering when a set shrinks from 46 to 19, and correctness sliding at 32k tokens are not edge cases. They are the default behaviour of capable models fed poorly curated context.
So my rule when an agent misbehaves is to look at the context before I blame the model. Nine times out of ten the fix is fewer tools, a tighter retrieval query, a summary of the last twenty turns, or a sub-agent to isolate a noisy sub-task. Context engineering is unglamorous plumbing, and it is the single highest-leverage work in building agents that hold up in production.