Context Engineering for LLM Agents: Write, Select, Compress

For two years the craft was called prompt engineering, and it mostly meant polishing one clever instruction until a chatbot behaved. That framing broke the moment I started shipping agents that run for hundreds of turns, call tools, and read documents. The wording of the prompt turned out to be a tiny fraction of what determined success. Everything else in the window, the retrieved chunks, the tool outputs, the conversation history, the memory, mattered far more.

In this post I walk through what context engineering actually is, why the term crystallised in 2025, the specific ways a context window fails, the four strategies I use to keep it healthy, and the published numbers that show the impact. Every figure here is drawn from a primary source linked at the end.

What context engineering is, and why the term arrived in 2025

The label went mainstream in June 2025. On 19 June, Shopify CEO Tobi Lutke wrote that he preferred context engineering over prompt engineering because it describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the model. Six days later, on 25 June, Andrej Karpathy endorsed it, calling context engineering the delicate art and science of filling the context window with just the right information for the next step. In September 2025 Anthropic published an engineering guide defining it as the strategies for curating and maintaining the optimal set of tokens during model inference.

The key mental shift is that prompt engineering is now a subset of context engineering. Writing a good instruction still matters, but in a real agent it is one input among many. Anthropic frames context engineering as the natural progression of prompt engineering: instead of crafting a single prompt, you manage the entire state of information the model sees across every turn.

How the context window fails

Longer context is not automatically better. Anthropic describes context rot: as the number of tokens grows, the model's ability to accurately recall information from that context decreases, because attention is stretched across more pairwise relationships. Drew Breunig catalogued four concrete ways contexts fail, each with evidence from public studies.

• Poisoning: a hallucination or error enters the context and is then referenced repeatedly. A Gemini agent playing Pokemon kept pursuing impossible goals because a wrong game state stayed pinned in its context.
• Distraction: the context grows so long the model over-focuses on it and neglects what it learned in training. A Databricks study found correctness for Llama 3.1 405b began declining around 32k tokens, well before the window was full.
• Confusion: superfluous tools or content drag quality down. In one benchmark a quantized Llama 3.1 8b failed with 46 tools loaded but succeeded when the set was trimmed to 19.
• Clash: newly added information conflicts with what is already there. A Microsoft and Salesforce study found models performed 39 percent worse when information arrived across several turns, with OpenAI's o3 dropping from 98.1 to 64.1.

Write, select, compress, isolate: the four strategies

LangChain groups the working techniques into four categories, using a useful analogy: the context window is like RAM, a finite resource an operating system curates. Almost everything I do in production maps onto one of these four verbs.

• Write: save information outside the window so it survives, using a scratchpad for the current task or persistent memory across sessions, then read it back only when needed.
• Select: pull in just the relevant tokens on demand. RAG over documents and even over tool descriptions, plus lightweight identifiers loaded just in time rather than everything upfront.
• Compress: keep only the essential tokens. Summarise older turns, trim stale messages, and post-process bulky tool outputs. Claude Code auto-compacts near 95 percent of window utilisation.
• Isolate: split context across sub-agents or sandboxes. Anthropic's sub-agents each work in their own window and return a condensed summary of roughly 1,000 to 2,000 tokens to the lead agent.

A context budget, in code

Here is the shape of a single agent turn when you treat context as a budget. It writes a small stable core, selects only the tools and documents that fit the goal, and compresses history the moment it nears the limit. The point is that context assembly is code you own, run on every step, not a one-off prompt.

# One agent turn: assemble context under a fixed token budget
MAX_TOKENS = 180_000
COMPACT_AT = 0.95  # Claude Code auto-compacts near 95% utilization

def build_context(state):
    # WRITE once, keep it small: system prompt + a few canonical examples
    ctx = [state.system_prompt, *state.canonical_examples]

    # SELECT: RAG over tool specs and docs — pull the few that fit the goal,
    # not all 40 tools (more tools measurably lowers tool-call accuracy)
    ctx += select_relevant(state.tools, state.goal, top_k=5)

    # SELECT just-in-time: keep lightweight IDs, load full docs only on demand
    ctx += retrieve(state.query, top_k=5)
    ctx += state.recent_messages

    # COMPRESS: if we near the budget, summarize the oldest turns and
    # keep decisions, open bugs, and file paths — then drop the raw history
    if count_tokens(ctx) > MAX_TOKENS * COMPACT_AT:
        summary = summarize(state.older_messages,
                            keep=["decisions", "open_bugs", "file_paths"])
        ctx = replace(ctx, state.older_messages, summary)

    return ctx  # ISOLATE heavy sub-tasks in sub-agents that return ~1-2k tokens

The measurable payoff

The reason this stopped being optional is that the numbers are large and repeatable. A 39 percent multi-turn drop, an o3 score collapsing from 98.1 to 64.1, tool accuracy recovering when a set shrinks from 46 to 19, and correctness sliding at 32k tokens are not edge cases. They are the default behaviour of capable models fed poorly curated context.

So my rule when an agent misbehaves is to look at the context before I blame the model. Nine times out of ten the fix is fewer tools, a tighter retrieval query, a summary of the last twenty turns, or a sub-agent to isolate a noisy sub-task. Context engineering is unglamorous plumbing, and it is the single highest-leverage work in building agents that hold up in production.