Context Engineering

Why Does This Exist?

Prompt engineering assumes the model already holds everything it needs, and the only open question is how you phrase the ask. That picture breaks the moment a system answers more than one self-contained question. Picture a support assistant mid-conversation: it needs its standing rules, the last dozen turns, two retrieved help-center articles, and the raw JSON the order lookup just returned. No sharper instruction rescues you if the order details never entered the window. A better sentence can't summon a missing fact.

That gap is what context engineering covers: deciding what the model actually sees when it reads your instruction. Wording still matters, and prompt anatomy covers how a single prompt is built. This page covers the assembly around it: the four lanes competing for one fixed budget, the order you trim them in, and why fitting is still not enough. The frame itself is context windows. The token math is context budgeting. The reason a full window still misbehaves is context degradation. And if you want the contrast in one place, the prompt versus context engineering comparison draws the line.

The one idea you need first: a context window is the fixed number of tokens a model reads in one request, as defined in context window. Anything outside it is not skimmed or deprioritized. It's gone, and the model will answer without it rather than tell you it's missing.

Think of It Like This

How It Actually Works

The four lanes sharing one window

Almost every production system fills the window from the same four lanes. The system prompt carries standing instructions: tone, escalation rules, what the assistant may promise. Its full treatment is in system prompts. History is what has been said this session; drop it and the assistant re-asks for the order number it already has. Retrieved passages arrive from a search step at request time, usually through the pipeline in RAG architecture; without them the model guesses at policy. Tool output is the raw result of an action just taken, delivered the way tool use and function calling describes; it's often the least compressible lane, because trimming it risks cutting the exact field the customer asked about.

Remove any one lane and a specific failure appears: no system prompt, drifting behavior; no history, repeated questions; no retrieval, confident wrong answers about policy; no tool output, an answer that ignores the live fact being asked about.

The budget rule and the cutting order

Write $T_{window}$ for the window size in tokens, and $T_{sys}$, $T_{hist}$, $T_{ret}$, $T_{tool}$ for what the four lanes request. Everything the model sees must satisfy:

When the left side is bigger, the difference is the overflow, and which lane absorbs it is a design decision, not an accident. The usual priority: the system prompt is trimmed last, because it's small and cutting it changes behavior unpredictably. Tool output is pinned next, because the answer typically depends on one field inside it. That leaves history and retrieval to absorb the pressure. History goes first, oldest turns first, by summarizing or dropping; retrieval goes second, by keeping the matching passage and cutting the rest. A well-built assistant compacts old turns into a running summary before it cuts a retrieved passage further: a stale summary is a smaller loss than a policy article missing the clause the customer needs.

Worked example: a 4,300 token request in a 4,000 token window

Take a request where the lanes ask for $T_{sys} = 400$, $T_{hist} = 2200$, $T_{ret} = 1200$ and $T_{tool} = 500$ tokens, with $T_{window} = 4000$. Add them: $400 + 2200 + 1200 + 500 = 4300$. The overflow is $4300 - 4000 = 300$ tokens.

History absorbs the cut first: $2200 - 300 = 1900$. Nothing still overflows, so retrieval stays at $1200$, and the pinned lanes stay at $400$ and $500$. The final window holds $400 + 1900 + 1200 + 500 = 4000$ tokens, exactly the budget. The diagram above draws this same fixture: four lanes in, a 300 token piece spilling off the frame, history trimmed to 1900.

Why fitting inside the window is still not enough

Here's the mistake the budget rule invites: treating the problem as solved once everything technically fits. A model can hold a fact in context and still use it badly, because position matters. Instructions at the very top lose force as thousands of tokens pile beneath them, and a fact buried mid-prompt gets weaker attention than the same fact placed at the end. Length alone degrades use, independent of relevance. That's context degradation, and it's why stuffing the window to the brim can score worse than a shorter, sharper assembly.

So the test for adding anything is never "does this fit." It's "does adding this raise the odds of the right answer." Every kept token costs attention weight on everything else, plus latency and serving cost. That's the trade-off: recall against precision and cost. Cut too much and you lose the clause the answer needed; keep too much and you drown it.

Two honest boundaries close the picture. When not to use any of this machinery: a single self-contained question whose prompt sits far under the window needs none of it, and plain prompt wording wins. And the whole practice assumes trimming the oldest history first is safe, which fails whenever the key fact arrived early. If the order number appeared in turn one and your policy drops turn one first, the assistant re-asks for what it was told. Pin load-bearing facts, or summarize with them kept, instead of trusting recency. What outlives the window entirely is a different system, covered in memory systems.

Code

A minimal budget allocator: four requested lanes and a fixed window, with history absorbing the overflow first and retrieval second, while system and tool output stay pinned.

1from dataclasses import dataclass2
3
4@dataclass5class Budget:6    system: int7    history: int8    retrieval: int9    tool_output: int10
11
12def allocate(window: int, requested: Budget) -> Budget:13    """Trim history first, then retrieval, until the lanes fit the window."""14    total: int = requested.system + requested.history + requested.retrieval + requested.tool_output15    overflow: int = max(0, total - window)16    history: int = max(0, requested.history - overflow)17    covered: int = requested.history - history18    retrieval: int = max(0, requested.retrieval - (overflow - covered))19    return Budget(requested.system, history, retrieval, requested.tool_output)20
21
22result = allocate(window=4000, requested=Budget(400, 2200, 1200, 500))23print(result)  # -> Budget(system=400, history=1900, retrieval=1200, tool_output=500)24print(400 + 1900 + 1200 + 500)  # -> 4000

Feed it the numbers from any request and it shows exactly which lane absorbs the overflow: history first, retrieval next, system and tool output untouched. The fixture above is the worked example, so the output doubles as the check.

Watch Out For

The Quick Version

• Context engineering decides what the model sees, not how the instruction is worded: four lanes, system, history, retrieval and tool output, share one fixed token window.
• The lanes must satisfy a budget rule, so any overflow is cut from somewhere, and the usual order trims history before retrieval while system and tool output stay pinned.
• A 4,300 token request in a 4,000 token window loses 300 tokens of history first, leaving 400 plus 1900 plus 1200 plus 500.
• Fitting is not enough, because position and length degrade how well the model uses what it holds.
• Add a token only when it raises the odds of the right answer, and pin load-bearing facts instead of trusting recency.

What to Read Next

• Context Windows
• Context Budgeting
• Context Degradation
• RAG Architecture