In-Context Learning

Why Does This Exist?

Changing what a model does used to mean changing its weights. You gathered data, ran a training loop, and shipped new numbers. That machinery is heavy: data to clean, gradients to compute, checkpoints to judge. When a behavior has to last, it's still the right tool, which is why when to fine-tune gets its own page.

But models trained at scale turned out to have a second gear. Paste three worked examples into the prompt and the very next answer follows their pattern. No optimizer runs. No weight moves. Delete the examples and the behavior vanishes like it was never there. That's startling if your whole picture of learning is gradient descent, because here one frozen function acts like a different machine depending on what you show it.

One idea first, since everything below leans on it. A language model trains on a single job: given some text, guess the next token, check, adjust. Repeat that across enough text and the weights soak up grammar, facts, and styles along the way. The full version lives in next-token prediction. In-context learning is what that same next-token machinery does at test time, when the text sitting before it happens to contain examples.

This page covers that test-time behavior: what varies, why it works, and where it breaks. The hands-on version, writing prompts that show instead of tell, is few-shot prompting. The permanent version, actually rewriting the weights, is instruction tuning.

Think of It Like This

How It Actually Works

Conditioning, not training

Write the model's behavior as $P_\theta(y \mid x)$. In plain words: $P$ is the probability distribution the model computes over possible outputs, $\theta$ (theta) is its weights, $y$ is the output tokens you have not seen yet, $x$ is the prompt tokens you supplied, and $\mid$ means "given". Zero-shot prompting asks for $P_\theta(y \mid x)$ with an instruction alone. Few-shot prompting asks for $P_\theta(y \mid E, x)$, where $E$ is a few demonstration pairs placed before the query. Only the right side of the bar changed. $\theta$ sits outside everything that moves.

All of it runs through the ordinary forward pass described in how LLMs work: embed the tokens, run the transformer blocks, sample the next token. There's no second mode. The weights answering a prompt with five examples are bit-for-bit the weights answering a prompt with none, so nothing persists. The next request starts from the same $\theta$ with a different $x$.

Zero-shot, one-shot, few-shot

The names just count examples. Zero-shot shows none and leans on the instruction alone (zero-shot prompting). One-shot shows a single pair. Few-shot shows a handful, usually two to five. The model never changes between these. Only the length and content of $x$ changes.

More examples usually pin down the pattern better, but each one spends context. A context window is finite, and every example token is one your actual task cannot use. Past a point you are paying rent on every request for examples that stopped adding signal several examples back.

Why examples steer at all

Pretraining text is full of pattern-then-continuation: glossaries, phrasebooks, worked exercises, tables. A model that predicts next tokens well across that diet has to get good at spotting "a pattern is being demonstrated" and continuing it, because finishing those passages demands exactly that. Few-shot prompting works because it hands the model the kind of passage it spent training learning to finish.

That is also the assumption the whole trick rests on: the examples select among patterns the model already knows. Break it and steering fails. Ask for a notation the model never saw, with three neat examples, and you'll often get something fluent and wrong, continuing the surface shape while missing the rule. Fluency is not comprehension, and examples cannot fix a missing capability.

Worked example: counting the prompt

Take sentiment classification with three examples and one query, the same prompt the diagram draws:

1"happy" -> positive2"awful" -> negative3"fine" -> neutral4"thrilled" ->

Count the words on each line. "happy" -> positive is three words. So is each of the next two lines. The query line, "thrilled" ->, is two. Total: $3 + 3 + 3 + 2 = 11$ words, against 2 words for the zero-shot version. Nine extra words buy the behavior shift.

Those nine words are re-sent, re-processed, and re-paid on every single request. Nothing is amortized, because nothing is stored. That is the trade in one line: instant adaptation at per-request rent, versus instruction tuning, which pays once during training and then charges nothing per request.

Code

The point of this snippet is the boundary it draws: the only object the code builds is a longer string. No model, no weights, and no optimizer appear anywhere in it.

1def build_prompt(examples: list[tuple[str, str]], query: str) -> str:2    lines = [f'"{text}" -> {label}' for text, label in examples]3    lines.append(f'"{query}" ->')4    return "\n".join(lines)5
6
7examples: list[tuple[str, str]] = [8    ("happy", "positive"),9    ("awful", "negative"),10    ("fine", "neutral"),11]12zero_shot = build_prompt([], "thrilled")13few_shot = build_prompt(examples, "thrilled")14
15print(zero_shot)16# -> "thrilled" ->17print(len(zero_shot.split()), len(few_shot.split()))18# -> 2 11

Two words versus eleven: the same numbers as the worked example and the diagram. Whatever behavior gap you observe between those two strings comes from the forward pass reading a longer $x$, because there is nothing else here that could cause it.

Watch Out For

The Quick Version

• In-context learning changes what a model does without touching its weights: examples in the prompt steer the next output, then vanish.
• It runs through the ordinary forward pass. There is no separate learning mode, and nothing persists between requests.
• Zero-shot shows no examples and few-shot shows a handful, both on identical weights.
• It works because next-token pretraining rewards continuing demonstrated patterns, so a prompt shaped like a pattern gets continued.
• Examples rent context on every request and order effects are real, so freeze one template and test it before shipping.
• When the behavior must persist across sessions or the model lacks the underlying skill, examples are the wrong tool: retrieve or fine-tune instead.

What to Read Next

• Few-Shot Prompting
• Zero-Shot Prompting
• When to Fine-Tune
• Next-Token Prediction