Gen AI

In-Context Learning

Place a few input-output examples directly in the prompt and the model's behavior shifts on the very next request, with no gradient update and no change to any weight.

A few input-output examples placed directly in the prompt shift the model's behaviour on the very next request, with no gradient update and no change to any weight

Why Does This Exist?

Every learning method covered up to this point in the curriculum — gradient descent, backpropagation, fine-tuning — works the same way: compute a gradient, update the weights, repeat. That's "learning" in the conventional sense a machine learning course teaches, and it's slow, it needs a training pipeline, and it permanently changes the model.

Large language models exhibit a second, very different kind of behavior change that has nothing to do with any of that. Show a model a handful of examples of a task directly inside its prompt — no training loop, no optimizer, no gradient — and its next output shifts to match the pattern in those examples, as if it had "learned" the task on the spot. This is genuinely surprising the first time you see it, because nothing about next-token prediction as a training objective explicitly asks for this capability. It's not something anyone programmed in deliberately; it emerged from training at scale on next-token prediction alone.

Think of It Like This

A substitute teacher reading the lesson plan on the way in

Imagine a substitute teacher who has never taught a particular class before, handed a lesson plan five minutes before walking in, showing a few example problems already worked out in the exact style the regular teacher uses. The substitute doesn't need to be retrained as a teacher — they read the examples, infer the pattern being demonstrated, and apply that same pattern to the next problem, immediately, using knowledge and skill they already had before ever seeing this specific lesson plan.

In-context learning is exactly this: the model isn't acquiring new knowledge or new skill from the examples in the prompt. It's using its existing, already-trained ability to recognize a pattern and apply it, with the examples serving purely as instructions for which pattern to apply to this request — nothing about the underlying substitute teacher changed, and nothing about the underlying model's weights changed either.

How It Actually Works

The mechanism is inference, not training

Every step involved in in-context learning is an ordinary forward pass — the exact pipeline in how LLMs work, with no modification. The examples in the prompt are just more tokens, embedded and processed by the same transformer blocks as any other input. There is no separate "learning mode" the model switches into; the weights that produce the model's response to a prompt with five examples in it are bit-for-bit identical to the weights that would produce a response to a prompt with none. Whatever "learning" happens, happens entirely within the forward pass, using the fixed function the weights already encode.

Zero-shot, few-shot, and what actually varies

Zero-shot prompting gives no examples at all, relying purely on an instruction. Few-shot prompting, shown in the diagram, includes several input-output pairs before the actual query — three labeled sentiment examples, then a new case to classify. What varies between zero-shot and few-shot isn't the model or its weights; it's how much demonstration of the exact desired pattern is present in the input the model is conditioning on. More, clearer examples generally narrow down which pattern the model should apply, the same way a clearer lesson plan narrows down what the substitute teacher should do.

Why this behavior showed up at all

The leading explanation ties back directly to the next-token training objective: a sufficiently large and diverse training corpus contains an enormous number of passages that themselves demonstrate pattern-following — worked examples in textbooks, FAQ-style question-answer pairs, tables, translated phrase lists. Predicting the next token well across that kind of text specifically rewards a model for recognizing "here is a demonstrated pattern, and here is where it continues," because that's literally what correctly continuing those training passages required. In-context learning, under this view, isn't a separate mechanism bolted onto next-token prediction — it's next-token prediction generalizing a skill the training data happened to demonstrate repeatedly.

Show Me the Code

Building a few-shot prompt programmatically, to make explicit exactly what's varying between a zero-shot and a few-shot request — only the prompt text, never any model parameter.

1def build_prompt(examples: list[tuple[str, str]], query: str) -> str:2    lines = [f'"{inp}" -> {label}' for inp, label in examples]3    lines.append(f'"{query}" ->')4    return "\n".join(lines)5
6
7examples = [("happy", "positive"), ("awful", "negative"), ("fine", "neutral")]8zero_shot = build_prompt([], "thrilled")9few_shot = build_prompt(examples, "thrilled")10
11print(zero_shot)12# -> "thrilled" ->13print(few_shot)14# -> "happy" -> positive15#    "awful" -> negative16#    "fine" -> neutral17#    "thrilled" ->

Nothing here touches a model weight or runs an optimizer — few_shot is purely a longer string than zero_shot, built by concatenating example text. Whatever behavior difference results comes entirely from what the model, unchanged, does with that longer input during its forward pass.

Watch Out For

Calling this 'training' or 'fine-tuning' the model

Because the model's behavior visibly adapts to the examples, it's tempting to describe in-context learning with the same vocabulary used for actual training — but no gradient is computed, no weight changes, and nothing persists once the request ends. The very next request, with a different prompt, gets the exact same underlying model with zero memory of the previous prompt's examples. Confusing this with fine-tuning leads to wrong expectations about persistence: examples given in one conversation have no effect whatsoever on a separate, later conversation.

Assuming more examples always helps, without checking the ordering and selection

In-context learning is surprisingly sensitive to details that shouldn't matter if it were "genuine" learning: the order examples appear in, which specific examples are chosen, and even minor formatting changes can shift results meaningfully, sometimes more than adding additional examples does. Treating example selection and ordering as an afterthought, rather than something worth testing directly for a given task, is a common way few-shot prompting underperforms its potential.

The Quick Version

In-context learning is a behavior shift driven entirely by what's in the prompt — no gradient update, no weight change, nothing persisted afterward.
It runs through the exact same forward pass as any other request; there's no separate mechanism or mode.
Zero-shot gives no examples, few-shot gives several, and both use the identical underlying model weights.
The leading explanation is that next-token training on text that itself demonstrates patterns rewards recognizing and continuing patterns generally.
Results are sensitive to example order and selection in ways that wouldn't matter for actual weight-updating learning.