Reasoning Distillation
You stop asking a small model to invent reasoning from scratch. A large teacher writes its step-by-step thinking, a checker keeps only the correct traces, and the small model learns those steps by heart.
Why Does This Exist?
Small models don't learn to reason by being told to try harder. If you run reinforcement learning on a small model from scratch, you pay for thousands of sampled rollouts and you often get flat tricks instead of real backtracking. The signal is thin. A single pass or fail at the end says nothing about which of the 200 thinking tokens went wrong, so search wanders and bills climb.
You've seen the expensive way to earn those traces. A large reasoning model trained with RLVR already writes long thinking with tries, slips, and corrections. Reasoning distillation reuses that work. You collect its traces, you keep only the ones that finish right, and you teach a small model to predict them token by token with supervised fine-tuning. No new RL run. Just imitation of good thinking.
This page covers that teacher to filter to student loop and its ceiling. Prompt-time stepwise work lives in chain of thought. Training a model to write traces at all lives in chain-of-thought training. Picking your own best samples without any teacher lives in rejection sampling fine-tuning. You need one idea before the mechanism below. SFT trains a model to predict demonstration tokens while the prompt stays out of the loss, which is next-token prediction aimed at a curated target. If that feels thin, read supervised fine-tuning first, then come back.
Think of It Like This
Copying the head chef's notebook
You apprentice in a busy kitchen. The head chef never just hands you the finished plate. She hands you her notebook. For one sauce it reads: tried salt first, too flat. Added acid, too sharp. Cut the heat, reduced ten minutes, tasted again, done. You cook from that notebook line by line until your hands know the order: season, taste, fix, taste again.
Where it stops: the notebook can't taste for you. If the chef misjudged a step yet the plate still passed, you copy the mistake along with the method. And you'll never invent a sauce the chef never wrote. Copying spreads skill. It doesn't create new skill.
How It Actually Works
Call the prompt tokens , in plain words the question you typed. Call the trace tokens , in plain words the steps written before the answer, made of pieces where is the count of trace tokens. Call the final answer tokens , in plain words the short answer at the end, with answer tokens. You'll train a student whose weights are , in plain words every number you will update.
What the teacher writes
For each prompt , the teacher writes candidate traces, where is the count of tries per question. A verifier checks each try, in plain words a checker that returns 1 when the final answer is right and 0 otherwise. For math that checker can be exact match. For code it can be hidden tests. Keep the triple only when . Drop the rest. The kept triples form the dataset , in plain words the list of examples the student will study.
This is the whole filter. You don't grade style. You don't rank elegance. You keep thinking that ends right and you discard thinking that ends wrong. That choice is also the weak point, and you'll see why in the pitfalls.
Worked example: 200 prompts become 340 traces
Say you start with 200 prompts and set tries each. That gives:
Total traces .
Say the verifier keeps 340 of them. Then:
Keep rate , which is 34 percent. The other traces go away.
Say each kept trace plus its answer averages 180 graded tokens. Then:
Graded tokens .
So the student sees 61,200 tokens of thinking that all end right, drawn from 200 starting questions. Same arithmetic lives in the diagram above and the snippet below, so you can check it twice.
The student update and its ceiling
The student learns with the same objective as ordinary SFT. For one kept example, write its graded tokens as , in plain words each trace token followed by each answer token in order. Write for the graded history before position , in plain words all kept tokens earlier in the same example. Write for the chance the student assigns to the correct token given the prompt plus that history. Then the loss is:
In plain words: walk each kept token, take the log of the chance you gave the right token, negate it, add it up. The prompt is read but never graded, which is masking. Low chance on a step means large loss on that step, so the gradient pushes you toward writing that step next time. You pay SFT cost and you get SFT stability. No rollouts. No value model. No clipping.
The trade is sharp. You buy reasoning behavior for the price of filtering plus SFT, far below a fresh RL run. You pay with a ceiling. The student copies the teacher's procedures, including its tics, and it won't beat the teacher on problems the teacher can't solve. The method assumes the verifier catches bad thinking. When a trace reaches the right answer through wrong steps, that assumption fails and you teach confident nonsense with full weight.
Skip this when you need new capability rather than cheaper capability. If no checker exists for the task, if the teacher itself is weak there, or if the job is single fact recall with no steps to show, distillation adds cost without gain. Use it when traces are checkable, the teacher is strong, and you want a small model that backtracks, retries, and shows its work.
Code
This snippet recomputes the worked example so you don't take the arithmetic on trust. It uses only the standard library.
def keep_rate(kept: int, total: int) -> float: """Fraction of generated traces the verifier keeps.""" return kept / total
def graded_tokens(kept: int, avg_per_trace: int) -> int: """Total trainable tokens across the kept set.""" return kept * avg_per_trace
prompts: int = 200per_prompt: int = 5total: int = prompts * per_promptkept: int = 340
print(total) # -> 1000print(keep_rate(kept, total)) # -> 0.34print(graded_tokens(kept, 180)) # -> 61200Total traces, keep rate, and graded tokens match the hand computation above: 1000 traces, 0.34 kept, 61,200 tokens to train on.
Watch Out For
Right answer through wrong steps slips past the checker
Symptom: the student writes tidy steps that sound right, then lands on the right answer for the wrong reason. You'll spot a made up intermediate number or a cited value that never appeared, wrapped in confident backtracking language it copied from good traces.
Fix: check steps, not just answers. Recompute arithmetic in kept traces, run code where code exists, and hand audit a sample before training. If polished but wrong traces keep passing, tighten the verifier or add process checks from process-vs-outcome-rewards rather than piling on more filtered data.
The student mimics the shape and stalls on new problems
Symptom: training loss falls, demo traces look fluent, yet held out reasoning stays flat. The small model learned longwinded style, extra self correction phrases and hedging, without the underlying procedure. Longer traces, same wrong answers.
Fix: never ship on training loss alone. Hold out fresh prompts, score them with the same verifier, and compare against the undistilled base. Mix trace data with base instruction data so fluency doesn't crowd out facts, keep trace length capped, and match student size to the task. If held out scores won't move, the teacher or the filter is the bottleneck, not the training run.
The Quick Version
- Reasoning distillation trains a small student to predict a large teacher's step-by-step traces with standard SFT instead of fresh RL.
- The dataset keeps only traces whose final answer passes a checker, so verification quality caps student quality.
- In the worked example, 200 prompts times 5 tries gives 1000 traces, 340 kept is a 34 percent keep rate, and 340 times 180 tokens gives 61,200 graded tokens.
- It buys backtracking and self correction at SFT cost, but the student copies teacher flaws and won't pass the teacher.
- Skip it where no checker exists, the teacher is weak, or the task has no steps to teach.