Skip to content
AI360XpertBeta

Reasoning Models

A reasoning model writes out its working before it answers. Those extra thinking tokens buy it time to try steps, catch slips, and only then commit to a final answer.

A reasoning model spends 46 thinking tokens working through Mary's age step by step before committing to 26, buying 46 extra forward passes to check its work
A reasoning model spends 46 thinking tokens working through Mary's age step by step before committing to 26, buying 46 extra forward passes to check its work

Why Does This Exist?

A standard model answers the moment you stop typing. Each new token costs one forward pass through next-token prediction, and when you demand the answer immediately, every bit of hard thinking has to fit inside the few passes that write those answer tokens. That works for one-hop facts. It breaks on anything with two or three moves, because there is nowhere to park a partial result and no way to take back a wrong turn.

Take a tiny word problem. Mary was 5 three years ago, John is twice her current age, and you want John in ten years. You can do it, but only by holding three numbers at once: 8, 16, 26. A model forced to blurt the last number has to get all three right in one shot, with no scratch space. Give it room to write the steps down and the job changes shape.

That's what this page is about: models that write a thinking trace before they answer, how that habit gets trained in, and what it costs. The prompt-time trick of asking for steps lives in chain-of-thought. Training a model to do it by default lives in chain-of-thought training. Spending more compute at serving time is test-time compute, the dial on that spend is reasoning effort controls, and shrinking the habit into a small model is reasoning distillation.

Think of It Like This

Measure twice, cut once

You need one shelf board cut to length for a bookcase. You don't fire up the saw first. You measure the gap, you pencil the length onto the board, you mark where the waste falls, and you read the marks back before you cut. The pencil marks are cheap and erasable. The cut is not.

A question is the gap. Each thinking token is one pencil mark: visible, revisable, and sitting there for the next measurement to use. The final answer is the cut, made once the marks agree.

Where it stops: pencil marks don't guarantee a straight cut. You can misread the tape, copy a number wrong, or draw tidy marks that point at a bad cut. A model can do the same: neat steps that don't actually support the answer. Marks give you room to check. They don't do the checking for you.

How It Actually Works

Here's the deal: one token always costs one forward pass. A reasoning model spends that currency on purpose, writing scratch work before it commits. You'll see the same three numbers from the intro all the way through, so you can check every line.

One token buys one forward pass

Call the question tokens xx, in plain words everything you typed. Call the thinking tokens zz, the scratch work, made of pieces z1z_1 through zTz_T. Call the final answer tokens yy. Here TT is the count of thinking tokens and AA is the count of answer tokens. Call PP the model's next-token distribution, and read ∣\mid as given, so P(y∣x)P(y \mid x) means the chance of the answer given the question.

A direct answer models P(y∣x)P(y \mid x). One shot, no scratch. A reasoning model factors the job as P(z,y∣x)=P(z∣x) P(y∣x,z)P(z, y \mid x) = P(z \mid x) \, P(y \mid x, z). First write the scratch given the question, then write the answer given the question plus the scratch. Because each zz token rejoins the context, the model's attention can reread an early line, spot a slip, and write the fix further down. Words like wait and let me recheck are not feelings. They are learned moves that predict a corrected line will follow.

Count the spend. A direct answer on our age problem uses A=4A = 4 tokens, so 4 forward passes. The reasoning version uses T=46T = 46 thinking tokens plus the same A=4A = 4, so 46+4=5046 + 4 = 50 passes. Those 46 extra passes are test-time compute: time and money traded for room to work.

Training turns the habit into weights

Prompting asks for steps. Training makes steps the default. The recipe has two stages, covered in full in chain-of-thought training.

First, supervised fine-tuning on triples (x,z,y)(x, z, y): question, worked trace, final answer. The loss grades the thinking tokens too, so a lucky guess with no work can't earn full credit.

Second, outcome-reward reinforcement learning. Call the reward RR, a plain 0 or 1: R=1R = 1 when the final answer matches the checkable correct answer, and R=0R = 0 otherwise. The update raises the chance of traces that ended right and lowers the rest. Backtracking survives this filter for a simple reason: traces that catch their own slips end right more often. A common way to run this update is GRPO.

This only works when the answer can be checked. Open-ended writing has no RR you can trust, so the filter can't tell good scratch from tidy fiction.

Products often hide the raw zz and show a short summary instead. The raw scratch can hold disallowed content, wrong turns, and messy self-talk that reads badly out of context. Treat the summary as a progress note, not a transcript.

Worked example: the age problem, step by step

Question xx: Mary was 5 three years ago. John is twice her current age. How old is John in ten years?

The trace zz runs three lines, with one catch along the way.

Line 1: Mary now. 5+3=85 + 3 = 8. The trap is reading 5 as her current age. A good trace writes the fix out loud: 5 was three years ago, so add the 3.

Line 2: John now. 2×8=162 \times 8 = 16. Check it: 2×82 \times 8 is 1616, not 1818.

Line 3: John in ten years. 16+10=2616 + 10 = 26.

Then the answer yy: 26. So T=46T = 46 thinking tokens earn 46 extra forward passes, and A=4A = 4 answer tokens close the job. Total 46+4=5046 + 4 = 50 passes against 4 for a direct guess.

Now the training view. Suppose two sampled traces answer the same xx. Trace one ends at 26, so R=1R = 1. Trace two ends at 24, so R=0R = 0. The update pushes the weights toward trace one. Nothing graded the prose style. The only thing that paid was ending right.

Code

The point here is the check, not the arithmetic. You run the steps as code and grade the ending the way outcome-reward training does: right or wrong, 1 or 0.

def john_in_ten_years(mary_years_ago: int, years_ago: int, years_ahead: int) -> int:    mary_now: int = mary_years_ago + years_ago    john_now: int = 2 * mary_now    return john_now + years_ahead

def outcome_reward(answer: int, correct: int = 26) -> int:    return 1 if answer == correct else 0

print(john_in_ten_years(5, 3, 10))  # -> 26print(outcome_reward(26))  # -> 1print(outcome_reward(24))  # -> 0

The first print proves the trace arithmetic: 5 plus 3 is 8, twice is 16, plus 10 is 26. The next two show the only signal training trusts. Same question, two endings, and the weights move toward the one that scored 1.

Watch Out For

Thinking hard about an easy question

Symptom: you route everything through full thinking and a one-hop fact like the capital of France burns hundreds of tokens before saying Paris. Latency climbs, the bill climbs, and the extra room can even invite a wander away from the obvious answer.

Fix: sort by difficulty before you spend. One-hop facts go to a direct answer or a low effort setting. Save the long trace for multi-step work where the scratch can pay. That's exactly what reasoning effort controls are for.

Quoting the trace as proof

Symptom: the summary reads clean, so you paste it as the justification. Then someone recomputes and finds a step that doesn't follow, or a tidy derivation pointing at a wrong number. The trace looked like reasoning but worked like decoration.

Fix: never trust the trace as proof. Recompute the key line yourself, run the code, or check the source. Use the final answer plus your own check as the evidence, and treat the trace as a hint about where to look.

The Quick Version

  • A reasoning model writes thinking tokens before its answer, turning one guess into steps it can reread and fix.
  • One token costs one forward pass, so a 46-token trace on the age problem buys 46 extra passes and ends at 26.
  • Training pairs supervised traces with outcome-reward updates that keep traces ending in the right answer.
  • Long thinking costs latency, money, and context, so easy questions should skip it.
  • A tidy trace can still be wrong, so the final answer needs its own check.