Chain of Thought
Ask the model to show its work before its answer. Those written steps become a scratchpad the model can reuse, so hard problems get solved piece by piece.
Why Does This Exist?
A language model writes one token at a time, always reading the tokens before it. That is next-token prediction: the full treatment lives there, and here you only need the one consequence.
If you demand the final answer as the very next tokens, you squeeze every bit of reasoning through a single pass. There is nowhere to park a partial sum. No chance to fix a slip. No extra compute. That works for a one-hop fact. It falls apart on anything with two or more moves, which is why direct answers go wrong on math word problems and multi-step logic.
Chain of thought changes the shape of the job. You ask for the steps first and the answer last, so each emitted step joins the context the next token can read. You have seen this in your own work: the scratchpad holds the parts while you assemble the whole.
This page covers that prompting move: the mechanism, the two ways to trigger it, and one hand-checked example. Voting over many chains belongs to self-consistency. Branching and backtracking belong to tree of thoughts. Training the habit into the weights belongs to reasoning models.
Think of It Like This
Splitting the bill on a napkin
You are out with friends and the bill needs splitting. Do it all in your head and you will drop something. Grab a napkin instead. You write each share on its own line, then you add the lines. You keep glancing back at what you wrote, and the total at the bottom is easy because the lines above it are visible.
The question is the bill. Each reasoning step is one line on the napkin. The final answer is the total at the bottom.
Where it stops: ink on a napkin does not lie, but model text can. You can still mis-add, invent a line, or write tidy steps that do not actually support the total. The napkin gives you room to work. It does not check your arithmetic.
How It Actually Works
Call the question tokens , in plain words everything you typed. Call the rationale tokens , in plain words the steps the model writes before answering, made of pieces . Call the final answer tokens . Call the model's next-token distribution, and read as given, so means the chance of the answer given the question.
Direct answering models . One shot. All the reasoning stays hidden inside a single forward pass, and the model must land on the answer with no place to set anything down.
Chain of thought models , which factors as . Read it left to right: first write steps given the question, then write the answer given the question plus the steps. Each step token costs another forward pass, so the model spends more compute on harder questions. That is the whole mechanism. Nothing mystical. The context window is the scratchpad.
Zero-shot: one cue
Append one line to the question: "Let's think step by step." No examples. The model falls back on decompositions it already saw in training and writes its own steps.
Use this when the task splits the obvious way and you have no format to enforce. Do not expect it to invent your house format. It picks the breakdown, not you.
Few-shot: show the shape
Give one to three worked exemplars, each shaped as question, steps, then a final-answer line, then put your real question last. This is few-shot prompting aimed at reasoning: you are teaching the shape of the work, not just the shape of the output.
Use this when the steps must follow your procedure, such as a fixed arithmetic layout or a domain checklist. The model copies the pattern you demonstrated.
Worked example: 14 times 16
Take and force the split . That distributive step is the rationale . Then:
Step 1: . Tens first. Nothing to carry.
Step 2: . Check it: and , and .
Final: . Hundreds with hundreds, tens with tens: , plus gives .
A direct answer has to produce with no intermediate tokens, and in the diagram it lands on , wrong by ten. The stepped version never holds more than one small product at a time, and each result stays in context for the addition. Same model, same weights. Different conditioning.
The trade is explicit. You pay for every step token in latency and cost, so you buy accuracy with output length. It helps on decomposable multi-step tasks with a model large enough to follow the decomposition. It assumes the written steps actually support the answer. When that assumption fails you get fluent nonsense with tidy numbering, which is why the pitfalls below matter.
Code
This snippet builds the cue, extracts a final-answer line, and checks the arithmetic from the worked example. No API calls. Run it as is.
def build_cot_prompt(question: str) -> str: return f"Q: {question}\nThink step by step, then write 'Final answer: <value>'.\nA:"
def extract_answer(draft: str) -> str: marker: str = "Final answer:" return draft.split(marker)[-1].strip()
prompt: str = build_cot_prompt("14 * 16 = ?")print(prompt)# -> Q: 14 * 16 = ?# -> Think step by step, then write 'Final answer: <value>'.# -> A:
draft: str = "14 * 10 = 140; 14 * 6 = 84; 140 + 84 = 224. Final answer: 224"print(extract_answer(draft)) # -> 224print(14 * 10, 14 * 6, 140 + 84) # -> 140 84 224The point is the contract, not the strings. Force a labeled final line so your code can find it, then verify the numbers yourself instead of trusting the prose around them.
Watch Out For
Spending steps on one-hop questions
You ask for the capital of a country or a single lookup, add "think step by step," and get a paragraph of throat-clearing before the same answer. Symptom: slower responses, larger bills, and sometimes a worse answer because the extra tokens gave the model more room to drift.
Fix: match the tool to the task. Direct answer for single facts and short rewrites. Save the scratchpad for questions with at least two moves, where an intermediate result must feed a later one.
Trusting neat steps that do not add up
The rationale reads beautifully and the arithmetic is wrong. Symptom: numbered steps, confident tone, and a total the steps do not entail, like writing 140 and 84 yet concluding 214. Skimming the shape fools you into skipping the sum.
Fix: check the numbers with code, not your eyes. Require the exact final-answer line from the snippet above, recompute each line independently, and for anything that matters sample twice or vote across chains with self-consistency. Never let tidy formatting stand in for a verified total.
The Quick Version
- Chain of thought asks for the reasoning steps before the final answer so each step becomes context for what follows.
- Direct answering squeezes everything through one pass, which is why it fails on multi-move problems it could solve stepwise.
- Zero-shot uses one cue with no examples, while few-shot demonstrates the exact step shape you want copied.
- Splitting 14 times 16 into 140 and 84 before adding to 224 is the pattern: one small product at a time, then the sum.
- Steps cost tokens and time, so skip the scratchpad on single facts and demand verifiable arithmetic where it counts.