Skip to content
AI360XpertBeta

Reasoning Effort Controls

A reasoning model thinks in hidden tokens before it answers. The effort dial sets how many of those tokens it may spend, so you choose between a quick guess and a checked answer.

Turning reasoning effort from low to high grows the hidden thinking budget from 150 tokens on one draft to 900 tokens across three drafts plus a verification check
Turning reasoning effort from low to high grows the hidden thinking budget from 150 tokens on one draft to 900 tokens across three drafts plus a verification check

Why Does This Exist?

Not every question deserves the same amount of thought. Sorting five words deserves a snap answer. Rewriting a locking mechanism deserves drafts, doubt, and a second pass. If you pay for deep thought on everything, you burn money and keep users waiting. If you pay for none of it, the hard questions come back wrong.

Reasoning models made this tunable. They spend hidden tokens thinking before they answer, so more tokens can mean a more carefully built answer. Test-time compute is the general trick behind that trade. This page is about the dial itself: what the low, medium, and high settings actually buy you, and how to pick one.

The one idea you need first: a reasoning model writes intermediate tokens you never see, then writes the answer you do. Those hidden tokens are the thinking, and they are billed like any other token. The full treatment is in reasoning models.

Think of It Like This

One sheet of scratch paper versus a whole pad

Picture a maths exam. In the first room you get a single sheet of scratch paper. You try the problem once, straight through, and whatever you land on goes in the answer box. In the second room you get a whole pad. You try three approaches, cross two out, recheck the survivor line by line, and then write the answer.

The sheet is the budget. Each attempt is a draft. Crossing out is discarding a bad branch. Rechecking is verification. The answer box is the final output, and it costs the same ink either way.

Where it stops: scratch paper never taught anyone maths. A bigger budget buys more attempts, not more knowledge. If the model does not know the fact at low effort, high effort will not invent it. It just gets more chances to assemble what it already knows.

How It Actually Works

The dial sets a cap, call it BB, on the hidden reasoning tokens for one call. The model then uses some number of reasoning tokens RR, with RR never above BB, and writes an answer of AA tokens. You are billed for the total T=R+AT = R + A. Turn the dial up and BB rises; turn it down and the model must go straight at the answer.

The dial is a token cap

Vendors expose this as named levels such as low, medium, and high, or as a numeric thinking budget in tokens. Exact caps differ per product and change over time, so forget memorising them. The shape is what matters: a small BB forces one straight pass, and a large BB permits drafts plus checks. Every number on this page comes from one running example, stated next, so prose, diagram, and code all agree.

What extra tokens buy

A big budget is spent on three moves. The model drafts several candidate lines of attack instead of one. It checks steps against each other instead of trusting the first version. And it backtracks: when a branch leads somewhere silly, it drops that branch and spends the remaining budget on another one. That is the whole difference between the two panels above. Low effort is one draft and an answer. High effort is three drafts, two crossed out, one check, and the same size of answer.

Worked example

Take a low budget of B=150B = 150 tokens and a high budget of B=900B = 900 tokens, with the visible answer fixed at A=120A = 120 tokens in both cases.

At low effort the model writes one draft of 150 tokens and stops. Reasoning used is R=150R = 150, and the billed total is T=150+120=270T = 150 + 120 = 270 tokens.

At high effort the model writes three drafts of 220 tokens each, drops two, keeps one, and spends 240 tokens rechecking the survivor. That is 3×220=6603 \times 220 = 660 draft tokens plus a 240 token check, so R=660+240=900R = 660 + 240 = 900. The billed total is T=900+120=1020T = 900 + 120 = 1020 tokens.

Now the two ratios that run your bill. Thinking grows 900/150=6900 / 150 = 6 times over. The total grows 1020/270≈3.781020 / 270 \approx 3.78 times over, less than sixfold because the answer is fixed. If the endpoint streams at an assumed 50 tokens per second, the wait is 270/50=5.4270 / 50 = 5.4 seconds at low and 1020/50=20.41020 / 50 = 20.4 seconds at high. That speed is an assumption to make the wait concrete, not a product claim.

The trade is now plain: high effort buys drafts and checks with tokens, money, and latency. Skip it where there is nothing to check. Lookup, formatting, and classification do not get better with a bigger BB. Spend it where an answer can be tested before it ships: sums that can be re-added, code with constraints that can be re-read, plans with steps that can be replayed. And note the assumption this rests on: the extra tokens must actually go into checking. When they do not, you get the failure in the second card below.

Code

The point of this snippet is the bill, not the API. Given the fixture above, totals follow from one addition, and the ratio shows most of the high effort price is thinking.

def total_tokens(reasoning_tokens: int, answer_tokens: int) -> int:    """Billed tokens for one call: hidden thinking plus the visible answer."""    return reasoning_tokens + answer_tokens

answer_tokens = 120low = total_tokens(150, answer_tokens)  # one draft, no recheckhigh = total_tokens(3 * 220 + 240, answer_tokens)  # three drafts plus a checkprint(low)  # -> 270print(high)  # -> 1020print(high / low)  # -> 3.7777777777777777

Low costs 270 tokens and high costs 1020, so high bills about 3.78 times more for the same 120 token answer. The 900 hidden tokens are the purchase; the drafts and the check are what they buy.

Watch Out For

High effort on a question with nothing to check

The symptom is a familiar receipt: the answer matches what low effort gave you, but the call used roughly four times the tokens and kept you waiting. "Translate this sentence" does not improve with three drafts and a verification pass. There was never a branch to discard.

Route by task shape instead of setting one level everywhere. Default to low for lookup, formatting, and classification. Reach for high when the task has steps worth doubting, like chain of thought problems, multi-file edits, or anything you would recheck yourself. If you cannot name what the model should verify, low is the right dial.

A long trace with no checking in it

The symptom is worse: a 900 token trace, a confident answer, and no discarded branch anywhere inside. The model filled the budget by restating the question three ways instead of testing anything. Longer thinking looked like better thinking right up until the answer came back wrong.

Treat an unchanged answer as information. Run the same prompt at two effort levels; if the answer does not move, the task is not search-shaped, so stop paying. Reasoning distillation is the cheaper fix for routine hard tasks: a small model taught on a teacher's checked traces, instead of a big thinking bill on every call. And when agents do the spending, the same budgeting instinct lives in agent cost control.

The Quick Version

  • Reasoning effort is a cap on hidden thinking tokens, not a smarter model. Low means one quick draft; high means drafts plus checks.
  • Billed tokens are thinking plus answer, so 150 plus 120 is 270 at low and 900 plus 120 is 1020 at high in this page's example.
  • Spend high effort where an answer can be tested before it ships: sums that re-add, code with re-readable constraints, plans with replayable steps.
  • Keep low effort for lookup, formatting, and classification. Extra thinking cannot fix missing knowledge.
  • If raising the dial leaves the answer unchanged, the task is not search-shaped. Stop paying.