Chain of Thought Agents

Let the agent write its plan in words before it touches a tool. Those written steps are what turn guessing into checking.

One Thought names the next missing number, one Action fetches it, one Observation returns 12 then 72 then 67, so the final answer is built rather than guessed.

Why Does This Exist?

Ask for the answer straight away and you get a confident guess. A stockroom question like how many good pens remain needs three multiplications and a subtraction held in order. A model that must answer in one go has nowhere to put the middle numbers, so it invents them.

Writing the steps down fixes that. Each written line becomes working memory the model can read back. That is the whole trick behind chain of thought: trade a few extra tokens for a place to keep partial results.

Chain-of-thought agents use those lines to steer tools. The Thought picks the next missing piece, the Action calls one tool for it, the Observation returns the fact. Code execution agents use the same pattern when the action is a script, agentic workflow evaluation grades the trace it leaves, and human-in-the-loop agents pause it before a risky call.

This page covers the Thought, Action, Observation loop. Tool choice itself lives with tool use, and the ReAct name for the loop is just shorthand for Reason plus Act.

Basics first: the model writes the next token from the tokens before it, so everything written so far is its memory. If that is new, read about next-token prediction first. Everything below assumes you know that writing more of the right tokens is how the model thinks longer.

Think of It Like This

The packing slip you fill as you count

You count pens in a stockroom. You do not trust your head, so you carry a slip and write each subtotal before moving on.

Boxes times packs per box gives packs. Packs times pens per pack gives pens. Pens minus broken gives good pens. The slip is the Thought trace. Walking to the next shelf is the Action. Reading the shelf label is the Observation.

Where it stops: your slip never lies about what you saw. A model can write a confident Observation it never received. That is why real agents must stop writing and wait for the tool reply instead of completing the slip from memory.

How It Actually Works

Call the goal $g$ , which means the question you asked. Call one Thought $t$ , which means a written line naming the next missing piece. Call one Action $a$ , which means a single tool call with arguments. Call one Observation $o$ , which means the tool reply pasted back into context. The loop is $t$ then $a$ then $o$ , repeated until $g$ is met.

The loop in order

The agent reads $g$ plus all past $t$ , $a$ , and $o$ lines. It writes the next $t$ , for example that it still needs packs before it can count pens. The framework parses the following $a$ , runs exactly that tool, and appends the real $o$ . The next $t$ can then use a number that actually came back, not one the model invented.

Worked example

Stockroom: 3 boxes, 4 packs per box, 6 pens per pack, 5 broken.

Thought 1: packs are missing, so count packs first. Action 1: multiply boxes by packs per box. Observation 1: $3 \times 4 = 12$ packs.

Thought 2: pens are missing, so multiply packs by pens per pack. Action 2: multiply. Observation 2: $12 \times 6 = 72$ pens.

Thought 3: good pens need the broken ones removed. Action 3: subtract. Observation 3: $72 - 5 = 67$ good pens.

A direct answer often says 70 or 66 because it merges the steps. The trace keeps 12 and 72 visible, so a wrong final line can be checked against them. That checkability is the point agentic workflow evaluation later grades.

Reach for this loop when the task needs two or more dependent facts. Skip it when one lookup answers the question, because each loop burns tokens and adds a place to drift.

Code

This loop runs the stockroom count with only the standard library. Each tool is a plain function, and the trace is just a list the agent reads back:

1def count_packs(boxes: int, per_box: int) -> int:2    return boxes * per_box3
4
5def count_pens(packs: int, per_pack: int) -> int:6    return packs * per_pack7
8
9def good_pens(total: int, broken: int) -> int:10    return total - broken11
12
13packs: int = count_packs(3, 4)14print(packs)  # -> 1215total: int = count_pens(packs, 6)16print(total)  # -> 7217print(good_pens(total, 5))  # -> 67

Three calls, three numbers that match the worked example. The agent version wraps the same calls with written Thoughts, but the arithmetic is identical.

Watch Out For

The model writes its own Observation

The trace shows an Action followed at once by a neat Observation, with no tool running in between. The model completed the pattern from memory.

The symptom is a plausible number that never came back from any tool, like 72 appearing when the shelf API timed out. Stop generation at the end of the Action line and let the framework paste the real reply. If your logs ever show Thought, Action, and Observation with the same timestamp, the pause is missing.

The same Action five times in a row

The agent needs a fact it cannot get, so it retries the identical call with the identical arguments and hopes.

The symptom is a trace where Action 3, 4, and 5 match character for character while the Observation repeats its error. Cap total steps, ban exact repeats, and add a fallback Thought such as trying a broader query or asking for help. A human checkpoint is the right fallback when the loop has no good next tool.

The Quick Version

Chain-of-thought agents write a Thought before each Action, so partial results stay visible for later steps.
One cycle is Thought naming the missing piece, Action calling one tool, Observation pasting back the real reply.
Three boxes times 4 packs gives 12 packs, times 6 pens gives 72 pens, minus 5 broken leaves 67 good pens.
Written steps are working memory, and each extra step costs tokens and adds drift.
Never let the model write its own Observation. Pause and paste the real tool reply.
Repeats of the same failing Action need a step cap and a fallback plan.