Code Execution Agents

Why Does This Exist?

A model predicts words, not sums. Ask it for the average of 120, 80, 200, and 150 and it may say 140 with full confidence. The shape looks right and the arithmetic is wrong. That failure grows with data size: a five-megabyte log cannot be counted by eye, and a guess is worthless when money or access depends on it.

Code execution agents stop guessing and start running. The agent writes a small script, the framework runs it in an isolated sandbox, and the printed number comes back as an observation. Chain-of-thought agents supply the plan that decides a script is needed, agentic workflow evaluation later scores whether the run was tidy, human-in-the-loop agents gate the risky runs, and groups of such agents can show emergent behavior when they share results.

This page covers the write, run, and read-back cycle. It does not cover which sandbox to buy or how to orchestrate many agents. Those are deployment choices that sit on top of this loop.

Basics first: a sandbox is an isolated place to run untrusted code, with no access to your files or keys. If that is new, think of it as a disposable computer that is wiped after each run. Everything below assumes code never runs on your own machine directly.

Think of It Like This

How It Actually Works

Call the plan $p$, which means the written decision that a script is needed. Call the script $s$, which means the code the model wrote. Call the result $r$, which means the text the sandbox printed back. The loop is $p$ then $s$ then $r$, and $r$ becomes the fact the next Thought uses.

From words to a run

The agent first writes $p$: load the four revenues and average them. It then writes $s$ in a fenced block the framework can extract. The framework takes $s$ into the sandbox, runs it with a time limit, and pastes $r$ back. If $r$ is a stack trace, the agent reads the error, fixes $s$, and tries again.

Worked example

Revenues: 120, 80, 200, 150.

Add them: $120 + 80 + 200 + 150 = 550$. Count them: 4. Divide: $550 / 4 = 137.5$.

A word-only answer often says 140 because it blends the numbers. The script returns 137.5 every time, and the diagram shows that number travelling back into the chat as the trusted value. The same loop scales to a file with a million rows, where the only change is that $s$ reads the file instead of listing four numbers.

Use this loop when the answer needs exact math, filtering, or parsing. Keep it off when a sentence is enough, because each run costs time and needs guardrails.

Code

This snippet is the trusted part of the loop with only the standard library. It averages the four numbers and shows the retry signal a sandbox returns:

1def mean(values: list[float]) -> float:2    return sum(values) / len(values)3
4
5revenues: list[float] = [120.0, 80.0, 200.0, 150.0]6print(mean(revenues))  # -> 137.57print(sum(revenues))  # -> 550.08print(len(revenues))  # -> 4

Three prints, one fact. The sum is 550.0, the count is 4, and the mean is 137.5. An agent would paste that 137.5 back as the observation instead of defending a guessed 140.

Watch Out For

The Quick Version

• Code execution agents write a script $s$ from a plan $p$ and read back the sandbox result $r$ as fact.
• Word-only math guesses. The script for 120, 80, 200, and 150 returns 550.0 over 4, which is 137.5 every time.
• Errors come back as text too, so the agent can fix the script and retry.
• Every run needs a timeout and a memory cap, because models write infinite loops.
• Never run agent code with your own permissions. Isolate it and gate risky calls with human approval.
• The loop is for exact work: math, filtering, and parsing. Sentences do not need it.

What to Read Next

• Chain of Thought Agents
• Agentic Workflow Evaluation
• Human-in-the-Loop Agents
• Emergent Agent Behavior