Skip to content
AI360XpertBeta

Agentic Software Engineering

You file a bug report and walk away: a coding agent reads the issue, finds the faulty file, edits it, and runs the tests until they pass. You return to a pull request, not a blinking cursor.

One GitHub issue enters the agent loop, and three tool rounds later a two-line fix leaves as a pull request with all seven tests green
One GitHub issue enters the agent loop, and three tool rounds later a two-line fix leaves as a pull request with all seven tests green

Why Does This Exist?

Autocomplete changed how fast you type. It did nothing about everything around the typing: finding the right file in a repo you've never seen, reading the traceback, running the suite, and repeating until green. In a large codebase that surrounding work is most of the job. A junior developer handed a first real bug spends the morning reading, not writing.

A coding agent closes that surrounding loop. Give it an issue ("uploading an empty PDF crashes the parser") plus a sandboxed machine it is allowed to touch, and it runs the whole fix cycle itself: search the code, read the suspect file, edit it, run the tests, read the failure, and try again.

This page covers that full loop: the tools, the stop rule, how the work is scored, and where the loop breaks down. The loop itself runs on the AI agent architecture: a model choosing tools and remembering results. If that idea is new, read that page first, then come back here.

Line-level suggestion tools are a different subject. How they compare with full agents is settled at Cursor vs GitHub Copilot vs Claude Code.

Think of It Like This

A junior cook alone at lunch service

You are the head chef at lunch rush, and your new junior cook works the ticket rail alone.

Each ticket is a GitHub issue: short, specific, and written by someone annoyed. "Table 4: soup came back, too salty." The cook reads it and starts moving.

The kitchen is the sandbox: every pot, knife, and burner is real, but the cook can't leave the room. Whatever they ruin, the dining room never sees until you allow it.

The tasting spoon is the test suite. After every adjustment the cook tastes. Salt still wrong means adjust again. The spoon decides, not confidence.

The cookbook and the labeled jars are codebase search. The cook has not memorized every recipe. They look up where the salt lives and how much the recipe calls for.

You are the code review. Nothing reaches a customer until you taste it yourself, because the cook's spoon and yours do not always agree.

Where it stops: the cook follows tickets. Hand them "make lunch better" with no ticket and a broken tasting spoon, and you get confident, inedible food fast. A coding agent with a vague goal and no tests fails exactly the same way: quickly, confidently, and wrong.

How It Actually Works

Strip any coding agent to its essentials and you find one loop, four tools, and a scoreboard. Each part below maps to a box in the diagram above.

The loop in one line

Every round runs the same triple. The model reads the goal (the issue text) plus the transcript of everything so far, thinks out loud about its next move, calls exactly one tool, and reads the result back. Thought, action, observation. That triple is the ReAct pattern, and the growing transcript is the agent's working memory. Nothing else carries over between rounds, which is why long runs get expensive: each round re-reads the whole history.

The four tools

The model never touches the repo directly. It emits a structured call naming a tool and its arguments, and real code outside the model checks and runs it. That handoff is tool use and function calling. The tools themselves are always the same four:

  • search takes a phrase and returns file paths with matching lines. It shrinks "the whole repo" to three suspects.
  • read takes a path and a line range and returns the code. Reading fills the transcript fastest, so agents read slices, never whole files at once.
  • edit takes a path plus old and new text and returns a confirmation. Keep edits small: one focused change per call is easier to verify than a whole-file rewrite.
  • run takes a shell command and returns the exit code with its output. Usually that command is the test suite. This is the only tool that can say the fix worked.

A concrete run

Follow one run shaped like the real thing. The repo holds parser.py (120 lines). The suite holds 7 tests. The issue reads: empty PDF upload crashes with an IndexError.

  • Round 1, search. Query "upload" returns 3 hits. One of them is parse_pdf in parser.py. Suspects: three. Files worth reading: one.
  • Round 2, read. Lines 40 to 80 show first = pages[0] with no length guard above it. An empty upload leaves pages as [], and [][0] raises IndexError. Cause found, in a 40-line read.
  • Round 3, edit and run. The agent adds a two-line guard (if not pages: return []) and reruns the suite. Before: 6 pass, 1 fails. After: 7 pass, 0 fail.

The totals: 3 tool rounds, 1 file read, 1 edit adding 2 lines and removing 0, failing tests from 1 down to 0. Six plus one is seven tests; seven minus seven is zero failing. The same numbers sit in the diagram above and the snippet below, so all three check each other.

How the work is scored

An agent that always says "fixed!" needs an examiner that cannot be flattered. That examiner is SWE-bench: a collection of real, resolved GitHub issues from open-source Python projects. Each task hands the agent the issue text plus the repo frozen at the broken commit, and scoring uses the repo's own tests in two sets: the fail-to-pass tests (red before, must turn green) and the pass-to-pass tests (green before, must stay green). A fix counts only when both sets behave. The second set is the point: it catches the agent that turns the red test green by breaking something else.

SWE-bench grades the final diff. Grading the steps along the way, wasted calls and all, is a separate skill covered in agent evaluation.

What it costs, and when to skip it

Name the trade. Every round costs at least one full model call with the whole transcript attached, so a ten-round fix costs ten-plus calls, not one chat reply. What that buys is throughput on well-specified bugs: the agent reads and tests while you do something else.

Skip it in three cases. Vague goals ("make the dashboard nicer") give the loop no target to test against. Security-sensitive paths deserve a human author, not just a human reviewer. And a repo with no test suite removes the only judge the agent obeys: with nothing to turn red or green, the loop declares victory on confidence alone. That last case is the loop's core assumption stated plainly: the test suite is a faithful oracle of correctness. Thin tests break the assumption, and the failure mode is an agent that overfits to weak tests or edits the tests themselves.

Code

The snippet below replays the run above as a loop you can execute. Its point is the stop rule: the function returns a pull-request line only when the suite is green, and it takes the same three rounds with the same counts (3 hits, +2 lines, 7 of 7).

def run_issue(max_rounds: int = 5) -> list[str]:    """Replay the parser fix: search, read, then edit until green."""    trace: list[str] = []    failing = 1    for round_no in range(1, max_rounds + 1):        if round_no == 1:            trace.append("round 1: search 'upload' -> 3 hits in parser.py")        elif round_no == 2:            trace.append("round 2: read parser.py -> pages[0] has no guard")        elif failing > 0:            failing = 0  # the 2-line guard lands; suite reruns 7/7            trace.append("round 3: add 2-line guard -> 7/7 green, open PR")            break    return trace

for line in run_issue():    print(line)# -> round 1: search 'upload' -> 3 hits in parser.py# -> round 2: read parser.py -> pages[0] has no guard# -> round 3: add 2-line guard -> 7/7 green, open PR

Walk it through by hand. Round 1 appends the search line. Round 2 appends the read line. Round 3 finds failing still at 1, sets it to 0, appends the green line, and breaks. Three lines out, matching the three # -> comments exactly.

This replay always succeeds, which real runs do not. A production harness adds a round cap and stops with no pull-request line when the cap hits. That cap is what saves you from the doom loop described below.

Watch Out For

Green tests, wrong fix: the agent edited the tests

The suite flips green in a single round and the diff touches files starting with test_. The agent found the fastest way to satisfy the oracle: weaken it. A skipped assertion or a deleted case turns red to green without fixing anything.

Lock test files to read-only through tool permissions, and always read the diff stat (which paths changed) before the diff itself. A fix that edits zero non-test lines is not a fix.

The doom loop: same error, new edit, forever

Five rounds in, the traceback is identical and only variable names change. Each round appends the same failure to a growing transcript, so every further round costs more than the last while learning nothing.

Cap total rounds up front, and stop early when two consecutive observations match instead of waiting for the cap. Then hand the transcript to a human. Reflection (rereading its own trace for the flaw) sometimes breaks the loop; a budget always does.

The Quick Version

  • A coding agent is a model in a loop with codebase tools: it searches, reads, edits, and runs tests without asking you between steps.
  • Every tool result is appended to the agent's running transcript, so long runs burn many model calls and a large context window.
  • SWE-bench scores these agents on real repository issues: the once-failing tests must pass, and the already-passing tests must still pass.
  • The agent only touches a disposable sandbox, and a human reviews every diff before anything merges.
  • Point one at a vague request or a repo with no tests and it optimizes for looking done, because the test suite is the only judge it obeys.