AI Agent Architecture

Why Does This Exist?

Ask a plain chatbot to book a flight under $400 and you get one guess. It writes a confident reply, maybe with a made-up time, because it can't check a real seat. One input, one output, and you're left holding the guess.

A real booking never fits in one guess. You check availability, see a $600 direct against a $400 budget, search again for one-stop options, find $340, and only then answer. That's three passes with one action outside the chat. An agent is a model wrapped in a loop that can do exactly that: call tools, keep notes across steps, and stop only when the answer holds up. Take the loop away and you're back to a chatbot that talks about booking instead of doing it.

This page stays on the loop itself: its four parts and how work moves around them. The call itself lives in tool use and function calling, the notes live in memory systems, the concrete think and act cycle lives in ReAct pattern, and the production stops live in agent harness design.

One idea first, briefly. A model call is stateless. It sees only the tokens in front of it right now and keeps nothing by itself between calls. That's why something outside the model has to carry state forward, and that something is the loop you're about to build.

Think of It Like This

How It Actually Works

Four pieces make the loop, and you've already met them in the picture above. What follows names each one, then runs the $400 booking through three steps with the arithmetic shown.

The four parts

The model is the reasoning core. It reads the current state and picks what happens next, and nothing else. It never touches a flight API directly.

The tools are its only way to reach outside the chat. Search flights, read a calendar, hit a booking API. No tool, no checking. You are back to text about flights rather than flights.

memory is whatever the loop carries forward. The $400 budget, the rejected $600 quote, the fact that one search already ran. If it isn't written into the next input, the model never saw it.

The loop is plain code around the model, often called the harness. It sends state in, reads the model's choice out, runs any tool call for real, appends the result to memory, and decides whether to go around again. Pull the loop out and you get one model call. That's a chatbot.

What the model actually outputs

Call $S_t$ everything the model sees at step $t$. In plain words, the goal plus every past result the harness chose to keep. Call $a_t$ the model's decision at that step, either a tool call or a final answer. Call $o_t$ the tool result fed back in after a call. Call $B$ the budget limit, here $B = 400$ dollars.

At each step the harness does the same small job. It hands $S_t$ to the model, reads $a_t$, and sorts it. If $a_t$ names a tool, the harness runs that tool for real and builds $S_{t+1}$ from $S_t$ plus $o_t$. If $a_t$ is a final answer, the harness stops and returns it. The model chooses with text. The harness acts with code. That split is the whole safety story, because only the harness can check a call before it runs, as tool use and function calling shows in detail.

Worked example

Fix $B = 400$ and follow the picture. You asked for the cheapest option at or under budget.

Step 1 starts with $S_1$ holding the goal and $B = 400$. The model picks $a_1$ as a direct flight search. The harness runs it and gets $o_1$ as a $600 direct. Check it: $600 - 400 = 200$, so the quote sits $200 over budget. Reject it, but keep it in memory. Forgetting it now means proposing it again later.

Step 2 starts with $S_2$ holding the goal, the budget, and the rejected $600. The model picks $a_2$ as a one-stop search. The harness runs it and gets $o_2$ as a $340 one-stop. Check it: $400 - 340 = 60$, so it sits $60 under budget. It also saves $600 - 340 = 260$ against the rejected option. Keep both numbers. The comparison is the point of memory, as memory systems explains.

Step 3 starts with $S_3$ holding both quotes and the budget. The model picks $a_3$ as the final answer: book the $340 option. The harness stops. Call $T$ the total step count. Here $T = 3$ steps with one rejection and one acceptance, which is the same run the diagram draws.

More passes buy more chances to recover, and that's the trade-off. Each extra step costs latency and tokens and gives a confused model one more chance to compound an error. Skip the loop entirely when one independent lookup settles the question. A single weather check or one database read with no follow-up needs a plain tool call, not a loop, because there's nothing for a later step to learn from. The loop assumes tools return timely and parseable results that the harness can feed back. It fails when a tool returns stale prices, an unparsable blob, or a side effect that happens before the failure, like a hold placed on a card before the booking call errors. Counting steps can't un-send that hold, which is why bounds and permissions in agent harness design sit next to the loop rather than inside it.

Code

The point of this snippet is the harness side budget check with memory, using the fixture numbers. It rejects the $600 quote and keeps the $340 one.

1budget: int = 4002memory: list[str] = ["budget 400, need direct flight"]3quotes: list[int] = [600, 340]4
5for q in quotes:6    if q <= budget:7        memory.append(f"accept {q}")8        break9    memory.append(f"reject {q}, over by {q - budget}")10
11print(memory)  # -> ['budget 400, need direct flight', 'reject 600, over by 200', 'accept 340']12print(len(memory))  # -> 313print(600 - 340)  # -> 260

The list ends with three entries: the starting note, the rejection with its $200 overage, and the acceptance. The length is 3 and the saving against the first quote is 260. Those are the same numbers the worked example computed by hand.

Watch Out For

The Quick Version

• An agent is a model plus tools, memory, and a loop. A chatbot is one model call with no tools and no second try.
• The model never runs code. It writes a proposed call and the harness runs it for real, then feeds the result back in.
• Memory is whatever the harness carries into the next step. Anything left out of that input is forgotten.
• In a 400-dollar booking run, a 600-dollar quote is rejected as 200 over and a 340-dollar quote is accepted as 60 under.
• Skip the loop for a single independent lookup. The loop earns its cost only when a later step depends on what an earlier step found.

What to Read Next

• ReAct Pattern
• Tool Use & Function Calling
• Agent Harness Design
• Memory Systems
• Embodied AI