Gen AI

Tool Use & Function Calling

Tool use lets a model ask real code to run something on its behalf, naming what to call and what to pass in, so a program can check it before running.

The model emits a structured call naming a function and its arguments, the harness executes the real function outside the model, and the result flows back into the conversation before the model continues

Why Does This Exist?

Ask a model to plan a trip and at some point it needs today's actual weather in the city you're going to. It doesn't have that. Its knowledge was frozen at training time, and weather isn't the kind of thing you can bake into weights anyway — it changes by the hour. Left alone, the model will still answer. It'll say "expect mild temperatures, maybe around 68°F," in the same confident tone it uses for things it actually knows. Fluent, plausible, and made up.

Tool use closes that gap without pretending the model can do things it can't. Instead of answering from memory, it says, in effect, "I need the real weather for this city and date — someone go get it." A separate piece of code, the harness, goes and gets it: hits a real weather API, gets a real number back, and hands that number to the model to finish the answer with. The model still writes the sentence. It just isn't inventing the fact inside it anymore.

This matters beyond casual chat: live lookups, a calculation the model would botch by hand, a database query, sending an email, booking something. All of it needs an explicit, checkable bridge between "the model wants this to happen" and "this actually happens" — one safe enough to leave unattended.

Think of It Like This

A concierge phoning a specialist instead of guessing

A hotel concierge doesn't track tomorrow's weather in every city a guest might fly to. When asked, the concierge phones the weather line, states exactly which city and day, and reads back whatever the specialist says. No faking the forecast from memory to avoid an awkward pause. Ask a specific question of a system built to know the answer, then relay it.

A model calling a tool plays the concierge, not the specialist. It names the city and date precisely enough for someone else to look up, then waits for the real answer instead of filling the silence with a guess.

How It Actually Works

Declaring a tool as a schema

Before a model can call anything, the harness tells it what's available, and that declaration is a schema, not a code snippet. For the weather lookup: a name (get_weather), a description ("look up the current weather forecast for a specific city and date"), and a typed argument shape — city as a string, date as a string. It's the same idea as structured outputs, applied to the request instead of the response: the model fills in a fixed shape instead of writing free-form text about what it wants.

The description carries more weight than it looks like it should, because it's the only thing the model uses to decide when a tool is relevant. A vague one ("gets weather info") gets ignored where it would help, or triggered where it wouldn't. A specific one gets reached for at the right moments and left alone otherwise.

The round trip

When the model decides the weather tool applies, it doesn't run any code. It emits a structured call — the tool name plus arguments, something like get_weather(city="Lisbon", date="2026-08-14") — formatted as data the harness can parse, not as a snippet it expects interpreted. The harness reads that call, and only the harness executes anything: it hits the real weather API and gets back a real forecast.

That result doesn't vanish into a log. It's inserted back into the conversation as if it were a new message, right where the model reads next. The model continues with an actual forecast in front of it instead of a gap to guess-fill. From the user's side this looks like one turn; underneath it's three handoffs between the model and the harness.

Error recovery and argument grounding

Two things can go wrong with the arguments, and each needs its own fix.

The first is a shape problem: the model forgets date, or sends quantity: "two" where an integer was required. A schema exists exactly to catch this. The harness checks the call before running anything, rejects it on mismatch, and sends an error back into the conversation as data — "missing required field: date." The model reads that and retries. Nothing outside the conversation was ever touched.

The second is a grounding problem, and it's harder, because it passes the schema check cleanly. Say the user only mentioned "the coast," no city named. A model pressured to fill a required city field might write "Lisbon" anyway — a valid string, never actually said. The harness has nothing to reject; the shape is correct. Catching this takes a different check: comparing the argument against what's genuinely in the conversation, not against a type. Structured outputs handles the first failure. Only reading the transcript handles the second.

Show Me the Code

A minimal harness-side check: validate a tool call's arguments against the schema before letting the real function run.

1from typing import Any2
3SCHEMA: dict[str, type] = {"city": str, "date": str}4
5
6def validate_call(args: dict[str, Any]) -> dict[str, str] | None:7    """Return an error dict if args are bad; None means safe to execute."""8    for field, kind in SCHEMA.items():9        if field not in args:10            return {"error": f"missing required field: {field}"}11        if not isinstance(args[field], kind):12            return {"error": f"{field} should be {kind.__name__}, got {type(args[field]).__name__}"}13    return None14
15
16call = {"city": "Lisbon", "date": 20260814}       # date sent as int, not string17error = validate_call(call)18print(error)  # -> {'error': 'date should be str, got int'}

Watch Out For

Executing a real side effect on unvalidated arguments

A schema check that runs only sometimes is no schema check. If the harness calls the real weather API — or a booking or payment API — directly on whatever the model sent, a hallucinated argument reaches a live system before anyone catches it. Validate every call at the code boundary, every time, before the function underneath it runs.

Writing a description too vague for the model to time correctly

A tool description like "handles weather stuff" gives the model nothing to decide with. It'll skip the tool when a forecast would help, or call it in a conversation that never asked for weather. Write it the way you'd explain the tool to a new teammate: what it does, specifically when it applies.

The Quick Version

A tool is declared as a schema: a name, a description the model uses to decide when to call it, and a typed argument shape.
The model never runs code. It emits a structured call naming the tool and its arguments; only the harness executes anything.
The result goes back into the conversation as a new message, and the model continues with real data instead of a guess.
A missing or wrong-shaped argument gets caught by schema validation before execution; the error goes back as a chance to retry.
A hallucinated-but-valid argument — a city the user never said — passes the schema and needs grounding against the transcript instead.