Skip to content
AI360XpertBeta

Tool Result Curation

A tool can hand back a phone-book-sized answer. Curation trims it to the few lines the model needs next, before those lines re-enter the prompt.

A 12,600-token raw tool reply is curated to a 200-token observation by keeping 3 fields from the top 5 of 200 rows, fitting the 2,000-token budget
A 12,600-token raw tool reply is curated to a 200-token observation by keeping 3 fields from the top 5 of 200 rows, fitting the 2,000-token budget

Why Does This Exist?

You ask your travel agent for cheap flights to Tokyo. It calls a flight search tool, and the tool answers with everything: 200 offers, each stuffed with logo URLs, legal paragraphs, and internal IDs. That one reply runs about 12,600 tokens, and the lane you set aside for a single observation holds 2,000. Send it raw and you either crash into the limit or drown the next thought in noise.

That loop is the ReAct pattern: the model writes a Thought, makes one call through function calling, then reads the observation before thinking again. The observation is the only part of that loop your code fully controls, so it is where you defend your context budget. Skip that defense and you get the failure named in context degradation: the model quotes a baggage-fee paragraph as the departure time, because the real time sat buried in the middle of 200 rows.

This page covers that one shaping step: turning a raw reply into an observation the next Thought can use. How the model places the call belongs to function calling. How you split the whole window belongs to context budgeting. How you shrink free text with a second model pass belongs to context compaction. Here you keep rows and fields, on purpose, with rules you can read.

Think of It Like This

The news desk and the reporter's notebook

You (the model) ask your reporter (the tool) for tonight's scores. The reporter drops a 40-page notebook on your desk: every play, every quote, every coffee stain.

You don't read the notebook. Your desk editor (the wrapper function) reads it and slides you a three-line brief: the scores, the standings, and one line saying where the full notebook sits and how it is sorted. You write from the brief.

Where it stops: a real editor uses news judgment and can phone back for more. Your wrapper follows fixed rules and cannot. It keeps exactly what you told it to keep, so a wrong rule deletes the answer without telling you. That is why the brief always names its sort order and its total.

How It Actually Works

Call the raw tool reply RR: everything the API returned, junk included. Call what the model will actually read the observation OO: the short note your wrapper builds from RR. Write T(x)T(x) for the token length of any text xx, counted the way your tokenizer counts. Set aside a budget lane BobsB_{obs} for one observation, the number context budgeting gave you. The rule is one line: send OO only when T(O)T(O) sits under BobsB_{obs}. Everything below serves that line.

Three moves that cover most replies

  1. Project: keep the keys the next Thought asked for and drop the rest. A flight row with 27 keys becomes 3: airline, price, and time. Copy them exactly; don't rephrase them.
  2. Truncate: keep the top kk rows under a stated sort, where kk is the small number you keep, and always attach the total found plus the sort order. Five cheapest with "200 found, sorted by price" beats a silent five.
  3. Compress: for free text with no keys to keep, summarize with a cheaper pass, as context compaction describes. Prefer the first two moves for structured rows: a summarizer can round a price or a time, while projection copies them digit for digit.

Skip all three when the raw reply already fits with room to spare. Curation then adds failure surface for no gain. Each move assumes you know what the next Thought needs: projection assumes the right keys, truncation assumes the right sort. Break the assumption and you hide the answer, which is what the pitfalls below are about.

Worked example: 200 flights in, 5 out

Your flight tool returns 200 offers. Each averages about 63 tokens of JSON once the junk keys are counted, so the raw reply runs:

200×63=12,600200 \times 63 = 12{,}600

Your budget lane holds Bobs=2,000B_{obs} = 2{,}000 tokens. The raw 12,600 does not fit, by a factor of six.

You project each row to 3 fields (airline, price, time) at about 32 tokens each, keep the top k=5k = 5 cheapest, and wrap them in a 40-token envelope carrying status, total, and sort:

5×32=160and160+40=2005 \times 32 = 160 \quad\text{and}\quad 160 + 40 = 200

So T(O)=200T(O) = 200 tokens, which fits the 2,000-token lane with room to spare. The saving is (12,600−200)/12,600≈0.984(12{,}600 - 200) / 12{,}600 \approx 0.984, about 98.4 percent smaller. Same flights, same question, one tenth of one percent of the confusion.

Code

The point of this snippet is the shape of a curating wrapper: rank, project to a slim schema, slice to the top kk, and always return the total with the sort. It builds the 200-row reply from the example, so the counts it prints are the example's counts.

from typing import Any

def curate_flights(raw: list[dict[str, Any]], k: int = 5) -> dict[str, Any]:    ranked = sorted(raw, key=lambda r: r["price_usd"])    top = [        {"airline": r["airline"], "price": r["price_usd"], "time": r["departure_time"]}        for r in ranked[:k]    ]    return {"status": "success", "total_found": len(raw), "sort": "price_asc", "top": top}

raw = [    {        "airline": f"Airline-{i % 8}",        "price_usd": 380 + (i * 37) % 420,        "departure_time": f"{10 + (i % 9)}:00",        "logo_url": "https://cdn.example/logo.png",        "legal": "Terms apply.",        "pilot_id": f"p-{i}",    }    for i in range(200)]
obs = curate_flights(raw, k=5)print(len(raw))  # -> 200print(len(obs["top"]))  # -> 5print(obs["total_found"])  # -> 200print(obs["top"][0])  # -> {'airline': 'Airline-0', 'price': 380, 'time': '10:00'}

Two hundred rows go in, five slim rows come out, and the printed top row is the cheapest of the 200. The envelope tells the model what happened: all 200 were seen, sorted by ascending price, five shown.

Watch Out For

You deleted the field the next thought needed

The symptom is a confident quote of something that never appeared in the observation. The model is not scheming; you removed every departure time and left it nothing to quote, so it filled the gap. This bites hardest right after you add a new question type without updating the projection.

Fix it by projecting from the question, not from the payload. Keep the identifiers plus each field the Thought asked for, and log the dropped keys while you develop. When the question is what time the USD 450 flight leaves, price and time both stay.

A silent top five looks like all there is

The symptom is the model writing there are no other flights when 200 matched. You sliced the list but never said so, so five reads as everything and the agent stops searching a store full of options.

Fix it by attaching the triple every time: never send a bare sliced list. Total found, sort order, and one line on how to narrow, such as asking for a date or a price cap. That triple is the 40-token envelope in the example above, and it is the cheapest accuracy you will buy.

The Quick Version

  • Tool result curation trims one raw tool reply to the fields and rows the next thought needs, before that text re-enters the prompt.
  • Send the observation only when its token length fits its budget lane: 200 tokens fits a 2,000-token lane, and 12,600 tokens does not.
  • Project rows to a slim schema, keep the top k with the total and the sort order, and save summarization for free text with no keys to keep.
  • Deleting the needed field causes invented facts: a model denied every departure time will quote one anyway, so keep what the question asked for.
  • A bare top five reads as everything: always state how many matched and how to ask narrower.