Agent Memory Types
Agent memory types split what an agent knows into working context for right now, episodic recall of past runs, and semantic facts, then assemble only what fits.
Why Does This Exist?
Every model call starts forgetful. The model reads the prompt in front of it and nothing else. Yesterday's fix, the user's corrected address, the preference stated three turns ago: all gone unless your code puts them back. For one question that hardly matters. For an agent running errands across an afternoon, it breaks everything. The agent re-asks answered questions, repeats dead ends, and contradicts choices it made an hour earlier.
Memory types exist to stop stuffing and start selecting. The naive fix pastes the whole chat into every prompt until the window bursts: 9500 tokens against an 8000 limit, and the run dies. The structured fix sorts knowledge into three stores with different rules, then a small assembler picks a little from each. The result is a 3500 token prompt that fits with 4500 to spare and carries the thread, the lessons, and the facts.
This page covers the three types and the assembly. How the stores are built and indexed lives in memory systems in agents. The loop that consumes the assembled prompt lives in agent architectures, and what moves between agents lives in agent communication protocols.
Basics first: the context window is the maximum tokens one call can read, prompt plus reply together. Everything below is triage under that ceiling. If windows are new, the worked example shows the arithmetic before any jargon arrives.
Think of It Like This
The chef's cutting board, journal, and encyclopedia
A chef works with three surfaces. The cutting board holds tonight's ingredients: chopped, small, wiped clean after service. That is working memory. A journal on the shelf holds dated notes: too much salt last Tuesday, the fix that saved Friday's soup. That is episodic memory. An encyclopedia on the stand holds undated facts: sugar balances acid, poultry rests before carving. That is semantic memory.
Service runs because the chef consults all three without confusing them. The board answers what is happening now. The journal answers what worked before. The encyclopedia answers what is true regardless of the night. Mix them and dinner fails: last Tuesday's salt correction means nothing without its date, and the encyclopedia never mentions that the oven runs hot.
Where it stops: the chef trusts the journal because they wrote it. An agent's episodic store fills with its own guesses, some wrong, all stated confidently. Recalled errors return as advice unless something stamps, scores, and expires them.
How It Actually Works
Name the three stores W for working, E for episodic, and S for semantic. One user message arrives. An assembler pulls a little from each, stamps dates, ranks by relevance, and builds a single prompt. The model answers from that prompt alone.
Working memory: the capped thread
W is the recent conversation plus the current task state, held directly in the prompt. It is fast, exact, and tiny. When the thread grows, older turns get summarized or dropped through a sliding window, keeping roughly the last 5 messages. Working memory answers who said what just now. It cannot answer anything from last week, and pretending otherwise is how windows burst.
Episodic memory: dated past runs
E is a persistent log of events outside the prompt: past conversations, tool results, fixes that worked, mistakes that cost an hour. Each entry carries a timestamp and a short summary. Retrieval keys on the current situation, usually by similarity, and returns a handful of episodes. Episodic memory answers what happened when we saw this before. It needs dates and outcomes, or a March correction will contradict a June move with equal confidence.
Semantic memory: undated facts
S is general knowledge the agent looks up: docs, policies, product facts, usually fetched by retrieval over a store. Entries carry sources, not dates, because water boils at the same point every Tuesday. Semantic memory answers what is true. It never remembers that it told you otherwise yesterday, which is exactly why the assembler keeps it separate from E.
Worked example: 9500 down to 3500
The chat has run to 9500 tokens against an 8000 window. The overage is 9500 − 8000 = 1500, so the full thread cannot travel. The assembler instead builds a 3500 token prompt: the last 5 messages at 2500 tokens from W, the 2 most relevant past episodes at 800 tokens from E, and 1 checked fact at 200 tokens from S. Add them: 2500 + 800 + 200 = 3500. The spare room is 8000 − 3500 = 4500, left open for the reply and the next tool result. The diagram draws exactly this assembly.
The trade is now visible. A bigger pull from E and S adds knowledge and noise together, and buried context gets ignored late in long prompts. Pull little, rank hard, and stamp every recalled line with its date and source. This assumes entries carry both: without timestamps, stale episodes win on similarity, and without sources, facts cannot be checked.
Code
The point here is the triage, not the storage. This stdlib-only sketch holds the three stores, caps the working thread at 5, and assembles the same 3500 token prompt:
class AgentMemory: def __init__(self, max_working: int = 5) -> None: self.working: list[str] = [] self.episodic: list[str] = [] self.semantic: list[str] = [] self.max_working = max_working
def add_turn(self, text: str) -> None: self.working.append(text) self.episodic.append(text) if len(self.working) > self.max_working: self.working.pop(0)
def assemble(self) -> dict[str, int]: plan = {"working": 2500, "episodic": 800, "semantic": 200} plan["total"] = plan["working"] + plan["episodic"] + plan["semantic"] return plan
memory = AgentMemory()for i in range(7): memory.add_turn(f"turn {i}")print(len(memory.working)) # -> 5print(len(memory.episodic)) # -> 7prompt = memory.assemble()print(prompt["total"]) # -> 3500print(prompt["total"] <= 8000) # -> Trueprint(9500 - 8000) # -> 1500The outputs match the worked example: 7 turns keep 5 in working memory while all 7 persist in the episodic log, the assembled prompt totals 3500 inside the 8000 window, and the naive full-thread overage is 1500. In production the token counts come from the tokenizer, but the cap-and-assemble shape is identical.
Watch Out For
Stuffing the window until the middle goes missing
The symptom is an agent that reads everything and uses nothing. Retrieved memories fill the prompt, the key instruction sits buried at token 4000 of 7000, and the answer ignores it. The run looks well-informed and behaves uninformed.
Fix it by ranking retrieved lines ruthlessly and capping each store's contribution before assembly. Keep the current task and the newest constraint near the end of the prompt, where the model attends best, and cut anything the present step cannot use.
Old episodes contradicting new facts
The symptom is confident time travel. The user moved in June, but a January episode about the old address scores higher on similarity and the agent ships the answer to the wrong city. Nothing errors, so the mistake reads as success until a human checks.
Fix it by stamping every episodic entry with a date, weighting recency alongside similarity, and writing explicit supersede rules for facts that change, like addresses and versions. Retrieve the past, then ask which of it is still true.
The Quick Version
- Agent memory splits into working context for now, episodic recall of dated runs, and semantic facts without dates.
- Working memory holds about the last 5 messages. Episodic recall returns a few similar episodes. Semantic lookup returns checked facts.
- A 9500 token thread overflows an 8000 window by 1500, while an assembled 3500 token prompt fits with 4500 spare.
- Rank retrieved lines hard and cap each store, or buried context gets ignored.
- Stamp every recalled episode with its date, or old addresses beat new ones on similarity alone.
- How the stores get indexed lives in memory systems, while this page covers what each type is for.