Episodic and Semantic Memory
Episodic memory keeps the raw transcript of what happened. Semantic memory keeps the lasting facts distilled from it, so your agent still knows you next month without rereading years of chats.
Why Does This Exist?
Every model call starts from nothing. It sees exactly the tokens you hand it in this request and keeps none of them afterward, so an assistant that learned your taste last week can't know it today unless something outside the model carried it forward.
The lazy fix is to paste the whole history into every prompt. That works for one evening, then breaks against the budget: the window is fixed, dead sessions crowd out today's task, and you pay for the same stale tokens on every call. Context budgeting polices that split, and it assumes you know what a context window is, so start there if the frame itself is fuzzy.
The way out is to split the job. Keep the raw record of what happened for the short term, and distill the durable facts out of it for the long term. Episodic memory is the first half, the transcript. Semantic memory is the second, the fact book. This page covers both plus the extractor pass between them. The wider system they belong to lives in memory systems, and the loop that spends both on real tasks lives in AI agent architecture.
Think of It Like This
A doctor's visit notes versus your chart
Your Tuesday visit produces two records. The nurse writes down everything that happened, word for word: you walked in at 2pm, said your head hurt, admitted you'd drunk no water, got told to drink water. That transcript is episodic memory. Then the doctor updates your chart with one line: prone to dehydration. That chart is semantic memory.
Next year you walk in and nobody rereads the Tuesday transcript. The doctor glances at the chart and asks about water first. The mapping holds point for point: transcripts serve the moment, the chart serves every future visit, and a background pass (the doctor, after you leave) does the distilling.
It stops where medicine stops. Your chart never argues with Tuesday's notes, but an agent's facts can contradict each other when you change. A chart also never forgets on purpose, while an agent must, or the fact book fills with trivia. The overwrite rule in the next section is the part with no medical equivalent.
How It Actually Works
Two stores, one background pass, and a budget that explains why both exist.
The two stores
Call the ordered log : messages, tool results, and timestamps, kept exactly as they occurred. is the transcript. It is cheap to write (append and forget) and expensive to read, because nothing in it is indexed by meaning. You search it by recency with a sliding window over the last few turns, or by keyword over older ones.
Call the fact table : one row per durable claim about the user or the world, each stamped with the episode it came from. is the chart. It is expensive to write, because every row costs judgment, and cheap to read, because a dozen rows fit anywhere.
The budget is what forces the split. Name the window: the most tokens one request may hold, input plus output. Name the history lane inside it. Raw history grows without bound while stays fixed, so can never hold more than the recent past. Everything older must either be distilled into or be forgotten. If a lane still overflows after that triage, context compaction shrinks the pieces that remain. That is the whole trade: buys fidelity and costs tokens, buys persistence and costs judgment.
The extractor pass
Nothing moves from to by itself. After a session ends (or on a timer, for long ones), a background pass reads the fresh episodes and proposes rows: new facts get added, confirmed facts get their stamps refreshed, contradicted facts get overwritten. Last write wins, and the stamp is what makes that rule safe: every row points back at the episode that last touched it, so you can always audit a fact back to the words it came from.
This pass is also the bouncer. Most of what happens in a session deserves no permanent row: one renamed variable, one clarified address, one joke. A good extractor writes little and overwrites willingly. If your fact table grows by hundreds of rows a week, the extractor is filing napkin sketches.
Do not build any of this for single-session tools. A summarizer, a classifier, or any task where yesterday says nothing about today pays the extractor's cost for rows it never reads. And never persist what you should re-fetch: stable traits (home city, taste, role) belong in , while volatile values (prices, availability, last night's score) belong in a tool call at read time.
Worked example: two thousand messages become twelve facts
Take a user with 2000 past messages averaging 40 tokens each. That is tokens of raw history. Against a fixture window of tokens, the full log overshoots by a factor of ten. Even a generous 200-message sliding window costs tokens, the entire window, leaving no room for instructions or the answer.
Now distill. Suppose the extractor keeps 12 durable facts averaging 15 tokens each: tokens. That profile is of the window, and times smaller than the raw log. Same user, same history: one fits in a lane and the other never could. The picture above shows the same idea at diary scale, with four raw episodes on the left, three stamped facts on the right, and one background pass between them.
The design assumes past behavior predicts future preference. When you change, old rows lie until some new episode contradicts them, which is why the overwrite rule is load-bearing and not polish.
Code
The point of this snippet is the overwrite rule, the one line the whole section depends on. Newer episodes replace rival rows instead of piling up beside them.
def consolidate(profile: dict[str, str], facts: list[tuple[str, str]]) -> dict[str, str]: updated = dict(profile) for key, value in facts: updated[key] = value # newer episode overwrites; rivals never pile up return updated
profile: dict[str, str] = {"location": "New York", "diet": "likes cilantro"}episode_facts: list[tuple[str, str]] = [("diet", "dislikes cilantro"), ("role", "chef")]
print(consolidate(profile, episode_facts)) # -> {'location': 'New York', 'diet': 'dislikes cilantro', 'role': 'chef'}print(len(consolidate(profile, episode_facts))) # -> 3The cilantro row flips instead of forking, and the role row appends because nothing rivals it. That is last-write-wins in four lines, and the diagram's right panel shows exactly this profile: New York, chef, dislikes cilantro.
Watch Out For
Appending rival facts instead of overwriting
After a year of chats the profile holds both "likes cilantro" and "dislikes cilantro". The symptom is unmistakable: the agent hedges, cites both, or picks one at random each session. It happens because the extractor only ever adds rows and nothing tells it to retract.
Fix it at the write path, not with cleanup scripts. Instruct the extractor to overwrite on conflict, stamp every row with its source episode, and let last write win. Then confirm with one glance per week that no key holds two rivals. Cleanup scripts treat the symptom; the overwrite rule removes the cause.
Distilling a misheard line into permanent fact
One garbled transcript suggests the user is vegetarian, though nobody said that. The extractor files it anyway, and for months the agent dodges every steakhouse with full confidence. The symptom is a confident, persistent, wrong personalization with no visible trail back to the words.
Fix it with provenance and doubt. Keep the source episode id on every row, distill only claims stated plainly (never guesses from one vague line), and re-read the transcript before acting on a high-stakes fact. A fact without a source is a rumor with tenure.
The Quick Version
- Episodic memory is the raw transcript of what happened; semantic memory is the distilled fact book that survives after the window closes.
- Model calls are stateless, so anything the agent "remembers" was written down somewhere and pulled back into the prompt.
- A background extractor moves facts from transcript to profile, overwriting rivals so the newest episode wins.
- A 2000-message history at 40 tokens each is 80000 tokens against an 8000-token window, while twelve 15-token facts cost 180. That ratio is the whole argument.
- Persist stable traits like home city and taste; re-fetch volatile values like prices with a tool call instead.
- Stamp every fact with its source episode, or the first misheard line becomes permanent.