Skip to content
AI360XpertBeta

Episodic and Semantic Memory

Episodic memory keeps the raw transcript of what happened. Semantic memory keeps the lasting facts distilled from it, so your agent still knows you next month without rereading years of chats.

Episodic memory keeps raw chat transcripts while a background extractor distills lasting facts into semantic memory
Episodic memory keeps raw chat transcripts while a background extractor distills lasting facts into semantic memory

Why Does This Exist?

Every model call starts from nothing. It sees exactly the tokens you hand it in this request and keeps none of them afterward, so an assistant that learned your taste last week can't know it today unless something outside the model carried it forward.

The lazy fix is to paste the whole history into every prompt. That works for one evening, then breaks against the budget: the window is fixed, dead sessions crowd out today's task, and you pay for the same stale tokens on every call. Context budgeting polices that split, and it assumes you know what a context window is, so start there if the frame itself is fuzzy.

The way out is to split the job. Keep the raw record of what happened for the short term, and distill the durable facts out of it for the long term. Episodic memory is the first half, the transcript. Semantic memory is the second, the fact book. This page covers both plus the extractor pass between them. The wider system they belong to lives in memory systems, and the loop that spends both on real tasks lives in AI agent architecture.

Think of It Like This

A doctor's visit notes versus your chart

Your Tuesday visit produces two records. The nurse writes down everything that happened, word for word: you walked in at 2pm, said your head hurt, admitted you'd drunk no water, got told to drink water. That transcript is episodic memory. Then the doctor updates your chart with one line: prone to dehydration. That chart is semantic memory.

Next year you walk in and nobody rereads the Tuesday transcript. The doctor glances at the chart and asks about water first. The mapping holds point for point: transcripts serve the moment, the chart serves every future visit, and a background pass (the doctor, after you leave) does the distilling.

It stops where medicine stops. Your chart never argues with Tuesday's notes, but an agent's facts can contradict each other when you change. A chart also never forgets on purpose, while an agent must, or the fact book fills with trivia. The overwrite rule in the next section is the part with no medical equivalent.

How It Actually Works

Two stores, one background pass, and a budget that explains why both exist.

The two stores

Call the ordered log EE: messages, tool results, and timestamps, kept exactly as they occurred. EE is the transcript. It is cheap to write (append and forget) and expensive to read, because nothing in it is indexed by meaning. You search it by recency with a sliding window over the last few turns, or by keyword over older ones.

Call the fact table SS: one row per durable claim about the user or the world, each stamped with the episode it came from. SS is the chart. It is expensive to write, because every row costs judgment, and cheap to read, because a dozen rows fit anywhere.

The budget is what forces the split. Name WW the window: the most tokens one request may hold, input plus output. Name HH the history lane inside it. Raw history grows without bound while WW stays fixed, so HH can never hold more than the recent past. Everything older must either be distilled into SS or be forgotten. If a lane still overflows after that triage, context compaction shrinks the pieces that remain. That is the whole trade: EE buys fidelity and costs tokens, SS buys persistence and costs judgment.

The extractor pass

Nothing moves from EE to SS by itself. After a session ends (or on a timer, for long ones), a background pass reads the fresh episodes and proposes rows: new facts get added, confirmed facts get their stamps refreshed, contradicted facts get overwritten. Last write wins, and the stamp is what makes that rule safe: every row points back at the episode that last touched it, so you can always audit a fact back to the words it came from.

This pass is also the bouncer. Most of what happens in a session deserves no permanent row: one renamed variable, one clarified address, one joke. A good extractor writes little and overwrites willingly. If your fact table grows by hundreds of rows a week, the extractor is filing napkin sketches.

Do not build any of this for single-session tools. A summarizer, a classifier, or any task where yesterday says nothing about today pays the extractor's cost for rows it never reads. And never persist what you should re-fetch: stable traits (home city, taste, role) belong in SS, while volatile values (prices, availability, last night's score) belong in a tool call at read time.

Worked example: two thousand messages become twelve facts

Take a user with 2000 past messages averaging 40 tokens each. That is 2000×40=800002000 \times 40 = 80000 tokens of raw history. Against a fixture window of W=8000W = 8000 tokens, the full log overshoots by a factor of ten. Even a generous 200-message sliding window costs 200×40=8000200 \times 40 = 8000 tokens, the entire window, leaving no room for instructions or the answer.

Now distill. Suppose the extractor keeps 12 durable facts averaging 15 tokens each: 12×15=18012 \times 15 = 180 tokens. That profile is 180/8000=2.25%180 / 8000 = 2.25\% of the window, and 80000/180≈44480000 / 180 \approx 444 times smaller than the raw log. Same user, same history: one fits in a lane and the other never could. The picture above shows the same idea at diary scale, with four raw episodes on the left, three stamped facts on the right, and one background pass between them.

The design assumes past behavior predicts future preference. When you change, old rows lie until some new episode contradicts them, which is why the overwrite rule is load-bearing and not polish.

Code

The point of this snippet is the overwrite rule, the one line the whole section depends on. Newer episodes replace rival rows instead of piling up beside them.

def consolidate(profile: dict[str, str], facts: list[tuple[str, str]]) -> dict[str, str]:    updated = dict(profile)    for key, value in facts:        updated[key] = value  # newer episode overwrites; rivals never pile up    return updated

profile: dict[str, str] = {"location": "New York", "diet": "likes cilantro"}episode_facts: list[tuple[str, str]] = [("diet", "dislikes cilantro"), ("role", "chef")]
print(consolidate(profile, episode_facts))  # -> {'location': 'New York', 'diet': 'dislikes cilantro', 'role': 'chef'}print(len(consolidate(profile, episode_facts)))  # -> 3

The cilantro row flips instead of forking, and the role row appends because nothing rivals it. That is last-write-wins in four lines, and the diagram's right panel shows exactly this profile: New York, chef, dislikes cilantro.

Watch Out For

Appending rival facts instead of overwriting

After a year of chats the profile holds both "likes cilantro" and "dislikes cilantro". The symptom is unmistakable: the agent hedges, cites both, or picks one at random each session. It happens because the extractor only ever adds rows and nothing tells it to retract.

Fix it at the write path, not with cleanup scripts. Instruct the extractor to overwrite on conflict, stamp every row with its source episode, and let last write win. Then confirm with one glance per week that no key holds two rivals. Cleanup scripts treat the symptom; the overwrite rule removes the cause.

Distilling a misheard line into permanent fact

One garbled transcript suggests the user is vegetarian, though nobody said that. The extractor files it anyway, and for months the agent dodges every steakhouse with full confidence. The symptom is a confident, persistent, wrong personalization with no visible trail back to the words.

Fix it with provenance and doubt. Keep the source episode id on every row, distill only claims stated plainly (never guesses from one vague line), and re-read the transcript before acting on a high-stakes fact. A fact without a source is a rumor with tenure.

The Quick Version

  • Episodic memory is the raw transcript of what happened; semantic memory is the distilled fact book that survives after the window closes.
  • Model calls are stateless, so anything the agent "remembers" was written down somewhere and pulled back into the prompt.
  • A background extractor moves facts from transcript to profile, overwriting rivals so the newest episode wins.
  • A 2000-message history at 40 tokens each is 80000 tokens against an 8000-token window, while twelve 15-token facts cost 180. That ratio is the whole argument.
  • Persist stable traits like home city and taste; re-fetch volatile values like prices with a tool call instead.
  • Stamp every fact with its source episode, or the first misheard line becomes permanent.