Why does my AI assistant keep bringing up outdated information after I've corrected it?

Because the correction was stored as a new memory instead of defeating the old one. Both facts remain retrievable, and the older one often wins on similarity when a question closely matches it. Corrections need supersession: the write path must find and demote every memory the new fact contradicts.

Should memories have a TTL?

Not a uniform one - that prices all knowledge at the same decay rate. Assign volatility classes (structural, preferential, procedural, situational) with class-appropriate half-lives, and attach dependencies (product version, org state) so forgetting can be event-driven rather than calendar-driven.

What's wrong with just down-ranking stale content instead of removing it?

Down-ranking reduces retrieval probability but loses to similarity exactly when a question most resembles the stale item - which is when it's most dangerous. Graduated states work better: flag suspect items visibly at retrieval, retrieve deprecated items only alongside their replacements, quarantine what must not surface.

How does the right to be forgotten interact with RAG and memory systems?

Deletion must cascade: distilled memories, knowledge-graph edges, derived tables, and validation counts all carry residue of source data. Without a derivation graph (provenance edges from every derived artifact back to sources), complete erasure requires reprocessing the entire corpus - so build provenance in at ingestion.

Do agents need forgetting within a single session?

Yes - an agent's context is working memory with second-scale half-lives. Superseded observations should be evicted or flagged when later evidence defeats them, and closed subtasks compressed to conclusions. Agents that never forget within-episode both thrash and act on defeated observations.

What does this essay explain about the feature nobody builds?

Go look at the roadmap of any AI product with a memory feature. I'll wait. You'll find remembering everywhere. Persistent memory, cross-session recall, "it knows you" - remembering is the demo, the differentiator, the launch tweet.

What additional detail does the "The Feature Nobody Builds" section cover?

You'll find remembering everywhere. Persistent memory, cross-session recall, "it knows you" - remembering is the demo, the differentiator, the launch tweet.

What does this essay explain about the workaround that wouldn't die?

In the memory essay in this series, I mentioned a support copilot that learned resolutions from validated tickets. This is the story of that system's worst day. Among its best-performing memories was a workaround for a sync failure in the product's v4.1: disable a consistency check, re-run the import, re-enable the check. Validated on many tickets. Genuinely correct - for v4.1.

What additional detail does the "The Workaround That Wouldn't Die" section cover?

Among its best-performing memories was a workaround for a sync failure in the product's v4.1: disable a consistency check, re-run the import, re-enable the check. Validated on many tickets. Genuinely correct - for v4.1.

What does this essay explain about there is no cold storage?

The deeper realization arrived while we were scoping the fix, and it's the observation this essay is built on. In conventional software, storing something is passive. Data at rest is inert; old rows sit in cold storage costing pennies, affecting nothing until explicitly queried. Our entire intuition about "keep it just in case" is built on that inertness - storage is cheap and harmless, so why ever delete?

What additional detail does the "There Is No Cold Storage" section cover?

In conventional software, storing something is passive. Data at rest is inert; old rows sit in cold storage costing pennies, affecting nothing until explicitly queried. Our entire intuition about "keep it just in case" is built on that inertness - storage is cheap and harmless, so why ever delete?

What key claim does the essay make about there is no cold storage?

In systems with semantic retrieval, there is no cold storage. Everything stored is effectively loaded - one similar query away from entering a context window with the authority of institutional knowledge. Storage decisions are therefore behavior decisions, and forgetting is a runtime capability, not cleanup.

What does this essay explain about retrieval liability?

That gives me the first concept I want to name properly. Retrieval liability: the expected future harm of a stored item, given the probability it will be retrieved into contexts where it's wrong.

What additional detail does the "Retrieval Liability" section cover?

Retrieval liability: the expected future harm of a stored item, given the probability it will be retrieved into contexts where it's wrong.

How should readers understand: Retrieval liability: the expected future harm of a stored item, given the probability it will be retrieved into contexts where it's wrong?

Every item in a memory store or corpus carries some of it.

What does this essay explain about volatility classes?

The fix conversation started, as these do, with someone proposing TTLs: expire every memory after ninety days. Terrible idea, instructively terrible. "The customer's deployment is on AWS" and "the customer was mid-migration last Tuesday" would expire on the same schedule. One of those facts is good for years; the other was stale within a week. A uniform TTL is a category error: it prices all knowledge at the same de…

What additional detail does the "Volatility Classes" section cover?

Terrible idea, instructively terrible. "The customer's deployment is on AWS" and "the customer was mid-migration last Tuesday" would expire on the same schedule. One of those facts is good for years; the other was stale within a week. A uniform TTL is a category error: it prices all knowledge at the same decay rate, and decay rates are precisely what distinguish kinds of knowledge.

What does this essay explain about the forgetting ladder?

The second mistake in the TTL proposal was treating forgetting as boolean: present or deleted. Forgetting done well is graduated - a state machine, not a delete statement. ACTIVE │ half-life elapsed, dependency changed, or contradiction detected ▼ SUSPECT — retrievable, but flagged; retrieval must carry the flag │ re-validation fails, or supersession confirmed ▼ DEPRECATED — retrieved ONLY alongside its replacement,…

What additional detail does the "The Forgetting Ladder" section cover?

ACTIVE │ half-life elapsed, dependency changed, or contradiction detected ▼ SUSPECT — retrievable, but flagged; retrieval must carry the flag │ re-validation fails, or supersession confirmed ▼ DEPRECATED — retrieved ONLY alongside its replacement, as history │ no longer referenced; retention window passed ▼ QUARANTINED — excluded from retrieval entirely; kept for audit │ deletion obligation or retention expiry ▼ DEL…

What does this essay explain about supersession: most forgetting is replacement?

Audit any mature memory store's transitions and you'll find something that reframes the whole problem: genuine expiration - a fact just fading into staleness - is the minority case. The majority case is *defeat*: a new fact arrives that contradicts, updates, or obsoletes an old one. The user didn't stop having a deployment platform; they moved to GCP. The workaround wasn't withdrawn; it was replaced by a fix. The pr…

What additional detail does the "Supersession: Most Forgetting Is Replacement" section cover?

The user didn't stop having a deployment platform; they moved to GCP. The workaround wasn't withdrawn; it was replaced by a fix. The preference didn't expire; it changed.

How should readers understand: Supersession check: every memory write first searches for existing memories it contradicts or updates, and must resolve the conflict - defeat the old memory, refine it, or flag the contradiction - before the write commits?

This is consolidation from the memory essay's production function, given teeth.

What does this essay explain about the deletion cascade?

Eventually a customer invoked their right to erasure, and we learned the difference between forgetting and deletion the way everyone learns it: by discovering how many places a fact can hide. Deleting the user's raw records was easy. Then the audit started.

What additional detail does the "The Deletion Cascade" section cover?

Deleting the user's raw records was easy. Then the audit started.

How should readers understand: The deletion cascade: forgetting a fact requires identifying and reprocessing everything ever derived from it - and that's only possible if derivation left a trail?

Here's the uncomfortable general form: every write-time intelligence strategy this series has recommended - distillation, consolidation, graph extraction, derived tables - *multiplies the places a source fact lives*.

What This Looks Like in Real Systems?

Personalization: the correction test. The most user-visible forgetting failure is the assistant that won't accept a correction - the user says "I don't work at that company anymore" and the old fact keeps resurfacing, because the correction was *added* (one new memory) rather than *supersessive* (defeating every memory it contradicts). Users experience this as the system arguing with them about their own lives. I no…

What additional detail does the "What This Looks Like in Real Systems" section cover?

Agents: episodic forgetting. Within a single long run, an agent's context is working memory, and the hygiene from the physics essay - dropping superseded observations, compressing closed subtasks - is this essay's ladder operating at second-scale half-lives: an observation is ACTIVE until the plan moves on, SUSPECT when a later tool result disagrees, and deserves eviction once its subtask closes. Same state machine,…

What does "Personalization: the correction test" mean in this essay?

The most user-visible forgetting failure is the assistant that won't accept a correction - the user says "I don't work at that company anymore" and the old fact keeps resurfacing, because the correction was *added* (one new memory) rather than *supersessive* (defeating every memory it contradicts).

What does "Agents: episodic forgetting" mean in this essay?

Within a single long run, an agent's context is working memory, and the hygiene from the physics essay - dropping superseded observations, compressing closed subtasks - is this essay's ladder operating at second-scale half-lives: an observation is ACTIVE until the plan moves on, SUSPECT when a later tool result disagrees, and deserves eviction once its subtask closes.

What does "Knowledge graphs: edge invalidation" mean in this essay?

Graphs rot edge-first - entities endure while relationships churn ("supplies," "owns," "reports to" are all volatility-classed facts wearing structural clothing.)

What does "RAG corpora: documents lie about their volatility" mean in this essay?

A corpus is a memory store where someone else did the writing - and documents don't announce which class their claims belong to; the 2022 pricing PDF asserts its numbers in the same present tense the current one uses.

What are the limits of the argument presented in this essay?

Premature forgetting destroys rare gold. The strongest counterargument: some of the most valuable memories are rare-case resolutions that go unretrieved for two years and then save a customer's week. Aggressive liability management would have quarantined them at month six. Frequency-of-retrieval is a bad proxy for value, and any forgetting policy tuned purely on usage will silently discard the long tail that justifi…

What additional detail does the "Where This Argument Breaks" section cover?

Misclassification compounds. Volatility classes are assigned by the same imperfect distillation pipeline that writes the memories. File a structural fact as situational and you'll churn a stable truth; file procedural as structural and you've rebuilt the v4.1 bug with extra steps. The classes reduce error versus uniform TTLs; they don't eliminate the judgment, they relocate it.

What does "Premature forgetting destroys rare gold" mean in this essay?

The strongest counterargument: some of the most valuable memories are rare-case resolutions that go unretrieved for two years and then save a customer's week.

What does "Misclassification compounds" mean in this essay?

Volatility classes are assigned by the same imperfect distillation pipeline that writes the memories.

What does "The machinery has a price floor" mean in this essay?

Supersession checks on every write, dependency tracking, derivation graphs - for a small system, this can cost more than it saves.

What does "Retention law cuts both ways" mean in this essay?

I've framed deletion as an obligation, but plenty of regulated domains have the opposite one: audit trails, legal holds, records that *must* survive.

What should readers take away about ai products design remembering as a capability and forgetting as a compliance chore?

AI products design remembering as a capability and forgetting as a compliance chore, but semantic retrieval eliminates cold storage: everything stored is one similar query away from entering a context window with institutional authority.

What should readers take away about every stored item therefore carries retrieval liability - expected future harm when it surfaces where it's wrong - and liability?

Every stored item therefore carries retrieval liability - expected future harm when it surfaces where it's wrong - and liability, not storage cost, is what accumulation accumulates.

What should readers take away about effective forgetting is graduated (active → suspect → deprecated → quarantined → deleted)?

Effective forgetting is graduated (active → suspect → deprecated → quarantined → deleted), event-driven via volatility classes and dependencies rather than calendar TTLs, and mostly implemented on the write path as supersession: new facts must defeat the old facts they contradict, leaving explicit lineage.

What should readers take away about true deletion requires a derivation graph?

True deletion requires a derivation graph, because write-time intelligence - distillation, graph edges, derived tables - multiplies where a source fact lives.

What should readers take away about the unifying principle: a system that cannot forget cannot be corrected?

The unifying principle: a system that cannot forget cannot be corrected, because a correction is only real when the old fact loses future retrieval contests.

Which series is Designing AI That Knows When to Forget part of?

Designing AI That Knows When to Forget is part 7 of the "Mental Models for AI Engineering" essay series on production AI mental models.

How does Designing AI That Knows When to Forget relate to LLM forgetting?

Designing AI That Knows When to Forget uses LLM forgetting as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does Designing AI That Knows When to Forget relate to memory decay AI agents?

Designing AI That Knows When to Forget uses memory decay AI agents as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does Designing AI That Knows When to Forget relate to stale memory LLM?

Designing AI That Knows When to Forget uses stale memory LLM as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does Designing AI That Knows When to Forget relate to right to be forgotten AI?

Designing AI That Knows When to Forget uses right to be forgotten AI as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does Designing AI That Knows When to Forget relate to memory TTL design?

Designing AI That Knows When to Forget uses memory TTL design as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does Designing AI That Knows When to Forget relate to agent memory architecture?

Designing AI That Knows When to Forget uses agent memory architecture as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does Designing AI That Knows When to Forget relate to knowledge invalidation?

Designing AI That Knows When to Forget uses knowledge invalidation as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

June 14, 2026

Designing AI That Knows When to Forget

In systems with semantic retrieval there is no cold storage - everything stored can resurface with authority. Forgetting is a capability, not cleanup.

Key Takeaways

AI products design remembering as a capability and forgetting as a compliance chore, but semantic retrieval eliminates cold storage: everything stored is one similar query away from entering a context window with institutional authority.
Every stored item therefore carries retrieval liability - expected future harm when it surfaces where it's wrong - and liability, not storage cost, is what accumulation accumulates.
Effective forgetting is graduated (active → suspect → deprecated → quarantined → deleted), event-driven via volatility classes and dependencies rather than calendar TTLs, and mostly implemented on the write path as supersession: new facts must defeat the old facts they contradict, leaving explicit lineage.
True deletion requires a derivation graph, because write-time intelligence - distillation, graph edges, derived tables - multiplies where a source fact lives.

Key Takeaways

FAQ

Related Expertise

Related Concepts

Related Research

Related Articles

Related Automations