Why AI forgets, and how memory works in Whisperer

Why AI forgets

An ordinary AI chat forgets not because the model is weak, but because the context window is working memory, not storage. Everything you said lives inside one conversation. The conversation ends and the whiteboard is wiped.

There are actually two kinds of forgetting, and they have different cures.

The second is the real problem. A 200,000-token window is a large whiteboard for one session; the moment that session closes the whiteboard is wiped, and enlarging it changes nothing. A different mechanism is needed: not "hold everything in your head" but write it down separately and retrieve it when relevant.

Memory is really about forgetting

It sounds backwards, but a system that remembers everything is useless in exactly the same way as one that remembers nothing.

Memory that only grows becomes a landfill within a year: the fact you need cannot be found among a thousand unimportant ones, and the token bill grows linearly with the age of the account. Memory that forgets too eagerly loses the very thing it was built for. All the engineering here is the balance between those two ways of failing.

Which gives four problems any honest implementation has to solve:

1. What to record at all — a filter on the way in.
2. How long to keep it — decay over time.
3. What to do with repeats — deduplication.
4. How much to show the model — a retrieval budget.

Below is how each is solved here, with numbers. The numbers are not there for effect: without them, any discussion of memory stays marketing.

1. What gets recorded: three kinds and a filter

Entries come in three kinds, and the kind determines how long an entry lives.

The spans are not guesses. A fact about a person stays true for months; a detail from the day before yesterday's call does not; a learned procedure lasts longest, because every repetition of the scenario confirms it.

Before anything is written, a structural filter runs — deliberately blunt and conservative: it is cheaper not to record something (the person will repeat it) than to pollute memory permanently. It does not judge whether a statement is true — that is the job of the model that extracted it. It looks only at properties checkable without a network call.

That last row is the most underrated. Recording "right now I'm working on the release" means confidently reporting a stale state of affairs as current a month later. That is worse than remembering nothing: wrong context looks like knowledge.

2. How long it lasts: retention from four signals

Entries are not deleted on a timer. Each has a retention score, and below a threshold of 0.15 it stops taking part in retrieval. The threshold is deliberately non-zero: an exponential never actually reaches zero, and without a floor nothing would ever be forgotten.

Retention combines four signals, each guarding against a different kind of mistake:

• Age — otherwise a one-off remark lives forever alongside a profession.
• Kind — a conversational detail goes stale faster than a property of a person.
• Confirmations — something said three times must not be dropped, however long ago. Each repetition extends the half-life.
• Importance — "I'm allergic to penicillin" should not fade on the same schedule as "prefers dark mode".

One separate signal is an explicit request. When someone says "remember that…", importance is set to maximum: a short everyday phrase they asked for should not evaporate at the same rate as a passing remark.

3. Repeats: a duplicate is confirmation, not clutter

Duplicates appear by themselves: people restate the same thing in different words, and every conversation adds another copy. Within a month, retrieval for "where do I work" consists of five variants of one sentence, having crowded out everything else.

The key decision: repetition is a signal of importance, not a reason to discard. Merging increments a confirmation counter and refreshes the last-touched timestamp rather than quietly losing one copy. Comparison runs at two levels: exact, on normalised text — it catches literal repeats and costs nothing — and near, on vectors, which catches "I work in fintech" against "my job is fintech".

4. What happens when there is too much memory

Recall competes for space with the conversation itself. Without a ceiling, memory behaves like a leak: the longer an account lives, the longer the system layer, the more every single turn costs, and the less window remains for the actual dialogue.

So the memory block has two ceilings at once:

So what reaches the prompt is not all of your memory but a dozen of the most relevant entries. This is not thrift for its own sake: the limit is the condition under which the assistant still works a year from now.

There is a second, less obvious half to the problem. Memory records the whole utterance — "work out how much time my meetings took last month" goes into the store exactly as said. On the hot path that is sensible: an extra model call on every turn would double the cost of the conversation. But what accumulates is then made of the person's commands rather than facts about them, and retrieval spends its slots on one-off requests.

So once a day a separate pass goes over what has accumulated and decides, for each entry: discard it (a one-off request, stale trivia) or rewrite it as a standalone fact — "is building their own RAG system". Rewritten entries are re-indexed, or search would keep finding the old text. The work is off the hot path, so a cheaper, simpler model does it than the one running the conversation.

What is deliberately not remembered

Equally important is what never enters memory.

• Pleasantries and one-word replies — rejected by the filter.
• Momentary states — "right now", "today", "at the moment".
• Long retellings — summarised before recording, or retrieval becomes a wall of text.
• Anything doubtful — the filter is fail-closed: if an entry does not pass cleanly, it is not stored.

Separately: conversations in no-logs mode are not stored at all — neither transcript nor model output — so no memory arises from them either.

Staying in control

Memory is useless if you cannot inspect and correct it. The list of memories is available through the assistant, as is deleting a specific entry. Deletion is irreversible, so it always requires confirmation and shows a server-side scope preview: how many entries will be erased, computed from the real arguments rather than narrated by the model. The confirmation mechanics are covered in the article on the agent that takes action.

A list, though, is not enough. Nobody reads several dozen lines of "works in fintech", "asked for shorter answers", "discussed the migration" from top to bottom — and the question people actually open that screen with is different: how does Leo see me? A summary answers it: the same entries, retold as a single paragraph. It is recomputed by content rather than on a schedule — while memory is unchanged the text comes from cache; a new entry changes the fingerprint, and the next visit rebuilds it.

Alongside it sits personal context — a short text about you that you write yourself and that is injected into every request. The distinction is simple: context you control by hand, memory accumulates on its own and decays by the rules above.

Limits and trade-offs

Four places where the solution is knowingly imperfect.

Importance is estimated crudely. Without calling a model, substance is approximated by length: very short entries tend to be fragments, very long ones retellings. Nothing better is possible inside a pure domain, and pulling a model in would mean paying for every entry.

The momentary-state filter had to be taught each language separately. It began knowing only Russian and English, and in the other fourteen "right now I'm working on the release" slipped straight into memory. Fixing it was not a matter of translating a word list: Chinese, Japanese and Thai have no word boundaries at all, while in Devanagari and Arabic the combining marks do not count as letters, so the usual "is this a standalone word" check never fires there. It took two separate lists — one with word boundaries, one without. All sixteen languages are covered now.

Twelve entries is few for a long history. The cap favours retrieval quality, but with a large memory some genuinely relevant facts will not reach the prompt.

Forgetting is irreversible. An entry that has fallen below the threshold does not come back on its own. If a fact matters again, confirm it in conversation — the confirmation counter will extend its life.

Frequently asked questions

How is memory different from the context window?

The context window is the working memory of a single conversation: everything in it is available now and disappears when the session ends. Long-term memory is stored separately from the conversation and retrieved when relevant in later ones. Enlarging the window fixes forgetting within a chat and does nothing for forgetting between chats.

Why does AI forget if it has a 200,000-token window?

Because window size governs one conversation. A new chat starts from a blank whiteboard no matter how large the whiteboard is.

What exactly does the assistant remember about me?

Stable facts, compressed episodes of past conversations, and learned procedures. Pleasantries, one-word replies and momentary states are not remembered.

Can I ask it to remember something specific?

Yes, and an explicit request is the strongest available signal: such an entry gets maximum importance, so a short everyday phrase will not fade at the same rate as a passing remark.

How long does a memory last?

It depends on kind and on whether it has been confirmed. Base half-lives: 180 days for a fact, 30 for an episode, 365 for a procedure. Each repetition extends the span, and high importance doubles it.

Can I delete what the assistant remembers about me?

Yes, entry by entry and with confirmation. Before deletion you are shown how many entries will be erased — a number computed by the server, not by the model.

What to do with this

If you are choosing an assistant with memory, do not ask whether it remembers. Ask three things: what it refuses to record, how fast it forgets, and how much reaches the prompt at once. A product with no answers to those has most likely been hoarding everything — and you will see the consequences six months in, not during the demo.

Our answers are above, with numbers. Seeing how it behaves on your own context takes one conversation: create an account, tell the assistant a couple of things about yourself, and come back the next day in a fresh chat.