Agent memory: an employee with amnesia every morning

Picture an excellent employee: quick to understand, precise, never tired. And every morning they arrive having forgotten everything. Who you are, what you are building, what you decided yesterday, why you rejected their suggestion last week.

That is an agent today. The gap between what it can do and what it is actually useful for is not intelligence. It is memory.

Why a context window is not memory#

The confusion is common and the difference is fundamental.

A context window is temporary workspace. Everything in it is present now, and when the conversation ends or the window fills, all of it disappears completely. There is nowhere it goes.

Memory is different: what persists after the conversation ends, and is retrieved in a future one when relevant.

The four kinds, and which you need#

The third row is the most neglected and the most valuable in an organisational setting. An agent that learns your procedures once, and does not need them re-explained every time, saves more than any improvement to the underlying model.

How to build one, in the simplest form that works#

You do not need elaborate architecture to start. The simplest thing that works:

After each session, extract what deserves keeping. Not the whole transcript. The facts, preferences and decisions.

Store them as short standalone sentences. Each one understandable on its own with no context.

Before each new session, retrieve only what relates to the current topic. Not everything, and this is where most implementations go wrong.

The third rule is not decoration. Memory accumulates for years, everything written into it stays, and any future breach exposes everything ever collected.

The problem nobody discusses: wrong memory#

We talk about memory as though it were purely good. It is not.

An agent that stored an incorrect fact will build on it forever, confidently, and you will not know where it came from. An agent that stored a preference you expressed once in a specific context will apply it in every context.

What this means for you#

If you are building an agent: start with procedural memory rather than user memory. Teaching an agent how your organisation works returns value faster than remembering individual preferences.

If you use tools with memory: open the settings and look at what they have stored about you. Most people never have, and the list is usually surprising.

If you own compliance: memory changes the system's classification. A tool without memory processes data; a tool with memory stores it, and those are entirely different obligations under most data protection frameworks.

In closing#

The next jump in usefulness will not come from a smarter model. It will come from a system that remembers the right things and forgets the rest.

And good memory is not the kind that keeps everything. It is the kind that knows what is not worth keeping.

Try this today: open an AI tool you use daily, find its memory settings, and read what it has collected about you. What you decide afterwards is your own business.

And if you want to build an agent that safely remembers your organisation's procedures, that is what we build in the retrieval assistant course.

Common questions#

What is the difference between a context window and agent memory?

A context window is temporary workspace that disappears when the session ends. Memory is what persists and is retrieved in a later session. The first is filled and emptied; the second accumulates.

What kinds of agent memory are there?

Episodic for what happened and when, semantic for stable facts, procedural for how tasks are done, and working memory for the state of the current task. Procedural is the most valuable inside an organisation.

How do I build a simple memory that works?

After each session, extract facts, preferences and decisions as short standalone sentences and store them; then retrieve only what relates to the current topic in the next session. Selective retrieval is the key part.

What is the risk with memory systems?

A wrong fact once stored gets built on confidently and nobody knows where it came from. That is why every memory must be viewable, deletable item by item, and dated so its age is visible.

Does memory change my regulatory obligations?

Usually yes. A tool without memory processes data; a tool with memory stores it, and those are different classifications under most data protection frameworks, with different retention, deletion and disclosure duties.