Agent memory is information an AI agent saves and reads back later. Short-term memory covers one conversation or task; long-term memory carries facts, past events and learned rules across conversations. A language model has no memory of its own between calls, so everything an agent remembers is stored by the application and placed back into the model’s context when it is needed.
Short-Term and Long-Term Memory
- Short-term memory is the state of one thread: the messages so far, tool results, intermediate decisions. LangGraph persists it with a checkpointer, so a conversation can continue after a restart and separate threads stay isolated.
- Long-term memory is shared across threads. LangGraph stores it as JSON documents in a store, organised by namespace (for example, per user or per organisation) and key (LangGraph memory concepts).
The same documentation divides long-term memory into three kinds, borrowed from cognitive science. The research framework CoALA (Cognitive Architectures for Language Agents) takes a similar view, treating memory as separate modules around the model rather than part of it (Sumers et al.).
| Kind | What it holds | Example |
|---|---|---|
| Semantic | Facts | ”This customer’s contract renews in March.” |
| Episodic | Past events and actions | How the agent resolved a similar ticket last week |
| Procedural | Rules for doing the task | An updated instruction in the agent’s system prompt |
How Memory Is Written and Read
Writing. Memories can be written in the hot path, while the agent is working, which keeps them current but adds latency and work to every turn. Or they can be written in the background, by a separate job after the conversation, which keeps responses fast but means a new fact is not available immediately.
Reading. At the start of a turn the application retrieves the memories that matter for this request, usually by namespace and then by similarity search over embeddings, and places them in the prompt. Retrieval from memory uses the same machinery as retrieval-augmented generation; the difference is that memory is written by the agent from its own interactions, while a RAG index is built from company documents.
Trimming. Long threads are summarised or truncated so they fit the context window without crowding out instructions.
When to Use Long-Term Memory and When Not
Long-term memory helps when the same user returns and repeating context wastes their time: preferences, ongoing cases, a project’s decisions. It is a poor fit for facts that already live in a system of record. A customer’s address belongs in the CRM, read through a tool each time, not copied into a memory that will go stale.
Common Failure Modes
- Stale facts. A remembered value outlives the change in the real system. Memories need a source and a date, and records of truth should be read live.
- Wrong-scope leakage. A memory saved under one user is retrieved for another. Namespaces must follow the same access rules as the data itself.
- Memory poisoning. Text from an untrusted document is saved as a “fact” and steers later runs. Treat memory writes as data input and validate them.
- Privacy obligations. Personal data in memory is personal data under GDPR, with the same rights to access and erasure.
- Clutter. Saving everything makes retrieval worse. Decide what is worth remembering.
In Practice
Before any storage is chosen, we list what an assistant may remember at all. A user’s preferred report format is worth keeping; an order status or a balance is not, because it changes and has to be read fresh from the source system each time. In Alltegrio’s AI agent development projects, short-term state sits in LangGraph checkpoints and long-term memory in PostgreSQL, partitioned by user and tenant, with each stored fact dated and linked to the conversation it came from. Users can see and delete what the assistant holds about them.