MnemoverseDocs

Appearance

Edward Izgorodin·Published Jun 23, 2026Updated Oct 17, 2026

AI agent memory: what it is, how it works, and how to choose ​

Most confusion about agent memory starts with one bad assumption: that storage and memory are the same thing.

They are not.

A database can hold facts. A cache can hold context. A vector store can retrieve similar text. None of that, by itself, gives an AI agent memory in the sense engineers usually need: continuity across sessions, selective recall, and some way to improve what gets remembered over time.

TL;DR

  • AI agent memory is the capability that lets an AI agent store, recall, and reuse knowledge across sessions instead of starting cold on every new conversation. Sources: IBM Think, Memory in the Age of AI Agents.
  • LLMs are stateless by design, so long-running work breaks when relevant context is lost. For background and the evidence base, see The AI Agent Memory Crisis and AI Agent Memory: The 2026 Landscape.
  • The main approach families are vector memory, graph memory, hybrid vector+graph systems, and OS-tiered or self-editing runtimes. They solve different parts of the problem.
  • The hard part is not storing text. The hard part is consolidation, learning from outcomes, and verification so bad memories do not pollute future work.

Published by Mnemoverse, a persistent-memory API for AI agents.

What is AI agent memory? ​

AI agent memory is the capability that lets an AI agent store, recall, and reuse knowledge across sessions — instead of starting cold on every new conversation. See IBM Think and the survey Memory in the Age of AI Agents.

That definition matters because large language models do not persist state on their own. Each call gets a context window. Then it ends. If you want continuity, something outside the model has to provide it.

This is why memory has become a core issue for coding agents, research agents, and multi-step assistants. In long-running work, the bottleneck is often not generation quality. It is missing context. As summarized in AI Agent Memory: The 2026 Landscape, developers report that AI often misses relevant context in refactoring, testing, and review (Qodo, 2025 survey). The same piece cites a METR study with a small sample size that found a 19% slowdown on familiar repositories, while also noting that the authors do not claim broad generalization.

So the practical question is not whether memory matters. It is what kind of memory you need.

Why AI agent memory matters for long-running work ​

A stateless LLM is a model that does not carry knowledge from one call into the next unless you explicitly pass that knowledge back in.

That statelessness is manageable for short tasks. It becomes expensive in ongoing collaboration.

An agent that cannot remember prior design decisions, failed experiments, tool outputs, or user-specific conventions has to reconstruct them from scratch. Sometimes it asks again. Sometimes it searches logs. Sometimes it guesses. All three paths add cost and risk.

This is where the phrase "memory is not storage" becomes useful.

Storage answers: can the system keep data?

Memory answers harder questions:

  • What should be kept?
  • In what form?
  • When should it be retrieved?
  • What should fade?
  • What should be merged?
  • What if the stored fact is wrong?

Those are memory questions, not database questions. In practice a memory layer has to perform four operations on top of storage: write what is worth keeping, consolidate it as duplicates and patterns accumulate, retrieve the right items at the right time, and update or retire what is stale or wrong.

For a deeper treatment of the failure mode itself, see The AI Agent Memory Crisis. For how memory feeds prompt assembly, see Context Engineering Needs a Compiler.

Do ChatGPT and Claude already have memory? ​

Yes, but that does not settle the category.

Platform memory is the cross-session memory shipped by major AI products such as ChatGPT, Claude, Gemini, and Copilot. As summarized in AI Agent Memory: The 2026 Landscape, these systems mainly store user preferences and continuity cues: tone, language, typical tools, recurring tasks, or project-level notes.

That is useful. It is also different from structured knowledge.

A real agent memory layer needs to do more than remember that a user prefers concise answers or writes Python. It may need to preserve a sequence of tool results, represent relationships between entities, track change over time, or learn that one remembered pattern helps while another does not.

So platform memory helps with personalization. It is not the same thing as consolidated, structured, cross-session knowledge.

How agent memory works: the main approach families ​

The field now clusters around a few clear design families.

text
┌─────────────────────────────────────────────────────────────────┐
│                      AI agent memory: approach families          │
├─────────────────┬─────────────────┬───────────────┬─────────────┤
│ Vector          │ Graph           │ Hybrid        │ OS-tiered   │
├─────────────────┼─────────────────┼───────────────┼─────────────┤
│ • Flat vectors  │ • Nodes & edges │ • Vector +    │ • In-context│
│ • Similarity    │ • Bi-temporal   │   graph + KV  │   vs recall │
│ • Fast, simple  │ • Relationships │ • Breadth +   │ • Self-edit │
│                 │                 │   structure   │   via tools │
└─────────────────┴─────────────────┴───────────────┴─────────────┘

Vector memory ​

Vector memory stores memories as embeddings and retrieves them by similarity.

This is the most common baseline because it is simple and scales well. It is also flat. Similarity search does not naturally represent hierarchy, explicit relationships, or whether a recalled memory led to a good outcome. A vector store can be part of memory, but it is not the whole system.

That distinction matters enough that it deserves its own article: Is Mnemoverse a vector database?.

Graph memory ​

Graph memory represents connected facts and how they change over time.

This approach fits cases where relationships matter more than plain similarity. Examples include Zep, its Graphiti project, and the associated paper on bi-temporal knowledge graphs (arXiv:2501.13956). Another example is Cognee and its open-source repository, which uses an Extract-Cognify-Load pipeline to build a self-hosted knowledge graph.

Graph approaches are often better at answering questions like "what changed?" or "how are these entities connected?" They also introduce more modeling complexity. Whether that complexity is worth it against plain retrieval, and what asserting entity identity actually costs, is examined in Knowledge Graph vs Retrieval for AI Agent Memory.

Hybrid vector + graph memory ​

Hybrid memory combines vector retrieval with graph or key-value structures.

This is increasingly the practical middle ground. Similarity search handles fuzzy recall. Graph or structured layers preserve relationships and stable facts. Mem0, its GitHub repository, and its paper (arXiv:2504.19413) are one example; AI Agent Memory: The 2026 Landscape traces how the field has converged on this pattern.

The reason hybrids keep appearing is simple: agents need both broad semantic recall and explicit structure.

OS-tiered or self-editing memory ​

OS-tiered memory lets the agent manage memory across in-context and out-of-context tiers.

This family treats memory less like a passive retrieval backend and more like part of the runtime. Letta and the MemGPT paper (arXiv:2310.08560) are the canonical references here. LangMem and its documentation bring similar ideas into LangGraph agents with semantic, episodic, and procedural memory abstractions.

This design is useful when the agent needs to actively edit, summarize, or move memories between working and long-term state.

How to choose a memory layer ​

The cleanest way to choose is to ask what the layer does beyond retrieval.

Start with four questions.

1. Where does the data live? ​

Does memory stay inside a platform product, inside your application, or in a dedicated external layer?

This affects portability. If your team works across multiple tools, portability may matter as much as recall quality. The protocol side matters too. See Memory MCP: How to Give AI Agents Persistent Memory, A2A vs MCP, and A2A Protocol (Agent2Agent), Explained.

2. What does the layer store? ​

Some systems store snippets. Some store extracted facts. Some keep entity graphs. Some preserve episodes or procedures.

The right format depends on the task. Coding assistants often need decisions, file-level facts, and prior outcomes. Research agents may need source-linked claims and provenance. Multi-agent systems may need shared state, which is a separate design problem covered in Shared Memory for Multi-Agent Systems.

3. What does it do besides recall? ​

This is the real separator.

A thin retrieval layer stores and fetches. A memory layer may also consolidate duplicates, decay stale items, preserve hierarchy, or re-rank what helped before. If it cannot learn which recalled items were useful, it may keep surfacing the same irrelevant memory forever.

For related ideas, see Hebbian memory for AI agents and Rescorla-Wagner for agent memory.

4. How is it evaluated? ​

If a system claims strong memory, ask how that claim was tested.

How AI agent memory is evaluated ​

LoCoMo (long-term conversational memory) is a benchmark for memory over long, multi-session dialogues.

Two other important benchmarks are LongMemEval (arXiv:2410.10813, ICLR 2025) and BEAM (arXiv:2510.27246, 2025), which tests memory over contexts up to 10M tokens.

The important caveat is that benchmark scores across vendors are indicative, not directly comparable. Different test harnesses use different subsets, judges, and grading behavior. That is why this page does not rank systems. For the deeper methodology, see How to Evaluate AI Agent Memory and LLM-as-a-Judge Patterns.

Open problems in AI agent memory ​

The open problems are what separate a useful memory layer from a storage wrapper.

Consolidation ​

A long-running agent can accumulate very large numbers of memories. The system has to compress and organize that experience without flattening away the nuance. This remains unsolved at scale, especially once memory counts move past 100,000 items, as discussed in AI Agent Memory: The 2026 Landscape.

Learning from outcomes ​

Most systems remember what happened. Few learn whether it worked.

That means retrieval often ignores the strongest signal available: outcome. If a remembered strategy led to success, it should become easier to recall. If it repeatedly failed, it should become harder to surface. This remains largely unexplored in production systems, according to AI Agent Memory: The 2026 Landscape.

Verification and trust ​

A stored hallucination is worse than a transient one.

If a bad fact enters memory, the agent can keep retrieving and reusing it across sessions. The error becomes persistent. That is why memory systems need verification, provenance, or both. For the failure mode, see When AI Cites What Doesn't Exist.

My AI agent forgets decisions made in previous conversations. How to give it long term memory? ​

Added 2026-10-17. This section answers the question as practitioners ask it; the sections above are the general primer.

A decision does not come back on its own: the code shows what was done, not what was considered and rejected, so an agent cannot rediscover a decision the way it finds a fact in the repository. Write each decision as its own record at the moment it is agreed (the decision, the reason, where it applies, the date, who decided, and what it replaces), and have the agent read the records for an area before it proposes anything there. When a decision is reversed, a new record replaces the old one and the old one stays, marked as replaced, which is what Michael Nygard asked of architecture decision records in 2011: "keep the old one around, but mark it as superseded."

A decision record is a short, dated entry that states one decision, the reason for it, where it applies, who decided, and what it replaces.

For a fictional project:

text
Decision: sessions stay in Postgres; no Redis for sessions.
Reason:   one less service to run; Postgres handles current load.
Scope:    api service.
Decided:  2026-09-29, proposed by the agent, accepted by Dana in review.
Status:   accepted. Replaces: nothing.
Source:   link to the thread or pull request.

A CLAUDE.md or another rules file does not hold these records by itself: a refused approach appears there when someone writes it down, and the repository will not grow that refusal on its own.

What does a decision record need? ​

Start with the reason: it gives the agent something to check before it suggests a rejected alternative again. Oliver Zehentleitner measured this for his project Keep the Why in a rejected-change experiment. Twenty fresh sessions got the same request to simplify a retry wrapper, a change already rejected: ten ran without the recorded rationale and ten with it. His write-up reports: "Without the recorded rationale, 7 of 10 sessions offered the already-rejected change as an option. With the rationale present in context/, 0 of 10 did, and all 10 of 10 explicitly identified it as already rejected." His own bound follows: "It does not prove universal behavior, but it makes the core claim testable rather than purely philosophical."

The person who decided gets a field of their own, because in a shared file an agent's conclusion and a person's decision look the same. In a Cursor forum thread on a three-day test of the Projects beta, Stonianua put the rule plainly: "Agent conclusions stay proposed until the user explicitly accepts them; silence is not acceptance." The status is a label on the record, not a gate on writing it: an agent that saves decisions as it goes still files them as proposed, and acceptance is a later change of status. The same post sets a minimum: "Persist at least: author (user vs agent), source turn/session, status (proposed / accepted / superseded / rejected), and a one-line reason." Lutz Leonhardt, who started the thread, found in the copies he made of his threads: "“Who decided what” is recorded nowhere." The fuller list of fields, and how a correction becomes project policy, are on the evidence and policy page.

When should the agent write a decision, and when should it read one? ​

Write it when it is agreed, not at the end of the session: a session can end abruptly and take the unwritten record with it. Then give the agent a standing instruction in the project's instructions file: before it proposes a change in an area, it reads the decisions recorded for that area and says which apply. Treat writing and reading as separate obligations.

Anthropic's memory documentation for Claude Code says "Two mechanisms carry knowledge across sessions", CLAUDE.md files and auto memory, and adds: "Both are loaded at the start of every conversation. Claude treats them as context, not enforced configuration." For something settled in a chat, the instruction is direct: "Add conversation-only instructions to CLAUDE.md to make them persist." Asking Claude to remember something saves it to auto memory instead. Loaded is not the same as obeyed (context, not enforcement), so run the check at the end of this section.

A record that is never read does nothing. Lutz Leonhardt reported from his three days: "The Project Notes, a short index of the current work with an archive for finished items, were written 111 times and explicitly read once." That is his observation, not a general measurement of coding assistants.

How does a decision made in someone else's thread reach the agent? ​

Ritik Kumar asked the team version of this question on dev.to: "The part I still struggle with is decisions made in threads I was not part of. How do you handle that on a team?"

One method: the decision line goes into the same diff as the change that carries it out, or that backs out the rejected approach, with the date and a link to the thread, and review checks that line the way it checks a test. The method has a limit. A decision that produces no code change (an approach rejected before anything was written) never reaches a diff. For those, whoever closes the thread writes the line.

Point to the records from the committed project CLAUDE.md; in Anthropic's words, "These instructions are shared with your team through version control". Madhur Mittal, in the Cursor thread, keeps two files apart: "A plan file the orchestrator is free to rewrite, and a decisions.md in the repo that is append-only."

What should happen when a decision is reversed? ​

Do not overwrite the old record, and do not delete it. Nygard's full sentence carries the reason in its bracket: "If a decision is reversed, we will keep the old one around, but mark it as superseded. (It's still relevant to know that it was the decision, but is no longer the decision.)" Without the old record, a team cannot tell that it once decided the opposite, or why.

Some decisions sit between current and replaced, because their assumptions may have changed. Glen Allen, commenting on the dev.to version of this question, wrote: "I’d also consider making revalidation an explicit state rather than treating every stored decision as either “active” or “forgotten.”" The evidence and policy page calls a similar mark disputed, and Zehentleitner's write-up notes that "The Keep the Why specification defines stable IDs, status, evidence, sources, relationships, and revisit conditions." Record what would trigger a review rather than treating acceptance as permanent.

Some memory layers keep the replaced record with a mark on it. For example:

  • Mem0: Dream, a feature of the Mem0 platform, "marks the older memory as superseded and links it to the newer fact that replaced it", and "Superseded memories are not deleted and not hidden by default." For the current state alone: "When you want only the current truth, ask for it explicitly with latest_only=true".
  • Zep gives facts time bounds: "The valid_at and invalid_at fields define the temporal bounds that Zep has derived for a fact." In its own example, "Zep attempts to invalidate the fact that Kendra loves Adidas shoes and creates two new facts". Its caveat sits beside the feature: "The fields do not establish that the source content or the derived fact is true."
  • Mnemoverse, which publishes this library: a write through the REST API (POST /memory/write) or the Python SDK can name, in its supersedes field, earlier records of its own organization that it replaces. "A target is not deleted; it gains a superseded_by pointer to the new atom and stays fetchable by ID". The filter that hides a replaced revision from default reads is off by default; an organization switches it on in the console.

Agent Memory Consolidation Compared lays out how six memory layers treat the old record when a new one replaces it.

How do you check that the agent kept the decision? ​

Run a test with a control.

  1. In one conversation, agree a decision with the agent and have it written down with its reason.
  2. Reason check: open a new conversation, with none of the first one's history, and ask the agent to propose a change in that area. It should find the decision and name the reason, which exists only in the record. Avoiding the old approach is not enough on its own: the code as it stands can lead the agent away from it without the record.
  3. Invention check: ask about an area where nothing was decided. The agent should say it found no decision rather than invent one. If it cites one anyway, find where that came from before trusting the first answer.

The test is the same for a file, for auto memory, and for any memory layer. It checks retrieval and restraint, not whether a write succeeded.

Common questions ​

What is AI agent memory? ​

AI agent memory is the capability that lets an AI agent store, recall, and reuse knowledge across sessions instead of starting cold on every new conversation. Unlike a single context window, it persists useful information over time and brings it back when needed.

How is agent memory different from a vector database? ​

A vector database stores embeddings and retrieves similar items. An agent memory layer may use vectors, but it also has to decide what to store, how to consolidate it, when to forget, and how to verify that recalled information is still trustworthy.

How do I give an AI agent memory? ​

You add a persistent layer outside the model that saves facts, events, preferences, or procedures and retrieves them across sessions. Common approaches include vector memory, graph memory, hybrid systems, and OS-tiered or self-editing runtimes.

How is AI agent memory evaluated? ​

Common benchmarks include LoCoMo, LongMemEval, and BEAM. Their scores are indicative, not directly comparable across vendors, because test harnesses, subsets, and LLM-judge settings differ.

Do ChatGPT and Claude already have memory? ​

Yes, major platforms now offer cross-session memory features. But those systems mainly store user preferences and basic continuity, not consolidated, structured knowledge that learns from outcomes or represents hierarchy.

My AI agent forgets decisions made in previous conversations. How to give it long term memory? ​

Write each decision as its own record when it is agreed (the decision, the reason, where it applies, the date, who decided, and what it replaces) and have the agent read the records for an area before it proposes a change there. When a decision is reversed, write a new record and keep the old one marked as superseded, the rule Michael Nygard set for architecture decision records in 2011. Test it in a new conversation: the agent should name the recorded reason, and should say it found no decision where none was made.

Sources ​

Decisions section, read 17 October 2026

Mnemoverse is a hosted persistent-memory API for AI agents, with outcome-aware recall, cross-tool memory via one API key, and multi-scale retrieval via SLoD (arXiv:2603.08965); details: Is Mnemoverse a vector database?, Rescorla-Wagner for agent memory, and free access at console.mnemoverse.com.