MnemoverseDocs

Appearance

Edward Izgorodin·Published Jul 19, 2026

Memory, From DRAM to Agents: One Word, Twelve Worlds ​

An engineer debugging an LLM agent can say “memory” three times before lunch and mean three different things: the GPU memory that holds the model’s weights, the KV cache that avoids recomputing attention, and the persistent store that remembers a user’s name across sessions. One word names a physical die, a correctness-transparent cache, and a stateful substrate built to change behavior. The ambiguity is not sloppy terminology. It is a fossil record of a 78-year-old loan that keeps getting repaid.

TL;DR

  • “Memory” entered computing as a biology borrowing in von Neumann’s 1945 EDVAC draft, which cited the McCulloch–Pitts neuron model.
  • Computing now uses the word for a dozen or so distinct concepts, from physical storage and paging to model weights and persistent agent state.
  • In 2023, PagedAttention and MemGPT separately re-borrowed operating-system memory ideas, while CoALA drew its agent-memory categories from cognitive psychology.
  • Our lens separates these meanings by horizon and transparency: caches must not change the answer, while weights and agent memory exist to change future behavior.

Memory is state that outlives a computation and shapes future behavior. The transparency test asks whether that state is allowed to change the answer.

The first definition holds across the whole stack, but only once “a computation” is indexed to a horizon — an instruction, a process, a request, a training run, and an agent’s lifetime draw different boundaries.

These are not stacked floors of one building. They are different worlds — different contracts, different failure modes, different reasons the state is allowed to exist at all — that happen to share a word.

This article maps the word. It does not claim that computers remember as brains do.

I come at this from both ends of the map. I get asked to consult on memory from opposite directions — one month a hardware team on processing-in-memory, the next a product team on how to shrink a KV cache. Same word, unrelated problems. This is the map I wish I could hand each of them.

And a word on the number in the title. Twelve is a clean, memorable count — the months, the apostles, the hours on a clock face — but I don't mean it as a census. The honest tally refuses to hold still, and that refusal is the argument. Is cache one sense or several: are a CPU line, DRAM, and disk three memories or one hierarchy? Is a process's working set the same kind of thing as a hardware cache? Where do SSDs, memory cards, and DDR generations belong — new senses, or subdivisions of ones already counted? Each line you draw splits or fuses a few more. A word that resolved into a fixed tally wouldn't be the overloaded word this article is about. Twelve is a lens on that overload, not a census of it.

Why is computer storage called memory? ​

The answer begins with von Neumann’s First Draft of a Report on the EDVAC, circulated on June 30, 1945. The founding document of the stored-program computer named the machine’s storage “memory” and its components “organs,” and modelled its logic elements on an idealized neuron. Section 4.2 cited the neuron model directly:

“Following W.S. MacCulloch (sic) and W. Pitts (‘A logical calculus of the ideas immanent in nervous activity,’ Bull. Math. Biophysics, Vol. 5 (1943), pp. 115–133) we ignore the more complicated aspects of neuron functioning …”

This was a formal, full citation, not a later analogy or a loose “notation.” The word entered computing as a documented borrowing from the brain sciences — which establishes terminology, not equivalence between machines and minds.

The loop took 78 years to close. In 2023, AI reached back to both of the word’s ancestors at once:

  • MemGPT, titled MemGPT: Towards LLMs as Operating Systems, re-borrowed hierarchical memory from the operating system.
  • CoALA organized language-agent memory through cognitive-science categories: working, episodic, semantic, and procedural.

MemGPT is the operating-system return. CoALA is the psychology return. Conflating them would erase the more interesting history.

Types of memory in computing: five bands ​

The bands below are not a ranking; they group these senses by the horizon over which state survives.

Silicon: physical store, cache, and replacement ​

At the hardware band, physical memory is a hardware store arranged in a hierarchy of registers, cache, DRAM, and disk, from fast and volatile to slow and persistent. Hennessy and Patterson’s Computer Architecture: A Quantitative Approach gives the standard treatment.

Cache added another sense. Wilkes described a fast store subordinate to main memory in “Slave Memories and Dynamic Storage Allocation” in 1965; Liptay’s 1968 System/360 Model 85 paper fixed “cache” as the name for the CPU-side buffer. Replacement then became a memory problem of its own. Belády’s 1966 optimal MIN algorithm evicts the item whose next use lies furthest in the future — an ideal benchmark for replacement, not a practical view into future accesses.

Operating systems: illusion, locality, and exposure ​

Denning’s 1970 survey settled the model of virtual memory: an address-space illusion maintained by paging between faster and slower tiers. His 1968 working set described the pages a process is actively using — an idea from program locality, not a borrowing from psychological working memory.

Shared memory uses the word again. Interfaces such as POSIX mmap and System V shmget let processes observe a common region. Memory becomes a coordination surface.

Memory safety names the address space as an attack surface. The White House Office of the National Cyber Director’s February 2024 report cited Microsoft’s estimate that about 70% of the vulnerabilities Microsoft assigns a CVE each year are memory-safety issues — a figure the report attributes to Microsoft, not CISA. Chromium’s security team reports the same shape independently: around 70% of its serious security bugs are memory-safety problems.

Runtimes and databases: reachability and price ​

McCarthy’s 1960 Lisp paper described automatic reclamation of unreachable storage. Here memory is governed by reachability: state that no live variable can reach becomes garbage. McCarthy later recalled:

“We already called this process ‘garbage collection’, but I guess I chickened out of using it in the paper — or else the Research Laboratory of Electronics grammar ladies wouldn’t let me.”

Garbage collection made forgetting a designed runtime operation. Persistent agents face a related design problem, though their criteria for consolidation and forgetting differ from heap reachability.

Databases add an economic meaning. Gray and Putzolu’s 1987 five-minute rule asked when keeping a page in memory costs less than re-reading it from disk: keep a roughly 1 KB page resident when reuse is expected within about five minutes.

ML systems: the wall, the cache, the window, the weights ​

The memory wall returns here. Wulf and McKee coined the term in 1995 for the gap between processor speed and memory speed. Gholami and colleagues revived it for AI in 2024, reporting that peak server FLOPS had scaled 3.0× every two years while DRAM and interconnect bandwidth scaled only 1.6× and 1.4×. Faster arithmetic does not remove a bandwidth bottleneck.

The KV cache is attention state retained during decoding. Shazeer’s 2019 paper named the bottleneck it removes: “the memory-bandwidth cost of repeatedly loading the large ‘keys’ and ‘values’ tensors.” The mechanics belong in KV Cache & Context Engineering.

From the field. When I am asked to make a KV cache cheaper, my answer is usually that the sharper lever sits a level up. The cache is expensive because the context is large; shrink what the model has to recompute each turn and the cache question shrinks with it. Optimizing the cache treats a symptom of a decision made one layer higher — worth doing, but rarely where the largest wins live.

The context window is the bounded set of tokens a model attends to during one request or conversation. Calling it working memory can orient discussion, but the psychological comparison has limits.

Weights supply the last sense. Lewis and colleagues’ 2020 RAG paper says “the parametric memory is a pre-trained seq2seq model,” while “the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever.” Parametric memory is hard to enumerate or delete item by item — though Geva and colleagues found that transformer feed-forward layers “operate as key-value memories,” and ROME then modified those weights “to update specific factual associations” in 2022.

Agents: state across sessions ​

Agent memory is persistent state that an agent stores, retrieves, and reuses across sessions instead of starting cold. CoALA gives this band its cognitive taxonomy; MemGPT supplies an operating-system framing that moves information between memory tiers. The broader engineering model sits at the map’s terminal cell: the longest horizon in the stack, and deliberately consequential — recalled state should change what the agent does next.

From the field. I build in this cell, which is why I will say plainly that it is the map’s least settled. As a research object, agent memory is still thin — more hot topic than science, and the core primitive is weak. That is an argument for rigor, not enthusiasm: the interesting work is making the primitive less crude, not celebrating that it exists at all.

The echoes: documented borrowings ​

The strongest connections across these bands are explicit borrowings, not visual similarities.

In September 2023, PagedAttention described itself as “inspired by the classical virtual memory and paging techniques in operating systems,” applying paging to KV blocks at the serving layer. One month later, MemGPT proposed virtual context management inspired by hierarchical operating-system memory, applying the same family of ideas to context at the agent layer. The same OS mechanism was borrowed twice in 2023, at two different bands of the stack.

Replacement vocabulary travelled too. H2O proposed “a KV cache eviction policy” and framed eviction as a dynamic problem; StreamingLLM reported that “keeping the KV of initial tokens will largely recover the performance of window attention.” The objects changed from hardware lines and virtual pages to attention state, but eviction stayed the governing operation.

Shared agent memory has an older AI precedent. The 1980 Hearsay-II paper described “a global working memory (the ‘blackboard’)” where distinct representations were integrated — the coordination pattern that multi-agent shared memory revisits today under different constraints. And the memory wall moved without disappearing: the 1995 term reappeared by title in 2024 because model serving met the same bandwidth constraint on a new workload.

The invariant: horizon and the transparency test ​

The following framework is our analytical lens, not a taxonomy taken from the cited literature. It has two axes.

The first is horizon — the boundary of the computation that state must outlive. The stack is a nesting of these boundaries, not a ranked ladder.

Horizon“A computation” is…Example
Instructionone operationregister, CPU cache
Processa program runheap, virtual memory
Session or requestone request or conversationKV cache, context window
Lifetimea training run or agent’s lifeweights, agent memory

The second axis is consequence. The transparency test asks whether a memory mechanism is allowed to change the answer. Transparent memory may change cost or latency, never the intended result: CPU caches, database buffer pools, and an exact KV cache all follow this contract, where a miss is only slower, not semantically different. Lossy KV-cache compression — the eviction policies noted earlier — is the deliberate exception, trading exactness for memory. Consequential memory exists to change the result: different weights produce different behavior, and recalled agent state should change later decisions.

This yields a clean parallel — CPU cache : program output :: an exact KV cache : model output — both correctness-transparent. Agent memory sits at the opposite corner: longest horizon, intentionally consequential. The context window marks the boundary: it survives a request but does not persist by itself, so a separate store must carry information into a later context — the point where active context becomes agent memory.

What the word hides ​

Some echoes are documented lineages. Others are parallels, and should stay labelled as such.

Parallel coinage. Denning’s “working set” (1968) and psychology’s “working memory” both name the subset currently in use, with no documented borrowing in either direction. The recurrence reflects a recurring problem, not proven intellectual descent — two fields independently needing a word for the part in active use.

Our observation. The five-minute rule and prompt-cache pricing share the same break-even shape: keep state resident when expected reuse justifies the cost. This is an analytical parallel, not a cited lineage; the KV-cache article covers the modern economics.

“Memory” came from the brain sciences into computing in 1945. In 2023, CoALA returned to psychology while MemGPT returned to the operating system. Between those points, one word became a map of different horizons and different promises.

The bottleneck was always the memory channel ​

That is the history of the word. Underneath all these worlds runs a second history — of the constraint the word names. John Backus named it in his 1977 Turing Award lecture, published in 1978. A von Neumann computer, he wrote, has a CPU, a store, and “a connecting tube that can transmit a single word between the CPU and the store” — and “I propose to call this tube the von Neumann bottleneck.” The task of a program, he added, is accomplished “entirely by pumping single words back and forth” through it. The word “memory” and the bottleneck to it were born in the same 1945 architecture.

That complaint has one continuous history. Backus named it qualitatively in 1978; Wulf and McKee quantified it as the memory wall in 1995; Gholami and colleagues re-fired it for AI in 2024, with compute scaling 3.0× every two years against memory bandwidth’s 1.6×. Same channel, restated as the workload changed — nearly eight decades of the constraint sitting between the processor and the store, not inside either.

The hardware response has always been to move computation toward the data instead of pumping the data past the processor. The idea is old — Harold Stone described “a logic-in-memory computer” in 1970 — but it now ships. UPMEM has sold general-purpose processing-in-memory DIMMs since 2019. Samsung (HBM-PIM) and SK hynix (GDDR6-AiM) have built and publicly demonstrated compute inside the memory die, though neither is a volume product. Mythic ships analog compute-in-memory at the edge, doing matrix-multiply inside flash arrays. IBM’s phase-change analog chip is still research. The census is lopsided — one shipping general-purpose part, the rest sampled or in the lab — but the direction is not in doubt.

Author’s view. If the constraint has been the channel for nearly eighty years, the resolution is not pushing words through the tube faster. It is memory that computes — not a processor with memory bolted onto the far end of a bus. That is a bet about where the field is heading, not a settled result; the census above shows it is still early. And the bet reaches the terminal cell of this map. Persistent agent memory is the weakest world on the list partly because we still build it as storage the model reads from. Whether it should instead be a substrate that participates in the computation, rather than one the model merely queries, is — to me — the open question worth working on.

Common questions ​

Why is computer storage called memory? ​

Von Neumann's 1945 EDVAC draft named the machine's storage memory and built its logic on the McCulloch–Pitts neuron model. The term entered computing as a documented borrowing from the brain sciences.

Is a KV cache the same as a CPU cache? ​

They operate at different layers, but both are transparent caches: they may cut latency and computation without changing the intended result. A CPU cache serves processor operations; a KV cache preserves attention keys and values during inference.

Is the context window the model's memory? ​

The context window is a bounded active set for one request or conversation, and it does not persist by itself. Agent memory begins when another system stores information beyond that horizon and later feeds it back into context.

What is parametric memory? ​

Parametric memory is knowledge encoded in a model's weights. RAG contrasts it with non-parametric memory, such as an external vector index that a retriever can query.

What is the memory wall? ​

The memory wall is the growing gap between compute throughput and memory bandwidth. A 2024 analysis reported peak server FLOPS scaling 3.0× every two years, against 1.6× for DRAM bandwidth and 1.4× for interconnect bandwidth.

How many kinds of memory are there in computing? ​

This map identifies roughly a dozen load-bearing senses across hardware, operating systems, runtimes, databases, machine-learning systems, and agents — the same proliferation of names that psychology shows. Where you draw the lines shifts the exact count, which matters less than the different contracts hidden behind one word.

Sources ​

The borrowed word (1945) and its return (2023) ​

  • von Neumann, J. (1945). First Draft of a Report on the EDVAC. Moore School of Electrical Engineering, University of Pennsylvania. (Circulated June 30, 1945.)
  • McCulloch, W.S., & Pitts, W. (1943). “A Logical Calculus of the Ideas Immanent in Nervous Activity.” Bulletin of Mathematical Biophysics 5:115–133.
  • Sumers, T., Yao, S., Narasimhan, K., & Griffiths, T. (2023). “Cognitive Architectures for Language Agents (CoALA).” arXiv:2309.02427 (TMLR 2024).
  • Packer, C., et al. (2023). “MemGPT: Towards LLMs as Operating Systems.” arXiv:2310.08560.

Silicon: hierarchy, cache, replacement, the wall ​

  • Hennessy, J.L., & Patterson, D.A. Computer Architecture: A Quantitative Approach. (Memory-hierarchy chapters.)
  • Wulf, W.A., & McKee, S.A. (1995). “Hitting the Memory Wall: Implications of the Obvious.” ACM SIGARCH Computer Architecture News 23(1):20–24.
  • Wilkes, M.V. (1965). “Slave Memories and Dynamic Storage Allocation.” IEEE Transactions on Electronic Computers EC-14(2):270–271.
  • Liptay, J.S. (1968). “Structural Aspects of the System/360 Model 85, II: The Cache.” IBM Systems Journal 7(1):15–21.
  • Belády, L.A. (1966). “A Study of Replacement Algorithms for a Virtual-Storage Computer.” IBM Systems Journal 5(2):78–101.

Operating systems: the illusion, the region, the attack surface ​

  • Denning, P.J. (1970). “Virtual Memory.” ACM Computing Surveys 2(3):153–189. doi:10.1145/356571.356573
  • Denning, P.J. (1968). “The Working Set Model for Program Behavior.” CACM 11(5):323–333.
  • Office of the National Cyber Director (2024). Back to the Building Blocks: A Path Toward Secure and Measurable Software. The White House. (Press release 2024-02-26; ~70% figure quoted from Microsoft, not CISA; Chromium reports the same shape independently.)

Runtimes and databases ​

  • McCarthy, J. (1960). “Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I.” CACM 3(4):184–195.
  • Gray, J., & Putzolu, F. (1987). “The 5 Minute Rule for Trading Memory for Disc Accesses and the 10 Byte Rule for Trading Memory for CPU Time.” SIGMOD 1987, pp. 395–398. doi:10.1145/38714.38755

ML systems: bandwidth, KV cache, weights ​

  • Gholami, A., et al. (2024). “AI and Memory Wall.” IEEE Micro 44(3):33–39. arXiv:2403.14123
  • Shazeer, N. (2019). “Fast Transformer Decoding: One Write-Head is All You Need.” arXiv:1911.02150
  • Lewis, P., et al. (2020). “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” arXiv:2005.11401
  • Geva, M., Schuster, R., Berant, J., & Levy, O. (2021). “Transformer Feed-Forward Layers Are Key-Value Memories.” arXiv:2012.14913 (EMNLP 2021).
  • Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022). “Locating and Editing Factual Associations in GPT (ROME).” arXiv:2202.05262 (NeurIPS 2022).

The echoes: the OS idea borrowed twice, eviction, the blackboard ​

  • Kwon, W., et al. (2023). “Efficient Memory Management for Large Language Model Serving with PagedAttention.” arXiv:2309.06180
  • Zhang, Z., et al. (2023). “H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.” arXiv:2306.14048 (NeurIPS 2023).
  • Xiao, G., Tian, Y., Chen, B., Han, S., & Lewis, M. (2023). “Efficient Streaming Language Models with Attention Sinks.” arXiv:2309.17453
  • Erman, L.D., Hayes-Roth, F., Lesser, V.R., & Reddy, D.R. (1980). “The Hearsay-II Speech-Understanding System.” ACM Computing Surveys 12(2):213–253.

Compute-in-memory and the bottleneck ​

  • Backus, J. (1978). “Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs.” CACM 21(8):613–641. (1977 ACM Turing Award Lecture.) doi:10.1145/359576.359579
  • Stone, H.S. (1970). “A Logic-in-Memory Computer.” IEEE Transactions on Computers C-19(1):73–78. doi:10.1109/TC.1970.5008902
  • Mutlu, O., Ghose, S., Gómez-Luna, J., & Ausavarungnirun, R. (2023). “A Modern Primer on Processing in Memory.” arXiv:2012.03112
  • UPMEM (2019). Commercial general-purpose processing-in-memory DIMMs. Benchmarking study: arXiv:2105.03814
  • Samsung (2021). HBM-PIM “Aquabolt-XL,” Hot Chips 33; SK hynix (2022). GDDR6-AiM, ISSCC 2022 — compute inside the memory die (demonstrated, not volume products).
  • Mythic (2022). M1076 Analog Matrix Processor — analog compute-in-memory in flash. IBM Research (2023). 64-core analog in-memory chip (phase-change memory), Nature Electronics. doi:10.1038/s41928-023-01010-1

— Edward Izgorodin · Last updated 2026-07-19