Memory Science
The cognitive science and neural foundations behind machine memory — from Hopfield nets and Hebbian learning to consolidation.
14 articles
Attention as a Hopfield Network
Transformer attention is equivalent to one update step of a modern Hopfield network under specific query, key, value substitutions.
Memory, From DRAM to Agents: One Word, Twelve Worlds
Types of memory in computing, mapped from DRAM and virtual memory to KV caches, model weights, context windows, and persistent agent memory.
Transactive Memory: Who Remembers What in a Team
Transactive memory divides the labor of remembering across a group — a directory of who knows what — mapped, with its limits, to multi-agent AI memory.
Why Agent Memory Needs Sleep
Agent memory consolidation turns raw writes into reusable structure between uses — what works, what ships, and what remains unproven.
Multimodal Memory Integration: Cross-Modal Binding in AI
Multimodal memory integration binds text, image, and audio into one representation. A sourced guide to binding: TPR, VSA/HDC, SDM, and Hopfield networks.
Self-Organizing Memory: ART, SOM & Growing Neural Gas
Self-organizing memory systems explained: Adaptive Resonance Theory, Self-Organizing Maps, and Growing Neural Gas, including GNG-U utility-based pruning.
Episodic vs Semantic Memory: Tulving for AI Agents
Tulving's episodic vs semantic split is a design decision for AI agents: an event log beside a fact store, plus a step deciding when episodes become facts.
Types of Memory: Why So Many Names?
Memory has dozens of named kinds because no one has drawn its boundary. A guided tour of the list, the reasons it grew, and why the seams never close.
Schema Formation: How Memory Builds Reusable Structure
Schema theory explains how memory turns episodes into reusable structure. For AI agent memory, it clarifies why episodes and consolidation should stay separate.
Working Memory: Capacity, Models, and AI Context
Working memory is the bounded active workspace of cognition; its capacity debate (Miller's 7 vs Cowan's 4) and models map onto AI context-window design.
Bernard Widrow: From the LMS Rule to Cognitive Memory
Bernard Widrow's 1960 LMS delta rule taught machines to learn and still runs in adaptive filtering; late in life he turned to content-addressable memory.
Geoffrey Hinton: The Boltzmann Machine and Generative Memory
Geoffrey Hinton's 1985 Boltzmann machine made memory generative: a stochastic, energy-based network with hidden units that learns a distribution and samples it.
Jeff Hawkins: Memory Exists to Predict
Jeff Hawkins argues the brain is a memory system for prediction; Hierarchical Temporal Memory uses sparse representations, sequences, and continual learning.
Hopfield Networks: The Memory Model That Became Attention
Hopfield's 1982 PNAS associative memory, the 0.138N capacity limit (Amit, Gutfreund, Sompolinsky, 1985), and how attention became one read from it.