Mnemoverse Docs

Appearance

Edward Izgorodin·Published Jun 7, 2026

Jeff Hawkins: Memory Exists to Predict ​

Here is the idea, in one line: your brain does not store the world so it can play it back — it stores the world so it can predict it. Jeff Hawkins has spent two decades arguing that prediction is the core operation of the cortex, that the cortex is essentially a memory system, and that getting this right is one proposed route to more robust machine intelligence. It is a different starting point from almost everyone else who built a memory into a network — and it came from the man who built the PalmPilot.

TL;DR ​

  • Hawkins's memory-prediction framework (On Intelligence, 2004): the cortex is a memory system whose core job is prediction — it constantly anticipates its next input from stored sequences.
  • Hierarchical Temporal Memory (HTM) builds this on three pillars: sparse distributed representations, sequence memory, and continual learning.
  • SDRs — high-dimensional binary vectors with only ~2% of bits active — are robust, high-capacity, and encode similarity through overlap.
  • A Thousand Brains (2021): not one model of the world but ~150,000 cortical columns, each using reference frames, combined by voting.
  • A distinct, neuroscience-first path — influential in ideas, not the benchmark-dominant paradigm.

The bet: prediction is the whole game ​

Most founders of network memory came from physics or engineering and asked a mechanical question — how do you store a pattern in weights and get it back? Hawkins, an engineer turned neuroscientist, asked a functional one: what is all that memory for?

In 2004, with Sandra Blakeslee, he answered in On Intelligence. The neocortex is a memory system, and its core operation is prediction: it continuously forecasts its next input from stored sequences, and notices the instant a forecast fails.

The memory-prediction framework: the cortex constantly predicts its next input from stored sequences; to understand something is to be able to predict it.

You run this loop constantly — the next note of a familiar song, the next word of a sentence, the feel of the next stair. When prediction holds, the world feels continuous; when it breaks, attention snaps to the surprise. Hawkins's claim is that this is not something intelligence does with memory. It is what memory is.

HTM: the theory, made buildable ​

To make the idea concrete, Hawkins and Numenta built Hierarchical Temporal Memory (HTM) — a model constrained by the biology of the neocortex. Three properties carry it.

Sparse distributed representations. Everything in HTM is an SDR: a long binary vector with only ~2% of bits active, mirroring how few cortical neurons fire at once.

A sparse distributed representation is a high-dimensional binary vector with very few active bits; which bits are on is the meaning.

Sparsity is not a detail. As Ahmad and Hawkins set out in Properties of Sparse Distributed Representations (2015), it buys robustness (drop a few bits, keep the meaning), enormous capacity, and similarity-as-overlap — two SDRs that share active bits are, by construction, related. In an SDR, similarity is part of the representation rather than something computed afterward by a separate distance metric.

Sequence memory. HTM learns sequences over time, so prediction falls out for free — see part of a sequence, predict the rest, and treat a violated prediction as an anomaly. The mechanism is set out in Hawkins and Ahmad's Why Neurons Have Thousands of Synapses (Frontiers in Neural Circuits, 2016), a theory of sequence memory in the neocortex.

Continual learning. HTM learns from the stream as it arrives, with no separate training phase and without wiping what it knew when the data shifts — demonstrated for streaming data in Cui, Ahmad & Hawkins, 2016. That is the behavior an always-on system actually needs.

A Thousand Brains ​

In 2021, A Thousand Brains sharpened the picture with a surprise: the cortex does not hold one model of the world but thousands. The Thousand Brains Theory proposes that the cortex is built from many near-identical cortical columns — on the order of 150,000 — each learning a full model of objects using reference frames — coordinate systems anchored to the thing being modeled, so a column can predict features relative to position and movement. Nothing is in charge; the columns vote, and the consensus is your single perception. Memory, in this view, is redundant and parallel on purpose.

Where it actually stands ​

Be straight about it: HTM did not win the benchmark race. The field scaled dense networks and attention, and Numenta's neuroscience-first program has stayed off to the side. But several of its bets have aged well — sparse representations, continual learning, and prediction as a first-class operation are all live concerns again. Hawkins's contribution is less a deployable product than a theory of what memory is for, plus a set of properties worth borrowing; the open-source Thousand Brains Project (2024) carries it forward.

What a memory builder should take from it ​

Three things transfer directly.

Judge a memory by what it predicts. Recall is table stakes; the harder, more useful test is whether a memory lets the system anticipate what comes next. A memory that predicts well has captured the structure of its domain; one that only retrieves has merely filed things away.

Store sequences, not just snapshots, and weigh sparse codes. Temporal structure is where anticipation comes from. And sparse, high-dimensional representations trade dense-vector convenience for robustness, capacity, and a clean overlap-equals-similarity property worth considering in a memory layer.

Learn from the stream. Continual, online learning — absorb new experience as it arrives, without a retraining cycle and without erasing the past — is exactly what a long-lived agent memory needs.

Hawkins belongs with the other pioneers because he asked the question their mechanisms answer in pieces: not how to store a memory, but what a memory is for. His answer — to predict — is a good compass for building one.

Common questions ​

Who is Jeff Hawkins? The creator of the PalmPilot who turned to brain theory — founder of the Redwood Neuroscience Institute (2002) and Numenta (2005), author of On Intelligence and A Thousand Brains, originator of HTM.

What is the memory-prediction framework? Hawkins's theory (2004) that the cortex's core operation is predicting its next input from stored sequences; understanding is successful prediction.

What is HTM? A biologically constrained model of the neocortex — sparse distributed representations, sequence memory, and continual learning — that learns streaming data unsupervised and predicts.

What are SDRs? High-dimensional binary vectors with ~2% of bits active — robust, high-capacity, with overlap encoding similarity (Ahmad & Hawkins, 2015).

What is the Thousand Brains Theory? The idea (2021) that the cortex is ~150,000 cortical columns, each modeling the world with reference frames, combined by voting.

Mnemoverse is a persistent-memory API for AI agents. Free key: console.mnemoverse.com · Docs: Getting Started