MnemoverseDocs

Appearance

Edward Izgorodin·Published Jun 18, 2026Updated Sep 24, 2026

Rescorla-Wagner for agent memory: why retrieval should learn from outcomes ​

TL;DR

  • Rescorla-Wagner is a classic 1972 prediction-error learning rule: update strength from the gap between expected and actual outcome.
  • In Mnemoverse that idea drives the feedback step — outcome feedback nudges a memory's valence after recall (/api/overview).
  • Valence changes ranking on later queries, so memories that helped move up and memories that misled rank lower; nothing is erased.
  • This is the feedback half of learning; concept associations are the Hebbian half — see the companion article.

Published by Mnemoverse, a persistent-memory API for AI agents.

Static retrieval never learns from what happened next. An embedding match that once worked can fail repeatedly with no mechanism to lower its future rank. Most retrieval systems stop at recall: they find what looks relevant, return it, and end the loop — leaving out the one signal that matters most, what happened after the memory was used. A memory that led to a good answer should not be treated the same on the next query as one that caused a failure — yet plain retrieval has no way to tell them apart. The Rescorla-Wagner rule supplies exactly that missing mechanism.

Rescorla-Wagner is an error-driven learning rule: it changes strength in proportion to prediction error — the gap between what was expected and what actually happened. Bigger surprise, bigger correction.

What Rescorla-Wagner actually is ​

Rescorla and Wagner published their model in 1972 to explain conditioning results earlier theories could not. The core insight: learning happens when events violate expectation, and the size of the update equals the size of the surprise. Machine learning had already arrived at the same principle twelve years earlier, as the Widrow-Hoff delta rule (1960) — two fields converging independently: in both, the update is proportional to the error, so large errors produce large adjustments and near-zero errors produce almost none.

That idea fits agent memory because outcome feedback is often the only clean signal available after retrieval. You may not know the full internal reason an answer failed, but you usually do know whether using a recalled memory helped or hurt.

How Mnemoverse uses it: the feedback step ​

In Mnemoverse the rule is used narrowly and on purpose. It is not the model for the whole memory system — it is the lineage of the feedback step that updates a memory's valence after a recalled memory proves helpful or unhelpful. The API overview describes this as "a prediction-error feedback signal (Rescorla-Wagner-style) updates each memory's valence so memories that led to good outcomes rank higher next time," a learning layer added on top of retrieval (/api/overview).

The documented lifecycle is write → read → feedback → consolidate, and feedback is the learning step (/api/overview). When you report an outcome, Mnemoverse performs two documented updates:

  • a Rescorla-Wagner-style prediction-error update to memory valence;
  • strengthening of Hebbian concept associations (and co-activation links between the query and result concepts).

That split separates two different questions — did this memory help? (outcome feedback) and which concepts should become more strongly linked? (association structure). Keep one clean boundary in mind: Rescorla-Wagner answers how a memory is updated after an outcome; Hebbian learning answers which concepts associate. Mnemoverse uses both, for different jobs — the association half is covered in Hebbian memory for AI agents. So "Mnemoverse learns from feedback" is true; "Rescorla-Wagner is the core model for the whole memory graph" would be wrong.

What valence does in recall ​

Valence is the outcome-polarity signal attached to a memory; updating it changes future ranking during recall. The system does not rewrite embeddings or edit text — it changes a scalar that influences future retrieval probability. The docs state the behavior plainly (/api/overview): memories that lead to good outcomes rank higher in future queries, memories that consistently fail rank lower, and the system improves with use; nothing is erased. Valence is observable as avg_valence in stats (/api/getting-started, /api/python-sdk).

Because valence rides on top of semantic similarity, a moderately-similar but historically-reliable memory can outrank a closer match that has consistently disappointed. Similarity gets a memory into the candidate set; outcome feedback changes how it is treated next time.

The feedback call ​

The outcome-feedback API is direct: feedback() takes an outcome float from -1.0 (failure) to +1.0 (success) for a set of recalled atom_ids (/api/python-sdk).

python
from mnemoverse import MnemoClient
client = MnemoClient(api_key="mk_live_YOUR_KEY")

# Recall, then act on a memory…
memories = client.read("how to handle timeouts?")

# …then report what happened. A Rescorla-Wagner-style prediction-error update
# nudges each memory's valence by the outcome, so good memories rank higher next time.
client.feedback(
    atom_ids=[item.atom_id for item in memories.items],
    outcome=1.0,            # -1.0 (failure) … +1.0 (success)
    query_concepts=memories.query_concepts,
)

atom_ids says which recalled memories were involved; outcome reports whether they helped; query_concepts preserves the concept context from the read. One call performs both halves — the prediction-error valence update and the Hebbian association strengthening (which also creates co-activation links between the query and result concepts). The full write-read-feedback walkthrough is in Getting Started under "Report Outcomes."

Calling feedback from your own bot, when the answer helped ​

This case is also on video, walked through rather than written out: 12:21, for anyone who would rather watch it move. The text below stands on its own and covers more of the mechanism than the film does.

More of these on our YouTube channel — the same arguments, drawn out rather than written up.

The code above assumes something that is true for a coding assistant and false for most other bots: that an outcome is lying around. An assistant inside an editor has a build, a test run, a linter, a command that exits zero or does not. If you wrote your own bot and call this API from your own code, none of that exists. The outcome has to come out of the conversation, and the conversation mostly says nothing at all.

The decisions, in the order they come up.

Where the call goes. After the turn, not inside it. The read happens while you are assembling the answer; the outcome is not known until the user has done something with it. Rating in the same breath as the read means rating the retrieval, not the advice.

Which ids. A read hands back items, each with its atom_id. Keep those ids with the turn that used them, not just with the session. When somebody comes back three turns later and says the fix worked, the ids you rate are the ones from the read that produced that advice, and the current turn has its own, different ones. A store that rates the latest read every time learns that whatever it said last is what helped.

What to send when nobody says anything. Most turns end in silence, and silence is not an outcome. outcome is a required argument with no default, and nothing calls feedback() for you: if your code does not decide, nothing is reported and the ranking stays as it was. That is the honest state. A small positive on every turn the user did not complain about is not the same as sending nothing: it moves the valence of every memory it touches, and a rise that lands everywhere tells the ranking nothing.

What does count, for a bot with no build to read: the user acting on the answer, coming back with the next step rather than the same question, accepting a draft, closing the thread. Those are signals you already have in your own code. They are yours to pick, and they are not in anybody's API.

What the rating touches. Two things carry a valence, and they are not the same thing. The memory does: valence is a field on every item a read returns, outcome polarity from -1.0 to +1.0, and it moves when feedback arrives. The links between concepts do too: a Hebbian edge carries its own weight alongside its own valence, which is why the call takes query_concepts as well as ids. A rating changes the first and updates the second.

By how much is not published. The reference names the fields, not the coefficient, and this page is not going to invent one. What you can observe is the field itself: each item a read returns carries its current valence, so the next read after a rating shows the new value.

What you cannot do, as of 2026-09-23. You cannot filter a read by outcome. The documented read takes a query, a top_k, a domain, a relevance floor, association expansion, history, concept hints and the Rooms author filters, and there is no valence or outcome parameter among them, on the reference and on the deployed schema, both read on 2026-09-23. The verb is re-rank, not filter. A memory with a poor history still comes back; it comes back lower. Nothing is deleted to make room for what worked.

And the two paths are not the same. The Python client takes query_concepts on the feedback call, so the loop between what you asked and what came back is closed from there. The MCP server does not send them, and says so itself: in @mnemoverse/mcp-memory-server 0.11.0, published 2026-09-23, memory_feedback takes the memory_ids being rated, the outcome score and a domain when you pass one, and the package's own text reads "This server does not send query_concepts, so live calls through this tool report 0". If you are wiring this through an MCP client you are rating memories, which is the part that changes what comes back first. The link between the question and the answer is taught from the Python path.

The smallest version, with no service at all ​

Nothing above needs a vendor to be understood. This is the whole behaviour in the standard library: a store, a read that ranks, a report, and the same read afterwards. The arithmetic is this file's own, chosen to be readable, and it is not any product's ranking.

python
BOOST = 0.5      # how much a memory's standing counts next to the words that matched
RATE = 0.4       # how fast a single report moves that standing


def score(item, query_concepts):
    """Words that matched, then the memory's standing on top of them."""
    overlap = len(set(item["concepts"]) & set(query_concepts))
    return overlap + BOOST * item["standing"]


def read(items, query_concepts, top_k=3):
    """Everything that matches at all, best first. Nothing is filtered out."""
    hits = [i for i in items if set(i["concepts"]) & set(query_concepts)]
    hits.sort(key=lambda i: (-score(i, query_concepts), i["id"]))
    return hits[:top_k]


def feedback(items, item_ids, outcome):
    """Report how a call went: outcome runs from -1.0 to 1.0, and you choose it.

    The move is a plain delta rule: the standing goes part of the way from where it is
    to what was just reported, so one report cannot decide everything and ten reports
    of the same kind settle down instead of running away.
    """
    for item in items:
        if item["id"] in item_ids:
            item["standing"] += RATE * (outcome - item["standing"])
            item["reports"] += 1


items = [
    {"id": 1, "text": "Retry the webhook three times, then drop it.",
     "concepts": ["webhook", "retry", "failure"], "standing": 0.0, "reports": 0},
    {"id": 2, "text": "Retry with backoff and keep the dead letter.",
     "concepts": ["webhook", "retry", "failure"], "standing": 0.0, "reports": 0},
    {"id": 3, "text": "Webhook signatures expire after five minutes.",
     "concepts": ["webhook", "signature"], "standing": 0.0, "reports": 0},
]

asked = ["webhook", "retry"]
print([i["id"] for i in read(items, asked)])   # [1, 2, 3]: the words cannot tell 1 and 2 apart

feedback(items, [2], 1.0)                      # the answer that carried a call that went well
print([i["id"] for i in read(items, asked)])   # [2, 1, 3]

feedback(items, [2], 1.0)                      # the same answer works again
print([i["id"] for i in read(items, asked)])   # [2, 1, 3], and 2 moved less the second time

Three things in that are worth keeping when you move to a real store. The first read cannot tell the two candidates apart, because the words are the same and nothing has been reported yet, so the tie is broken by id and the wrong one wins. The second report moves the standing less than the first, which is what a delta rule does and what stops a single loud signal from deciding everything. And the answer that lost is still in the list, one place down, because it was out-ranked and not erased.

Rescorla-Wagner vs vector database retrieval ​

MnemoverseVector database
RetrievalSemantic similarity (the floor)Semantic similarity (the whole system)
Learns from outcomesYes — prediction-error update to valenceNo — ranking is static unless vectors are recomputed
Effect over timeRecall re-ranks by what proved usefulSame query, same result

A vector database returns nearest neighbors by a fixed distance. If two memories are similarly relevant but one repeatedly helps and the other repeatedly misleads, static retrieval has no native reason to prefer the better one next time. Outcome-aware memory does. Retrieval remains the performance floor; feedback supplies the learning signal that raises it. That is also why feedback belongs in production, not just evaluation — skip it and you skip the part where the memory learns from experience.

What this does not mean ​

A precise claim is a stronger claim. This article does not say Mnemoverse implements the full competitive multi-cue Rescorla-Wagner model — no cue competition, blocking, or overshadowing. The public description supports a narrower statement: a per-memory, single-cue, delta-rule-style update on valence from the reported outcome. That narrower claim is enough to explain why outcome feedback changes ranking over time, and why memories improve or degrade with actual use. For benchmark numbers, link the benchmarks page rather than quoting scores out of context; for why memory needs more than plain retrieval at scale, see building memory that scales.

Common questions ​

What is Rescorla-Wagner in agent memory? ​

Rescorla-Wagner is an error-driven learning rule from 1972: update strength in proportion to prediction error, the gap between expected and actual outcome. In Mnemoverse that idea drives the outcome-feedback step — after recall, a memory's valence is nudged by the reported outcome, so useful memories rank higher later.

How does prediction-error feedback work in Mnemoverse? ​

After a memory is recalled and acted on, you report an outcome from -1.0 (failure) to +1.0 (success) via feedback(). That outcome acts as a prediction-error signal that moves each recalled memory's valence, which changes future ranking.

Does Rescorla-Wagner control the associations in Mnemoverse? ​

No. Rescorla-Wagner is the lineage for the valence/feedback update, not the concept-association model. The association half is Hebbian, covered in the companion article.

What does valence do in recall? ​

Valence is the outcome-polarity signal attached to a memory. Updating it re-ranks future recall: memories tied to good outcomes rank higher, memories that consistently fail rank lower; nothing is erased. It is observable as avg_valence in stats.

How is this different from a vector database? ​

A vector database returns what is similar to the query, the same way every time; retrieval itself does not learn from what happened next. Mnemoverse adds a feedback step after recall, so outcome data updates valence and improves future ranking with use.

How do I report an outcome from my own bot when the user never says whether it helped? ​

Send nothing. outcome is a required argument with no default and nothing calls feedback() for you, so a turn you do not rate leaves the ranking as it was, which is the honest state. Rating every quiet turn slightly positive teaches the store that everything worked. What counts as a signal for a bot with no build or test run is behaviour you already see in your own code: the user acting on the answer, coming back with the next step instead of the same question, accepting a draft, closing the thread.

Can I filter a read by positive outcome? ​

No. The documented read takes a query, a top_k, a domain, a relevance threshold, association expansion, history, concept hints and the Rooms author filters, and none of them is an outcome or valence filter, on the reference and on the deployed schema, both read 2026-09-23. Outcome feedback re-ranks rather than filters: a memory with a poor history still comes back, lower, and nothing is deleted to make room for what worked.

The point is modest and practical: if your agent can report whether a recalled memory helped, the memory system should use that signal — that is what turns recall into a system that learns.

— Edward Izgorodin · Last updated 2026-09-23