The Rescorla-Wagner model (1972), developed by Robert Rescorla and Allan Wagner, is the most influential formal model of associative learning. It proposes that learning occurs when there is a discrepancy between what is expected and what actually happens — a prediction error. When an outcome is surprising (better or worse than expected), strong learning occurs. When an outcome is fully predicted, no learning occurs. This simple principle accounts for a remarkable range of conditioning phenomena.
Key Structures
- Ventral tegmental area (dopamine prediction error) — A midbrain dopamine nucleus that projects to limbic and cortical regions, central to reward prediction and motivation, particularly in relation to dopamine prediction error.
- Striatum — The input structure of the basal ganglia, comprising the caudate and putamen, critical for reward learning and habit formation.
- Amygdala — An almond-shaped structure in the medial temporal lobe that processes emotional significance, particularly threat and fear, and modulates emotional memory formation.
- Extinction — The process by which a conditioned response weakens when the reinforcing stimulus is no longer presented, revealing that extinction is new learning, not erasure.
- Insight — The sudden, conscious realization of the solution to a problem — the 'aha!' or 'eureka' moment — often preceded by an impasse and accompanied by a feeling of certainty and surprise.
Key Functions
Mathematically model associative learning as driven by prediction error: learning occurs when outcomes are surprising, and ceases when outcomes are fully predicted.
The Equation
ΔV_A = change in associative strength of CS_A
α_A = salience (learning rate) of CS_A
β = learning rate parameter for the US
λ = maximum associative strength the US can support
ΣV = total associative strength of all CSs present
The key term is (λ − ΣV): the prediction error. When the sum of all CSs' associative strengths (ΣV) is less than the maximum (λ), there is a positive prediction error and learning occurs. When ΣV equals λ, the outcome is fully predicted and no further learning occurs. When ΣV exceeds λ (as can happen in extinction), there is a negative prediction error and associative strength decreases.
Phenomena Explained
The model elegantly accounts for acquisition (prediction error decreases as learning proceeds), extinction (negative prediction error reduces associative strength), and blocking (when CS_A already predicts the US, adding CS_B to the compound produces no learning about B because the prediction error is zero). It also explains conditioned inhibition, overshadowing, and the effects of US magnitude on learning.
Wolfram Schultz's discovery that midbrain dopamine neurons encode reward prediction errors — firing more than baseline when rewards are unexpected, at baseline when rewards are predicted, and below baseline when expected rewards are omitted — provided a stunning neural implementation of the Rescorla-Wagner prediction error. This connection between a computational model of animal learning and a specific neurotransmitter system has been enormously influential, linking associative learning theory to reinforcement learning in computer science and to dopamine's role in motivation, addiction, and decision-making.
Limitations and Extensions
The model cannot account for all learning phenomena. It fails to explain latent inhibition (pre-exposure to the CS without the US slows subsequent learning), the relative validity effect, and backward blocking. Extensions such as the Pearce-Hall model (which focuses on attention to stimuli), the temporal difference (TD) learning model (which handles temporal aspects), and Bayesian models of learning have addressed various limitations while preserving the core insight that prediction error drives learning.
Disorders
- Prediction error dysfunction in schizophrenia
- Aberrant learning signals in addiction
- Extinction — Failure to detect a contralesional stimulus only when a competing ipsilesional stimulus is presented simultaneously; single stimuli detected normally.