RESEARCH DOSSIER
Associative Learning and Prediction-Error Models
Abstract. Associative learning is investigated as the experience-dependent acquisition and revision of predictions about consequential events. This fundamental research monograph critically synthesizes primary experimental studies, influential formal theories, contemporary computational models, and convergent neural evidence. Beginning with contingency and conditioning experiments, it develops the Rescorla–Wagner prediction-error formulation and its experimental implications for blocking, overshadowing, conditioned inhibition, overexpectation, and extinction. It compares fixed and adaptive associability, including the attentional theories of Mackintosh and Pearce–Hall, and examines temporal-difference learning, representation of cue compounds, sensory outcome identity, latent-cause inference, and volatility-dependent updating. The empirical discussion covers human causal judgments, animal conditioning, electrophysiological reward-prediction signals, functional neuroimaging, and contemporary studies through October 2026 on verbally induced blocking, temporal causal learning, cue-specific certainty, continuous outcomes, and psychophysiological limits of prediction-error measurement. It separates observed effects, model-generated predictions, neural correlates, and mechanistic claims, with attention to experimental controls, latent-variable identifiability, model recovery, replication, and external validity. The synthesis establishes cue competition and expectation-dependent acquisition as durable experimental phenomena, identifies the complementary explanatory functions of error, attention, timing, and context, and develops a comparative experimental program for distinguishing these mechanisms. Artificial algorithms serve as fully specified computational comparators within the Artificial Era. This work is a critical narrative synthesis of source-identified scholarship; no original experiments or systematic-review protocol are claimed.
Keywords: associative learning; prediction error; Rescorla–Wagner model; blocking; overshadowing; conditioned inhibition; overexpectation; extinction; Mackintosh model; Pearce–Hall model; associability; temporal-difference learning; reward prediction error; latent causes; volatility; uncertainty; causal learning; computational psychology
Associative Learning as a Predictive Research Object
Associative learning is an experience-dependent alteration in the influence of predictive stimuli, actions, or conjunctions of events on later behavior. Its experimental object is the change in a conditional response as the learner encounters a structured sequence of cues and outcomes. A response can be driven by acquired relations among events, by representations of their causal organization, by attentional allocation, and by the motivational consequences of an outcome. The central research question is therefore which features of experience alter the learner's expectation of an outcome, and under which procedures that expectation can be inferred from behavior. Controlled variation of probabilities, compound cues, temporal relationships, and test conditions makes the question experimentally tractable. Rescorla's contingency experiments demonstrated why the probability of an event outside the conditioned stimulus must enter any explanation of acquisition; isolated cue–outcome pairing frequencies omit an essential part of the informational environment Rescorla (1968).
Prediction-error models describe learning as adjustment driven by a discrepancy between an anticipated and an encountered event. In the simplest case, the discrepancy is signed: an unexpectedly present outcome produces a positive update and an unexpectedly absent outcome produces a negative update. This framework connects classical conditioning, human causal judgment, and some computational descriptions of reward learning, but each extension requires an explicit account of what is predicted, how competing predictors are combined, and how predictions influence the measured behavior. The Rescorla–Wagner family begins with a scalar aggregate of associative strength; attention-based models introduce changes in the cue's capacity to acquire new strength; latent-state approaches allow beliefs about the identity of the current situation to change Rescorla & Wagner (1972) Pearce & Hall (1980) Gershman (2015).
The scope of this monograph is the formal and empirical explanation of predictive associative acquisition, with special attention to competition among cues, signed error, associability, temporal credit assignment, and discriminating experimental tests. Neural reward circuits are considered where they provide tests of formal variables, while the full biology of dopamine belongs to a separate neuroscience inquiry. Planning and extended instrumental control enter as boundary comparisons. This scope matters because a good fit to a conditioned response does not by itself establish the neural implementation of a learning rule, and a neuronal signal correlated with error does not uniquely establish the rule that governs later behavior. Explanation proceeds across identifiable levels: the experimental contingency, the represented prediction, the computational update, the response-production process, and a possible biological implementation Schultz et al. (1997) Gläscher et al. (2010).
Historical Foundations: Contingency, Conditioning, and Expectation
Early conditioning experiments established that behavior could change systematically when an initially neutral signal preceded a consequential event. Subsequent research moved from the occurrence of pairings to their informational structure. The theoretical issue sharpened when investigators realized that a stimulus can be followed regularly by an outcome and nevertheless produce little conditioning if that outcome is equally likely when the stimulus is absent. In an influential conditioned-fear experiment, Rescorla manipulated probabilities of shock in the presence and absence of a conditioned stimulus, thereby demonstrating that the correlation between signal and outcome had explanatory force beyond their joint occurrences. This design helped make predictive contingency a central object of learning research and still illustrates why a baseline condition is essential Rescorla (1968).
A second transformation concerned learning with multiple simultaneously presented signals. If separate elements were learned independently whenever an outcome followed them, a newly added cue should acquire strength under reinforced compound training even when a previously established signal already predicts the outcome. Blocking procedures produced an important alternative pattern: a prior predictor can strongly reduce learning about the newly introduced cue. The observation establishes cue competition under relevant procedures and motivated models that compute error from the combined predictions of all present elements. The experimental finding, however, does not dictate a unique psychological process. Attention, inferential judgments, changes in effective representation, and retrieval at test can also contribute. Later assessments of the Rescorla–Wagner model accordingly judged it by the scope and limits of the total experimental record rather than by blocking alone Miller et al. (1995) Mackintosh (1975).
The history of quantitative associative learning includes attempts to explain why the effectiveness of a cue changes over training. Mackintosh proposed that cues which are relatively good predictors can gain associability, whereas Pearce and Hall emphasized that uncertain or surprising outcomes can preserve attention to cues whose consequences remain unresolved. These models formalized distinct relationships between predictive accuracy and attention; they did not simply append an unspecified attention mechanism to conditioning. They also led to tests in which learning about a cue is measured after a transfer to a new outcome, helping separate the amount already learned from the ease of learning something new Mackintosh (1975) Pearce & Hall (1980).
Experimental Definitions of Prediction, Error, and Associative Strength
An association is operationally inferred from a change in performance attributable to a history of exposure to one or more event relations. Associative strength is a model variable, often denoted V, whose relation to an observed response must be specified through a response function. It cannot be read directly from salivation, freezing, expectancy ratings, choice, or pupil dilation. Prediction is likewise task-dependent: the expected probability of shock, the anticipated magnitude of a reward, and the identity of an expected sensory outcome are different quantities. A mathematical model must identify which quantity its state variable represents. Human causal ratings also require interpretation because participants can apply explicit knowledge about instructions, task structures, and hypothetical alternatives Rescorla (1968) Corlett et al. (2004).
A signed prediction error is commonly expressed as the realized target minus the predicted target. In an outcome coded 0 or 1, omission creates a negative error when an outcome was expected, and unexpected occurrence creates a positive one. Learning rate, sometimes denoted alpha, converts that discrepancy into an update. If a learning model fits trialwise behavior, the estimated rate may reflect how quickly a particular representation is updated within that model; alternative representations and response rules can produce similar apparent rates. This identification problem requires parameter-recovery and model-recovery analyses. The inference from trialwise response data to a computational variable is an explicit modeling step rather than a direct observation Sutton (1988) Gershman (2015).
Surprise, uncertainty, and signed error need distinct measures. Absolute error captures the size of a mismatch regardless of sign; uncertainty concerns the dispersion of plausible outcomes or the reliability of inferred contingencies; volatility concerns the frequency with which the environment's underlying contingencies change. An unexpected event can be highly surprising in a stable environment, or a changing environment can be volatile while its current outcome remains predictable. Attention models and hierarchical learning models assign different functional roles to these quantities. A study that calls all of them prediction error conceals the very mechanism it seeks to test Pearce & Hall (1980) Yu & Dayan (2005) Behrens et al. (2007).
Acquisition, Expression, and the Interpretation of Behavior
Learning experiments observe behavior under specific performance conditions. The amount of a conditioned response depends on available response systems, the motivational status of an outcome, competing actions, temporal context, and the testing protocol. A weak conditioned response can reflect a weak acquired prediction, suppressed behavioral expression, or altered retrieval of an existing relation. This distinction becomes decisive in extinction, where repeated nonreinforcement typically reduces performance, but renewal or recovery in other circumstances shows that the original acquisition history can retain behavioral influence. Models that equate the observed response with current associative strength must be evaluated against these dissociations Gershman et al. (2010) Miller et al. (1995).
Training protocols also differ in what the organism experiences as the relevant unit. Simultaneous compounds can be encoded as individual elements, configural patterns, or mixtures. Instructions in human tasks can transform a nominal cue into a symbol for a proposition, and response requirements can induce strategic hypotheses. A full design therefore reports cue identity, modality, duration, timing, outcome magnitude, intertrial interval, prior cue exposure, and test order. It should separate acquisition trials from tests because unreinforced tests may themselves change predictive expectations. These controls support an interpretation of trialwise change without treating a performance curve as a transparent image of a hidden association Miller et al. (1995) Kelly & Ludvig (2025).
The decisive methodology is a comparison of interventions that competing accounts predict differently. Blocking alone supports the existence of cue competition under the chosen procedure; a transfer test of associability probes whether predictive history changes attention to that cue. Selective revaluation after compound learning probes whether representation and retrieval matter. Unexpected omission during overexpectation probes whether summed predictions can exceed the received outcome. Such convergent evidence places stronger constraints on a model than a single acquisition curve with adjustable parameters Pearce & Hall (1980) Corlett et al. (2004) Miller et al. (1995).
Research Questions and the Boundaries of the Monograph
The first research question asks how well a common-error rule explains acquisition and competition among predictors. A second asks whether cue associability remains fixed or changes with predictiveness, surprise, or previous experience. A third asks how predictions are allocated across time, outcomes, and representational states. A fourth concerns the evidential relation between formal learning rules and observed neural signals. A fifth asks which model comparisons, replications, and manipulations permit robust inference from behavior. These questions define an integrated assessment of associative prediction rather than a claim that a single algorithm explains the entire psychology of learning Rescorla & Wagner (1972) Mackintosh (1975) Pearce & Hall (1980) Schultz et al. (1997).
Four levels of explanation organize the investigation. Behavioral analysis characterizes conditional responding, expectancy judgments, choice, and physiological indices. Computational analysis describes error terms, associative weights, learning rates, and posterior beliefs. Mechanistic analysis studies attention, memory retrieval, neural activity, and their interventions. Comparative analysis asks what transfers across animal preparations, human laboratory judgment, and artificial learning algorithms. The levels can corroborate one another but are not logically interchangeable: the same computational trajectory can arise from distinct processes, while a system may carry an error-like signal yet use it only indirectly to regulate behavior Gläscher et al. (2010) Gershman (2015).
Neighboring work addresses the broader science of reinforcement learning and model-based control, the cellular and systems neuroscience of reward-prediction signaling, and other varieties of learning. This dossier examines their points of contact when they constrain the learning models themselves. Its principal synthesis is a program for testing precisely specified predictions against differentiated experimental evidence, preserving the distinction between a mathematical rule, its fit to observations, and its possible implementation in a particular learner Miller et al. (1995) Niv & Schoenbaum (2008).
The Rescorla–Wagner Error-Correction Rule
The Rescorla–Wagner model expresses trialwise associative change for a cue as ΔV_i = α_i β (λ − Σ_j V_j), where V_i is the associative strength assigned to cue i, α_i is its salience or fixed associability parameter, β scales updating with the outcome, λ represents the outcome-supported asymptote, and the summation is taken across cues currently represented as present. Alternative notational conventions combine α and β into a single learning rate. The central substantive commitment is that the experienced outcome is compared with the aggregate expectation generated by present stimuli. Each cue's update follows the shared prediction error, weighted by its own acquisition parameter. This means that learning about an element depends on predictions generated by other elements, a principle that changes the unit of analysis from isolated cue–outcome pairings to compound trials Rescorla & Wagner (1972) Miller et al. (1995).
Consider a trial on which signal A is already a strong predictor of a unit outcome and new signal X is added. If the animal expects nearly the full outcome from A, the compound A+X produces little prediction error, so X receives only a small update. If A had carried little predictive strength, the same A+X reinforcement would supply a substantial update to both elements. This basic comparison explains blocking under assumptions about representation and response mapping. Conversely, when the outcome is absent after a strongly predictive compound, both present cues receive negative increments. The model's appeal is its ability to derive apparently different conditioning effects from one trialwise operation; its precision also exposes its limits whenever those effects depend on representation, context, or attention beyond the original formulation Rescorla & Wagner (1972) Miller et al. (1995).
The learning rule concerns the state of a model at trial boundaries. Estimating a trajectory from behavior requires a likelihood or observation model, for example a logistic mapping from latent value to a binary choice or an appropriate link between value and conditioned-response intensity. The same response curve can often be generated by different combinations of initial values, learning rates, ceiling effects, and lapse probabilities. Researchers should therefore report the assumed observation function and parameter constraints, not merely the equation governing changes in V. A successful reconstruction of aggregate response rates is weaker evidence than discriminating predictions for new compound training, reversals, and unreinforced tests Sutton (1988) Gershman (2015).
Cue Competition: Blocking and Overshadowing
Blocking is conventionally tested by training A as a predictor of an outcome, then presenting reinforced compounds A+X, followed by an independent test of X. A comparison group receives a history that leaves A less predictive or controls for compound exposure. When responding to X is reduced relative to the control, the prior information supplied by A has altered what X acquires or expresses. A Rescorla–Wagner account attributes this reduction to small prediction error during compound training, while attentional and inferential accounts can attribute part of it to how the learner processes or subsequently evaluates X. Appropriate controls are essential because merely observing low responding to X cannot distinguish failure to encode X from learned irrelevance, overshadowing, or response competition Miller et al. (1995) Mackintosh (1975).
Overshadowing commonly involves training two cues together and finding that one, often the more salient, acquires less control than it would have acquired alone. In an error-correction model, higher initial associability can allow the salient element to capture a larger share of a limited aggregate prediction during acquisition. The resulting allocation also depends on exposure duration and whether compound elements are represented separately. Overshadowing and blocking therefore implicate related but experimentally distinguishable aspects of competition: prior learning for blocking, differential acquisition during compound learning for overshadowing. They should be analyzed with factorial manipulations of cue history and salience rather than treated as interchangeable labels Rescorla & Wagner (1972) Miller et al. (1995).
Cue competition can also appear when absence of an expected outcome is informative. Studies of discriminations involving signals for reinforcement and nonreinforcement illustrate how stimuli acquire control from the difference between predicted and obtained consequences, sometimes including inhibitory or occasion-setting properties Seraganian & Vom Saal (1969) Rescorla (1971). Importantly, variation in compound responses after training may arise at acquisition, at retrieval, or during the response test. A rigorous experimental sequence includes element tests, compound tests, and transfer controls to establish which prediction a given computational account uniquely entails.
Conditioned Inhibition, Overexpectation, and Negative Error
A conditioned inhibitor is a cue whose history signals reduction or omission of an otherwise expected outcome. Under an error-correction account, compound trials pairing an excitatory cue A with a candidate inhibitor X and omitting the expected outcome can drive X toward negative associative strength. Because a simple decrease in responding during A+X could be caused by distraction or generalization decrement, standard inhibitory assessment combines a summation test, in which X reduces responding to another independently trained excitor, with a retardation test, in which X is slower to acquire excitation when reinforced. Interpretations depend on the controls and on whether the inhibitor acts on an outcome representation or alters the significance of the occasion Rescorla (1971) Miller et al. (1995).
Overexpectation provides an especially informative test of aggregate prediction. Two separately trained signals, A and B, each predict an outcome. When A+B is then reinforced with an outcome no larger than the one individually predicted by either signal, the summed model prediction exceeds the actual outcome. A shared negative error results even though the compound remains reinforced, and subsequent tests can reveal weakened responding to the elements. This signature is difficult to explain by a simple rule that strengthens every cue whenever reinforcement occurs. The size of the predicted effect nevertheless depends on the learner's treatment of compound salience, whether expected intensities sum, and how the test maps associative strength to behavior Rescorla & Wagner (1972) Miller et al. (1995).
Negative prediction errors are experimentally consequential because they reveal what a learner believed would happen before the event. With well-controlled schedules, the failure of a predicted outcome can reduce value even in the absence of an aversive or punishable event. At the same time, a scalar coding of omission may be inadequate when outcomes have multiple identities, sensory dimensions, or motivational values. An organism can receive an equally valuable but different outcome and show learning about its identity. Such results motivate richer outcome representations and multiple error channels, while retaining the basic principle that the computational significance of feedback depends on prior expectation Niv & Schoenbaum (2008) Gläscher et al. (2010).
Extinction, Recovery, and the Persistence of Learned Relations
In basic extinction procedures a conditioned cue is presented repeatedly without its former outcome. An error-correction rule reduces the cue's predictive strength through successive negative errors until the expected magnitude approaches the encountered value. The process predicts a decline in conditioned performance and often provides an adequate descriptive account of initial extinction. But extinction does not erase the experimental history: renewed responding after a context change, spontaneous recovery with time, and reinstatement after unsignaled outcome presentations show that the relation between old and new learning is structured by retrieval and context. A model with one undifferentiated strength for a cue faces difficulty representing acquisition and extinction knowledge that govern behavior under different latent situations Gershman et al. (2010) Miller et al. (1995).
Latent-cause analyses propose that the learner may infer whether trials belong to the same generating context or to different hidden causes. If extinction observations are assigned to a new cause, the old association can remain available and later reemerge when retrieval favors its original cause. This framework links apparent response loss to probabilistic inference about state identity and explains why spacing, temporal separation, and environmental similarity can affect relapse-like patterns. It makes discriminating predictions about generalization across contexts and about the consequences of manipulating apparent boundaries between learning phases. The theory illustrates the value of distinguishing an error signal from the representation whose value is being updated Gershman et al. (2010) Gershman (2015).
The broader scientific implication is that a decline in response after omission is evidence for altered expressed expectations under a specified test; determining whether a stored relation was weakened, a second relation was formed, or its retrieval changed requires additional experiments. For clinical analogies involving fear extinction, this distinction is often consequential, but interventions cannot be prescribed from a generic conditioning equation. Translational interpretation must attend to species, response system, reinforcement schedule, and the separate evidence base for therapeutic practice. The present analysis treats extinction as a mechanistic case study of how associative models handle contradictory evidence Gershman et al. (2010) Pearce & Hall (1980).
Mathematical Predictions and Model Identifiability
A formal learning model earns explanatory leverage by specifying both its latent-update rule and its observable predictions. If distinct models can reproduce the same gradual acquisition curve after their learning rates are adjusted, that curve does not identify the underlying algorithm. A stronger evaluation chooses test sequences on which models disagree: blocking after altered prior validity, associability transfer after changes in uncertainty, overexpectation with controlled compound salience, or extinction separated across contexts. Parameter recovery examines whether known generating parameters can be estimated from simulated data under the planned design; model recovery examines whether the selection procedure can identify which candidate generated those data. These tests address practical identifiability before theoretical preferences are assigned to the result Gershman (2015) Miller et al. (1995).
For binary outcomes, a trialwise modeling framework can generate predicted outcome probabilities and evaluate predictive likelihood for held-out trials. For ordinal ratings or continuous conditioned measures, appropriate observation distributions and task-dependent measurement noise are required. Cross-validation must respect temporal structure: random trial shuffling can leak information from later learning into earlier model fitting. Group-average fit can obscure substantial individual differences in learning strategy, particularly when participants switch between explicit reasoning and incremental learning. Hierarchical estimation can stabilize noisy parameters, but its priors and shrinkage assumptions must be disclosed. The proper benchmark is prediction of experimentally independent data, not the flexibility to interpolate the training sequence Daw et al. (2011) Behrens et al. (2007).
An interpretable modeling report states initial values, outcome coding, input features, choice or response rule, free parameter bounds, fitting objective, cross-validation unit, uncertainty intervals, and recovery results. It also reports whether an alternative model with fewer parameters makes comparable predictions. A high correlation between a fitted error regressor and brain activity offers a useful hypothesis but can be sensitive to parameter choice and shared trial structure. Causal identification strengthens when manipulations selectively alter the error, associability, or latent-state variable and the observed response changes as specified. The distinction between descriptive adequacy and mechanistic identification is the methodological foundation for the subsequent experimental comparisons Corlett et al. (2004) Roesch et al. (2012).
Mackintosh's Theory of Learned Predictive Attention
Mackintosh's attentional theory proposes that the capacity of a stimulus to enter new associations changes with its relative predictive accuracy. A cue that forecasts the outcome better than other available cues becomes more associable, whereas a cue that is less informative loses priority for subsequent learning. The account treats selective attention as something that experience teaches, making an organism's allocation of processing resources sensitive to competitive reliability. This idea addresses a limitation of constant-salience error-correction rules: a cue that has repeatedly failed to predict an outcome need not be treated as equally eligible for future acquisition simply because an outcome later becomes surprising. The model yields experimentally assessable consequences for learning about previously reliable and irrelevant stimuli when their outcome assignments change Mackintosh (1975) Miller et al. (1995).
An associability transfer procedure can distinguish attention from previously learned associative strength. During an initial phase, one cue is consistently predictive while another varies independently of the relevant outcome. In a later phase both cues are assigned new consequences, and the rate at which the learner acquires those new relations reveals how prior predictive status affects processing. The key challenge is to balance initial associative values, perceptual familiarity, and test-induced generalization, because faster transfer may arise from several sources. Mackintosh's proposal is strongest where the observed pattern tracks relative accuracy under controlled comparisons; the empirical record has also motivated hybrids that allow multiple attentional processes to coexist Mackintosh (1975) Pearce & Hall (1980).
Human studies introduce an additional interpretive layer. Participants can explicitly track which features a task rewards them for attending to, and instructions can alter perceived relevance before any reinforcer is experienced. Learned predictive attention may consequently be instantiated through multiple kinds of representation, including explicit judgments about informative dimensions. Evidence for the general principle does not uniquely determine whether associability changes at an early perceptual stage, through strategic allocation, or through a common computational weighting of cue errors. An informative experiment combines response-based learning rates with independent measures of gaze, recall, recognition, and transfer to separate these possible realizations Mackintosh (1975) Kelly & Ludvig (2025).
Pearce–Hall Associability and Unsigned Surprise
Pearce and Hall proposed a different relationship between prediction and attention. When an outcome is poorly predicted, cues present on that trial retain or acquire the capacity to support subsequent learning; when the outcome becomes well predicted, further attention to those cues declines. In simple formulations, associability on the next trial is proportional to the absolute prediction error on a preceding trial. This unsigned surprise variable differs from the signed error used to change associative strength. A cue may be eligible for rapid updating because its consequences are uncertain even when the eventual direction of a value change is not yet known. The distinction creates a testable difference between learning how much a cue predicts and learning how much to learn from that cue Pearce & Hall (1980) Roesch et al. (2012).
A volatile cue that continues to generate unexpected outcomes can therefore retain high associability, whereas a dependable cue may be processed economically once its consequences are established. This view captures an important adaptive tradeoff: attention should remain available when predictions prove unreliable, but processing can be reduced when the environment is stable. The model does not entail that all surprising events are functionally identical; stimulus salience, temporal attention, and outcome identity can constrain which cues receive the enhanced update. Furthermore, apparent increases in attention measured through eye movements or orienting may not map one-to-one onto an abstract associability parameter. Such parameters should be subjected to independent transfer and response tests Pearce & Hall (1980) Yu & Dayan (2005).
Unsigned surprise also needs to be distinguished from tonic uncertainty and from the absolute deviation generated by a possibly misspecified model. When a learner changes its hypothesis about the current context, a sudden behavioral shift can appear as a rapid learning-rate change even if the underlying computational operation is belief revision rather than faster incremental error correction. Model comparisons should include both dynamic associability and explicit latent-state alternatives, especially in reversal or change-point tasks. The goal is to identify whether surprising outcomes induce sustained increases in subsequent processing, alter state inference, or simply contribute large one-time signed updates Behrens et al. (2007) Gershman (2015).
Contrasting Attentional Accounts Through Experimental Transfer
The two attentional traditions make especially different predictions when a cue is highly predictive but therefore unsurprising. In a Mackintosh-style account, relative reliability can increase its eligibility for new learning; in a Pearce–Hall account, low residual error can reduce its continued eligibility. Carefully designed transfer tasks expose this difference by measuring the speed of acquiring a new relation under matched exposure, initial strength, and stimulus salience. Some data favor one emphasis under particular tasks, while other experiments and analyses suggest that predictiveness-based and uncertainty-based influences can operate together. A broad scientific synthesis must therefore distinguish the empirical question of which effect exists from the model-selection question of which architecture most economically reproduces the pattern Mackintosh (1975) Pearce & Hall (1980) Roesch et al. (2012).
A decisive study should randomize and counterbalance cue identities, manipulate relative accuracy and residual surprise separately, and ensure that neither condition can be solved by a simple strategy of repeating a familiar response. When a test phase introduces new outcomes, participants should receive equivalent opportunities to detect them. Observed transfer differences can then be modeled alongside explicit expectancy reports, eye tracking, and independent learning-rate proxies. The inferential benefit of collecting multiple measures is that a single behavioral curve often mixes attention, value, memory, and decision caution. Each measure has its own limitations, but a constrained pattern across modalities is more informative than a single fitted free parameter Li et al. (2011) Roesch et al. (2012).
Neuroscientific comparisons have reported correlates compatible with both unsigned and signed learning variables, including cases in which the corresponding signals occupy partially different networks. Such findings motivate, rather than settle, multi-process accounts: a region's response may encode salience, action requirements, context change, or a correlated regressor. The scientific test becomes whether independently manipulated predictiveness and surprise produce dissociable changes in future learning, and whether neural measures improve out-of-sample prediction after relevant competing variables are controlled Li et al. (2011) Roesch et al. (2012) Corlett et al. (2004).
Temporal-Difference Learning and Prediction Across Time
Trial-boundary rules compare an outcome with an expectation at discrete learning events. Many conditioning preparations additionally require a model of when an outcome will occur. Temporal-difference learning extends error correction to a sequence of successive predictive states, using discrepancies between adjacent estimates to adjust predictions earlier in time. In a common formulation, δ_t = r_t + γV(s_(t+1)) − V(s_t), with r_t the current reward signal and γ a discount factor. This structure allows learning from changes in predicted future outcomes before the final reward appears. It supplies a formal connection between conditioning models and sequential learning without assuming that every psychological predictor is a scalar reward value Sutton (1988) Schultz et al. (1997).
The temporal-difference framework explains why an initially outcome-locked prediction-error response can move toward a reliably preceding cue as learning progresses. Once the cue predicts the outcome, the outcome itself becomes less surprising; omission of a predicted event can generate a negative error at the expected time. Classical observations of midbrain dopamine responses are broadly compatible with this account under particular appetitive tasks. Yet temporal precision depends on the representation of states and elapsed time. If an analyst arbitrarily divides an interval into time bins, the model's predictions may change with the chosen discretization. Distinguishing an experimentally discovered temporal representation from a fixed modeling convenience remains an important problem Schultz et al. (1997) Waelti et al. (2001) Sutton (1988).
Recent theory has developed approaches that jointly infer causal relations and temporal distributions of events. Hamou, Gershman, and Reddy examined how scale-invariant representations of intervals could accommodate timing phenomena and error-correction signatures within a common causal-learning framework. Their work illustrates a productive direction of inquiry: a successful account should predict acquisition as a function of the ratio between intertrial and cue–outcome intervals, and should characterize the timing of anticipatory responses rather than fitting only their aggregate magnitude. The relevant results are theoretical and computational explanations of selected published patterns, with specific predictions open to direct discrimination against alternatives Hamou et al. (2025).
Elemental, Configural, and Outcome-Specific Representations
An elemental model assigns strengths to component cues, while a configural model allows a compound to have an identity with predictive properties not reducible to those of its isolated elements. A stimulus compound can contain both shared and unique representational features, and those choices influence predictions for generalization, discrimination, and transfer. For example, if training rewards a compound but not its individual elements, the learner may need a representation that distinguishes the conjunction from its parts. A failure of a simple elemental summation rule on this task can therefore reflect a representational limitation rather than the failure of error correction itself. Formal comparison must specify how inputs are encoded before comparing rules for changing their values Miller et al. (1995) Gershman (2015).
Outcome specificity introduces another dimension. Learning that a signal predicts a particular food differs from learning only that something rewarding will occur. Distinct sensory outcomes may evoke different anticipatory responses, support sensory-specific revaluation, or be used differently in choice. Scalar-value models can describe overall appetitive acquisition while remaining unable to predict a shift in outcome identity at constant subjective value. This makes the choice of target variable an empirical commitment. Behavioral tests should include outcome-specific responding, devaluation, and changes in contingency, then assess whether the model requires separate outcome dimensions or a richer predictive distribution Niv & Schoenbaum (2008) Pool et al. (2023).
A meaningful computational synthesis therefore treats learning rule and representation as interacting hypotheses. A broad feature representation with a simple incremental update can sometimes match the output of a narrow representation with a more complex update. Without independent control of the representation, fitted parameters can absorb this difference, producing misleading claims about attention or neural error computation. Cross-context generalization and transfer to novel compounds constrain representational assumptions, while temporal manipulations constrain how states are defined. Representation is part of the explanatory model and must be evaluated with the same rigor as the update equation Gershman et al. (2010) Hamou et al. (2025).
Neural Evidence for Reward Prediction Errors
Electrophysiological recordings in primates established a major neural test of formal prediction. In classic appetitive tasks, midbrain dopamine neurons responded to unpredicted rewards, acquired cue-related responses when a predictor became reliable, and showed reductions in activity when an expected reward was omitted. Schultz, Dayan, and Montague interpreted this pattern in the language of temporal-difference learning, proposing a relation between neural responses and reward-prediction errors Schultz et al. (1997). The value of the observation is its trialwise correspondence to expectations under a defined schedule. The warranted conclusion concerns signals measured in particular neuron populations and conditions; it does not identify dopamine with pleasure, conscious anticipation, or every learning operation in the brain.
Waelti, Dickinson, and Schultz supplied a more discriminating test by examining cue competition. Their dopamine observations followed key qualitative predictions of formal learning theory in a blocking-related task, suggesting that the response depended on whether prior learning rendered a reward expected Waelti et al. (2001). Bayer and Glimcher further investigated quantitative scaling of neuronal firing with reward prediction error, adding constraints on error coding beyond the simple presence or absence of phasic responses Bayer & Glimcher (2005). Together these studies connect behavioral rules to physiological measurements, while leaving open how different dopaminergic projections, recording methods, and task structures generalize across reinforcers and actions.
A neural correlate of error is not necessarily an exclusive causal learning signal. Sensory novelty, movement, salience, state transitions, and outcome identity may influence recorded activity. Inference improves when researchers manipulate expectation independently of outcome magnitude, estimate behavioral learning on each trial, compare multiple candidate neural regressors, and intervene on the system with temporal specificity. The study of distinct state- and reward-prediction signals in human neuroimaging illustrates why computational variables must be separated even when they co-occur in a task Gläscher et al. (2010) Niv & Schoenbaum (2008).
Human Learning, Expectation Reports, and Neural Dissociations
Human experimental protocols often combine predictive judgments with measures of attention, choice, and brain activity. Corlett and colleagues used retrospective revaluation of causal associations to study whether prediction-error-like neural activity could accompany changes in human judgments when the relevant causal history was reconsidered Corlett et al. (2004). The finding is important because participants can revise expectations in the absence of a simple immediately experienced cue–outcome pairing for every affected relation. It therefore creates an empirical bridge between associative mechanisms and inferential processes while also showing how the interpretation of a fitted error term depends on an account of representation.
Li and colleagues investigated distinct contributions of human striatum and amygdala in associative learning, providing evidence relevant to the coexistence of error-driven acquisition and attentional processes Li et al. (2011). Roesch and colleagues explicitly examined neural correlates corresponding to Pearce–Hall and Rescorla–Wagner variables in related learning systems Roesch et al. (2012). These studies support a research landscape in which multiple formal signals can be measured. They do not authorize an inference that any one brain area uniquely implements an entire textbook learning algorithm. Interpretation requires task-specific computational fits, control of correlated regressors, and an explicit measurement model linking latent variables to recorded signals.
Human experiments also face strategic and linguistic influences. A participant instructed that one cue is predictive can use that statement to allocate attention or infer the irrelevance of another cue, without reproducing every increment prescribed by a low-level associative algorithm. Kelly and Ludvig reported that explicit verbal descriptions could produce blocking of subsequent experiential learning in human reward-causality judgments, demonstrating a route from symbolic information to a classic competition pattern Kelly & Ludvig (2025). The result extends the empirical phenomena to be explained; it remains an open question whether the verbal manipulation changes the same underlying learning variables, changes higher-level task beliefs, or both.
A 2026 study by Liu, Linnell, and Bach provides an unusually direct warning against interpreting psychophysiological outcome reactions as pure prediction-error measures. The investigators analyzed a pooled set of eight previously published partial-reinforcement fear-conditioning studies and a separate experiment that varied the probability of the unconditioned stimulus. Across skin conductance, pupil, heart-period, and respiratory indices, outcome occurrence and omission were distinguishable. The abstract nevertheless reports that the pattern did not conform to a simple signed or unsigned prediction-error encoding across the measured modalities in the first dataset. This negative mechanistic finding is scientifically useful: sensitivity to an event or omission is not equivalent to quantitative representation of the model's error term. The precise physiological signals and their relation to formal predictions require separate validation Liu et al. (2026).
Reward, State, and Sensory Outcome Prediction
The variable labeled prediction error differs across models and experiments. A reward error concerns the discrepancy between received and expected motivational value. A state-prediction error concerns an unanticipated transition between task states, and a sensory error concerns a mismatch in particular outcome features. The same event can carry zero reward error and substantial state or identity surprise. Gläscher and colleagues reported dissociable human neural signals relevant to state and reward prediction in a task comparing model-based and model-free learning Gläscher et al. (2010). This division makes the model's target space a scientifically testable choice rather than a terminological detail.
An organism can learn that a previously appetitive cue predicts an aversive outcome, that a reward occurs at a new time, or that the same reward now arrives in a different sensory form. These changes produce different kinds of uncertainty and potentially recruit different behavioral systems. A scalar strength update predicts only the aggregate direction of value correction unless supplemented by representations of identity, timing, and context. Models should define how component predictions are integrated and which errors train each component. Studies of devaluation-sensitive and -insensitive Pavlovian learning in humans underline how changes in an outcome's current value can reveal multiple routes by which cues influence behavior Pool et al. (2023) Niv & Schoenbaum (2008).
This multiplicity does not diminish the usefulness of signed reward error. It locates that variable within a larger predictive architecture in which an expectation concerns several properties of the environment. Good empirical work estimates the error for the property actually manipulated and checks that the apparent neural or behavioral correlation is not better explained by correlated surprise or action preparation. A model of associative learning gains scientific specificity by stating what it can predict and by identifying a discriminating task for every proposed error channel Gläscher et al. (2010) Daw et al. (2011).
Learning Rates Under Uncertainty and Volatility
A fixed learning rate is a useful approximation when the statistical properties of the environment remain stable. In a changing task, however, an adaptive learner should distinguish an isolated unlikely outcome from evidence that the underlying contingency itself has changed. Yu and Dayan developed a theoretical account linking uncertainty, neuromodulation, and attention; Behrens and colleagues examined how human learning adapts to estimates of environmental volatility Yu & Dayan (2005) Behrens et al. (2007). The resulting research program asks when uncertainty should dampen an update, when a detected change should increase learning, and whether these adjustments can be recovered from trialwise behavior.
Stable noise and volatility are distinct experimental factors. A noisy but stable cue can remain unreliable across many observations without changing its true outcome distribution. A volatile cue can be precisely predictive within a short block yet undergo abrupt reversals. Factorial variation of outcome randomness and reversal frequency tests whether participants maintain separate estimates of irreducible uncertainty and changes in environmental structure. A simple surprise-based associability model can respond to unexpected outcomes in both cases, whereas hierarchical models may infer different long-term consequences. Successful discrimination depends on adequately long sequences, counterbalanced reversals, and held-out predictions of adaptation after changes Behrens et al. (2007) Pearce & Hall (1980).
At the neural level, uncertainty-associated activity may reflect attention allocation, arousal, policy changes, or inferred hazard rather than a purified learning-rate parameter. At the behavioral level, a rise in apparent responsiveness may result from switching hypotheses about the current environment. A full model should generate predictions for choices, explicit expectations, and recovery after a return to prior conditions. The explanatory question is whether learning-rate control itself changes or whether the learner changes which representation receives the same update. This question joins associability models to latent-state inference without collapsing them Gershman et al. (2010) Yu & Dayan (2005).
Latent Causes, Context, and Bayesian Association Learning
Latent-cause models propose that observations arise from hidden states or causes whose identities must be inferred along with their predictive consequences. In this approach, the learner allocates a new trial to a familiar latent cause or infers a new one according to the fit between the observation, the context, and prior beliefs. Associative predictions are then conditioned on that inferred cause. This changes the meaning of extinction and reversal: a contradicted outcome can prompt an update to an established state or evidence for a distinct state. Gershman, Blei, and Niv applied this perspective to context, learning, and extinction, showing how context-dependent retrieval can explain dissociations difficult for a single unstructured strength Gershman et al. (2010).
A probabilistic interpretation can sometimes unify features previously expressed through separate associative rules. Gershman developed a framework in which traditional conditioning phenomena are related to inference over latent causes, prediction, and uncertainty Gershman (2015). The theoretical achievement is not an automatic claim about a brain's exact representational format. It is the derivation of testable response patterns from explicit probabilistic assumptions. Whether the learner performs approximate Bayesian inference, samples hypotheses, or uses an incremental heuristic requires further behavioral and neural evidence, especially on tasks designed to separate these mechanisms.
The crucial experimental tests manipulate context boundaries, temporal gaps, stimulus similarity, and the rate at which contingencies change. If the learner infers a new latent cause, a previously acquired prediction may reappear when context once again favors the old cause. If the learner instead overwrites one cue value, similar recovery is harder to derive without additional machinery. Comparisons should specify the number and complexity of latent states allowed and guard against the flexibility of models that can explain any irregularity by postulating an additional state. Parsimony and predictive recovery, especially after untrained context transitions, remain decisive Gershman et al. (2010) Hamou et al. (2025).
Retrospective Revaluation and Learning Beyond Immediate Pairings
Retrospective revaluation occurs when later learning about one cue changes the judged or expressed predictive role of another cue that is no longer present during the later events. In backward-blocking and related designs, compound acquisition is followed by separate training for one element; subsequent tests ask whether the absent partner's inferred causal strength changes. Such patterns challenge a strictly local rule that updates only cues present at the moment feedback occurs. Dickinson and Burke investigated how within-compound associations could mediate revaluation of human causal judgments, thereby providing a route through which a later observation can affect the assessment of a previously paired but currently absent stimulus Dickinson & Burke (1996).
Several computational explanations remain possible. Within-compound associations can permit mediated updates or changes in the retrieved representation of the other cue. Bayesian causal inference can revise the inferred contribution of an absent cause when new evidence changes the plausibility of alternatives. An associative account with explicitly specified retrieval processes can produce retrospective changes without assuming unrestricted reasoning about every possible hypothesis. Ghirlanda showed that patterns of retrospective revaluation could be modeled using a comparatively simple associative learning architecture, illustrating why theoretical disputes require exact models rather than generic labels such as associative and cognitive Ghirlanda (2005) Gershman (2015).
Neuroimaging experiments contribute a further kind of constraint when later information changes causal judgments. Corlett and colleagues examined prediction-error-related brain responses in retrospective revaluation, linking neural signals to experimentally inferred expectations during human learning Corlett et al. (2004). Such evidence can support computational plausibility while leaving underdetermination about the representation and the locus of updating. A stringent replication should include control conditions for cue familiarity, compound learning, instruction comprehension, and the accessibility of earlier associations; ideally it should test predictions for novel stimuli after the revaluation phase.
Contemporary Evidence: Verbal Blocking and Temporal Causal Learning
A contemporary extension of blocking asks whether explicit descriptions of one stimulus's predictive role can influence what is learned when another cue is subsequently experienced. Kelly and Ludvig used human reward-causality tasks and reported two experiments on blocking after verbal information. Their first experiment included conventional experience-based blocking and evidence that descriptions could also inhibit later acquisition; their second manipulated prior training conditions and found that the description effect depended on the earlier task structure Kelly & Ludvig (2025). These results enlarge the explanatory problem because they demonstrate competition between verbally communicated relations and trialwise experiential learning within a common behavioral paradigm.
The interpretation requires care. Verbal information may initialize an expectancy, set an explicit task hypothesis, change attention to a compound member, or alter the participant's willingness to report learning. A useful model comparison would fit the same observation structure across description-only, experience-only, and combined training conditions while allowing the competing accounts to make different predictions about later reversals. The study provides evidence for the observed blocking effect under the reported conditions and an invitation to examine its mechanism, not proof that propositional instructions necessarily operate through one biologically implemented error-correction rule Kelly & Ludvig (2025) Mackintosh (1975).
The temporal dimension has received renewed formal attention. Hamou, Gershman, and Reddy proposed a hierarchical temporal causal-learning model that estimates the distribution of interstimulus intervals and the inferred cause of a reward. Their computational analysis showed how a scale-invariance assumption can relate intertrial timing to acquisition and reproduce selected prediction-error signatures Hamou et al. (2025). The paper is especially significant because the learner's representation of elapsed time is treated as part of what must be explained. Its claims should be evaluated according to the distinction between patterns reproduced by the model, novel behavioral predictions, and neural dynamics that remain to be independently validated in the proposed forms.
Contemporary human evidence also tests whether a cue's previous excitatory or inhibitory history changes its update in a compound. Jahani and Üngör reported experiments using hypothetical food–inflammation relations with outcome magnitudes that could be positive, zero, or negative. Their abstract describes a dissociation across training arrangements: in two experiments, the cue associated with the greater individual prediction error showed greater associative change on a compound test, whereas another experiment favored the cue characterized by lower certainty and individual error. Additional confidence tests examined the inferred uncertainty of the cues. These differences are theoretically informative because they challenge an unrestricted prediction that either uncertainty or individual error alone controls compound revaluation. The paper appeared online in September 2026; this analysis confines detailed claims to its publicly available abstract Jahani & Üngör (2026).
Chow, Don, Colagiuri, and Livesey evaluated the prediction of continuous-valued outcomes in four human contingency-learning experiments. Their work contrasts a simple delta-model architecture, which primarily approximates an outcome's expected magnitude, with a distributed representation that preserves information across outcome values. The authors report that the distributed model fit the experimental data better for nearly all participants under their comparison. The result broadens the explanatory target from learning whether an event will occur to learning structured distributions of outcome magnitude. It also illustrates a general caution: a predictive-error algorithm's adequacy depends on what its representational layer can encode, so a comparison of update rules with incompatible representations must disclose that difference Chow et al. (2026).
Behavioral Paradigms and Controls for a Comparative Test
Associative-learning studies employ conditioned appetitive and aversive responses, human causal ratings, predictive choice, physiological anticipation, and neuroimaging. These measures have different error structures and different dependencies on task instructions. A proper blocking experiment should establish that the initial signal acquired predictive control, that the added cue was perceptually detectable, that compound and test exposure were equated across groups, and that the final test does not simply measure response saturation. Overexpectation designs should establish the summed expectation independently rather than infer it from a subsequent decrease in response. Associability-transfer designs should equate familiarity while changing predictive history. Each control asks which latent explanatory claim follows from the observable Miller et al. (1995) Pearce & Hall (1980).
The interaction between stimulus identity and temporal organization deserves particular scrutiny. Learning can depend on the duration of the conditioned stimulus, the delay to the outcome, the intervals between trials, and background outcome probability. In temporally structured tasks, researchers should record not only whether the outcome appears but when it appears relative to each candidate cause. Carefully controlled intertrial-interval manipulations can separate simple accumulated pairing counts from theories that learn rates, temporal maps, or causal interval distributions. Hamou and colleagues' proposed timing architecture supplies concrete comparative predictions for these manipulations Hamou et al. (2025) Sutton (1988).
Measurements should preserve trial history in a machine-readable sequence rather than aggregate responses solely by condition. Preregistered exclusion criteria, complete reporting of stimulus schedules, individual-level model fits, and reproducible simulation code enable independent reanalysis. Outcome expectancy can be probed intermittently to reduce the risk that constant questioning changes the task. For neuroimaging, fitted learning variables should be estimated without using the neural response under evaluation as the fitting target. Each of these choices makes it easier to distinguish a robust computational regularity from a pipeline-specific correlation Daw et al. (2011) Corlett et al. (2004).
Comparative Evidence Across Species and Tasks
Animal conditioning allows tight experimental manipulation of reinforcement schedules and sometimes cellular recordings or causal neural interventions. Human causal-judgment experiments allow direct questions about perceived relations and the influence of symbolic instructions. Both traditions offer evidence for competition among predictors, but their measured endpoints are different. Freezing, approach, key pressing, and explicit ratings must be mapped to constructs using task-appropriate observation functions. A shared qualitative blocking pattern can support a computational family across preparations while still permitting different representational and neural implementations Rescorla (1968) Waelti et al. (2001) Kelly & Ludvig (2025).
The comparative standard is a common set of experimentally manipulated quantities with clear outcome mapping. Researchers can vary baseline outcome probability, prior signal validity, compound reinforcement, reversal frequency, temporal spacing, and cue similarity in analogous preparations. Where a finding appears in one species but not another, differences in sensory capacity, prior experience, motivation, and training intensity should be examined before inferring a species-specific learning law. Across studies, it is especially important to distinguish an absent effect from an effect that could not be detected because the measurement had low sensitivity or the task lacked a valid acquisition control Miller et al. (1995) Niv & Schoenbaum (2008).
Computational algorithms provide a third comparison class. A value learner, a dynamic-attention learner, and a latent-state learner can be exposed to identical outcome sequences to generate quantitative predictions, with code and fitted parameters inspected directly. This comparison is useful for clarifying which environmental statistics drive each algorithm. Behavioral similarity between an artificial model and a biological participant is evidence about an input-output relation under that schedule; identifying a shared mechanism requires further causal evidence. The emerging cross-system research program is strongest where algorithmic simulations create discriminating experiments rather than stand in for them Sutton (1988) Gershman (2015).
Evidence Quality, Reproducibility, and Alternative Explanations
The literature on associative prediction includes foundational controlled experiments, formal modeling papers, targeted neural recordings, and later reviews that assess the explanatory range of models. These sources answer different questions. A controlled manipulation can establish that a training history changes later behavior; a formal model can demonstrate that a rule generates the pattern; a neural experiment can reveal activity correlated with the hypothesized quantity; and a synthesis can compare convergent and conflicting findings. One category of evidence should not be silently substituted for another. The long-running evaluations of the Rescorla–Wagner model are exemplary because they address both major experimental successes and patterns that require supplementary mechanisms Miller et al. (1995) Rescorla & Wagner (1972).
Several recurrent threats to validity deserve systematic attention. Unbalanced stimulus salience can simulate differences in learning rate; ceiling and floor effects can conceal cue competition; repeated testing can alter the associations being measured; task instructions can lead to explicit inference; and analyst-selected model regressors can correlate strongly with alternative latent variables. Neural studies face additional limits from temporal resolution, selection of recorded populations, and the interpretation of mixed signals. These issues do not nullify the established effects, but they specify what a credible mechanism claim must control Corlett et al. (2004) Roesch et al. (2012) Li et al. (2011).
The strongest research design incorporates independent replication, transparent schedules, uncertainty estimates, model-recovery simulations, and preregistered contrasts when confirmatory inference is intended. Contradictory observations should be retained in the scientific synthesis rather than absorbed through ad hoc parameter changes. Meta-analysis is appropriate only where comparable measures and adequate reporting permit a defensible statistical synthesis. This monograph is a critical narrative review of source-identified experiments and theories, not a systematic review or an original dataset. Its conclusions reflect the reliability and limits of the included evidence and generate hypotheses for further testing Gershman (2015) Behrens et al. (2007).
The Place of Prediction Error Within Learning, Memory, and Motivation
Associative change is embedded in a broader psychological economy. Predictions acquired in one phase can be stored, retrieved, generalized, or suppressed in later contexts, and their impact on action depends on current motivational states. A conditioned cue that signals a reward may elicit anticipation, modify attention, or influence instrumental behavior; each outcome requires an explanatory path from learned relation to response. Outcome devaluation is especially informative because it can separate current incentive value from historically learned response tendencies. Research on parallel devaluation-sensitive and -insensitive Pavlovian processes in humans indicates that the same learning history can support distinguishable influences on behavior Pool et al. (2023).
Memory operations shape what counts as the effective experience on which error correction operates. A learner can update an active cue, retrieve another cue linked through a prior compound, or reassign the trial to a different latent cause. The resulting prediction depends on both current input and accessible prior knowledge. This structure connects retrospective revaluation and context renewal to general questions about retrieval and representation, without collapsing the science of memory into trialwise prediction error. Researchers should identify whether behavior at test reflects newly acquired associations, changed access to previously encoded relations, or altered response valuation Dickinson & Burke (1996) Gershman et al. (2010).
Motivational interpretation also depends on the organism's needs, action opportunities, and outcome identity. A scalar reward variable can efficiently summarize task-specific expected value, but may omit distinctions among wanting, consummatory behavior, choice, and physiological regulation. The research task is to specify which outcome property is represented and show that the proposed error quantitatively predicts acquisition under controlled manipulation. This methodological discipline permits productive links to motivation and affect while protecting the central learning account from claims it has not been designed to explain Niv & Schoenbaum (2008) Schultz et al. (1997).
Artificial Learning Systems as Controlled Computational Comparators
Artificial learning algorithms provide a precisely specified comparison for psychological theories of prediction. A programmed Rescorla–Wagner learner supplies known associative weights and an explicit update rule; a temporal-difference learner makes state transitions and value estimates inspectable; an inferred-state learner expresses a posterior over hypotheses about the environment. Presenting the same trial schedule to these systems permits controlled ablation of representation, associability, error sign, or temporal structure. Such simulations are valuable because they show whether an explanation truly follows from the proposed operations and reveal patterns that can be sought in human or animal experiments Rescorla & Wagner (1972) Sutton (1988) Gershman (2015).
Within the Artificial Era, psychological comparison across Homo and Artificial begins from operationally defined tasks. A computational system's apparent blocking can arise when its training algorithm shares an error-correction structure, when it explicitly reasons over supplied descriptions, or when an interface changes which predictors are accessible. These implementations can produce similar behavioral effects while differing in their update machinery. Scientific attribution therefore proceeds from algorithmic inspection, intervention on artificial internal states, and transfer to novel schedules. Human-style expectancy questions directed at an artificial system do not automatically reveal a human-style internal representation. Conversely, computational transparency can make a model particularly effective for testing theoretical sufficiency Kelly & Ludvig (2025) Daw et al. (2011).
The most constructive relationship between artificial and biological research is reciprocal constraint. Psychological experiments specify phenomena that computational theories must explain; artificial models generate exact counterfactual predictions about future trials and isolate mechanisms that can then be probed experimentally. The comparison should disclose differences in inputs, training objectives, memory, state representation, and feedback conditions. This preserves the distinction between a functional analogy and a mechanistic identity while extending the empirical scope of associative-learning research to systems whose learning procedures can be directly audited Gershman (2015) Hamou et al. (2025).
A Discriminating Research Program for Associative Prediction
A decisive experimental program should test three families of explanations against common data: fixed-associability shared-error models, adaptive-attention models, and latent-cause or temporal-inference models. The first design crosses prior predictive validity with subsequent surprise, then measures transfer learning about previously reliable and unreliable cues. The second introduces a fully specified change in background contingency while holding cue–outcome pairings constant where possible. The third varies the ratio between intertrial and cue–outcome intervals and tests both the rate and timing of anticipatory responding. By construction, the models disagree about what changes at each manipulation: value, associability, inferred context, or temporal representation Mackintosh (1975) Pearce & Hall (1980) Hamou et al. (2025).
Experiments should include preplanned calibration of outcome salience, counterbalancing of stimulus identities, randomized learning schedules where theoretically compatible, and independent tests of value and associability. A preregistered model set should be implemented before inspection of confirmatory data. Simulation-based parameter recovery can establish whether the number and spacing of trials permit discrimination; hierarchical fits can quantify between-participant heterogeneity without forcing every person into one learning strategy. Held-out blocks should contain qualitatively diagnostic transitions, including untrained compounds, unexpected omissions, and contextual reinstatements. The central evaluative quantity is the probability assigned to observations the model did not use for fitting Behrens et al. (2007) Gershman et al. (2010) Daw et al. (2011).
Neural recording or perturbation, where ethically and technically appropriate, adds a complementary test. Separate regressors should index signed outcome error, absolute surprise, inferred volatility, and state transition prediction, with controls for movement, stimulus onset, and reward delivery. To interpret a putative neural learning signal causally, interventions should have a predicted effect on later cue acquisition rather than only on contemporaneous activity. When experiments cannot isolate these regressors, the correct outcome is a constrained explanatory comparison rather than a unique assignment of an algorithm to tissue Schultz et al. (1997) Li et al. (2011) Gläscher et al. (2010).
Critical Scientific Synthesis and Unresolved Problems
The strongest common result across the evidence is that associative acquisition depends on prior prediction and competition among potential predictors. The Rescorla–Wagner family expresses this principle through a compact quantitative rule and accounts for central effects including blocking, conditioned inhibition, and overexpectation under defined assumptions. Selective attention models show why the capacity of a cue to acquire new relations may itself depend on predictive history. Temporal-difference and causal-interval models address when outcomes are expected, while probabilistic latent-cause approaches address which situation an observation is assigned to. The models form a structured scientific landscape of increasingly explicit commitments about representation, learning, and response Rescorla & Wagner (1972) Mackintosh (1975) Pearce & Hall (1980) Hamou et al. (2025).
Major unresolved problems concern how predictive features are represented before learning begins, how error is allocated among compounds and hidden causes, how multiple outcome dimensions influence the same behavior, and how the learner distinguishes stable uncertainty from environmental change. The relation between signed error, unsigned surprise, and measured attention remains especially sensitive to design choices. Neural evidence supports the physiological relevance of prediction-related variables in selected circuits, but a mapping from one fitted computational quantity to a unique and general biological mechanism has not been established by correlation alone. Open questions also concern generalization beyond short controlled tasks, the contribution of instructions and explicit inference, and the durability of learning across contexts Roesch et al. (2012) Gershman (2015) Kelly & Ludvig (2025).
Scientific progress requires experiments that make different accounts fail for different reasons. This means comparing fully implemented models with matched observation functions, recovering parameters under known generative conditions, predefining discriminating behavioral contrasts, and including independent transfer and replication tests. The core objective is explanatory compression: a model should derive multiple observations from a constrained set of assumptions while specifying the conditions under which it predicts a divergent result. Predictive adequacy, biological evidence, and mechanistic identification remain separate benchmarks that together define a mature science of associative learning Miller et al. (1995) Daw et al. (2011) Hamou et al. (2025).
The 2026 literature sharpens three independent unresolved questions. Jahani and Üngör demonstrate why cue-specific certainty and prediction error need factorial separation in compound human learning; Chow and colleagues show how distributed coding of continuously valued outcomes can improve prediction beyond a single expected-value delta model; and Liu and colleagues report psychophysiological outcome sensitivity that does not consistently instantiate signed or unsigned prediction error as specified by a simple formal model. Together these findings support a methodological requirement: models must specify the encoded outcome distribution, the source of error assigned to each cue, and the mapping from latent variables to measured response channels. They also indicate which apparent contradictions may reflect differing task representations rather than mutually exclusive theories Jahani & Üngör (2026) Chow et al. (2026) Liu et al. (2026).
Conclusion
Associative Learning and Prediction-Error Models traces the development of a psychological research program that understands experience-dependent change through the relation between prior expectations and subsequent events. Contingency experiments made baseline outcome probability scientifically indispensable; cue-competition experiments revealed that one signal's acquired prediction can constrain learning about another; and formal error-correction models supplied precise, falsifiable rules for changing associative strength. Later attentional, temporal, representational, and probabilistic theories extended the framework toward questions that scalar acquisition curves leave unresolved. Across these traditions, what is predicted, which cues contribute to that prediction, and what changes after an unexpected event are the central explanatory variables Rescorla (1968) Rescorla & Wagner (1972) Pearce & Hall (1980) Hamou et al. (2025).
The evidence favors a plural, discriminating modeling practice. Rescorla–Wagner rules account economically for important experimental patterns; learned-attention models identify changes in subsequent associability; temporal-difference theories clarify predictive updating across states and time; and latent-cause frameworks formalize contextual inference and recovery. Human and animal results converge on several computationally useful principles while preserving differences in task, response, representation, and potential implementation. Neuronal signals related to prediction error offer a powerful line of evidence, but the identification of precise learning mechanisms requires experiments that connect their manipulation to future acquisition rather than relying solely on correlated activity Schultz et al. (1997) Waelti et al. (2001) Gershman et al. (2010).
The enduring scientific task is to determine which combination of predictive structure, attention, timing, memory, and inference is necessary to explain learning under specific conditions. Future research can answer this question through controlled manipulations of prior validity, surprise, uncertainty, temporal structure, and contextual identity, combined with preregistered model comparisons and independent replication. In the larger context of psychological science, associative prediction becomes a rigorously testable account of how past experience guides future anticipation across diverse learners. The same operational framework supports comparisons with artificial systems when their algorithms, environments, and evaluation conditions are explicitly specified Gershman (2015) Kelly & Ludvig (2025) Hamou et al. (2025).
References
- Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47(1), 129–141. https://doi.org/10.1016/j.neuron.2005.05.020
- Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214–1221. https://doi.org/10.1038/nn1954
- Chow, J. Y. L., Don, H. J., Colagiuri, B., & Livesey, E. J. (2026). Predicting continuous outcomes: Some new tests of associative approaches to contingency learning. PLOS Computational Biology, 22(6), Article e1013220. https://doi.org/10.1371/journal.pcbi.1013220
- Corlett, P. R., Aitken, M. R. F., Dickinson, A., Shanks, D. R., Honey, G. D., Honey, R. A. E., Robbins, T. W., Bullmore, E. T., & Fletcher, P. C. (2004). Prediction error during retrospective revaluation of causal associations in humans: fMRI evidence in favor of an associative model of learning. Neuron, 44(5), 877–888. https://doi.org/10.1016/j.neuron.2004.11.022
- Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron, 69(6), 1204–1215. https://doi.org/10.1016/j.neuron.2011.02.027
- Dickinson, A., & Burke, J. (1996). Within-compound associations mediate the retrospective revaluation of causality judgements. The Quarterly Journal of Experimental Psychology: Section B, 49(1), 60–80. https://doi.org/10.1080/713932614
- Gershman, S. J. (2015). A unifying probabilistic view of associative learning. PLOS Computational Biology, 11(11), Article e1004567. https://doi.org/10.1371/journal.pcbi.1004567
- Gershman, S. J., Blei, D. M., & Niv, Y. (2010). Context, learning, and extinction. Psychological Review, 117(1), 197–209. https://doi.org/10.1037/a0017808
- Ghirlanda, S. (2005). Retrospective revaluation as simple associative learning. Journal of Experimental Psychology: Animal Behavior Processes, 31(1), 107–111. https://doi.org/10.1037/0097-7403.31.1.107
- Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585–595. https://doi.org/10.1016/j.neuron.2010.04.016
- Hamou, N., Gershman, S. J., & Reddy, G. (2025). Reconciling time and prediction error theories of associative learning. Nature Communications, 16, Article 10265. https://doi.org/10.1038/s41467-025-65137-9
- Jahani, A., & Üngör, M. (2026). Associative changes in human learning when cues with different learning histories are trained in compound. Journal of Experimental Psychology: Animal Learning and Cognition. Advance online publication. https://doi.org/10.1037/xan0000446
- Kelly, T., & Ludvig, E. A. (2025). Blocking of associative learning by explicit descriptions. Cognition, 256, Article 106015. https://doi.org/10.1016/j.cognition.2024.106015
- Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., & Daw, N. D. (2011). Differential roles of human striatum and amygdala in associative learning. Nature Neuroscience, 14, 1250–1252. https://doi.org/10.1038/nn.2904
- Liu, H., Linnell, J., & Bach, D. R. (2026). Psychophysiological outcome responses in human Pavlovian fear conditioning: A prediction error analysis. Psychophysiology, 63(4), Article e70300. https://doi.org/10.1111/psyp.70300
- Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82(4), 276–298. https://doi.org/10.1037/h0076778
- Miller, R. R., Barnet, R. C., & Grahame, N. J. (1995). Assessment of the Rescorla-Wagner model. Psychological Bulletin, 117(3), 363–386. https://doi.org/10.1037/0033-2909.117.3.363
- Niv, Y., & Schoenbaum, G. (2008). Dialogues on prediction errors. Trends in Cognitive Sciences, 12(7), 265–272. https://doi.org/10.1016/j.tics.2008.03.006
- Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87(6), 532–552. https://doi.org/10.1037/0033-295X.87.6.532
- Pool, E. R., Pauli, W. M., Cross, L., & O'Doherty, J. P. (2023). Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans. Nature Communications, 14, Article 8057. https://doi.org/10.1038/s41467-023-43747-5
- Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology, 66(1), 1–5. https://doi.org/10.1037/h0025984
- Rescorla, R. A. (1971). Variation in the effectiveness of reinforcement and nonreinforcement following prior inhibitory conditioning. Learning and Motivation, 2(2), 113–123. https://doi.org/10.1016/0023-9690(71)90002-6
- Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). Appleton-Century-Crofts. https://www.columbia.edu/~rk566/Session4/Theory%20of%20Pavlovian%20Conditioning.pdf
- Roesch, M. R., Esber, G. R., Li, J., Daw, N. D., & Schoenbaum, G. (2012). Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. European Journal of Neuroscience, 35(7), 1190–1200. https://doi.org/10.1111/j.1460-9568.2011.07986.x
- Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. https://doi.org/10.1126/science.275.5306.1593
- Seraganian, P., & Vom Saal, W. (1969). Blocking the development of stimulus control when stimuli indicate periods of nonreinforcement. Journal of the Experimental Analysis of Behavior, 12(5), 767–772. https://doi.org/10.1901/jeab.1969.12-767
- Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1), 9–44. https://doi.org/10.1007/BF00115009
- Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412, 43–48. https://doi.org/10.1038/35083500
- Yu, A. J., & Dayan, P. (2005). Uncertainty, neuromodulation, and attention. Neuron, 46(4), 681–692. https://doi.org/10.1016/j.neuron.2005.04.026