Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

Neural Mechanisms of Reward Prediction and Learning

Abstract. Neural mechanisms of reward prediction and learning transform experience with beneficial outcomes into predictions and adaptive future behavior. This fundamental research monograph critically synthesizes electrical self-stimulation, primate dopamine recordings, quantitative reward prediction errors, temporal-difference interpretations, corticostriatal synaptic plasticity, causal manipulation of dopamine pathways, distributional and feature-specific coding, hidden-state inference, and interactions between learning, motivation, and movement. Evidence is evaluated across electrophysiology, neurochemical measurements, imaging, behavioral conditioning, and circuit-specific stimulation. The synthesis distinguishes correlational coding from causal teaching, somatic firing from terminal release, scalar reward value from outcome identity, and retained learning from immediate performance. It reports a critical narrative synthesis of published research, no original experiments, and no newly validated independent theory.

Keywords: reward prediction error; reinforcement learning; dopamine neurons; ventral tegmental area; substantia nigra; nucleus accumbens; corticostriatal plasticity; temporal-difference learning; reward uncertainty; associative learning; optogenetic causality; distributional reinforcement learning; feature-specific prediction errors; hidden-state inference; reward identity; behavioral motivation; neural recordings; neurochemistry; prediction and learning; Artificial Era

Introduction: The Neural Problem of Learning from Reward

Reward prediction and learning concern the transformation of experience into expectations that guide subsequent perception, choice, and action. The neural problem is to explain how an outcome delivered now changes the probability of a later response, how cues acquire predictive force, and how circuits assign credit when events are separated in time. Experiments have established coordinated activity in dopaminergic midbrain neurons, striatal targets, frontal cortical systems, and distributed valuation networks, yet the computations inferred from these measurements depend on task structure and recording modality. This monograph examines how those neural systems represent discrepancies between expected and received outcomes, how their activity can alter synapses and behavior, and where established results leave competing mechanistic interpretations Schultz et al. (1997) Montague et al. (1996) Reynolds et al. (2001).

The central explanatory framework is reward prediction error: an outcome may be better than expected, worse than expected, or in agreement with expectation. A scalar signed discrepancy can support the updating of learned reward estimates, but experimental organisms also learn stimulus identities, temporal relations, hidden causes, and action contingencies. A theory that identifies the firing of all dopamine neurons with one universal value error would therefore make stronger claims than the evidence warrants. Population heterogeneity, terminal release dynamics, and causal manipulations motivate a circuit-specific account in which teaching, motivation, and action preparation are distinguishable empirical roles and can sometimes converge during the same decision Dabney et al. (2020) Mohebi et al. (2019) Sharpe et al. (2017).

The scientific contribution of this monograph is a critical synthesis across levels of analysis. We begin with early reinforcement experiments and extracellular recordings, examine equations as hypotheses about neural coding, follow candidate signals through synaptic plasticity and corticostriatal selection, and assess causal tests using stimulation and projection targeting. Human imaging, developmental questions, computational model comparison, and artificial learning are included where they sharpen rather than replace mechanistic inference. Throughout, a response to reward, a model fit to trialwise behavior, a measured neural correlate, and a causally demonstrated learning mechanism are treated as evidentially different observations Steinberg et al. (2013) Parker et al. (2016) Starkweather et al. (2017).

Defining Reward, Prediction, Error, and Learning

A reward is an outcome that can support reinforcement or organized approach under specified biological and experimental conditions; different tasks operationalize reward through food, fluid, money, sensory feedback, or other consequential events. Reward value is a learned or inferred quantity that depends on motivational state, availability, delay, and alternatives. Reward prediction denotes an expectation conditioned on available cues, action history, and latent environmental structure. Learning denotes a measurable change in future predictive or instrumental behavior attributable to experience, rather than momentary vigor or consumption alone. The definition demands explicit stimulus sequences, schedules, choices, and controls, because an animal can work harder for a familiar reward even when its learned estimate remains unchanged Hamid et al. (2016) Mohebi et al. (2019) Tobler et al. (2005).

A commonly used formal expression is delta equals actual reward minus expected reward, or, in temporal-difference learning, delta at time t equals reward at time t plus discounted next-state value minus current-state value. This equation names a proposed computational variable; whether neural activity implements it requires independent tests of reward magnitude, predicted probability, timing, and model state. Positive errors can occur when unexpectedly valuable outcomes appear, negative errors when anticipated outcomes fail to occur, and near-zero errors when observations match an established prediction. Interpreting neuronal inhibition as a negative error must account for low baseline firing and the limited dynamic range of decreases Schultz et al. (1997) Bayer & Glimcher (2005) Hollerman & Schultz (1998).

Reward-related signals also vary according to what is being predicted. In a sensory-identity reversal, an outcome may change its identity without a comparable change in scalar pleasantness or expected magnitude. That manipulation separates a sensory-specific surprise from a common-currency value surprise. Similarly, an aversive event can be behaviorally salient without serving as an appetitive reinforcer. The terms prediction error, surprise, motivational salience, subjective pleasure, and action value therefore receive distinct operational definitions in this study. Their neural overlap is an empirical proposition to test using factorial tasks and independent measures rather than a basis for terminological equivalence Howard & Kahnt (2018) Kahnt et al. (2014) Cohen et al. (2012).

From Electrical Self-Stimulation to Phasic Reward Signals

Early experimental work established that direct stimulation of particular brain sites could reinforce behavior. Olds and Milner observed that rats learned to perform responses that produced electrical stimulation in septal and related regions. Their 1954 study helped create the experimental field of intracranial self-stimulation and strengthened the hypothesis that reward-related neural systems can be studied through reinforcement contingencies. Electrical stimulation, however, activates heterogeneous fibers and local circuits, and successful self-stimulation does not identify a unique neurotransmitter or psychological feeling. Its enduring role is to show that the brain contains manipulable substrates sufficient to shape instrumental behavior under the tested conditions Olds & Milner (1954).

Single-neuron recordings subsequently transformed the question from where stimulation is reinforcing to what information naturally occurring neural responses carry. Mirenowicz and Schultz reported that dopamine-neuron responses to appetitive events depend strongly on their unpredictability. Hollerman and Schultz further showed that the timing of an expected reward affects the signal, with unexpected delivery and omission creating different activity patterns. These experiments made quantitative timing and expectation indispensable components of reward neuroscience. Their findings are most secure for the identified neuronal samples and conditioning protocols; the anatomy and physiology of other populations may require additional mapping Mirenowicz & Schultz (1994) Hollerman & Schultz (1998).

Montague, Dayan, and Sejnowski proposed a predictive Hebbian account of mesencephalic dopamine systems in 1996, and Schultz, Dayan, and Montague articulated in 1997 how observed neuronal responses resembled a teaching signal familiar from temporal-difference algorithms. The conjunction of computational prediction and physiological recording gave reward learning a testable cross-level framework. It was not a demonstration that all relevant synapses implement a particular algorithm in the same way. Subsequent research has progressively asked whether prediction-error-like activity can cause learning, what downstream cells receive the signal, how dopamine release varies by target, and whether the population supports a distribution rather than a single expected value Montague et al. (1996) Schultz et al. (1997) Dabney et al. (2020).

Prediction, Contingency, and Temporal Structure

Conditioning experiments separate the predictive relation between a cue and its consequence from the mere frequency with which the two occur. When a reliable cue precedes reward, activity that initially follows the unexpected reward can come to follow the cue itself. When an established prediction is violated, a brief suppression may occur at the anticipated outcome time. Such response redistribution provides a biological signature of learning about the temporal structure of events. Yet a cue response can also reflect orienting, anticipation, or motor preparation, so task designs must vary reward probability and timing independently of movement and sensory novelty Schultz et al. (1997) Hollerman & Schultz (1998) Mirenowicz & Schultz (1994).

A second problem is temporal credit assignment. An organism must connect outcomes with earlier cues and actions, often over intervening events. Temporal-difference models distribute error over successive state transitions, while synaptic theories propose that recently active inputs carry eligibility traces that can be modified when a later neuromodulatory signal arrives. A corresponding empirical claim is that dopamine-dependent plasticity should occupy a restricted temporal window relative to activity at the synapse. Cellular studies of structural plasticity and corticostriatal strengthening support temporally organized modulation, while the scale and specificity of such windows depend on the preparation and measured endpoint Montague et al. (1996) Yagishita et al. (2014) Reynolds et al. (2001).

Real tasks introduce uncertainty about time itself. A missing reward might signal that its delivery has been delayed, a hidden context has changed, or an outcome is genuinely absent. Starkweather and colleagues investigated dopamine signals in tasks requiring inference about unobserved environmental states and found patterns compatible with belief-state-sensitive prediction errors. This extends the framework beyond a fixed clock and a fully observed state, while also introducing model-selection challenges: several latent-state representations may fit behavior and neural activity. Experiments must therefore manipulate temporal hazard, uncertainty, and environmental transitions to distinguish an implementation of expected value from an implementation of a particular inference model Starkweather et al. (2017) Fiorillo et al. (2003).

A Critical Narrative Method for Linking Brain and Behavior

This review organizes evidence according to the inferential claim made by each method. Electrophysiology measures spikes or local electrical activity in specified cells; voltammetry and genetically encoded sensors estimate chemical fluctuations in particular targets; fMRI samples hemodynamic responses at relatively coarse temporal and anatomical resolution; optogenetic and electrical interventions alter neural activity with distinct selectivity profiles. Behavioral probes establish whether subjects acquired a cue association, changed their choice policy, or modified their effort. Convergence matters when recording and perturbation target a well-characterized circuit and the behavioral endpoint directly tests the proposed learning operation Hart et al. (2014) Mohebi et al. (2019) Steinberg et al. (2013).

The corpus was selected through a critical narrative synthesis of historically foundational primary experiments, targeted mechanistic studies, cross-level computational analyses, and methodological comparisons. It is neither a protocol-registered systematic review nor a statistical meta-analysis: there is no claim of exhaustive retrieval, formal risk-of-bias aggregation, or pooled effect estimation. Accordingly, the argument weighs experimental design, anatomical and cell-type identification, model assumptions, alternative behavioral explanations, replication across preparations, and whether outcomes are actually manipulated. A report limited to an abstract is treated as evidence only for claims conveyed in that abstract, and numerical claims requiring original tables are omitted when their precise estimates have not been checked Schultz et al. (1997) Shen et al. (2008) Dabney et al. (2020).

The principal boundaries are established in the project’s scientific architecture. Formal algorithms for associative and reinforcement learning, considered independently of neural implementation, belong to a neighboring Research Dossier; motivational pleasure and incentive salience have their own scientific object. This monograph studies neural reward prediction, local and long-range mechanisms of learning, and evidence capable of identifying causal contribution. Its scope includes model equations only when they lead to falsifiable physiological hypotheses and includes Artificial Era comparisons only where artificial learning provides an experimental or conceptual contrast. Mechanistic mapping is evaluated at the circuit and task level rather than asserted on the basis of similar vocabulary Montague et al. (1996) Parker et al. (2016) Sharpe et al. (2017).

Midbrain Dopamine Cells and Their Projection Fields

Midbrain dopaminergic neurons are found in anatomically distinct and physiologically diverse groups including the substantia nigra pars compacta and ventral tegmental area. Their axons innervate striatal, cortical, and other targets with unequal densities and connectivity, making projection-defined measurement essential to functional interpretation. The ventral striatum is prominent in reward-learning studies, whereas dorsal striatal subregions support different mixtures of action selection, movement, and reinforcement. A claim about dopamine in one terminal field should not automatically be transferred to all others. Tracing, genetically targeted recordings, and terminal measurements are valuable precisely because they reveal variation hidden by a single global dopamine label Parker et al. (2016) Menegas et al. (2017) Mohebi et al. (2019).

Neural identity is a methodological issue as well as an anatomical classification. Extracellular recordings historically identified putative dopamine cells by waveform, firing pattern, and recording location; modern genetic tagging and optotagging improve assignment but also require controls for expression specificity and sampling. Cohen and colleagues used cell-type-targeted methods to distinguish reward- and punishment-related signals among different neuronal populations within the ventral tegmental area. This result illustrates why a mixture of glutamatergic, GABAergic, and dopaminergic components cannot be treated as one readout of reward value. Projection identity, transmitter phenotype, baseline dynamics, and behavioral event timing jointly define the neuronal population under study Cohen et al. (2012) Parker et al. (2016).

Interpretation also requires distinguishing somatic activity from chemical transmission at a target. Spikes in midbrain neurons can contribute to terminal dopamine release, but local regulation, receptors, uptake, and circuit state shape the relationship. Two experiments may therefore appear inconsistent because one records cell bodies on a millisecond timescale and another monitors extracellular dopamine during sustained work. Their differences provide mechanistic information rather than mere measurement noise. Integrative accounts must ask how terminal fields encode particular task variables, when those variables become available, and how target-specific signals influence downstream plasticity or response vigor Mohebi et al. (2019) Hamid et al. (2016) Parker et al. (2016).

Phasic Prediction Errors in Primate Recording Experiments

The classic primate electrophysiology experiments are foundational because they manipulated the difference between expected and obtained reward. Early in training, an unexpected appetitive event could evoke a burst of dopamine-cell firing; after learning, the reward-predictive cue could evoke a response while the now-expected reward elicited little additional activity. Omission at the predicted delivery time was associated with suppression relative to ongoing firing. These three observations resemble a signed error computed against an internal expectation. They offer a principled explanation for how a temporally distant cue becomes predictive through reinforcement experience Schultz et al. (1997) Hollerman & Schultz (1998) Mirenowicz & Schultz (1994).

The formal analogy remains a hypothesis requiring controls. A cue response may coincide with a change in orientation, sensory uncertainty, or action preparation, and a reward response may vary with physical magnitude or internal motivational state. In a well-designed prediction-error experiment, the probability and timing of reward are manipulated independently of visual or auditory stimulus novelty, response requirements, and motor execution. Within the recorded sample, a monotonic dependence of the response on unexpectedly delivered reward supports quantitative coding, while failure to respond when reward is fully predicted supports an expectancy component. The result does not by itself identify the synaptic learning rule or establish necessity for updating Bayer & Glimcher (2005) Tobler et al. (2005) Steinberg et al. (2013).

Historical attribution also matters. The 1997 framework integrated previous recording findings with computational reinforcement learning, building on a 1996 predictive Hebbian proposal; it did not originate all concepts of reinforcement, dopaminergic reward responses, or temporal-difference algorithms. The scientific achievement was to connect a reproducible physiological signature to a formal prediction problem. Later experiments strengthened this connection with target release, causal stimulation, and evidence that the same neural system can represent richer statistics. A historical narrative should preserve both the power of the original explanatory proposal and the way its scope has changed under subsequent tests Montague et al. (1996) Schultz et al. (1997) Dabney et al. (2020).

Quantitative Dopamine Responses, Probability, and Context

Reward prediction error is most informative when a neural response varies systematically with the numerical difference between expectation and outcome. Bayer and Glimcher recorded midbrain dopamine neurons under reward conditions that varied expectation and reported a quantitative relationship between neuronal responses and reward prediction errors. Tobler, Fiorillo, and Schultz showed adaptive coding of reward value, linking neuronal responses to the distribution of possible rewards in a context. These studies advance beyond a binary unexpected-versus-expected contrast, because reward scale, local reference frame, and probability can make different predictions for neural firing. The precise mapping remains conditional on the trained environment Bayer & Glimcher (2005) Tobler et al. (2005).

Fiorillo, Tobler, and Schultz also examined reward probability and uncertainty and distinguished activity patterns associated with different aspects of an uncertain outcome. A slowly evolving component during anticipation may track uncertainty while phasic responses signal particular outcomes. Such results suggest that reward circuits can maintain multiple variables on different timescales. They do not establish that a single neuron computes a fully calibrated probability distribution. A rigorous analysis must compare averaged firing across probabilities, temporal dynamics within trials, and trial-by-trial variation, while acknowledging that uncertainty can influence attention, anticipation, and behavior simultaneously Fiorillo et al. (2003) Dabney et al. (2020).

Context-dependent scaling raises a key difficulty for identifying a universal biological currency. A neural response to the same physical volume of fluid may differ when alternative rewards are larger, smaller, more variable, or more costly. Models of normalized or adaptive value prediction can explain such changes, but equal fits to a firing-rate curve do not prove equal circuit mechanisms. Experiments must test model predictions outside the fitted range and distinguish reward identity, magnitude, risk, and motivational condition. Evidence of adaptation is thus compatible with a broad family of neural encodings that are discriminated by carefully controlled generalization tests Tobler et al. (2005) Eshel et al. (2016) Kahnt et al. (2014).

Firing Rates, Striatal Release, and Measurement Timescales

Dopaminergic cell-body spiking and extracellular dopamine concentrations are related biological measurements rather than interchangeable readouts. Phasic action potentials can help determine transient release, but extracellular signals are also shaped by release probability, uptake, spatial diffusion, local neuromodulators, and projection-specific activity. Hart and colleagues showed that phasic dopamine release in rat nucleus accumbens can encode a signed prediction-error-like term. This strengthens the mechanistic bridge between a classic midbrain recording phenomenon and transmitter availability within a recipient circuit. The observation remains target-, task-, and measurement-specific, which is exactly why it is informative Hart et al. (2014) Parker et al. (2016).

Mohebi and colleagues compared somatic dopamine neuron activity and dopamine release dynamics while animals performed motivational tasks, providing evidence for dissociable relationships between prediction-related spiking and motivational fluctuations at terminals. Hamid and colleagues likewise investigated mesolimbic dopamine in relation to the value of work. These findings complicate a simple equation between all dopamine concentration changes and one teaching-error variable. Sustained dopamine variation can covary with effort, reward rate, or motivation even when a particular phasic prediction contrast is held constant. The task is to identify which variable is represented at each timescale and whether it predicts acquisition, performance, or both Mohebi et al. (2019) Hamid et al. (2016).

Measurement technology shapes this inference. Fast-scan cyclic voltammetry, fluorescent dopamine sensors, calcium imaging, and electrophysiology have different temporal filters and molecular specificity. Sensor signals can integrate neuromodulator availability over space and time, whereas spikes retain cellular timing but may sample a restricted number of neurons. Cross-method confirmation is stronger when the same task has independently characterized movement, cue timing, reward delivery, and stimulation artifacts. A change in fluorescence following a rewarded action cannot on its own discriminate prediction error from the movement or sensory event that accompanied it. Explicit observation models are indispensable for mapping recorded traces onto neural computations Hart et al. (2014) Mohebi et al. (2019) Parker et al. (2016).

Negative Errors, Omission, and Valence Asymmetries

Negative reward prediction errors are central to theories in which the absence of an expected reward weakens future expectation. In neuronal recordings, omission can cause a dip below tonic firing around the expected time. Yet the quantitative range of negative firing is restricted by baseline activity, and a silent neuron cannot decrease its spike rate indefinitely. Consequently, equal positive and negative errors in a behavioral model need not have symmetrically sized neural representations. Experimental claims of symmetric error coding are better tested with transmitter-release measurements, carefully defined baselines, and outcome schedules that separate omission from temporal uncertainty Hollerman & Schultz (1998) Bayer & Glimcher (2005) Hart et al. (2014).

Reward omission is also distinct from punishment. An omitted appetitive event can represent a negative change in expected benefit, whereas an aversive stimulus has its own sensory and motivational consequences. Cohen and colleagues characterized cell-type-specific responses to reward and punishment in the ventral tegmental area, showing that different neuronal populations signal distinct valence-related variables. A protocol that groups all unpleasant or surprising events together obscures these differences. The appropriate factorial design compares expected and unexpected reward, omission, and aversive events while tracking anticipatory state and motor behavior Cohen et al. (2012) Kahnt et al. (2014).

Interpretations of error polarity must account for what the organism believes. The absence of a reward at one instant may provide little negative evidence when an uncertain delivery delay remains plausible. A negative response can therefore depend on temporal hazard and inferred hidden state rather than simply on a static reward schedule. This makes uncertainty manipulation a substantive test of neural models: error traces can be evaluated against explicit beliefs about event timing and task switches. The synthesis is that signed errors are a valuable empirical construct, while their implementation is shaped by physiological bounds, learned temporal structure, and circuit context Starkweather et al. (2017) Fiorillo et al. (2003) Eshel et al. (2016).

Corticostriatal Architecture and Action Values

Reinforcement learning requires a route through which information about consequences changes action selection. The striatum receives widespread cortical input and is modulated by midbrain dopamine, placing corticostriatal synapses in a position to integrate contextual representations, recently selected actions, and ensuing outcomes. Anatomical subdivisions differ in connectivity, and experiments in ventral and dorsal striatum emphasize partly different behavioral functions. It is therefore useful to specify whether a neural measure predicts cue attraction, effort, choice among actions, or acquisition of a learned contingency. None of those behaviors alone establishes the particular synaptic process that implements the update Parker et al. (2016) Tai et al. (2012) Hamid et al. (2016).

In an instrumental task, changing the expected consequences of alternative actions can produce a choice bias. Such biases can be explained by updated action values, but choice frequency also depends on exploration, vigor, perseverance, and the response costs associated with each option. Tai and colleagues used transient stimulation of distinct striatal neuron populations to influence action selection in ways resembling changes in action value. The intervention is mechanistically informative because it alters circuit activity rather than relying exclusively on correlational recordings. Nevertheless, a stimulation-induced choice shift may reflect the expression of learned preference or the update that creates it, depending on intervention timing and subsequent tests Tai et al. (2012) Steinberg et al. (2013).

A circuit model therefore needs separate stages for cortical representations of task state, striatal selection and learning, dopaminergic or other modulatory signals, and downstream motor implementation. Cortical areas can contribute estimates of sensory identity and contingency, while striatal circuits can modify how states and actions bias one another. Reward learning is not localized to a single node; the causal effect of a midbrain pulse depends on the inputs, receptor landscape, and activity of the receiving circuit. Testing mechanistic necessity entails temporally and anatomically precise manipulations coupled to later learning probes Sharpe et al. (2017) Shen et al. (2008) Reynolds et al. (2001).

Dopamine-Dependent Synaptic Plasticity

Long-term learning calls for mechanisms that preserve effects after a brief reward signal has vanished. Corticostriatal synapses are candidates because prior activity can mark particular connections for modulation, and dopamine receptor pathways can affect the direction and magnitude of plastic changes. Reynolds, Hyland, and Wickens identified a cellular mechanism linking reward-related signaling to synaptic change in the striatum. Their work established an experimental bridge between reinforcement phenomena and plasticity, while leaving open how many of the required activity patterns occur naturally in intact behavior Reynolds et al. (2001) Shen et al. (2008).

Shen, Flajolet, Greengard, and Surmeier provided influential evidence for distinct dopaminergic control of plasticity in striatal pathways. Their results motivate models in which receptor expression, cell type, and signaling state determine whether a particular synapse potentiates or depresses. The basic lesson concerns conditional plasticity rules: a neurotransmitter pulse is not a self-sufficient instruction to strengthen every recently active connection. Synaptic modification depends on stimulation sequence, the state of the receiving cell, pharmacology, and interactions between cortical input and basal ganglia circuitry. Extrapolation to an entire animal therefore requires evidence that the relevant synapses were engaged during natural learning Shen et al. (2008) Tai et al. (2012).

Interpretive rigor requires distinguishing ex vivo induction protocols, in vivo circuit manipulation, and behavioral learning. A slice preparation can reveal necessary cellular machinery under controlled conditions while lacking the full task state and competing inputs of an awake brain. An optical manipulation can show sufficiency for a behavioral acquisition effect while stimulating many terminals simultaneously. The strongest cross-level argument combines a learning-relevant temporal window, identified projection, a synaptic consequence, and behavior measured after stimulation has ceased. This is an achievable experimental program, not an assumption licensed by the general phrase dopamine-dependent learning Yagishita et al. (2014) Steinberg et al. (2013) Sharpe et al. (2017).

Eligibility Traces and Temporal Credit Assignment

Reward often arrives after the neural events that made it possible. An eligibility trace is a formal or biological state that temporarily marks recently active representations as candidates for reinforcement. Under a three-factor learning rule, coincident pre- and postsynaptic activity establishes eligibility, and a later modulatory signal determines whether selected connections change. This class of models offers a solution to delayed credit assignment without requiring the dopamine signal itself to encode the full sensory history of an episode. The central empirical questions concern how the trace is created, how quickly it decays, and whether its timescale matches observed learning under natural behavior Montague et al. (1996) Reynolds et al. (2001) Yagishita et al. (2014).

Yagishita and colleagues examined dopamine-dependent structural plasticity in dendritic spines and identified a restricted time window in which dopamine could promote plastic change after preceding synaptic activity. Their finding supports a temporally specific interaction between prior activity and neuromodulatory action. It does not follow that the duration measured in one preparation supplies a universal eligibility constant for all learning contexts, cell types, or synapses. Behaviorally relevant intervals can be bridged by recurrent activity, hierarchical state representations, or other mechanisms. Experiments must separately measure the persistence of cellular eligibility and the persistence of a computationally inferred reward expectancy Yagishita et al. (2014) Starkweather et al. (2017).

Temporal precision is especially important in causal intervention studies. A pulse delivered at an unexpected reward can reinforce an antecedent cue, whereas the same pulse at cue onset or during movement may have different consequences. Timing manipulations can distinguish a permissive role in arousal from an instructive role in credit assignment, provided controls match sensory artifacts and stimulation exposure. A complete mechanism must account for specificity: if many synapses were active, how does a global or broadly distributed modulator preferentially alter the appropriate ones? Eligibility rules provide part of that explanation but demand circuit-level validation across learning tasks Steinberg et al. (2013) Sharpe et al. (2017) Reynolds et al. (2001).

Circuit Causality in Associative Blocking and Extinction

Blocking and related contingency procedures offer tests of whether a signal behaves as a teaching error. If one cue already predicts an outcome, pairing a new cue with the same outcome can yield comparatively little learning about the new cue. When an unexpected increment appears, learning about additional predictors can occur. A neural manipulation that supplies a synthetic positive prediction signal at the otherwise fully predicted outcome can test whether teaching is restored. Causal inference is strongest when later behavior is measured in the absence of intervention and control groups receive matched sensory and contextual experience Steinberg et al. (2013) Sharpe et al. (2017).

Steinberg and colleagues used temporally targeted activation of dopamine neurons to test whether a phasic signal can support learning in circumstances where an error is experimentally manipulated. This experiment advanced the field beyond correlation between recorded firing and surprise. The appropriate conclusion concerns the particular animal preparation, behavioral task, intervention parameters, and acquisition measure. Altering cell activity may recruit downstream effects beyond the theoretical prediction-error variable, so controls for movement, immediate preference, and enduring association remain essential. Nevertheless, the study supplied influential causal support for a teaching role of dopamine transients Steinberg et al. (2013).

Sharpe and colleagues tested dopamine transients in acquisition of associations involving more than a scalar cached value and reported sufficiency and necessity within their experimental framework. These results matter because animals learn rich relationships among predictors, outcomes, and task structure. They also caution against assuming that all effects of dopamine can be compressed into one action-value update. Extinction introduces additional complexity: a reduced response may reflect inhibitory learning, contingency change, context inference, or motor suppression rather than unlearning of one synapse. Mechanistic studies should pair intervention with outcome-specific probes, reinstatement or renewal tests, and task-state analysis Sharpe et al. (2017) Starkweather et al. (2017) Hollerman & Schultz (1998).

Dopamine Contributions to Pavlovian Incentives and Action

A reward-predictive cue can influence later action through several pathways. It may provide information about the likely outcome, attract attention, become an incentive target, invigorate a response, or change the expected utility of an available action. Pavlovian conditioned approach and instrumental choice therefore cannot be treated as identical readouts of one latent reward value. Saunders and colleagues investigated dopamine-neuron contributions to Pavlovian stimuli with circuit-defined motivational properties. Their intervention-based results highlight a motivational route through which learned cues influence behavior and the importance of specifying the projection involved Saunders et al. (2018) Parker et al. (2016).

Different learning and motivational effects can coincide. When a cue reliably announces a desirable event, neurons involved in prediction may also influence behavioral vigor during anticipation. Hamid and colleagues investigated mesolimbic dopamine as related to the value of work, an important measurement because willing effort is distinct from the acquisition of a new cue–outcome relation. Mohebi and colleagues later provided evidence that dopamine dynamics relevant to learning and motivation can be dissociated. These results should not be forced into an exclusive single-function account. The appropriate model explains how current state, learned expectation, and action costs jointly control transmitter dynamics and behavioral output Hamid et al. (2016) Mohebi et al. (2019).

The distinction between wanting and hedonic impact has clinical and theoretical importance. An animal's pursuit of a reinforcer can intensify even when direct measures of pleasurable reactions do not increase in parallel. Research Dossier PSY-D-912 concentrates on neural signals relevant to prediction and updating; a separate dossier examines reward, pleasure, and incentive salience in their own right. For the present object, rewarding efficacy should be inferred from changes in subsequent learning and action within well-defined tasks. The presence of dopamine activity during cue approach is evidence of neural recruitment, while causal claims about incentive formation require intervention and longitudinal behavioral tests Saunders et al. (2018) Steinberg et al. (2013) Mohebi et al. (2019).

Spatially Heterogeneous Dopamine Signals

Anatomical differences among dopamine projections complicate claims that a single reward error governs all recipient structures. Projection-defined recordings reveal that dopamine activity and release can depend on the striatal target, task, and measured event. Parker and colleagues compared terminals of midbrain dopamine neurons innervating different striatal regions and reported reward- and choice-related encoding that depended on the target. The finding implies that the same behavioral episode can have different learning-relevant features across corticostriatal loops. It also raises a measurement question: are differences caused by the presynaptic population, local terminal regulation, or different behavioral representations within interconnected networks? Parker et al. (2016) Mohebi et al. (2019).

Menegas and colleagues studied dopamine signals in ventral and posterior striatum and found different initial responses to novel cues. That result is especially instructive because novelty and the beginning of association acquisition are circumstances in which reward predictions have not yet stabilized. It shows how the sign and behavioral interpretation of a dopamine signal may depend on target region and learning phase. Rather than assigning a universal valence to novelty-associated activity, experiments should combine recording location, stimulus histories, reward contingencies, and eventual behavioral outcomes. Heterogeneity is scientifically meaningful only when neural location and experimental context are both specified Menegas et al. (2017) Cohen et al. (2012).

A multi-projection model can preserve the explanatory usefulness of prediction errors while recognizing that different circuits learn different features. Some populations may encode differences in expected appetitive benefit, others an outcome's identity or movement-related variable, and still others aspects of uncertainty and state inference. The distinction does not require every single cell to have one pure function. Population coding can be mixed, and individual neurons may combine signals that evolve over learning. Experiments should therefore report the distribution of responses, unit-selection criteria, target identity, within-session stability, and out-of-task predictions before generalizing from an averaged trace Dabney et al. (2020) Lee et al. (2024) Parker et al. (2016).

Distributional and Feature-Specific Representations

Standard reward-learning equations describe an expected value as a single number, but rewards are uncertain and organisms can adjust differently to distributions with identical means. Dabney and colleagues proposed and tested a distributional reinforcement-learning account in which heterogeneous dopamine responses support representation of multiple aspects of future reward distributions. Single-unit recordings in mice offered evidence consistent with such a code. The central advance is the possibility of encoding outcome variability through a population of learning signals rather than requiring each neuron to reproduce the same scalar expectation. Assessing this interpretation requires careful model comparisons and tests across reward distributions not used to fit neuronal response functions Dabney et al. (2020).

Distributional coding also illustrates the productive connection between computational neuroscience and artificial reinforcement learning. A mathematical architecture developed for algorithmic learning yields new physiological predictions about asymmetries in neuronal responses to positive and negative outcomes. Evidence that some neural responses exhibit such patterns lends support to a representation-level hypothesis, but implementing the same formal computation in software and biological tissue are different claims. Biological circuits include local neurotransmitter regulation, nonlinear receptor dynamics, and recurrent interactions that may realize related predictions through different mechanisms. Cross-system similarity becomes scientifically powerful when it motivates discriminating experiments rather than establishing identity by analogy Dabney et al. (2020) Montague et al. (1996).

Feature-specific models broaden the scope further. When reward outcomes differ in sensory identity or task-relevant features, teaching signals may reflect errors in those dimensions even when overall value is approximately conserved. Recent work on heterogeneity has examined whether observed dopamine-neuron differences can be explained by feature-specific prediction-error models. The test is whether such models predict how neural populations respond to untrained feature changes, reversals, and mixed outcomes. Empirical claims about separate features should be evaluated against simpler alternatives such as different motivational salience, novelty, and movement; without this discrimination a richer model can fit signals by flexibility alone Lee et al. (2024) Howard & Kahnt (2018) Kahnt & Schoenbaum (2025).

Belief States, Hidden Causes, and Temporal Inference

Animals rarely observe the complete state of their environments. A cue may be ambiguous, an outcome delayed, or a task rule change without an explicit announcement. When a learner must infer a hidden state, the relevant prediction is conditional on beliefs rather than on the immediately visible stimulus alone. Starkweather and colleagues investigated dopaminergic error signals under temporally structured uncertainty and reported evidence consistent with hidden-state inference. This research reframes the teaching signal as depending on what the animal believes will happen, not simply on a table of fixed rewards associated with fixed cues Starkweather et al. (2017).

Belief-state models produce specific requirements for experiment design. Investigators must measure behavioral evidence of the inferred state, manipulate the probability of transitions, and examine whether neural activity follows predictions derived from the participant's information rather than the experimenter's omniscient description. A plausible model might predict a dopamine response when an outcome is surprising under one inferred context but unsurprising under another. To distinguish a belief-dependent error from nonspecific surprise, the same physical reward and sensory events should be presented under different informational histories, with enough observations to estimate state uncertainty Starkweather et al. (2017) Fiorillo et al. (2003).

Neural prediction and state inference can reside in interacting circuits. Cortical systems may encode expected outcome identity and inferred task structure, while dopaminergic signals participate in updating associations based on those representations. Under this account a teaching signal cannot be understood independently of the model that defines what counts as expected. The causal chain requires evidence identifying how state representations arrive at midbrain and recipient circuits, whether they change neural errors, and how intervention alters later behavior. The evidence establishes that richer representations can shape reward-related activity while leaving substantial questions about precise circuit implementation open Howard & Kahnt (2018) Sharpe et al. (2017) Kahnt & Schoenbaum (2025).

Value, Identity, Salience, and Orbitofrontal Interactions

An outcome has multiple psychologically relevant attributes: amount, probability, sensory identity, delay, and motivational importance. A cue predicting one flavored food may change meaning when a different flavor arrives, even if both outcomes have comparable scalar value. Howard and Kahnt investigated identity prediction errors in the human midbrain and related them to updated expectations in orbitofrontal cortex. This work is informative because it asks how neural circuits learn specific outcomes rather than only whether an outcome is better or worse overall. It also requires distinguishing a human fMRI response from direct evidence about the transmitter released by identified dopamine neurons Howard & Kahnt (2018).

Value and salience are often correlated in experimental tasks: highly valuable rewards are attention grabbing, and surprising outcomes induce both neural and behavioral responses. Kahnt and colleagues used human neuroimaging to address distinctions between neural representations of value and salience. A strong mechanistic explanation must generate predictions for conditions in which these variables dissociate, such as equally salient desirable and undesirable outcomes or identity substitutions without a difference in total utility. Motor demands, arousal, reaction time, and task difficulty also require control because they can alter the same neural measurements Kahnt et al. (2014) Cohen et al. (2012).

Orbitofrontal interactions invite an explicitly multilevel account. Outcome-specific expectations can be represented in cortical networks, compared with actual sensory input, and updated through neuromodulatory or other synaptic mechanisms. Dopamine may influence acquisition of predictive associations whose content is richer than a common scalar value, as causal animal studies suggest. This is a significant extension of classical reward prediction theory, and its scope is being debated in current reviews. Future tests should identify how content-specific errors are computed, whether they involve distinct dopaminergic subpopulations, and how the learning rule relates to receptor-specific plasticity downstream Sharpe et al. (2017) Howard & Kahnt (2018) Kahnt & Schoenbaum (2025).

Learning, Motivation, and Movement as Coupled Processes

Learning and performance are measured on different causal axes. A manipulation can change what an animal expects about future reward, its willingness to expend effort for that reward, or the speed and form of an already learned action. All three can alter choice behavior, yet they imply different functions for the manipulated neural signal. Studies relating dopamine to work value and motivated behavior demonstrate that a reward-relevant neural variable can operate during performance rather than exclusively at unexpected outcomes. Interpreting increased responding as increased learning without subsequent probe trials would therefore overstate the evidence Hamid et al. (2016) Mohebi et al. (2019).

Mohebi and colleagues observed dopamine dynamics that differed across measurements tied to prediction-related and motivational processes. Their work illustrates why recording location, timescale, and behavioral mode need to be integrated. A phasic signal associated with the receipt of unexpected reward can coexist with slower changes during vigorous pursuit. Such findings do not invalidate a teaching role for dopamine transients; they constrain its domain and expose additional neural functions. Formal accounts benefit from representing policy execution and policy updating separately and then explaining how common modulatory pathways may influence both Mohebi et al. (2019) Hart et al. (2014).

Movement is another essential confound and potential mechanism. Striatal dopamine is strongly implicated in action control, and interventions can change movement initiation or choice without identifying an altered belief about reward. Experimental analysis should include trial-by-trial kinematics, action timing, instrumental contingencies, and transfer probes when a reward-learning claim is made. A decisive design manipulates the signal during experience and tests behavior later without the manipulation, separating immediate performance from retained learning. These distinctions are valuable for clinical neuroscience, which must distinguish deficits in reinforcement acquisition from altered vigor, exploration, or motor capacity Tai et al. (2012) Parker et al. (2016) Steinberg et al. (2013).

Experimental Paradigms and Measurement Validity

The most informative reward-learning experiments manipulate a learner's prediction before manipulating outcome delivery. Probabilistic cues, temporal schedules, blocking, reversal learning, omission, reward magnitude, and sensory-identity substitutions test different components of expectation. When data are modeled trial by trial, a model should be fitted to behavior and used to predict neural responses on held-out conditions; otherwise neural-model agreement can be circular. Reward contingencies must be disentangled from cue novelty, movement, outcome salience, satiety, reaction time, and differences in exposure. A failure to control these variables weakens mechanistic attribution even when a strong signal follows reward Fiorillo et al. (2003) Bayer & Glimcher (2005) Starkweather et al. (2017).

Recording methods answer different questions. Single-unit recordings reveal spike timing in identified or putative cells, multiunit recordings sample neuronal ensembles, fMRI tracks hemodynamic changes within spatially defined regions, and voltammetry or fluorescent probes detect neurochemical dynamics with modality-specific temporal and spatial resolution. Ex vivo electrophysiology can characterize synaptic plasticity under precisely controlled stimulation, while in vivo photostimulation tests the causal role of targeted populations during learning. Researchers should document targeting and expression, sensor calibration, anatomical boundaries, experimental exclusions, and the correspondence between intervention timing and the hypothesized learning variable Shen et al. (2008) Mohebi et al. (2019) Steinberg et al. (2013).

The inference hierarchy runs from event-locked response to model-consistent coding, to intervention-dependent learning, to verified circuit implementation. A single strong correlation supports the first levels but cannot substitute for an independent test of necessity or sufficiency. Necessity experiments require control for behaviorally disruptive intervention side effects; sufficiency experiments need controls for unintended stimulation reinforcement and sensory artifacts. Multiple converging modalities increase confidence when their measurement errors and biological dependencies differ. Transparent reporting of null results, target-specific effects, and alternative computational fits is crucial because appealing prediction-error narratives can otherwise be mapped onto many different event-related neural responses Parker et al. (2016) Sharpe et al. (2017) Dabney et al. (2020).

Human Translation, fMRI, and Computational Neuropsychiatry

Human reward-learning experiments often combine monetary or other consequential outcomes with trialwise choices, response times, and hemodynamic imaging. A model-derived prediction error can be entered as a regressor to identify regions whose activity covaries with the estimated variable. This approach relates behavior to neural systems, but the regressor inherits assumptions about subjective expectations, learning rates, and the chosen state representation. Furthermore, a signal in the human midbrain does not establish that recorded voxels uniquely reflect dopaminergic transmission. Stronger interpretation requires anatomical care, behavioral model comparison, and tests involving multiple outcome attributes Howard & Kahnt (2018) Kahnt et al. (2014).

Theoretical translation is most valuable when it specifies dissociable predictions for clinical or neurobehavioral states. An apparent failure to learn from reward could reflect inaccurate value updating, altered perceptual access to the cue, reduced effort, impaired action initiation, or unstable beliefs about context. A computational model can help separate these possibilities when it is identifiable from the task, predicts new observations, and resists misleading parameter tradeoffs. No single fitted learning-rate parameter constitutes a diagnosis or a direct assay of dopamine concentration. Clinical application requires independent validation, representative participant samples, and caution about treatment effects and task reliability Starkweather et al. (2017) Mohebi et al. (2019) Kahnt & Schoenbaum (2025).

Work on subjective evaluation adds another layer. Rutledge and colleagues investigated computational and neural relationships between recent experiences and momentary reported well-being. Such studies show that prediction-related valuation can inform the dynamics of affective judgment in human tasks, yet affect ratings, anticipatory choice, and synaptic teaching are distinct measurements. Translational frameworks must specify which process is being modeled, how its latent variables are inferred, and whether those variables generalize across laboratory rewards and life circumstances. This monograph offers no patient-specific diagnostic or therapeutic recommendations; its clinical significance is methodological and mechanistic Rutledge et al. (2014) Howard & Kahnt (2018).

Development, Experience, and Ecological Generalization

Reward prediction is shaped by learning history, internal state, and the ecology of available outcomes. The same nominal reward may have different values under hunger, fatigue, social competition, or varying action costs, and an animal's prior exposure changes which cues are surprising. Laboratory tasks offer exceptional causal control by holding contingencies stable, but the mechanisms they reveal must be tested in environments with changing objectives and partially observed states. Biological generalization requires attention to the species, age, training intensity, receptor context, and behavioral repertoire of the studied population Hamid et al. (2016) Starkweather et al. (2017) Fiorillo et al. (2003).

Developmental variation is scientifically plausible because corticostriatal circuits, motivational regulation, and learning histories change across the lifespan. Yet this monograph's primary cellular and causal sources overwhelmingly derive from specific adult animal preparations or adult human tasks; they do not establish a universal developmental trajectory for the studied mechanism. Research involving younger and older participants should directly measure task comprehension, reward valuation, behavioral strategy, and neural recording comparability. Age-related changes in acquisition may reflect differences in exploration or state inference rather than an isolated impairment of phasic error encoding. Future work needs within-subject learning probes and comparable operational definitions across ages Tai et al. (2012) Howard & Kahnt (2018).

Ecological reward learning also features long delays, multiple outcomes, sensory-rich predictors, and social feedback. A single short laboratory trial can establish whether a putative teacher signal updates a selected contingency, while long-term behavior requires additional memory, planning, and policy-control systems. Cross-species work offers mechanistic leverage only when the compared tasks share relevant computational structure and when differences in sensory and motor capacities are explicitly modeled. The appropriate standard is predictive generalization: circuit accounts should explain new tasks with overlapping contingencies without requiring unconstrained reinterpretation of every neural response Sharpe et al. (2017) Parker et al. (2016) Menegas et al. (2017).

Artificial Learning Systems as Analytical Comparators

Artificial reinforcement learning provides formal models that clarify what a biological teaching signal would have to calculate. Temporal-difference algorithms explicitly separate expected future return from observed outcomes; distributional algorithms maintain information about variability across possible returns; partially observable models update beliefs about hidden states. The value of these comparisons lies in generating predictions about timing, response asymmetry, uncertainty, and generalization that can be tested in neurophysiology. The foundational dopamine prediction-error framework and subsequent distributional work are examples of productive exchanges between formal modeling and experiment Montague et al. (1996) Schultz et al. (1997) Dabney et al. (2020).

Correspondence between algorithms and brains has multiple dimensions. Functional correspondence concerns whether systems solve a comparable learning problem; representational correspondence concerns whether variables encode similar information; mechanistic correspondence concerns whether a particular circuit and update rule actually instantiate the computation. These distinctions prevent category errors. A trained artificial agent can exhibit error-driven policy adjustment through numerical optimization without using dopamine receptors, basal ganglia pathways, or neuronal spike timing. A neural system may exhibit distributional response patterns without implementing the exact parameters or training procedure of a published software model Dabney et al. (2020) Lee et al. (2024).

In the Artificial Era, cross-system comparison can extend to adaptive feedback, distributed memory, human–artificial cooperation, and learned preferences, provided every hypothesis receives operational definitions. A formal model may predict what information an artificial system retains about expected reward and how new evidence changes its behavior, while a biological study investigates the cellular processes supporting an analogous adjustment. Such research benefits from carefully specified tests that transfer across systems without presuming identical internal mechanisms. The present dossier maintains its neural scope and uses artificial learning as a conceptual comparator; independent studies of actual artificial learning belong to dedicated scientific dossiers Starkweather et al. (2017) Dabney et al. (2020).

Conclusion

Evidence across more than seven decades of reinforcement neuroscience supports a coherent account of reward prediction and neural learning while requiring disciplined distinctions among levels of evidence. Electrical self-stimulation demonstrated that neural interventions can reinforce actions. Primate recordings revealed responses sensitive to reward expectation and timing. Computational models identified a precise family of candidate teaching signals. Cellular and corticostriatal studies supplied plausible substrates for activity-dependent modulation, and targeted causal experiments demonstrated that dopamine-neuron transients can alter associative acquisition in defined tasks. Together these results strongly support a role for dopaminergic circuits in updating reward-relevant behavior Olds & Milner (1954) Schultz et al. (1997) Reynolds et al. (2001) Steinberg et al. (2013).

The mechanistic picture is more distributed than a single-value teaching equation. Dopamine neurons and terminal fields vary across projections, reward dimensions, uncertainty, and behavioral phases. Population responses may carry distributional or feature-specific information; hidden-state inference changes the expectations against which outcomes are compared; terminal dopamine dynamics can relate to work and motivation as well as learning. These findings preserve the importance of prediction-error accounts while clarifying their domain. The central unresolved question is how circuit-specific signals are transformed into temporally selective synaptic updates that generate outcome-specific learning across different ecological circumstances Parker et al. (2016) Starkweather et al. (2017) Mohebi et al. (2019) Dabney et al. (2020).

Future progress depends on triangulating precise recordings, identifiable computational models, well-controlled causal interventions, and subsequent behavior measured without perturbation. Experiments should compare alternative neural encodings in held-out contexts, map signals to defined projection targets and receptor pathways, distinguish immediate performance from retained associations, and examine how age, species, physiological state, and social environments influence generalization. The resulting synthesis establishes reward prediction and learning as a tractable multilevel scientific object whose explanatory architecture joins computation, neural transmission, plasticity, and behavior. Its broader relevance extends to psychology, neuroscience, and Artificial Era comparative inquiry through testable mechanisms rather than mere verbal analogy Sharpe et al. (2017) Yagishita et al. (2014) Kahnt & Schoenbaum (2025).

References

  • Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47(1), 129–141. https://doi.org/10.1016/j.neuron.2005.05.020
  • Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., & Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature, 482, 85–88. https://doi.org/10.1038/nature10754
  • Dabney, W., Kurth-Nelson, Z., Uchida, N., Starkweather, C. K., Hassabis, D., Munos, R., & Botvinick, M. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577, 671–675. https://doi.org/10.1038/s41586-019-1924-6
  • Eshel, N., Tian, J., Bukwich, M., & Uchida, N. (2016). Dopamine neurons share common response function for reward prediction error. Nature Neuroscience, 19, 479–486. https://doi.org/10.1038/nn.4239
  • Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299(5614), 1898–1902. https://doi.org/10.1126/science.1077349
  • Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., Kennedy, R. T., Aragona, B. J., & Berke, J. D. (2016). Mesolimbic dopamine signals the value of work. Nature Neuroscience, 19, 117–126. https://doi.org/10.1038/nn.4173
  • Hart, A. S., Rutledge, R. B., Glimcher, P. W., & Phillips, P. E. M. (2014). Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. Journal of Neuroscience, 34(3), 698–704. https://doi.org/10.1523/JNEUROSCI.2489-13.2014
  • Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1, 304–309. https://doi.org/10.1038/1124
  • Howard, J. D., & Kahnt, T. (2018). Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex. Nature Communications, 9, Article 1611. https://doi.org/10.1038/s41467-018-04055-5
  • Kahnt, T., Park, S. Q., Haynes, J.-D., & Tobler, P. N. (2014). Disentangling neural representations of value and salience in the human brain. Proceedings of the National Academy of Sciences, 111(13), 5000–5005. https://doi.org/10.1073/pnas.1320189111
  • Kahnt, T., & Schoenbaum, G. (2025). The curious case of dopaminergic prediction errors and learning associative information beyond value. Nature Reviews Neuroscience, 26, 169–178. https://doi.org/10.1038/s41583-024-00898-8
  • Lee, R. S., Sagiv, Y., Engelhard, B., Witten, I. B., & Daw, N. D. (2024). A feature-specific prediction error model explains dopaminergic heterogeneity. Nature Neuroscience, 27, 1574–1586. https://doi.org/10.1038/s41593-024-01689-1
  • Menegas, W., Babayan, B. M., Uchida, N., & Watabe-Uchida, M. (2017). Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. eLife, 6, e21886. https://doi.org/10.7554/eLife.21886
  • Mirenowicz, J., & Schultz, W. (1994). Importance of unpredictability for reward responses in primate dopamine neurons. Journal of Neurophysiology, 72(2), 1024–1027. https://doi.org/10.1152/jn.1994.72.2.1024
  • Mohebi, A., Pettibone, J. R., Hamid, A. A., Wong, J.-M. T., Vinson, L. T., Patriarchi, T., Tian, L., Kennedy, R. T., & Berke, J. D. (2019). Dissociable dopamine dynamics for learning and motivation. Nature, 570, 65–70. https://doi.org/10.1038/s41586-019-1235-y
  • Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936–1947. https://doi.org/10.1523/JNEUROSCI.16-05-01936.1996
  • Olds, J., & Milner, P. (1954). Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47(6), 419–427. https://doi.org/10.1037/h0058775
  • Parker, N. F., Cameron, C. M., Taliaferro, J. P., Lee, J., Choi, J. Y., Davidson, T. J., Daw, N. D., & Witten, I. B. (2016). Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target. Nature Neuroscience, 19, 845–854. https://doi.org/10.1038/nn.4287
  • Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001). A cellular mechanism of reward-related learning. Nature, 413, 67–70. https://doi.org/10.1038/35092560
  • Rutledge, R. B., Skandali, N., Dayan, P., & Dolan, R. J. (2014). A computational and neural model of momentary subjective well-being. Proceedings of the National Academy of Sciences, 111(33), 12252–12257. https://doi.org/10.1073/pnas.1407535111
  • Saunders, B. T., Richard, J. M., Margolis, E. B., & Janak, P. H. (2018). Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties. Nature Neuroscience, 21, 1072–1083. https://doi.org/10.1038/s41593-018-0191-4
  • Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. https://doi.org/10.1126/science.275.5306.1593
  • Sharpe, M. J., Chang, C. Y., Liu, M. A., Batchelor, H. M., Mueller, L. E., Jones, J. L., Niv, Y., & Schoenbaum, G. (2017). Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nature Neuroscience, 20, 735–742. https://doi.org/10.1038/nn.4538
  • Shen, W., Flajolet, M., Greengard, P., & Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science, 321(5890), 848–851. https://doi.org/10.1126/science.1160575
  • Starkweather, C. K., Babayan, B. M., Uchida, N., & Gershman, S. J. (2017). Dopamine reward prediction errors reflect hidden-state inference across time. Nature Neuroscience, 20, 581–589. https://doi.org/10.1038/nn.4520
  • Steinberg, E. E., Keiflin, R., Boivin, J. R., Witten, I. B., Deisseroth, K., & Janak, P. H. (2013). A causal link between prediction errors, dopamine neurons and learning. Nature Neuroscience, 16, 966–973. https://doi.org/10.1038/nn.3413
  • Tai, L.-H., Lee, A. M., Benavidez, N., Bonci, A., & Wilbrecht, L. (2012). Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nature Neuroscience, 15, 1281–1289. https://doi.org/10.1038/nn.3188
  • Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005). Adaptive coding of reward value by dopamine neurons. Science, 307(5715), 1642–1645. https://doi.org/10.1126/science.1105370
  • Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R., Urakubo, H., Ishii, S., & Kasai, H. (2014). A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science, 345(6204), 1616–1620. https://doi.org/10.1126/science.1255514