Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

Prediction and Surprisal in Language Comprehension

Abstract. Prediction and Surprisal in Language Comprehension investigates how readers and listeners use linguistic and situational context to anticipate possible continuations and how the conditional probability of incoming information relates to processing difficulty. This critical narrative research monograph reconstructs experimental and computational traditions, defines conditional probability, lexical and structural surprisal, entropy and predictive preactivation, and evaluates evidence from cloze completions, eye movements, self-paced reading, event-related potentials, neuroimaging and naturalistic corpora. Particular attention is given to the distinction between referential anticipation, semantic facilitation, and form-specific lexical preactivation; to the nine-laboratory replication of prenominal article effects; to the interactions of memory, syntax, goals, proficiency and discourse; and to discrepancies between human reading and language-model probabilities. The synthesis supports distributed, probabilistic contextual sensitivity while identifying unsettled questions about representation, causal mechanisms, cross-linguistic generalization, model calibration and prediction's functional role. The work reviews existing research and makes no claim to having conducted a new experiment, systematic review or meta-analysis.

Keywords: predictive language comprehension; surprisal; word predictability; conditional probability; cloze probability; psycholinguistics; N400; visual world paradigm; anticipatory eye movements; syntactic prediction; semantic preactivation; lossy-context surprisal; naturalistic reading; language models; prediction error; Artificial Era

Introduction: Why Anticipation Matters for Comprehension

Language arrives in time. A listener hears a verb before its object; a reader receives one word while still interpreting the preceding phrase. Comprehension therefore requires continual coordination of incoming evidence with accumulated context. The research question is whether that coordination includes anticipatory probability distributions over what might come next, what kinds of representations are involved, and how those distributions affect observable effort. The answer depends on distinguishable phenomena: contextual facilitation at a presented word, advance allocation of attention toward a likely referent, and preactivation of the phonological or orthographic form of a word not yet encountered. The empirical record supports each claim to different degrees, and a responsible theory must specify its measurements rather than treating all context effects as equivalent Kuperberg & Jaeger (2016) Pickering & Gambi (2018).

This monograph examines prediction as a family of computational and psychological processes and surprisal as one mathematically specified measure of the unexpectedness of an observed event under a conditional probability model. It integrates controlled experiments, event-related potentials, eye movements in scenes and text, naturalistic corpora, linguistic probability models, and tests of neural activity during continuous speech. The evidence is organized by inferential strength: a causal manipulation of context establishes its influence on processing; a temporal measure can constrain when that influence begins; and a computational fit tests whether a particular quantity predicts variation. None of these results automatically identifies the mechanism generating the effect. Foundational formulations by Hale and Levy provide an expectation-based quantitative account of syntactic processing, while reading-time analyses test generality beyond carefully selected examples Hale (2001) Levy (2008) Smith & Levy (2013).

The research boundaries follow the approved Psychology of Language dossier architecture. Local syntactic parsing, lexical ambiguity resolution, discourse integration, and language production have their own neighboring research objects. The present work concentrates on conditional expectations linking those processes to upcoming linguistic and referential events, the difficulty of low-probability continuations, and the evidence for or against specific predictive commitments. It also considers language models as scientific measuring instruments, including cases where computationally superior prediction departs from human reading behavior. This perspective connects human psycholinguistics to the Artificial Era through testable questions about inference, calibration, experience, and cognitive partnership, while retaining direct empirical evidence about human language understanding as the primary object of explanation Oh & Schuler (2023b) Kaye & Gordon (2025).

Probability, Prediction, and Operational Definitions

Let a context be the sequence of previously encountered linguistic and relevant situational information. The conditional probability P(w|c) represents how likely a candidate word w is under a model conditioned on context c; it is a distribution over possible continuations, not a report of one consciously anticipated word. Lexical predictability may be estimated by cloze completion, in which people supply a continuation, or by a statistical language model assigning probabilities to observed strings. These estimators need not agree because cloze tasks demand explicit production and may amplify a dominant response, whereas model estimates depend on training distributions, tokenization, and conditioning context. Kuperberg and Jaeger describe prediction at multiple representational levels, and Luke and Christianson show why an expectation for semantic or morphosyntactic features can exist when no single lexical candidate has high probability Kuperberg & Jaeger (2016) Luke & Christianson (2016).

Predictive preactivation is an empirical claim about a representation becoming more available before the corresponding bottom-up signal arrives. It may concern a visual object, an event role, a grammatical category, a semantic feature, a lexical item, or a sound pattern. Commitment denotes a stronger state in which one continuation receives sufficient processing priority that a different continuation might require revision or incur a measurable cost. A prediction error is the discrepancy between expected and observed information, but theoretical traditions operationalize discrepancy differently: as a signed residual, as probability reduction after observation, or as a change in an internal representation. Naming these constructs separately prevents an effect in one measure from silently becoming evidence for another Huettig (2015) Pickering & Gambi (2018).

Expectation can be explicit, implicit, or inferred from behavior. A participant may know that a completion is likely without ever naming it; an eye movement can indicate attention to a semantically compatible object without proving that its name was preactivated. Likewise, a low-amplitude N400 can reflect semantic access, contextual fit, or predictive facilitation depending on experimental design and competing accounts. This dossier treats expectations as graded statistical constraints whose psychological implementation must be tested rather than assumed. It requires each claim to name its representational level, context, processing time, dependent variable, and admissible alternative explanations. Such specification makes theories mutually comparable and helps prevent the same observation from being counted several times under different theoretical labels Kutas & Federmeier (2011) Huettig et al. (2011).

The Experimental History of Context and Anticipation

Early electrophysiological evidence established that the processing of a word is sensitive to its position in meaningful linguistic context. Kutas and Hillyard found that the amplitude of the N400 to sentence-final words varied inversely with expectancy measured by cloze probability. Unexpected but semantically related completions produced smaller responses than unrelated unexpected completions, demonstrating that semantic organization and contextual expectancy interact. The critical historical contribution was the possibility of observing a context-related neural response on a word-by-word time scale, rather than relying exclusively on end-of-sentence judgments. This work did not by itself decide whether preactivation occurred before the word or whether the response indexed easier semantic processing after the word appeared Kutas & Hillyard (1984) Kutas & Federmeier (2011).

Subsequent evidence broadened the representational question. Federmeier and Kutas compared expected exemplars with unexpected category-related and category-unrelated completions, showing that semantic memory organization shapes context effects even when the most likely item fails to appear. Altmann and Kamide then showed that listeners viewing a scene often shifted gaze toward an appropriate target after hearing a constraining verb and before the target noun began. These demonstrations established distinctive methods for testing semantic relatedness and referential anticipation, respectively. They also made clear why predicting a category, object, or thematic role can be experimentally easier to establish than anticipating a precise word form Federmeier & Kutas (1999) Altmann & Kamide (1999).

Quantitative computational formulations transformed the field by expressing incremental difficulty as a function of probability distributions over possible sentence continuations. Hale proposed using the surprisal of a word within a probabilistic parser to model processing load, and Levy elaborated expectation-based syntactic comprehension as a theory in which the probability of continuation matters even without an actual structural contradiction. The resulting program connected corpora, formal grammar, and chronometric data and made predictions across grammatical constructions. Later empirical work examined how far this mathematics extends to natural reading, neural signals, and individual differences. Its historical importance lies in providing falsifiable functional relations; an adequate mechanistic account still has to explain how people acquire, represent, and exploit the relevant conditional distributions Hale (2001) Levy (2008) Demberg & Keller (2008).

Information Theory: Surprisal and Entropy

Surprisal is defined as S(w|c) = −log P(w|c), with the logarithm base determining the unit. A base-two logarithm expresses information in bits; a natural logarithm yields nats. An observed word with probability one has zero surprisal, and decreasing conditional probability increases surprisal. Entropy, by contrast, is the expected surprisal across the whole distribution of potential next events: H(W|c) = −Σw P(w|c) log P(w|c). Surprisal concerns the word that actually arrived; entropy concerns the uncertainty remaining before it arrives. Both quantities are mathematical properties of a specified model and reference distribution. They become psychological hypotheses only when connected to independently measured reading, listening, or brain responses Hale (2001) Levy (2008).

Suppose a model assigns probability one half to a candidate and one thirty-second to another. Under base-two logarithms their surprisals are one bit and five bits. This illustrative contrast clarifies the scale without claiming that actual human effort rises by precisely four processing units. A linear relationship between processing time and surprisal predicts a logarithmic relationship between processing time and raw conditional word probability. Smith and Levy investigated that functional form in reading and reported a robust logarithmic relationship over a large estimated predictability range. The result is important because it tests a quantitative gradient rather than merely contrasting categorically expected and unexpected words; its interpretation remains dependent on the probability estimator and the model of reading times Smith & Levy (2013).

Entropy reduction is another candidate account of processing. When a word eliminates several plausible syntactic or semantic possibilities, the posterior distribution may contract sharply; however, a reduction in uncertainty and the observed word's surprisal are not mathematically identical. A model can be highly confident before a word arrives and nevertheless be wrong, producing high surprisal; a model can also remain uncertain after an unsurprising word. Computational studies should therefore describe whether they measure word surprisal, syntactic surprisal, entropy, entropy reduction, or an information-theoretic approximation to memory costs. Treating these quantities as interchangeable would obscure the very distinctions experimental designs aim to test. Their empirical interpretation also depends on structural hypotheses and the grain of the lexical units Frank et al. (2015) Futrell et al. (2020).

Mechanisms and Levels of Predictive Representation

Anticipation may be generated by learned associations between words, combinatorial constraints from grammar, semantic relations between events and their likely participants, discourse-level pragmatic inference, or simulation of a speaker's production choices. These mechanisms need not produce the same representational output or operate under the same timing constraints. A strongly associated pair can facilitate recognition even when the larger sentence predicts a different continuation; a grammatical cue can restrict gender or number without privileging one specific noun. Huettig argues that a single universal mechanism does not adequately cover the empirical range and identifies multiple routes of prediction. Pickering and Gambi place production-based processes among the candidates, while acknowledging alternative sources of activation Huettig (2015) Pickering & Gambi (2018).

Representational levels are causally interdependent. Knowledge about an unfolding event influences plausible referents; a referent constrains semantic concepts; semantic concepts narrow lexical candidates; lexical candidates carry phonological or orthographic features. But inference from a high-level expectation to the advance activation of every lower-level feature requires evidence about the actual propagation process. A listener may look at an edible object after hearing a verb related to eating while leaving several different names or descriptions equally possible. Conversely, function words or agreement morphology can carry information that reveals a lexical expectation before its noun is spoken. The strength of these designs arises from their ability to probe specific levels, provided attention, lexical competition, and integration accounts are controlled Altmann & Kamide (1999) Van Berkum et al. (2005) DeLong et al. (2005).

The concept of probabilistic prediction also accommodates adaptive allocation of resources. Generating fine-grained predictions can be useful when contextual reliability is high, linguistic input is rapid, and the task rewards early preparation; it may be less useful when opportunities for continuation are diffuse or attention is divided. A theory must therefore explain when comprehenders estimate broad distributions, when they sharpen onto particular candidates, and when they rely predominantly on processing the incoming signal. Evidence from prediction reviews suggests interactions among goals, experience, costs, and expected usefulness rather than an obligatory lexical guess at every position. This layered view guides the empirical chapters that follow and keeps behavioral facilitation distinct from claims about the architecture generating it Kuperberg & Jaeger (2016) Van Petten & Luka (2012) Luke & Christianson (2016).

Cloze Completion and the Measurement of Expected Words

Cloze probability is usually obtained by presenting a sentence fragment and asking participants to provide the word they would naturally use next. The proportion selecting a continuation estimates its production probability under that task and participant population. The measure is attractive because it does not assume that a machine language model faithfully reproduces human expectations. At the same time, it is sensitive to stimulus truncation, instructions, participant sampling, response normalization, spelling variation, and the availability of plausible alternatives. An independently collected cloze sample is therefore an important component of studies that characterize words as strongly or weakly expected. Comparing different completion populations can reveal variability, while using cloze scores from the same participants who later perform the target task risks dependencies requiring explicit analysis Luke & Christianson (2016) Staub (2015).

Cloze scores do not establish that a reader committed to the response with highest frequency. A distribution may contain many weak competitors; one semantic category might be strongly expected even when no single word receives a high cloze proportion. Luke and Christianson collected norms for thousands of words in connected text, finding that highly predictable content words were relatively uncommon and that a more probable alternative existed for most content words. Their eye-tracking results showed facilitation associated with graded predictability and did not identify the anticipated penalty for failing to receive a single privileged lexical item. The study is particularly relevant to the distinction between broad constraint and form-specific guessing Luke & Christianson (2016).

Reliable measurement demands separate attention to the number of response options, the number of cloze participants, treatment of morphological variants, and the target language's writing system. A zero observed cloze response does not prove that the true probability of that word is mathematically zero; it may simply be unobserved in a finite sample. Computational studies sometimes smooth these probabilities or use corpus-trained model estimates instead, introducing additional assumptions about experience and representation. When a study reports a correlation between cloze and response time, the researcher should check whether word frequency, sentence position, length, semantics, and individual differences have been addressed. The estimate is most useful when treated as an operational predictor with a known sampling process rather than a transparent readout of private lexical forecasts Smith & Levy (2013) Kuperberg & Jaeger (2016).

Anticipatory Eye Movements in the Visual World

In the visual-world paradigm, participants view objects or events while hearing language, and their gaze provides a continuous behavioral measure of changing attention. Altmann and Kamide's classic experiment manipulated a verb's semantic restrictions in scenes containing a uniquely compatible object. When the verb referred to eating, looks toward the edible object often began before the noun named it; when the verb described a less restrictive action, target-directed eye movements occurred later. The manipulation links linguistic information to prospective referential attention at a moment when the spoken object expression has not yet arrived. It thus provides strong evidence of anticipatory use of verb and scene constraints in that task, with a temporal resolution unavailable to final comprehension questions Altmann & Kamide (1999).

Interpretation nevertheless depends on the relationship between visual targets and linguistic forms. A target-directed saccade may be driven by object semantics, event plausibility, association between verbs and pictured affordances, or an anticipated referential role. The timing of saccade initiation, visual display complexity, fixation availability, and acoustic properties of the verb all affect observed trajectories. Huettig, Rommers, and Meyer emphasize that the paradigm is especially informative about interactions between language and displayed visual information, while generalizing its results to comprehension without any visual display requires additional evidence. In particular, a look toward a cake cannot independently demonstrate that the comprehender preactivated the word cake rather than a broader concept such as edible object Huettig et al. (2011).

Strong visual-world experiments specify an appropriate analysis window relative to the earliest auditory cue, estimate saccadic latency, and control how often a target is present and how visually salient it is. Designs can contrast highly plausible competitors that share semantic features, sound onset, grammatical class, or event roles, thereby probing the information underlying early looks. The central inference concerns a distribution over potential referents and its influence on attention, not necessarily the generation of a single lexical candidate. These methods also illuminate practical comprehension in visually grounded communication, but scene restriction may make anticipation easier than ordinary conversation. Accordingly, the evidence is best integrated with text reading and brain measurements instead of being treated as a universal demonstration that all linguistic inputs trigger lexical forecasts Altmann & Kamide (1999) Huettig (2015).

Eye Movements and Predictability in Silent Reading

Natural reading generates several dependent measures: first-fixation duration, gaze duration, go-past time, skipping probability, and rereading. Contextual predictability affects these measures, although they represent overlapping processes and respond differently to parafoveal preview and oculomotor constraints. A high-probability word may be recognized more quickly or skipped, while a low-probability continuation can require additional processing; the resulting associations have been observed both in controlled stimuli and in naturalistic corpora. Staub's critical review argues that graded activation across candidate words offers a plausible explanation for many predictability effects and examines how cloze probabilities interact with frequency and parafoveal information. No one eye-movement measure cleanly separates lexical access from integration or attention allocation Staub (2015).

Smith and Levy combined computational probability estimation with major reading datasets to analyze the shape of the predictability–latency function. Their evidence supported a logarithmic relationship rather than a simple effect limited to a few strongly expected words. Naturalistic approaches improve ecological coverage because they observe words across diverse linguistic contexts, but sentence-level variability and correlated lexical predictors complicate causal interpretation. Demberg and Keller compared surprisal with dependency locality theory using the Dundee eye-tracking corpus and found different patterns depending on grammatical category and cost measure. Their results underscore that model fit is a comparison among explicitly implemented accounts under particular statistical controls, rather than an automatic validation of a general theory of neural prediction Smith & Levy (2013) Demberg & Keller (2008).

An especially important limitation concerns the existence of unexpected competitors. Luke and Christianson observed that the presence of a more predictable alternative need not cause a detectable extra cost when the target appears, even though contextual probability facilitates the target's reading. The result challenges models requiring strong commitment to a single word and substantial recovery whenever that commitment fails. Shain's analyses of naturalistic reading further show that word frequency and contextual predictability can be difficult to distinguish statistically once both are modeled, complicating claims of separately identified memory-retrieval and predictive mechanisms. Cross-dataset replication should report not only significant coefficients but their stability across texts, participant groups, model choices, and reading-time distributions Luke & Christianson (2016) Shain (2019).

The N400 and Semantic Expectancy

The N400 is an event-related potential response associated with meaning processing, commonly analyzed within a post-stimulus window around several hundred milliseconds. Its amplitude varies with several properties of incoming meaningful input, including contextual expectancy, semantic relationship, and ease of accessing relevant knowledge. Kutas and Hillyard's sentence-completion study established a relationship between cloze expectancy and N400 amplitude; Federmeier and Kutas extended the evidence by considering semantic category structure. These findings provide a neural correlate of contextual sensitivity and constrain possible chronologies of semantic processing. They do not alone distinguish whether the brain carried an explicit lexical forecast before the word arrived or facilitated access once the stimulus became available Kutas & Hillyard (1984) Federmeier & Kutas (1999).

Kutas and Federmeier's comprehensive review emphasizes that N400 phenomena extend far beyond sentences, including varied meaningful perceptual inputs, and that their interpretation requires a theory of knowledge activation and contextual integration. For language research, amplitude changes can result from lexical frequency, semantic association, discourse coherence, or predictability. A valid preactivation experiment must therefore probe an effect before the target or use a property of an earlier expression diagnostic of the upcoming target's identity. N400 differences at the target itself demonstrate that context changed processing of that target, a substantial finding that still leaves open several psychological routes to the effect. This distinction is crucial when experimental interpretations move from an observed ERP component to claims about predictive coding Kutas & Federmeier (2011) Van Petten & Luka (2012).

Frank and colleagues linked probabilistic estimates to ERP amplitudes during sentence reading and found a relationship between N400 size and model-derived word surprisal. Their comparative analyses also showed that the chosen computational representation matters; a model's inability to explain additional variance under one dataset and model specification does not rule out hierarchical syntax in human comprehension. The N400 result indicates that statistical expectation can be a useful predictor of neural responses, but the correspondence does not identify the predictive representation's origin or prove that the N400 measures information-theoretic surprisal alone. Neural effects can reflect multiple interacting factors, requiring careful feature controls, baseline selection, temporal correction, and independent model comparison Frank et al. (2015) Brennan et al. (2016).

Preactivation Before the Noun and the Replication Question

Evidence for advance activation of an exact word form is more demanding than evidence for semantic facilitation. DeLong, Urbach, and Kutas exploited the English a/an alternation, where the initial sound of an anticipated noun can constrain the indefinite article preceding it. They reported a graded ERP effect at the article as a function of the cloze probability of the later noun, along with effects at the noun. If an article response reliably tracked the anticipated noun's phonological onset, it would support a form-specific prediction occurring before the predicted word was presented. The experiment accordingly became a central test case for the distinction between semantic and phonological preactivation DeLong et al. (2005).

Nieuwland and colleagues conducted a preregistered direct replication distributed across nine laboratories, with 334 participants. They replicated the association for the nouns, but the predicted article effect was not established in their preregistered analyses; exploratory Bayesian work suggested that any article effect could be substantially smaller than the original report. This result directly constrains the claim that fluent readers routinely activate the phonological forms of specific upcoming words to a reliably detectable degree. It does not establish the absence of all word-form prediction in every context, and it leaves intact other kinds of anticipatory evidence, including referential gaze and semantic constraints. The contrast between the two studies provides a concrete example of why independent replication changes the weight assigned to a mechanistic explanation Nieuwland et al. (2018).

Cross-study comparisons must attend to stimulus differences, participant variation, cloze estimation, ERP analysis windows, statistical power, and grammatical reliability. The 2005 and 2018 studies targeted a specific advance-form prediction indexed through one English article alternation, so generalizing their disagreement to all languages and all anticipated features would exceed the tests. Studies of discourse-mediated gender agreement show that other prereferential morphosyntactic cues can be informative: Van Berkum and colleagues found differential responses to prediction-inconsistent gender information in Dutch stories, with a control condition supporting a role for discourse. The stronger synthesis is therefore representationally specific: form-level commitment is context- and cue-dependent, and robust evidence for contextual probability does not license a universal preactivation mechanism Van Berkum et al. (2005) Kuperberg & Jaeger (2016) Nieuwland et al. (2018).

Probabilistic Parsing and Expectation-Based Syntax

Probabilistic parsing supplies an explicit bridge between grammatical structure and predictions for upcoming words. In a probabilistic grammar, a prefix maintains weighted alternatives for how a sentence might continue. When a new word arrives, its occurrence changes the relative probabilities of surviving analyses, and the total probability mass assigned to that word determines its lexical or structural surprisal. Hale modeled processing difficulty using a probabilistic Earley parser and demonstrated how disconfirmed expectations could account for classic ambiguity effects and differences between relative-clause constructions. Levy's broader account places expectation at the center of incremental syntactic comprehension and connects resource allocation to statistical knowledge. These proposals make precise claims about the relation between a model's distribution and observed processing cost Hale (2001) Levy (2008).

A probabilistic parser does not need to choose one winning analysis before a disambiguating word. Multiple syntactic hypotheses can remain active, each contributing probability to the word that appears next. A garden-path pattern can then arise because one continuation carried substantially more probability than another, even if the less expected continuation was grammatically possible throughout. This flexibility is a substantial theoretical advantage, since fluent comprehension often proceeds without explicit ungrammaticality, and unexpected input frequently requires updating rather than rebuilding an interpretation from scratch. However, the predicted costs depend on the chosen grammar, how probabilities are learned, and whether a structural representation maps adequately onto the comprehender's actual experience. Probability over strings is experimentally observable only through a particular estimation procedure Levy (2008) Frank & Bod (2011).

Expectation-based parsing must be evaluated against memory-centered alternatives. Gibson's locality account emphasizes the processing costs of keeping dependencies available and integrating structurally distant material. Demberg and Keller compared locality and surprisal effects in corpus eye movements and found that competing measures described different subsets of the data. The interpretation is not that syntax can be reduced to one scalar at all positions; rather, context-conditioned uncertainty and memory demands may make distinct predictions, especially as syntactic dependencies become long and complex. Good studies estimate incremental predictions on held-out text and investigate whether adding memory variables improves out-of-sample explanation after lexical, structural, and position-related confounds have been considered Gibson (1998) Demberg & Keller (2008) Futrell et al. (2020).

Memory Constraints and Lossy-Context Surprisal

Humans interpret new input using a remembered context, not a perfect record of everything previously encountered. Context information can decay, be overwritten, lose syntactic detail, or be represented as a compressed semantic summary. Futrell, Gibson, and Levy formalized this insight as lossy-context surprisal, allowing expectation-based and memory-based processing effects to be described within a common information-theoretic framework. In that approach, the distribution conditioning a word is based on noisy or incomplete context representations. The same word can be surprising to a listener who has lost an earlier dependency while remaining unsurprising to a model conditioned on the full text. This distinction matters for long sentences, discourse comprehension, and differences between short controlled trials and extended natural reading Futrell et al. (2020).

Lossy-context accounts also illustrate why linguistic probability alone is insufficient as an explanation. The same grammatical sequence can imply different processing demands depending on whether the relevant context survives in working memory and what type of representation remains. Such models predict interactions among distance, interference, and conditional expectation, and they invite experimental manipulations of intervening material rather than mere associations with word order. Gibson's analyses of structural locality helped establish memory demand as a competing resource-based explanation, while expectation-based work demonstrated that frequency information also accounts for substantial variance. Modeling both requires identifying which effects are distinct and which arise when contextual information is represented with uncertainty Gibson (1998) Levy (2008) Futrell et al. (2020).

A theoretical model of memory loss must specify what information is degraded and how the degree of loss is estimated or independently constrained. One could assume approximate retention of lexical identities, partial preservation of thematic roles, or probabilistic maintenance of syntactic relations, but these alternatives can yield different predictions. Without constraints, a highly flexible memory-noise function could fit many outcomes after the fact and lose explanatory force. The most informative evaluation jointly fits or tests reading-time, recall, and grammatical-interference measures, using genuinely held-out sentences and multiple complexity conditions. This research program changes the meaning of surprisal from a purely textual property to the information a particular comprehender can compute from a cognitively available history, while preserving the need to validate the memory mechanism empirically Futrell et al. (2020) Smith & Levy (2013).

Neuroimaging and Naturalistic Spoken Language

Continuous listening experiments address whether laboratory findings generalize to ordinary narratives and whether computational estimates correspond to brain activity during fluent comprehension. Willems and colleagues related model-derived prediction measures to activity recorded as participants processed natural language, providing evidence that contextual statistical properties are relevant outside isolated sentence completions. Brennan and colleagues compared predictions from models based on sequential dependencies with those using hierarchical grammatical structure against fMRI time courses. The findings indicated that both classes can capture neural variance, with different contributions in language-related temporal regions. The practical implication is that measured linguistic information and neural effects depend on the structure and timescale of the model and the signal, so different computational accounts require direct comparison Willems et al. (2016) Brennan et al. (2016).

Temporal resolution changes the inferences available. Functional MRI integrates activity over a hemodynamic response and offers spatially informative correlations, but it cannot by itself distinguish pre-word prediction from neural processing immediately after a particular word. Magnetoencephalography and electroencephalography measure more rapid neural changes yet require careful alignment of speech onset, phoneme boundaries, lexical timing, and overlapping responses. During natural listening, acoustic properties, articulation rate, word frequency, and syntactic position co-vary. Analyses that partial out these features and employ time-resolved regression are consequently more compelling than simple correlations between a linguistic surprisal series and a neural time series. The underlying challenge is to establish what psychological computation the fitted neural predictor actually indexes Heilbron et al. (2022) Kutas & Federmeier (2011).

Heilbron and colleagues used naturalistic audiobook listening and model-based predictors spanning different levels of linguistic representation to study hierarchical predictions in neural recordings. Their findings support a relationship between probabilistic expectations and ongoing processing across multiple levels, providing a useful complement to sentence-final N400 paradigms. Nevertheless, the model-defined hierarchy and human neural processes are connected inferentially through the fit to data; the result is not a direct microscopic observation of every intermediate predictive state. Together with fMRI studies, such work suggests that distributed, context-sensitive activity is a promising general property of comprehension. Determining how acoustic evidence, lexical probabilities, syntactic constraints, and semantic expectations interact causally remains a separate experimental task Heilbron et al. (2022) Brennan et al. (2016).

Language Models as Instruments of Cognitive Measurement

A statistical language model supplies estimated conditional word probabilities that can be used to calculate surprisal for every position in a corpus. Its value for psycholinguistics is methodological: it creates a reproducible numerical predictor across many expressions and contexts where collecting exhaustive human cloze judgments would be impractical. Goodkind and Bicknell examined the relationship between language-model quality and the predictive power of surprisal for reading times, showing that better probability estimates can improve behavioral modeling within particular model ranges. Yet model perplexity evaluates average predictive performance on text, while psychological fit evaluates how estimated probabilities correspond to human reaction measures. Those criteria are related but logically distinct and can diverge as model training and capacity change Goodkind & Bicknell (2018) Oh & Schuler (2023b).

Recent transformer studies show why the distinction is decisive. Oh and Schuler found that larger pretrained models with lower perplexity can yield surprisal estimates that explain human reading times less well, and identified systematic residual patterns involving uncommon names and function words. A separate analysis varying training data reported that the best behavioral fit could arise before additional training improved model language prediction. Later work implicated frequency and models' enhanced capacity to predict rare words in this divergence. The results imply that artificial systems can become exceptionally effective predictors of text while departing from the learning histories and processing constraints characteristic of human readers. Surprisal is only as psychologically appropriate as the estimated distribution that generates it Oh & Schuler (2023b) Oh & Schuler (2023a) Oh et al. (2024).

Calibration and tokenization create further methodological difficulties. A model may assign probabilities to subword units, so computing the conditional probability of a written word requires combining token probabilities in a consistent manner. Liu, Škrjanec, and Demberg found that temperature scaling could improve fits to reading times, with improvements influenced strongly by words composed of multiple subword tokens. This finding cautions against equating a transformed probability score with an independently verified human expectation distribution. Cross-language evaluations reinforce the issue: Boeve and Bogaerts examined Dutch language-model surprisal across sentence, paragraph, and book-reading material, indicating the need to test probability estimators in different contexts and languages. The key scientific object remains a human processing model whose assumptions can be evaluated separately from engineering performance Liu et al. (2024) Boeve & Bogaerts (2025).

Sequential Statistics, Hierarchical Grammar, and Explanatory Fit

Models can learn contingencies between nearby words, dependencies among grammatical categories, or explicitly hierarchical relations between constituents. Their comparative success helps clarify which regularities predict human processing, although statistical fit alone does not prove that participants implemented the best-fitting algorithm. Frank and Bod compared sequential and hierarchical probability models against sentence reading times and did not find additional variance explained by their hierarchical models under the tested conditions. Frank and colleagues later reported a similar advantage for certain nonhierarchical estimates in an N400 analysis. These are carefully delimited model-comparison results; the actual human knowledge of phrase structure can contribute to comprehension through mechanisms that the particular fitted predictors do not isolate Frank & Bod (2011) Frank et al. (2015).

Other evidence supports an independent role for structural representations. Brennan and colleagues used fMRI responses during narrative listening to evaluate computations informed by hierarchical grammars and observed associations in temporal regions after considering sequential models. This apparent tension is theoretically productive because the studies measure different outputs, employ different materials, and quantify structure in different ways. A syntactic model might capture a neural correlate of hierarchical parsing without improving one particular reading-time regression, or a sequential predictor might approximate many dependencies found in ordinary text without representing grammar in the same way as human learners. The relevant question is which model components deliver generalizable explanatory advantages under matched data, matched baselines, and preregistered comparisons Brennan et al. (2016) Levy (2008).

The strongest synthesis retains several levels of explanation. Contextual statistical regularities constrain expected words; grammar provides structured relationships among candidates; memory limitations affect which past cues remain available; and mechanisms of attention and semantic access determine how conditional information influences observable effort. Formal surprise can be calculated under any of these representations, but the psychological interpretation of a numerical coefficient depends on what the model regards as context. Future comparisons should prioritize held-out participants, unseen texts, multiple languages, and independent neural and behavioral validation. They should also test whether the same model makes successful predictions when discourse conditions and dependency complexity are experimentally manipulated, rather than relying exclusively on correlations collected from a single distribution of sentences Futrell et al. (2020) Heilbron et al. (2022) Smith & Levy (2013).

Discourse, World Knowledge, and Pragmatic Constraint

Context in comprehension includes more than the immediate preceding words. A story establishes participants, locations, intentions, prior events, and an evolving model of what is likely to occur. These resources can narrow the expected meaning of later utterances before local grammatical information is complete. Van Berkum and colleagues demonstrated discourse-dependent expectations in Dutch materials, showing that an upcoming noun's grammatical gender could be relevant before the noun itself appeared. Their control conditions helped separate context-driven anticipation from a generic response to locally anomalous morphology. The result establishes that expectations may propagate from discourse into local linguistic processing under sufficiently constraining circumstances, while preserving the question of which intermediate representations are actually preactivated Van Berkum et al. (2005).

World knowledge affects predictions through event plausibility and social conventions. If a sentence describes a person preparing to drink, several beverage-related nouns may become more likely; a distinctive conversational setting may shift the distribution toward an unusual item. Such shifts can facilitate semantic access without any participant having internally rehearsed an exact continuation. Accounts that assume one dominant response may fail when discourse supports a broad semantic field or multiple plausible intentions. Reviews by Kuperberg and Jaeger and by Pickering and Gambi emphasize that comprehension can combine probabilistic priors, communicative goals, and bottom-up evidence, possibly with different processing commitments at different levels. The testable question is how context changes the distribution of candidate interpretations and how that change affects measured effort Kuperberg & Jaeger (2016) Pickering & Gambi (2018).

Discourse experiments face special control problems because changing one antecedent or character often changes many correlated properties of the text. A target word's cloze probability may co-vary with plausibility, semantic association, lexical repetition, emotional salience, and syntactic predictability. Eye-tracking measures can reveal anticipatory object attention, and neural measures can reveal rapid context effects, but strong causal inference still requires careful matching and targeted manipulations. Studies should report whether participants could recover the discourse information, whether the same effect survived in texts with a different genre or speaker, and whether the anticipation was driven by the intended narrative relation. Discourse-level predictions remain a meaningful component of comprehending connected language, even when their exact form-level consequences vary Van Berkum et al. (2005) Van Petten & Luka (2012) Huettig et al. (2011).

Grammar, Morphology, and the Time Course of Commitment

Agreement features, gender marking, case, and grammatical dependencies can create opportunities to test anticipatory processing at a finer grain than conceptual referent selection. If a preceding phrase fixes the gender of a likely noun, a gender-marked adjective can conflict with that expectation before the noun appears. Van Berkum and colleagues used such logic in Dutch stories to probe prospective effects, and DeLong and colleagues used the English indefinite article alternation to probe anticipated phonological onset. Both strategies are ingenious because they attempt to examine prediction prior to the critical content word. Their differences also matter: one examines language-specific morphosyntactic marking in discourse and the other uses a particularly weak phonological cue in English Van Berkum et al. (2005) DeLong et al. (2005).

The generality of each result depends on how diagnostic the pretarget cue is. In languages with richer morphology, earlier grammatical forms may constrain possible lexical continuations more substantially than they do in English; in languages with less obligatory marking, the experimenter has fewer opportunities to test the same proposition. Even within a language, speakers vary in their familiarity with agreement patterns and the importance they attach to them during rapid comprehension. A cue can produce an early processing difference because it is incompatible with a constructed interpretation, because it violates grammatical distributional expectations, or because a word form has already become active. Distinguishing these accounts calls for orthogonal manipulations of semantic expectation and cue-specific grammatical probability Kuperberg & Jaeger (2016) Huettig (2015).

Prediction also interacts with integration over time. A comprehender can use an article to revise the relative plausibility of upcoming nouns, then integrate the noun using both the revised context and the acoustic or written form. A pretarget ERP effect would constrain the chronology of this updating; its absence does not imply that prediction began only after target onset, because some anticipatory processes may be too small, diffuse, or variable for the instrument to detect. The nine-laboratory replication of the English article result is therefore a precise limit on a specific strong version of phonological preactivation. Future research should compare diagnosticity across morphological systems and analyze both pretarget and target-locked outcomes in the same statistical framework Nieuwland et al. (2018) Kutas & Federmeier (2011).

Individual Differences, Goals, and Learning Histories

Readers and listeners enter a sentence with differing language experience, working-memory constraints, knowledge of the subject, and immediate goals. These variables can change conditional expectations before any difference in processing skill becomes visible in response time. A technical expert may anticipate terminology that a novice treats as surprising; a second-language learner may rely more on familiar local associations than on distant discourse or syntactic dependencies. These possibilities follow directly from defining probability relative to a learner's knowledge and accessible context. Kuperberg and Jaeger argue that preactivation may depend on the expected utility of prediction and the reliability of contextual evidence. Such a claim requires studies measuring actual experience and manipulating task demands rather than inferring individual strategies from group-average surprisal effects Kuperberg & Jaeger (2016).

Some mechanistic proposals link anticipatory comprehension to processes ordinarily used for producing speech. Under such accounts, a listener can partially simulate the speaker's likely production or exploit learned forward relations between intended messages and linguistic forms. Other approaches emphasize associative priming, combinatorial restrictions, or more general prediction mechanisms. Pickering and Gambi review evidence across levels and propose production-based routes while noting that such explanations must be distinguished from competing pathways. Individual variability provides a crucial test: if predictive effects depend on production fluency or task demands, those relations should be observable under carefully controlled measurement. Yet a correlation between language proficiency and anticipation would not by itself identify production simulation as the cause Pickering & Gambi (2018) Huettig (2015).

Language-model comparisons offer a complementary approach to experience, but they introduce their own assumptions. Large models may have seen vastly more text than a human participant and may treat rare names and technical expressions as more predictable. Oh and Schuler document divergences between increasingly trained transformer probabilities and measured human reading times, illustrating why the scale and distribution of learning history matter. Boeve and Bogaerts broaden the methodological landscape to Dutch reading across different text lengths, while Kaye and Gordon discuss the use of contemporary large language models as tools for understanding human sentence processing. Individual-difference research should therefore calibrate computational probabilities against human groups and tasks instead of treating one global artificial distribution as a universal model of readers Oh & Schuler (2023b) Boeve & Bogaerts (2025) Kaye & Gordon (2025).

Cross-Linguistic and Ecological Generalization

The strongest established demonstrations of predictive language processing come from particular combinations of language, task, stimulus format, and participant population. English reading studies dominate quantitative surprisal research, whereas Dutch investigations provide especially informative tests of gender-related pretarget cues. Generalizing from these cases requires explicit attention to word order, morphology, script, sentence segmentation, and language experience. A finding that depends on English a/an alternation cannot be assumed to hold in a language without an equivalent alternation. Likewise, correlations obtained from eye movements while reading alphabetic text may change in scripts with different word-boundary conventions or orthographic processing demands DeLong et al. (2005) Van Berkum et al. (2005) Nieuwland et al. (2018).

Ecological validity concerns more than the length of a stimulus. Real comprehension occurs amid distractions, changing topic knowledge, imperfect acoustics, varying speaker accents, and uncertain conversational intentions. Naturalistic reading corpora and audiobook neuroimaging expand the range of contexts sampled, but they also reduce the experimental control available for isolating one causal explanation. Heilbron and colleagues show the promise of neural measurements during extended listening, and Demberg and Keller demonstrate how corpus eye movements can adjudicate computational processing accounts. The question is whether an effect replicated in several laboratories under controlled conditions also predicts variability across genuinely distinct contexts. Both types of study are needed: controlled manipulation for causal specificity and naturalistic observation for the breadth of generalization Heilbron et al. (2022) Demberg & Keller (2008).

Cross-language comparison also tests what counts as a word for surprisal estimation. Some languages have morphologically complex units and others use writing conventions that challenge simplistic token boundaries. Subword tokenization decisions already influence English fits of transformer-based surprisal to reading times, as demonstrated by Liu and colleagues. A multilingual theory must specify how candidate continuations are represented and how probability mass is assigned across word forms, morphemes, and syntactic features. Boeve and Bogaerts' systematic evaluation in Dutch exemplifies the need for language-specific validation of model measures. The appropriate claim is that conditional expectations are a promising shared theoretical currency, while the psychologically operative representations and useful measurements require independent cross-linguistic tests Liu et al. (2024) Boeve & Bogaerts (2025).

Research Design, Reliability, and Causal Inference

A strong experiment identifies one predictive claim, manipulates its proposed cause, and measures the resulting change with a method appropriate to its timing and representational content. Cloze norms can establish the intended manipulation of probability; visual-world gaze can test prospective referential attention; prereferential morphology can constrain form-level anticipation; and reading times or the N400 can assess the consequences of encountering a continuation. No single method covers all theoretical levels, so convergent designs are especially valuable. The response measure must be interpreted in light of alternative explanations such as word frequency, plausibility, association, integration, acoustic cues, and strategic behavior. The history of the article-prediction controversy shows why transparent confirmatory contrasts are essential DeLong et al. (2005) Nieuwland et al. (2018) Staub (2015).

Many psycholinguistic datasets contain repeated observations from the same participants and linguistic items. Statistical evaluation should therefore account for clustering by reader and item, ideally including appropriate participant-level and item-level variation in probability effects. Multiple time windows and dependent measures create researcher degrees of freedom when selected after seeing the data. Preregistering primary outcomes, documenting preprocessing, preserving exclusions, reporting uncertainty intervals, and maintaining a clear distinction between confirmatory and exploratory analyses make interpretations more stable. The nine-laboratory replication of the prenominal article effect illustrates a useful combination of preregistered analyses and explicitly described exploratory alternatives. Its value extends beyond a single disputed result to the standard of evidence required for general claims about prediction Nieuwland et al. (2018).

Computational validation adds another set of safeguards. Surprisal estimates should be generated without leakage from test items into tuning decisions wherever feasible, compared across plausible model families, and evaluated on unseen text or participants. Regression analyses should test whether an added predictor explains meaningful variance beyond lexical and positional baselines, and examine residuals for patterns concentrated in rare words or multi-token items. Oh and colleagues' transformer analyses and Liu and colleagues' calibration tests show that probability-model choice can reverse substantive conclusions about humanlike processing. A credible synthesis therefore separates empirical regularities that survive estimator changes from theoretical interpretations dependent on one algorithmic implementation. This distinction helps organize accumulated evidence without overstating agreement across tasks and measurement traditions Oh et al. (2024) Liu et al. (2024) Shain (2019).

Competing Accounts of Facilitation and Surprise

At least four explanatory families must be kept distinct. A preactivation account proposes that context raises the readiness of likely representations before input arrives. An integration account attributes easier processing to how readily an observed item fits the developing interpretation. A memory-based account emphasizes which contextual dependencies remain available when processing takes place. A probabilistic resource-allocation account links difficulty to shifts in the distribution of possible continuations. These accounts can generate overlapping predictions for reading times and N400 amplitudes, making empirical discrimination difficult when an experiment manipulates only the contextual likelihood of a target. Research reviews recommend considering representational specificity and temporal order, rather than classifying every expectancy effect as evidence for the same predictive mechanism Van Petten & Luka (2012) Kuperberg & Jaeger (2016) Futrell et al. (2020).

Early referential eye movements constitute particularly relevant evidence against a view in which context influences comprehension only after the target noun appears, because the attention shift precedes that noun. Yet they remain compatible with several forms of anticipatory event modeling and do not require sound-form preactivation. Conversely, a smaller N400 to expected words is consistent with both preactivation and facilitated semantic access after onset. The DeLong–Nieuwland comparison further establishes that an initially influential pretarget index of phonological expectation did not generalize as strongly as first proposed. These observations support a differentiated account in which prediction is common at some levels and conditions, while a uniform lexical commitment mechanism is empirically too strong Altmann & Kamide (1999) Kutas & Federmeier (2011) Nieuwland et al. (2018).

Formal surprisal models add predictive specificity but cannot by themselves settle whether the processing system performs explicit anticipatory activation. A conditional probability distribution may summarize statistical regularities available to an ideal observer even when the human mechanism uses associative activation or rapid post-input inference. Theories differ in what is represented, when it is available, and which computation creates the latency cost. Experiments capable of separating them should orthogonally manipulate probability, semantic plausibility, lexical competition, and memory availability; they should also test outcomes before target onset where technically possible. The decisive result would be a consistent pattern of interactions predicted by one mechanism and excluded by its alternatives, replicated with independent materials and populations Hale (2001) Levy (2008) Pickering & Gambi (2018).

Integrating Evidence Across Behavioral and Neural Methods

The evidence is strongest for continuous contextual sensitivity during comprehension. Word expectancy is associated with N400 amplitudes, natural reading measures exhibit graded relationships with contextual predictability, and anticipatory gaze shows prospective use of verbs and scenes. Quantitative surprisal accounts explain meaningful variance across reading tasks, and naturalistic neural research links linguistic model predictions with ongoing activity. These conclusions originate from independent operational traditions, increasing confidence in the broad phenomenon. Their convergence is functional rather than exhaustive: the same conceptual label does not guarantee that EEG, eye tracking, and model-based reading regressions index an identical computation Kutas & Hillyard (1984) Smith & Levy (2013) Altmann & Kamide (1999) Heilbron et al. (2022).

The evidence is more conditional for exact lexical form preactivation, strong commitment costs, and universal representational hierarchies. The large preregistered article-effect replication reproduced noun expectancy effects but failed to establish the predicted article response; natural-text eye tracking revealed that highly predictable words constitute a small subset of content words and did not support a general penalty for receiving a less expected competitor. Model comparisons have also yielded disagreements about whether hierarchical syntax contributes explanatory variance beyond sequential context, and results vary with neural region, language model, and data type. Such findings limit broad mechanistic claims without negating anticipatory referential or semantic processing under appropriately constrained conditions Nieuwland et al. (2018) Luke & Christianson (2016) Frank & Bod (2011) Brennan et al. (2016).

A practical evidence hierarchy distinguishes experimentally manipulated anticipatory behavior, neural pretarget differences that survive independent replication, post-target responses to contextual fit, and correlational model predictions. Each offers a different inferential warrant. Experimental manipulation identifies causal effects of the tested cue; prereferential timing can exclude some post-target explanations; post-target measures show context-dependent processing without uniquely naming its temporal origin; and naturalistic computational regression establishes generalizable association if evaluated on independent data. This framework also highlights the need to report uncertainty intervals, analytical choices, and the sample of texts and participants. Scientific confidence grows when a conclusion remains valid after plausible changes to stimuli, signal measurement, and conditional-probability estimator Nieuwland et al. (2018) Staub (2015) Oh & Schuler (2023b).

Human–Artificial Comparisons in the Artificial Era

Machine language models create a new opportunity to formalize and test hypotheses about linguistic expectations. A model can assign a probability to an observed sequence and thereby supply word-by-word surprisal estimates for comparison with human data. It can also generate stimulus sets in which lexical probability, word frequency, syntactic structure, and semantic plausibility are systematically varied. These uses are scientific instruments rather than evidence that the model and a human reader employ identical mechanisms. Results showing deterioration of the fit to reading times as model size or training increases make the distinction empirically consequential. The Artificial Era broadens comparative inquiry because nonbiological language systems offer powerful, inspectable sources of candidate computational descriptions Oh & Schuler (2023b) Oh et al. (2024).

Comparisons should specify the dimensions of likeness under investigation. Similar output distributions, matching correlations with reading time, shared representational structure, and identical causal mechanisms are different propositions with different tests. A model trained on large quantities of text may predict rare words better than human participants while providing worse fits to their processing latencies. A temperature adjustment may improve prediction of measured data without demonstrating that the underlying representation is psychologically equivalent. Kaye and Gordon examine the methodological opportunities of language models for explaining reading, and Liu and colleagues show how tokenization and calibration can change the apparent quality of a surprisal metric. Together, these results recommend model-based experiments rather than casual analogies between computation and human experience Kaye & Gordon (2025) Liu et al. (2024).

Potential applications include adaptive reading support, assessment of text difficulty, study of comprehension in specialized domains, and human–artificial collaborative writing. Each application depends on more than a generic language-model surprisal score. Human reading difficulty reflects familiarity, goals, attention, discourse coherence, and available memory, so a model optimized solely for average text likelihood can misestimate the needs of a specific reader. Research on expert-adapted or language-specific probability estimation offers promising routes to personalization, but requires independent evaluation against readers' actual behavior. A responsible psychology of human–artificial relations will examine which predictions improve comprehension and which create misleading impressions of humanlike cognition, keeping functional performance and mechanisms analytically distinct Boeve & Bogaerts (2025) Oh & Schuler (2023a) Kuperberg & Jaeger (2016).

Unresolved Questions and an Empirical Research Agenda

The central unresolved representational question is how far predictions propagate before input arrives. Does an anticipated event activate a referent, a semantic field, a grammatical feature, a lexical candidate, and its sound pattern simultaneously, or does the system prioritize only a subset under task-dependent conditions? The answer requires experiments that manipulate cues diagnostic of each level and compare neural and behavioral effects within common participants and materials. Competing mechanisms must make distinguishable predictions for pretarget time windows, interference from unexpected but semantically related alternatives, and the circumstances under which prediction failure leads to measurable revision. The preactivation debate illustrates why precise distinctions among these outcomes are more informative than broad declarations about whether comprehension is predictive DeLong et al. (2005) Nieuwland et al. (2018) Huettig (2015).

A second problem concerns generalization and mechanism identification. Conditional probability estimates derived from language models must be calibrated against the reader's knowledge, including the influence of context length, discourse genre, technical specialization, and morphosyntactic structure. The best computer predictor of text may differ from the best model of human processing, and the differences themselves can identify relevant memory or learning constraints. Cross-linguistic investigations should compare tokenization policies, morphological features, and direct human expectation measures. Studies of naturalistic brain activity and reading should be complemented by experiments that intentionally change one contextual factor at a time, so the strength of prediction can be related to its actual causes Futrell et al. (2020) Oh & Schuler (2023b) Liu et al. (2024) Heilbron et al. (2022).

A third problem is methodological accumulation. Large independent replications and transparent synthesis procedures are essential when effects depend on flexible preprocessing or narrow stimulus properties. Future investigations should preregister theoretically diagnostic contrasts, report null and contradictory evidence, and archive item-level probabilities, stimuli, code, and analysis decisions wherever permissions allow. Individual-difference designs should examine whether apparent variation in prediction reflects distinct experience, memory, motivation, or measurement reliability. The long-term aim is a model that explains which representations are predicted, how uncertainty is updated, how surprisal relates to difficulty, and when an anticipated continuation is actively represented before its arrival. Each element should be supported by evidence appropriate to its explanatory level Nieuwland et al. (2018) Luke & Christianson (2016) Kutas & Federmeier (2011).

Conclusion

Prediction and surprisal provide a coherent framework for investigating the temporal structure of language comprehension. Conditional probabilities describe graded expectations over possible continuations; surprisal quantifies the unexpectedness of the continuation that actually occurs. Across controlled experiments, natural reading, visual-world eye movements, event-related potentials, and neuroimaging, linguistic context measurably affects processing and frequently supports anticipation of referents, semantic properties, or grammatical possibilities. Quantitative expectation-based accounts have generated successful predictions about reading difficulty and have linked formal probability estimation to neural responses. This convergence establishes contextual probabilistic sensitivity as a central empirical feature of comprehension, while the specific representational and causal machinery remains a subject of active investigation Levy (2008) Smith & Levy (2013) Heilbron et al. (2022).

The evidence also disciplines the scope of explanation. Referential gaze does not entail activation of an exact word form; a post-target N400 difference does not uniquely identify pretarget lexical prediction; and computational surprisal fit does not show that humans calculate probabilities by the model's algorithm. The preregistered failure to establish the influential English prearticle effect at the strength originally reported is a substantive constraint on routine phonological preactivation claims. Findings on naturalistic eye movements, memory limitations, and large model behavior likewise support graded, contextual, and task-dependent mechanisms instead of a single obligatory guess-and-repair strategy. A durable theory must preserve these distinctions and incorporate positive, negative, and incompatible results within the same explanatory program Nieuwland et al. (2018) Luke & Christianson (2016) Futrell et al. (2020).

The resulting scientific synthesis defines a clear research trajectory: measure uncertainty with psychologically validated estimators; discriminate semantic, referential, grammatical, lexical, and phonological predictions; identify how retained context and communicative goals modify processing; and compare models across languages, readers, and modalities using independent tests. Computational models, including artificial language systems, extend the range of hypotheses that can be examined but do not substitute for causal evidence about human comprehension. The fundamental object is the dynamic relation between contextual knowledge, upcoming linguistic alternatives, observed input, and the effort of constructing meaning. Its study connects foundational psycholinguistics to the comparative science of language in the Artificial Era Kuperberg & Jaeger (2016) Pickering & Gambi (2018) Kaye & Gordon (2025).

References

  • Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73(3), 247–264. https://doi.org/10.1016/S0010-0277(99)00059-1
  • Boeve, S., & Bogaerts, L. (2025). A systematic evaluation of Dutch large language models' surprisal estimates in sentence, paragraph and book reading. Behavior Research Methods, 57(9), Article 266. https://doi.org/10.3758/s13428-025-02774-4
  • Brennan, J. R., Stabler, E. P., Van Wagenen, S. E., Luh, W.-M., & Hale, J. T. (2016). Abstract linguistic structure correlates with temporal activity during naturalistic comprehension. Brain and Language, 157–158, 81–94. https://doi.org/10.1016/j.bandl.2016.04.008
  • DeLong, K. A., Urbach, T. P., & Kutas, M. (2005). Probabilistic word pre-activation during language comprehension inferred from electrical brain activity. Nature Neuroscience, 8(8), 1117–1121. https://doi.org/10.1038/nn1504
  • Demberg, V., & Keller, F. (2008). Data from eye-tracking corpora as evidence for theories of syntactic processing complexity. Cognition, 109(2), 193–210. https://doi.org/10.1016/j.cognition.2008.07.008
  • Federmeier, K. D., & Kutas, M. (1999). A rose by any other name: Long-term memory structure and sentence processing. Journal of Memory and Language, 41(4), 469–495. https://doi.org/10.1006/jmla.1999.2660
  • Frank, S. L., & Bod, R. (2011). Insensitivity of the human sentence-processing system to hierarchical structure. Psychological Science, 22(6), 829–834. https://doi.org/10.1177/0956797611409589
  • Frank, S. L., Otten, L. J., Galli, G., & Vigliocco, G. (2015). The ERP response to the amount of information conveyed by words in sentences. Brain and Language, 140, 1–11. https://doi.org/10.1016/j.bandl.2014.10.006
  • Futrell, R., Gibson, E., & Levy, R. P. (2020). Lossy-context surprisal: An information-theoretic model of memory effects in sentence processing. Cognitive Science, 44(3), e12814. https://doi.org/10.1111/cogs.12814
  • Gibson, E. (1998). Linguistic complexity: Locality of syntactic dependencies. Cognition, 68(1), 1–76. https://doi.org/10.1016/S0010-0277(98)00034-1
  • Goodkind, A., & Bicknell, K. (2018). Predictive power of word surprisal for reading times is a linear function of language model quality. In Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics (pp. 10–18). Association for Computational Linguistics. https://doi.org/10.18653/v1/W18-0102
  • Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics (pp. 1–8). Association for Computational Linguistics. https://doi.org/10.3115/1073336.1073357
  • Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022). A hierarchy of linguistic predictions during natural language comprehension. Proceedings of the National Academy of Sciences, 119(32), e2201968119. https://doi.org/10.1073/pnas.2201968119
  • Huettig, F. (2015). Four central questions about prediction in language processing. Brain Research, 1626, 118–135. https://doi.org/10.1016/j.brainres.2015.02.014
  • Huettig, F., Rommers, J., & Meyer, A. S. (2011). Using the visual world paradigm to study language processing: A review and critical evaluation. Acta Psychologica, 137(2), 151–171. https://doi.org/10.1016/j.actpsy.2010.11.003
  • Kaye, N. G., & Gordon, P. C. (2025). Sentence processing by humans and machines: Large language models as a tool to better understand human reading. Psychonomic Bulletin & Review, 32(6), 2719–2733. https://doi.org/10.3758/s13423-025-02756-9
  • Kuperberg, G. R., & Jaeger, T. F. (2016). What do we mean by prediction in language comprehension? Language, Cognition and Neuroscience, 31(1), 32–59. https://doi.org/10.1080/23273798.2015.1102299
  • Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology, 62, 621–647. https://doi.org/10.1146/annurev.psych.093008.131123
  • Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307(5947), 161–163. https://doi.org/10.1038/307161a0
  • Levy, R. (2008). Expectation-based syntactic comprehension. Cognition, 106(3), 1126–1177. https://doi.org/10.1016/j.cognition.2007.05.006
  • Liu, T., Škrjanec, I., & Demberg, V. (2024). Temperature-scaling surprisal estimates improve fit to human reading times—but does it do so for the right reasons? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol. 1, pp. 9598–9619). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.519
  • Luke, S. G., & Christianson, K. (2016). Limits on lexical prediction during reading. Cognitive Psychology, 88, 22–60. https://doi.org/10.1016/j.cogpsych.2016.06.002
  • Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., Von Grebmer Zu Wolfsthurn, S., Bartolozzi, F., Kogan, V., Ito, A., Mézière, D., Barr, D. J., Rousselet, G. A., Ferguson, H. J., Busch-Moreno, S., Fu, X., Tuomainen, J., Kulakova, E., Husband, E. M., . . . Huettig, F. (2018). Large-scale replication study reveals a limit on probabilistic prediction in language comprehension. eLife, 7, e33468. https://doi.org/10.7554/eLife.33468
  • Oh, B.-D., & Schuler, W. (2023). Transformer-based language model surprisal predicts human reading times best with about two billion training tokens. In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 1915–1921). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-emnlp.128
  • Oh, B.-D., & Schuler, W. (2023). Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times? Transactions of the Association for Computational Linguistics, 11, 336–350. https://doi.org/10.1162/tacl_a_00548
  • Oh, B.-D., Yue, S., & Schuler, W. (2024). Frequency explains the inverse correlation of large language models' size, training data amount, and surprisal's fit to reading times. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Vol. 1, pp. 2644–2663). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.eacl-long.162
  • Pickering, M. J., & Gambi, C. (2018). Predicting while comprehending language: A theory and review. Psychological Bulletin, 144(10), 1002–1044. https://doi.org/10.1037/bul0000158
  • Shain, C. (2019). A large-scale study of the effects of word frequency and predictability in naturalistic reading. In Proceedings of NAACL-HLT 2019 (Vol. 1, pp. 4086–4094). Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1413
  • Smith, N. J., & Levy, R. (2013). The effect of word predictability on reading time is logarithmic. Cognition, 128(3), 302–319. https://doi.org/10.1016/j.cognition.2013.02.013
  • Staub, A. (2015). The effect of lexical predictability on eye movements in reading: Critical review and theoretical interpretation. Language and Linguistics Compass, 9(8), 311–327. https://doi.org/10.1111/lnc3.12151
  • Van Berkum, J. J. A., Brown, C. M., Zwitserlood, P., Kooijman, V., & Hagoort, P. (2005). Anticipating upcoming words in discourse: Evidence from ERPs and reading times. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(3), 443–467. https://doi.org/10.1037/0278-7393.31.3.443
  • Van Petten, C., & Luka, B. J. (2012). Prediction during language comprehension: Benefits, costs, and ERP components. International Journal of Psychophysiology, 83(2), 176–190. https://doi.org/10.1016/j.ijpsycho.2011.09.015
  • Willems, R. M., Frank, S. L., Nijhof, A. D., Hagoort, P., & van den Bosch, A. (2016). Prediction during natural language comprehension. Cerebral Cortex, 26(6), 2506–2516. https://doi.org/10.1093/cercor/bhv075