RESEARCH DOSSIER
Multimodal Gesture and Language
Abstract. Multimodal gesture and language examines the coordinated production, comprehension, and learning of linguistic meaning through speech and visible communicative action. This critical integrative research monograph synthesizes the history of gesture studies; representational, interactional, and simulated-action accounts; controlled congruence and semantic-disambiguation experiments; electrophysiological and neuroimaging research; gesture timing and anticipatory processing; developmental longitudinal studies; experimental learning interventions; multilingual listening conditions; and cross-linguistic differences in event representation. Evidence includes original research, a published meta-analysis of 63 samples, and recent 2023–2025 theoretical and experimental studies. The synthesis distinguishes experimentally identified comprehension effects, prospective developmental associations, and task-specific instructional causal effects from competing mechanistic models. Scientific boundaries separate co-speech gesture from visual phonetic binding and autonomous signed languages, while outlining testable questions for human–artificial multimodal communication. The monograph reports no original experiments or registered systematic review.
Keywords: multimodal language; co-speech gesture; gesture–speech integration; iconic gesture; deictic gesture; psycholinguistics; visible action; semantic congruence; language comprehension; gesture production; electrophysiology; N400; language development; gestural prediction; developmental communication; gesture–speech mismatch; embodied cognition; cross-linguistic variation; gesture learning; gesture and attention; human–artificial communication
Introduction: Language as Coordinated Vocal and Visible Action
Human language ordinarily unfolds through concurrent vocal, visual, bodily, and environmental activity. A spoken utterance may identify an object while a pointing hand selects its referent, or describe a trajectory while a moving hand depicts a path unavailable in the words alone. Multimodal gesture and language is the study of how such visible communicative action and linguistic expression are produced, perceived, interpreted, and coordinated in time. Its central question concerns the organization of meaningful action across modalities rather than a comparison between two isolated channels. Natural conversation supplies the ecological problem, whereas tightly controlled gesture–speech experiments offer tests of particular mechanisms Kendon (2004) Holler & Levinson (2019).
This research dossier concentrates on hand movements, depiction, pointing, pragmatic positioning, and co-speech gestures as contributors to language production and comprehension. It also examines developmental gestures preceding spoken combinations, instructional gestures, neurocognitive integration, and differences across linguistic communities. Visual speech, including lip movements, provides an important neighboring source of auditory–visual information; the perceptual fusion of articulatory cues is addressed as a boundary case rather than the primary object. Conventional signed languages are complete linguistic systems, not simply co-speech gestures, and their autonomous grammatical structure deserves separate description McNeill (1992) Kendon (2004).
The review method is a critical integrative narrative synthesis: original behavioral experiments, developmental longitudinal observations, electrophysiological and imaging studies, controlled pedagogical interventions, scholarly monographs, and an existing quantitative meta-analysis are compared by their research questions and inferential reach. Evidence is evaluated for manipulation, naturalness, participant population, coding reliability, and degree of causal identification. This is an examination of existing studies, with no original participants, systematic-search protocol, or newly computed pooled effect size Hostetter (2011) Novack & Goldin-Meadow (2017).
Scientific Definitions and the Unit of Analysis
A gesture can be defined operationally as a visible bodily movement interpreted within an interaction as contributing communicative, representational, or organizational information. This definition requires specifying whose interpretation determines communicative relevance, whether a movement is intentionally produced, and how the researcher separates meaningful movement from instrumental action. In Kendon's framework, visible action counts as gesture through its participation in an utterance. A hand moving a cup and a hand depicting the movement of a cup may have similar kinematics while playing different roles in communication Kendon (2004) Streeck (2009).
Co-speech gesture identifies manual or bodily movement coordinated with spoken language, frequently classified by its referential, iconic, metaphoric, deictic, or interactional work. An iconic movement depicts selected aspects of an event, such as a circular motion for turning; a deictic gesture directs attention toward a visible or imagined locus. Beat-like movements may organize discourse timing without depicting referential content. These categories describe functions in context and often overlap within a single gesture. The coding unit may be a gesture stroke, a full preparation–stroke–retraction sequence, an utterance segment, or a complete interactional turn McNeill (1992) Kendon (2004).
Gesture–speech congruence concerns semantic compatibility between the verbal and visible contributions, while gesture–speech redundancy measures the amount of shared information. The two variables must be separated: a gesture can be semantically compatible and still introduce information absent from the words. Integration denotes evidence that processing one source changes the interpretation or processing of the other. Mere co-occurrence provides a stimulus description, whereas experimental interference, facilitation, altered neural response, or interlocutor uptake provides stronger evidence for functional linkage Kelly et al. (2010) Hostetter (2011).
Historical Formation of the Modern Research Problem
The contemporary field developed through converging traditions of linguistic anthropology, conversation analysis, psycholinguistics, developmental psychology, and cognitive neuroscience. Scholars of interaction examined pointing, embodied conduct, and the organization of visible utterances in face-to-face activity. Experimental psychologists studied how speech and hand movements jointly convey meaning, while developmental researchers noticed that children's gestures communicate knowledge not yet verbalized. These strands continue to differ in their preferred observational units and explanatory priorities Kendon (2004) Streeck (2009).
David McNeill's Hand and Mind advanced the study of spontaneous gestures as representations coupled to speech and thought, giving particular importance to temporally synchronized speech–gesture expressions. Susan Goldin-Meadow's Hearing Gesture developed the developmental argument that gestures reveal and can transform knowledge. Adam Kendon's Gesture: Visible Action as Utterance treated gesture as communicative conduct organized within situated social interaction. These books established distinct emphases: image–speech unity, learning and cognition, and the sequential organization of visible action McNeill (1992) Goldin-Meadow (2003) Kendon (2004).
Later approaches asked how gestures arise from action simulation, how language-specific packaging shapes depiction, and how brains integrate semantic information arriving through speech and hand movement. Empirical experiments moved from descriptions of gesture frequency toward explicit congruence manipulations, electrophysiological evidence, and counterfactual comparisons of instructional movements. The history thus documents an increasing capacity to distinguish production, comprehension, and causal learning effects even as the conceptual object remains multimodal situated conduct Hostetter & Alibali (2008) Kita & Özyürek (2003) Özyürek et al. (2007).
Forms, Functions, and Discourse Organization
Iconic depiction selects aspects of visible, imagined, or remembered events and renders them in manual form. The hand may trace a motion path, embody an object, describe a size, or enact an agent's action. These representational choices are selective: the gesture usually captures a spatial or dynamic feature rather than duplicating the full linguistic proposition. Deictic gestures establish or recover a referent, including objects outside immediate reach or entities represented in an imagined discourse space. The spatial locus can subsequently anchor further reference, making the gesture part of the evolving discourse McNeill (1992) Streeck (2009).
Gestures also organize conversation at the level of turn management, stance, and coordination. An open palm may orient an invitation or presentation, a movement may emphasize a contrast, and a visible response can signal readiness to answer before a spoken turn begins. Interpretation depends on position in the sequence of action, gaze, participant roles, and cultural convention. The same hand form cannot be assigned a universal psychological meaning independently of its interactional setting Kendon (2004) Holler et al. (2018).
Classifications are useful only when linked to the measurement purpose. A study of discourse structure may code gesture functions relative to speech clauses; a study of embodied conceptualization may code movement trajectory and imagery; a study of comprehension may manipulate whether a visible action supports or conflicts with the utterance. High coding agreement about gross movement onset does not automatically establish agreement about a gesture's semantic function. Researchers should report segmentation rules, coder independence, adjudication and whether gestures were assessed with or without access to accompanying speech Streeck (2009) Hostetter (2011).
Theoretical Architectures of Speech–Gesture Coordination
Integrated accounts propose that speech and gesture arise through coordinated representational activity and jointly contribute to a communicative message. The proposal predicts tight temporal relations, semantic alignment, and cross-modal influences during comprehension, while allowing different modes to express different features of a thought. McNeill's approach gives gesture a distinctive role in visible imagistic presentation; its central challenge for experimental research is to identify observable predictions that separate a shared conceptual origin from strategic coordination between otherwise separable processes McNeill (1992) Kelly et al. (2010).
The information-packaging perspective explains how spatial thinking is shaped by the linguistic commitments of a speaker's language. In motion-event descriptions, languages differ in how they distribute manner and path information across clauses and phrases. Kita and Özyürek compared speech and gesture across linguistic systems and argued for an interface representation linking spatial conceptualization with linguistic formulation. This approach predicts that gestural depiction is sensitive to the semantic units selected for speech, rather than a context-free visual transcript of an event Kita & Özyürek (2003) Özyürek et al. (2008).
The gestures-as-simulated-action account connects spontaneous movements with perceptual and motor simulations involved in thinking and speaking. It yields predictions about when people gesture, which movements they make, and what happens when gestures are inhibited. An embedded or extended perspective emphasizes how external movement reshapes the speaker's material and social problem-solving environment. These perspectives address distinct causal layers: internal simulation, linguistic formulation, communicative coordination, and the feedback of movement into cognition. Research advances when a study specifies which layer its manipulation can distinguish Hostetter & Alibali (2008) Hostetter & Alibali (2019) Pouw et al. (2014).
Production: Timing, Conceptualization, and Lexical Formulation
Speech and gesture production demands coordination of several processes with partially different temporal constraints. Speakers select concepts, construct a message, retrieve words, articulate, and move the body while adapting to an addressee. Gesture strokes often align with semantically relevant portions of speech, but preparation can precede the corresponding word and movement completion may follow it. This temporal organization makes frame-by-frame audio and video analysis central to a production account, especially when the proposed mechanism depends on what becomes available before verbal formulation McNeill (1992) Kendon (2004).
A depictive gesture may provide an additional pathway for representing a spatial relation or an event, while the accompanying speech serializes and grammatically packages that event. Cross-linguistic variation is especially informative because it separates stimulus content from linguistic constraints. In English and Turkish developmental comparisons, adult patterns of speech and co-speech depiction differed with language-specific event encoding, and children's acquisition of those patterns offered a developmental test of the coupling between linguistic structure and gesture Özyürek et al. (2008) Kita & Özyürek (2003).
Production models face competing causal explanations. Gesture may be a product of conceptual activation, a communicative aid for the listener, a support for lexical search, or a functional component of the speaker's own thinking; these roles can coexist within one conversation. Gesture suppression experiments, speech-disfluency measures, listener visibility manipulations, and dual-task studies can help dissociate the alternatives, yet each may change social context, attention, and movement load simultaneously. A credible theory must therefore specify the competing predictions and identify what is held constant Hostetter & Alibali (2008) Hostetter & Alibali (2019).
Comprehension: Semantic Integration Across Modalities
In a multimodal utterance, listeners construct a message-level interpretation from partially overlapping streams of evidence. Words provide lexical and grammatical constraints, whereas visible hand movements can contribute depiction, direction, reference, or stance. Interpretation is often incremental: information available from gesture can constrain the meaning assigned to upcoming words and a spoken phrase can constrain what an ambiguous gesture depicts. This account predicts measurable changes in comprehension when the same words accompany different gestures, with the strongest inference arising from experimental controls that preserve verbal content Kelly et al. (2010) Holler & Levinson (2019).
Kelly, Özyürek, and Maris (2010) used action primes followed by bimodal targets and varied whether the spoken and gestural information was congruent. Participants' speed and accuracy changed with compatibility, and weak versus strong mismatches produced different interference. The reported bidirectional interactions in the first experiment and obligatory effect proposed in the second support functional coupling in the tested action-word paradigm. The design establishes cross-modal effects for controlled stimuli; it does not require that every type of natural gesture is automatically integrated under every attentional or social condition Kelly et al. (2010).
Redundancy must be treated as an empirical variable. A compatible gesture repeating the word's content can improve confidence or speed, while a complementary gesture may change which proposition is inferred. A gesture giving information absent from the speech often has greater potential communicative value, yet too much detail or an unfamiliar depictive convention may impose processing demands. The choice between a redundancy benefit and a complementarity benefit depends on the outcome measure, topic, listener characteristics, and baseline difficulty Hostetter (2011) Holler & Levinson (2019).
Ambiguity Resolution and the Consequences of Conflict
Verbal ambiguity permits a stringent test of gesture's interpretive contribution. A gesture that depicts an alternative sense of a homonym can influence what listeners expect or retrieve before the speech resolves the ambiguity. Holle and Gunter used audiovisual sentences containing ambiguous expressions and measured event-related potentials as later words supplied a disambiguating context. They observed N400 variation with gesture congruence under an explicit matching task, providing evidence that a visible iconic cue affects subsequent linguistic semantic processing Holle & Gunter (2007).
The N400 is an electrophysiological response sensitive to semantic expectancy and integration demands across many task types. An N400 difference indicates a change in neural processing under the contrast, not a direct readout of a single semantic representation or a unique anatomical localization. Care is therefore required when translating a waveform difference into claims about automaticity, knowledge representations, or a universal gesture–speech faculty. Eye movements, accuracy, response latency, and comprehension decisions supply complementary evidence about where the gesture affects interpretation Holle & Gunter (2007) Özyürek et al. (2007).
Conflict experiments also expose interpretive alternatives. A mismatching gesture can trigger surprise, draw attention, or introduce a response conflict even where the listener does not integrate it into the semantic representation. Strong studies control physical movement salience, gesture comprehensibility, task instructions, and timing relative to the target word. A graded conflict manipulation can help distinguish semantic distance from simple mismatch detection; independent measures of interpretive choice help test whether integration affects meaning rather than merely detection Kelly et al. (2010) Hostetter (2011).
Neurocognitive Evidence: Electrophysiology and Imaging
Electrophysiological studies address the timing of semantic interaction between gesture and speech. Özyürek, Willems, Kita, and Hagoort (2007) examined how semantic information from the two channels enters the unfolding comprehension process and reported event-related potential evidence for online integration. Such measurements are sensitive to the timing of processing, although the voltage observed at the scalp sums activity across distributed neural generators. The scientific question is therefore which contrast changes the time course of multimodal comprehension and under which stimulus and attentional conditions Özyürek et al. (2007).
Functional imaging complements temporal evidence by testing whether interpreting gesture and verbal content engages shared and distinct cortical regions. Willems, Özyürek, and Hagoort (2007) found that difficult integration of verbal and gestural information into prior linguistic context engaged inferior frontal regions, whereas mismatching observed actions modulated premotor areas. This pattern supports overlap in higher-order contextual integration alongside sensitivity to observed action. Hemodynamic co-activation, however, is compatible with several circuit-level computations, and anatomical overlap alone cannot establish an identical mechanism for speech and gesture Willems et al. (2007).
Neural evidence is strongest when it converges with behavioral necessity and experimentally altered context. A semantic ERP effect can appear even when the response demands are minimal; fMRI patterns can reveal areas recruited by a contrast; controlled lesions or stimulation could then identify causal contributions. Yet paradigms vary in whether they use isolated verbs, narratives, or face-to-face communication. Generalization to spontaneous interpersonal understanding must be earned through designs closer to real interaction and through replication across language communities and age groups Holler & Levinson (2019) Willems et al. (2007).
Attention, Listening Conditions, and Cue Availability
Listeners do not encounter uniformly clear speech or uniformly visible gestures. Noise, visual occlusion, divided attention, familiarity with a speaker, and knowledge of a language change how informative each signal becomes. A gesture may become especially useful when the auditory channel is uncertain, but its usefulness also depends on whether the viewer can extract an intelligible meaning from the movement. Testing this prediction requires factorial designs that vary auditory quality and gestural information separately while holding the target utterance constant Drijvers & Özyürek (2017) Drijvers & Özyürek (2018).
Drijvers and Özyürek examined iconic gestures and visible speech under clear and noise-vocoded listening conditions. Their comparison of blurred lips, visible articulatory information, and co-speech gesture was designed to separate two visual supports with partly different informational content. Follow-up EEG work comparing native and highly proficient non-native listeners found differences in N400 patterns across clear and degraded speech, demonstrating that fluency and listening demands can change the accessibility of gestural semantics. Such variation limits simple prescriptions that adding gestures always improves understanding Drijvers & Özyürek (2017) Drijvers & Özyürek (2018).
An adequate account of multimodal benefit must specify the visual field, gesture familiarity, whether the hearer expects relevant manual information, and the nature of the noise. An isolated action word with an experimentally salient iconic gesture can produce effects quite different from a dense classroom explanation with several interlocutors. Studies of ecological listening should assess both objective comprehension and the cognitive effort required to integrate cues; improved accuracy can sometimes coexist with greater effort Holler & Levinson (2019) Hostetter (2011).
Gesture, Turn-Taking, and Situated Conversation
In face-to-face interaction, gesture enters a sequential system in which speakers manage shared attention and coordinate responses. Its temporal importance extends beyond word meanings: a gesture can make an upcoming question, referent, or action projectable before the spoken utterance is complete. Holler, Kendrick, and Levinson studied questions with visible gesturing in conversational material and found faster responses when gestures accompanied questions. This supports the proposition that visible bodily information can participate in rapid conversational coordination Holler et al. (2018).
Conversation research differs from tightly scripted comprehension tests because participants construct their own utterances and their partners respond within ongoing activity. Gestures become relevant through uptake: a recipient follows a point, repairs a misunderstanding, or uses a projected referent. Gesture meaning is shaped by the task, location, prior talk, shared objects, and interactional expectations. The same physical motion may therefore carry different conversational significance depending on its position and the identities of the participants Kendon (2004) Streeck (2009).
Naturalistic analyses have considerable ecological value but provide weaker counterfactual control. Observed fast responses to gesturing questions might reflect the question type, speaker certainty, or the recipient's ability to anticipate a turn. Researchers can strengthen causal identification by pairing corpus annotation with preregistered replication, experimentally manipulating cue visibility, and matching utterance types. The resulting evidence can join conversation analysis with psycholinguistic processing without flattening situated action into a static signal-integration task Holler et al. (2018) Holler & Levinson (2019).
Quantitative Evidence and Heterogeneity of Gesture Benefits
Hostetter's (2011) meta-analysis brought together 63 samples comparing comprehension of speech alone with comprehension of speech accompanied by gesture. The pooled conclusion was a statistically significant moderate benefit for communication, with evidence that the magnitude varied by the content depicted, overlap between channels, and listener age. Gestures depicting motor actions were more useful than those concerning abstract topics; less redundant gestures had greater communicative effects; children often gained more than adults. These moderators directly challenge a universal, context-free effect Hostetter (2011).
A meta-analytic finding aggregates operationally heterogeneous studies. Comprehension may be assessed by recall, action selection, correct referent identification, or response latency, and gesture stimuli can range from spontaneous naturalistic movements to deliberately exaggerated pedagogical demonstrations. Effects also depend on whether an experimental contrast eliminates gesture altogether or changes the particular information it carries. Without reanalysis of individual effect sizes, it is inappropriate to assign a precise common effect magnitude to every communicative context Hostetter (2011).
The strongest synthesis combines meta-analytic evidence with experiments that isolate functional mechanisms. Controlled mismatch studies show that gesture can influence processing; quantitative aggregation shows a benefit across many sampled contexts; naturalistic conversational research shows that timing and interaction matter. Each supports a different inferential claim. Questions about selective reporting, between-study heterogeneity, generalization to multilingual populations, and contemporary multimodal digital settings remain active directions for evidence updating Kelly et al. (2010) Holler et al. (2018) Hostetter (2011).
Development: Gestures Before and Alongside Words
Early communicative gestures appear during the period in which children are building a spoken vocabulary and beginning to combine meanings. Pointing can identify an object while a single word predicates something about it, creating a two-element message distributed across modalities before the same child produces comparable two-word spoken combinations. Iverson and Goldin-Meadow's (2005) observational study followed ten children during this transition and found that gesture use anticipated particular lexical and combinatorial changes in speech. The association makes gesture a useful prospective marker of language development in the population studied Iverson & Goldin-Meadow (2005).
Prediction must be distinguished from causal explanation. A child's gesture–word combination may reveal a communicative intention or conceptual differentiation already developing internally. Alternatively, the gesture may elicit a more specific verbal response from caregivers, thereby altering the learning environment. Repeated measures can establish sequence, while carefully controlled intervention and analysis of caregiver responses are needed to evaluate whether modifying gesture would change subsequent vocabulary or syntax. Development is a reciprocal process involving motor maturation, social attention, linguistic input, and symbolic practice Iverson & Goldin-Meadow (2005) Goldin-Meadow & Wagner (2005).
Research on early gesture also illuminates how a child enters shared reference. The child must coordinate attention toward an object, produce a visible sign, and infer that another person will use the sign to understand the intention. Gesture thus sits at the intersection of language, joint attention, and social learning. Differences in when a child begins to point or combine modalities should be interpreted in relation to sampling, opportunity for interaction, language environment, and the conventions of the child's community rather than through a single universal timetable Rowe & Goldin-Meadow (2009a) Holler & Levinson (2019).
Longitudinal Prediction, Caregiver Input, and Social Inequality
Rowe and Goldin-Meadow (2009) followed fifty families, recording parent–child interaction at fourteen months and assessing children's vocabulary at fifty-four months. Their analysis linked socioeconomic differences in caregiver gestures to early child gesture, which in turn helped account for later vocabulary differences. This result placed manual communication within a wider social ecology of language input and opportunity. It was a longitudinal observational association, not a randomized intervention assigning families to socioeconomic conditions Rowe & Goldin-Meadow (2009b).
The study's interpretation requires attention to mediation assumptions and unmeasured confounding. Caregiver verbal input, education, time available for interaction, shared activity, and child characteristics may affect both early gesture and later vocabulary. Statistical adjustment for measured speech does not eliminate all alternatives, while the reported sample is not a population-representative census. The defensible conclusion is that early gestural communication was a meaningful predictor and candidate pathway warranting further causal investigation Rowe & Goldin-Meadow (2009b).
A related study found early gesture selectively predicted different aspects of later language. Such selectivity is theoretically informative because deictic and representational gestures may map more closely to vocabulary growth, while gesture–speech combinations may anticipate combinatorial structure. The validity of predictive markers depends on the coding of gestures, timing of follow-up, and the stability of measurement across social and linguistic settings. Educational applications should therefore combine observation with evidence from randomized instructional work Rowe & Goldin-Meadow (2009a) Iverson & Goldin-Meadow (2005) Goldin-Meadow et al. (2009).
Gesture–Speech Mismatch and Readiness to Learn
Children sometimes produce a verbal explanation and a gesture that convey different strategies. Such gesture–speech mismatches became an important developmental measure because the nonverbal component can reveal an emerging solution or representation absent from the child's explicit words. Goldin-Meadow and Singer (2003) studied how information distributed across children's hands and speech can influence adult instruction and the learning process. Gesture can thus reveal cognitive transition while also changing the feedback a learner receives Goldin-Meadow & Singer (2003) Goldin-Meadow (2003).
Mismatch is a relational coding judgment: researchers must know what problem is being solved, what each hand movement represents, and what the speech says. An apparent mismatch can stem from a coder's uncertainty, playful movement, motor noise, or an ambiguous depictive convention. Its predictive usefulness is established when mismatches discriminate later learning beyond baseline ability, task difficulty, and observable verbal knowledge. A movement's diagnostic value must be established separately for each domain and age group Goldin-Meadow & Wagner (2005) Goldin-Meadow & Singer (2003).
The mechanism also has two plausible paths. A teacher who notices a child's gesture may respond to an unspoken idea, indirectly supporting learning; the child's own gesture may reorganize attention and representation, directly supporting learning. These effects are jointly possible, and studies that manipulate student movement while controlling instruction provide stronger evidence about the direct pathway. The educational implication is to examine both what gesture reveals and what producing a particular gesture changes Goldin-Meadow et al. (2009) Novack et al. (2014).
Experimental Gesture Instruction and Conceptual Transfer
Goldin-Meadow, Cook, and Mitchell (2009) experimentally manipulated the gestures children were required to produce during mathematical instruction. Learners instructed to make correct gestures showed greater subsequent learning than groups given partially correct gestures or no gestures. The result linked the informational content of action, rather than movement alone, to acquiring and expressing new mathematical ideas. Its strongest causal inference concerns the assigned instructional movements in the specific mathematical task and population Goldin-Meadow et al. (2009).
Novack, Congdon, Hemani-Lopez, and Goldin-Meadow (2014) compared action on objects, concrete representational gesture, and abstract gesture during instruction in mathematical equivalence. The conditions supported learning on trained problems, but gesture offered an advantage on generalization problems requiring transfer. Representational distance mattered: moving hands to represent a relation may help abstract a pattern beyond the physically manipulated objects. This is evidence for a task-specific advantage of depictive action in generalization, not evidence that free gesturing invariably improves every lesson Novack et al. (2014).
The research suggests a mechanism of representational recoding: a learner's manual act selects relations, removes incidental detail, and permits those relations to be aligned with words. But differences among groups may also depend on attention, effort, teacher cues, or how explicitly the gestures designate mathematical structure. Generalization studies should assess delayed retention, transfer to distinct content, and comparability of task instructions. Controlled pedagogy shows how gesture can contribute to learning while defining the limits of the causal claim Novack et al. (2014) Novack & Goldin-Meadow (2017).
Cross-Linguistic Variation in Depiction and Event Framing
Languages select different grammatical and lexical resources for describing motion, path, manner, and event segmentation. Consequently, speakers can describe the same perceived event while placing emphasis on different semantic components. Kita and Özyürek (2003) argued from cross-linguistic comparisons that gesture's semantic organization is influenced by how spatial meaning is packaged for speech. The result supports a coordinated interface between visual–spatial thought and language-specific formulation Kita & Özyürek (2003).
Özyürek and colleagues (2008) investigated the development of language-related variation in speech and gesture for motion events in English and Turkish. Such comparisons matter because they hold broadly comparable events constant while exposing different language resources. They also distinguish age-related learning of locally conventionalized description patterns from putatively universal motor affordances. An observation about English speakers should not be promoted to a universal principle of gesture timing or meaning without considering the linguistic typology of the utterance Özyürek et al. (2008).
Culture matters beyond grammar. Gesture communities may differ in emblem conventions, norms of pointing, spatial discourse practice, and the social acceptability of particular bodily actions. Cross-cultural studies need situated examples, local annotators, translation checks, and explicit comparison of communicative functions rather than imposed shape categories. The emerging lesson is that the body supplies representational possibilities while interactional histories and linguistic practices help determine which possibilities become communicatively relevant Kendon (2004) Streeck (2009).
Multilingualism, Second-Language Use, and Accessibility
Multilingual conversation can place different demands on lexical access and on interpreting conventional or iconic movements. Language learners may depend more heavily on deictic and depictive cues when comprehension is uncertain, yet their ability to interpret a gesture depends on familiarity with its form and the context of the utterance. Gullberg and McCafferty's work on gesture and second-language acquisition highlights the need to study communicative performance as situated interaction, with speech and bodily conduct analyzed together Gullberg & McCafferty (2008) Holler & Levinson (2019).
Experiments with native and proficient non-native listeners under degraded speech show how multimodal support varies with listening circumstances. Drijvers and Özyürek (2018) observed different neural integration patterns across the groups, with gestural benefit constrained by the availability of spoken acoustic cues. This result warns against treating multilingualism as a single deficit or assuming that a visually salient movement automatically resolves uncertain auditory language. Proficiency, language pairing, task demands, and exposure all require explicit measurement Drijvers & Özyürek (2018).
Accessibility research should distinguish linguistic signs, visible articulation, captioning, spontaneous gestures, and augmentative systems, since each has different communicative structures and user needs. A signed language is a full natural language with its own grammar; gesturing alongside speech is a particular multimodal phenomenon. For inclusive experimental design, researchers should report sensory access, visual presentation conditions, and adaptations rather than excluding participants whose communicative modalities differ from the standard audiovisual laboratory setup Kendon (2004) Holler & Levinson (2019).
Prediction: Gesture Before the Spoken Word
Temporal precedence provides a more discriminating test of gesture's contribution than simultaneous compatibility alone. If a gesture is completed or becomes informative before a target word, a listener may use its depictive information to anticipate the word's meaning. Ter Bekke, Drijvers, and Holler examined the timing of gestures in conversation and reported that gestures may be available early enough to support prediction. This observation identifies an opportunity for prediction, but temporal precedence is a necessary condition rather than a sufficient demonstration of anticipatory neural processing ter Bekke et al. (2024).
Ter Bekke, Drijvers, and Holler (2025) tested anticipation with Dutch adults hearing questions from a virtual avatar. Iconic gestures or meaningless control movements preceded target words with an intervening pause. A Cloze experiment found that gestures improved explicit expectations about upcoming words; a separate EEG experiment reported changes in prestimulus alpha and beta power and reduced N400 responses to subsequent targets. The experimental ordering of gesture and word makes the evidence particularly pertinent to the distinction between advance prediction and postword integration ter Bekke et al. (2025).
Prediction nevertheless requires careful operationalization. Better guessing of a word can arise from a restricted experimental vocabulary, a highly informative prime, or familiarity with the avatar's gestural repertoire. Prestimulus oscillatory changes may accompany anticipation, attention, or preparation rather than uniquely indexing a semantic prediction. A next generation of studies should vary gesture informativeness probabilistically, test unexpected-but-compatible continuations, and track the costs of violated predictions in spontaneous dialogue ter Bekke et al. (2025) Holler & Levinson (2019).
Experimental Paradigms and Measurement Decisions
Gesture research uses three broad methodological families. Controlled comprehension experiments cross spoken content with congruent, incompatible, or unrelated movements; observational studies annotate natural interactions; developmental and educational interventions alter the gestures a participant sees or produces. These designs make different variables identifiable. Controlled contrasts can test causal effects of a visible movement, corpus observation describes real coordination, and longitudinal records establish developmental sequence and prospective association Kelly et al. (2010) Iverson & Goldin-Meadow (2005) Goldin-Meadow et al. (2009).
Stimulus design is a central measurement problem. Researchers must control gesture duration, velocity, amplitude, position, visibility, depictive conventionality, semantic informativeness, and the timing of its stroke relative to speech. A gesture may be recognizable with no sound, or may become meaningful only after hearing the corresponding verb. Participants can additionally distinguish natural movements from deliberately theatrical demonstrations. A single word–gesture stimulus set cannot establish representativeness of gestures in everyday conversations; multiple speakers and item-level random effects help guard against conclusions that depend on a few highly diagnostic clips Hostetter (2011) ter Bekke et al. (2025).
An evidentially strong analysis reports raw or modeled accuracy and latency, confidence intervals, preregistered contrast definitions, exclusions, statistical handling of participants and stimuli, and complete coding conventions. For production studies, aligned video and audio enable estimates of gesture onset, peak stroke, lexical onset, and speech prosody. For neurophysiological studies, the timing of baseline periods and artifact treatment is essential because large bodily movements can alter signal quality. In longitudinal research, baseline language ability and caregiver input should be treated as alternative explanatory variables Özyürek et al. (2007) Rowe & Goldin-Meadow (2009b).
An especially important design choice is how researchers establish that a gesture conveys a particular meaning. Researchers can obtain independent comprehensibility ratings for movements presented without speech, collect paraphrases from raters who hear only speech, and then quantify the new information available when the channels are combined. These tests separate familiar gestures from ambiguous hand movements whose interpretation depends entirely on the accompanying word. Manipulation checks can also establish whether a participant noticed a movement, how confidently it was interpreted, and whether the intended interpretation differed from the audience's actual construction. The scientific benefit of this strategy is an independently measured stimulus-level variable that can be tested against later comprehension rather than assigned after results are known Hostetter (2011) Kelly et al. (2010).
Theoretical Comparison and Falsifiable Predictions
An integrated representational account predicts content-specific effects of visible movement on interpretation even when the gesture is task-irrelevant. A communication-centered account predicts stronger gesture production and listener use when visible access and common ground make the gesture useful. A simulated-action account expects depictive movement to be related to sensorimotor simulation and to vary with the action content of the message. A language-packaging account predicts language-specific patterns of gesture segmentation and depiction. These positions are distinguishable most clearly when a single experiment varies more than one of the proposed causal drivers McNeill (1992) Hostetter & Alibali (2008) Kita & Özyürek (2003).
Experiments should cross communicative intent with visibility, linguistic encoding with movement possibilities, and perceptual prediction with semantic informativeness. For example, if a gesture facilitates a target word while a speaker is absent, that supports a comprehension effect but cannot establish that the original speaker intended the hand movement for this listener. If constraining movement changes later explanation accuracy, that may reflect functional cognitive support, attentional diversion, or frustration. A theory gains explanatory value by forecasting which conditions reverse or eliminate a benefit rather than describing every movement as evidence of integration Kelly et al. (2010) Goldin-Meadow et al. (2009).
Recent accounts also consider gestures in signed and spoken language together. Kita and Emmorey (2023) surveyed their relations to cognition and identified opportunities to connect representational action, conventional linguistic structure, and modality. The comparative program demands disciplined category definitions: a signed lexical form participates in the grammar of a language, while a spontaneous depictive movement accompanying speech may be organized differently. Treating both as visible language behavior can generate comparative hypotheses without collapsing the distinct objects Kita & Emmorey (2023).
Evidence Quality, Replication, and Generalizability
The research literature supports several converging findings: gestures can change immediate comprehension; they can precede and predict developmental vocabulary changes; instructionally relevant movements can improve some forms of learning and transfer; and visible cues can assist conversational anticipation. Confidence differs by proposition. Experimental cue manipulations identify effects under tested conditions, longitudinal prediction leaves causal mediation less certain, and theoretical proposals about imagery or memory mechanisms remain explanations to be tested rather than directly measured outcomes Kelly et al. (2010) Rowe & Goldin-Meadow (2009b) Novack et al. (2014).
Replication requires more than recovering a significant difference with the same videos. Independent studies should vary languages, narrators, cultural groups, age ranges, hearing conditions, gesture forms, stimulus sampling, and comprehension tasks while preserving the causal contrast. Publication bias is especially relevant to studies with many possible gesture categories and outcomes. Hostetter's quantitative review established a broad benefit across older studies, yet the field needs transparent updates that assess study-level bias, heterogeneity, and the modern range of conversational and educational uses Hostetter (2011) Holler & Levinson (2019).
One prominent source of overgeneralization is a leap from group differences to stable traits of individuals. A laboratory response to a scripted iconic gesture cannot reveal a participant's general intelligence, personality, or communicative capacity. Likewise, an association between caregiver gesture and child vocabulary does not establish that low gesture frequency causes language difficulty. Conclusions should state the manipulated variable, population and task, temporal scale, and quality of the counterfactual before discussing possible applications Rowe & Goldin-Meadow (2009b) Novack & Goldin-Meadow (2017).
The inferential reach of developmental findings also depends on temporal ordering. If early gesture precedes measured vocabulary, the observation supports temporal prediction, yet a reciprocal developmental process may generate both changes. Cross-lagged analyses with multiple assessments and independently measured caregiver responses offer more information about possible feedback loops, though causal assumptions remain demanding. Instructional experiments can complement such models by randomizing the gestural information produced or observed, provided assignment, teacher behavior, and additional practice are controlled. Integrating these methods could reveal whether language development depends primarily on social response, internal representational activity, or their repeated interaction across learning episodes Iverson & Goldin-Meadow (2005) Rowe & Goldin-Meadow (2009b) Goldin-Meadow et al. (2009).
Attention, Memory, and Candidate Learning Mechanisms
Gesture may influence learning through a sequence of mechanisms rather than a single added information channel. Depiction attracts attention to a relation, directs the learner's selection of task features, can be rehearsed or reconstructed, and may change how a spoken explanation is encoded and retrieved. The crucial distinction is between immediate acquisition, delayed memory retention, and transfer to structurally related new problems. An intervention that increases short-term accuracy but leaves delayed transfer unchanged supports a narrower claim than one that benefits flexible abstraction Novack et al. (2014) Goldin-Meadow et al. (2009).
Guo, Rosenberg, Bainbridge, and Goldin-Meadow (2025) proposed a framework connecting gesture's learning effects to bottom-up and top-down attention and memory processes. Their review positions these pathways as candidates for experimental tests. It does not establish a single validated neural mechanism for all gestural benefit. A rigorous intervention can independently manipulate which relation a movement highlights and whether learners must produce or simply watch it, then test attention during exposure and retention at delayed intervals Guo et al. (2025).
Representational action also varies in its degree of abstraction. Acting directly on objects may preserve many incidental physical features, whereas a gesture representing a relation can suppress those specifics. The transfer advantage reported by Novack and colleagues is consistent with this abstraction account, although the mechanism would be strengthened by direct tests of attentional allocation and memory organization. Combined behavioral and eye-tracking measures could establish which intermediate processes account for the observed learning differences Novack et al. (2014) Guo et al. (2025).
Scientific Boundaries: Visual Speech, Sign Languages, and Gesture
Multimodal gesture and language is contiguous with, but scientifically distinguishable from, visual phonetic integration. Visible mouth movements and speech sounds may combine at phonetic or lexical stages; iconic hand gestures more often depict actions, referents, discourse relations, or event structure. The classic McGurk and MacDonald demonstration of interacting heard and seen syllables addresses an audiovisual speech-perception problem. Its phenomena should therefore be interpreted as a neighboring comparison, not evidence that a particular co-speech gesture produces the same effect McGurk & MacDonald (1976) Kelly et al. (2010).
Signed languages offer a second crucial comparison. Signed linguistic expressions are organized through manual and nonmanual phonological, morphological, syntactic, and discourse systems. Co-speech gestures can share modality and representational affordances with signs, yet their relation to concurrent spoken language differs. Comparative studies can examine iconicity, temporal coordination, spatial reference, and embodied cognition while documenting whether a movement is a lexical sign, a depictive construction, a conventional emblem, or a spontaneous gesture in a specific communicative system Kita & Emmorey (2023) Kendon (2004).
These boundaries sharpen the research object. A full theory of language in face-to-face settings must eventually connect speech, gesture, gaze, posture, conventional signs, and interactional context, but each component has its own methods and explanatory levels. This dossier investigates how gesture and linguistic meaning are coordinated and how visible action can alter language processing, learning, and discourse; work on elementary audiovisual binding and the autonomous grammar of signed languages remains a complementary research program Holler & Levinson (2019) Kita & Emmorey (2023).
Applications in Education and Public Communication
Gestures can make instruction more informative when they depict task-relevant relationships that would otherwise remain implicit in words. Experimental evidence in children's mathematical reasoning shows that systematically taught movements may support abstraction and transfer, while developmental studies show that learners' spontaneous movements can communicate strategies unavailable in their verbal explanations. Educational use should therefore distinguish a diagnostic interpretation of a student's gesture from an intervention that causes learning by assigning a gesture. These are separate claims supported by different kinds of data Goldin-Meadow & Singer (2003) Novack et al. (2014).
Designers of educational communication should assess whether a gesture helps the particular learner identify a relation, whether the movement is culturally and linguistically interpretable, and whether it imposes a competing attention demand. A lecturer's illustrative gesture, a screen-based avatar movement, and a child's embodied reconstruction of a solution are not interchangeable interventions. Randomized studies can compare observing gestures, producing gestures, acting on physical objects, and hearing speech alone, with delayed comprehension and transfer measured independently Goldin-Meadow et al. (2009) Guo et al. (2025).
Research on public communication also has implications for communicative accessibility. The visibility of gesture, the availability of clear audio, and the linguistic background of an audience shape the usefulness of multimodal cues. Captions or visual speech can support access by different routes than spontaneous hand movement. Inclusive design should provide intelligible linguistic information through appropriate channels and treat gesture as one possible aid, not as a universal substitute for accessible speech or signed language Drijvers & Özyürek (2017) Kita & Emmorey (2023).
Human–Artificial Multimodal Communication in the Artificial Era
The Artificial Era introduces mediated settings in which human speech is accompanied, interpreted, or generated by artificial visible agents. Digital assistants, avatars, and embodied robots may produce pointing or depictive motion while supplying linguistic instructions, and sensor-equipped systems may recognize some forms of human gesture. The scientific bridge to established gesture research lies in measurable variables: semantic alignment, gesture timing, prediction of upcoming speech, communicative uptake, and effects on comprehension or task performance. No theoretical designation of an artificial system's psychological status is established by a gesture–speech compatibility score Holler & Levinson (2019) ter Bekke et al. (2025).
A controlled artificial-agent experiment should compare the same linguistic message delivered with a congruent gesture, an irrelevant movement, an incongruent gesture, and no visible movement. Further factorial conditions can manipulate whether the movement precedes the word, whether the agent is described as responsive, and whether the user can correct its interpretation. Primary outcomes include intended referent identification, explanation accuracy, repair effort, calibrated confidence, and delayed learning. Such tests would identify the behavioral consequences of specific design choices while resisting an inference from fluid multimodal performance to unmeasured internal experience Kelly et al. (2010) ter Bekke et al. (2025).
Human–artificial interaction also creates a provenance problem: a generated gesture may be semantically plausible while the system's underlying interpretation of a user's intent remains erroneous. A gestural output can therefore influence trust independently of informational quality. Scientific evaluation should retain the recorded dialogue, the agent's visible movement stream, decision timings, task ground truth, and correction episodes. These observations can support testable accounts of successful and failed cognitive partnership without treating the existence of a humanoid form as a psychological explanation Holler & Levinson (2019) Guo et al. (2025).
Evidence Synthesis: What the Findings Jointly Establish
Across experimental comprehension studies, iconic and referential gestures supply information capable of changing the interpretation and processing of concurrent speech. Controlled congruence effects, neural responses to semantic mismatch, and the more recent separation of prestimulus anticipation from subsequent word integration converge on a temporally organized multimodal system. They do not determine a single mandatory mechanism for all communicative situations, because attention, movement meaning, expectancy, and social interaction vary across paradigms Kelly et al. (2010) Özyürek et al. (2007) ter Bekke et al. (2025).
Across developmental and instructional studies, gesture serves both as evidence of emerging linguistic or conceptual knowledge and as a manipulable component of some learning environments. Early gesture–word combinations predict later spoken combinations, and caregiver gestures predict aspects of children's later vocabulary, while experimentally assigned educational movements can affect learning. The evidential sequence must be preserved: longitudinal prediction supports potential markers and pathways; randomized instructional assignment supports causal effects under the intervention's exact conditions Iverson & Goldin-Meadow (2005) Rowe & Goldin-Meadow (2009b) Goldin-Meadow et al. (2009).
Across theoretical and cross-linguistic investigations, speech and gesture display partial semantic coordination while remaining sensitive to the grammatical and interactional structures through which people express events. The strongest general account thus treats multimodal communication as dynamically constrained by conceptual content, linguistic formulation, bodily possibilities, interlocutor expectations, and local conventions. This synthesis integrates multiple research traditions; it does not nominate one as an empirically settled universal explanation Kita & Özyürek (2003) Hostetter & Alibali (2008) Kendon (2004).
Open Problems and a Cumulative Research Agenda
A first unresolved issue concerns the causal sequence between gesture production and thought. Does a movement directly stabilize a representation, does it merely reveal a concept already prepared, or does it recruit a response from an interlocutor that improves later reasoning? Orthogonal manipulations of physical movement, visible communicative intent, verbal formulation, and receiver feedback would sharpen the alternatives. Delayed learning tests and direct measures of attention and recall would help arbitrate between candidate pathways Hostetter & Alibali (2019) Guo et al. (2025).
A second issue concerns generalizability from edited videos, experimental avatars, and laboratory lexical decisions to spontaneous social interaction. Natural conversation contains unpredictable turn transitions, moving objects, variable visibility, and participant-specific conventions. Multisite data sets with registered annotation protocols, item and participant random effects, language-diverse samples, and archived synchronized video and speech can make comparisons more robust. Research ethics must address the identifiability of visible human faces and the permissions required to share interaction recordings Holler et al. (2018) ter Bekke et al. (2025).
A third issue is the scientific status of multimodal integration models. Theories need explicit intermediate variables, quantitative predictions for degraded cues, thresholds for gesture informativeness, and well-designed counterexamples. Research should report situations in which gestures distract, convey misleading information, or fail to improve communication, rather than sampling only beneficial configurations. Comparative work with signed language, human–human dialogue, and human–artificial communication should preserve the distinct representational and social characteristics of each system Hostetter (2011) Kita & Emmorey (2023).
Conclusion
Multimodal gesture and language is a coherent field of psychological research devoted to the organization of meaning across visible action and linguistic expression. Its object extends from the selection and timing of a depictive hand movement to the interpretation of spoken language, early communication, conversational coordination, and learned conceptual relations. A gesture can convey information not expressed verbally, alter an addressee's expectation before a word arrives, and influence what is remembered or learned. These findings rest on multiple independent methods rather than on a single definition of gesture Kelly et al. (2010) ter Bekke et al. (2025) Hostetter (2011).
The evidence establishes the importance of semantic relation, temporal position, audience, language, and learning context. Controlled experiments identify momentary effects of congruent or contrasting movements; longitudinal studies identify predictors and developmental associations; teaching interventions identify specific causal learning advantages; electrophysiological and imaging studies constrain the timing and neural organization of interpretation. Their findings are most informative when reported with their populations, controls, measured outcomes, and alternative explanations intact Özyürek et al. (2007) Rowe & Goldin-Meadow (2009b) Novack et al. (2014).
The next fundamental step is a cumulative account of how visible communicative action participates in flexible, situated understanding across human languages, developmental stages, and new mediated environments. That account will require shared annotations, causal paradigms capable of separating prediction from postword integration, precise models of attention and memory, and meaningful tests across cultures and artificial interlocutors. The established scientific conclusion is that language understanding and use frequently depend on coordinated information distributed through bodies, speech, and the shared scene Holler & Levinson (2019) Guo et al. (2025) Kita & Emmorey (2023).
References
- Drijvers, L., & Özyürek, A. (2017). Visual context enhanced: The joint contribution of iconic gestures and visible speech to degraded speech comprehension. Journal of Speech, Language, and Hearing Research, 60(1), 212–222. https://doi.org/10.1044/2016_JSLHR-H-16-0101
- Drijvers, L., & Özyürek, A. (2018). Native language status of the listener modulates the neural integration of speech and iconic gestures in clear and adverse listening conditions. Brain and Language, 177–178, 7–17. https://doi.org/10.1016/j.bandl.2018.01.003
- Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Belknap Press of Harvard University Press. https://doi.org/10.2307/j.ctv1w9m9ds
- Goldin-Meadow, S., & Singer, M. A. (2003). From children's hands to adults' ears: Gesture's role in the learning process. Developmental Psychology, 39(3), 509–520. https://doi.org/10.1037/0012-1649.39.3.509
- Goldin-Meadow, S., & Wagner, S. M. (2005). How our hands help us learn. Trends in Cognitive Sciences, 9(5), 234–241. https://doi.org/10.1016/j.tics.2005.03.006
- Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. A. (2009). Gesturing gives children new ideas about math. Psychological Science, 20(3), 267–272. https://doi.org/10.1111/j.1467-9280.2009.02297.x
- Gullberg, M., & McCafferty, S. G. (2008). Introduction to gesture and SLA: Toward an integrated approach. Studies in Second Language Acquisition, 30(2), 133–146. https://doi.org/10.1017/S0272263108080285
- Guo, X. (H.), Rosenberg, M. D., Bainbridge, W. A., & Goldin-Meadow, S. (2025). How gesture benefits learning: A working framework for examining attention and memory mechanisms. Annual Review of Developmental Psychology, 7, 41–63. https://doi.org/10.1146/annurev-devpsych-111323-112334
- Holle, H., & Gunter, T. C. (2007). The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience, 19(7), 1175–1192. https://doi.org/10.1162/jocn.2007.19.7.1175
- Holler, J., & Levinson, S. C. (2019). Multimodal language processing in human communication. Trends in Cognitive Sciences, 23(8), 639–652. https://doi.org/10.1016/j.tics.2019.05.006
- Holler, J., Kendrick, K. H., & Levinson, S. C. (2018). Processing language in face-to-face conversation: Questions with gestures get faster responses. Psychonomic Bulletin & Review, 25(5), 1900–1908. https://doi.org/10.3758/s13423-017-1363-z
- Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2), 297–315. https://doi.org/10.1037/a0022128
- Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15(3), 495–514. https://doi.org/10.3758/PBR.15.3.495
- Hostetter, A. B., & Alibali, M. W. (2019). Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review, 26(3), 721–752. https://doi.org/10.3758/s13423-018-1548-0
- Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language development. Psychological Science, 16(5), 367–371. https://doi.org/10.1111/j.0956-7976.2005.01542.x
- Kelly, S. D., Özyürek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21(2), 260–267. https://doi.org/10.1177/0956797609357327
- Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge University Press. https://doi.org/10.1017/CBO9780511807572
- Kita, S., & Emmorey, K. (2023). Gesture links language and cognition for spoken and signed languages. Nature Reviews Psychology, 2(7), 407–420. https://doi.org/10.1038/s44159-023-00186-9
- Kita, S., & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48(1), 16–32. https://doi.org/10.1016/S0749-596X(02)00505-3
- McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746–748. https://doi.org/10.1038/264746a0
- McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press. https://press.uchicago.edu/ucp/books/book/chicago/H/bo3641188.html
- Novack, M. A., & Goldin-Meadow, S. (2017). Gesture as representational action: A paper about function. Psychonomic Bulletin & Review, 24(3), 652–665. https://doi.org/10.3758/s13423-016-1145-z
- Novack, M. A., Congdon, E. L., Hemani-Lopez, N., & Goldin-Meadow, S. (2014). From action to abstraction: Using the hands to learn math. Psychological Science, 25(4), 903–910. https://doi.org/10.1177/0956797613518351
- Özyürek, A., Kita, S., Allen, S., Brown, A., Furman, R., & Ishizuka, T. (2008). Development of cross-linguistic variation in speech and gesture: Motion events in English and Turkish. Developmental Psychology, 44(4), 1040–1054. https://doi.org/10.1037/0012-1649.44.4.1040
- Özyürek, A., Willems, R. M., Kita, S., & Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19(4), 605–616. https://doi.org/10.1162/jocn.2007.19.4.605
- Pouw, W. T. J. L., de Nooijer, J. A., van Gog, T., Zwaan, R. A., & Paas, F. (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Frontiers in Psychology, 5, Article 359. https://doi.org/10.3389/fpsyg.2014.00359
- Rowe, M. L., & Goldin-Meadow, S. (2009). Differences in early gesture explain SES disparities in child vocabulary size at school entry. Science, 323(5916), 951–953. https://doi.org/10.1126/science.1167025
- Rowe, M. L., & Goldin-Meadow, S. (2009). Early gesture selectively predicts later language learning. Developmental Science, 12(1), 182–187. https://doi.org/10.1111/j.1467-7687.2008.00764.x
- Streeck, J. (2009). Gesturecraft: The manu-facture of meaning. John Benjamins Publishing Company. https://doi.org/10.1075/gs.2
- ter Bekke, M., Drijvers, L., & Holler, J. (2024). Hand gestures have predictive potential during conversation: An investigation of the timing of gestures in relation to speech. Cognitive Science, 48(1), Article e13407. https://doi.org/10.1111/cogs.13407
- ter Bekke, M., Drijvers, L., & Holler, J. (2025). Co-speech hand gestures are used to predict upcoming meaning. Psychological Science, 36(4), 237–248. https://doi.org/10.1177/09567976251331041
- Willems, R. M., Özyürek, A., & Hagoort, P. (2007). When language meets action: The neural integration of gesture and speech. Cerebral Cortex, 17(10), 2322–2333. https://doi.org/10.1093/cercor/bhl141