Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

Emotional Expression and Emotion Recognition

Abstract. Emotional expression and emotion recognition are coupled processes through which visible and audible behavior becomes evidence for social judgments about affective states. This critical narrative monograph analyzes the historical study of expression, definitions of expression and inference, basic-emotion and contextual theories, facial and vocal cues, nonverbal vocalizations, body movements, spontaneous and posed displays, cross-cultural regularities, development, social relationships, and multimodal integration. Original experiments and published quantitative syntheses are examined for construct validity, sampling, causal inference, and ecological generalization. Evidence supports informative but context-dependent patterns of emotional signaling and substantial variation in the relation between observable behavior, observer judgments, and a sender's experienced emotion. The monograph concludes with methodological requirements for future multimodal research and artificial emotion-recognition systems. It presents a critical narrative synthesis, not an original experiment, systematic review, or meta-analysis.

Keywords: emotional expression; emotion recognition; facial expression; affective science; emotional prosody; nonverbal communication; context effects; multimodal emotion recognition; cultural variation; emotion inference; expressive behavior; affective computing

Introduction: Expressions, Observers, and Emotional Inference

Emotional expression is the production of facial, vocal, bodily, and interactional behavior that may communicate an affectively significant condition. Emotion recognition is an observer's attribution of emotion on the basis of available evidence, expectations, and categories. These processes are causally related in some circumstances, yet they need separate operational definitions: a moving face is an observable behavior, whereas anger, grief, relief, and amusement designate psychological interpretations that may require additional evidence. A theory of expression must explain production; a theory of recognition must explain decoding, inference, and error. Barrett et al. (2019) Hall et al. (2019)

The object of this research dossier is the relation between expressive behavior, available cues, perceiver knowledge, and warranted inference about another person's state. It encompasses the face, prosody, nonverbal vocalizations, posture, gesture, social situation, cultural conventions, and their temporal coordination. It asks how displays arise, when observers agree, when they are accurate, how laboratory formats alter answers, and which assumptions survive tests with naturally occurring behavior. Emotion categories as a general ontology, clinical diagnosis, and linguistic pragmatics remain adjacent scientific subjects rather than substitutes for this object. Scherer (2003) Goel et al. (2024)

Three distinctions organize the inquiry. Production and perception represent different measurable processes. Recognition accuracy must be distinguished from agreement with a researcher-supplied label. Classification of a picture must be separated from inference about a living person in a changing event. These distinctions make apparent disagreements between schools more tractable, because one experiment may measure discrimination among stereotyped portrayals while another examines whether facial movements predict self-reported experience in uncontrolled circumstances. Both can produce valuable results without estimating the same quantity. Russell (1994) Durán & Fernández-Dols (2021)

This work is a critical narrative synthesis of original experimental reports, historical theoretical papers, methodological analysis, and published quantitative reviews. It documents what each design can establish and where rival interpretations remain plausible. Sources are evaluated at the level of task, stimulus construction, population, reference criterion, and generalization beyond the sample. No original participant study, preregistered systematic-review search, or de novo meta-analysis is claimed. The result is a structured account intended for affective science, psychology education, social interaction research, and the assessment of machine-based emotion inference. Mauss & Robinson (2009) Steward et al. (2025)

Scientific Definitions and the Problem of Ground Truth

An expressive cue is a perceivable feature such as lip-corner motion, brow configuration, speech rhythm, vocal pitch, tears, or body orientation. A display is a temporally assembled configuration of cues. A recognition judgment is a response assigning an emotion word, valence estimate, intensity, or probability to a target. The emotional condition of the target is a separate construct that can be sampled through contemporaneous report, event history, behavioral consequences, physiological measures, or combinations of indicators. The reliability and validity of those measurements determine which accuracy claims the investigator can make. Hall et al. (2019) Mauss & Robinson (2009)

Ground truth differs across experimental tasks. In a posed-photo classification experiment, the nominal correct answer may simply be the expression the actor was instructed to portray. In a spontaneous-expression study, a participant's timely self-report may serve as the criterion, although reports depend on language, introspective access, and demand characteristics. In a social inference task, agreement about what a protagonist probably feels may be measured without observing that protagonist's private experience. Each criterion defines a different target, so apparent success in one task cannot automatically validate another. Barrett et al. (2019) Goel et al. (2024)

Reliability refers to consistency among judgments, instruments, or repeated observations. Validity concerns whether the recorded score represents the claimed construct. High interrater agreement demonstrates a shared interpretation of cues and category labels; the agreement may reflect common cultural learning or stimulus selection. Predictive validity requires a separate criterion of the relevant real-world state, while causal validity requires evidence linking an intervention or mechanism to changed expression or inference. A comprehensive account must describe these evidentiary layers explicitly before using the word accurate. Russell (1994) Elfenbein & Ambady (2002)

Emotion recognition can be framed as probabilistic inference: a perceiver estimates the relative plausibility of candidate states given visible and audible behavior, context, personal history, and prior knowledge. This framing identifies uncertainty as a property of evidence rather than a defect to be hidden by categorical output. It also emphasizes that the receiver's decision may be well calibrated even when no single label is warranted. The critical question is what evidence supports a probability, what population supplied it, and whether the conditional relation remains stable outside the sampled situations. Goel et al. (2024) Zaki et al. (2008)

Historical Formation of Expression Research

Modern investigation of expression developed from nineteenth-century comparative descriptions of visible behavior, bodily accounts of emotion, and later programs of experimental photography and observation. Charles Darwin's 1872 discussion of expression in humans and other animals placed continuity, inherited tendencies, habits, and communicative appearance within one biological argument. The book was historically significant as a program for comparative questions, even though its examples and causal interpretations cannot be treated as contemporary controlled evidence. Later physiology and psychology transformed the explanatory question into the relationship between eliciting events, bodily activity, facial actions, and emotional experience. Keltner et al. (2019) Barrett et al. (2019)

During the twentieth century, investigators developed standard photographs and naming procedures to ask whether observers in different societies associate similar facial appearances with similar emotion categories. Ekman and Friesen's 1971 cross-cultural experiment became central evidence for regularities in identifying portrayals of particular emotions. Subsequent programs extended the catalog of displays and explored proposed mechanisms. The enduring importance of these studies is methodological as well as theoretical: the standardized stimulus made comparison feasible, while the restricted response task raised questions about language, category prompting, and ecological representativeness. Ekman & Friesen (1971) Russell (1994)

Historical controversies soon turned toward the meaning of recognition. Were researchers finding culturally general production of particular facial movements, shared perceptual discrimination, familiar conceptual labels, or a combination? Russell's 1994 methodological review asked whether findings from forced-choice recognition tasks established universality under more natural conditions. Later experiments with remote populations and dynamic visual generation directly varied conceptual and cultural circumstances. The historical movement was consequently toward more precise questions, in which regularities can be tested without presupposing an invariant one-to-one dictionary of faces and feelings. Russell (1994) Jack et al. (2012) Gendron et al. (2014b)

Contemporary work links social perception, behavioral science, psychometrics, neuroscience, and computer vision. Expression studies increasingly compare naturally produced and acted signals; decoding work manipulates situation and person knowledge; cross-cultural projects examine tasks that do not supply Western emotion labels; and meta-analysis estimates the influence of design differences. The 2019 synthesis of facial-inference evidence by Barrett and colleagues and the basic-emotion update by Keltner and colleagues show that scientific disagreement is now focused on the specificity, variability, functions, and inferential value of expressive movements. Barrett et al. (2019) Keltner et al. (2019)

Observable Signals and Measurement Systems

Facial movement is commonly quantified in terms of anatomically interpretable muscle-related actions, their onset, amplitude, duration, and combinations. The measurement of an action and the designation of an emotion category are distinct operations: a contraction pattern can be documented without presuming that it means anger or fear. Video annotation, human coding, and computer vision can extract candidate features, but each relies on lighting, camera geometry, training exemplars, and coding conventions. Temporal movement information may discriminate actions lost in a single still frame. Barrett et al. (2019) Keltner et al. (2019)

Vocal expression involves fundamental frequency, amplitude, spectral properties, speech timing, articulation, breathiness, and other acoustic measures. A speaker's pitch can change with activation, conversational intent, language, and individual vocal physiology. A valid vocal inference therefore distinguishes physiological production, acoustic transmission, and the listener's interpretive response. Scherer's Brunswikian lens approach remains valuable because it makes separate links between an emotional condition, observable vocal cues, and perceiver judgments, thereby allowing failures of production and decoding to be investigated separately. Scherer (2003)

Body expression includes posture, head movement, gait, orientation, approach and withdrawal, speed of action, and interpersonal distance. These features often emerge as dynamic configurations rather than independent tokens. An observer may recognize urgency, reluctance, confidence, or distress through whole-body motion even when a cropped face provides limited information. Valid measurement requires specification of the act and the contextual alternatives that produce similar geometry, including physical effort, social role, and camera perspective. Aviezer et al. (2012) Hall et al. (2019)

Every signal source carries measurement noise. Image-based studies must consider actor selection, posing instructions, digital alteration, frame sampling, and demographic composition. Acoustic studies require language and recording controls. Human annotation must test coder agreement without confusing it with affective validity. Automated systems require separate evaluation on held-out people, contexts, devices, and populations. A sufficiently rigorous expression instrument reports what was observed, which category was imposed, and which inferential step converts the one into the other. Bänziger et al. (2009) Poria et al. (2017)

Basic-Emotion and Communicative-Function Accounts

Basic-emotion theories propose that evolutionarily important affective responses display structured regularities in physiology, behavior, subjective experience, and sometimes expression. Within that tradition, expressive patterns may convey information relevant to danger, affiliation, submission, disgust, or social repair. Ekman's 1992 argument presented differentiated emotional families as meaningful organizing categories; later theoretical developments recognize additional expressive states and interactional functions. The empirical question concerns the stability and specificity of particular links, including variation across situation, individual, and developmental history. Ekman (1992) Keltner et al. (2019)

Evidence for recurrent configurations is important without implying that every instance of an emotion produces one complete stereotyped face. Displays can differ by intensity, communicative goal, and social audience. Smiles can serve several affiliative functions, while widened eyes can occur during orienting and surprise-like events. An evolutionary explanation needs to specify the environment, function, and behavioral mechanism that could favor a signal, along with the observable conditions that would distinguish that account from learned conventional displays. Keltner et al. (2019) Barrett et al. (2019)

Within contemporary accounts, expression may both reveal a producer's condition and change another person's behavior. A signal that obtains help, warns of danger, negotiates social rank, or invites cooperation can have consequences independently of whether it maps uniquely onto a private feeling. Functional interpretations can be tested through audience effects, interaction sequences, observer behavior, and cross-context variation. The resulting unit of analysis often expands from a facial configuration to a relationship in which sender and receiver influence one another over time. Hall et al. (2019) Hess & Fischer (2013)

A useful comparison separates claims of common perceptual tendencies from claims of universal categorical production. Cross-cultural above-chance classification can support the first claim under specified test conditions; expression–experience co-occurrence studies inform the second. Their results must be synthesized rather than substituted for one another. The review by Keltner and colleagues develops the case for rich expressive repertoires, while Barrett and colleagues examine the limits of inferring individual emotion from a facial movement alone. Both perspectives demand careful investigation of task design and context. Elfenbein & Ambady (2002) Barrett et al. (2019)

Dimensional, Constructed, and Contextual Models

Dimensional approaches describe affective variation using coordinates such as pleasantness and activation. For expression research, these dimensions provide an alternative target to discrete emotion labels: a voice may communicate high activation more consistently than it identifies a specific reason for that activation. Category recognition and dimensional judgment need separate benchmarks, because a stimulus can be clear in valence and uncertain in its categorical interpretation. Dimensional measurement also allows gradations and mixtures that forced categorical selection can hide. Russell (1994) Cowen & Keltner (2017)

Constructed-emotion perspectives emphasize the role of concepts, prior learning, bodily information, and situated interpretation in making sense of experience and expression. Their relevance to recognition is especially concrete: perceptual categories can depend on linguistic prompts and local cultural practices. An observer's judgment involves inference about an action or situation as well as visual feature detection. These models can be tested by manipulating category availability, contextual descriptions, and task instructions while retaining identical physical stimuli. Barrett et al. (2019) Gendron et al. (2014b)

Appraisal-oriented approaches relate emotions to evaluations of goals, agency, certainty, novelty, control, and anticipated outcomes. The same facial movement may be interpreted differently when the surrounding event implies triumph, social embarrassment, frustration, or relief. Context-sensitive recognition is therefore compatible with models in which an expression communicates only some aspects of the emotional condition while circumstances determine its full interpretation. This arrangement motivates cross-modal designs that independently alter facial behavior and event information. Goel et al. (2024) Scherer (2003)

Competing models should be compared through discriminating predictions. A fixed-label account predicts robust emotion-specific decoding across unfamiliar people and circumstances; a dimensional account predicts more stable broad affective judgments than precise categories; a contextual inference model predicts substantial changes in attribution after situational information changes. No single study exhausts these alternatives, and mixed outcomes are scientifically informative. The enduring challenge is to construct experiments that separate genuinely informative movement patterns from category instructions and the observer's expectations. Jack et al. (2012) Steward et al. (2025)

Facial Movement, Configuration, and Temporal Dynamics

The human face supports a wide range of visible actions that can accompany communication, sensory orienting, attention, and emotion-related change. Researchers study the eyebrows, eyelids, mouth, jaw, nose, and coordinated movement across facial regions, together with timing and intensity. A face is a moving biomechanical system embedded in communicative activity, so an isolated photograph loses information about onset, sequence, synchronization, and offset. Facial recognition research must identify which aspects of a moving pattern carry reliable information and which arise from ordinary speech or social conventions. Barrett et al. (2019) Keltner et al. (2019)

Statistical regularities in expression can be studied by comparing facial features with independently measured experiences and circumstances. Their interpretation depends on the heterogeneity of eliciting events and people: a single smile may accompany amusement, politeness, reassurance, or discomfort, and a scowl may emerge from concentration or bright sunlight. The appropriate analysis estimates conditional associations, rather than assigning a unique psychological cause to a movement. Accuracy of a static facial classifier is thus only one component of a scientifically valid inference system. Durán & Fernández-Dols (2021) Hall et al. (2019)

Motion provides information about intentional control and interaction. A facial change can develop in anticipation of an audience, during a conversational turn, or following a shift in social attention. An experimental sequence can compare the same movement in isolation, within a continuous video, and within the actual event that elicited it. This isolates the contributions of kinematics, person identity, and social meaning. It also tests whether apparent categorical recognition in photographs survives the more complex conditions of ordinary encounters. Goel et al. (2024) Steward et al. (2025)

Individual morphological differences, lighting, camera angles, cultural display practices, and observers' expectations further modify facial perception. Researchers who pool people without examining these sources of variance risk attributing a property of the stimulus collection to emotion generally. Robust science therefore requires multiple independent senders, recordings across activities, and stratified estimates of performance. Interpretability improves when human observers and algorithms are evaluated against the same clearly defined reference outcome, while their cue sensitivities are measured separately. Barrett et al. (2019) Poria et al. (2017)

Spontaneous Expression and Deliberately Posed Displays

Posed expressions are elicited by instructions such as acting surprised, frightened, or angry. They provide experimental control and recognizable exemplars, but their relation to spontaneous emotional events must be established. The actor's intended category may be known even when the actor does not experience that emotion. A classifier trained on posed stimuli may primarily identify culturally familiar conventions of portrayal. This is a legitimate perceptual task whose findings must be described using the task's actual target, not generalized as proof of access to another person's feeling. Russell (1994) Barrett et al. (2019)

Spontaneous displays introduce other problems. A laboratory film, game, or social encounter may elicit mixed states, changes of attention, conversational self-presentation, and behavioral regulation. The emotional experience must be indexed without interfering excessively with the expressive process. Repeated reports, event coding, independent physiological measures, and contextual information can be combined while preserving each indicator's limits. A useful sample includes both emotional and ordinary events, because classification among carefully selected emotion trials alone can exaggerate performance under realistic base rates. Mauss & Robinson (2009) Durán & Fernández-Dols (2021)

Meta-analytic co-occurrence evidence asks a different question than photograph recognition: how reliably does a named emotion accompany its theoretically predicted facial action? Durán and Fernández-Dols evaluated this co-occurrence literature and challenged assumptions about consistent one-to-one correspondence. Their work motivates precision about temporal windows, expression definitions, emotional criterion measures, and publication selection. A low co-occurrence estimate cannot show that expressive behavior never communicates emotion; it identifies limits of the hypothesized regularity under the assembled designs. Durán & Fernández-Dols (2021)

Strong protocols compare deliberately posed, induced, and naturalistic events with common recording and analysis standards. They document the strength of emotion induction, the presence of an audience, the participant's awareness of cameras, and expressive inhibition or exaggeration. Cross-condition findings then identify which signals are robust and which depend on social performance. This design makes the production problem experimentally visible and connects it to the observer's subsequent judgments without assuming that the two stages are equivalent. Hall et al. (2019) Keltner et al. (2019)

The Evidence on Expression–Emotion Correspondence

Expression–emotion correspondence is the association between a measurable display and a separately identified emotional condition. It can be evaluated in both directions: given an emotion, how often does a display occur, and given a display, how likely is the emotional condition? The two conditional probabilities differ whenever expression prevalence, situational context, and alternative causes differ. A common conceptual error treats their interchangeability as self-evident. The scientific task is to estimate them with representative denominators, clear temporal criteria, and evidence about other events that produce similar movements. Durán & Fernández-Dols (2021) Barrett et al. (2019)

Experimental induction offers some causal control over circumstances but often leaves several processes changing together. Threat films can alter startle, attention, vigilance, facial behavior, and body posture; no isolated one of these changes is a uniquely sufficient criterion of fear. Studies that operationalize the emotional target using the same visible cue used to test it become circular. A more informative design compares expression to independently sampled experience and event parameters, with transparent criteria for missing or ambiguous reports. Mauss & Robinson (2009) Scherer (2005)

The 2019 review by Barrett and colleagues surveyed evidence for six frequently studied emotion categories and examined whether specific facial configurations can reliably diagnose individual emotional states. Its central methodological contribution is the separation of expressive action from an observer's judgment and from the producer's actual condition. Critiques of a universal expression dictionary are especially relevant to automatic face-based emotion detection, where an algorithm can give a precise category even when the scientific target remains inadequately defined. Barrett et al. (2019)

Evidence for expressive regularity is stronger when converging indicators agree across independent studies and less secure when agreement is confined to stereotyped test items. Researchers should report sensitivity, specificity, calibration, and variation by context, not only overall classification accuracy. They should include cases in which no selected category fits and should examine whether perceived certainty tracks actual success. This evidentiary program permits qualified findings about social signaling while preventing a symbolic category label from being mistaken for direct observation of private feeling. Elfenbein & Ambady (2002) Goel et al. (2024)

Emotional Prosody in Spoken Language

Speech carries information through words and through the timing, pitch, intensity, rhythm, and timbre of the voice. Emotional prosody studies examine how these features vary with an eliciting condition and how listeners attribute affective significance to them. A crucial distinction separates the linguistic content of a sentence from the paralinguistic characteristics of its delivery. The same words can sound reassuring, sarcastic, distressed, or excited because the listener interprets their acoustic form within a communicative situation. Scherer (2003) Hall et al. (2019)

Scherer's review of vocal research paradigms recommends modeling emotion-related production, acoustics, and listener perception as linked stages. That approach reveals which acoustic features reliably vary with circumstances and which merely influence impressions. For example, higher pitch can be associated with increased activation, vocal effort, or language-specific intonation. A study of anger identification must determine whether success depends on features distinctive of anger or on broadly activated speech, lexical hints, or the restricted set of alternative categories. Scherer (2003)

Speech production also varies across speakers because of age, anatomy, language, health, social role, and interpersonal goal. Cross-speaker classification is harder than recognizing a familiar actor whose baseline is known, and recordings must be normalized carefully to prevent equipment or demographic features from becoming unintended category cues. A rigorous validation dataset separates speakers and contexts between training and testing, reports uncertainty, and compares performance with simple acoustic baselines. Poria et al. (2017) Cowie et al. (2001)

Vocal inference is socially consequential because a voice may suggest urgency, fatigue, certainty, concern, or emotional investment before a speaker has finished an utterance. But these impressions should be interpreted as context-dependent evidence rather than unmediated emotion measurements. A listener can misunderstand irony, miss a deliberate performance, or infer more from a narrative than from the acoustic signal. A comprehensive design records the perceived label, the sender's experience, the words, the acoustic structure, and the interactional purpose. Hall et al. (2019) Zaki et al. (2008)

Laughter, Crying, and Non-Speech Vocalizations

Non-speech vocalizations such as laughter, sobbing, screams, sighs, and gasps offer a complementary domain of expressive behavior. Their distinctive acoustic structure can provide socially useful information even when no linguistic message is present. Yet the relation between a vocal form and a psychological condition remains many-to-many: laughter can communicate amusement, affiliation, tension, or derision, and crying may accompany grief, relief, pain, or social appeal. Experimental inference should therefore examine both acoustics and the social event in which the sound occurs. Sauter et al. (2010) Hall et al. (2019)

Sauter and colleagues' cross-cultural experiments tested recognition of nonverbal vocalizations, adding a crucial auditory dimension to disputes previously dominated by facial photographs. Their results support some cross-cultural identification under the tested conditions, while the selected vocal samples and response formats remain important boundaries on interpretation. Follow-up work by Gendron and colleagues demonstrates that cultural experience and task organization can affect vocalization perception. Taken together, these studies show why claims about general recognition must be specified for a modality, stimulus set, category, and sample. Sauter et al. (2010) Gendron et al. (2014a)

Crying and laughter also change an interaction directly: they can elicit comfort, facilitate group affiliation, signal the end of tension, or provoke avoidance. A developmental or interpersonal experiment might observe whether others respond with assistance, approach, or a particular emotion label. Those outcomes distinguish communicative effectiveness from accurate decoding of private experience. An action that successfully recruits care need not tell the observer exactly which feeling prompted it. Hess & Fischer (2013) Hall et al. (2019)

Naturalistic research should include spontaneous vocal behavior, variability in sound type, and temporal coupling with speech, facial movement, and posture. It should also record physical causes such as pain, exertion, illness, or respiratory effort when relevant. With such controls, scientists can estimate how much a listener learns from sound alone, how much from context, and how the combined evidence changes inference. This is a stronger approach than equating a distinctive waveform with a universal emotion category. Bänziger et al. (2009) Scherer (2003)

Posture, Movement, Gesture, and the Whole Body

The body communicates through movement trajectory, stance, head and torso orientation, gait, muscular tension, gesture, and interpersonal spacing. These features can express readiness to act, protective withdrawal, affiliation, and engagement. Observers often combine them with the face so quickly that an isolated-face experiment misses the actual basis of judgment. When bodily signals conflict with facial appearance, the perceived emotional meaning of the face itself may change. Aviezer et al. (2012) Hall et al. (2019)

Aviezer, Trope, and Todorov examined extremely intense positive and negative moments and found that the body was more informative than the isolated face for discriminating affective valence in their stimuli. Their experimental recombination of incongruent faces and bodies produced systematic contextual effects on face judgments. The finding matters because it documents a causal effect of bodily context on observers' interpretation under controlled stimulus manipulation. It does not entail that bodies always dominate faces or that every ordinary emotional episode displays the same pattern. Aviezer et al. (2012)

Whole-body behavior is embedded in physical tasks and cultural practices. Arms raised after a competition can suggest triumph, celebration, appeal, or instruction depending on the event. A hunched posture may follow fatigue, cold, physical discomfort, or social deference. Valid body-based studies must therefore encode the preceding circumstances, action constraints, movement duration, and accompanying speech. A posture label derived from a short isolated clip may capture a visible action more reliably than an inner psychological condition. Hall et al. (2019) Barrett et al. (2019)

Temporal and multimodal methods can make this complexity tractable. Investigators can record synchronized video and audio, identify body trajectories, and obtain independent self-reports and observer judgments. Experimental editing can then assess the incremental contribution of each modality while retaining an unedited condition as a benchmark. Researchers should report performance for familiar and unfamiliar people, actual event types, and cross-cultural samples. Such designs allow body expression to be studied as an active component of social coordination rather than a decorative supplement to facial recognition. Bänziger et al. (2009) Steward et al. (2025)

Situational Context and the Construction of Social Inference

An expressive act acquires meaning through its surrounding event. The observer considers what happened, who was involved, what the target knew, what goals were at stake, and what expression would be appropriate in that relationship. These inferences may be supported by a brief story or by years of familiarity with the person. Contemporary experimental work shows that situational information can be at least as informative for a judgment as isolated facial movement, depending on the emotion category and experimental conditions. Goel et al. (2024) Barrett et al. (2019)

Goel and colleagues compared judgments that could be based on facial features, situations, or their integration across an archival and five empirical datasets. Their results revealed considerable variation across categories and observers, including cases in which situational knowledge alone captured judgments as well as combined cues. These findings question the expectation that every perceiver integrates all available channels equally. They are directly relevant to the cognitive architecture of recognition: people may deploy different strategies shaped by familiarity, category, cue diagnosticity, and social knowledge. Goel et al. (2024)

Context can also alter how identical movements appear. The experimenter can hold a face constant while changing a body, background event, or description, then compare resulting emotion attributions. A judgment shift under such manipulation establishes causal influence of the manipulated contextual information on perception or interpretation. It does not by itself establish that the new attribution matches the target's true emotion, because the target's emotional state may be absent or simulated. This distinction between causal perceptual effect and criterion validity is fundamental. Aviezer et al. (2012) Steward et al. (2025)

The 2025 meta-analysis by Steward and colleagues synthesized visual face–context studies and identified moderators such as congruency and clarity, providing a more general evidentiary base for contextual effects than any single display experiment. Its findings support the need to model both face and scene, while leaving the question of the particular target's experience dependent on the selected criterion. A sophisticated theory consequently treats situated inference as a structured process with empirically estimated dependencies rather than a fixed sequence from muscle pattern to emotion label. Steward et al. (2025) Goel et al. (2024)

Multimodal Integration, Conflict, and Cue Weighting

Everyday observers encounter faces, words, voices, body movements, interpersonal distance, and scenes as coordinated streams. Multimodal recognition research asks whether combining these sources improves a specified judgment and how the improvement occurs. An additive advantage means independent channels supply extra information; interaction means the interpretation of one channel changes with another. Both mechanisms can operate within a single encounter, and their relative importance depends on stimulus reliability, event type, and what the observer is asked to infer. Bänziger et al. (2009) Steward et al. (2025)

The Multimodal Emotion Recognition Test developed by Bänziger, Grandjean, and Scherer assesses recognition using face, voice, and body expressions, providing a standardized method for examining performance beyond facial photographs alone. The test expands available information but remains subject to the selection of portrayals and the criterion used to define a correct category. Its importance lies in forcing measurement designers to state the contribution of each expressive modality and to examine the individual differences that emerge when cues are combined. Bänziger et al. (2009)

Conflicting cues make integration processes visible. A positive voice with a distressed expression may evoke interpretations of irony, concealment, ambivalence, or uncertainty. If the task forces a single emotion label, the measured choice also reflects the observer's response strategy. Experimental manipulations can compare congruent and incongruent combinations while controlling the naturalness of edited stimuli. Response confidence, graded probability judgments, and reaction time can add information about whether the observer truly combines cues or simply privileges one modality. Aviezer et al. (2012) Goel et al. (2024)

A multimodal advantage demonstrated on a curated dataset does not guarantee better real-world inference. The channels may share spurious labels, the training and testing samples may contain the same actors, or contextual details may reveal the intended category without requiring expression recognition. Proper evaluation includes unimodal baselines, context-only baselines, independent senders, a clear emotional reference measure, and out-of-distribution tests. The theoretical goal is to estimate what each cue contributes and how it changes others, with uncertainty preserved where evidence conflicts. Poria et al. (2017) Cowie et al. (2001)

Cross-Cultural Regularities and Cultural Specificity

Cross-cultural research investigates which expressions or judgments are stable across populations and which depend on acquired conventions, local concepts, or social relationships. The tradition began with standardized recognition tests and expanded into free-sorting methods, recognition of unfamiliar expressions, and studies of societies with limited exposure to Western media. Because experiments vary in labels, pictures, sampling, and communication conditions, the proposition of universality requires a precisely stated outcome. Similarity in selected tasks cannot establish invariance of every expressive act, and observed cultural difference does not establish complete communicative separation. Ekman & Friesen (1971) Russell (1994)

Elfenbein and Ambady's meta-analysis found above-chance cross-cultural recognition alongside an in-group advantage, indicating both shared tendencies and sensitivity to cultural familiarity under included designs. These aggregate findings are informative about the recognition tasks studied, but interpretation still depends on the methods and comparison baselines. The in-group effect encourages investigation of learning, exposure, conversational convention, and local interpretive practices. It also challenges evaluators who assume that a classifier's performance on one population must transfer unchanged to another. Elfenbein & Ambady (2002)

Jack and colleagues used dynamic facial stimuli to examine culturally variable representations of emotional expression. Gendron and colleagues asked participants in a remote Namibian setting to sort posed facial expressions with differing degrees of conceptual guidance. Their results revealed strong dependence of categorization on the availability of experimenter-supplied emotion concepts and on local interpretive organization. Crivelli and colleagues further studied emotion perception in indigenous societies, contributing evidence against treating a familiar Western face–label mapping as a self-evident human universal. Jack et al. (2012) Gendron et al. (2014b) Crivelli et al. (2016)

Auditory findings add a useful comparison. Sauter and colleagues documented cross-cultural recognition for selected nonverbal emotional vocalizations, while Gendron and colleagues demonstrated cultural relativity in vocalization perception under other task conditions. The divergent outcomes require close attention to emotion categories, stimulus production, choice formats, and social experience rather than a simple declaration about universal or nonuniversal emotion. Future evidence benefits from collaboration with local researchers, participant-defined categories, multiple response formats, and direct tests of measurement equivalence across languages. Sauter et al. (2010) Gendron et al. (2014a)

Emotion Words, Categories, and Recognition Tasks

An emotion word is a culturally and linguistically organized category that allows observers to describe, remember, and compare states. Recognition research frequently presents a photograph together with a small list of words, then records the participant's choice. This design makes standardized scoring possible, but it also supplies conceptual information and constrains acceptable answers. An isolated scowl can be categorized differently under an open description, a story-completion task, a valence scale, or a forced choice among familiar labels. Russell (1994) Gendron et al. (2014b)

The association between category and perception may be bidirectional. Familiar categories influence the features a perceiver attends to, and repeated encounters teach which expressive patterns are likely under particular circumstances. The question is experimentally tractable through label availability, training, priming, and test generalization. A strong design records spontaneous descriptions before introducing category choices, thereby revealing whether the label was independently produced or selected because the response format offered it. Jack et al. (2012) Barrett et al. (2019)

Research on the semantic organization of emotional experience provides a cautionary parallel. Cowen and Keltner analyzed reports elicited by a large and diverse set of videos, identifying numerous distinguishable categories connected by gradual transitions. This study concerns subjective reports associated with viewed events, not direct proof that all such experiences have corresponding unique facial expressions. Its relevance here is to show how a small predetermined set of emotion words can compress the actual semantic variability of affective life and obscure uncertainty or mixed states. Cowen & Keltner (2017)

Cross-language comparisons must examine conceptual translation rather than assume exact equivalence between dictionary entries. Researchers can compare participant descriptions, local social situations, and category judgments at different levels of abstraction. They should distinguish a population's failure to select an unfamiliar label from an inability to detect distress, approach, or avoidance in another person. Inference can remain socially useful at the level of goals and action tendencies even when categories differ. Such an approach joins linguistic and nonverbal evidence while protecting the independent status of each measured construct. Gendron et al. (2014b) Hall et al. (2019)

Development, Learning, and the Acquisition of Recognition

Emotion recognition develops through perceptual maturation, attention to caregivers, language learning, social interaction, and experience with expressive events. The developmental question is not simply when children can identify a canonical happy face. Investigators must determine which perceptual discriminations are available, how emotion words become attached to observed behaviors, and how children learn to infer a state from a situation. These competencies may emerge on different schedules, especially when experimental response methods demand verbal labels or complex instructions. Widen & Russell (2008) Hall et al. (2019)

Widen and Russell examined young children's acquisition of emotion categories and found developmental differentiation rather than an instant complete adult taxonomy in their tested tasks. Their evidence reminds researchers that an apparent recognition error may reflect limited vocabulary, emerging conceptual boundaries, memory demands, or difficulty following task instructions. A longitudinal approach can distinguish learning of a label from enhanced sensitivity to a facial feature by repeatedly assessing the same children with complementary nonverbal and verbal measures. Widen & Russell (2008)

Social learning affects which movements children observe and what they are told those movements mean. Families and institutions differ in explicit emotion talk, norms of display, and the expression of conflict or care. Repeated associations between situation, gesture, and subsequent action build expectations that later guide recognition. Research should measure these experiences directly rather than treating country, race, or age as explanatory mechanisms by themselves. A context-sensitive developmental model also examines when children learn that a person can conceal, exaggerate, or perform an emotional display. Gendron et al. (2014b) Zaki et al. (2008)

Methodologically, child studies need age-appropriate tasks, checks on language comprehension, independent samples of facial and situational cues, and developmental measurement invariance. Training programs can raise accuracy on tested exemplars without demonstrating generalization to spontaneous social interaction. Investigators therefore should evaluate transfer to new senders, different event contexts, and naturally occurring encounters. Developmental findings become especially meaningful when the outcome is defined as successful interpersonal inference rather than memorization of a short list of adult-designed expression prototypes. Widen & Russell (2008) Elfenbein & Ambady (2002)

Social Relationships, Expressivity, and Emotional Mimicry

Expressions function within relationships characterized by familiarity, trust, status, cooperation, and conflict. The same facial or vocal behavior may be interpreted differently when produced by a close friend, stranger, competitor, or authority figure. Recognition involves previous knowledge of the person's habits and the current purpose of the encounter. Thus, an observer may accurately infer distress from an unusually quiet response even when the face displays no textbook prototype. Studies restricted to unfamiliar static faces cannot capture this relational information. Hall et al. (2019) Zaki et al. (2008)

Zaki, Bolger, and Ochsner studied empathic accuracy as a dyadic process and found that the relation between perceiver empathy and accuracy depended on target expressivity. This finding highlights a basic statistical fact: an observer cannot extract information the sender does not make available through the relevant channels. Interpersonal accuracy must be investigated across both members of the interaction, including expressive opportunities, the perceiver's attention, and the reference measure of the target's experienced state. Zaki et al. (2008)

Emotional mimicry involves a perceiver producing behavior partly resembling a sender's expression. It has been discussed as a process contributing to coordination, affiliation, or emotion understanding. Hess and Fischer emphasize its situational and relational conditions, arguing against a purely automatic matched-motor account. Mimicry can depend on the social meaning of a display and on affiliation goals, which means similar visible movements can participate in different social processes. A correspondence between movements is consequently a phenomenon to explain rather than proof of shared emotion or accurate attribution. Hess & Fischer (2013)

Researchers can disentangle interpersonal functions by manipulating social affiliation, observing reactions over multiple conversational turns, and comparing expression identification with actual prosocial behavior. A sender may intentionally exaggerate concern to request help, smile to preserve a relationship, or suppress an immediate response to avoid conflict. These examples are empirically investigable through interactional records and contemporaneous reports, with due attention to consent and privacy. The broad implication is that expression and recognition are coupled components of social regulation whose causal relationships depend on the interaction's history. Hall et al. (2019) Hess & Fischer (2013)

Individual Differences, Bias, and Perceptual Calibration

Emotion recognition varies across individuals because of familiarity with senders, opportunities for social learning, perceptual sensitivity, vocabulary, attention, and strategies for using context. These influences can differ for facial, vocal, and situational tasks. An observer who reliably detects broad distress may be uncertain whether it reflects sadness, pain, fear, or fatigue. Treating all such judgments as equivalent errors conceals the distinction between perceptual discrimination and interpretation of another person's psychological condition. A useful model therefore measures both accuracy against a justified criterion and the uncertainty accompanying each response. Bänziger et al. (2009) Hall et al. (2019)

Systematic biases may arise when observers generalize from a narrow group of actors or rely excessively on expected patterns associated with demographic features. A familiar expression can be interpreted through stereotypes or prior expectations about social role. Such biases should be evaluated through controlled comparisons that balance target groups, expression types, and contextual narratives, while considering the actual base rates of emotional conditions. Differences in group performance must also be interpreted in light of measurement equivalence and differential exposure to the stimuli. Elfenbein & Ambady (2002) Barrett et al. (2019)

Repeated-measures designs can estimate how stable an individual's cue-weighting pattern is across tasks and time. Goel and colleagues' evidence for moderately stable variation in reliance on contextual information suggests that these differences are a meaningful target for further investigation. Yet stability within a particular task does not establish the trait's generality across social settings. Investigators must separate enduring preferences from stimulus-specific strategies, learning during the experiment, and the tendency to select familiar labels when unsure. Goel et al. (2024)

Calibration analysis examines whether confidence matches observed success. Researchers can ask for graded judgments or probability distributions, use proper scoring rules, and assess whether observers become appropriately less certain when information is incomplete or conflicting. This practice supports more realistic communication of interpersonal uncertainty than a winner-take-all accuracy score. For applications in education, organizational life, or automated systems, the cost of confident false inference may exceed that of acknowledged uncertainty, so error types must be evaluated against the actual decision context. Hall et al. (2019) Poria et al. (2017)

Experimental Paradigms and Ecological Validity

Emotion expression experiments differ in whether they measure production, perception, or interaction. Production designs manipulate eliciting events and record subsequent behavior, ideally alongside independent reports of experience. Recognition designs present preselected displays and ask observers to judge a nominated target. Interaction designs examine what communicators do in response to one another over time. Each paradigm produces information about a different causal link, and conflating them can create the appearance of broad agreement where only a narrow task has been validated. Mauss & Robinson (2009) Hall et al. (2019)

Static photographs make it possible to control visible features precisely, while dynamic video preserves timing and multimodal recording permits analysis of coordination. Context experiments can independently alter scenes, written descriptions, and faces. Free-response tasks examine spontaneous meaning, whereas forced-choice formats quantify selection among supplied categories. Researchers need to justify which task corresponds to the scientific question and include direct comparisons when alternative procedures yield different interpretations of the same stimuli. Russell (1994) Goel et al. (2024)

Ecological validity concerns the relationship between experimental circumstances and the everyday process the investigator wishes to explain. Naturally occurring encounters contain ambiguous events, mixed emotions, social feedback, variable base rates, and strategic displays. An idealized stimulus can isolate a cue while departing considerably from those conditions. The appropriate response is to use complementary laboratory and field designs, with clear statements about what transfers between them. Naturalism alone also fails to identify causality, making carefully controlled experiments indispensable for testing specific mechanisms. Barrett et al. (2019) Steward et al. (2025)

Studies should sample multiple senders, perceivers, languages, eliciting events, and recording environments. Within-person baselines can improve sensitivity to unusual changes, but they introduce privacy and longitudinal stability questions. Transparent preregistration, retention of ambiguous trials, independent replication, and accessible stimulus descriptions improve interpretability. Above all, the outcome must be defined before analysis: intended portrayal, modal observer label, concurrent self-report, or a behaviorally relevant criterion. Each answers a distinct question about expression and recognition. Durán & Fernández-Dols (2021) Elfenbein & Ambady (2002)

Statistical Evidence, Effect Heterogeneity, and Replication

An evidence review must distinguish descriptive regularity, predictive association, experimental causation, and theoretical explanation. A correlation between brow position and an emotion label may reveal a cue used by judges. A randomized change in context can demonstrate an effect on attribution. A prospective prediction of a sender's independently reported emotion tests a stronger criterion. The meaning of an effect size depends on the chosen outcome, sampling frame, and baseline against which performance is compared. Goel et al. (2024) Steward et al. (2025)

Meta-analysis provides a way to summarize heterogeneous studies while evaluating moderators and the precision of estimates. Elfenbein and Ambady synthesized cross-cultural recognition, Durán and Fernández-Dols analyzed co-occurrence of predicted expressions and emotions, and Steward and colleagues synthesized face–context interactions. These are complementary bodies of evidence rather than interchangeable estimates of a universal expression-recognition parameter. Researchers should inspect how each review defines observations, handles dependence, chooses inclusion criteria, and accounts for variations in experimental task. Elfenbein & Ambady (2002) Durán & Fernández-Dols (2021) Steward et al. (2025)

Confidence intervals, sampling uncertainty, publication bias, missing outcomes, and selective stimulus construction matter especially when studies have many expressions or observer subgroups. Mean accuracy obscures categories that are recognized differently, circumstances in which performance fails, and distributions of participant-specific effects. Hierarchical models and cross-classified analyses can represent both sender and perceiver variability, while sensitivity analyses assess whether findings depend on particular stimuli or coders. Statistical sophistication remains subordinate to construct validity: precise estimates of an inappropriate criterion do not become psychologically meaningful through mathematical precision. Russell (1994) Goel et al. (2024)

Reproducibility requires code and stimulus descriptions sufficient to regenerate an analysis, and replication requires sufficiently independent observations to test its generality. A large dataset of near-duplicate pictures cannot substitute for new senders and contexts; similarly, agreement among annotators trained on the same label scheme does not test whether the labels reveal felt emotion. A compelling cumulative science combines controlled evidence, independent behavioral criteria, cross-cultural sampling, and transparent uncertainty. These requirements apply to human judgment studies and machine-learning benchmarks alike. Barrett et al. (2019) Poria et al. (2017)

Affective Computing and the Limits of Machine Emotion Recognition

Automated emotion recognition systems analyze speech, facial action, text, movement, or combinations of signals to produce categorical or continuous predictions. Such systems can be useful for identifying observable patterns and studying the computational organization of expressive data. Their scientific status depends on the outcome that training labels represent. If the label is an actor's portrayal instruction or an annotator's interpretation of a short clip, the model learns to approximate that label under the dataset's conditions; it has not thereby demonstrated direct access to the actor's subjective state. Cowie et al. (2001) Poria et al. (2017)

Early research on human–computer emotion recognition, including Cowie and colleagues' work, emphasized combining psychological theory with speech and face signal processing. Later multimodal surveys mapped increasingly complex fusion architectures. The unresolved issue is frequently the scientific validity of the target, rather than the computational capacity to predict it. Systems may exploit background, speaker identity, recording artifacts, and stereotyped acting styles unless evaluation blocks those shortcuts. Tests require actor-disjoint partitions, culturally varied populations, naturalistic conditions, calibrated confidence, and transparent failure reporting. Cowie et al. (2001) Poria et al. (2017)

The 2019 review by Barrett and colleagues specifically examined consequences of facial emotion inference for commercial and institutional applications. A model that assigns anger or deception from a face without independent evidence may introduce unwarranted psychological claims into education, hiring, or surveillance. Responsible deployment begins by defining observable behavior, intended decision, and scientific benchmark; it also requires safeguards for privacy, consent, discrimination, and misuse. Precision in a laboratory benchmark cannot by itself justify high-stakes inference about another person's motives or mental health. Barrett et al. (2019) Hall et al. (2019)

In the Artificial Era, artificial cognition and human–artificial relations expand the research field to interactions with conversational and embodied systems. An Artificial system can generate an emotional expression, interpret human signals, or maintain a probabilistic social model; each is an independently testable functional capacity. Anthropomorphic communication and reliable recognition require distinct assessments. Future research should compare human and artificial observers on identical contextual inference tasks and evaluate whether their predictions improve cooperative understanding without overclaiming access to inner experience. Goel et al. (2024) Bänziger et al. (2009)

Critical Synthesis and Unresolved Research Problems

The accumulated evidence supports a layered account in which observable expressive events, social communication, and emotion attribution must be modeled together. Facial, vocal, and bodily movements carry information, but their diagnostic value changes with eliciting circumstances, social intention, modality, and cultural learning. Regularities established in controlled expression sets remain meaningful results within those settings. Their extension to individual emotional diagnosis requires separate empirical validation, especially with naturalistic signals and independently assessed emotional conditions. Keltner et al. (2019) Barrett et al. (2019)

The first unresolved problem is causal identification: which physiological or cognitive mechanisms generate a particular display, and under what conditions does the display contribute to communication? The second concerns inference: how do observers integrate partially dependent facial, bodily, vocal, and situational information? The third concerns generalization across ages, languages, societies, individuals, and social relationships. The fourth concerns measurement: can a stable reference criterion for emotion be constructed without circularly defining it through the expression under investigation? Goel et al. (2024) Durán & Fernández-Dols (2021)

Advancing the field requires experiments that record both production and perception with synchronized multimodal measures. Naturalistic longitudinal datasets should include self-report, social context, participant-defined categories, and relevant behavior; controlled perturbations can then isolate mechanisms. Statistical analyses should estimate conditional probabilities, uncertainty, and subgroup heterogeneity rather than suppress them. Direct replications should test the same claim with new populations, new stimuli, and alternative measurement procedures. Such a program can reconcile apparently opposing findings by specifying the conditions under which expressive cues are truly informative. Steward et al. (2025) Elfenbein & Ambady (2002)

Scientific explanation is strengthened when theories make falsifiable predictions about the difference between expression, portrayal, recognition judgment, and experienced emotion. A successful model must predict situations in which a facial movement is informative, cases in which context dominates, and conditions in which the appropriate response is uncertainty. This criterion also establishes the boundary of machine emotion analysis: computational prediction is evaluated against a defined target, while claims about psychological states require independent evidence. Barrett et al. (2019) Poria et al. (2017)

Conclusion

Emotional expression and emotion recognition form a dynamic psychological relationship between behavioral production, available cues, cultural and personal knowledge, and interpretation by an observer. The face, voice, body, and situation contribute information through different channels and with different degrees of specificity. Their effects can be measured and experimentally compared, but interpretation requires careful identification of the state or judgment the study seeks to explain. This conceptual discipline provides a common foundation for classical expression theory, contemporary contextual models, and computational approaches. Hall et al. (2019) Scherer (2003)

Historical studies demonstrated systematic recognition of selected expressive portrayals, and later experiments established consequential cultural, situational, and modality-dependent variation. Cross-cultural meta-analysis identified recognition regularities alongside in-group advantages; co-occurrence synthesis examined the correspondence between expressions and experienced emotion; contextual experiments and more recent meta-analysis tested how surrounding information changes observer judgments. Together these literatures indicate a rich, structured, and variable signaling system whose properties depend on the measurement task and the circumstances in which expression occurs. Elfenbein & Ambady (2002) Durán & Fernández-Dols (2021) Steward et al. (2025)

The strongest scientific conclusion is methodological as well as substantive. Researchers should specify the sender's expressive act, the observer's task, the reference criterion, and the proposed causal process. They should evaluate alternative explanations, uncertainty, and transfer to new contexts, and they should not confuse shared interpretation with direct evidence of private psychological experience. These requirements sustain empirically responsible applications in social psychology, development, education, and human–artificial relations. Goel et al. (2024) Barrett et al. (2019)

The scientific future of the field rests on integrated, longitudinal, and internationally collaborative research capable of connecting visible movement to social action and independently characterized emotional experience. Its decisive questions concern conditional reliability, causal organization, contextual calibration, cultural learning, and the boundaries of artificial inference. A mature science of emotional expression and emotion recognition will describe those relations with sufficient precision to explain both successful communication and meaningful disagreement. Keltner et al. (2019) Zaki et al. (2008)

References

  • Aviezer, H., Trope, Y., & Todorov, A. (2012). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 338(6111), 1225–1229. https://doi.org/10.1126/science.1224313
  • Bänziger, T., Grandjean, D., & Scherer, K. R. (2009). Emotion recognition from expressions in face, voice, and body: The Multimodal Emotion Recognition Test (MERT). Emotion, 9(5), 691–704. https://doi.org/10.1037/a0017088
  • Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
  • Cowen, A. S., & Keltner, D. (2017). Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences, 114(38), E7900–E7909. https://doi.org/10.1073/pnas.1702247114
  • Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001). Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, 18(1), 32–80. https://doi.org/10.1109/79.911197
  • Crivelli, C., Jarillo, S., Russell, J. A., & Fernández-Dols, J.-M. (2016). Reading emotions from faces in two indigenous societies. Journal of Experimental Psychology: General, 145(7), 830–843. https://doi.org/10.1037/xge0000172
  • Durán, J. I., & Fernández-Dols, J.-M. (2021). Do emotions result in their predicted facial expressions? A meta-analysis of studies on the co-occurrence of expression and emotion. Emotion, 21(7), 1550–1569. https://doi.org/10.1037/emo0001015
  • Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129. https://doi.org/10.1037/h0030377
  • Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6(3–4), 169–200. https://doi.org/10.1080/02699939208411068
  • Elfenbein, H. A., & Ambady, N. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128(2), 203–235. https://doi.org/10.1037/0033-2909.128.2.203
  • Gendron, M., Roberson, D., van der Vyver, J. M., & Barrett, L. F. (2014). Cultural relativity in perceiving emotion from vocalizations. Psychological Science, 25(4), 911–920. https://doi.org/10.1177/0956797613517239
  • Gendron, M., Roberson, D., van der Vyver, J. M., & Barrett, L. F. (2014). Perceptions of emotion from facial expressions are not culturally universal: Evidence from a remote culture. Emotion, 14(2), 251–262. https://doi.org/10.1037/a0036052
  • Goel, S., Jara-Ettinger, J., Ong, D. C., & Gendron, M. (2024). Face and context integration in emotion inference is limited and variable across categories and individuals. Nature Communications, 15, Article 2443. https://doi.org/10.1038/s41467-024-46670-5
  • Hall, J. A., Horgan, T. G., & Murphy, N. A. (2019). Nonverbal communication. Annual Review of Psychology, 70, 271–294. https://doi.org/10.1146/annurev-psych-010418-103145
  • Hess, U., & Fischer, A. (2013). Emotional mimicry as social regulation. Personality and Social Psychology Review, 17(2), 142–157. https://doi.org/10.1177/1088868312472607
  • Jack, R. E., Garrod, O. G. B., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences, 109(19), 7241–7244. https://doi.org/10.1073/pnas.1200155109
  • Keltner, D., Sauter, D., Tracy, J. L., & Cowen, A. (2019). Emotional expression: Advances in basic emotion theory. Journal of Nonverbal Behavior, 43(2), 133–160. https://doi.org/10.1007/s10919-019-00293-3
  • Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209–237. https://doi.org/10.1080/02699930802204677
  • Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017). A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion, 37, 98–125. https://doi.org/10.1016/j.inffus.2017.02.003
  • Russell, J. A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102–141. https://doi.org/10.1037/0033-2909.115.1.102
  • Sauter, D. A., Eisner, F., Ekman, P., & Scott, S. K. (2010). Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. Proceedings of the National Academy of Sciences, 107(6), 2408–2412. https://doi.org/10.1073/pnas.0908239106
  • Scherer, K. R. (2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40(1–2), 227–256. https://doi.org/10.1016/S0167-6393(02)00084-5
  • Scherer, K. R. (2005). What are emotions? And how can they be measured? Social Science Information, 44(4), 695–729. https://doi.org/10.1177/0539018405058216
  • Steward, B. A., Mewton, P., Palermo, R., & Dawel, A. (2025). Interactions between faces and visual context in emotion perception: A meta-analysis. Psychonomic Bulletin & Review, 32(5), 1987–2003. https://doi.org/10.3758/s13423-025-02678-6
  • Widen, S. C., & Russell, J. A. (2008). Children acquire emotion categories gradually. Cognitive Development, 23(2), 291–312. https://doi.org/10.1016/j.cogdev.2008.01.002
  • Zaki, J., Bolger, N., & Ochsner, K. (2008). It takes two: The interpersonal nature of empathic accuracy. Psychological Science, 19(4), 399–404. https://doi.org/10.1111/j.1467-9280.2008.02099.x