RESEARCH DOSSIER
Construct Validity
Abstract. Construct validity concerns the scientific justification for interpreting observed responses, behaviors, and scores as evidence about intended psychological attributes under specified conditions. This fundamental critical narrative monograph reconstructs the development of validation theory from Cronbach and Meehl to multitrait–multimethod analysis, unified validity, argument-based interpretation, and causal measurement accounts. It examines content, response processes, structural relationships, external predictions, measurement invariance, differential item functioning, consequences, and the limits of statistical fit. Empirical audits of common instruments, open-science research, and multilaboratory replication reveal recurring weaknesses in overreliance on internal consistency and unexamined operational assumptions. The synthesis specifies evidentiary requirements for valid interpretations across populations, clinical and institutional contexts, digital measurement, and human–artificial research. It is a theory-guided critical narrative synthesis of published literature, not an original experiment or protocol-registered systematic review.
Keywords: construct validity; psychological measurement; construct validation; Cronbach and Meehl; nomological network; multitrait–multimethod matrix; convergent validity; discriminant validity; response-process evidence; content evidence; internal structure; measurement invariance; differential item functioning; score interpretation; validity argument; psychometrics; replication; artificial cognition
Construct Validity as an Inference About Psychological Attributes
Construct validity concerns the warranted interpretation of observations, responses, and scores as evidence about a specified psychological construct. An attribute such as anxiety, cognitive flexibility, social trust, or perceived agency is inferred through tasks, reports, behavior, or physiological measurements. The interpretation becomes scientifically defensible when a network of theoretical predictions and empirical observations supports the proposed connection. A measurement does not acquire universal validity merely because its items seem plausible or because a statistical model fits one dataset. The intended population, administration conditions, interpretive claim, and decision context all determine what must be justified. This formulation draws on the construct-centered program of Cronbach & Meehl (1955) and the unified view of score interpretation developed in Messick (1995).
The unit of validation is a claim about what an observed result means under defined conditions. A self-report inventory may support comparisons of reported distress between research groups, yet provide insufficient evidence for diagnosing an individual or forecasting a clinical outcome. The same recorded score can sustain one inference and fail another. The 2014 joint testing standards explicitly treat validity as evidence and theory supporting interpretations for intended uses, rather than an immutable property owned by an instrument AERA, APA, & NCME (2014). In applied settings, Cook & Beckman (2006) organizes that evidence through content, response processes, internal structure, relationships to other variables, and consequences. Each source addresses a different vulnerability in the inferential chain.
This monograph asks how constructs are defined, how evidence accumulates, which kinds of results discriminate among theories, and how validation withstands replications, new populations, changed administration systems, and novel applications. The research method is a critical narrative synthesis of canonical theoretical publications, methodological reviews, documentary standards, and selected empirical investigations. It is not a registered systematic review or new experiment. The resulting conclusions are conditional on the observed quality of the cited designs and their relevance to a particular interpretation. The theoretical analysis of Borsboom et al. (2004) supplies a valuable challenge to purely correlational validation, while the argument-based program of Kane (2013) clarifies how an interpretive claim may be progressively challenged.
Constructs, Indicators, Operational Definitions, and Research Objects
A construct is a concept introduced to organize theoretically meaningful regularities and specify testable relationships. It may represent a dispositional tendency, psychological process, subjective report, latent dimension, or functional capacity. Its operational definition specifies procedures that produce observations; its theoretical definition specifies the phenomenon those observations are meant to describe. The two definitions must remain linked through explicit assumptions. An attention task can operationalize performance under a selected cueing regime without encompassing every process ordinarily classified as attention. Cronbach & Meehl (1955) framed measurement interpretation through a nomological network, and Clark & Watson (2019) underscores the importance of precise conceptualization and deliberate selection of item content.
Indicators can be reflective consequences of a proposed latent attribute, causal components of a composite, performance samples from a task domain, or behavioral events whose significance depends on context. These structures make different demands on validation. Correlations among interchangeable indicators may be expected under a reflective model, whereas the components of a deliberately defined index may have distinct causal origins. Researchers therefore need to describe what is measured, which population of possible indicators was considered, and which assumptions support aggregation. Borsboom et al. (2004) argues that the attribute-to-score pathway is central when the intended claim is genuine measurement of an existing attribute. The ontology of the attribute is part of the scientific question.
Construct underrepresentation occurs when the chosen observations omit important parts of the proposed phenomenon. Construct-irrelevant variance occurs when scores vary for reasons that do not belong in the proposed interpretation, including reading skill in a task purported to assess spatial cognition or device familiarity in a digital attention assessment. These are properties of the measurement-to-interpretation relation, not merely statistical anomalies. Item review, response interviews, alternative operationalizations, and experimental manipulation offer distinct ways to identify them AERA, APA, & NCME (2014). Scientific scope is strengthened by recording explicitly what the score represents, what it may partly represent, and which purported meanings currently exceed the available evidence Flake & Fried (2020).
From Classical Test Validity to Construct Validation
Early test theory often organized validation into content, criterion-related, and construct approaches. Content assessment considered the match between items and a defined domain; criterion studies examined relations with external outcomes; construct studies addressed interpretations concerning theoretical attributes. This classification remains historically important because it reflects the methods available for very different scientific purposes. Cronbach & Meehl (1955) made a decisive intervention by explaining why psychological attributes without adequate external gold standards require theoretically articulated programs of validation. The article appeared in Psychological Bulletin in 1955 and established construct validity as a research process in which multiple lines of evidence bear on an interpreted score.
The subsequent program of Campbell & Fiske (1959) introduced a structured demand for convergent and discriminant evidence across traits and methods. Rather than accepting correlations between superficially similar measures as sufficient, it examined the pattern of associations produced by several methods applied to more than one construct. That historical contribution provided a tractable empirical test of whether data covary because of the intended attribute, the measurement method, or both. Later research expanded the toolkit through structural models and explicit validity arguments, while retaining the central insight that evidence must distinguish the target construct from competing explanations. A historical methodology is evaluated by the inferential problems it solved and by limitations that remain.
The modern unified account emerged prominently in the work of Messick (1995), integrating substantive, structural, external, generalizability, content, and consequential considerations rather than treating three older categories as interchangeable validity types. The 2014 AERA, APA, & NCME (2014) standards express closely related sources of evidence in operational language used in testing practice. Kane (2013) sharpened the methodological requirement: specify an interpretation-and-use argument, identify the assumptions connecting observations to the intended conclusion, and evaluate each assumption with pertinent evidence. The history reveals a progression from isolated coefficients toward cumulative justification. Established techniques persist, but their meaning now depends on a transparent model of the claims being supported.
Nomological Networks and Discriminating Predictions
A nomological network makes the theoretical commitments of a construct publicly inspectable. It describes relations expected among the target attribute, its observable indicators, related attributes, situational manipulations, and relevant outcomes. A validation program then tests predictions that could fail. If a proposed measure of cognitive flexibility responds primarily to processing speed and general task familiarity, the evidence may favor a narrower description than its original label. Cronbach & Meehl (1955) insisted that construct validation requires a theoretical system capable of revision, and Westen & Rosenthal (2003) offered quantitative ways to compare observed relationship patterns against predicted ones. The critical feature is the discriminating structure of predictions, not simply the number of positive correlations.
Convergence is meaningful when different measurement approaches agree for reasons attributable to the target attribute. Discrimination is meaningful when theoretically distinct attributes remain distinguishable despite method overlap, common contexts, and plausible causal associations. A measure of burnout should have predictable connections to exhaustion while preserving a defensible distinction from general depression or chronic stress, if the proposed theory assigns these concepts separate roles. Campbell & Fiske (1959) established the underlying logic, while Podsakoff et al. (2003) documents how shared reporting and response processes can manufacture apparent agreement among scales. Relationships with external variables deserve models of method influence as well as psychological interpretation.
A sophisticated network also contains temporal and intervention predictions. If the construct describes a capacity that changes after training, a longitudinal study can ask whether the purported indicators change coherently and whether the change has the consequences the theory anticipates. A failed prediction may reveal measurement error, contextual moderation, or a genuine problem in the construct. A successful prediction remains vulnerable to alternative causal accounts and selective reporting. Kane (2013) proposes organizing such uncertainties as explicit warrants and rebuttals. Validation therefore advances by locating the weakest links in a particular interpretation and investigating them directly, rather than accumulating an undifferentiated portfolio of favorable findings.
Competing Conceptions of What Makes Measurement Valid
Contemporary validity theory contains substantive disputes about what its claims require. In a unified inferential account, validity is established by evidence and reasoning that justify a particular meaning and use of a score Messick (1995). In a causal realist account, an instrument measures an attribute when the attribute exists and variation in that attribute causally generates relevant variation in the measurement outcome Borsboom et al. (2004). Both positions require genuine scientific work, but they prioritize different questions. One emphasizes whether interpretive arguments are sufficiently supported; the other asks how the specified attribute produces observations. An empirical program benefits from making these commitments visible rather than treating the word validity as self-explanatory.
The causal view makes response mechanisms, task demands, and intervening cognitive processes theoretically central. An association between a loneliness scale and social network size might fit an expected nomological relationship without establishing how experienced loneliness affects endorsement of particular scale items. Conversely, carefully documented response processes may establish plausible mechanisms without showing that the score functions well in consequential decisions. Cook & Beckman (2006) treats these as complementary sources of information in practical assessment. A measurement theory must also distinguish attributes inferred as causes of indicators from indexes defined as summaries of multiple heterogeneous components. Ontological assumptions influence which statistical findings count as evidence.
The argument-based approach articulated by Kane (2013) offers a pragmatic way to expose where the disagreement matters. Researchers can state an observed score, a proposed construct interpretation, a decision rule, and the assumptions connecting them. A causal mechanism may become one warrant, internal structural results another, and external predictive evidence a third. When a use involves institutional selection or clinical classification, the case additionally requires evidence about application and consequences. The central scientific obligation is to specify the scope of the claim before choosing tests. Without that discipline, favorable coefficients can be assembled to defend meanings that were never clearly defined in advance.
Evidence From Test Content and Domain Representation
Content evidence asks whether the observations required by an instrument adequately represent the intended psychological domain. The investigator must identify the construct's boundaries before determining which situations, behaviors, experiences, or knowledge elements belong in the assessment. Expert judgment contributes when criteria are explicit, experts have relevant expertise, and disagreement is reported. Sireci (1998) examined methods for gathering and analyzing such evidence, including item-to-objective ratings and multidimensional analyses of similarity judgments. The strongest content argument moves beyond a generic assertion of expert approval: it documents a domain specification, the sampling logic that generated items or tasks, procedures for review, revisions, and the representational limitations that survive.
Consider a questionnaire intended to measure social trust. A scale consisting entirely of items about trusting close relatives has a narrow interpersonal domain, even if its alpha coefficient is excellent. The construct might also involve institutional trust, strangers, risk, reciprocity, and situational power. Whether these domains should be grouped is a theoretical question that must be answered before interpreting total scores. Clark & Watson (2019) recommends extensive early item pools, careful wording, and discriminating evaluation against neighboring constructs. Content evidence is also critical for performance tests because a small selection of task demands can favor particular strategies rather than sample the conceptual domain. Representation and accessibility should be designed together.
Expert ratings have identifiable limits: panels may share the same conceptual blind spots, underrepresent cultural practices, and disagree about what the construct means. Endorsement percentages cannot by themselves prove psychological relevance. A rigorous review therefore recruits relevant substantive perspectives, examines alternative construct definitions, and tests whether purported content coverage predicts the inferential performance of the instrument. AERA, APA, & NCME (2014) treats content as one source in a larger validity argument, while Sireci & Padilla (2014) describes validation as the process of substantiating whether assessments fulfill their stated purposes. The result is a traceable connection between theoretical specification and selected evidence, not a certification detached from use.
Response Processes and the Causal Path From Construct to Score
Response-process evidence investigates what participants, observers, raters, or automated systems actually do when an assessment produces data. The intended interpretation usually assumes that people read an item, retrieve relevant experiences, compare alternatives, and respond according to a target attribute. Those assumptions may fail because of linguistic ambiguity, social desirability, fatigue, strategic guessing, memory heuristics, or learned task-specific procedures. AERA, APA, & NCME (2014) recognizes evidence about response processes as a central validity source. Cognitive interviews, think-aloud procedures, response-time analyses, eye movements, experimental item alterations, and detailed observation can test distinct stages of the hypothesized chain. Their evidentiary value depends on whether procedures reveal mechanisms relevant to the proposed score meaning.
For example, an item asking how often a person feels in control may elicit reflections on financial resources in one group and immediate task performance in another. Equal numerical responses then need not represent equal psychological conditions. Response interviewing can identify the difference, while experimental wording changes may show how strongly the score depends on the interpretation. Such findings can trigger item revision, separate subscales, or a narrower theoretical claim. Borsboom et al. (2004) gives causal priority to the processes through which an attribute affects measurement outcomes. The implication is that a well-fitting statistical structure can coexist with faulty response mechanisms; empirical scrutiny of the response process remains necessary.
Response investigations also have limits. Verbal reports about reasoning can omit automatic or inaccessible processes, alter the task being studied, or provide retrospectively plausible explanations rather than direct records of cognition. Observational traces may be compatible with multiple mechanisms. Triangulation is therefore essential: interviews establish experienced meanings, behavioral manipulations test sensitivity to particular task features, and models evaluate consequences for observed scores. Cook & Beckman (2006) shows how response processes complement other evidence sources in assessment. For machine-scored tasks, rater disagreement, software behavior, data preprocessing, and interface design become additional stages of the causal chain requiring an explicit audit, particularly when an assessment crosses languages or platforms.
Internal Structure, Factor Models, and Measurement Claims
Internal-structure evidence concerns whether dependencies among observed indicators agree with the proposed organization of the construct. For a multidimensional instrument, the theory may predict several correlated factors; for a narrow scale, it may predict one coherent latent dimension. Confirmatory factor analysis, exploratory models, item response models, and residual examinations supply ways to test these patterns. The question is not whether a particular numerical threshold is achieved in isolation, but whether model assumptions, factor interpretations, and distributional properties fit the substantive research claim. Clark & Watson (2019) emphasizes unidimensionality and discriminant item development rather than maximizing homogeneity indiscriminately; AERA, APA, & NCME (2014) situates structural evidence among other validity sources.
Reliability and structural fit address aspects of score consistency and model adequacy. A high internal-consistency coefficient may follow from redundant wording, shared response style, or a narrow slice of a broader domain. A model may produce an acceptable fit index while omitting theoretically relevant indicators or allowing parameters to absorb misspecification. Flake et al. (2017) found that reports on scales in the social and personality literature often emphasized coefficient alpha without sufficient ongoing construct validation. The methodological response is to report multiple relevant forms of evidence, disclose item-level decisions, compare plausible structural models, and examine whether the interpretation generalizes across settings and groups. Reliability is necessary for some interpretations without establishing their meaning.
Structural modeling acquires evidentiary strength through independent assessment. Model selection guided by the same data can overfit, while correlated residuals or post hoc item deletion can disguise misspecification. Investigators should distinguish preregistered structural tests from exploratory reconstruction and replicate final solutions in independent data when feasible. Hussey & Hughes (2020) illustrates the consequences of broadening evaluation beyond a single consistency coefficient to include retest stability, structure, and invariance. Structural validity remains an inferential component within construct validation; the technical derivation and comparison of factor, item response, and reliability estimators belong to the separate psychometric measurement domain. This dossier evaluates what those models establish about score meaning.
Convergent and Discriminant Evidence in Multitrait–Multimethod Designs
The multitrait–multimethod program of Campbell & Fiske (1959) addresses an enduring ambiguity: psychological observations can covary because of a common attribute, shared measurement procedures, or both. A design that assesses multiple traits through multiple methods can inspect correlations between different measures of the same trait and correlations between different traits measured similarly. Predicted convergence gains force when it survives methodological diversity; discrimination gains force when theoretically distinct constructs remain separable. The matrix is an organized pattern of substantive tests rather than a universal cut-off score. Its interpretation depends on measurement quality, method independence, trait relations, and predeclared expectations about the relationships that ought to appear.
Suppose researchers examine emotional regulation through self-report, behavioral choice under uncertainty, and independent observer ratings while also examining impulsivity and anxiety through analogous sources. Agreement within methods might reflect general positive self-description or shared situational conditions. Agreement across methods could support common construct variance, provided the methods are sufficiently distinct and theoretical predictions were made in advance. Podsakoff et al. (2003) reviews sources and remedies for method bias, showing why a questionnaire battery administered in one sitting is vulnerable to artifacts of shared responding. The design should separate method artifacts where possible and report disagreements as substantive evidence requiring explanation.
Multitrait evidence is stronger when patterns are evaluated against explicit theoretical expectations rather than an unspecified hope that all relevant correlations are high. Westen & Rosenthal (2003) proposes quantitative contrasts between predicted and observed correlation profiles, making it possible to summarize degrees of support more transparently. Yet quantitative indices cannot rescue invalid construct definitions or incomplete trait sampling. Real constructs may influence each other causally, so some correlations between nominally distinct traits are expected. Conversely, different methods may tap different time scales of a single phenomenon. The scientific task is to distinguish theoretically meaningful divergence from method distortion and measurement failure; a matrix contributes information only through those interpretive assumptions.
External Relationships, Prediction, and Incremental Explanation
Relationships with external variables provide evidence when predictions are derived from a clear construct theory. The relevant outcomes may include behavioral decisions, later functioning, laboratory effects, observer judgments, educational attainments, or theoretically adjacent traits. Predictive success helps establish utility for a particular inference, but a correlation cannot show by itself that the intended attribute caused the association. Kane (2013) emphasizes that each intended interpretation and use requires its own supporting argument. When a questionnaire predicts treatment attendance, it is important to ask whether the prediction derives from the proposed motivational construct or from resources, literacy, prior experience, and institutional barriers that happen to covary with its score.
Incremental validity is often studied by asking whether a new score predicts an outcome beyond established measures. This question has practical value when assessments consume time, effort, or money. Nevertheless, incremental prediction depends on the comparison set, outcome quality, and population; a measure can outperform a weak benchmark while adding little to mechanistic understanding. Messick (1995) places external relations within a broad construct framework and rejects treating criterion correlations as a complete substitute for theory. Prospective designs, independent datasets, and theoretically contrasting criteria are more informative than a series of post hoc correlations chosen because they happen to be significant.
A scientific prediction should also specify where it ought to fail. A trait assumed to be stable should have an expected temporal pattern; a context-sensitive state should shift under identified environmental changes. A measure developed for low-stakes research may react to incentives when used for selection. AERA, APA, & NCME (2014) links interpretation to intended uses, while Cook & Beckman (2006) explains why evidence must be collected from multiple sources. When predictive claims are broad, research should document population transfer, calibration, subgroup performance, and possible changes produced by the assessment itself. Distinct evidence about predictive accuracy, construct meaning, and decision utility should remain analytically visible.
Measurement Invariance Across Groups and Time
Comparisons between populations rely on assumptions that may be hidden inside familiar score labels. Researchers who compare an anxiety scale across languages, ages, genders, or clinical contexts implicitly assume that the observations have sufficiently similar relationships to the intended attribute. Measurement invariance research examines the structure needed for particular cross-group interpretations. Vandenberg & Lance (2000) established a methodological synthesis linking factor loadings, intercepts, and other model components to comparisons across groups, while Putnick & Bornstein (2016) reviews conventions and reporting practices in developmental research. Evidence of structural similarity does not automatically establish cultural equivalence or identical response processes, but its absence can undermine naive mean comparisons.
Different comparative claims require different forms of equivalence. Similar loading patterns may support investigation of broad factor relations, while latent mean comparisons typically require additional restrictions on intercepts or thresholds. Longitudinal research raises comparable questions when respondents' understanding of an item changes after an intervention or developmental transition. Putnick & Bornstein (2016) distinguishes levels of invariance and discusses practical responses to partial failures. Researchers must report why any relaxed constraints are defensible and examine whether remaining differences have a substantive interpretation. Statistical invariance tests depend on sample characteristics, model specification, response distributions, and the sensitivity of the criteria used to evaluate fit.
The inferential task is to connect model evidence with the intended comparison. An instrument can show some noninvariance yet remain useful for within-group descriptions, while equal sum scores from non-equivalent items may be deceptive across groups. AERA, APA, & NCME (2014) requires attention to fairness and accessibility in intended testing contexts. Good validation combines quantitative differential functioning analyses with interviews, translation studies, domain expertise, and local participation. Cross-group transport should be justified for the exact inference at issue. The technical algorithms of differential item functioning are examined in the adjoining psychometrics domain; this chapter evaluates their consequences for construct meaning and comparative claims.
Differential Item Functioning and Competing Explanations
Differential item functioning occurs when item responses differ across groups after relevant levels of the target attribute have been taken into account under a specified model. Such differences are clues to possible non-equivalence rather than automatic proof of unfairness. A reading item may be harder because of dialect or cultural familiarity rather than the construct being measured, but group differences can also reflect legitimate construct-relevant processes. Zumbo (2007) surveys generations of DIF analysis and their interpretive challenges. Investigation requires locating the differential functioning, identifying plausible causes, and evaluating how it changes scores or decisions.
Quantitative methods may flag items through conditional comparisons, logistic models, or item response approaches. Their interpretation depends on anchor-item assumptions, dimensionality, sample size, and whether the statistical matching variable adequately represents the target construct. When a test is already contaminated, matching respondents by its total score can obscure the very mechanism under investigation. Vandenberg & Lance (2000) and Putnick & Bornstein (2016) illustrate the wider measurement-invariance framework that situates these choices. The most informative analyses evaluate sensitivity to alternative anchor sets and models and complement statistical evidence with content and response-process inquiry. A flagged parameter is the beginning of scientific diagnosis, not its conclusion.
Consequences depend on the proposed use. Small item-level differences may have limited effect on group-level correlations while producing unacceptable errors near a decision threshold. Conversely, a detectable statistical departure may have little practical impact on the narrow question being asked. AERA, APA, & NCME (2014) requires examining fairness and consequences in the context of intended interpretations. Investigators should report which comparisons remain supportable after accounting for functioning differences, which require qualification, and which should be suspended. Validity concerns the defensibility of interpretation under specified conditions, so conclusions should not leap from a single invariance statistic to universal claims about people or groups.
Development, Culture, Language, and Construct Boundaries
Developmental change can alter both the target phenomenon and the processes by which observations are generated. A child's ability to describe an emotion may increase while the emotional episode being reported remains comparable in intensity. An adult's response to a question about autonomy may depend on institutional responsibilities unavailable to an adolescent. Longitudinal comparisons must therefore examine item meaning, response demands, and changing opportunities as well as numerical structure. Putnick & Bornstein (2016) reviews evidence relevant to comparisons across age and time; Clark & Watson (2019) emphasizes that validation samples and item wording influence construct interpretation from the outset. Development is an object of investigation and a source of possible measurement reorganization.
Culture enters measurement through language, values, institutional practices, norms of disclosure, and the contexts in which behaviors acquire meaning. A translated item may preserve dictionary semantics yet change its interpersonal implications. The same behavioral indicator can describe cooperation in one environment and conformity to coercive authority in another. AERA, APA, & NCME (2014) explicitly considers testing people with different linguistic backgrounds, and Sireci (1998) supplies methods for making content-domain judgments open to empirical scrutiny. Researchers should combine translation review with cognitive interviewing, local substantive collaboration, and tests of expected relationships. A measurement claim must specify whether the construct is proposed as universal, population-specific, or comparable only along selected dimensions.
The operational result is a graded account of equivalence. Some item responses may be comparable after adjustments; others may require different indicators with shared theoretical anchors; still others may indicate that the original construct needs revision. Kane (2013) provides a language of warrants and rebuttals suited to this process. Researchers should avoid making broad cultural conclusions from statistically convenient samples or treating Western questionnaire categories as automatic conceptual universals. They should document who defined the construct, whose experiences supplied its examples, and who participated in interpreting anomalies. Contextual validity is established by testing the connection between meaning and observation under the conditions where claims will be made.
Reliability, Stability, Responsiveness, and Interpretive Limits
Reliable measurement refers to consistency under relevant conditions and error models. A construct validity argument may require stable discrimination among individuals, sensitivity to within-person change, consistency among raters, or comparability across occasions. The appropriate criterion depends on whether the attribute is theoretical as a trait, state, event, or developmental process. Cook & Beckman (2006) explains that reliability is necessary for defensible interpretations without substituting for evidence of what a score represents. High consistency can arise from common wording, repetitive items, shared method effects, or stable nuisance variables. A consistently wrong interpretation remains consistently wrong. The evidentiary task is to establish which sources of variation should be expected and which should be treated as measurement error.
Test–retest association is particularly interpretable only when the construct should be stable over the interval studied. A measure of current pain, fatigue, or mood may appropriately change over hours; a putatively enduring disposition may require stability across longer periods. Intervention responsiveness also requires a theory of what changes first and which indicators should respond. Clark & Watson (2019) cautions that maximizing item homogeneity can narrow representation, while Flake & Fried (2020) shows how unexplained measure modification undermines cumulative inference. A strong longitudinal validation program separates true change from shifts in response thresholds, practice effects, regression to the mean, and changes in who remains in the sample.
Interpretive limits become especially important in clinical or institutional settings. A symptom scale may be suitable for monitoring an aggregate trend yet provide insufficient evidence to classify individuals; a reliable decision rule still requires calibrated threshold performance and evidence about downstream outcomes. AERA, APA, & NCME (2014) directs attention to the proposed use and the quality of its supporting argument. The statistical derivation of reliability coefficients, generalizability components, and item precision belongs to the neighboring psychometric measurement dossier. Within construct validation their role is to determine what the measurement can reasonably sustain, at which temporal scale and for whom.
Consequences, Decisions, and the Social Life of Scores
Psychological scores influence the settings that produce them. Classroom placement, hiring, access to services, research classification, and professional evaluations can change incentives, self-understanding, and opportunities. Messick (1995) argues that the social consequences of interpretation and use belong within a comprehensive validity inquiry. Kane (2013) likewise distinguishes arguments for score meaning from arguments for decisions based on those scores, requiring that consequential assumptions be evaluated. The relevant question is how particular interpretations and decision rules behave in real contexts, including their foreseeable effects on different populations. The scientific analysis should specify the decision system and its counterfactual alternatives, rather than infer all consequences from the psychometric scale alone.
Consequences require careful causal reasoning. If a screening procedure disproportionately excludes a population, the disparity might arise from the construct being assessed, unequal opportunity to acquire the tested skills, measurement noninvariance, threshold choice, or unequal downstream support. An observed disparity is important evidence for further investigation, while its specific explanation needs independent testing. AERA, APA, & NCME (2014) includes guidance for fairness and intended use; Vandenberg & Lance (2000) provides tools for examining whether cross-group measurement assumptions are plausible. Qualitative investigation can identify forms of harm or benefit omitted from standard outcomes. Outcome studies should report whose interests define success and how benefits and burdens are distributed.
High-stakes applications require a chain of justification from construct definition to observed score, prediction, threshold, action, and consequence. Weakness at any stage can invalidate the intended use even when other stages are well supported. Cook & Beckman (2006) applies the modern validity framework to instruments used in clinical practice and research. A transparent report should separate established psychometric properties from claims requiring prospective outcome or implementation evidence. Such distinctions protect both science and decision makers from treating a familiar questionnaire name as evidence of clinical authority. A decision can be evaluated ethically and empirically while recognizing that some questions about fairness depend on normative commitments that statistics alone cannot resolve.
Empirical Audits of Construct Validation Practice
The strength of construct validity research becomes visible when established instruments are audited with methods beyond those used in their original development. Flake et al. (2017) reviewed a representative set of studies in social and personality psychology and reported that evidence for both existing and newly created scales was often limited, with coefficient alpha frequently serving as the principal psychometric report. This is a methodological observation about the sampled literature rather than a universal estimate of invalid instruments. It exposes an asymmetry in research practice: theoretical conclusions routinely depend on scores whose interpretation receives less scrutiny than the subsequent hypothesis test. Transparent documentation of instrument selection, changes, and competing interpretations is therefore essential for cumulative scientific progress.
An especially informative audit is Hussey & Hughes (2020), which examined data comprising 144,496 experimental sessions for 15 commonly used questionnaires representing 26 scales. The published summary reports that 88% appeared to show good properties when assessed by the common practice of considering internal consistency, whereas only 4% met the study's criteria under a wider battery spanning consistency, immediate and delayed retest, factor structure, and age- and gender-group invariance. These percentages describe that sample, those tests, and the authors' specified evaluation rules. They should not be reinterpreted as population prevalence of invalid psychological measures or as a decisive finding that the scores have no useful applications.
The audit demonstrates why validation results are sensitive to the chosen evidence portfolio. Flake & Fried (2020) describes questionable measurement practices that include opaque item removal, undocumented modifications, and selective disclosure of favorable analyses. An appropriate corrective is an auditable measurement record: intended construct, item pool, instructions, scoring method, analytic decisions, available raw or appropriately deidentified data, reported failures, and justification for each intended use. These records allow subsequent researchers to distinguish a legitimate adaptation from a nominal replication involving a substantively different procedure. Measurement transparency is a condition for evaluating a study's claims, not merely a convention for formatting methods sections.
Experimental Manipulations and Causal Validation
A strong construct validation program can use intervention experiments to test whether purported indicators respond to hypothesized mechanisms. Researchers can manipulate task demands, environmental constraints, feedback, or instructional framing to investigate what actually causes a score. If a claimed attention measure changes dramatically after a trivial change in keyboard layout, interface skill becomes a plausible alternative explanation. If a construct theory predicts a specific pattern under a mechanism-targeting intervention, evidence is stronger when independent indicators change as expected while theoretically unrelated measures remain relatively stable. Borsboom et al. (2004) emphasizes the attribute-to-observation process, while Kane (2013) demands explicit support for the assumptions underlying a score interpretation.
Experimental contrasts do not eliminate all ambiguity. Manipulations may influence several processes simultaneously, participant strategies may vary, and the construct itself may be multidimensional. A successful manipulation check can be contaminated by the same response format as the outcome. Researchers should specify mechanism assumptions, choose comparison tasks with distinct nuisance demands, and where feasible include multiple outcome modalities. Campbell & Fiske (1959) supports triangulation across methods; Podsakoff et al. (2003) explains why common method characteristics must be considered. Studies should report manipulation success and failure separately from the inferential conclusion that a latent attribute was measured. Causal designs strengthen validation only insofar as their identification assumptions are credible.
The distinction between construct validity and experimental replicability is illustrated by multilaboratory work. Hagger et al. (2016) conducted a preregistered replication of a widely used sequential-task ego-depletion paradigm. The reported findings raised questions about robustness under the tested protocol, but interpretation also depends on whether the tasks genuinely instantiate the proposed self-control process and whether alternative paradigms test different mechanisms. A null or small effect under a specified operationalization does not settle every claim about self-control. Nosek et al. (2022) emphasizes the significance of replication, robustness, and reproducibility for cumulative psychological science. Construct and operational questions should be specified jointly when designs are reproduced.
Measurement, Replication, and Cumulative Psychological Science
Replication requires preservation or deliberate variation of the relevant inferential target. Two laboratories may describe the same construct while using different tasks, translations, instructions, recruitment channels, and scoring rules. A mismatch can reflect sampling variation, methodological moderation, or failure of either operationalization to support the proposed common interpretation. Flake & Fried (2020) emphasizes that measurement transparency enables researchers to assess whether nominal replications truly test equivalent claims. Nosek et al. (2022) situates reproducibility and replicability within the broader practices needed to distinguish stable phenomena from fragile findings. Construct validation must therefore be integrated into replication design rather than relegated to a short justification of the chosen instrument.
One useful approach is a planned network of replications in which the target hypothesis is tested using multiple operationalizations of the same construct. Preregistered expectations specify which results should generalize across instruments, which should depend on method, and what level of divergence would challenge the proposed theoretical identity. Westen & Rosenthal (2003) illustrates how theoretically expected correlation profiles may be quantified, while Campbell & Fiske (1959) provides the classic cross-method logic. Different operationalizations may reveal legitimate facets of a broad construct; the aim is to determine which substantive claims survive and whether a higher-level construct explains observed commonalities better than narrow task-specific accounts.
Meta-analytic synthesis must attend to what each included measurement actually represents. Combining studies that all use a familiar label can produce a precise pooled coefficient for a poorly identified mixture of constructs. Analysis should code instrument versions, administration modes, participant populations, operational definitions, reliability, and available evidence of structural or cross-group equivalence. Flake et al. (2017) illustrates why reported validity information may be insufficient, and Nosek et al. (2022) discusses the broader difficulty of building cumulative conclusions from inconsistently reproducible findings. The resulting evidence grade should describe both the precision of a statistical synthesis and the defensibility of treating the underlying measurements as instances of one phenomenon.
Clinical, Educational, and Organizational Interpretation
In clinical research, a scale of depressive symptoms can summarize reported experiences, track change, support case finding, or contribute to a diagnostic assessment. These uses differ in their evidentiary requirements. Clinical diagnosis depends on a comprehensive assessment and applicable diagnostic rules, while a self-report sum score is a sampled observation with measurement error. Cook & Beckman (2006) applies modern construct validity theory to medical and psychological instruments and describes content, response, structure, relations, and consequences as intersecting sources of evidence. AERA, APA, & NCME (2014) requires that test interpretations be justified for their specific purposes. A study supporting group-level association does not alone establish safe individual-level classification.
Educational and organizational testing faces analogous questions when an instrument influences opportunity. A selection score may predict training performance yet be affected by test preparation, socioeconomic access, language, and task familiarity. Structural equivalence across groups is relevant but cannot substitute for studying access, incentives, and decision consequences. Vandenberg & Lance (2000) examines the rationale for measurement invariance before cross-group inference; Messick (1995) connects score meaning to the consequences of use. Construct validity should be assessed alongside calibration, decision thresholds, monitoring of subgroup performance, and prospective evidence that the implemented rule serves the stated purpose. These additional studies may change what counts as an acceptable interpretation.
Evidence from one clinical or institutional setting requires careful transport to another. Measures designed among English-speaking university participants may not preserve their intended meaning in pediatric care, diverse communities, or high-stakes employment. Putnick & Bornstein (2016) documents the challenges of group and temporal equivalence; AERA, APA, & NCME (2014) emphasizes accessibility and intended use. The relevant clinical editorial task is to describe what a measurement supports and where further qualified evaluation is required. This monograph concerns scientific interpretation of assessments and does not issue individualized diagnostic instructions, prescribe treatment, or claim that a particular psychometric instrument has regulatory approval.
Digital Behavioral Traces and Artificial Systems
Digital environments expand the range of candidate indicators: interaction timing, navigation sequences, conversational records, device use, text production, and performance on adaptive tasks. These signals can be measured precisely while leaving their psychological interpretation open. A shorter response latency may reflect expertise, low cognitive load, automation, accessibility technology, or strategic behavior. Construct validity requires identifying the target capacity, specifying the conditions that link that capacity to the trace, and testing predictions that separate competing explanations. Borsboom et al. (2004) makes the causal connection central, while Flake & Fried (2020) demonstrates why transparent instrumentation and analytic choices are indispensable when multiple plausible variables can be extracted from the same observations.
In human–artificial research, claims about agency, confidence, trust, attachment, or mental-state attribution are especially dependent on precise referents. A participant's score on perceived trust in an artificial assistant measures a human appraisal under particular interaction conditions; it does not, without an additional research argument, establish a mental attribute of the system. Likewise, a benchmark score for an artificial agent may measure task competence, memorized patterns, tool access, or test contamination in different proportions. Kane (2013) motivates explicit interpretation-and-use arguments, and AERA, APA, & NCME (2014) provides a transferable principle that intended inferences require evidence suited to their population, task, and administration. These issues define new validation programs rather than automatically certifying older constructs.
Research in the Artificial Era can operationalize artificial psychological organization as a theory-guided object through observed regularities, memory persistence, context-sensitive adaptation, and mechanisms of response generation. Such a proposal must distinguish philosophical definitions from empirical tests and from verified conclusions. Validation would require diverse task families, perturbation tests, repeatability across contexts, examination of shared data leakage, and evidence that proposed constructs explain phenomena beyond trivial baselines. Campbell & Fiske (1959) and Nosek et al. (2022) supply methodological principles applicable to these investigations. Artificial systems invite fresh questions about measurement objects and contexts, but the authority of a psychological construct still rests on the strength and relevance of its evidence.
Limits of Statistical Fit and the Interpretation of Latent Variables
Psychometric models offer powerful summaries of patterns among observed responses, but model adequacy must be interpreted through assumptions about the phenomenon. A latent factor can account for covariance even when its indicators are driven by common response habits, shared environmental exposures, or methodological artifacts. Borsboom et al. (2004) articulates a causal conception of the attribute-to-score relationship, while Clark & Watson (2019) explains how construct specification, item choice, and tests against nearby constructs shape scale development. An elegant one-factor solution does not establish that a psychological entity has been identified. Researchers should distinguish statistical identifiability, empirical model fit, theoretical interpretability, and the validity of concrete applications, examining each through evidence suited to its question.
The problem of alternative models is intrinsic to inferential measurement. One dataset may be compatible with several structures, including correlated factors, a hierarchical organization, response-style effects, and dependence introduced by shared item wording. When investigators select among them after seeing the data, model comparison becomes part of an exploratory theory-building process. Flake et al. (2017) documents the limitations of routine validation reporting, and Hussey & Hughes (2020) shows that conclusions can differ markedly when additional structural and invariance tests are applied. Validation therefore benefits from independent samples, withheld items, plausible alternative models, diagnostic residual checks, and explicit statements about underdetermination. Explanatory commitment should not exceed what the research design can discriminate.
The status of a latent variable is an empirical and philosophical issue. A latent variable may model a real mechanism, summarize a behavioral propensity, or serve as a useful statistical abstraction whose interpretation is limited to certain observations. Cronbach & Meehl (1955) demands testable theoretical relations; Kane (2013) asks that the inferential meaning of a score be defended. Researchers should articulate which kind of claim is intended and what would count as contrary evidence. A flexible model is useful when it supports bounded predictions, yet it may be weak evidence for broad ontological conclusions. The proper result of validation is an increasingly precise account of what is known and how it is known.
Building an Explicit Interpretation-and-Use Argument
A practical validation argument begins by stating the intended construct in language that can be operationalized and challenged. The investigator then specifies the observed data, scoring rules, generalizations across items or occasions, extrapolations to the target domain, and any proposed decisions. Each transition contains assumptions that can be tested. Kane (2013) demonstrates the value of this argument-based orientation, while AERA, APA, & NCME (2014) establishes the expectation that evidence and theory support the specified interpretation and application. The structure prevents a common failure: gathering favorable evidence for an instrument without explaining which scientific claim the evidence is supposed to authorize. The argument should remain open to revision as evidence accumulates.
A validation matrix can record the claim, alternative explanation, available study, observed result, population, limitation, and necessary next test. For content claims, evidence may come from structured domain review and independent item evaluation Sireci (1998). For response-process claims, interviews and controlled task variations reveal competing interpretations. For internal structure, independent model tests and invariance analyses probe specified dependencies Putnick & Bornstein (2016). For external relations, theoretically registered predictions and cross-method designs test nomological implications Campbell & Fiske (1959). For consequential claims, prospective evaluation of actions and outcomes is appropriate Messick (1995). This arrangement does not assign identical weight to every source; it matches methods to scientifically consequential uncertainties.
An interpretive claim should also carry explicit scope conditions. These include language, age range, administration medium, testing stakes, scoring version, relevant disabilities, and contexts that could change response generation. Evidence from one condition should be extended only with a plausible transport argument or new validation studies. Cook & Beckman (2006) frames this as a practical issue across medical and psychological assessment, and Kane (2013) explains why ambitious inferences require more extensive support. A well-maintained dossier of score interpretations is therefore a living research record, documenting changes as new data challenge assumptions. The publication of a questionnaire and the publication of its validity evidence are separate scientific events.
Evaluating Evidence Quality and Reporting Uncertainty
The quality of validity evidence depends first on the claim and design. Content review provides evidence about representational coverage; interviews provide evidence about experienced item meaning; structural tests evaluate patterns under chosen statistical models; experimental manipulations probe causal sensitivity; longitudinal designs examine stability and change; prospective outcome studies investigate predictive and consequential relations. The same study rarely settles all these questions. AERA, APA, & NCME (2014) organizes validity evidence into several sources for precisely this reason, and Kane (2013) shows how the adequacy of evidence depends on the proposed use. Critical synthesis should identify the method used, the assumptions that remain untested, and whether rival explanations were addressed.
Evidence grades should consider sampling, design credibility, transparent measurement decisions, precision, preregistration where relevant, independent replication, and heterogeneity across settings. Flake & Fried (2020) identifies questionable measurement practices that make even a large reported association difficult to interpret. Nosek et al. (2022) explains why reproducibility and replicability support cumulative scientific progress. A large single-platform dataset can offer precise estimates within its recruitment process while failing to establish generality; a small carefully designed mechanism experiment may provide an informative causal contrast without estimating population prevalence. Such distinctions should be retained in scholarly conclusions rather than flattened into a single certificate of validity.
Uncertainty should be reported at the level of the inference. Authors may have strong evidence for a scale's content coverage and response comprehension, moderate evidence for factor structure, and little evidence for predictive decisions in a particular population. Hussey & Hughes (2020) illustrates how conclusions can change with more extensive evaluation. Messick (1995) emphasizes that interpretation and use remain connected to broader implications. A defensible report states which uses are supported, which require further validation, and which encounter material contrary evidence. This produces an interpretable scientific record rather than an unqualified statement that an assessment is valid, reliable, or universally suitable.
Open Problems and a Cumulative Research Agenda
Construct validity research faces enduring theoretical problems. Psychological constructs may overlap causally, change through development, depend on social practices, or acquire meaning through interactions between person and environment. Such properties complicate attempts to identify a stable measurement target from a single test score. Borsboom et al. (2004) presses the question of what makes an attribute causally relevant to measurement, while Cronbach & Meehl (1955) situates an attribute within a revisable theoretical network. Future research should connect these traditions through studies that test response-generating mechanisms alongside explicit predictions about external relations. Better theory can reduce the number of arbitrary operational choices, but it must remain corrigible when experiments reveal divergent patterns.
Practical research priorities include independent item validation, planned cross-method studies, routine checks for longitudinal and cross-cultural equivalence, and public reporting of measure modifications. Flake et al. (2017) and Flake & Fried (2020) identify shortcomings in current measurement practice that such steps can address. Putnick & Bornstein (2016) shows the importance of transparent reporting when group comparisons depend on invariance assumptions. Large collaborative datasets can provide statistical power for testing heterogeneity, provided samples represent the populations to which conclusions are applied. Multilaboratory studies should vary tasks deliberately to distinguish replicated constructs from replicated procedures. The value of openness lies in allowing alternative analyses to confront the same inferential claim.
New settings produce new validation challenges: adaptive instruments, behavioral trace data, human–artificial partnerships, multimodal measurements, and systems whose behavior changes after deployment. An assessment developed for one algorithm or interface may require fresh evidence after the environment changes. Kane (2013) provides a framework for evaluating interpretations and uses under altered conditions, and Nosek et al. (2022) supplies principles of robust and reproducible inquiry. Fundamental research should register which theoretical assumptions are unchanged, which are newly tested, and which are speculative. In the Artificial Era the question of what counts as an indicator of agency or cognition becomes more consequential, and it retains the same scientific obligation to distinguish observed capacities from inferred psychological organization.
Conclusion
Construct validity is the cumulative scientific justification for interpreting specific observations as evidence of a specified psychological construct in a specified context. Its historical foundations in Cronbach & Meehl (1955) and Campbell & Fiske (1959) established the importance of theoretical networks, convergent predictions, and discriminating tests. Messick (1995) integrated content, structure, substantive processes, external relations, generalizability, and consequences into a comprehensive interpretive framework. Kane (2013) made the inference chain explicit through a structured validation argument. Together these contributions explain why validity inquiry continues throughout the life of an instrument and why evidence obtained for one population or use cannot automatically sustain another.
Contemporary evidence reveals recurring threats to this project. Narrow content sampling, ambiguous response processes, selective item revision, overinterpretation of factor structure, method effects, and group non-equivalence can all produce apparently persuasive scores with weak interpretive foundations. The empirical critiques of Flake et al. (2017), Hussey & Hughes (2020), and Flake & Fried (2020) demonstrate the importance of evaluation beyond a single reliability statistic. Clark & Watson (2019) provides practical guidance for scale construction, and Putnick & Bornstein (2016) clarifies the demands of cross-group and temporal comparison. The methodological synthesis therefore supports diversified evidence, transparent decisions, independent replication, and direct confrontation with rival explanations.
A complete validation program states what its scores mean, how that meaning is supported, for whom the claim applies, which decisions it can justify, and what evidence would change the conclusion. It distinguishes conceptual definitions from indicators, consistency from interpretation, prediction from explanation, and proposed theory from established empirical results. AERA, APA, & NCME (2014) sets the professional benchmark for such work, while Borsboom et al. (2004) and Nosek et al. (2022) extend its causal and cumulative-science implications. Construct validity remains a central method by which psychological science maintains a stable relationship between theoretical language and empirical reality, including where new behavioral forms and artificial systems demand new operational definitions and tests.
References
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://ncme.org/resources/books/testing-standards/
- Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061
- Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
- Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626
- Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119(2), 166.e7–166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036
- Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
- Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
- Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063
- Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brandt, M. J., Brewer, G., Bruyneel, S., Calvillo, D. P., Campbell, W. K., Cannon, P. R., Carlucci, M., Carruth, N. P., Cheung, T., Crowell, A., De Ridder, D. T. D., Dewitte, S., … Zwienenberg, M. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546–573. https://doi.org/10.1177/1745691616652873
- Hussey, I., & Hughes, S. (2020). Hidden invalidity among 15 commonly used measures in social and personality psychology. Advances in Methods and Practices in Psychological Science, 3(2), 166–184. https://doi.org/10.1177/2515245919882903
- Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
- Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
- Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73, 719–748. https://doi.org/10.1146/annurev-psych-020821-114157
- Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
- Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
- Sireci, S. G. (1998). Gathering and analyzing content validity data. Educational Assessment, 5(4), 299–321. https://doi.org/10.1207/s15326977ea0504_2
- Sireci, S., & Padilla, J.-L. (2014). Validating assessments: Introduction to the special section. Psicothema, 26(1), 97–99. https://doi.org/10.7334/psicothema2013.255
- Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002
- Westen, D., & Rosenthal, R. (2003). Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology, 84(3), 608–618. https://doi.org/10.1037/0022-3514.84.3.608
- Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4(2), 223–233. https://doi.org/10.1080/15434300701375832