Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

Psychometric Measurement of Mental Processes

Abstract. Psychometric measurement of mental processes connects observable responses to theoretical claims about cognition, affect, motivation, and other psychological attributes. This fundamental research monograph traces the history of quantitative psychological measurement and critically synthesizes classical test theory, coefficient alpha and omega, generalizability theory, exploratory and confirmatory factor models, bifactor interpretation, item response theory, adaptive assessment, differential item functioning, measurement invariance, and longitudinal responsiveness. It examines how sampling, construct specification, response mechanisms, task reliability, score precision, and statistical identification influence the interpretation of psychological differences. Primary methodological studies, official testing standards, and empirical audits of cognitive tasks and questionnaire scales provide distinct forms of evidence. The synthesis establishes the separate roles of reliability, validity, dimensionality, conditional information, fairness, and context-specific comparability. It identifies unresolved problems in measuring change, transporting scores across populations and technologies, and assessing Artificial systems without assuming equivalence to human psychological constructs. The central conclusion is that meaningful psychometric inference requires an explicitly defined attribute, a defensible response-generating model, reproducible error estimation, and independent evidence for each proposed interpretation and use.

Keywords: psychometrics; mental processes; psychological measurement; classical test theory; measurement error; reliability; coefficient alpha; coefficient omega; generalizability theory; factor analysis; confirmatory factor analysis; structural equation modeling; bifactor model; item response theory; adaptive testing; measurement invariance; differential item functioning; construct validity; longitudinal measurement; Artificial Era

Psychometric Measurement as an Inference About Mental Processes

Psychometric measurement of mental processes is the construction and evaluation of quantitative inferences about cognition, affect, motivation, and other psychological functions from observed responses or behavior. The observable datum may be an item endorsement, choice, reaction time, accuracy, examiner rating, or longitudinal report. The measured attribute is represented through a model linking those observations to a theoretically specified variable. Scores therefore acquire scientific meaning through the joint specification of construct, response mechanism, population, administration, scale, and intended interpretation. The AERA, APA, & NCME (2014) testing standards provide an encompassing framework for evaluating the evidentiary warrant of interpretations and uses, while Messick (1995) situates score meaning within a unified validity argument. Even when two procedures output the same number, their units, uncertainty, and inferential boundaries may differ substantially.

A measurement system has several separable layers. A substantive theory defines the process, a sampling scheme selects situations or indicators, an observation protocol records responses, a scoring rule combines observations, and a statistical model describes variation attributable to persons, occasions, items, and error. Finally, an interpretation connects the estimated quantity to an empirical claim. Failure at any layer can make a precise numerical output scientifically ambiguous. A factor score estimated from inconsistent indicators, for example, can have a small conditional standard error under a misspecified model. Borsboom et al. (2004) gives a causal account of what it means for an attribute to influence test responses; its emphasis on response-generating mechanisms complements the broader validation inquiry rather than independently demonstrating that a proposed attribute exists.

The central questions of this dossier are how scores represent mental processes, which assumptions support their comparability, how different error sources are identified, which psychometric models perform adequately for specific observations, and what independent empirical evidence warrants interpretation. This is a critical narrative research synthesis based on original methodological articles, books, public standards, and illustrative empirical investigations; it is not a prospectively registered systematic review or a statistical meta-analysis. Its scope concerns the technical foundations of score interpretation, reliability, factor structure, item response models, generalizability, invariance, differential item functioning, and change measurement. Instrument design and norm construction are addressed where they affect the measurement model, while the separate research object Psychological Test Development, Standardization, and Norming governs complete production and deployment procedures.

Historical Origins of Quantitative Psychological Measurement

Quantification of sensation and judgment developed through nineteenth-century psychophysics, later joined by quantitative individual-differences research. The historical importance of these traditions is their conversion of a psychological question into operations that yield repeatable observations. Early investigators confronted a problem still familiar in contemporary testing: observed variation contains both changes of psychological interest and variation introduced by stimuli, observers, tasks, and recording conditions. Subsequent correlational approaches investigated shared individual differences across multiple performances. These developments should be distinguished from the much stronger claim that a single numerical scale faithfully describes every aspect of a mental faculty. Modern standards require the scale interpretation to be specified and corroborated in the context of the proposed use. AERA, APA, & NCME (2014) expresses this continuing evidentiary obligation.

Classical measurement research gained an enduring formal instrument when Cronbach (1951) related the internal structure of tests to coefficient alpha and item covariance. Midcentury psychometrics subsequently expanded from reproducibility toward the interpretation of theoretical attributes. Cronbach & Meehl (1955) framed construct validation as the testing of a network of theoretical relations, while Campbell & Fiske (1959) supplied the multitrait–multimethod strategy for distinguishing a construct from the method used to assess it. These milestones changed psychometrics from arithmetic applied to questionnaires into a research program about the relation between theoretical variables and observable responses. Their continuing value is methodological: they direct attention to the hypotheses a score must satisfy, rather than equating scientific success with a single satisfactory statistic.

Later developments increased both model flexibility and the burden of justification. Probabilistic response modeling associated with Rasch (1960) and the graded-response investigations of Samejima (1968) enabled researchers to examine which items are informative at different attribute levels. Generalizability theory formalized multiple design facets Brennan (2001). Factorial invariance gave mathematical expression to comparability across populations Meredith (1993). Modern psychometric software can fit all of these families, but computational availability does not select the right model or establish validity. Their historical sequence illustrates a widening recognition of contingent score interpretation, multidimensionality, and the need for independent evaluation of model assumptions. Contemporary practice should retain both the precision of formal modeling and the substantive reasoning that made these techniques necessary.

The Latent Construct, Indicator, and Response Process

A construct such as working-memory capacity, emotion regulation, or threat sensitivity is defined by a theoretical account of how a psychological process operates. Its operational indicators can be reflective, when an assumed latent variable helps generate responses, or composite and formative, when selected components are assembled into an index by a rule. These architectures have different implications. In a reflective factor model, indicator covariance is explained by shared latent influences subject to residual structure; in a formative index, components may be important precisely because they describe distinct domains. Treating every index as a reflective scale can misstate the construct. Borsboom et al. (2004) emphasizes the causal path between measured attribute and response, while Messick (1995) calls for substantive and structural evidence about meaning.

Response processes introduce systematic sources of difference beyond the target construct. A rating about anxiety depends on comprehension, autobiographical recall, response style, social context, and interpretation of the response anchors. A time-pressured cognitive task is influenced by speed–accuracy tradeoffs, practice, fatigue, hardware latency, and strategy. These influences sometimes belong to the intended construct and sometimes contaminate its assessment. A rigorous model identifies which processes belong in the construct and which should be separated or controlled. Cognitive interviewing, protocol analysis, experimental manipulation of instructions, and comparisons across formats can interrogate these mechanisms. The AERA, APA, & NCME (2014) framework recognizes response-process evidence as one contributor to validity arguments, alongside content, internal structure, relations to external variables, and consequences.

The same observable pattern can support multiple substantive explanations. High correlations between attention scores may arise from shared executive demands, motor speed, education, common task format, or restricted samples. Factor analysis is therefore a tool for evaluating modeled covariance, not an automatic discovery of psychological ontology. Eisenberg et al. (2019) illustrates how large-scale batteries and data-driven analyses can reconsider taxonomies of self-regulation, while remaining dependent on the coverage and properties of chosen tasks. An interpretable measurement program makes competing explanations explicit, collects discriminating evidence, and records uncertainty about the construct's boundaries. This approach also gives cross-domain inquiries a stable starting point: a score of an artificial system and a score of a human may coincide numerically while requiring different response-process accounts before any psychological analogy follows.

Validity as an Evidence-Based Interpretive Argument

Validity concerns the adequacy of evidence and theory supporting specific interpretations and uses of scores. It is accumulated through a research program, rather than transferred wholesale from a test's name or reputation. Cronbach & Meehl (1955) placed construct validity at the intersection of theoretical explanation and empirical tests. Messick (1995) subsequently integrated content, substantive, structural, generalizability, external, and consequential considerations into an account of score meaning. A scale can show stable administration and still fail to represent its proposed domain. Conversely, an experimental process measure may be sensitive to a manipulation while having limited value for ranking stable individual differences. Each claimed use demands its own inferential bridge.

The Campbell & Fiske (1959) multitrait–multimethod framework makes one bridge especially clear. If the same trait is measured using questionnaires and behavioral observations, convergent evidence should persist across methods, while discriminant evidence should separate it from genuinely different traits. A strong correlation between two questionnaires written in similar language may reflect shared response style rather than psychological identity. Evidence from known-group comparisons, interventions, longitudinal prediction, cognitive interviews, and external criteria may further constrain competing accounts. A validation argument remains revisable when sampling, technology, linguistic context, or intended decisions change. The decisive question is which claims the observed pattern justifies and which alternative pathways remain empirically plausible.

Recent examinations of measurement practice expose the cost of abbreviated validation. Flake et al. (2017) documented frequent reliance on coefficient alpha and insufficient reporting of construct-validation evidence in sampled social and personality research. Flake & Fried (2020) described questionable measurement practices, including undisclosed choices about indicators, scoring, and analytic specifications, that can alter conclusions. Hussey & Hughes (2020) evaluated 15 commonly used measures with several complementary psychometric checks and reported substantial discrepancies between apparent quality based on internal consistency alone and quality under more demanding criteria. Such findings are field- and sample-specific, not proof that every established questionnaire is invalid. They demonstrate why validity evaluation must follow the actual claims made with scores and must be independently reproducible.

Data Types, Scaling, and the Logic of Comparison

The numerical appearance of a response does not settle its measurement level. Binary correct–incorrect records, ordered response categories, bounded accuracy proportions, reaction times, count outcomes, continuous signals, and repeated subjective ratings imply different likelihood functions and error structures. An ordered category such as 'often' may reflect increasing intensity without guaranteeing equal distances between adjacent options. A reaction-time difference may be sensitive to skew, trimming rules, and task conditions. These properties affect parameter estimation and substantive interpretation. The item-response framework introduced in Embretson & Reise (2000) explicitly models binary and ordered-category data; the multidimensional possibilities reviewed in Reise (2012) further show how apparently simple total scores may conceal heterogeneous information.

Meaningful comparison requires a common scale interpretation. Within a fixed task, a raw total can be sufficient for a limited descriptive purpose. Between instruments, languages, time periods, or groups, linking and invariance must be investigated before differences are treated as changes in the same attribute. Standard scores and percentiles locate observations relative to a defined reference distribution, not to an immutable universal psychological unit. Latent scores additionally depend on the fitted model and its identification constraints. Millsap (2011) develops methods for examining bias and comparability across groups; Putnick & Bornstein (2016) reviews how measurement equivalence is tested and reported across developmental research. A scale transformation can preserve within-sample ordering while changing the meaning of a one-unit difference.

The units of analysis also require care. Item-level observations are nested in persons, trials in tasks, ratings in raters, and repeated observations in time. Ignoring clustering can understate uncertainty and confuse contextual variability with individual differences. When a task is modified, a scoring rule is changed, or a software implementation alters timing, the same nominal score may cease to represent the same operation. Measurement practice therefore begins with a transparent description of observations and preprocessing decisions. It then models the appropriate sampling and dependence structure, tests plausible alternatives, and gives an uncertainty estimate aligned with the intended comparison. These principles will recur throughout classical, latent-variable, adaptive, and longitudinal approaches.

Classical Test Theory and the Decomposition of Score Error

Classical test theory begins from the expression X = T + E, in which the observed score X is modeled as a true-score expectation T plus error E under a defined conceptual repetition of measurement. The true score is conditional on the relevant measurement procedure and hypothetical replications; it should not be mistaken for a permanently existing quantity that an investigator observes directly. Under standard assumptions, error is uncorrelated with true scores, and reliability is the proportion of observed-score variance attributable to true-score differences in a specified population and design. This representation helps explain why identical test instruments can have different reliability estimates in different samples. The AERA, APA, & NCME (2014) standards require attention to the precision appropriate for each interpretation rather than a universal certificate of reliability.

Three forms of error have different substantive implications. Random transient influences alter scores between equivalent observations, systematic nuisance processes shift responses in predictable directions, and model error arises when the assumed measurement relation is misspecified. Classical reliability formulas concentrate on reproducibility under designated assumptions and do not by themselves identify or correct systematic construct contamination. A standardized task could yield extremely stable scores because participants consistently misunderstand its instructions. The reliability estimate would describe stability of that output without validating the intended attribute. Cronbach (1951) formalized alpha from item covariances, providing a historically influential summary, but the interpretation of any estimated coefficient remains dependent on item sampling and the conditions under which responses were elicited.

The standard error of measurement translates reliability into an approximate statement about uncertainty on the observed-score metric. Under the usual homogeneous-error assumptions, SEM = SD(X) × sqrt(1 − r), where r denotes the reliability coefficient appropriate to the target score. This conditional calculation is useful for understanding why apparent differences between two observed scores may carry little information. The formula presumes a defined reliability model and should not be casually applied to heterogeneous error or untested transformations. Precision also varies across the score distribution under nonlinear and item-response models. Reporting score spread, standard errors, intervals, sample characteristics, and intended decision rules gives the user a more informative description than reporting reliability alone.

Internal Consistency, Alpha, and Assumption-Sensitive Alternatives

Internal consistency concerns the extent to which item-level responses contribute dependable information to a specified composite within one administration. Coefficient alpha can be expressed as k/(k−1) times one minus the ratio of the sum of item variances to the total-score variance, with k representing the number of items. Cronbach (1951) established the coefficient's connection to test structure and item covariance. Because the covariance matrix also reflects the number and redundancy of items, a high alpha can result from repeated content and correlated residuals. It does not demonstrate unidimensionality, a coherent psychological mechanism, interpretability across populations, or responsiveness to an intended intervention.

Model assumptions matter. Under suitable tau-equivalence and uncorrelated-error conditions, alpha can correspond to reliability of a composite; substantial departures change its relationship to the target parameter. Sijtsma (2009) analyzed common interpretations and limitations, while Revelle & Zinbarg (2009) distinguished alpha, beta, omega, and other coefficients. Factor-based omega can be estimated from loadings and error variances when a plausible measurement model is fitted. Dunn et al. (2014) provides an accessible methodological account of omega estimation and interval reporting. Omega is not a magic replacement: if factors are misidentified, cross-loadings are omitted, or response categories are mishandled, its numerical value can also mislead.

Reporting internal consistency responsibly therefore requires an explicit composite, the item response format, estimated dimensionality, missing-data handling, confidence intervals, and enough detail to reproduce the calculation. A scale intentionally sampling distinct facets may require subscale coefficients or a multidimensional score model instead of a single total-score index. Very high consistency can indicate inadequate domain coverage as readily as desirable precision. The alternative coefficients proposed in Dunn et al. (2014) and discussed by Revelle & Zinbarg (2009) are most informative when selected from a credible account of the score-generating process. Investigators should compare estimates under substantively defensible assumptions and connect the resulting uncertainty to the magnitude of effects their research seeks to detect.

Temporal Stability, Rater Agreement, and Dependability

Test–retest reliability examines similarity of scores across administrations separated by an interval appropriate to the intended construct. A stable trait, a rapidly fluctuating state, and an intervention-sensitive process require distinct interpretations of change. Low temporal correlation can indicate random measurement error, authentic within-person variation, a changed context, learning, or a change in the measurement relation itself. Consequently the interval, practice exposure, stimulus set, attrition, and measurement conditions belong in the reported result. Enkavi et al. (2019) investigated long-term test–retest reliability across self-regulation measures and found that many widely used task-based measures were less stable than questionnaires, demonstrating the significance of explicitly specifying the individual-differences purpose.

When measurements depend on observers, inter-rater reliability and absolute agreement answer related but different questions. A correlation may remain high when one rater systematically scores everyone above another, whereas absolute-agreement indices respond to that difference. Shrout & Fleiss (1979) distinguished intraclass correlation formulations according to how raters and targets are sampled and whether interest concerns a single rating or an aggregate. The design must establish whether raters are treated as fixed or sampled from a population, whether the same raters assess all participants, and whether agreement about absolute values matters. Reporting only an unqualified 'ICC' conceals these decisions and makes reproducibility and application difficult.

Reliability is also a property of the inference population. Strong agreement among experienced coders viewing high-quality recordings cannot automatically be extended to inexperienced assessors using low-resolution clips. A longer test may improve precision when items supply independent information, but can worsen missingness or fatigue. Brennan (2001) provides the broader variance-component language for treating such conditions as explicit facets of a measurement design. For repeated measures of rapidly changing psychological functioning, investigators should estimate within-person variation and model temporal dependence rather than assuming all differences between sessions are noise. The question is whether the instrument can distinguish the psychological changes the study claims to investigate.

Generalizability Theory and the Design of Measurement Facets

Generalizability theory extends classical reasoning by recognizing multiple sources of score variation simultaneously. Person, task, rater, occasion, setting, and their interactions can be represented as facets in a variance-components model. A generalizability study estimates how much variation is associated with each source, and a decision study asks how precision would change if the number of tasks, raters, or observations were altered. Brennan (2001) develops this mathematical and design-based framework. The target universe of permissible observations must be stated: generalizing across two trained raters in one laboratory is different from generalizing across school systems, languages, task families, and years.

Relative decisions concern ordering people or units, whereas absolute decisions concern their deviation from a criterion or expected level. These purposes imply different definitions of error. A stable occasion effect shared by all examinees might leave rank ordering almost unchanged while distorting an absolute clinical threshold. Crossed designs, where each rater scores every task, and nested designs, where different raters encounter different participants, offer different opportunities to identify variance components. A large generalizability coefficient estimated in a narrowly sampled universe may provide little support for a broad claim. The conceptual contribution of Brennan (2001) is to make the intended universe and the sampling design part of the scientific meaning of precision.

Generalizability analyses are especially valuable for performance assessments, developmental observations, interactive tasks, and multimethod studies. Consider a cognitive battery in which an individual's apparent executive score varies by stimulus class, task order, examiner, and testing day. Adding items from the same class may improve apparent consistency while leaving between-class generalization unresolved. Sampling across independent tasks could be more useful, even if the composite's coefficient initially declines. An informative report describes the variance-component estimation method, sample adequacy, uncertainty intervals, missing cells, and decision consequences. The purpose is to allocate measurement effort to the error sources that matter, not simply to maximize one reliability index.

The Reliability Paradox in Cognitive Tasks

Experimental paradigms are often developed to elicit strong average effects: an incongruent condition slows responses relative to a congruent condition, or memory accuracy decreases with interference. The same design can produce relatively little stable variation among participants in the difference score used to represent an individual ability. Hedge et al. (2018) called attention to this reliability paradox: tasks optimized to demonstrate robust within-subject experimental effects may be poor instruments for ranking individual differences. A replicated group contrast answers whether an experimental manipulation systematically changes performance under defined conditions. Reliable individual-differences assessment requires a different set of variance and measurement properties.

Difference scores introduce additional problems when component measures are correlated and contain independent error. A participant's average response speed, strategic variation, and trial sampling can shape the magnitude of the subtraction. Within-session split-half consistency may overstate long-term stability when trial-to-trial associations do not persist across occasions. Enkavi et al. (2019) supplied direct retest evidence from a broad self-regulation battery, and Eisenberg et al. (2019) examined structural patterns across behavioral tasks and self-report measures. Together these empirical investigations show why conceptual labels such as impulsivity or inhibition cannot guarantee that multiple tasks share a stable measured attribute. Model-based task parameters are promising but require the same independent reliability evaluation as raw difference scores.

Improvement strategies depend on the scientific objective. Additional trials may reduce sampling error; hierarchical estimation may separate subject-level parameters from trial noise; carefully varied tasks can test transfer across paradigms; longitudinal designs can distinguish enduring differences from states. None of these interventions guarantees construct validity. A complex model can yield stable estimates of a nuisance process, while a simple task can be fit for investigating a sharply defined causal manipulation. The rational evaluation begins by deciding whether the intended output is a condition effect, an individual's standing, within-person change, or prediction of an external outcome. The research program developed by Hedge et al. (2018) makes that distinction essential to task selection and score interpretation.

Exploratory Factor Analysis and the Discovery of Covariance Structure

Exploratory factor analysis seeks a parsimonious account of associations among observed variables through latent dimensions and residual components. Its premise is that correlations can arise partly from shared underlying variation, with the precise structure of factors and residual dependencies requiring investigation. Factor retention, extraction methods, rotations, scaling, and the treatment of ordinal categories are modeling decisions that can change apparent dimensionality. A factor named 'executive control' is an interpretation of a pattern, not a measurement supplied directly by the covariance matrix. Reise (2012) reviews several alternative factor structures and explains why general and group dimensions may coexist in complex psychological measures.

An exploratory solution can be underdetermined by observed data. Equivalent rotations may describe the same covariance matrix, and differently specified models can reproduce very similar observed associations. Overextracting factors can turn random local dependencies into named constructs; underextracting factors can conceal theoretically important heterogeneity. Item wording, reverse-keyed response formats, method effects, and sampling restrictions can also create apparent dimensions. Exploratory procedures should therefore be combined with parallel evidence from item content, response processes, cross-sample stability, and substantive hypotheses. Flake et al. (2017) documents how insufficient construct-validation reporting leaves such choices opaque. Declaring that a statistically extracted factor is a psychological faculty requires independent tests of what produces that pattern.

The design of an exploratory factor study begins with a defensible item domain and a sufficiently varied sample, followed by a declared correlation metric, extraction method, factor-selection rationale, and scrutiny of residuals. Factor loadings and correlations should be interpreted together with uncertainty and potential cross-loadings. Replication in an independent sample tests whether the solution is a population-stable regularity rather than a sample-specific arrangement. Where indicators are highly skewed or have few response categories, ordinal models may better match the data than assumptions of continuous normal responses. Exploratory analysis has explanatory value when it narrows competing conceptual structures and generates predictions that later data can genuinely challenge.

Confirmatory Factor Analysis and Structural Equation Measurement

Confirmatory factor analysis expresses explicit hypotheses about which observed variables depend on which latent dimensions and what residual dependencies are allowed. Structural equation models extend this measurement component to relations among latent variables, observed predictors, and outcomes. The key advantage is an articulated model whose fit, parameter behavior, and implications can be inspected. A good global fit statistic alone cannot establish a unique true structure. Different models can fit similarly, and large samples may expose substantively minor deviations while small samples may fail to detect important misspecification. Reise (2012) details relationships among correlated-factor, hierarchical, and bifactor structures that can complicate attempts to identify one definitive representation.

Latent variables commonly require constraints fixing their scales, such as a reference loading or fixed variance. Standardized loadings describe relationships under those constraints and in the population studied. Residual covariances can represent substantive local processes, repeated wording, task overlap, or omitted variables. Modifying a model in response to fit indices without independent theory creates capitalization on chance. A substantive validity argument should therefore state its anticipated dimensions, inspect local fit and residual patterns, use data-independent hypotheses where possible, and evaluate alternatives. Messick (1995) places internal structure within a larger framework of evidence, while Flake & Fried (2020) highlights the inferential risks of undisclosed measurement decisions.

Measurement and structural components require separate scrutiny. A purported relation between a latent self-regulation factor and external behavior depends on both how the factor is identified and how the external criterion is measured. Common-method variance can create misleading associations when all observed indicators share the same format or rater. Multimethod evidence, prospective prediction, and task manipulations may help distinguish conceptual relations from measurement artifacts Campbell & Fiske (1959). When fitting models to ordinal response items, researchers should describe whether thresholds and appropriate estimators were used. Numerical precision is conditional on a chosen model, the assumed distribution, the available sample, and the missing-data process, and therefore cannot replace a substantive argument about meaning.

Bifactor Models, Dimensionality, and Score Interpretation

Bifactor models partition item covariance into a general dimension and one or more domain-specific dimensions under a specified pattern of constraints. Their attraction is the possibility of investigating whether a total score is dominated by shared variance or whether subscale scores retain substantial distinctive information. Reise (2012) explains historical bifactor developments and modern estimation approaches, including relations to earlier hierarchical decompositions. A model with a strong general factor may support a broad summary within a given sample, but it does not by itself establish a single causal mechanism underlying all items. Strong loadings can reflect broad response tendencies, common item presentation, or shared task demands as well as theoretically meaningful common functioning.

A bifactor solution also inherits challenges of identification and interpretability. Specific factors can be poorly determined when few indicators remain after the general factor is considered, and assumptions about orthogonality may conflict with substantive theory. Reporting total-score omega alongside hierarchical omega or other variance partitions can clarify which score interpretation is supported, yet such coefficients depend on the fitted factor structure. Revelle & Zinbarg (2009) emphasizes that the definition of reliability differs across candidate composite models. If a scale contains both general distress and symptom-specific domains, the validity of reporting a single overall score should be weighed against clinically and scientifically important distinctions among those domains.

Substantive evaluation compares bifactor results with plausible correlated-factor, second-order, and item-level alternatives. Cross-validation should assess whether the apparent structure survives independent data and changes in sample composition. Factor determinacy, estimated score uncertainty, and external validity are crucial when factor scores are used as individual measurements. Researchers should also check whether a model merely absorbs redundant item wording rather than explaining psychologically separable processes. Flake et al. (2017) cautions against treating internal structure as a one-time ceremonial test; validation accumulates as the use of scores expands. The interpretation of a dimensional structure is strongest when alternative models make different predictions about behavior, development, or intervention response.

Item Response Theory and the Information Supplied by Individual Items

Item response theory models the probability of an observed response as a function of a latent attribute and item-specific parameters. Under familiar dichotomous logistic models, item difficulty locates the attribute level at which responses become more likely, and discrimination describes how steeply response probability changes. More complex models may incorporate guessing, multidimensionality, response time, or local dependence. Rasch (1960) developed a foundational probabilistic measurement program, and Embretson & Reise (2000) surveys the assumptions, estimation strategies, and extensions that are especially relevant in psychological testing. Unlike a single global consistency statistic, IRT can characterize measurement precision across different levels of the attribute.

Item information is local: a task can sharply distinguish some attribute levels while offering little precision elsewhere. Conditional standard errors derived from a fitted item-response model can guide selection of items and interpretation of estimated person parameters. Yet a high information value is meaningful only when item parameter estimates are stable, the latent scale is adequately identified, and the item response function is substantively appropriate. Local independence is frequently assumed conditional on the modeled attribute; repeated stimuli, shared wording, and strategy clusters may violate it. Model fit should be inspected at the item and person levels, with attention to response categories, residual associations, and possible population differences Embretson & Reise (2000).

IRT does not automatically transform rating responses into universally comparable psychological units. The latent metric is established under model constraints and calibration data; linking to another item pool requires common anchors or equivalent design information and empirical checks. Item parameters can drift with language, population, technology, or administration. If the construct changes across contexts, improved estimation of a shared mathematical parameter can conceal a substantive comparability problem. Item-response research thus connects the precision question to both construct validity and measurement invariance. The long-term scientific value of an IRT bank rests on sustained calibration, independent validation, and transparent reporting of its domains and limits.

Ordinal Responses, Graded Models, and Adaptive Assessment

Many psychological instruments ask respondents to choose among ordered categories such as never, rarely, sometimes, and often. The spacing between categories is not automatically equal, and the response process may differ across items or groups. Samejima (1968) developed foundational methods for estimating latent ability from graded response patterns, providing a basis for modern polytomous IRT approaches. Category thresholds can describe points along a latent continuum where endorsement probabilities change. Their interpretation requires both model fit and a plausible account of how respondents use anchors. Collapsing categories after inspection of outcomes may affect dimensionality and item information, so the rationale and sensitivity of such modifications need to be reported.

Computerized adaptive testing selects subsequent items in response to previous answers, usually to increase precision or shorten administration. The instrument becomes a decision rule over an item bank rather than a single fixed form. Cella et al. (2010) describes an important implementation in the PROMIS initiative, which developed and tested standardized patient-reported outcome item banks intended to permit flexible yet comparable assessment. The claim of comparable scores rests on calibration, dimensionality, item functioning, and linkage; it cannot be obtained by adaptivity alone. Practical evaluation should compare scoring precision, item exposure, completion burden, nonresponse, and external validity against conventional fixed-form designs.

Adaptive designs present further scientific and operational risks. Maximum-information selection can overuse highly discriminating items, raise security and familiarity problems, and provide uneven content coverage. Termination at a chosen precision threshold requires safeguards for participants with atypical response patterns. Adaptive tests used repeatedly need evidence that exposure and learning do not alter the construct measured. In health-related measurement, Prinsen et al. (2018) highlights assessment of multiple measurement properties when synthesizing evidence for patient-reported outcome instruments. For general psychological processes, researchers should determine whether response selection modifies engagement, strategy, or affect in ways that change the meaning of the inferred score. Calibration and response-process inquiry remain prerequisites for trustworthy efficiency gains.

Measurement Invariance Across Groups and Populations

Measurement invariance asks whether a score has a sufficiently comparable relationship to its intended construct across groups, contexts, or occasions. In factor models, configural invariance concerns the basic pattern of indicators and dimensions, metric invariance concerns factor loadings, and scalar invariance concerns intercepts or ordinal thresholds, subject to the particular model. These conditions support different comparisons: similar patterns can permit analysis of associations without establishing that observed or latent means are directly comparable. Meredith (1993) provided a formal framework for factorial invariance, while Vandenberg & Lance (2000) organized its implications for applied organizational research. The key scientific issue is the meaning of comparison, rather than the mechanical passage of a statistical threshold.

Testing invariance usually requires an adequately defined reference model and explicit alternative explanations for noninvariance. A model fitted separately in each group may not identify all relevant differences because parameter scales and constraints can vary. Multigroup analyses impose and relax restrictions systematically, then inspect parameter estimates, local misfit, sample composition, and substantive context. Chen (2007) used Monte Carlo studies to evaluate how changes in fit indices respond to different patterns and proportions of noninvariant parameters. The simulation showed why a fixed change-index cutoff does not have the same meaning under every configuration. Researchers should report the size and locus of departures and their effect on the comparisons actually being made.

Language, age, educational exposure, disability, social context, and test format may influence item functioning. A difference in the distribution of latent attributes does not by itself establish biased measurement, just as the absence of a global model-fit difference does not establish fairness. Putnick & Bornstein (2016) reviewed conventions and reporting practices in the developmental literature and emphasized the need to examine equivalence before interpreting cross-group or longitudinal differences. Millsap (2011) analyzes related problems of measurement bias across diverse applications. The strongest practice combines quantitative tests with cultural and linguistic investigation of how items are understood. When invariance is incomplete, partial alignment or item-specific adjustments may be defensible, provided their assumptions and sensitivity are reported.

Differential Item Functioning and Response Bias

Differential item functioning, or DIF, occurs when individuals from comparison groups with the same value of the modeled attribute have different probabilities of responding to an item in a particular way. Uniform DIF describes a relatively consistent directional difference over the attribute range, while nonuniform DIF involves differences that vary with attribute level. DIF may indicate translation problems, culturally specific knowledge, different task strategies, or legitimate multidimensional influences omitted from the conditioning model. It is a diagnostic clue about comparability, not an automatic verdict about unfairness. Millsap (2011) provides a general latent-variable account of testing bias, connecting item-level differences to the broader assumptions required for valid cross-group interpretation.

DIF can be investigated through item-response models, ordinal regression, multigroup factor models, and related approaches. Each procedure conditions on a matching variable or estimated latent trait; if the matching variable itself contains biased items, estimated DIF can become distorted. Anchor-item selection, purification, sample size, multiple testing, and effect-size interpretation are therefore essential. A statistically detectable difference may have limited influence on total scores, while a small set of influential items can materially alter decisions near a threshold. Researchers should examine differential test functioning as well as individual items. Meredith (1993) provides the conceptual foundation for discussing invariance and fairness, and Chen (2007) illustrates how patterns of departures influence conventional fit summaries.

Response bias is a broader category that includes acquiescence, extreme-category use, impression management, speeded guessing, and language-related interpretation. Some are stable person tendencies; others depend on immediate incentives or task instructions. Including method factors can help identify systematic variance but may also absorb theoretically meaningful processes, so such models require external corroboration. Campbell & Fiske (1959) emphasized separating trait and method contributions through deliberately varied measurement procedures. Fair interpretation further depends on what the assessment is used for, how consequences fall across groups, and whether proposed corrections preserve the construct. Measurement equity is therefore a continuous research and governance task spanning response processes, model diagnostics, and decision outcomes.

Longitudinal Measurement and the Quantification of Psychological Change

Measuring change in memory, emotion, personality, or self-regulation requires a stable interpretive bridge across occasions. If an item becomes easier because of practice, if a response anchor changes its personal meaning, or if the construct's internal organization develops, a numerical difference may combine authentic psychological change with altered measurement. Longitudinal invariance tests examine whether loadings, intercepts or thresholds, and other relevant parameters are sufficiently comparable across waves. Putnick & Bornstein (2016) treats invariance across time as a central condition for interpreting developmental differences. Such tests should accompany, rather than be displaced by, sophisticated growth-curve analysis; modeling change precisely on an unstable scale remains substantively ambiguous.

Observed difference scores have error contributed by both measurements, and their reliability depends partly on the covariance between occasions and on the variance of authentic change. In a longitudinal design, researchers must distinguish rank-order stability from absolute level change, practice effects from development, and regression to the mean from intervention response. Enkavi et al. (2019) provides evidence that the temporal properties of self-regulation tasks vary, limiting unqualified claims about stable individual differences. Latent change models can separate measurement error from modeled trajectories when their constraints are plausible, but parameter identification and missing-data assumptions must be documented. The minimum detectable change for one individual is also a distinct problem from detecting a statistically reliable population-average change.

Responsiveness concerns an instrument's ability to detect the kind of change that matters for a specified purpose. A highly reliable measure of enduring personality may be deliberately insensitive to short-term states; a state measure designed for ecological sampling may appropriately vary from day to day. Prinsen et al. (2018) organizes evaluation of measurement properties for patient-reported outcomes and shows why precision, responsiveness, interpretability, and construct validity require separate consideration. Anchor-based change thresholds, distribution-based indices, and causal intervention analyses answer different questions. A study that reports clinically meaningful improvement should specify the criterion of meaningfulness, the population, the timeframe, and the uncertainty of change estimates rather than relying solely on statistical significance.

Cross-Cultural, Developmental, and Contextual Comparability

Psychological processes are measured within languages, developmental stages, educational systems, and institutional practices. The same item can recruit different memories, concepts, or norms across contexts, even when translated literally. Cross-cultural comparability therefore demands more than matching vocabulary. Translation and back-translation, cognitive interviews, expert content review, representative sampling, and statistical invariance tests supply complementary evidence. Vandenberg & Lance (2000) demonstrates why comparisons depend on established measurement equivalence, and Putnick & Bornstein (2016) documents the practical challenge of doing so across heterogeneous developmental studies. A method that performs well in an adult convenience sample may behave differently among children, older adults, or individuals with different communication needs.

Development complicates the notion of an unchanged attribute. A young child's task may require motor coordination or vocabulary that later becomes nearly automatic; the intended memory or inhibitory process can remain relevant while the behavioral indicators shift. A measurement bridge may therefore require age-appropriate tasks and linked parameters rather than identical item forms. Millsap (2011) emphasizes that models for assessing bias must be matched to substantive measurement questions. If a task changes its cognitive requirements with age, even stable statistical loadings could hide a response-process shift. Developmental inferences benefit from multiple methods, task families, and experimentally probed mechanisms that clarify what remains comparable.

Comparability is also contextual within the same person. Clinical symptoms reported in private can differ from answers given in an employment setting; risk-sensitive decisions change motivations for disclosure. Remote testing may vary in device latency and distraction, while intensive experience sampling changes the meaning of a recalled time window. Across all these cases, the scientific requirement is to establish the equivalence of the inference rather than the superficial similarity of the interface. AERA, APA, & NCME (2014) ties appropriate interpretations to documented validity and fairness evidence. Cross-context claims should describe the populations and environments in which evidence has been gathered, identify untested extrapolations, and give priority to designs that measure plausible response-process differences directly.

Model Identification, Fit, Estimation, and Statistical Uncertainty

A statistical model can be estimable in software while being weakly identified by the available observations. Latent-variable scales require constraints; sparse categories, highly correlated factors, underrepresented dimensions, and boundary estimates can lead to unstable parameters. Model identification asks whether distinct values of the parameter vector generate distinguishable observable distributions under stated assumptions. Goodness-of-fit assessment asks whether observed data are sufficiently consistent with the constrained model for the intended purpose. Both questions differ from substantive validity. Reise (2012) demonstrates how alternative factor structures can resemble one another; Embretson & Reise (2000) describes estimation and fit issues in item-response families. A parameter without defensible meaning remains a mathematical estimate, however small its standard error.

Evaluation should include global and local diagnostics, uncertainty intervals, estimation sensitivity, convergence behavior, residual structure, and independent replication. Chen (2007) shows that fit-index changes are sensitive to particular noninvariance conditions, undermining the practice of treating every cutoff as universal. Large item banks can give significant global misfit for minor irregularities, whereas small samples may conceal influential local discrepancies. Bayesian estimation adds prior assumptions whose effect should be reported; frequentist estimation requires attention to sampling design and the adequacy of asymptotic approximations. Simulation studies can probe bias and parameter recovery under realistic sample sizes and missing-data mechanisms, but their conclusions are restricted to the data-generating scenarios examined.

Multiverse and sensitivity analyses are especially useful when several defensible scoring or model choices exist. Researchers can compare ordinal and continuous item treatments, alternative factor counts, methods for missing responses, or different anchor sets in invariance analysis. Such comparisons are interpretable when the specifications are declared and their substantive implications explained. Flake & Fried (2020) identifies undisclosed researcher flexibility in measurement as a threat to cumulative evidence. Reporting that a conclusion survives reasonable modeling decisions strengthens confidence; reporting instability identifies a real boundary of present knowledge. Both outcomes advance psychometrics when accompanied by reproducible code, appropriate data access, and complete methodological disclosure.

Empirical Evidence, Replicability, and Measurement Quality Audits

Psychometric evidence has to be evaluated according to study design. A simulation can establish how an estimator behaves when a particular data-generating model is correct; it does not prove that real responses follow that model. A test–retest study can estimate stability in a population over a specified interval; it cannot alone determine what the scale represents. A multitrait study can challenge shared-method interpretations without establishing the causal structure of a latent construct. Flake et al. (2017) provides a methodological audit of published construct-validation practices, whereas Hussey & Hughes (2020) evaluates the properties of particular commonly used scales under multiple criteria. These lines of inquiry are complementary but have different populations, sampling frames, and inferential endpoints. Scientific synthesis must preserve those distinctions.

The sample of 15 questionnaires examined by Hussey & Hughes (2020) illustrates why methodological breadth matters. Their study reports that evaluations confined to internal consistency gave a substantially more favorable impression than a combination of factor structure, retest, and invariance checks. The findings demonstrate a concrete measurement-audit problem in a defined body of social and personality scales; they do not estimate the prevalence of invalid measurement throughout psychology. Hedge et al. (2018) investigated a different problem: robust average cognitive-task effects can coexist with weak ability to rank individuals reliably. Enkavi et al. (2019) extended the discussion through a large battery of self-regulation measures. These studies provide empirical constraints on broad claims that a familiar task is automatically suitable for individual differences.

An evidence audit should therefore record whether a source reports primary data, a mathematical derivation, a simulation, a review, or professional consensus. It should assess participant recruitment, item coverage, response-process validity, missingness, model fit, reliability uncertainty, independent replication, subgroup performance, and the distance between observed outcomes and the authors' conclusion. Prinsen et al. (2018) supplies a structured methodology for synthesizing properties of patient-reported instruments, illustrating how a review must assess distinct measurement properties separately. The present monograph compares such work narratively and does not estimate pooled reliability, perform a risk-of-bias meta-analysis, or claim exhaustive database coverage. Empirical conclusions are correspondingly restricted to the published designs and outcomes actually inspected.

Psychometric Decisions in Cognitive, Personality, and Health Research

In cognitive science, the unit of measurement may be an experimentally manipulated performance effect, a trial-level process parameter, or a stable person-level attribute. These uses call for different reliability and validity evidence. A time-based attention task can establish an average interference effect while offering little retest precision for selecting individuals. Hedge et al. (2018) and Enkavi et al. (2019) make this distinction empirically salient. Personality research often relies on multiple item ratings aggregated to subscales, in which content coverage, factor structure, response styles, and group equivalence dominate the validity argument. Flake et al. (2017) showed that a widespread reliance on alpha alone is insufficient for such claims. The test's domain, time scale, and intended inference should drive its technical evaluation.

In educational and organizational settings, score differences can have consequential effects on placement, employment, or access to support. Comparisons therefore depend on adequate standardization, uncertainty near decision thresholds, and evidence about subgroup functioning and fairness. AERA, APA, & NCME (2014) addresses testing practice and the relationship between score interpretation, use, and the populations concerned. Millsap (2011) develops methods for examining bias, while Meredith (1993) formalizes invariance questions relevant to equitable comparison. No single coefficient can establish that a decision rule is appropriate. The quality of assessment includes the consequences of error, the stability of measurement under changing conditions, and a documented process for responding to newly discovered limitations.

In health research, instruments may evaluate pain interference, fatigue, physical function, anxiety symptoms, or quality of life, often through patient-reported outcomes. Cella et al. (2010) describes standardized item-bank development in PROMIS, which seeks flexible precision across measurement applications. Prinsen et al. (2018) explains why systematic evaluation of such instruments involves multiple distinct properties and different review judgments. A precise score should not be interpreted as a diagnosis unless the applicable diagnostic validation and clinical standards have also been established. In longitudinal intervention trials, investigators must establish that the measure is responsive to meaningful change, that baseline and follow-up administrations are comparable, and that missing observations or treatment expectancy do not distort conclusions.

Digital Assessment, Artificial Systems, and New Measurement Contexts

Digital environments increase the range of observable behavior: response latency, sequences of choices, scrolling, writing, physiological signals, and repeated ecological reports can all generate potential indicators. These richer streams permit intensive within-person modeling but also introduce device effects, nonrandom missingness, platform-dependent behavior, privacy issues, and dependence among observations. A large number of data points collected from one person does not substitute for representative participants or demonstrate between-person generalizability. The same principles used for traditional tests remain relevant: formulate the construct, establish response mechanisms, evaluate precision at the intended level, and test transfer across technology and context AERA, APA, & NCME (2014). Scale or algorithmic complexity should remain proportionate to the explanatory question.

Computerized adaptive testing and modern item banks show what responsible technological integration can accomplish. Cella et al. (2010) offers a documented example of item-bank development intended to improve flexibility and conditional precision. Machine-generated questions or automated rating systems could extend this approach, but each change in item generation or scoring requires calibration and validation of its own. A language model can produce fluent explanations and familiar psychometric scores without those scores establishing equivalence with human cognitive functions. The interpretive problem is whether the system's observed responses instantiate the same measurement relation or require a distinct explanatory model. The philosophical horizon of the Artificial Era makes this issue important, while the empirical standard remains explicit, reproducible evidence.

Comparisons between Homo and Artificial systems should begin with operationally defined tasks, measurement conditions, and error sources. System updates, sampling parameters, prompts, memory configuration, interface constraints, and interactions with users can shift performance and the meaning of repeated scores. Human norms cannot simply be applied to artificial systems as if both were samples from a single established reference population. Experimental interventions, within-system retests, model-specific invariance checks, and longitudinal reproducibility can form a research program for artificial measurement. Borsboom et al. (2004) supplies a useful general question about whether an assumed attribute actually generates the observed scores; Eisenberg et al. (2019) demonstrates the need to test construct structure rather than inherit a conventional taxonomy. Claims about artificial psychological organization remain hypotheses unless the proposed constructs and their indicators have passed such tests.

Critical Scientific Synthesis and Unresolved Research Problems

The reviewed literature supports a coherent conclusion: psychometric quality is multidimensional and inferentially local. Classical theory formalizes reproducibility under defined conditions; factor models characterize shared item variation; IRT describes conditional response probabilities and information; generalizability theory partitions design-dependent variability; and invariance analysis investigates whether a scale preserves its interpretation across contexts. These frameworks address different questions and can sometimes be combined within one research program. Brennan (2001), Embretson & Reise (2000), Reise (2012), and Millsap (2011) provide complementary mathematical accounts, while Messick (1995) and AERA, APA, & NCME (2014) specify the broader interpretive responsibility. A defensible score claim identifies the model's explanatory role, assumptions, population, uncertainty, and intended use.

Unresolved issues arise where technical performance outruns substantive explanation. Latent-variable models can fit while leaving response processes ambiguous; predictive accuracy can improve through nuisance variables; group invariance tests can miss local harms; and repeated behavior can change because the act of assessment alters its context. Flake & Fried (2020) highlights the consequences of opaque measurement decisions. Hussey & Hughes (2020) shows how several familiar scales can fail more demanding validation checks, while Hedge et al. (2018) reveals that highly reproducible group-level effects need not measure stable between-person differences. The appropriate response is to improve experimental discrimination among explanations, expose preprocessing and scoring decisions, test independent samples, and give model uncertainty a role in substantive conclusions.

Future work should connect mechanistic models of response generation to psychometric predictions, develop stronger transportability assessments across devices and cultures, and distinguish stable attributes from context-sensitive states using repeated designs. Adaptive assessments need continuous monitoring for item drift and differential functioning; digital systems need explicit governance of algorithmic scoring changes. Human–Artificial comparisons need experimental tests of measurement equivalence before terms such as cognition or agency can support quantitative identity claims. These are scientific research priorities rather than claims of an established unified theory. Their shared principle is empirical accountability: every extension of a score into a new population, timescale, or substrate carries an additional evidentiary burden.

Conclusion

Psychometric measurement of mental processes is a disciplined inference from observations to theoretically defined psychological attributes under specified conditions. Its foundation lies in a model connecting construct, response process, observation, scoring, uncertainty, and interpretation. The historical development from classical reliability and construct validity to generalizability theory, factor models, item-response modeling, and measurement invariance has expanded the questions that researchers can answer. It has also made the conditions of defensible comparison more explicit. Cronbach (1951), Cronbach & Meehl (1955), Rasch (1960), Brennan (2001), and Meredith (1993) represent distinct foundational stages in that development. Their methods remain useful insofar as contemporary research specifies and tests the assumptions under which each one supports its intended conclusion.

The evidence reviewed here shows that reliability, internal structure, conditional precision, invariance, and predictive performance are complementary sources of information about a measurement claim. None can independently establish a complete interpretation. Original empirical studies of cognitive tasks and self-regulation have demonstrated serious mismatches between common uses of scores and their temporal or between-person properties Hedge et al. (2018) Enkavi et al. (2019). Audits of psychological scales have uncovered underreported and inconsistent validation evidence Flake et al. (2017) Hussey & Hughes (2020). Methodological research on ordinal models, subgroup comparisons, and patient-reported outcomes shows how clearer designs and transparent diagnostics can strengthen the quality of inference Samejima (1968) Putnick & Bornstein (2016) Prinsen et al. (2018).

The practical outcome is a unified standard of reasoning rather than a single favored coefficient or test model. Researchers must establish what is being measured, select observations that can express it, document the fitted model, quantify uncertainty, assess alternative explanations, and verify whether conclusions generalize to the populations and decisions in question. Psychometrics will remain central to the Psychology of Homo and to research on artificial cognitive systems because both require disciplined relations between observable performances and proposed psychological attributes. In the Artificial Era, a common measurement vocabulary can support comparative inquiry when response mechanisms and scale equivalence have been established. Every authoritative claim about mental processes ultimately depends on the specific evidence that connects its theoretical object to observable variation.

References

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://ncme.org/resources/books/testing-standards/
  • Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061
  • Brennan, R. L. (2001). Generalizability theory. Springer. https://doi.org/10.1007/978-1-4757-3456-0
  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016
  • Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Amtmann, D., Bode, R., Buysse, D., Choi, S., Cook, K., DeVellis, R., DeWalt, D., Fries, J. F., Gershon, R., Hahn, E. A., Lai, J.-S., Pilkonis, P., Revicki, D.,. PROMIS Cooperative Group. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. https://doi.org/10.1016/j.jclinepi.2010.04.011
  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
  • Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
  • Eisenberg, I. W., Bissett, P. G., Enkavi, A. Z., Li, J., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019). Uncovering the structure of self-regulation through data-driven ontology discovery. Nature Communications, 10, Article 2319. https://doi.org/10.1038/s41467-019-10301-1
  • Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates. https://www.routledge.com/Item-Response-Theory/Embretson-Reise/p/book/9781410605269
  • Enkavi, A. Z., Eisenberg, I. W., Bissett, P. G., Mazza, G. L., MacKinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2019). Large-scale analysis of test–retest reliabilities of self-regulation measures. Proceedings of the National Academy of Sciences, 116(12), 5472–5477. https://doi.org/10.1073/pnas.1818430116
  • Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063
  • Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
  • Hedge, C., Powell, G., & Sumner, P. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50, 1166–1186. https://doi.org/10.3758/s13428-017-0935-1
  • Hussey, I., & Hughes, S. (2020). Hidden invalidity among 15 commonly used measures in social and personality psychology. Advances in Methods and Practices in Psychological Science, 3(2), 166–184. https://doi.org/10.1177/2515245919882903
  • Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. https://doi.org/10.1007/BF02294825
  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
  • Millsap, R. E. (2011). Statistical approaches to measurement invariance. Routledge. https://www.routledge.com/Statistical-Approaches-to-Measurement-Invariance/Millsap/p/book/9781848728189
  • Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1147–1157. https://doi.org/10.1007/s11136-018-1798-3
  • Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
  • Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research. https://search.worldcat.org/title/1037107923
  • Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555
  • Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
  • Samejima, F. (1968). Estimation of latent ability using a response pattern of graded scores. ETS Research Bulletin Series, 1968(1), i–169. https://doi.org/10.1002/j.2333-8504.1968.tb00153.x
  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
  • Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107–120. https://doi.org/10.1007/s11336-008-9101-0
  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/10.1177/109442810031002