RESEARCH DOSSIER
Psychological Test Development, Standardization, and Norming
Abstract. Psychological test development integrates construct specification, representative item sampling, response-process investigation, empirical calibration, standardized administration, population norming, validity arguments, and the maintenance of score interpretations. This critical narrative monograph reconstructs the historical foundations of psychological assessment and systematically analyzes classical test theory, item response models, dimensionality, precision, normative sampling, percentile ranks, standard-score transformations, score linking and equating, invariance, differential item functioning, fair use, translation, adaptive testing, and re-norming. It distinguishes experimental findings, mathematical assumptions, professional testing standards, and scientific limitations. The evidence supports a lifecycle conception of test quality in which every numerical comparison and institutional decision depends on a documented chain from construct definition to task performance, scoring rules, reference population, and intended use. The study draws on original methodological publications, empirical instrument-development studies, and international professional guidance; it is a critical narrative synthesis rather than a protocol-registered systematic review or an original experiment.
Keywords: psychological test development; test standardization; norm-referenced testing; psychometric norms; norming sample; test construction; item analysis; classical test theory; item response theory; measurement precision; percentile ranks; test equating; differential item functioning; measurement invariance; fairness in assessment; cross-cultural adaptation; computerized adaptive testing; score interpretation; psychological assessment
The Research Object: Engineering Defensible Psychological Test Scores
Psychological test development is the coordinated construction of a procedure that elicits observations, transforms those observations into scores, and supports specified interpretations and decisions. Standardization establishes controlled procedures and defensible comparability, while norming characterizes score distributions in a stated reference population. These activities are connected throughout the instrument life cycle: item content influences who can respond meaningfully, administration conditions influence observed performance, and the composition of the norming sample determines the meaning of a percentile. The 2014 joint testing standards AERA, APA, & NCME (2014) frame testing as a chain of evidence about intended interpretations and uses. The methodological synthesis of Boateng et al. (2018) describes sequential phases of item development, scale construction, and evaluation, making visible the decisions often concealed beneath a final questionnaire or examination form.
The fundamental unit of evaluation is a score interpretation under specified conditions. A five-point self-report rating may be useful for comparing group trends but lack the precision required for a high-stakes individual decision. A cognitive performance test standardized on one language community can show stable internal consistency yet produce misleading comparisons when used with unfamiliar linguistic or technological demands. Clark & Watson (2019) places theoretical definition and deliberate sampling of content at the beginning of instrument design; Kane (2013) requires the claims connecting responses, scores, extrapolation, and action to be stated explicitly. Therefore a developer must specify the target attribute, respondent population, response mechanism, testing setting, scoring rule, and proposed inference before any empirical coefficient is treated as sufficient.
This monograph critically synthesizes primary psychometric theory, original methodological contributions, published instrument-development evidence, and professional testing guidance. It traces the historical transformation of tests, the design and calibration of items, normative sampling, score equivalence, adaptations, decision thresholds, technological administration, and the maintenance of interpretations over time. It is a narrative scientific synthesis rather than a preregistered systematic review or an original empirical study. The recurring analytic question is whether a given body of evidence actually supports the intended comparison or decision, and which additional observations would be needed when test conditions change AERA, APA, & NCME (2014) Kane (2013).
Definitions: Test, Measure, Standardization, Norm, and Validation
A test is an intentionally specified sample of performances or responses, recorded under defined rules and interpreted in relation to a construct, performance domain, or criterion. Measurement describes the assignment of values under a defensible representational or model-based relationship; scaling describes how different observed response patterns are mapped to a numerical continuum. Standardization has at least two operational senses: the uniformity of administration and scoring, and the development of reference distributions or calibrated scales. Keeping these senses distinct prevents a standardized administration script from being mistaken for proof that a norm table is representative or that its scores have the claimed validity AERA, APA, & NCME (2014) Angoff (1971).
Norm-referenced interpretation locates a score within a distribution derived from a defined reference group. Criterion-referenced interpretation evaluates performance against a substantively specified expectation or performance standard. A percentile rank states a distributional position; it does not identify a percentage of items answered correctly, a percentage of an underlying trait possessed by the examinee, or an individual's probability of success. Standard scores are transformations of scores relative to a reference distribution and inherit its sample limitations. Angoff (1971) establishes the conceptual architecture of scales, norms, and equivalent scores, while Holland & Dorans (2006) distinguishes statistical linking from the stronger claim that forms are interchangeable for a particular use.
Validation evaluates the evidence and theory that connect observed results to proposed meaning and action. Reliability addresses a family of consistency or precision questions; validity asks what inferences the test warrants. Evidence that an instrument is internally coherent does not settle whether it covers the intended domain, whether respondents interpret items as designed, or whether the same score has the same meaning across cultural groups. Cronbach & Meehl (1955) articulates the construct-oriented validation problem, Messick (1995) unifies sources of validity evidence, and AERA, APA, & NCME (2014) turns these requirements into practice. Sound test development makes each of these questions separately inspectable and maintains their connections.
Historical Formation of Psychological Testing and Normative Comparison
Modern psychological testing developed at the intersection of experimental psychology, statistical analysis of human differences, school administration, clinical assessment, and institutional selection. Late nineteenth-century measurement programs collected bodily and sensory observations as candidate indices of ability, while subsequent research challenged how these data related to complex intellectual work. Such early projects matter historically because they established reproducible procedures and population comparison as goals, but the social hierarchies and eugenic premises attached to parts of the tradition require explicit scrutiny. The lasting scientific lesson is that a quantity extracted from a person acquires psychological meaning only through a defended relation between task performance and the proposed attribute Cronbach & Meehl (1955) AERA, APA, & NCME (2014).
Twentieth-century test construction increasingly combined representative content sampling, standardized instructions, statistical item analysis, and organized normative data. Educational examinations highlighted the difficulty of comparing performances across forms; clinical and personality instruments raised distinct questions about symptom reporting, response tendencies, and norm-group composition. The classical reliability tradition provided methods for analyzing score consistency, including the 1937 formula program of Kuder & Richardson (1937) and the internal-structure coefficient proposed by Cronbach (1951). These contributions were mathematically influential, yet neither formula alone establishes that an instrument measures its intended target or that a score conveys the same meaning in different contexts.
Later developments made the assumptions underlying scores increasingly explicit. Lord (1980) formulated applications of item response theory to practical testing, while Samejima (1968) extended latent-response modeling to graded responses. Holland & Dorans (2006) clarified statistical designs necessary for linking and equating different forms. Messick (1995) and the joint AERA, APA, & NCME (2014) standards enlarged professional attention to score meaning, consequences, fairness, and appropriate use. A defensible history therefore tracks both technical achievements and changes in the kinds of claims that test developers are expected to justify; the current research object is a maintained evidence system rather than a once-completed test form.
From Intended Uses to a Documented Test Development Program
Development begins with a specification of use: who will take the test, which decisions the score may inform, what opportunity the respondent has to demonstrate the intended attribute, and which consequences follow a false classification. The instrument's purpose determines the target level of precision, desirable content coverage, length, security requirements, and acceptable operational burden. Kane (2013) shows why each step in the interpretation-and-use argument needs evidence. When a scale is designed for group research, individual diagnostic use requires a new evidentiary argument even if the item text remains identical. The design brief should state such limits at the outset, so implementation does not inadvertently authorize claims for which evidence was never collected.
A development program translates that brief into measurable milestones: construct explication, domain mapping, blueprint approval, item pool creation, cognitive pretesting, field piloting, statistical calibration, norm collection, standard setting, independent validation, publication of administration materials, and scheduled re-evaluation. Boateng et al. (2018) organizes major parts of this sequence into item generation, scale construction, and psychometric evaluation. Clark & Watson (2019) adds a crucial discipline: create an initially inclusive item pool, compare the proposed scale with neighboring constructs, and examine dimensionality independently of internal-consistency optimization. These milestones are interconnected; poor domain coverage cannot be repaired by a large calibration dataset.
The responsible project should preserve an auditable decision record describing the question being answered at each phase, the data used, the criteria for item revision, and the effect of changes on the interpretation claim. There is scientific value in documenting rejected items and failed model assumptions, because their exclusion can alter the meaning of the final scale. Flake & Fried (2020) describes questionable measurement practices that arise when investigators modify instruments without reporting consequential decisions. A test manual should therefore identify the currently supported versions and populations while separating completed evidence from planned validation and ongoing operational monitoring AERA, APA, & NCME (2014).
Construct Specification, Domain Sampling, and the Test Blueprint
A construct definition must distinguish the intended psychological process or attribute from the particular stimuli selected to observe it. For an educational achievement test, the domain may be organized by skills, contexts, and complexity levels; for a self-report scale, the intended dimension may require explicit definitions of experiences, time frames, and functional implications. A blueprint links the theoretical definition to observable tasks and establishes how many items should sample each part of the domain. Sireci (1998) explains the value of structured content-evidence programs; Clark & Watson (2019) emphasizes theoretical clarity, overinclusive item generation, and comparison with related traits. A blueprint is a scientific sampling design, not merely a layout for a convenient questionnaire.
Underrepresentation occurs when the test excludes important behaviors or facets, making performance too narrow to support the intended inference. Irrelevant variance arises when response patterns depend substantially on reading difficulty, test familiarity, language conventions, fatigue, motor demands, or device accessibility unrelated to the intended construct. The test blueprint should identify both hazards and describe strategies to study them. Review of response processes may show that apparently straightforward items invite interpretations different from those assumed by authors. AERA, APA, & NCME (2014) treats content and response-process evidence as distinct resources; Terwee et al. (2018) demonstrates a formal methodology for studying relevance, comprehensiveness, and comprehensibility in patient-reported measures.
The blueprint must remain traceable through revisions. Developers can record the operational definition for each domain, its intended weight, the response format, the expected difficulty or endorsement range, and the independent review procedures governing changes. If later psychometric selection removes nearly every item concerning a theoretically important facet, the final score becomes a different operational construct. Boateng et al. (2018) explains item reduction as a phase requiring substantive as well as statistical judgment. A scientifically defensible instrument treats the content map as an explicit constraint on calibration rather than surrendering its intended meaning to whichever items maximize a convenient summary coefficient.
Item Generation, Wording, and Response Formats
Item development translates a domain definition into a deliberately varied set of response opportunities. The initial pool should span the intended range of intensity, difficulty, context, and behavioral manifestation. Item wording should state one interpretable task or proposition, use time frames consistently, and avoid cues that impose literacy or cultural demands outside the construct. Clark & Watson (2019) recommends an overinclusive pool and careful attention to wording because narrowly selected items may yield deceptively coherent but incomplete scales. A performance item requires inspection of the correct solution and plausible distractors; a rating item requires careful construction of response categories, anchors, instructions, and a reference period. Each design creates its own response-process assumptions.
Response formats make consequential choices about the information recorded. Dichotomous scoring may provide operational simplicity but obscure partial competence; ordered-category ratings may capture gradients yet invite variation in how respondents interpret labels such as often, somewhat, or strongly agree. Forced-choice formats, situational judgments, observations, and timed tasks change the structure of dependencies among responses. Developers should choose formats for their theoretical fit and study effects of item direction, acquiescence, extreme responding, fatigue, and missingness. Boateng et al. (2018) distinguishes item formulation from later statistical reduction; AERA, APA, & NCME (2014) requires evidence about whether the resulting scores support the proposed use.
A field-ready item bank preserves item provenance, proposed subdomain, version history, response instructions, scoring keys, expected misunderstandings, and accessibility requirements. Content revisions should be recorded before and after pilot data are examined, especially when alternative phrasings change the task. Adverse item characteristics are evidence about the design, not merely an inconvenience to be optimized away. Flake & Fried (2020) connects transparent reporting to the integrity of psychological measurement. An instrument with many plausible questions but unclear scoring meaning is still a collection of prompts; it becomes a test through documented procedures and evidence supporting a particular interpretation.
Expert Content Review and Response-Process Evidence
Expert assessment should evaluate the fit between each item and the domain definition, including relevance, representative coverage, potential bias, technical clarity, and consistency with the intended administration. Independent reviewers with appropriate subject knowledge may use structured ratings, justification fields, and adjudication procedures to identify missing facets. Sireci (1998) provides a methodological treatment of collecting and analyzing content-validity information. Numerical agreement summaries can organize expert judgments, but they depend on reviewer selection and the underlying specification; their existence alone does not validate the instrument. A theoretically coherent domain map allows conflicting ratings to reveal substantive disagreements rather than being silently averaged.
Response-process studies examine what participants actually do when they encounter an item. Cognitive interviews, think-aloud protocols, comprehension probes, and observations during task completion can reveal whether respondents interpret concepts, recall periods, and answer categories as designers intended. These investigations are particularly important when test materials are adapted for different ages, languages, abilities, or digital interfaces. Terwee et al. (2018) treats comprehensibility as one component of content validity in patient-reported measurement; Beaton et al. (2000) emphasizes conceptual and experiential equivalence in cross-cultural adaptation. The observed response process is evidence about interpretation, and revisions should trigger renewed testing when wording or response demands change.
Effective content review and response-process inquiry serve complementary functions. Expert panels investigate whether items adequately cover a scholarly or practical domain, while respondent interviews investigate how the item is encountered and understood. Their evidence can diverge: an expert may judge an item highly relevant while intended respondents interpret it differently. The developer should retain those divergences and test revised language with diverse target users before large-scale field administration. AERA, APA, & NCME (2014) locates these data within the broader validation argument; Clark & Watson (2019) cautions that psychometric elegance cannot replace conceptual clarity. The revision record should identify which interpretation problems were resolved and which remain plausible.
Pilot Administration, Feasibility, and Item Diagnostics
A pilot study evaluates whether planned procedures operate as specified in real respondents and testing environments. Its tasks include timing, instruction comprehension, item omissions, response-category use, administration fidelity, score distribution, data capture, and accessibility. Pilots may be intentionally small and qualitative when diagnosing usability; their evidence cannot be confused with definitive population parameters. Boateng et al. (2018) identifies pretesting and subsequent survey administration as distinct phases, because a clear question at the cognitive-interview stage may still produce ceiling effects, missingness, or unanticipated correlations in field data. Each pilot should have explicit questions that the study is capable of answering.
Item facility, discrimination, distractor analysis, corrected item-total correlations, and category functioning can identify anomalies, but interpretation requires a theory of what successful performance should mean. A negative item-total relation might reveal a wrongly keyed item, a multidimensional scale, systematic subgroup differences, or a scientifically informative contradiction. Excluding an item solely because it reduces alpha may discard rare but important aspects of the construct. Clark & Watson (2019) places dimensional coherence and discriminant validation ahead of automatic optimization of consistency. Cronbach (1951) formalizes internal structure through an important coefficient but does not turn maximal covariance into the objective of every instrument.
Pilot results should determine which questions can be answered in the next study. For example, a compressed score distribution may require a wider task-difficulty range; repeated missing data on particular items may require changes to administration or an investigation of privacy concerns. All such revisions create a new documented form. When pilot participants overlap with later calibration or validation samples, dependence should be disclosed and evaluated. AERA, APA, & NCME (2014) links evidence quality to the intended use and population; Flake & Fried (2020) shows why opaque measurement changes threaten replication. The outcome of the pilot phase is a reasoned decision about readiness for representative or purpose-built field testing.
Calibration Designs, Recruitment, and Data Independence
Calibration estimates how observed response patterns relate to the proposed scale within a particular sample and model. The sampling design should therefore reflect the intended population, range of performance, language demands, testing contexts, and anticipated sources of variation. Recruitment from a single convenience pool may supply large amounts of information about its participants while remaining weak evidence for a general normative claim. The joint standards AERA, APA, & NCME (2014) demand transparent description of examinee populations and intended applications. Boateng et al. (2018) similarly places sampling and survey administration at the center of scale construction. Developers must distinguish statistical sample size from representativeness and support claims of precision with a design-specific rationale.
Different phases may require different samples. A broad, deliberately heterogeneous calibration dataset is valuable for studying item functioning across the trait continuum; a probability-based reference sample may be required for population norm estimation. External validation should test predictions in data that did not guide item selection, factor modeling, or scoring decisions. When several analyses reuse the same participants, apparent confirmation may partly reflect model selection. Clark & Watson (2019) advocates thoughtful validation samples and independent assessment of substantive claims. Kane (2013) clarifies why each stage of an interpretation argument may require its own evidence, rather than treating one well-fitting calibration study as a universal certificate.
Sampling plans should anticipate differential participation, nonresponse, clustering, mode effects, and restrictions of range. The testing manual should describe exclusions, recruitment locations, demographic coverage, accessibility provisions, scoring omissions, and any weighting or adjustment procedures. Data-management protocols should distinguish raw responses from operational scores and preserve version identifiers for every form and administration mode. AERA, APA, & NCME (2014) frames documentation as essential for judging the transport of evidence. Scientific test development establishes traceability between the field conditions that produced parameters and the future settings in which those parameters are expected to hold.
Scoring Architecture, Missing Data, and Auditability
Before calibration results are finalized, developers should state the scoring model: item keys, reverse-coded items, treatment of partial credit, missing-response rules, subscales, composites, and allowable transformations. The score-generating procedure should be executable from a clearly identified input and produce the same output when run again. Under a classical composite, a raw score can be a sum or mean of specified responses; under an item response model, ability estimates require model parameters and an estimation procedure. Boateng et al. (2018) describes both weighted and unweighted approaches, while Lord (1980) shows how parameterized response models support practical scoring. Scoring choices should be evaluated for their effects on interpretability, precision, and fairness.
Missingness cannot be handled by a default rule detached from the response mechanism. Omissions may result from timing, misunderstanding, fatigue, deliberate withholding, inaccessible materials, or adaptive test routing. Proration by completed items changes the scoring assumption and may privilege people who systematically skip difficult questions; treating omitted items as incorrect can encode an administrative artifact as low ability. Developers should record missing-data patterns and test alternative rules against explicit scientific and operational criteria. AERA, APA, & NCME (2014) requires users to know conditions that affect scores. For person-level decisions, the consequences of uncertain scoring should be part of the validity argument Kane (2013).
An auditable scoring system includes the precise version of the test form, parameters, software, scoring keys, transformations, and a verification corpus of known input-output cases. It should preserve whether the score is observed, model-estimated, norm-referenced, or interpreted by a cut score. Digital systems introduce risks of silent scoring changes following updates; manual formats introduce inter-rater and transcription variation. Flake & Fried (2020) identifies inconsistent measurement decisions as a threat to cumulative scientific results. The methodological goal is a transparent relation between what the examinee did, what the scoring procedure produced, and what the resulting value is authorized to mean.
Classical Test Theory and Error in Observed Scores
Classical test theory organizes the observed score around a conceptual decomposition into a true-score component and measurement error under defined assumptions. The framework is useful for asking how much variation in observed scores is associated with stable individual differences versus circumstances or random fluctuations under a particular measurement procedure. Classical coefficients are properties of scores in a population and testing design; they should not be described as permanent characteristics of a questionnaire. Kuder & Richardson (1937) derived early internal-consistency estimators and Cronbach (1951) generalized the analysis with coefficient alpha. These methods changed test construction by enabling item-level evaluation, but the inferential question is which sources of error a particular coefficient actually samples.
Internal consistency describes patterns of inter-item association under assumptions and is affected by test length, item covariance, and dimensional structure. It does not automatically estimate stability over time, agreement among raters, precision for a single individual, or the truthfulness of the proposed construct interpretation. Revelle & Zinbarg (2009) contrasts alpha with alternative reliability estimates and discusses why interpretation depends on model structure. Clark & Watson (2019) stresses that unidimensionality and substantive coverage matter more than mechanical pursuit of high alpha. Adding near-duplicate items can increase covariance without improving the breadth of the measured domain.
The standard error of measurement translates an uncertainty model into score units, but its use requires clarity about which population, score transformation, and error sources are represented. Confidence or prediction intervals around estimated scores should reflect the actual reliability model and cannot be transferred freely between distinct norm groups or forms. AERA, APA, & NCME (2014) expects precision evidence relevant to an intended interpretation; Kane (2013) makes the inference and proposed use central. Developers should report multiple reliability studies when error may arise from occasion, raters, forms, or administration, and distinguish uncertainty about a test score from uncertainty about a future outcome.
Item Response Theory and Localized Measurement Precision
Item response theory models the probability of an observed item response conditional on a latent variable and specified item parameters. Models for dichotomous items may include difficulty and discrimination; polytomous models represent ordered response categories through additional parameters. Lord (1980) developed practical applications of this framework, and Samejima (1968) treated graded-score response patterns. Such models can help identify where along the scale an item provides information and support item banks or tailored tests. The latent scale, however, remains dependent on dimensionality, model fit, calibration conditions, and how the psychological variable is defined.
Item information is usually uneven across the trait continuum. A bank calibrated mostly for the center of a distribution may estimate extreme scores with considerably poorer precision even when its average reliability appears satisfactory. Standard errors derived from a fitted item response model are conditional on that model and on the calibration population; unmodeled local dependence or multidimensionality can make them misleading. Hambleton et al. (1991) introduces assumptions and diagnostics relevant to practical item response applications, while AERA, APA, & NCME (2014) treats empirical support for model assumptions as part of test-quality evidence. Reporting precision across intended score regions is more informative than presenting a single summary coefficient.
Item response theory facilitates linking forms and selecting different items for different examinees, provided parameters remain meaningfully comparable. This conditional is the substantive issue. Exposure to coaching, alternative modes of delivery, language translation, or population shift can alter item functioning and invalidate an existing calibration. Holland & Dorans (2006) distinguishes strong equating requirements from looser score linking. Developers should therefore evaluate item fit, dimensional structure, local independence, parameter stability, group differences, and out-of-sample behavior before using model output to guarantee equivalence. The presence of a sophisticated latent-variable algorithm does not by itself establish measurement.
Factor Structure, Dimensionality, and Construct Coverage
Dimensionality analysis asks whether a score represents one dominant response process, several correlated dimensions, a hierarchical structure, or a composite of substantively different facets. Exploratory procedures identify patterns that may guide construct refinement; confirmatory procedures test prespecified restrictions but remain sensitive to item selection and model alternatives. Clark & Watson (2019) makes this distinction central to valid scale construction. Boateng et al. (2018) describes extraction and independent testing of latent dimensions as different phases. Researchers should interpret factor structure alongside item content and response-process evidence rather than equating statistical fit with discovery of a psychological entity.
Shared wording, response habits, method effects, and item dependence may create correlations that resemble a substantive factor. Conversely, the existence of a broad general component does not make meaningful specific abilities irrelevant. Decisions to score a total scale or report subscales must therefore be tied to the research question and the available reliability and validity evidence for each composite. Cronbach & Meehl (1955) locates measurement meaning in a wider network of construct predictions. Revelle & Zinbarg (2009) demonstrates why internal-consistency coefficients must be interpreted in light of latent organization. A useful structural model clarifies the limits of the score and supports falsifiable predictions about external relations.
Strong testing programs use independent samples or cross-validation for structural claims, compare plausible alternatives, inspect residual dependence, and test whether the chosen structure remains interpretable across relevant groups and times. Uncritical post-hoc model adjustment can capitalize on idiosyncratic data while obscuring the original substantive theory. Flake & Fried (2020) explains how flexible measurement decisions undermine confidence in reported effects. AERA, APA, & NCME (2014) frames internal-structure evidence as one source among several, so a theoretically meaningful factor solution must still be supported by content, response-process, external, and consequential evidence for its actual uses.
Generalizability, Raters, Occasions, and Forms
A psychological score may vary when the examinee encounters a different rater, testing room, occasion, set of tasks, or administrative mode. These facets create distinct questions of consistency. Repeated administration can examine temporal stability, but genuine change in the construct and memory of earlier items complicate interpretation. Agreement studies can examine scoring across raters, but their conclusions depend on which raters and circumstances were sampled. AERA, APA, & NCME (2014) recommends precision evidence matched to use. The generalizability perspective expands test evaluation from one residual error term to specified sources of observed-score variation, allowing developers to prioritize the conditions most threatening a target inference.
Designing a test for individual progress monitoring requires particular care. A change score combines uncertainty from two occasions and may be affected by regression toward the mean, practice, treatment, development, or drift in administration. A high single-occasion reliability coefficient does not guarantee sensitive measurement of change. Kane (2013) distinguishes the generalization and extrapolation assumptions that support an interpretation; Cronbach (1951) supplies one internal consistency tool but not a complete account of longitudinal score comparability. Studies should identify the smallest differences their design can meaningfully distinguish rather than assert that every observed movement represents psychological change.
Multiple forms also introduce content-sampling error and unequal challenge. Two collections of tasks can cover nominally identical domains yet differ in difficulty distribution, language, or exposure to coaching. The developer should study whether scores from both forms support the same interpretation across relevant ability ranges, not assume that a common label makes them substitutable. Holland & Dorans (2006) develops the logic of linking and equating, including designs that use common-item anchors. The quality of the evidence depends on anchor stability, sampling assumptions, and the intended comparison. Generalizability is therefore an experimentally investigable feature of the measurement procedure and decision context.
Validity as an Interpretive and Evidentiary Argument
Validation asks whether the evidence and theory support the interpretations and uses attached to an observed score. Cronbach & Meehl (1955) reconceived construct validity as the testing of a network of theoretically specified relationships. Messick (1995) integrated evidentiary and consequential considerations rather than treating different data sources as independent validity certificates. Kane (2013) makes the inferential chain explicit: observations must be scored, generalized, extrapolated to the intended domain, and sometimes used to support an action. Test development can furnish evidence for these transitions, but no single coefficient provides a global verdict on every possible interpretation.
Evidence includes domain relevance, response-process studies, internal structure, relationships with other variables, group comparability, and consequences of testing. Sources may disagree because they investigate different assumptions. For instance, a test can reproduce a stable factor structure while predicting an important criterion poorly; another instrument can predict an outcome in one sample while containing items that primarily reflect group-specific access to schooling. AERA, APA, & NCME (2014) demands that validity evidence be judged for the proposed use, while Clark & Watson (2019) insists on examining discriminant relations to neighboring constructs. These distinctions prevent performance prediction alone from substituting for the broader claim that a score measures a particular psychological attribute.
The best validation program states counterhypotheses, seeks independent corroboration, publishes contrary findings, and updates permitted score uses when new evidence appears. A positive correlation found after choosing among many analysis options supports a weaker inference than a replicated, prespecified result with a plausible mechanism. Flake & Fried (2020) treats transparency about measurement decisions as a prerequisite for cumulative scholarship. Kane (2013) clarifies that additional evidence should address the most consequential untested links in the argument, particularly when scores influence placement, clinical evaluation, employment, or access to services. Validation is therefore part of ongoing test governance.
Normative Populations and the Meaning of a Reference Group
Norms describe how scores are distributed among persons in a specified reference population under specified testing procedures. A norm table is consequently a claim about population composition and time, not an abstract property of the test. Developers should state age ranges, educational and linguistic distributions, geographic coverage, recruitment frames, sampling dates, weighting, exclusions, and the administration mode associated with each normative sample. Angoff (1971) establishes the systematic distinction between raw scores, normative scales, and equivalent scores. AERA, APA, & NCME (2014) requires users to understand the adequacy and relevance of the reference group for each proposed interpretation. When normative sampling is incomplete, the limitation must accompany the displayed score.
Representative norming requires design attention to who could have entered the sample and whether selection differed systematically by psychological attributes related to performance. Large online samples can reveal detailed patterns within an accessible platform while underrepresenting people lacking digital resources or requiring accommodations. Conversely, stratified recruitment can better cover prespecified subpopulations while leaving uncertainties about nonresponse and within-stratum diversity. Poststratification weights may improve a documented mismatch but cannot eliminate all selection mechanisms. Developers should compare the realized sample with its sampling frame and report achieved rather than merely intended representation AERA, APA, & NCME (2014). Kane (2013) explains why extrapolation from observed participants to a proposed target population is a distinct inferential step.
Subgroup norms require substantive justification. Age-specific or language-specific tables can be warranted when developmental or linguistic differences systematically affect performance, but demographic partitioning can also institutionalize unequal expectations or conceal structural causes of disparities. The choice of reference group changes what an equivalent percentile implies about comparative position. Norms must be explained as descriptive comparisons rather than innate capacities or moral rankings. Messick (1995) connects test-score interpretation with consequences of use. For every norm table, responsible documentation identifies whom it represents, how current it is, and which cross-group comparisons are supported by independent measurement-equivalence evidence Putnick & Bornstein (2016).
Raw Scores, Percentile Ranks, and Standard Score Transformations
Raw test scores count or aggregate responses under specified scoring rules. A percentile rank locates a value in the empirical distribution of a reference group; its exact computation depends on the treatment of ties and the convention used for ranks. A score at the 80th percentile does not mean 80 percent of items were correct or that the examinee possesses 80 percent of the construct. A standardized z score expresses distance from a reference-group mean in units of its standard deviation, while other familiar normative scales apply linear transformations of z. Angoff (1971) shows why these quantities represent different aspects of an observed distribution and must not be mixed indiscriminately.
Percentile ranks are ordinal descriptions of distributional position; their numerical intervals cannot normally be interpreted as equal psychological distances. A move from the 40th to the 50th percentile is not generally equivalent in raw-score difference to a move from the 90th to the 100th. In skewed or bounded distributions, equal distances on a standard-score scale may also fail to correspond to equal changes in the underlying attribute. Percentiles estimated in small reference samples can have substantial uncertainty, especially near distributional tails. Norm manuals should state estimation procedures, smoothing assumptions, precision, and limits of extrapolation rather than presenting fine-grained ranks as exact individual facts AERA, APA, & NCME (2014) Angoff (1971).
Norm conversions inherit every upstream choice about item content, administration, population sampling, and scoring. A published norm chart should therefore identify version numbers and the precise reference sample from which it was derived. When users compare results across groups or decades, apparently identical percentile values may refer to different distributions. The intended decision may require absolute performance standards or outcome prediction rather than relative rank. Kane (2013) provides a framework for distinguishing these interpretation claims; Messick (1995) emphasizes the need to examine their real-world implications. Clear score reports display uncertainty and state what the scale does and does not express.
Standardization Across Administrators, Settings, and Modalities
Administration standardization seeks to make differences in recorded responses interpretable rather than products of inconsistent testing conditions. Developers specify instructions, practice opportunities, timing, permitted prompts, environmental conditions, accommodations, examiner qualifications, handling of interruptions, and procedures for unusual responses. The objective is to control or document sources of variation that matter to the score interpretation. AERA, APA, & NCME (2014) treats administration and scoring quality as central test responsibilities. Standardized presentation should nevertheless be evaluated for access, because formally identical instructions can impose substantially different unrelated demands on people with different sensory, linguistic, or motor circumstances.
Equivalence between paper, computer, remote-proctored, and mobile administration cannot be presumed from identical wording. Layout, latency, screen size, input devices, distraction, supervision, and perceived privacy may change the tasks respondents perform. For timed measures, technical differences can affect whether the score expresses the intended psychological capacity or the testing platform. A mode-change study should separate population composition from modality effects, assess subgroup interactions, and examine whether score distributions and item parameters shift under controlled conditions. AERA, APA, & NCME (2014) calls for evidence concerning conditions relevant to score comparability; Putnick & Bornstein (2016) supplies an analytic framework for checking measurement equivalence across groups.
Standardization includes quality assurance after procedures have been written. Observed examiner adherence, automated logs, inter-rater agreement, equipment reliability, and incident records can reveal drift. Deviations should be recorded at the level necessary to determine whether scores are comparable or require exclusion, adjustment, or cautious interpretation. Developers should distinguish approved accommodations from untested changes to measurement conditions and document their evidence base. Kane (2013) makes these considerations part of the generalization argument; Flake & Fried (2020) highlights the costs of unreported measurement modifications. Consistent practice makes later normative and equating analyses scientifically interpretable.
Linking, Equating, and Parallel Forms
Test linking places results from two forms or scales into a common interpretive framework; equating makes a stronger claim that suitably transformed scores from forms can be used interchangeably for the specified application. Holland & Dorans (2006) identifies the conceptual distinction and designs supporting equating, including random-groups, single-group, and nonequivalent-groups anchor-test designs. Angoff (1971) provided an earlier comprehensive account of equivalent scores and norm systems. A statistical conversion equation does not itself prove interchangeability, because the forms must meet substantive and measurement assumptions about their content, reliability, target populations, and comparability.
Common-item anchors are informative only when they represent the construct and function comparably in the relevant groups and administrations. If an anchor has been exposed through coaching, changes difficulty after translation, or measures a mode-specific skill, the resulting transformation may be biased. Linear equating and equipercentile methods answer different distributional questions and have different sampling sensitivity; item response methods add parameter-invariance requirements. Holland & Dorans (2006) describes these conditions as methodological rather than purely computational matters. The calibration record must disclose the design, anchor choice, linking error, population definition, and sensitivity to alternative procedures.
Equating programs face cumulative problems when new forms are linked repeatedly across several testing cycles. Small sampling discrepancies and changing populations can propagate through chains of conversions, especially when the underlying construct or curriculum evolves. Developers should evaluate linkage to earlier reference forms and compare results from independent anchor sets when possible. AERA, APA, & NCME (2014) requires support for the intended interpretation and use of scores across forms. For high-stakes applications, a transparent record should distinguish a provisional scale-linking study from a sufficiently supported assertion of exchangeable scores. Similar scale labels are not empirical evidence of equivalent measurement.
Measurement Invariance, Differential Item Functioning, and Fairness
Measurement invariance concerns whether observed responses can be related to the same underlying construct across groups, times, or settings under a specified statistical model. Configural, loading, and intercept or threshold restrictions answer increasingly demanding questions about comparability, although their interpretation depends on the model and data. Putnick & Bornstein (2016) reviews reporting conventions and their limitations. Differential item functioning concerns item-level differences in responses between groups after conditioning on a chosen matching variable. Raju (1988) supplies a formal method based on differences between item characteristic curves. Neither a detected group difference nor the absence of a significant test automatically determines whether an item is substantively fair.
Bias analysis asks whether group differences arise from construct-relevant ability or from barriers such as language-specific interpretations, opportunities to learn, unfamiliar contexts, and inaccessible administration. Statistical DIF is a diagnostic signal to investigate; substantive evidence, response-process inquiry, and intended use determine whether a difference is threatening. Group comparisons can also fail when the matching score itself is biased or when sample size is too small to detect meaningful heterogeneity. AERA, APA, & NCME (2014) places fairness within the full testing life cycle; Beaton et al. (2000) illustrates why adaptation requires more than literal translation. A transparent procedure records both statistical findings and the substantive rationale for retaining or modifying flagged items.
Norm-group separation cannot substitute for measurement invariance. Separate means and standard deviations may describe distributional variation accurately while failing to establish that a given score assesses the same attribute in each group. Conversely, rigid insistence on every model constraint may obscure acceptable practical comparability when deviations are understood and small for the decision context. Putnick & Bornstein (2016) encourages transparent testing and reporting; Kane (2013) asks whether the specific inference can withstand plausible alternatives. Fairness audits should be repeated following item revision, migration to digital platforms, substantial demographic change, or introduction of new decision rules.
Norm Obsolescence, Population Change, and Re-Norming
A normative interpretation is dated because populations, curricula, exposure to test content, public health conditions, and technologies change. Even without test-item revision, the relation between a raw score and its reference-group percentile may shift. The research agenda surrounding long-term changes in cognitive test performance is one prominent reminder that the historical date of a norming sample matters. Yet one observed trend cannot be generalized automatically to every ability, age group, or nation. AERA, APA, & NCME (2014) requires documentation of normative evidence and its appropriate use. Developers should schedule periodic checks of distributional stability rather than assume that a once representative sample remains sufficient indefinitely.
Re-norming should begin with a clearly specified question: has the target population changed, have administration conditions changed, or has the underlying construct or task difficulty shifted? Sampling a new cohort permits estimation of an updated reference distribution, but comparing it with older norms requires attention to form comparability and changing recruitment. If item parameters have drifted, an equating study may be needed before shifts in score levels can be interpreted as changes in the population. Holland & Dorans (2006) explains linking assumptions, and Putnick & Bornstein (2016) shows why invariance across time cannot simply be taken for granted.
Updating norms creates a communication responsibility. A person may receive a different percentile for the same raw score under a new reference distribution, without any change in observed performance. Reports should clearly identify the normative edition and explain whether changes arise from recalibration, resampling, or revisions to administration and scoring. Where longitudinal decisions depend on consistent scales, users may need documented bridge studies. Angoff (1971) establishes the importance of scale semantics, while Kane (2013) relates technical transformations to actual inferences. Norm maintenance is therefore an element of scientific interpretation rather than routine cosmetic publishing.
Cut Scores, Classification, and Decision Consequences
Some assessments use scores to make decisions about eligibility, placement, referral, treatment planning, or professional selection. These applications require evidence beyond the distributional meaning of a percentile. A cut score is a rule that partitions a continuum of results into decision categories; its defensibility depends on the intended outcome, the definition of minimally adequate performance, and the consequences of mistakes. AERA, APA, & NCME (2014) frames the justification of score uses as an essential obligation. Kane (2013) makes the interpretive and decisional argument explicit: an observed performance must be scored, generalized, extrapolated, and used in ways supported by evidence. No percentile alone supplies the social or scientific rationale for a decision threshold.
Classification studies should report sensitivity, specificity, predictive values where relevant, uncertainty around thresholds, and how these quantities change with the prevalence or base rate of the target condition. The appropriate reporting vocabulary differs between clinical screening, educational mastery, and employment selection, but the general question is the same: what errors result from applying a particular decision rule in a specified population? For clinical contexts, predictive estimates and normative rankings should never be converted into individual diagnoses without a validated protocol and professional evaluation. AERA, APA, & NCME (2014) requires attention to the risks and benefits of test use, and Messick (1995) connects these consequences to the broader validity discussion.
Decision thresholds should be audited when the tested population, available interventions, resource constraints, or relative costs of errors change. Developers can compare plausible thresholds using external outcome data and sensitivity analysis rather than selecting the value that maximizes apparent fit on one development sample. Independent assessment is especially important where the same data informed test-item choice and threshold optimization. Flake & Fried (2020) warns about the general consequences of flexible and unreported measurement decisions. The valid application of a score is a joint property of evidence, decision context, and implementation, and should be revised when any of these elements materially changes.
Translation, Cross-Cultural Adaptation, and Accessibility
Translation of psychological tests involves the equivalence of concepts, task demands, response categories, scoring rules, and interpretive conditions. Literal replacement of words is insufficient when cultural practices, idioms, educational opportunities, or the social meaning of an item differ. International Test Commission (2017) offers guidelines governing preconditions, development, confirmation, administration, interpretation, and documentation of test adaptation. Beaton et al. (2000) describes a cross-cultural adaptation procedure including translation, synthesis, back translation, expert review, and pretesting in self-report measures. These processes are methodological aids; the resulting version still requires empirical validation for its intended population and uses.
Adaptation studies should test whether the response process is comprehensible and whether item functioning and scale meaning are stable across language groups. A concept may be familiar to respondents yet expressed differently through conversational norms, while a formally parallel item can involve unequal familiarity with specific experiences. Cognitive interviewing, content review, invariance tests, and differential item functioning analyses address different parts of this problem. Terwee et al. (2018) places content comprehensibility and comprehensiveness at the center of patient-reported measure evaluation; Putnick & Bornstein (2016) discusses the reporting of invariance tests. Where equivalent scores are not established, developers should state precisely which within-group interpretations remain supported.
Accessibility is also a measurement issue. A sensory, motor, or interface barrier can introduce construct-irrelevant variability even when the test is administered exactly according to its original manual. Accessible formats and reasonable accommodations should be designed for the construct and evaluated empirically rather than judged only by visual similarity of prompts. AERA, APA, & NCME (2014) addresses fairness, accommodations, and appropriate interpretation; Kane (2013) links departures from original administration to the strength of generalization and extrapolation claims. An inclusive development program records what accommodations were evaluated and what new evidence is needed when a substantially different delivery mode is introduced.
Digital Test Platforms, Adaptive Administration, and Item Banks
Computerized assessment allows item banking, automated scoring, adaptive routing, rich response records, and rapid feedback. In computerized adaptive testing, the next item can be selected using an estimate of the respondent's current position on a calibrated latent continuum. The method is efficient only under assumptions about stable item parameters, sufficiently broad item coverage, and meaningful construct interpretation. Lord (1980) established the application of response models to practical testing, while Hambleton et al. (1991) explains model foundations and limitations. Developers must examine where the item bank supplies information, how routing operates at scale boundaries, and how test length interacts with precision and accessibility.
The Patient-Reported Outcomes Measurement Information System provides an influential applied example of constructing and evaluating calibrated patient-reported outcome item banks. Cella et al. (2010) describes the development and first-wave testing of adult health outcome banks, illustrating how domain definitions, calibrated item content, and administration strategies can be integrated. This work demonstrates feasibility and methodological opportunity, not universal proof that adaptive administration eliminates bias or guarantees interpretive transport. Every item bank requires its own evidence regarding populations, score precision, response processes, clinical relevance, and maintenance of parameters after content revisions.
Digital platforms additionally create software and governance problems. Changes to display rendering, language models used to generate items, user authentication, device timing, privacy conditions, and automated scoring code can alter the effective test. Operational monitoring must therefore include version pinning, scoring regression tests, accessibility audits, security, anomalous item exposure, and independent data-quality checks. AERA, APA, & NCME (2014) supplies enduring requirements for reliable administration and justified use; Flake & Fried (2020) underlines the importance of transparent measurement decisions. In an Artificial Era context, tests applied to artificial agents or human–artificial interaction require explicit task and population definitions; evidence from human norm samples cannot be treated as directly transferable to nonbiological systems.
Empirical Evaluation, Replication, and Evidence Quality
Evidence for a new test accumulates through different study designs. Content studies can establish coverage and relevance; cognitive interviews probe response processes; calibration studies estimate item properties; independent structural studies evaluate dimensionality; longitudinal designs study stability or change; prospective prediction studies examine outcomes; and randomized administration studies can isolate modality effects. The nine-step program of Boateng et al. (2018) organizes these tasks, while Clark & Watson (2019) explains why good instrument development also requires discriminating tests against related constructs. A strong evaluation program aligns each method with an explicit inference rather than presenting a collection of favorable coefficients as a single validation score.
Evidence quality depends on sample selection, experimental controls, transparency, confidence intervals, independent replication, and plausible alternative explanations. A scale may appear reliable in a homogeneous pilot yet behave differently in the population for which decisions are intended. Equating may fit one administration cohort while drifting in another. Unreported item deletions and post-hoc model changes compromise out-of-sample confirmation. Flake & Fried (2020) describes measurement practices that threaten the credibility of psychological findings. Putnick & Bornstein (2016) explains why subgroup invariance analyses need careful reporting rather than automated pass-fail interpretation. These issues apply even when algorithms produce precise numerical estimates.
Contradictory findings are an essential part of test evaluation. Differences across sites may reveal uncontrolled context effects, changing response processes, or theoretical overgeneralization. The appropriate response is a design that can discriminate among explanations, not simply a new sample with larger numbers. Kane (2013) recommends concentrating new evidence on the weakest inferential links in a proposed use; AERA, APA, & NCME (2014) emphasizes that test documentation should accurately communicate limits. Independent audits of scoring, wording, recruitment, and item functioning should be integrated into the scientific record. Developers must explicitly distinguish evidence for group description, individual prediction, and institutional decisions.
Test Manuals, Lifecycle Governance, and Open Research Problems
A test manual is the public-facing record of the current evidence system. It should describe the construct and intended uses, forms and scoring, administration instructions, normative samples and their dates, precision across score ranges, evidence supporting interpretation, fairness analyses, adaptations, cut-score justifications, and known limitations. AERA, APA, & NCME (2014) establishes professional expectations for documentation and responsible testing. The manual should preserve version identifiers and a revision history that explains substantive changes, allowing readers to determine whether results from different editions remain meaningfully comparable. Technical transparency is especially important when software changes can alter administration or scoring without visibly changing an instrument's title.
Three recurrent open problems define future research. First, test developers must distinguish genuine psychological change from changing reference populations and measurement drift. Second, adaptive and digital assessment must sustain score comparability across devices, accessibility needs, and evolving software. Third, normative and validity evidence must extend to populations historically excluded or poorly represented in test development. Holland & Dorans (2006) offers principles for evaluating cross-form score exchangeability; International Test Commission (2017) provides a framework for translated and adapted tests; Putnick & Bornstein (2016) identifies unresolved practical questions in invariance testing. These programs are complementary and require empirical studies tailored to each new use.
Artificial systems introduce a further methodological challenge: their output distributions depend on architectures, prompts, training histories, interaction contexts, and policy changes. A psychological task developed for people may measure task completion in a machine without licensing the same interpretation of latent human attributes. Norms, response processes, and theoretical assumptions require explicit reconstruction when crossing system categories. Kane (2013) supplies the general discipline of justifying interpretations and uses, while Clark & Watson (2019) supplies principles of construct definition and measurement design. The productive research agenda consists of operational definitions, independently replicated observations, explicit uncertainty, and documented limits to transfer.
Contemporary Testing Guidance, Evidence Appraisal, and Emerging Technologies
Recent methodological guidance makes explicit that test selection and test construction are related but distinct scientific tasks. Mokkink et al. (2024) updated the COSMIN approach to systematic reviews of patient-reported outcome measures, emphasizing advance specification of the review question, study-level risk of bias, evaluation of the measurement properties of subscales, and graded certainty of the cumulative evidence. This framework is important for prospective test developers because it reveals why a new instrument should be created only after examining existing measures and identifying a substantiated unmet measurement need. COSMIN procedures are intended for formal reviews and should not be attributed to an ordinary narrative synthesis. Their relevance here lies in the standards they establish for comparing evidence about alternative instruments.
In contemporary clinical and psychological measurement, the 2025 guide by Stefana et al. (2025) presents five phases and eighteen steps covering assessment of need, conceptual clarification, item generation, scale construction, evaluation, and finalization. Its value is as an explicit framework that links instrument development to a usable manual; the article synthesizes methodological literature and applied experience rather than presenting a new validation trial. The earlier Boateng et al. (2018) nine-step account and the construct-oriented recommendations of Clark & Watson (2019) converge on the importance of representative content, pretesting, principled item reduction, and independent evaluation. A developer should adapt the sequence to the intended use and retain methodological traceability when stages overlap.
The updated July 2025 ITC & ATP (2025) guidance addresses digital assessment design, administration, scoring, security, privacy, accessibility, fairness, and the implications of artificial intelligence and generative systems. Its contemporary importance lies in treating technical change as a possible change to the measurement procedure and to the evidence needed for interpretation. Model-based item generation and automated scoring can expand operational capacity, but they also introduce training-data dependence, version drift, explainability challenges, and exposure of sensitive responses. A digital test must specify how its implementation and modifications are evaluated empirically; AERA, APA, & NCME (2014) and Kane (2013) supply the broader requirements of interpretive justification that remain applicable across technological generations.
Conclusion
Psychological test development establishes a durable relationship between an intended research object, controlled observations, score construction, and defensible interpretation. Standardization makes specific testing operations comparable; norming establishes the reference distribution against which relative position is interpreted. The research reviewed here shows that these achievements require distinct studies and ongoing maintenance. No amount of statistical precision can repair omitted construct content, an irrelevant norm group, unstable item meaning, or an unexamined decision rule. The major testing standards AERA, APA, & NCME (2014) and the interpretation-centered framework of Kane (2013) provide an organizing principle: every assertion about what a score means must be supported by evidence suited to the people, procedures, and uses concerned.
The historical progression from internal consistency and classical score theory to item-response models, modern validation arguments, and contemporary item banks has expanded the available scientific toolkit. Cronbach (1951) and Kuder & Richardson (1937) advanced methods for consistency assessment, Lord (1980) and Samejima (1968) developed response-model approaches, and Holland & Dorans (2006) clarified the conditions of cross-form equivalence. Content, response-process, structural, fairness, and consequential investigations provide different tests of the inferences attached to these numerical outputs. A test is scientifically mature when those distinct kinds of evidence are integrated, assessed critically, and made visible in documentation.
The resulting methodological principle is one of traceable comparability. A test score can support an individual assessment, scientific comparison, or administrative action only through a transparent chain from construct definition to item content, administration, scoring, calibration, norms, inference, and decision. That chain must be revised when populations, technologies, contexts, and theoretical understanding change. For human psychology and future cross-system research alike, disciplined test development requires well-specified objects, informative designs, empirical challenge, and interpretive restraint where evidence remains incomplete. Psychological measurement progresses as a cumulative research practice whose conclusions are explicitly tied to the observations that support them Clark & Watson (2019) AERA, APA, & NCME (2014).
References
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://ncme.org/resources/books/testing-standards/
- Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). American Council on Education. https://www.ets.org/research/policy_research_reports/publications/chapter/1971/jekp.html
- Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. https://doi.org/10.1097/00007632-200012150-00014
- Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, Article 149. https://doi.org/10.3389/fpubh.2018.00149
- Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Amtmann, D., Bode, R., Buysse, D., Choi, S., Cook, K., DeVellis, R., DeWalt, D., Fries, J. F., Gershon, R., Hahn, E. A., Lai, J.-S., Pilkonis, P., Revicki, D., Rose, M., Weinfurt, K., & Hays, R. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. https://doi.org/10.1016/j.jclinepi.2010.04.011
- Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626
- Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
- Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
- Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393
- Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. SAGE Publications. https://www.sagepub.com/shop/buy-a-book/fundamentals-of-item-response-theory-1-3067
- Holland, P. W., & Dorans, N. J. (2006). Linking and equating. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 187–220). Praeger. https://www.ets.org/research/policy_research_reports/publications/chapter/2006/hshn.html
- International Test Commission & Association of Test Publishers. (2025). Guidelines for technology-based assessment (Version 1.1, July 2025). International Test Commission and Association of Test Publishers. https://www.intestcom.org/page/16
- International Test Commission. (2017). The ITC guidelines for translating and adapting tests (2nd ed.). International Test Commission. https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf
- Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
- Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160. https://doi.org/10.1007/BF02288391
- Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates. https://www.ets.org/research/policy_research_reports/publications/book/1980/jexj.html
- Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066X.50.9.741
- Mokkink, L. B., Elsman, E. B. M., & Terwee, C. B. (2024). COSMIN guideline for systematic reviews of patient-reported outcome measures version 2.0. Quality of Life Research, 33(11), 2929–2939. https://doi.org/10.1007/s11136-024-03761-6
- Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
- Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53(4), 495–502. https://doi.org/10.1007/BF02294403
- Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/10.1007/s11336-008-9102-z
- Samejima, F. (1968). Estimation of latent ability using a response pattern of graded scores. ETS Research Bulletin Series, RB-68-02. https://doi.org/10.1002/j.2333-8504.1968.tb00153.x
- Sireci, S. G. (1998). Gathering and analyzing content validity data. Educational Assessment, 5(4), 299–321. https://doi.org/10.1207/s15326977ea0504_2
- Stefana, A., Damiani, S., Granziol, U., Provenzani, U., Solmi, M., Youngstrom, E. A., & Fusar-Poli, P. (2025). Psychological, psychiatric, and behavioral sciences measurement scales: Best practice guidelines for their development and validation. Frontiers in Psychology, 15, Article 1494261. https://doi.org/10.3389/fpsyg.2024.1494261
- Terwee, C. B., Prinsen, C. A. C., Chiarotto, A., Westerman, M. J., Patrick, D. L., Alonso, J., Bouter, L. M., de Vet, H. C. W., & Mokkink, L. B. (2018). COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A Delphi study. Quality of Life Research, 27(5), 1159–1170. https://doi.org/10.1007/s11136-018-1829-0