Regression to the mean is the statistical tendency for unusually high or unusually low observations to be followed, on average, by observations closer to the relevant mean when the two measurements are imperfectly correlated. It explains why an exceptional examination score, a very bad sales month, an unusually high symptom rating, or a remarkable athletic performance may be followed by a more ordinary result. Regression is a property of the relationship between observations, not an invisible force pushing outcomes toward normality. It creates particular problems for causal judgment because an expected statistical change can be mistaken for the effect of praise, punishment, treatment, management, or technology.
A person who receives treatment at the peak of a fluctuating symptom may subsequently feel better partly because of natural variation. A company launching a corrective program immediately after its worst quarter may see improvement partly because the original quarter was unusually poor. In neither case does regression establish that the intervention worked or failed. The appropriate scientific question is how observed follow-up compares with a justified statistical prediction and, when causation is the issue, with the outcome that would have occurred without the intervention. The methodological review by Barnett, van der Pols, and Dobson (2005) explains why this distinction matters in research with repeated measurements.
What Regression to the Mean Means
A definition tied to repeated observations
Suppose X is an initial measurement and Y is a later measurement of the same outcome or a related quantity. If their relationship is positive but imperfect, people observed at the extreme of X generally have a less extreme expected value of Y in standardized terms. When X and Y have the same mean and standard deviation, and the conditional relationship is linear, the expected standardized follow-up equals the first standardized observation multiplied by the correlation. If the initial observation is two standard deviations above average and the correlation is 0.60, the expected follow-up is 1.20 standard deviations above average. The expected change is 0.80 standard deviations toward the mean. These values illustrate an assumed model, not a universal psychological effect size.
The same model applied to an initial observation two standard deviations below average predicts a follow-up 1.20 standard deviations below average. Regression is therefore not intrinsically improvement or deterioration. Its practical meaning depends on whether larger or smaller values are desirable. A high score on an examination may be followed by a lower score; a high pain rating may be followed by a lower rating; an exceptionally low sales total may be followed by a higher total. The change can be real even when its causal interpretation is wrong.
The formal conditional prediction
The familiar linear regression expression is E[Y | X = x] = μY + r(σY/σX)(x − μX). Here μX and μY denote the respective means, σX and σY the standard deviations, and r the correlation between baseline and follow-up. With equal means and spreads, this simplifies to E[Y | X = x] = μ + r(x − μ). Under a bivariate normal model, the expression gives the actual conditional expectation. Outside that model, it represents the best linear prediction under ordinary least-squares criteria, while the true conditional expectation can be nonlinear.
The formula shows why the mean used for comparison must be specified. If the population becomes healthier, students learn material, or an instrument changes its calibration, the second mean may differ from the first. A prediction using the wrong reference group can confuse a population trend with regression. Researchers also need to distinguish raw scores from standardized values: if standard deviations differ between time points, a change in standardized extremity is not necessarily a change of the same size on the original measurement scale.
Why the prediction is probabilistic
A conditional expected value summarizes the center of a distribution. It does not instruct every individual outcome to move in the predicted direction. Someone with an exceptional first result can outperform it on the second occasion; a very ill participant can become worse; an athlete can set another record. Regression describes the average follow-up among comparable observations under specified conditions. The appropriate individual forecast also needs an uncertainty interval and any relevant predictors of stable differences between people.
Historical Origins and the Psychology of Prediction
Galton's discovery
The nineteenth-century statistician Francis Galton described regression toward mediocrity while examining familial stature. Children of exceptionally tall parents tended to be tall, yet their average height was less extreme relative to the population than the parents' height; a corresponding pattern applied to unusually short parents. His published account established a language and geometry later generalized to many kinds of paired variables (Galton, 1886). That historical study involved measurements of different people across generations. The same statistical principle appears when a single person's outcome is measured more than once, provided the joint relationship meets the relevant conditions.
The nineteenth-century wording reflected the historical setting. Modern statistical usage does not treat average outcomes as more valuable or more desirable. Stephen Stigler's reconstruction of the discovery shows how an apparently simple pattern became a major conceptual development in statistics, while repeated misunderstandings of its meaning survived (Stigler, 1997).
Kahneman and Tversky on extreme predictions
A separate psychological question is whether people understand how far an initial impression should predict a future result. In their research on the psychology of prediction, Daniel Kahneman and Amos Tversky showed that people could make extreme predictions on the basis of descriptions that were highly representative of an outcome even when the evidence had limited reliability. The research connected overly extreme forecasts with neglect of prior probabilities and predictive validity (Kahneman and Tversky, 1973). Their wider account of judgment under uncertainty placed representativeness alongside availability and anchoring as important heuristic processes (Tversky and Kahneman, 1974).
The distinction is precise. Regression to the mean is a statistical pattern that can occur whether anyone notices it. A regression-related judgment error occurs when a person disregards predictive reliability or attributes statistically expected change to an event without a suitable causal comparison. The statistical model supplies a criterion against which a forecast can be evaluated; psychological experiments investigate how people actually reason in particular tasks.
Why Extreme Values Become Less Extreme
Temporary disturbances and measurement reliability
Observed outcomes often combine relatively persistent influences and temporary variation. An examination score may reflect subject knowledge, but also sleep, illness, anxiety, question selection, and guessing. Sales totals reflect skill and customer relationships alongside short-lived fluctuations in demand. Symptom measurements reflect underlying conditions and momentary biological or reporting variations. Selection of the most extreme observed cases will disproportionately include occasions on which temporary disturbances made values especially high or low.
If the temporary component is less likely to repeat in the same direction, follow-up results usually include less extreme temporary contributions. This produces regression without requiring the underlying stable attribute to change. Greater measurement error or weaker baseline-follow-up correlation can make the phenomenon more conspicuous in selected samples. Barnett and colleagues explicitly identify measurement variability and selection using baseline as important determinants of apparent change (Barnett, van der Pols, and Dobson, 2005).
Real fluctuations and instrument error are different
The observed pattern may occur even when instruments are accurate. A genuine one-day increase in temperature, pain, concentration, or performance can be transitory. The second measurement may record a different real state, not a correction of an earlier recording mistake. Conversely, a stable underlying quantity may produce varying observations because instruments or respondents are imperfect. Those mechanisms should be distinguished when designing a study. Labeling every return toward typical levels as instrument error overlooks meaningful variation in the underlying process.
A simple true-score-plus-error model is useful but depends on assumptions about independence, stability, and the meaning of the latent quantity. Some psychological characteristics change across development, treatment, learning, or repeated experience. In such settings, predictive models must allow underlying states to evolve and may need time-varying covariates. A test-retest correlation mixes imperfect measurement with actual temporal instability unless a richer design separates them.
Selection on an unusually high or low baseline
Regression becomes particularly visible when participants enter a study because a first measurement crosses a threshold. A program recruits students with exceptionally low scores, patients with high symptom ratings, sales teams after poor performance, or applicants whose interview ratings were unusually strong. Those selection criteria create a baseline distribution enriched for extreme results. Subsequent results are sampled from a related distribution, and the follow-up average tends to be less extreme when repeated measurements are imperfectly correlated.
The original extreme observation is part of the selection process. Comparing selected participants with their own earlier values can therefore make a program appear to achieve change that would have happened in the absence of that program. This threat applies even if the investigators have measured every participant honestly and reported all their available results. It is a design problem that requires an appropriate comparison and analysis, not merely reassurance that the observations are genuine (Bland and Altman, 1994).
Strength, Boundary Conditions, and Interpretation
How correlation changes the expected amount
In the simple equal-mean, equal-variance model, the expected movement from an extreme baseline toward the mean is the baseline deviation multiplied by (1 − r). If a person starts 3 standard deviations above average and r = 0.50, the expected follow-up is 1.50 standard deviations above average. With r = 0.95, the expected follow-up is 2.85. A more reliable first measurement therefore supports an expected result that remains closer to the original extremity. The numerical values illustrate the effect of an assumed correlation. They should not be quoted as typical measurements across educational, clinical, or occupational contexts.
When r = 1 with equal means and spreads, the second score is perfectly predicted by the first and no regression of the conditional expected standardized value occurs. When r = 0, the best linear prediction of Y is its mean no matter how extreme X was. Negative correlations can reverse expected deviations rather than merely reduce them. The relation between correlation and regression is not a claim that high correlations make all interventions effective or that low correlations make all changes artificial.
Different means, skewness, and bounded scales
Real measurements often violate the idealized symmetric model. Test scores may have ceilings, symptom scales have floors, and monetary outcomes can be severely skewed. Between two examinations, mean performance may increase because of practice or instruction. A treatment may genuinely shift the entire follow-up distribution. In these cases, researchers should model changing means, unequal variances, and nonlinear conditional relationships instead of applying the textbook expression mechanically.
Consider a class in which every student learns considerably between tests. Some initially outstanding students may rise in raw points while becoming less outstanding relative to the improved class. Calling that movement deterioration would misrepresent both the raw data and the relevant comparison population. Similarly, a population-wide improvement and regression among initially distressed people can occur simultaneously. Statistical regression helps decompose expectations; it does not rule out genuine changes in a group's level.
Heterogeneity between individuals and contexts
Different subgroups may have different stable means, variation, and repeatability. Comparing an elite athlete with an average adult athlete could produce a profoundly misleading mean. Some patients have chronically elevated readings, while others exhibit sharp episodic changes. A good model accounts for persistent individual differences, relevant covariates, context, and measurement intervals. Regression toward a subgroup or personalized expected level may be more meaningful than regression toward a broad population average.
The size of regression should be estimated with uncertainty rather than characterized as universally large or small. It is usually possible to measure the baseline-follow-up relationship in historical data, provided those data resemble the target population. Even then, sampling error, attrition, time trends, and changing selection criteria can undermine transportability. There is no universal percentage of psychological change that must be due to regression to the mean.
Statistical Regression and Nearby Concepts
Regression to the mean versus the law of large numbers
The law of large numbers describes how averages from growing collections of suitably structured observations approach expected values under specified assumptions. Regression to the mean describes the conditional distribution of a second measurement after selection of an extreme first value. These mathematical ideas can intersect but serve different purposes. A fair coin's cumulative fraction of heads may become less extreme as the number of tosses grows, yet the next independent toss does not change its probability because the earlier sequence was unusual.
Regression versus the gambler's fallacy
The gambler's fallacy expects independent random events to compensate for earlier outcomes. After several heads, a person may think tails is due. In an independent fair-coin process the probability of tails on the next toss remains one-half. Regression does not imply that a corrective event must occur. It instead describes expectations for correlated observations or an expanding average under a different sampling question. A distinction between event probabilities and averages across observations prevents many misleading intuitions about streaks.
Regression versus a genuine hot hand
The hot-hand question concerns whether success on one attempt changes the probability of success on subsequent attempts, perhaps because of skill, feedback, momentum, defensive responses, or selection. Regression to the mean may occur even if some serial dependence exists, and a continuing streak can occur even if group-level conditional predictions regress. Sport-specific evidence is therefore needed to decide whether performance dependence is real. A return to customary performance after a remarkable run does not prove the run was purely chance, while a second remarkable run does not disprove statistical regression.
Regression versus dynamic mean reversion
In time-series analysis, mean reversion can describe an underlying dynamical process that pulls a variable toward a long-run equilibrium. Regression to the mean in repeated measurements can appear even when no restorative force operates. Selection on an extreme initial value, imperfect stability, and the form of the joint distribution are enough. Time-series predictions require assumptions about stationarity, autocorrelation, external shocks, and structural changes. Equating statistical regression with a causal tendency to return to equilibrium risks inventing a mechanism from an observed association.
Regression versus a cognitive bias or disorder
Regression itself is a statistical characteristic of data. Overextreme prediction, inaccurate causal attribution, and selective interpretation are possible psychological errors involving that characteristic. A logical fallacy concerns a faulty inferential structure; a statistical error concerns an invalid calculation or model; a heuristic is a strategy or process for making judgments. No one has a psychiatric diagnosis simply because they misunderstand regression to the mean. These categories answer different scientific questions and should remain distinct.
Psychological Mechanisms Behind Misinterpretation
Representativeness and extreme predictions
A persuasive profile of a brilliant student or promising employee can elicit a forecast of equally remarkable future performance. The observer may focus on how strongly the profile resembles success instead of how well such profiles predict actual outcomes. When evidence is imperfect, a statistically justified forecast should be tempered according to its predictive validity and the relevant prior distribution. Kahneman and Tversky's experimental research directly investigated this kind of overextreme prediction (Kahneman and Tversky, 1973).
An appropriate forecast does not simply say that everyone is average. Persistent differences in achievement and skill matter. If the evidence is highly reliable and the candidate belongs to a genuinely exceptional subgroup, an unusually high prediction may be warranted. Psychological error arises when extremity exceeds what the available evidence justifies, not whenever someone forecasts an exceptional result.
Praise, punishment, and the illusion of causal effect
Imagine that an instructor praises a student immediately after an exceptional score. The next score is less exceptional, and the instructor concludes that praise was harmful. A different student performs exceptionally poorly, receives a stern reprimand, and subsequently scores better. The instructor concludes that punishment works. The timing encourages a causal story, yet the comparison is built around extreme first outcomes. Some movement toward usual performance might have occurred without either response.
This example is an illustration of confounding by selection, not experimental evidence that feedback has no effect. Praise, criticism, learning, effort, and classroom conditions can change performance in their own right. To estimate a feedback effect, investigators need an appropriate control or comparison with the expected untreated trajectory. A compelling before-after sequence supplies a hypothesis, but the statistical effect of selection must be accounted for first.
Hindsight and outcome-based stories
Once an outcome is known, explanations often become more convincing than they would have been prospectively. A recovery follows a new supplement; a sales rebound follows a leadership announcement; a team wins after changing its routine. The observer remembers the intervention and constructs a story from the sequence. Regression provides a statistical alternative that should be evaluated alongside real changes, seasonality, and other causes. The precise contribution of hindsight bias, confirmation bias, or motivated reasoning is a further empirical question rather than an automatic consequence of regression.
Organizational incentives
Organizations can inadvertently make regression errors more consequential by evaluating managers after an unusually bad quarter, ranking teachers by changes after exceptionally weak scores, or selecting clinical participants on their highest observed distress. Such selection procedures create strong incentives for improvements that partly reflect a natural statistical rebound. Transparent pre-specified inclusion rules, historical benchmarks, uncertainty ranges, and controlled comparisons reduce the risk of rewarding or penalizing people for random fluctuations.
Scientific Evidence and Reproducibility
The 1973 prediction experiments
Kahneman and Tversky studied judgments based on descriptions, prior probabilities, and imperfect diagnostic evidence. Their findings supported the importance of representativeness in predictions and showed that intuitive forecasts could be too extreme relative to available predictive evidence. Their original participants and tasks support conclusions about these experimental judgments, not the claim that every human judgment in every culture displays an identical error (Kahneman and Tversky, 1973). The mathematical condition for regression and the prevalence of regression-related cognitive mistakes are separate kinds of evidence.
Methodological research across repeated measurements
Bland and Altman's statistical notes clarify that selecting an extreme group based on one measurement and then measuring it again can produce a group mean closer to the overall mean. They explicitly connect the magnitude of the shift with the correlation between measurements and offer concrete examples in scientific reporting (Bland and Altman, 1994a; Bland and Altman, 1994b). Barnett, van der Pols, and Dobson synthesized practical approaches to the same problem in epidemiology, emphasizing reliability, selection, design, and analysis (Barnett et al., 2005).
These papers explain and demonstrate a statistical principle; their purpose is not to supply a single effect size applicable to psychology. Differences in measurements, populations, time intervals, and selection criteria preclude such a universal estimate. When investigators report a regression-related bias in a particular trial, its magnitude must be estimated in that trial or justified through an applicable external model.
What replication means in this domain
Conditional regression under a specified probability model can be verified algebraically or through simulation. Empirical reproducibility raises different questions: do particular populations show the assumed correlation, do people systematically neglect reliability under a particular task framing, and does a specific intervention improve judgments outside its training context? Designs should report sample selection, analytic specifications, uncertainty, distributions, preregistration where appropriate, and independent assessments. Replicating a mathematical relationship does not automatically replicate a psychological mechanism.
Cultural and methodological limits
Numeracy, familiarity with statistical concepts, language, incentives, and local decision practices may change how people answer prediction questions. Laboratory studies with students cannot establish global rates of misunderstanding. Results from standardized tasks may also differ from expert decisions based on extensive feedback. An authoritative scientific account therefore treats the statistical pattern as established within its assumptions and treats specific claims about recognition, causal storytelling, or debiasing as empirical and potentially context dependent.
Measurement and Research Design
Visualizing the conditional relationship
A scatterplot of initial and later measurements can make regression visible. With comparable scales and positive correlation below one, a fitted conditional prediction line generally rises more gradually than the identity line in standardized coordinates. Cases selected at an extreme baseline lie near the edges of the horizontal axis, while the conditional average of their second values is less extreme. Scatterplots should show raw points, confidence or prediction uncertainty where appropriate, and potential nonlinear patterns; a line fitted to aggregated values can conceal important subgroup structure.
Test-retest designs and multiple baselines
When the objective is to estimate usual performance, several measurements are often more informative than one. Repeated observations can help distinguish short-lived disturbances from a persistent level. They do not solve every problem: repeated tests may cause learning, fatigue, sensitization, or changes in measurement conditions. Designs should account for time dependence and distinguish within-person variation from between-person variation. The measurement model, not mere repetition, determines what can be inferred.
Why uncontrolled before-after comparisons mislead
A treatment group recruited because of unusually high scores is not its own unbiased counterfactual. Its average value may fall at follow-up because of regression, natural history, measurement variation, attrition, seasonal changes, or a real treatment effect. Comparing follow-up with baseline establishes an observed difference; it does not by itself partition those possibilities. A contemporaneous comparator selected by the same criteria or a sufficiently credible longitudinal model is required to make stronger causal statements.
Randomization and baseline-adjusted analysis
In randomized trials, regression can occur in both treatment and control groups when entry depends on an extreme baseline. The causal estimand is the difference between appropriately comparable outcomes under treatment and control, not the treated group's difference from its own selected baseline. For suitable continuous-outcome trials, analysis of covariance that adjusts follow-up for baseline commonly offers advantages over a naive comparison of change scores. Vickers and Altman discuss the methodological rationale and conditions for these approaches (Vickers and Altman, 2001).
Baseline adjustment should be pre-specified and interpreted according to the study design. In observational analyses, baseline status may interact with confounders, selection, and measurement error in ways that require more elaborate models. Statistical sophistication cannot rescue a causal claim when crucial assumptions are implausible or unexamined.
Percent change and mathematical coupling
Analysts sometimes compute percentage improvement as the difference between follow-up and baseline divided by baseline. Because the starting value appears directly in the change metric, correlations between baseline and change can arise mechanically, especially when baseline is noisy or close to zero. A high apparent improvement among initially extreme cases may therefore involve mathematical coupling as well as true regression. Researchers should specify their estimand and prefer interpretable absolute outcomes or appropriately adjusted comparisons when percentage measures obscure the mechanism.
Education, Talent, and Performance Evaluation
Tests and selection programs
Students selected for tutoring after unusually low screening scores can improve on a second test even without instruction, especially when the screening score contains temporary disturbances. A credible evaluation compares students under equivalent selection rules and accounts for practice, learning, timing, and differences between test forms. The same issue affects enrichment programs that select only the highest scoring students: later results can be less exceptional even when the program is beneficial.
Hiring and talent identification
An unusually strong interview, assessment center result, or sales month can reflect high ability and favorable transient influences. Predicting future performance as if the complete extreme result were perfectly repeatable leads to overconfident rankings. Better prediction combines multiple validated observations, estimates reliability, and incorporates appropriate information about the candidate and work context. Shrinking forecasts toward a justified reference distribution can improve expected accuracy under particular models; a blanket rule that exceptional people become average would ignore genuine persistent differences.
Praise, criticism, and management incentives
A manager might introduce strict oversight after a disastrous project and credit a rebound to the new policy, or withdraw rewards after a stellar quarter and blame subsequent ordinary performance on declining motivation. Those narratives are hypotheses. Performance evaluations should examine comparable periods, workload, clients, seasonal cycles, and counterfactual controls. Policies that evaluate supervisors only by change from an extreme baseline risk exaggerating their contribution and misallocating resources.
Health, Clinical Studies, and Safety
Screening thresholds and fluctuating symptoms
Health research frequently recruits participants because an observation crosses a high or low threshold. Pain severity, blood pressure, and self-reported symptoms may fluctuate over time. A follow-up value nearer the typical level is compatible with regression, treatment response, concurrent intervention, or natural disease progression. The review by Barnett and colleagues specifically discusses how baseline selection can distort interpretation of apparent clinical changes (Barnett et al., 2005).
Statistical regression never justifies ignoring an abnormal or alarming clinical result. Repeat testing, urgent assessment, or treatment must follow the relevant professional standard and the person's individual presentation. The proper research lesson is to estimate causal effects carefully, not to dismiss individual health concerns as statistical noise.
Placebo groups and clinical trial interpretation
A treatment offered during a symptom peak may be followed by improvement partly because of regression. Yet placebo-group improvement can also reflect natural history, expectancy, reporting shifts, and concurrent care. Those processes are distinguishable. A randomized comparison can help evaluate the difference attributable to a specific treatment under the trial's design assumptions. Researchers should avoid using regression as a catch-all explanation for every change in untreated participants.
Longitudinal psychological scales
A rating scale may combine stable individual tendencies, current emotional or cognitive states, response style, and measurement variability. If researchers select participants at exceptionally high baseline scores, follow-up averages can decline without any new therapy. Reliability, measurement invariance, attrition, and changes in the construct itself all matter. The measured pattern is not a clinical symptom or diagnostic indicator. A scale's score and a clinician's diagnosis also serve different purposes; regression should be analyzed as a property of the repeated outcome.
Sports, Finance, and Technology
Athletes and winning streaks
Athletic performances are shaped by stable skill, opponents, injuries, strategies, opportunities, and chance. Someone selected after a record-setting season may post a less extreme subsequent season. The reference distribution must nevertheless reflect that athlete's level of competition and persistent ability. Genuine improvements in training or injuries can alter the trajectory, and some streaks can involve true dependence. Regression alone neither confirms nor refutes a particular hot-hand effect.
Investment and business results
A fund, business unit, or sales representative selected as the highest performer in one period may be less exceptional in the next. This is especially important when many candidates were ranked and only the most extreme selected. Selection of a winner increases the chance that favorable transient variation contributed to the observed rank. Future performance still depends on economics, strategic changes, skill, risk, and market structure. Regression to the mean is not an investment trading rule that makes past winners certain to lose.
A/B testing and product analytics
Digital product teams often respond to a suddenly bad metric by launching a change. If they then compare the revised period only with the unusually bad starting interval, natural variation can masquerade as success. Randomized experiments with concurrent controls, clearly pre-specified outcomes, sufficient observation windows, and honest accounting for seasonality help separate effects. Analysts should also check autocorrelation, data quality, composition of traffic, and simultaneous interventions. Regression is part of the diagnostic reasoning, not a substitute for experimental design.
Human–AI decision support
Automated systems can overstate causal stories too. A forecasting model trained on selected extreme outcomes may produce poorly calibrated results when deployed on more typical cases. A language-model summary of a before-after chart may narrate the apparent improvement as evidence of a policy's success without considering baseline selection. Responsible decision support should expose how cases were selected, display uncertainty, distinguish prediction from causality, and evaluate recommendations against appropriate reference data. These are empirically testable design propositions for human–AI cognition and decision support within the broader technological context of the Artificial Era.
Shrinkage, Bayesian Prediction, and Calibration
Statistical shrinkage is not identical to regression
Shrinkage methods move uncertain estimates toward a reference value or model-based distribution as a deliberate estimation procedure, often to reduce prediction error under a specified loss function. Regression to the mean is a relationship between measurements or their conditional distributions. The mathematical connections are important, but the concepts differ in purpose: a shrinkage estimator is constructed by an analyst, while regression can be observed without any estimator being applied. Efron and Morris demonstrated the practical value of Stein-type estimators in applications including baseball averages (Efron and Morris, 1975).
Bayesian updating and reliability
Bayesian prediction combines an appropriate prior distribution with information supplied by the observed data and the likelihood model. When a measurement is noisy, the posterior estimate of a latent characteristic can be less extreme than its observed value. This parallels regression's intuition that limited information should support limited confidence, but the degree of Bayesian shrinkage depends on the prior, likelihood, and loss function. It should not be assumed that any movement toward a grand population average is justified. Subgroups, measurement bias, and distribution shift require explicit modeling.
Calibrated forecasts
A forecast is well calibrated when predicted probabilities and observed frequencies agree appropriately over an evaluation set. Regression-aware prediction can improve calibration when models recognize unstable extreme signals. Out-of-sample validation should examine systematic underprediction of persistently exceptional people and overprediction of people selected after lucky outcomes. The appropriate scoring criterion must be specified, whether squared prediction error, log loss, a proper probabilistic score, or another outcome-based standard.
Preventing Errors in Everyday and Professional Decisions
Ask what would have happened anyway
Before attributing improvement to a policy, treatment, reward, or reprimand, ask what trajectory comparable cases would have followed without that action. This counterfactual question is essential when the original decision to intervene was triggered by an extreme outcome. Randomization can answer it under favorable conditions. In other settings, a carefully designed comparator, plausible natural experiment, or credible preintervention time series may help, with limitations stated explicitly.
Collect a defensible baseline
One extreme observation can be a poor summary of an underlying process. Multiple high-quality baseline measurements, collected at pre-specified intervals, can reduce the influence of temporary disturbance and allow a more realistic estimate of normal variation. Care is needed when repeated measurement changes the phenomenon, as with practice effects or treatment initiated during observation. In clinical and safety-critical contexts, needed intervention should never be delayed solely to create a neater baseline.
Estimate reliability and conditional expectations
Use empirical correlations and appropriate predictive models to estimate how much movement would be expected from a selected extreme score. Report the distribution, uncertainty, and relevant subgroup differences. For illustration, an initial standardized score of +2 with r = 0.70 predicts +1.40 under the simple common-scale linear model. The implied mean movement is 0.60 standard deviations toward the reference mean; the actual individual outcome remains uncertain. This estimated statistical movement should be considered before claiming a specific intervention caused the observed change.
Prevent selective baseline selection
A report becomes misleading if analysts choose an unusually bad month after seeing later improvements and present it as the sole starting point. Predefine eligibility criteria, outcomes, comparison windows, and analytic methods. Show the broader history whenever feasible, including typical periods and other extreme episodes. Documenting the full timeline helps distinguish a sustained trend from a short-lived deviation and reduces selective causal storytelling.
Statistical education and decision aids
Experiments by Fong, Krantz, and Nisbett found that statistical instruction could improve aspects of everyday reasoning about sampling and the law of large numbers in particular settings (Fong, Krantz, and Nisbett, 198690001-0)). Morewedge and colleagues reported improvements on measured judgment biases after interventions using games or videos, with particular biases and experimental outcomes rather than a universal guarantee of transfer (Morewedge et al., 2015). Neither study establishes that a short lesson eliminates every regression-specific causal error in clinical, business, or technological decisions. Useful decision aids require direct evaluation of transfer and real-world outcomes.
Scientific Status and Open Questions
What is established
Given a specified joint distribution and statistical assumptions, regression to the mean is a mathematical consequence of the conditional relationship. Its relevance to repeated measurements and extreme-baseline selection is extensively established in statistical methodology. The conditional formula under a bivariate normal model is exact; the best-linear-prediction interpretation is more general. The size and direction in an actual population depend on estimated means, spreads, correlations, measurement processes, and selection conditions.
What remains empirical
The degree to which observers recognize regression in a given decision task, the role of representativeness or causal storytelling, and the effectiveness of teaching interventions are empirical questions. Results vary with task presentation, statistical information, incentives, participant experience, and research design. A culture-independent mathematical relation does not establish culture-independent rates of psychological errors. Replication of an inferential behavior must be assessed independently from verification of the underlying theorem.
Implications for causal inference
An apparent effect after an extreme baseline is not sufficient to identify a cause. The strongest conclusion available from a simple before-after contrast is that measured outcomes differed across occasions. Estimating intervention effects requires an appropriate comparison and assumptions about randomization or exchangeability, missing data, time trends, and measurement. Recognizing regression is part of responsible research design and trustworthy decision-making; it does not automatically remove all other biases.
Frequently Asked Questions
Does regression to the mean mean every exceptional result gets worse?
No. It concerns the expected second measurement among cases with extreme first measurements under an imperfectly correlated relationship. Some individuals improve further, remain exceptional, or deteriorate much more. Predictions need distributions and uncertainty, not a deterministic rule about what happens next.
Is regression caused only by inaccurate measurement?
No. Genuine temporary changes in performance, symptoms, environmental conditions, and other influences can create imperfect stability even with accurate instruments. Measurement error is one important pathway. Investigators must examine which sources of variation their instruments and design capture.
Can a treatment work when participants' results also regress toward the mean?
Yes. Regression can coexist with a real beneficial, harmful, or null treatment effect. The appropriate controlled comparison evaluates the additional change attributable to treatment beyond natural and statistical movement. Clinical decisions should follow individual assessment and established professional evidence.
Is regression the same as the gambler's fallacy?
No. The gambler's fallacy wrongly expects independent events to compensate for a prior streak. Regression refers to conditional expectations or repeated measurements under a different statistical relationship. A fair independent coin remains a fifty-fifty proposition on its next toss, whatever sequence preceded it.
How can researchers reduce the problem?
They can use credible control groups, pre-specified eligibility criteria, repeated high-quality baseline measures, baseline-adjusted outcome analysis where justified, suitable longitudinal models, and transparent uncertainty. No single analytical technique is appropriate for every design.
How can I tell whether an apparent improvement is statistically expected?
Start with the reference population and the reason the extreme initial observation was selected. Estimate the empirical relationship between baseline and follow-up, allowing for time trends and measurement reliability. Compare actual outcomes with a justified predictive or causal model. Without these ingredients, a before-after change alone cannot identify the contribution of regression.
Is regression to the mean a mental-health symptom?
No. It is a statistical property of measurements. Incorrect causal reasoning about it is an inferential error, not a clinical diagnosis. Psychological or clinical interpretation of an individual's condition involves separate evidence.
Does AI eliminate regression-related errors?
No. Models can be trained on selected extreme outcomes, and automated narratives may overlook causal design. Systems should be evaluated for predictive calibration, selection sensitivity, and appropriate causal comparisons. Human oversight matters, but it must itself be informed by sound statistical reasoning.
Conclusion
Regression to the mean is essential to understanding why the second observation after an extreme first result is often less extreme. It links measurement reliability, predictive correlation, sampling rules, and causal interpretation. The scientific task is to separate what a statistical model already expects from what a treatment, policy, or other intervention plausibly changes. The psychological task is to recognize the attraction of vivid narratives built around unusually good or bad starting points and to replace such stories with calibrated forecasts and defensible comparisons. Responsible judgment begins by specifying the reference distribution, measuring uncertainty, and evaluating causes against appropriate counterfactual evidence.
Related Articles
Availability Heuristic: How Recall Shapes Probability Judgments — Examine how ease of recall shapes frequency estimates and expectations.
Base Rate Neglect: Why Prior Probabilities Matter — Understand how population distributions and predictive information inform forecasts.
Anchoring Bias: How Initial Numbers Influence Estimates — Explore how starting values shape numerical judgments.
Cognitive Debiasing Strategies: Which Interventions Work? — Review structured approaches to improving judgment.
Cognitive Biases: Definition, Types, and Psychological Research — Situate regression-related errors in the science of judgment.
References
Barnett, A. G., van der Pols, J. C., and Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it. International Journal of Epidemiology, 34(1), 215–220. https://doi.org/10.1093/ije/dyh299
Bland, J. M., and Altman, D. G. (1994a). Regression towards the mean. BMJ, 308, 1499. https://doi.org/10.1136/bmj.308.6942.1499
Bland, J. M., and Altman, D. G. (1994b). Some examples of regression towards the mean. BMJ, 309, 780. https://doi.org/10.1136/bmj.309.6957.780
Efron, B., and Morris, C. (1975). Data analysis using Stein's estimator and its generalizations. Journal of the American Statistical Association, 70(350), 311–319. https://doi.org/10.1080/01621459.1975.10479864
Fong, G. T., Krantz, D. H., and Nisbett, R. E. (1986). The effects of statistical training on thinking about everyday problems. Cognitive Psychology, 18(3), 253–292. https://doi.org/10.1016/0010-0285(86)90001-0
Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246–263. https://doi.org/10.2307/2841583
Kahneman, D., and Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251. https://doi.org/10.1037/h0034747
Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H., and Kassam, K. S. (2015). Debiasing decisions: Improved decision making with a single training intervention. Policy Insights from the Behavioral and Brain Sciences, 2(1), 129–140. https://doi.org/10.1177/2372732215600886
Stigler, S. M. (1997). Regression towards the mean, historically considered. Statistical Methods in Medical Research, 6(2), 103–114. https://doi.org/10.1177/096228029700600202
Tversky, A., and Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131. https://doi.org/10.1126/science.185.4157.1124
Vickers, A. J., and Altman, D. G. (2001). Analysing controlled trials with baseline and follow up measurements. BMJ, 323, 1123–1124. https://doi.org/10.1136/bmj.323.7321.1123