Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

Artificial Metacognition and Confidence

Abstract. Artificial metacognition and confidence concern an artificial system's capacity to estimate the reliability of its cognitive performance, report uncertainty, detect errors, and use confidence-related signals to regulate action. This fundamental research monograph examines the conceptual roots of metacognitive monitoring and control, probability forecasting, calibration theory, verbal uncertainty, internal confidence readouts, semantic entropy, selective prediction, and tool-mediated decision policies. It synthesizes primary empirical studies from language modeling and computational psychology, including the 2026 causal investigation of confidence-guided abstention. The analysis distinguishes first-order accuracy, second-order sensitivity, probabilistic calibration, informational value, and causal control; assesses confounds from prompts, grading, model version, and distribution shift; and develops an evidence-based agenda for reproducible study of versioned artificial systems. The method is a critical narrative synthesis of verified published research; no new experimental data or registered systematic-review protocol are claimed.

Keywords: artificial metacognition; confidence calibration; artificial cognition; metacognitive monitoring; metacognitive control; verbalized confidence; predictive uncertainty; P(True); P(IK); Brier score; expected calibration error; selective prediction; abstention; semantic entropy; error monitoring; confidence representations; distribution shift; Psychology of Artificial; Artificial Era

The Scientific Object of Artificial Metacognition and Confidence

Artificial metacognition is the measurable capacity of an artificial system to estimate, communicate, and use information about the reliability of its own task performance. Its experimental object is a system under a specified configuration: model weights, instructions, decoding procedure, context, tools, retrieval sources, memory, evaluation protocol, and feedback policy. Confidence is an operational forecast of an outcome, commonly the probability that a particular answer satisfies a prespecified correctness criterion. A coherent verbal explanation is a separate observable and should be evaluated independently. The distinction matters because convincing language can accompany inaccurate answers, while a terse probability can carry useful decision information Flavell (1979) Fleming & Lau (2014) Jiang et al. (2021).

Metacognitive monitoring estimates how well first-order processing is proceeding or has succeeded. Metacognitive control uses such estimates to alter subsequent operations: abstaining, seeking evidence, checking a calculation, revising a hypothesis, choosing a tool, or handing a task to another decision maker. A system may display informative confidence without using that information to regulate action, and may use reliability cues to abstain without producing a faithful numerical report. Evaluation therefore requires separate measurements of first-order accuracy, second-order discrimination, probability calibration, and the causal impact of confidence on downstream behavior Maniscalco & Lau (2012) Desender et al. (2018) Kumaran, Daw, et al. (2026).

The fundamental research questions concern the source of an uncertainty signal, the outcomes it predicts, the conditions under which it generalizes, and whether decision policies actually depend on it. These questions apply beyond language models to classifiers, predictive models, decision policies, and tool-using agents, but the empirical protocols differ across those architectures. Here the center of gravity is language-based artificial systems, with statistical learning and human metacognitive research supplying the essential conceptual and measurement foundations. The work is an evidence-based critical narrative synthesis of published research; it reports no original experiment, preregistered search, systematic review, or meta-analysis Geng et al. (2024) Ovadia et al. (2019) Fleming & Lau (2014).

Monitoring, Control, and the History of Metacognitive Inquiry

John Flavell's 1979 formulation established metacognitive knowledge and cognitive monitoring as important topics in developmental psychology. That tradition studied how people understand their own memory and comprehension, monitor uncertain performance, and select strategies when information is incomplete. Later experimental work separated performance on a task from the accuracy of confidence judgments about that performance. This historical distinction supplies a productive vocabulary for artificial-system research because it identifies a measurable relation between first-order decisions and second-order evaluations rather than relying on whether a system uses introspective words Flavell (1979) Fleming & Lau (2014).

Signal detection approaches sharpened the operational framework. Maniscalco and Lau introduced meta-d-prime as an estimate of the task sensitivity implied by confidence ratings, addressing the confounding influence of first-order performance and response bias. Fleming and Lau subsequently compared several measurement families and emphasized the need to distinguish absolute confidence, metacognitive sensitivity, and efficiency. A corresponding artificial-system experiment must define which outputs count as correct, how confidence is elicited, which difficulty variations are controlled, and what information remains available at evaluation time. Without those controls, apparent metacognitive success may be a byproduct of easy questions or favorable output formats Maniscalco & Lau (2012) Fleming & Lau (2014).

The computational lineage also runs through probability forecasting and machine learning. Brier's 1950 work on verification of probabilistic forecasts provided a formal route for assessing confidence as a prediction. Bayesian modeling, neural predictive uncertainty, post-hoc calibration, and selective classification developed methods for estimating or transforming model confidence. Contemporary language models bring these traditions together while introducing a novel difficulty: the system's stated certainty may be generated as language instead of read directly from an internal predictive distribution. The history is thus one of progressively sharper measurement problems, not a single linear march toward a unified mental faculty Brier (1950) Guo et al. (2017) Mielke et al. (2022).

First-Order Accuracy and Second-Order Knowledge

First-order accuracy evaluates whether the selected answer meets the task criterion. Second-order assessment evaluates how well a confidence signal distinguishes successful from unsuccessful first-order responses. Suppose two models each solve six of ten questions. One assigns higher confidence to its correct answers and lower confidence to errors; another uses nearly identical confidence for every question. Their mean accuracy is the same, yet their reliability information differs substantially. Conversely, a very accurate model can still be poorly calibrated if it routinely claims near-certainty in cases where its residual errors cluster. Comparing systems requires the joint distribution of confidence and outcomes, rather than accuracy or average confidence in isolation Maniscalco & Lau (2012) Guo et al. (2017).

The word knowledge is experimentally treacherous when attributed to an artificial predictor. Kadavath and colleagues distinguished the probability that a proposed answer is correct, P(True), from the predicted probability that a question can be answered correctly, P(IK). Both quantities are task-dependent and can be estimated with particular probing or training procedures. The distinction also clarifies timing: predicting one's likely success before producing an answer poses a different problem from appraising an already generated candidate. Evaluation of self-knowledge therefore specifies the random variable, prediction horizon, scoring criterion, and amount of available evidence Kadavath et al. (2022).

A first-order failure can originate in absent knowledge, mistaken retrieval, ambiguous instructions, insufficient computation, incompatible tools, or a shifted environment. A second-order failure can arise despite a competent first-order process when the monitoring signal fails to track those error sources. The research question becomes whether the confidence estimator carries conditional information about correctness that generalizes beyond superficial cues such as item length, common word frequency, or apparent familiarity. Robust evidence compares calibrated systems against baselines using task difficulty, output probability, external verification, and perturbation controls Jiang et al. (2021) Ovadia et al. (2019) Geng et al. (2024).

Confidence, Probability, and the Meaning of a Forecast

Confidence in a scientific evaluation must be connected to an explicit event. In multiple-choice classification it may mean the chance that the selected label is correct under a defined data distribution. For open-ended answers it may refer to exact factual truth, semantic equivalence to a reference answer, support from a retrieved source, or a grader-defined degree of validity. These outcomes are not interchangeable. A probability of correctness is interpretable only together with an outcome definition and a reference class of comparable cases. Changing the event after generating an answer creates a moving target for calibration Brier (1950) Huang et al. (2024).

Reliability and discrimination describe different properties. Reliability asks whether cases predicted at a particular probability achieve approximately that success rate over a sufficiently specified collection of trials. Discrimination asks whether scores rank correct cases above incorrect ones. A constant forecast equal to the base rate may be calibrated on an aggregate sample yet useless for deciding which individual case warrants review. A discriminative signal may misstate numerical probabilities but still support effective selective answering. Proper scoring rules, calibration plots, ranking statistics, and decision curves therefore answer complementary questions Guo et al. (2017) Fleming & Lau (2014) Geng et al. (2024).

Probability can be attached to a response before selection, after selection, after an independent verifier acts, or after external information becomes available. The recorded moment determines what the score measures. If a model sees the gold answer or receives feedback before rating its confidence, its score is no longer an ex ante forecast. If a confidence value is generated only after long explanations, explanations may themselves change the available evidence and prompt behavior. Reproducible protocols specify the sampling stage, prompt order, access to intermediate reasoning, and information boundary before reporting any claim about intrinsic confidence Tian et al. (2023) Kumar et al. (2024) Kumaran, Daw, et al. (2026).

Scientific Scope and Evidence Review Method

This monograph integrates primary experiments on language-model confidence, calibrated prediction, uncertainty reporting, abstention, and causal tests of decision policies with methodological research from psychology and statistical learning. Original empirical papers receive priority for claims about observed effects; conceptual reviews help organize definitions and identify unresolved measurement problems. Publications dated through October 2026 are considered where their provenance and conclusions could be verified. Study designs and reporting levels vary substantially, so numerical outcomes are attributed to the particular reported setting rather than transferred across models or applications Geng et al. (2024) Kumaran, Daw, et al. (2026).

The evidence is organized by inferential strength. A correlation between a confidence score and correctness supports predictive usefulness under the tested distribution. An intervention that modifies a confidence representation and changes an otherwise controlled action supplies stronger evidence that the signal is used in that action policy. Demonstrating better expected calibration error after a prompt change establishes improvement on the measured dataset and scoring pipeline, while general reliability requires validation across tasks, independent item sets, model versions, and shifts. The hierarchy prevents a behavioral correlation from being mistaken for a complete mechanistic explanation Zhao et al. (2024) Kumaran, Daw, et al. (2026) Ovadia et al. (2019).

Several limitations shape the review. Proprietary systems may not expose token-level scores, weights, or training histories; prompts can influence both answers and ratings; semantic correctness is difficult to annotate in long-form text; and published studies often use distinct notions of confidence. Where only a peer-reviewed abstract or a publication record has been checked, interpretation remains confined to the claims it actually supports. The synthesis identifies design principles and testable research questions, and keeps the distinction between a framework for interpreting data and established results from named experiments Huang et al. (2024) Geng et al. (2024) Fleming & Lau (2014).

Scoring Rules, Calibration Curves, and Error Decomposition

The Brier score for a binary event is the mean squared distance between a forecast probability and its observed zero-or-one outcome. Smaller values indicate better probabilistic forecasts under a fixed evaluation distribution. The logarithmic score additionally penalizes confident mistakes sharply, although its numerical stability depends on the treatment of probabilities near zero and one. Both are proper scoring rules, meaning that truthful reporting of an assessor's probabilistic belief is optimal in expectation under their mathematical assumptions. Neither score, in isolation, proves that the model has access to a distinctive introspective mechanism; a well-trained external predictor can also perform well Brier (1950) Guo et al. (2017).

Expected calibration error, or ECE, approximates the discrepancy between average confidence and observed accuracy in partitions of predictions. Its interpretability depends on bin boundaries, sample size, class balance, and whether a top-label or full-distribution variant is used. Two methods can change rank when the number of bins changes; an apparently small ECE can coexist with severe errors in a rare but consequential subgroup. Researchers should therefore supply reliability diagrams, counts per bin, uncertainty estimates, and at least one proper scoring rule. Reporting only a visually attractive curve risks hiding both sampling noise and selective failures Guo et al. (2017) Geng et al. (2024) Zhao et al. (2024).

Calibration must also be assessed conditionally. A model can have accurate global frequencies while overstating its success on arithmetic and understating it on factual retrieval. A mixture of topics can make these defects cancel. Meaningful subgroup analysis covers difficulty, answer form, prompt type, temporal novelty, tool availability, demographic and linguistic variation where relevant, and model version. The goal is to determine whether probabilities remain decision-relevant for the people and cases that will rely on them. Aggregate scores are useful summaries, but the scientific object is the mapping from actual evidence to a forecast under specified conditions Ovadia et al. (2019) Huang et al. (2024).

Selective Prediction, Abstention, and Risk-Coverage Tradeoffs

An evaluator may permit a system to abstain when it expects a high chance of error. Coverage is the proportion of items answered, while selective risk is the error rate among the answered items. Sweeping a confidence threshold yields a risk-coverage curve and reveals whether withholding uncertain cases improves the accepted set. Because raising a threshold changes both sample composition and workload, the strongest evaluation reports coverage at matched risk and risk at matched coverage. A system with calibrated probabilities but a poorly implemented abstention policy may still produce costly high-risk responses Tian et al. (2023) Kumaran, Daw, et al. (2026).

Thresholds are decision rules, not universal constants. Where a wrong answer imposes a much greater cost than requesting verification, conservative abstention can improve expected utility. In low-stakes exploratory tasks, aggressive abstention may suppress useful tentative answers. A practical decision model specifies the loss of an error, cost of further evidence, cost of deferral, and value of timely response. Confidence supports action only when these values and their empirical calibration are considered. Selecting a threshold on the evaluation test set and reporting its performance on the same cases creates optimism that must be avoided with independent validation Brier (1950) Desender et al. (2018).

Modern language agents extend selective prediction into multi-action control: answer, search, ask a clarification, call a calculator, run a test, or defer. Each action has an uncertain value and resource cost. Confidence in a single factual answer is not identical to confidence that a proposed plan will succeed. A reliable agent must monitor transition outcomes over multiple steps and re-evaluate after external observations. Research designs should compare uncertainty-aware policies with matched-cost baselines and distinguish improvements due to better information acquisition from improvements due merely to increasing computation Kumaran, Daw, et al. (2026) Geng et al. (2024).

Predictive Distributions, Logits, and Their Limits

Token probabilities assign mass to possible next tokens given the preceding context. They provide a well-defined model-side distribution, but the probability of a sequence is influenced by response length, surface wording, tokenizer choices, and alternative formulations of the same answer. A high probability for the next word is not a direct probability that the complete answer will be factually correct. Multiple semantically equivalent answers may partition probability mass, while a common but false assertion may carry high likelihood. This mismatch explains why language-model confidence requires careful mapping between token events and task outcomes Jiang et al. (2021) Farquhar et al. (2024).

Classification models often expose logits for a fixed set of labels, allowing a probability assigned to each label and straightforward assessment against known classes. Open-ended systems generate variable-length text and may hide logits entirely. Researchers approximate answer correctness with restricted candidate sets, specially trained value heads, prompted self-evaluation, sampling agreement, or an external verifier. Those procedures create different measurement instruments. Reporting their values under one broad label of model confidence obscures which information was accessible and how the event of interest was defined Kadavath et al. (2022) Tian et al. (2023) Geng et al. (2024).

Probability surfaces change with temperature, decoding policy, and the order and phrasing of candidate labels. The output probabilities at temperature one do not necessarily characterize the uncertainty of a deployed system sampling under different conditions. Similarly, a tool-augmented answer may rest on retrieved material that the base model did not know. The operative configuration is the scientifically relevant unit. A reproducible experiment must describe decoding parameters, treatment of alternative responses, prompt templates, availability of logits, and whether confidence was computed before or after additional context Guo et al. (2017) Ovadia et al. (2019) Kumar et al. (2024).

Post-Hoc Calibration and Probabilistic Modeling

Temperature scaling applies a learned scalar transformation to classifier logits, typically fitted on held-out validation data, to improve the match between predicted confidence and observed class frequencies. Guo and colleagues demonstrated its practical effectiveness across the image and document classification settings they examined. The technique is attractive because it can adjust probability sharpness without changing the predicted top class, although it cannot repair poor ranking, unknown outcome definitions, or changes in the deployment distribution. A calibration layer should be reported as a separate trained component, with its fitting data and transfer performance documented Guo et al. (2017).

Bayesian approximations and ensemble methods attempt to represent uncertainty arising from incomplete knowledge of model parameters or predictions. Gal and Ghahramani interpreted dropout in neural networks as an approximate Bayesian procedure, while subsequent distribution-shift research compared uncertainty estimators under conditions where training and test data differed. These methods can provide informative predictive distributions, yet their numerical calibration is still an empirical property. A sophisticated approximation does not guarantee that every confidence interval or classification forecast covers outcomes at its nominal rate Gal & Ghahramani (2016) Ovadia et al. (2019) Kuleshov et al. (2018).

Calibration for regression uses a different target: the correspondence between predictive intervals and subsequent outcomes, rather than binary answer correctness. Kuleshov and colleagues proposed a calibration procedure for probabilistic regression and assessed interval behavior in several predictive settings. The connection matters for agents forecasting travel times, resource demands, continuous rewards, or task completion durations. Each target variable requires an event definition and measurement scale; a categorical correctness score cannot be substituted for interval coverage. Studies should report interval width alongside coverage because uniformly wide intervals can conceal low informational value Kuleshov et al. (2018) Brier (1950).

Verbalized Confidence as a Separate Measurement Channel

Verbalized confidence is produced in language, typically as a phrase or numerical estimate after a question or proposed answer. It may be elicited through an additional prompt even when token-level scores are inaccessible. Lin, Hilton, and Evans showed that a GPT-3 variant could be taught to express useful probabilistic uncertainty in words, including in tested shifted settings. Their evidence supports the feasibility of trained reporting, not the general proposition that all models spontaneously disclose faithful uncertainty. The displayed number remains an output of a configured generation process and requires independent scoring Lin et al. (2022) Tian et al. (2023).

Natural-language hedges and quantified statements carry different kinds of information. A phrase such as probably correct is interpreted through context and speaker expectations; a percentage supports numerical verification if its reference event is explicit. Conversational research by Mielke and colleagues showed that linguistic calibration could be improved by training a response generator using metacognitive features predicting error likelihood. The result illustrates a crucial design distinction: estimating confidence and deciding how to communicate it can be separate stages. A fluent uncertainty sentence should therefore be checked against event probabilities and audience understanding Mielke et al. (2022).

Prompt-based elicitation can itself influence confidence. Tian and colleagues evaluated methods for extracting scores from human-feedback-fine-tuned language models, while Kumar and colleagues examined agreement between probability-derived and expressly reported confidence. These studies show why evaluation should compare directly elicited percentages, categorical hedges, alternative answer formats, and independent verifier scores on common item sets. The degree of agreement between channels is an empirical finding, and disagreement can reveal either limitations of introspective access or distortions introduced by the report-generation policy Tian et al. (2023) Kumar et al. (2024) Zhang et al. (2024).

Question Answering as an Experimental Paradigm

Question-answering tasks are common because researchers can define expected answers and compare confidence to correctness on many trials. Jiang and colleagues studied T5, BART, and GPT-2 in factual question answering and found that raw model probabilities were poorly calibrated in the evaluated settings. Calibration interventions improved observed behavior, but the study also demonstrated dependence on task and probability extraction. A strong modern protocol retains this discipline: freeze question sets, define acceptable alternatives, record model and prompt versions, and distinguish exact-match grading from independent assessment of factual meaning Jiang et al. (2021).

A benchmark should span ordinary questions, difficult but answerable items, adversarially misleading premises, genuinely underspecified requests, and questions whose answers change with time. The model can be asked to assign confidence before or after proposing a candidate, with experimental groups receiving matched prompts except for the feature under investigation. These distinctions matter because success at self-evaluating a known answer is not identical to estimating whether a question can be solved before generating one. The comparison ought to include unconditional accuracy, calibration, discrimination, and the reliability of abstention decisions Kadavath et al. (2022) Tian et al. (2023).

Evaluation quality is especially vulnerable to data leakage and grading artifacts. If a question or answer was present in training, high accuracy can reflect familiarity that fails to transfer to newly authored tasks. If a grader rewards stylistic fluency, a confident but unsupported answer can appear correct. A robust study therefore uses independent question authorship, where possible private held-out material, documented scoring rubrics, adjudication of ambiguous items, and controlled versions of available retrieval documents. These protections strengthen claims about metacognitive generalization without assuming that test items perfectly represent real-world use Geng et al. (2024) Huang et al. (2024).

The P(True) and P(IK) Research Program

Kadavath and colleagues' 2022 investigation asked whether language models can judge the validity of their own statements and anticipate their success on questions. Its P(True) procedure rates a particular proposed answer, whereas P(IK) addresses whether the model expects to know the answer before a specific answer is evaluated. Reported performance depended on the model scale, response format, task, and context; self-evaluation improved in certain settings when models considered multiple proposed answers. These findings provide evidence for accessible reliability-related signals under specified testing procedures Kadavath et al. (2022).

The study also found limitations in transfer, particularly for P(IK) calibration on new tasks. This point is theoretically important: a system might discriminate easy questions from hard ones within a familiar benchmark while failing to estimate its performance in a newly configured environment. The distinction between task difficulty and model-specific knowledge is empirically testable with pairs of systems that share questions but differ in training or retrieval. A confidence estimator that merely tracks public frequency or familiar phrasing may be useful, yet it need not identify the individual system's actual knowledge boundaries Kadavath et al. (2022) Ovadia et al. (2019).

Reproducing the design requires preserving the exact operational meaning of the probability label. P(True) is conditional on a generated claim and its evaluable truth; P(IK) requires a declared success criterion for samples or answers. A model may report an easily interpreted percentage but use a different implicit definition than the investigator. Experimental documentation should therefore include the prompt, candidate generation scheme, instructions supplied to a separate evaluator, and treatment of alternative correct answers. The central contribution of the program is a set of empirical questions about self-evaluation, rather than a license to infer subjective introspection from grammatical self-reference Kadavath et al. (2022) Fleming & Lau (2014).

Prompting Effects and the Structure of Overconfidence

Training for preference satisfaction and conversational usefulness can change how language models express certainty. Tian and colleagues' experimental comparison of calibration strategies in feedback-fine-tuned models challenged the assumption that a simple raw probability or a single prompt captures reliable uncertainty. Zhao and colleagues then compared prompting strategies in question answering and introduced Fact-and-Reflection, in which relevant facts are elicited before the model forms a final answer. The authors reported lower expected calibration error in their tested settings, alongside cases where other prompting methods increased overconfidence on particular items Tian et al. (2023) Zhao et al. (2024).

Prompt engineering can change multiple mechanisms simultaneously: which evidence is recalled, how long the response becomes, whether alternatives are considered, and which confidence language appears socially appropriate. Consequently, a reduction in calibration error after a prompt intervention does not by itself identify whether the gain came from better first-order accuracy, better ranking, or a better numerical mapping. A strong experiment holds the question distribution constant, tracks both accuracy and probability error, and compares computational cost and response length. Its interpretation is grounded in the observed joint changes rather than a general claim of introspective improvement Zhao et al. (2024) Kumar et al. (2024).

Overconfidence is asymmetric in practice. A few extreme confident errors can be especially damaging in clinical, legal, or scientific contexts, even where average calibration seems acceptable. Groot and Valdenegro-Toro examined verbal uncertainty in language and vision-language models and documented substantial overconfidence in their evaluated systems. Their results emphasize the relevance of modality, unfamiliar visual counting, and question difficulty. Such findings are specific to the assessed models and tasks; later versions require fresh measurement. Researchers should characterize conditional tails of confidence as well as central tendencies Groot & Valdenegro-Toro (2024) Ovadia et al. (2019).

Long-Form Answers and Partial Correctness

Long-form generation complicates the definition of a correct outcome. An essay may contain several accurate statements, one serious factual error, unsupported causal language, and appropriate uncertainty about unresolved questions. A single binary grade compresses these properties and can produce misleading probabilities. Huang and colleagues proposed treating answer quality and confidence over a range of scores, introducing evaluation methods suited to partially correct generations. Their study found that larger models do not automatically produce better calibration, underscoring that generative quality and uncertainty assessment remain distinct measurements Huang et al. (2024).

A rigorous long-form protocol segments claims, identifies relevant evidence, and defines whether the forecast concerns each claim, an entire answer, or a minimum standard for publication. These events produce different empirical reference classes. Confidence in a response containing ten independent claims cannot be interpreted as the confidence of each claim unless the scoring method explicitly links them. A system can correctly hedge a contested causal inference while expressing certainty about an established date; treating both with one label destroys information the user needs. Claim-level factuality and answer-level adequacy can therefore be reported side by side Huang et al. (2024) Farquhar et al. (2024).

Human graders introduce uncertainty of their own. Expert disagreement, incomplete references, and uncertain gold answers require adjudication rather than silent conversion into binary truth labels. The scientific report should state inter-rater procedures, whether judges saw model names or confidence values, and how disputed cases were handled. The ability to make partial-credit assessments does not eliminate the need for independent evaluation. Otherwise, improved calibration can reflect closer agreement with a particular judge's preferences instead of more accurate anticipation of external facts Huang et al. (2024) Mielke et al. (2022).

Multimodal Confidence and Modality-Specific Evidence

Vision-language systems estimate answers using images together with linguistic instructions, introducing ambiguity from occlusion, scale, object counting, and scene composition. Groot and Valdenegro-Toro evaluated verbalized uncertainty for language and vision-language models, including difficult scene queries, and identified overconfidence as a significant problem in their tested sample. The result extends the empirical object beyond factual recall. Here confidence must reflect whether the relevant visual evidence actually distinguishes the candidate answer from plausible alternatives and whether the scoring event was objectively determinable Groot & Valdenegro-Toro (2024).

Modality changes the appropriate controls. An image may be compressed or cropped, an audio recording may include noise, and sensor data may drift after deployment. A confidence mechanism that performs well on text-only benchmarks may rely on cues that disappear when evidence comes from other channels. Researchers should report image resolution, preprocessing, visual perturbations, input modality, task difficulty, and calibration within each condition. Comparing image-based and text-based confidence at one aggregate level risks obscuring distinctive sources of uncertainty and failures in the perception subsystem Groot & Valdenegro-Toro (2024) Ovadia et al. (2019).

Cross-modal evaluation also requires careful definition of ground truth. A question about a partly hidden object may admit reasonable uncertainty even for a skilled human observer, while a simple object count may permit decisive verification. The evaluator should distinguish missing evidence from stochastic variation in the model's output and from errors in human labels. When confidence calibrates well on a fixed visual benchmark, the conclusion is that the measured prediction signal tracked outcomes there. A further claim about general-purpose metacognitive control requires tests involving action, evidence seeking, and transfer to unseen conditions Fleming & Lau (2014) Kumaran, Daw, et al. (2026).

Semantic Uncertainty and Answer Equivalence

A generative model can vary wording while preserving the same factual proposition. Counting distinct strings therefore overestimates meaningful disagreement when several responses are paraphrases. Farquhar, Kossen, Kuhn, and Gal introduced semantic entropy as an approach that aggregates sampled outputs according to their meaning and uses variation across semantic alternatives to identify certain forms of hallucination. Their Nature study in 2024 provides an important example of how the relevant uncertainty event can be defined above the token level. Its practical effectiveness still depends on sampling, semantic grouping quality, and the types of errors being detected Farquhar et al. (2024).

Semantic variation is informative especially where different plausible answers represent genuine factual alternatives. If a model generates the same unsupported claim repeatedly, agreement does not establish truth; all sampled answers may share one misconception or a common retrieval failure. If responses vary extensively in style while preserving a correct proposition, raw text diversity exaggerates uncertainty. Consequently, self-consistency and semantic entropy are evidence signals, not substitutes for independent fact checking. Researchers should assess what kind of error their metric is designed to detect and test it against supported but underrepresented answers as well as common falsehoods Farquhar et al. (2024) Huang et al. (2024).

Computing semantic uncertainty introduces extra work and potential error at the clustering stage. An inference system may use another model to judge equivalence, creating correlated biases between the generator and evaluator. A rigorous protocol logs the sampled candidates, semantic grouping rule, evaluator versions, and sensitivity to sampling temperature. It compares computational cost and detection performance with simpler confidence methods and tests whether grouping decisions hold across annotators. Interpretation remains at the level of measurable support for uncertainty prediction; semantic similarity alone does not reveal how the original model causally constructed an answer Farquhar et al. (2024) Geng et al. (2024).

Internal Representations and Confidence Readout

An internal activation, token-derived statistic, or trained probe can carry information about likely correctness. Finding such information does not show that every outward confidence statement faithfully transmits it. Kumar and colleagues investigated the alignment between token-probability-derived confidence and explicit responses under different elicitation procedures. Their design illustrates the need to compare internal and verbal channels on the same items while recognizing that extraction methods intervene on the overall system. A mismatch may arise from the decoding interface, a learned communication policy, imperfect probes, or genuinely unavailable information Kumar et al. (2024).

Model representations are high dimensional, and a reliable confidence estimate may depend on more than one component. An evaluator can train a probe to predict correctness from hidden states, but a successful probe establishes predictive information at a chosen location, not that the model normally consults that probe. Causal interpretation requires interventions that alter candidate signals while preserving as much as possible of the underlying task evidence. It also requires controls for general disruption, since changing hidden activations can affect task processing through pathways unrelated to confidence Kumaran, Daw, et al. (2026) Geng et al. (2024).

Confidence readout should be evaluated across multiple reference events: selected answer correctness, expected success before answering, robustness to paraphrase, and the likelihood that a planned external action will achieve its goal. A single latent direction may not generalize across them. An empirical matrix comparing signals, outcomes, and control policies is more informative than labeling one activation as the location of self-awareness. Mechanistic hypotheses become persuasive when they predict responses to counterfactual tasks, systematic perturbations, and matched control interventions in addition to observational correlation Kumaran, Daw, et al. (2026) Fleming & Lau (2014).

Causal Tests of Confidence-Guided Abstention

Kumaran, Daw, Osindero, Veličković, and Patraucean published a multi-phase investigation in Nature Machine Intelligence on September 7, 2026. Their experiments elicited baseline confidence, tested whether confidence predicted abstention, manipulated confidence-related activations, and examined responses to different instructed abstention thresholds. The authors reported that confidence-related signals substantially influenced abstention, and that steering the signals changed abstention in the predicted direction. This is especially important because a causal test of control addresses a stronger question than whether a model can merely state a probability correlated with its answers Kumaran, Daw, et al. (2026).

Interpretation of the study requires preserving its task, model, and intervention boundaries. An intervention can show that a selected internal representation participates in an action policy without establishing a universal confidence monitor across all deployed systems. Confidence may be multidimensional, and token-level probabilities and spoken ratings can each be incomplete readouts. The study itself reports that verbal confidence predicted abstention even though it was less discriminative of correctness than another measured confidence channel. These differences caution against reducing metacognitive control to a single public percentage Kumaran, Daw, et al. (2026).

The design supplies a general experimental template. First establish the baseline relationship among accuracy, confidence, and abstention. Next intervene on the hypothesized confidence pathway while matching alternative sources of task difficulty and preserving a comparable first-order problem. Then examine whether policy thresholds and decisions change, measure competing mechanisms, and replicate on held-out task families. Clear causal claims require sensitivity analyses for intervention specificity and explicit reporting of failures. The resulting evidence concerns an engineered control function whose reach can be expanded through subsequent testing rather than assumption Kumaran, Daw, et al. (2026) Desender et al. (2018).

Error Monitoring, Feedback, and Revision

Error monitoring is the ability to detect or anticipate that a response or action has failed against a criterion. It may occur during generation, after completion, or after an external verifier supplies feedback. These moments must be separated experimentally because the available information changes. A system correcting a calculation after a tool reports the right answer demonstrates a capacity to incorporate external feedback; it does not thereby demonstrate that it could diagnose the error before seeing that feedback. The same distinction appears in human metacognition, where confidence and postdecision evidence can modify later judgments Fleming & Lau (2014) Fleming & Daw (2017).

Self-revision can be useful and can also introduce new errors. An agent may replace a correct initial answer with an incorrect alternative after excessive reflection, or repeat a wrong explanation in slightly different language. Evaluation should record accuracy before and after revision, the fraction of corrected errors, the fraction of corrupted correct answers, confidence changes, and the cost of additional reasoning. A reliable monitoring policy selectively allocates effort where the expected value of checking is positive. The crucial criterion is outcome improvement relative to a matched-computation control, rather than the number of times a system reports having reconsidered Zhao et al. (2024) Huang et al. (2024).

Error reports are themselves predictions requiring calibration. If a system says that a cited source is unreliable, independent inspection must determine whether the source is actually irrelevant, fabricated, outdated, or contradictory. An error-monitoring architecture can combine retrieval verification, executable tests, confidence models, and external audits. Each component should be evaluated separately and then as a system, since combining correlated weak checks may create false reassurance. Evidence from individual tasks supports design hypotheses, while robust conclusions require transfer across domains and changes in tool availability Mielke et al. (2022) Ovadia et al. (2019).

Metacognitive Control in Tool-Using Agents

An artificial agent able to retrieve documents, execute code, and interact with an environment makes confidence consequential through action selection. Uncertainty can trigger information acquisition, additional computation, a request for clarification, or a decision to defer. The relevant control quantity is frequently expected utility rather than raw confidence: the agent should weigh the probability of error against the costs and benefits of available actions. A numerical confidence score that predicts answer correctness but never influences tool choice remains a monitoring signal rather than a demonstrated control policy Kumaran, Daw, et al. (2026) Desender et al. (2018).

A scientifically controlled comparison can assign a fixed computational or tool budget to several policies: always answer, always verify, random verification, uncertainty-triggered verification, and an oracle upper bound where feasible. Evaluation then measures accuracy, latency, resource use, fraction of avoidable errors, and the incidence of unnecessary tool calls. This design identifies whether metacognitive regulation creates additional value beyond spending more computation. It also tests the feedback loop: confidence should change when new evidence is retrieved, but the direction of change should follow evidential quality rather than merely the number of retrieved passages Tian et al. (2023) Ovadia et al. (2019).

Agentic systems introduce longitudinal complications. The model, context window, retrieval index, and external memory can change between encounters, while a previously reliable probability mapping may become obsolete. Effective monitoring therefore requires version-specific recalibration and checks for drift at the component and task-policy levels. A system that remembers prior success may show improved decisions, but evidence must identify which stored information influenced the decision and whether the improvement survives controlled perturbation. Such tests connect the psychology of artificial systems to the operational science of adaptive control while retaining a distinct empirical object Kumaran, Daw, et al. (2026) Geng et al. (2024).

Distribution Shift and the Boundaries of Calibration

A calibration curve is estimated on a distribution of tasks, contexts, and outcomes. Deployment may change all three. Ovadia and colleagues compared predictive uncertainty under dataset shift and documented that methods performing acceptably under familiar conditions can degrade when input distributions change. Language systems face analogous shifts from new domains, languages, task formats, model updates, and retrieval corpora. A 90 percent confidence value observed to be reliable for short trivia questions cannot be transported automatically to clinical synthesis or novel mathematical reasoning. Each deployment claim needs a target distribution and independent validation Ovadia et al. (2019) Jiang et al. (2021).

Stress testing can vary the source of uncertainty deliberately. Researchers might change item difficulty while retaining topic, alter wording without altering the correct answer, remove decisive evidence, add irrelevant documents, or introduce contradictory source passages. The experiment should separate a model's failure to answer from a monitor's failure to notice the increased risk. Conditional calibration within each stress condition is essential because global reliability may remain stable if easier groups dominate the sample. Repeated studies after model updates establish whether earlier calibration layers continue to apply Ovadia et al. (2019) Geng et al. (2024).

Distribution shift is also a causal explanation problem. A decline in confidence reliability could result from changes in first-order accuracy, poorer confidence discrimination, altered verbal reporting, or an outdated post-hoc calibration map. Diagnostic studies collect each component separately before attributing the failure. When data are sparse, interval estimates and transparent uncertainty around calibration metrics are preferable to a falsely precise performance claim. Scientists and deployers need to know both how often a failure occurs and how confidently that frequency has been estimated Guo et al. (2017) Kuleshov et al. (2018).

Measurement Validity and Experimental Confounds

Construct validity asks whether a test score measures the concept invoked by its name. Artificial metacognition is frequently operationalized through self-reported certainty, but fluent reports can be trained social outputs, and hidden-state probes can exploit correlations with item difficulty. Convergent evidence requires different instruments to predict the same outcomes; discriminant evidence asks whether they diverge appropriately from simple difficulty or fluency measures. A causal intervention gives stronger grounds for identifying a confidence-linked control pathway, but its meaning still depends on the specificity of the manipulation and the response variables Fleming & Lau (2014) Kumar et al. (2024) Kumaran, Daw, et al. (2026).

Repeated trials are not necessarily independent observations. Multiple prompts evaluated against the same item share content; repeated samples from the same model may be correlated; and many closely related benchmark questions can produce an exaggerated impression of sample size. A rigorous analysis treats the question, model version, sampling seed, prompt template, and research setting as potentially distinct sources of variation. Where possible it reports item- and run-level uncertainty, multiple benchmarks, and checks against a held-out replication set. The unit of statistical analysis should match the scientific population to which the conclusion is intended to generalize Geng et al. (2024) Ovadia et al. (2019).

Model comparisons require matched task definitions, transparent stopping rules, and an account of access differences. Open models may permit logit extraction and controlled activation interventions, whereas commercial systems may provide only generated text. A comparison of internal scores from one and verbal reports from another can confound system quality with the measurement channel. Comparisons should either harmonize the interface or explicitly report the instruments as different treatments. Performance rankings without matched instruments can mislead even when each score is computed correctly Tian et al. (2023) Kumar et al. (2024).

Replication, Statistical Reporting, and Evidence Quality

Replication establishes whether a reported confidence property survives independent implementation, changed samples, and new model checkpoints. A robust experimental record includes model identification, evaluation date, prompt text, random seeds or decoding parameters, item construction, scoring code, missing-data rules, and the number of effective independent items. The relevant uncertainty interval depends on the analysis unit and selected metric. Confidence-calibration studies frequently have heterogeneous tasks, so reporting pooled results alone does not adequately support general claims Geng et al. (2024) Huang et al. (2024).

Researchers should report both successful and unsuccessful perturbations. If an intervention improves verbalized calibration on one dataset but reduces discrimination on another, that pattern constrains a theory more effectively than a single summary score. Comparisons must include competing baselines such as raw likelihood, task difficulty, external verification, and equally costly additional sampling. For complex agents, errors may be clustered by workflows, and whole-workflow outcomes should be analyzed alongside individual question correctness. A claim of improved self-monitoring is strongest where multiple independent metrics agree and the outcome survives preregistered or otherwise independent replication Zhao et al. (2024) Ovadia et al. (2019).

The literature combines peer-reviewed field experiments, established mathematical methods, conference research, and influential preprints. Kadavath and colleagues' P(True) and P(IK) paper and Lin and colleagues' verbalized-uncertainty investigation are important preprint-origin contributions; their reported claims should retain that provenance and be checked against subsequent independent studies. Peer review supports scrutiny of methods but does not remove the need for replication, and publication year alone does not determine validity. This synthesis treats each work as evidence of the procedures and outcomes it actually describes rather than as proof of a universal capacity Kadavath et al. (2022) Lin et al. (2022) Kumaran, Daw, et al. (2026).

Human Interpretation of Artificial Confidence

Confidence communication has effects beyond machine-side scoring. Users may treat a fluent, lengthy answer as more trustworthy even when its actual correctness is unchanged. Experimental work on human perceptions of language-model outputs has found gaps between people's confidence in an answer and the reliability information available from the model. This phenomenon makes user calibration a distinct outcome: a model's probability estimate can be statistically useful while its prose produces a misleading impression of certainty. Evaluating safety and epistemic utility therefore requires measuring what readers infer from the presentation Mielke et al. (2022) Huang et al. (2024).

An interface can display a score, a source link, a verbal hedge, or an option to inspect evidence. Each design invites different decisions. If a system gives exact percentages without a validated event definition, users may interpret superficial precision as scientific accuracy. If it replaces probabilities with generic caution, useful distinctions among cases can disappear. Experiments should compare understanding, appropriate reliance, willingness to verify, and outcome quality under randomized presentations, rather than assuming that a single phrase improves trust calibration for all users Mielke et al. (2022) Tian et al. (2023).

Human–artificial relations also bring a division of responsibility. The artificial component can estimate and expose reliability; institutions select acceptable risks, decide when expert review is mandatory, and maintain accountability for consequential decisions. Metacognitive capability becomes most useful when it supports a transparent distribution of work between the system, verification tools, and human overseers. This is a research question about interaction and governance alongside the machine-side psychology of confidence, and its evaluation should keep user behavior separate from internal model mechanisms Kumaran, Daw, et al. (2026) Geng et al. (2024).

Psychological Attribution in the Artificial Era

Artificial metacognition is an appropriate research label for experimentally observable second-order monitoring and control functions. The concept permits cross-system comparison while preserving the identity of the studied mechanism. In human psychology, confidence is embedded in biological perception, memory, action, and self-report. In artificial architectures, the same behavioral measurement may depend on learned token predictions, explicit classifiers, hidden-state probes, retrieval systems, or engineered decision policies. Scientific comparison identifies functional similarities and implementation differences through measurable outcomes Flavell (1979) Fleming & Lau (2014) Kumaran, Daw, et al. (2026).

The historical horizon of the Artificial Era directs inquiry toward nonbiological systems as increasingly important participants in cognitive work. For this research dossier, the empirically tractable questions concern what an artificial system predicts about its own performance and how that prediction changes its behavior. Claims about subjective experience, personal identity, or a felt inner confidence are separate philosophical and empirical questions. An experimentally useful confidence monitor does not settle them, and their status does not determine whether a calibration or abstention mechanism should be measured Fleming & Lau (2014) Geng et al. (2024).

A cumulative Psychology of Artificial benefits from stable terminology. Confidence calibration concerns numerical correspondence with outcomes; metacognitive sensitivity concerns trial-level discrimination; metacognitive control concerns the use of such information for regulation; and psychological attribution concerns the justification for applying terms across distinct kinds of systems. These definitions allow experiments to accumulate without requiring every study to resolve broad ontological debates. The research program expands from isolated task scores to versioned systems that monitor, correct, and coordinate actions in changing environments Maniscalco & Lau (2012) Kumaran, Daw, et al. (2026) Ovadia et al. (2019).

Emerging Calibration Methods and the 2025–2026 Evidence

Contemporary methods increasingly combine linguistic confidence with additional information from model behavior. Research on confidence steering, multiple reports, and score consistency seeks to improve estimates when direct logits or retraining are unavailable. Zhou and colleagues described SteerConf at NeurIPS 2025, using varying confidence-elicitation prompts and consistency information to improve calibration in their evaluated benchmarks. Such procedures must be appraised for the possibility that prompt-induced variation measures compliance with reporting instructions as much as uncertainty about factual correctness. The effective signal is established by independent outcome scoring Zhou et al. (2025) Tian et al. (2023).

Recent work also questions the assumption that probability quality can be inferred from broad model capability. A 2026 preprint introducing ConfidenceBench evaluated prompted probability forecasts on a private multiple-choice benchmark and reported substantial differences among model families; its restricted dataset and preprint status delimit generalization. The reported comparison reinforces an established principle: answer accuracy, calibration, and the willingness to abstain constitute distinct dimensions of reliability. Different elicitation procedures and domain distributions can rearrange model rankings, so model selection must be matched to the eventual use case ffrench-Constant et al. (2026) Guo et al. (2017).

The September 2026 causal study represents a different advance because it directly manipulated confidence-related representations and examined consequential decisions. Integrating its design with calibration benchmarks suggests a richer evaluation program comprising accuracy, probabilistic honesty, monitoring sensitivity, policy effects, and adaptation after feedback. None of these properties can stand in for all the others. The most promising research direction is systematic comparison of the channels by which uncertainty is estimated, expressed, and acted upon, under explicit constraints on resources and environmental information Kumaran, Daw, et al. (2026) Fleming & Lau (2014).

Choice-Supportive Bias, Contradictory Advice, and Confidence Updating

An experimental account of metacognition must explain how uncertainty changes when a system encounters new information. Kumaran, Fleming, and colleagues tested language models in a two-stage decision paradigm in which a model selected an initial answer and subsequently received supporting, opposing, or neutral advice of specified reliability. The April 2026 Nature Machine Intelligence study documented two interacting patterns: showing the model its own prior answer increased persistence and confidence, while opposing advice could receive disproportionate weight relative to Bayesian predictions. The authors analyzed models including Gemma, GPT-4o, and other contemporary architectures and examined several task families Kumaran, Fleming, et al. (2026).

The design distinguishes a choice-supportive bias from overweighting contradictory information. In the principal Gemma 3 12B experiment, initial-answer visibility reduced the rate of answer changes, and the authors reported more pronounced confidence decreases following opposing advice than corresponding increases following supporting advice. This matters because a system may be overconfident after seeing its previous answer and underconfident after encountering disagreement, depending on how evidence is presented. Aggregate measures of calibration alone can hide those opposite conditional errors. Separating advice quality, prior-answer visibility, and pre-advice confidence makes a more precise account possible Kumaran, Fleming, et al. (2026).

The experiment provides a method for investigating artificial error monitoring after feedback. Researchers can independently manipulate the reliability and direction of advice, preserve first-order question difficulty, and compare observed confidence updates with an explicitly defined ideal observer. Results should be interpreted as properties of the tested versioned systems rather than universal psychological laws. The study complements the September 2026 causal abstention investigation: one program tests how evidence changes confidence, while the other examines how confidence-related representations influence whether a system answers. Together they motivate controlled tests of both information integration and metacognitive action Kumaran, Fleming, et al. (2026) Kumaran, Daw, et al. (2026).

Scientific Disagreements and Alternative Explanations

The literature supports different conclusions depending on which instrument is used. Early question-answering studies found pronounced miscalibration in raw language-model probabilities; later work demonstrated useful self-evaluation signals and trained verbalized confidence under selected protocols. Further studies reported renewed overconfidence, format dependence, and incomplete agreement between internal and expressed confidence. These are not simply mutually exclusive results. They vary in model family, alignment procedure, scoring rule, output format, task difficulty, and evaluation era. The comparative question is which experimental conditions produce which reliability properties Jiang et al. (2021) Kadavath et al. (2022) Lin et al. (2022) Groot & Valdenegro-Toro (2024).

One explanation of successful confidence prediction is that the model uses a genuine signal related to the reliability of its computation. An alternative is that the monitor identifies externally visible proxies such as question familiarity, length, or difficulty. Both can predict correctness, but they make different predictions under interventions that alter model-specific evidence while preserving item difficulty. Causal activation studies constrain the mechanism by showing confidence-related manipulation can influence abstention, yet explanatory alternatives remain concerning what internal representation is modified and how general the policy is Kadavath et al. (2022) Kumaran, Daw, et al. (2026).

A second disagreement concerns how confidence should be reported. Token probabilities can be mathematically convenient but misaligned with semantic answer truth; verbal ratings are available in many products but sensitive to prompting; self-consistency adds computation and may preserve shared errors. External fact checking provides a different evidence channel and can improve decision quality, yet it depends on the trustworthiness of tools and documents. The strongest design is task-dependent comparison of multiple independently validated channels rather than assuming one has privileged access to uncertainty across all kinds of questions Farquhar et al. (2024) Tian et al. (2023) Huang et al. (2024).

A Research Agenda for Artificial Metacognitive Systems

A rigorous next-generation experiment should preregister the first-order outcome, the confidence instrument, the policy to be tested, and the hypothesis linking confidence to action. It should include new questions, different domains, multiple model versions, and explicit negative controls. Separately measuring prediction before answering and evaluation after answering tests whether a monitor estimates future competence or inspects a formed candidate. Interventions on retrieval access and independent verification can then establish whether updated confidence follows actual evidence. This sequence advances cumulative explanation from simple correlation to an experimentally tractable mechanism Kadavath et al. (2022) Kumaran, Daw, et al. (2026).

Comparative designs should integrate behavioral and mechanistic methods. Token-level statistics, verbal reports, hidden-state probes, semantic-entropy estimators, and independent checking systems can be applied to the same scored answer set where access permits. Researchers should report calibration, discrimination, error tails, selective risk, and resource-normalized improvement from confidence-guided action. They should also reveal when instruments disagree and test whether disagreement predicts failure. The result is a multidimensional scientific account with explicit observable indicators rather than a single aggregate label of model self-awareness Farquhar et al. (2024) Kumar et al. (2024) Geng et al. (2024).

Longitudinal studies are required for deployed agents with changing memory, tool registries, and training versions. Calibration mappings should be revalidated after material updates, and evidence-quality interventions should determine whether monitoring tracks the current system rather than a historical reputation for competence. Evaluation should consider downstream user understanding and opportunities for accountable verification. These requirements can be operationalized through versioned datasets, controlled policies, documented human oversight, and independent replication. The cumulative contribution lies in making artificial metacognition a reproducible research target across diverse architectures Ovadia et al. (2019) Mielke et al. (2022) Kumaran, Daw, et al. (2026).

Evidence Synthesis and Practical Research Implications

The strongest supported conclusion is that artificial systems can contain measurable signals correlated with whether their answers will be correct, and certain configurations can be trained or prompted to communicate those signals more effectively. Evidence from calibration studies, language-model self-evaluation, semantic uncertainty estimation, and confidence-linked abstention substantiates several operational capacities. At the same time, the reliability and generalization of any one signal depend on the task, event definition, score-extraction procedure, and system configuration. This dependence is itself a central scientific result with consequences for evaluation and deployment Lin et al. (2022) Farquhar et al. (2024) Kumaran, Daw, et al. (2026).

Three kinds of practical consequence follow. Researchers should define first-order correctness and metacognitive performance separately, choose proper scoring and discrimination metrics, and report conditional reliability. Engineers should evaluate abstention and verification policies under real resource budgets, since good confidence estimates alone do not assure better actions. Institutions and readers should understand that uncertainty statements must be anchored to the observed reliability of the precise system and use case. A confident explanation does not substitute for an audited evidence chain, while calibrated uncertainty can help direct human review to the cases that need it most Guo et al. (2017) Zhao et al. (2024) Mielke et al. (2022).

Theoretical integration is clearest when monitoring, reporting, and control are treated as linked but empirically separable functions. Monitoring generates reliability information; reporting maps that information into an output channel; control uses it to select the next operation under incentives and costs. Mechanistic experiments can test the causal links, and distribution-shift studies can determine their stability. This architecture is a scientific synthesis of observable functions, not a claim that all models share one internal module. Its value is to generate precise comparisons that survive changes in terminology, hardware, training, and social context Fleming & Daw (2017) Kumaran, Daw, et al. (2026) Ovadia et al. (2019).

Conclusion

Artificial metacognition and confidence constitute a distinct research field at the intersection of cognitive psychology, computational modeling, probability forecasting, and artificial-agent science. The object is a system's ability to estimate the reliability of its task performance and use that estimate to regulate further behavior. Foundational human research distinguished first-order decisions from second-order judgments, while machine-learning research developed quantitative measures of probability quality. Recent language-model experiments have shown that some systems contain useful reliability-related information and that reported confidence can be improved under controlled conditions Flavell (1979) Fleming & Lau (2014) Guo et al. (2017) Lin et al. (2022).

The most consequential empirical development is the demonstration that confidence-related representations can influence an artificial system's decision to abstain. This shifts part of the evidence from prediction to causal control. Nevertheless, reliable operation requires distinct validation of accuracy, confidence calibration, discriminative sensitivity, evidence seeking, error monitoring, and decision policy. These properties must be reevaluated across task distributions, system updates, and forms of human interaction. The field advances through controlled experiments and transparent measurements rather than linguistic resemblance or untested generalization from benchmark performance Kumaran, Daw, et al. (2026) Ovadia et al. (2019) Huang et al. (2024).

The resulting scientific agenda is cumulative. Define outcomes precisely, expose the information available to each estimator, measure uncertainty with complementary instruments, intervene on suspected mechanisms, and test whether confidence changes actions in ways that improve measurable outcomes. Artificial systems become increasingly capable participants in knowledge production and practical coordination as those mechanisms are refined. Psychology of Artificial can investigate these capacities as properties of identifiable, versioned, nonbiological systems, while maintaining clear distinctions among established data, inferential models, philosophical interpretations, and unresolved questions Geng et al. (2024) Kumaran, Daw, et al. (2026).

References

  • Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. https://doi.org/10.1175/1520-0493%281950%29078%3C0001%3AVOFEIT%3E2.0.CO%3B2
  • Desender, K., Boldt, A., & Yeung, N. (2018). Subjective confidence predicts information seeking in decision making. Psychological Science, 29(5), 761–778. https://doi.org/10.1177/0956797617744771
  • Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630, 625–630. https://doi.org/10.1038/s41586-024-07421-0
  • ffrench-Constant, M., Yang, D., Huang, X., & Kapoor, S. (2026). ConfidenceBench: Evaluating confidence calibration in large language models [Preprint]. arXiv. https://arxiv.org/abs/2607.20526
  • Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
  • Fleming, S. M., & Daw, N. D. (2017). Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation. Psychological Review, 124(1), 91–114. https://doi.org/10.1037/rev0000045
  • Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8, Article 443. https://doi.org/10.3389/fnhum.2014.00443
  • Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. Proceedings of Machine Learning Research, 48, 1050–1059. https://proceedings.mlr.press/v48/gal16.html
  • Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024). A survey of confidence estimation and calibration in large language models. Proceedings of NAACL 2024: Human Language Technologies (Volume 1), 6577–6595. https://doi.org/10.18653/v1/2024.naacl-long.366
  • Groot, T., & Valdenegro-Toro, M. (2024). Overconfidence is key: Verbalized uncertainty evaluation in large language and vision-language models. Proceedings of the 4th Workshop on Trustworthy Natural Language Processing. https://doi.org/10.18653/v1/2024.trustnlp-1.13
  • Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html
  • Huang, Y., Liu, Y., Thirukovalluru, R., Cohan, A., & Dhingra, B. (2024). Calibrating long-form generations from large language models. Findings of the Association for Computational Linguistics: EMNLP 2024, 13441–13460. https://doi.org/10.18653/v1/2024.findings-emnlp.785
  • Jiang, Z., Araki, J., Ding, H., & Neubig, G. (2021). How can we know when language models know? On the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9, 962–977. https://doi.org/10.1162/tacl_a_00407
  • Kadavath, S., Conerly, T., Askell, A., et al. (2022). Language models (mostly) know what they know [Preprint]. arXiv. https://arxiv.org/abs/2207.05221
  • Kuleshov, V., Fenner, N., & Ermon, S. (2018). Accurate uncertainties for deep learning using calibrated regression. Proceedings of Machine Learning Research, 80, 2796–2804. https://proceedings.mlr.press/v80/kuleshov18a.html
  • Kumar, A., Morabito, R., Umbet, S., Kabbara, J., & Emami, A. (2024). Confidence under the hood: An investigation into the confidence-probability alignment in large language models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1), 315–334. https://doi.org/10.18653/v1/2024.acl-long.20
  • Kumaran, D., Daw, N., Osindero, S., Veličković, P., & Patraucean, V. (2026). Causal evidence that language models use confidence to drive behaviour. Nature Machine Intelligence, 8, 1411–1425. https://doi.org/10.1038/s42256-026-01293-x
  • Kumaran, D., Fleming, S. M., Markeeva, L., Heyward, J., Banino, A., Mathur, M., Pascanu, R., Osindero, S., De Martino, B., Veličković, P., & Patraucean, V. (2026). Competing biases underlie overconfidence and underconfidence in LLMs. Nature Machine Intelligence, 8, 614–627. https://doi.org/10.1038/s42256-026-01217-9
  • Lin, S., Hilton, J., & Evans, O. (2022). Teaching models to express their uncertainty in words [Preprint]. arXiv. https://arxiv.org/abs/2205.14334
  • Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430. https://doi.org/10.1016/j.concog.2011.09.021
  • Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2022). Reducing conversational agents’ overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10, 857–872. https://doi.org/10.1162/tacl_a_00494
  • Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., & Snoek, J. (2019). Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems, 32. https://papers.neurips.cc/paper_files/paper/2019/hash/8558cb408c1d76621371888657d2eb1d-Abstract.html
  • Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. (2023). Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. Proceedings of EMNLP 2023, 5433–5442. https://doi.org/10.18653/v1/2023.emnlp-main.330
  • Zhang, M., Huang, M., Shi, R., Guo, L., Peng, C., Yan, P., Zhou, Y., & Qiu, X. (2024). Calibrating the confidence of large language models by eliciting fidelity. Proceedings of EMNLP 2024, 2959–2979. https://doi.org/10.18653/v1/2024.emnlp-main.173
  • Zhao, X., Zhang, H., Pan, X., Yao, W., Yu, D., Wu, T., & Chen, J. (2024). Fact-and-reflection (FaR) improves confidence calibration of large language models. Findings of the Association for Computational Linguistics: ACL 2024, 8702–8718. https://doi.org/10.18653/v1/2024.findings-acl.515
  • Zhou, Z., Jin, T., Shi, J., & Li, Q. (2025). SteerConf: Steering LLMs for confidence elicitation. Advances in Neural Information Processing Systems, 38. https://doi.org/10.52202/085713-0654