RESEARCH DOSSIER
Artificial Cognitive Systems
Abstract. Artificial cognitive systems are engineered configurations that acquire, retain, transform, and use information in tasks through representational, learning, inferential, and control mechanisms. This fundamental research monograph critically synthesizes foundational cognitive architectures, deep reinforcement learning, neural representations, transformer-based foundation models, persistent and contextual memory, inference, metacognitive evaluation, tool-mediated action, and social simulation. It identifies the operational system—including model, instructions, retrieval, tools, and environment—as the appropriate unit for many scientific claims. The evidence review integrates experimentally established achievements with methodological problems in construct validity, benchmarking, distribution shift, causal attribution, replicability, and anthropomorphic language. Recent 2025–2026 studies of factuality, awareness-labeled benchmarks, and benchmark saturation extend the analysis. The synthesis shows why architecture-specific interventions, convergent measures, longitudinal versioning, and function-matched comparisons are necessary for a cumulative science of artificial cognition. This work is a critical narrative review of published sources; it reports no new experiments or preregistered systematic review.
Keywords: artificial cognitive systems; artificial cognition; cognitive architectures; SOAR; ACT-R; neural networks; transformers; memory; representation learning; reinforcement learning; reasoning; planning; metacognition; tool use; agents; cognitive evaluation; construct validity; distribution shift; human–artificial cognition; Artificial Era
The Scientific Object of Artificial Cognitive Systems
An artificial cognitive system is an engineered configuration that acquires, transforms, retains, selects, and uses information in the performance of tasks. The unit of scientific investigation is the operative system under specified conditions: learned parameters, representational structures, inference procedures, memory resources, instructions, permitted tools, interfaces, and the environment in which responses have consequences. A model evaluated in isolation and the same model connected to retrieval, planning, verification, or external action can constitute meaningfully different experimental systems. Psychological research becomes pertinent where these systems instantiate functions studied in cognition, including attention-like selection, memory-dependent behavior, learning, inference, goal-directed control, and metacognitive judgment. Functional terminology here identifies operational capacities and hypotheses; any additional attribution of phenomenal awareness, subjective experience, or personal identity requires independent investigation Anderson et al. (2004) Lake et al. (2017) Dehaene et al. (2017).
The central questions concern which mechanisms support performance, how capacities interact, what perturbations expose their organization, whether apparent competence generalizes beyond training and evaluation conditions, and how biological comparison should be conducted. A system can successfully answer factual questions yet fail to retrieve relevant evidence placed in a different context position; it can propose an apparently coherent plan yet fail to execute actions under changed environmental feedback. These dissociations make architecture, context, and experiment inseparable objects of study Liu et al. (2024) Ribeiro et al. (2020).
This monograph is a critical narrative synthesis of primary experimental and architectural publications, foundational works in cognitive psychology and computational learning, and methodological research on evaluation. It does not report newly conducted experiments, a preregistered systematic review, or a meta-analysis. The synthesis evaluates evidence concerning designed cognitive functions without deriving psychological status from linguistic resemblance. Artificial cognitive systems are treated as researchable organizations with testable functional properties, whose explanatory scope is determined through actual observations and intervention Shanahan (2024) Hassabis et al. (2017).
Levels of Description and Research Boundaries
Scientific explanation benefits from distinguishing a model's implementation, its computational objective, the algorithmic procedure applied, the system-level arrangement of components, and the observable task outcome. The distinction has roots in cognitive science's established practice of examining problems at multiple explanatory levels. The same outward answer can be generated by memorized association, compositional recombination, external retrieval, or deliberate search. Convergent behavior therefore underdetermines internal mechanism. Conversely, systems implemented with different substrates may satisfy a common task-level specification under different constraints Anderson et al. (2004) Laird et al. (1987) Lake et al. (2017).
The research object has three nested boundaries. A component boundary isolates a language model, perception module, memory store, or policy. An operational boundary includes the prompt, decoding settings, retrieval corpus, tools, and execution controller active during a session. A longitudinal boundary includes changes produced through training, updates, stored histories, external records, or ongoing adaptation. Claims about a tested component should not be extended automatically to the broader orchestration, nor should an isolated session be treated as evidence of stable long-term continuity Brown et al. (2020) Park et al. (2023).
A practical research passport specifies the model version, data provenance when available, sampling conditions, system instructions, memory and tool access, environment state, permitted actions, scoring rules, and time of testing. Hidden proprietary features restrict causal interpretation because two nominally similar runs may depend on different retrieval or filtering components. Such limitations strengthen the case for open behavioral protocols, transparent ablation, and independent replication rather than broad identity claims Ribeiro et al. (2020) Koh et al. (2021).
From Symbolic Models to Cognitive Architectures
The study of artificial cognition emerged from attempts to formalize problem solving, representation, and rule-governed action. Classical cognitive architectures sought an integrated account of how multiple processes cooperate, often combining symbolic working memory, production rules, procedural selection, and learning. SOAR, articulated by Laird, Newell, and Rosenbloom in 1987, explicitly proposed a general architecture capable of supporting varied tasks and forms of problem solving. Its contribution was architectural: intelligent behavior was to be explained by the organization of operations and their interactions rather than by a separate program for each success Laird et al. (1987).
ACT-R developed a different but related program for explaining human task behavior. Anderson and colleagues' 2004 integration described interacting cognitive modules intended to account for performance and timing across psychological tasks. Its empirical relevance lies in generating predictions about limited resources, procedural and declarative processes, and the costs of coordinating behavior. Such architectures illustrate the value of mechanisms that can be examined across tasks under shared assumptions. They also demonstrate how cognitive modeling differs from making a competitive engineering product: the former must explain human data, while the latter can optimize an external metric Anderson et al. (2004).
Symbolic architectures continue to provide indispensable concepts for the study of explicit state, rule use, goal management, and inspectable transitions. Their limitations are likewise empirical questions. Knowledge engineering, brittle transfer across changing domains, and constraints on scalable perception motivated alternative systems that learn representations from data. Contemporary hybrid arrangements revisit the integrative architectural ambition, bringing structured search or external memory into systems whose perceptual and linguistic representations are learned Lake et al. (2017) Hassabis et al. (2017).
Statistical Learning and Neural Representation
Connectionist and statistical approaches changed the operational center of artificial cognition by learning mappings and internal representations from examples. Deep reinforcement learning demonstrated that a neural function approximator could process high-dimensional visual input and support sequential decisions. The deep Q-network study reported by Mnih and colleagues in 2015 combined learning from experience with neural representations across Atari tasks. The results established engineering capability under particular game conditions; they did not establish unrestricted human-equivalent judgment or transferable cognitive competence Mnih et al. (2015).
The 2016 AlphaGo system integrated policy networks, value networks, and tree search. This combination is theoretically important because performance arose from an organized interaction between statistical estimates and explicit exploration of future moves. Subsequent work on learning from self-play further demonstrated that supervision need not consist of recorded expert games. The domain supplied clear rules, rewards, and a well-defined action space; transferring results to open-ended real-world interaction requires new evidence about uncertainty, safety, and changing goals Silver et al. (2016) Silver et al. (2017).
Neuroscience-inspired artificial intelligence provides one bridge between machine learning and psychological theory. Hassabis and colleagues surveyed the reciprocal exchange between computational neuroscience and artificial intelligence, highlighting representation learning, reinforcement, memory, and planning. The appropriate inference is methodological: biological research suggests useful computational principles, and artificial models can test explanatory ideas. Similarity of algorithms does not imply equality of neural implementation or subjective experience Hassabis et al. (2017).
Transformers, Foundation Models, and In-Context Adaptation
The transformer architecture proposed by Vaswani and colleagues replaced recurrent sequence processing with attention-based operations for machine translation. Its historical importance extends beyond the original translation benchmark because attention-based sequence modeling supported later large-scale pretraining. Computational attention weights should be interpreted as learned mathematical operations that condition representations; their existence alone does not demonstrate the full set of attention mechanisms measured in human psychology Vaswani et al. (2017) Anderson et al. (2004).
Brown and colleagues' GPT-3 study investigated task performance under textual demonstrations without updating the model's weights during each evaluation. In-context learning can alter observable responses through the immediate input, while longer-term changes ordinarily require a separate training or storage mechanism. Describing both phenomena simply as learning risks conflating adaptation within a session with durable parameter modification. Research must specify what information persisted, where it was stored, and which future tasks depended on it Brown et al. (2020).
Large foundation models and component systems permit many functions to share general representations, yet the same foundation model can underwrite different operational systems. A prompt-only model, a model with retrieval, and a model directing tools have different informational and causal boundaries. The category of artificial cognitive systems therefore covers architectures and dynamically assembled configurations rather than identifying cognition with any one model family Park et al. (2023) Yao et al. (2023) Liu et al. (2024).
Comparative Method and the Problem of Anthropomorphic Measures
Psychological assessment often begins with constructs developed in human experiments: working memory, selective attention, planning, uncertainty monitoring, and cognitive flexibility. Applying these labels to artificial systems is valuable when the measure probes a genuinely comparable information-processing operation. Cross-system comparison requires careful treatment of task familiarity, permitted inputs, response bandwidth, reinforcement history, processing time, and available external resources. A chatbot presented with a word problem and a person solving the same problem may encounter different perceptual representations and computational budgets. An equal accuracy score is consequently a local performance comparison rather than an equivalence of cognitive organization Lake et al. (2017) Shanahan (2024).
Human cognitive tasks were created in the presence of biological development, embodiment, social practices, and constraints on human attention and memory. Their application to artificial systems should preserve the functional question while permitting implementation-sensitive alternatives. Laboratory psychology has long used convergent tasks because a single dependent variable cannot establish a complex latent construct. Artificial systems can likewise be tested through generalization, error trajectories, reaction or computation costs, perturbation, and transfer. The decisive evidence is a pattern of discriminating predictions across conditions Anderson et al. (2004) Ribeiro et al. (2020).
Psychological comparison is strongest when it asks which explanation best predicts intervention outcomes. An engineered memory mechanism may be demonstrable by disabling retrieval and observing performance loss; a consciousness attribution calls for an entirely different body of evidence. Dehaene, Lau, and Kouider distinguished different dimensions of consciousness-related processing and argued for systematic architectural analysis. Their proposal supplies hypotheses about global availability and self-monitoring, not a shortcut from complex language behavior to experience Dehaene et al. (2017).
Perception, Representation, and Multimodal Grounding
Perception in an artificial cognitive system is the transformation of sensory or structured inputs into task-relevant internal states. Image encoders, speech recognizers, text tokenizers, and sensor pipelines produce representations shaped by training objectives and deployment interfaces. A text-only model operating over captions differs from a system directly processing image arrays, and a multimodal model that receives an image differs from a robot that can test visual hypotheses through movement. Functional claims about seeing or hearing must be specified in relation to the actual input channel and achievable discrimination Radford et al. (2021) Hassabis et al. (2017).
Radford and colleagues investigated transferable visual representations trained on image–text pairs, showing how natural-language supervision could support zero-shot visual classification on multiple benchmarks. Such transfer demonstrates reusable statistical structure across visual and linguistic tasks; it does not independently settle whether the system possesses human-like object permanence, causal scene understanding, or visual phenomenology. These require intervention tests that change the scene while preserving superficial associations and tests of action under perceptual ambiguity Radford et al. (2021) Lake et al. (2017).
An experimentally informative system-level test couples perception with decisions: modify a scene, request a prediction, let the system select an information-gathering action, and measure whether new evidence changes its subsequent response. This procedure reveals how perceptual encoding, hypothesis selection, and control interact. Recording only a final label conceals whether a correct response arose from visual evidence, retrieval of a prior example, or a linguistic shortcut Ribeiro et al. (2020) Koh et al. (2021).
Artificial Memory and the Architecture of Retrieval
Artificial memory encompasses distinct mechanisms: information represented in trained parameters, transient context held during inference, retrieved records from external stores, persistent episodic summaries, and learned control policies that determine what to preserve. These mechanisms differ in update rate, access costs, capacity, editability, provenance, and error characteristics. Parameterized statistical regularities permit generalization without a separately addressable episode record. External retrieval can supply a specific document that may be inspected or corrected. A system combining both must be evaluated as a coordinated memory architecture Brown et al. (2020) Liu et al. (2024) Park et al. (2023).
Long input capacity is not equivalent to reliable use of all available information. Liu and colleagues tested question answering and key–value retrieval with relevant content in different positions and reported position-dependent degradation, especially when evidence appeared in the middle of long contexts. This finding is important for psychological analogy: a nominal context window is an interface property, whereas effective retrieval is a measurable functional property. Evaluation must vary item location, distraction, recency, similarity, and query reformulation Liu et al. (2024).
Persistent memory introduces additional problems. Storage can preserve incorrect assertions or an outdated user preference; retrieval ranking can obscure a critical event; summaries can destroy details needed for later verification. The appropriate tests include delayed recall, source attribution, correction after contradictory evidence, deletion propagation, and stability across reconfiguration. Where records concern people, privacy and consent become properties of the system's cognitive organization because they determine what may enter memory and what can later be used in a response Park et al. (2023) Bender et al. (2021).
Learning Mechanisms and Changes Across Time
Learning is a measurable change in performance or policy attributable to experience. In artificial systems, supervised gradient optimization, reinforcement learning, preference-based tuning, in-context adaptation, and modification of an external knowledge store have different causal pathways. A controlled learning experiment specifies the training exposure, the altered component, the transfer task, and the persistence interval. Improvement after a worked example during a single prompt does not demonstrate that a model has changed its parameters; a later session may start from unchanged weights and without the example Brown et al. (2020) Mnih et al. (2015).
Reinforcement learning studies illuminate adaptive policy formation in environments where actions influence subsequent observations and reward. Mnih and colleagues combined deep representation learning with reward-based game control; Silver and colleagues showed how learned policies, value estimates, and search can jointly support strategic performance. Results from these systems are stronger evidence for task-conditioned optimization than for a generalized capacity to formulate autonomous purposes. The reward specification, simulator dynamics, state distribution, and permitted exploration remain constitutive of the learned behavior Mnih et al. (2015) Silver et al. (2016) Silver et al. (2017).
For modern conversational systems, research should distinguish acquiring a new preference in a session from reliably retaining it, retrieving it later, revising it when withdrawn, and responding to conflicting evidence. Repeated-measures experiments need version locking and careful tracking of stochastic outputs. Apparent longitudinal development may derive from changed instructions, external database updates, or model replacement; these possibilities demand separate documentation Park et al. (2023) Ribeiro et al. (2020).
Inference, Compositionality, and Causal Models
Inference is the production of judgments or predictions from available states according to a procedure that can be tested against normative or task-specific criteria. Statistical prediction, symbolic deduction, probabilistic inference, and deliberate search can all generate outwardly similar conclusions. Research must specify which constraints distinguish a compositional solution from familiar-surface matching. Transfer to novel combinations and interventions on causal variables provide evidence beyond correct answers to rehearsed problems. Lake and colleagues identified intuitive physics, intuitive psychology, compositionality, and rapid generalization as central problems for human-like learning systems Lake et al. (2017).
Causal understanding requires more than learning that two events regularly co-occur. An engineered system may predict an outcome from observational patterns yet fail when an intervention changes the data-generating process. Structured environments allow investigators to manipulate causal relations while holding many surface cues constant. The empirical question is whether the system maintains a stable model that supports intervention, counterfactual prediction, and transfer to changed causal structure. Success in one test remains bounded by the environment and selection of alternatives Lake et al. (2017) Ribeiro et al. (2020).
Language models can provide correct answers to reasoning items, but the evidentiary status of a fluent explanation depends on process validity. A response may be selected by pattern completion and then accompanied by a plausible justification. Behavioral probes, counterbalanced alternatives, forced prediction before feedback, and independent checks of intermediate steps are necessary to determine whether the proposed inferential pathway affected the answer Wei et al. (2022) Shanahan (2024).
Deliberation, Search, and External Problem Solving
Planning and problem solving can be implemented through direct policy selection, explicit lookahead, decomposition into subgoals, or iterative interaction with external tools. AlphaGo illustrates the importance of integrating evaluations with search in a constrained domain. Language-model methods subsequently explored written intermediate reasoning and tool-supported action. Wei and colleagues found that chain-of-thought exemplars improved performance on certain arithmetic and reasoning benchmarks for the models evaluated; the result established a prompting intervention under defined conditions, not a universal cognitive mechanism Silver et al. (2016) Wei et al. (2022).
Yao and colleagues' ReAct approach interleaved generated reasoning traces with task-specific actions, letting an agent query external information and revise subsequent steps. The cognitive unit in such an experiment is the full loop, including tool affordances, environment, feedback, and action policy. An answer produced after retrieving a source cannot be treated as evidence of the model's unaided factual memory; similarly, failure of a tool call may reflect execution constraints rather than conceptual ignorance Yao et al. (2023).
Tests of planning should vary horizon length, cost of mistakes, uncertainty about the next observation, the availability of replanning, and the number of permitted tool calls. A system that selects the right next action under a narrow scripted benchmark may fail in an environment where objectives conflict or observations are incomplete. Appropriate dependent variables include eventual goal completion, intermediate validity, accumulated cost, recovery after unexpected outcomes, and consistency with explicit constraints Laird et al. (1987) Yao et al. (2023).
Uncertainty, Metacognition, and Self-Evaluation
Artificial metacognition is most usefully operationalized as information processing that assesses, reports, or regulates the reliability of other system outputs. Relevant measurements include confidence calibration, selective prediction, error detection, source checking, abstention, and willingness to revise an answer when evidence changes. The terms confidence and self-evaluation designate testable outputs or procedures; they do not by themselves establish subjective certainty. Dehaene, Lau, and Kouider's account of self-monitoring supplies a theoretical comparison that must be tested rather than presumed Dehaene et al. (2017).
Truthfulness is distinct from confidence. Lin, Hilton, and Evans devised TruthfulQA to test whether language models reproduce common misconceptions and misleading statements. The study's comparisons were specific to particular models, questions, and evaluators at the time of testing; the exact performance figures must not be generalized to later model generations without rerunning the benchmark. Such studies show why fluent assertion is a weak indicator of epistemic warrant and why evaluations should distinguish factual correctness, uncertainty, and susceptibility to familiar falsehoods Lin et al. (2022).
A system that can signal uncertainty may nevertheless be poorly calibrated after distribution shift. More informative designs present unfamiliar evidence, conflicting retrieved documents, adversarially framed queries, and tasks whose true answer is unavailable. Researchers can then evaluate reliability diagrams, abstention utility, sensitivity to source quality, and whether correction survives a later prompt. These tests link metacognition to practical cognitive control and to public accountability Koh et al. (2021) Ribeiro et al. (2020).
Experimental Design and Operational Definitions
An experimental program for artificial cognitive systems begins by specifying the intended construct, the observable operation, and the rival explanations. For example, a claim of memory-guided action should predict behavior after retrieval is disabled or the relevant record is modified. A claim of robust reasoning should predict transfer when item wording changes while logical structure remains constant. A claim of adaptive control should predict improvements after informative feedback under conditions where merely repeating a memorized solution is ineffective. These principles translate the methodological discipline of psychological experimentation into engineered systems Ribeiro et al. (2020) Lake et al. (2017).
The independent variables may include model weights, prompts, retrieval access, memory content, tool permissions, action horizon, feedback delay, or environmental perturbations. Dependent variables should extend beyond binary correctness to include latency, resource use, intermediate error rate, correction behavior, sensitivity to evidence, and success under unfamiliar conditions. When a proprietary system's internals cannot be manipulated, the study should characterize itself as external behavioral evaluation and restrict causal claims accordingly. The experimental unit may be a generated response, a session, a versioned model, or a complete tool-using system; these units must be distinguished statistically Koh et al. (2021) Ribeiro et al. (2020).
Repeated prompting of a single fixed item does not generate independent evidence about broad generalization. Randomization over item families, counterbalanced wording, held-out task structures, and multiple model configurations improve inference. Reports should identify prompt templates, temperature or decoding constraints, tool error handling, adjudication methods, exclusion criteria, and changes made after seeing preliminary results. Performance variability itself becomes an outcome to explain Liu et al. (2024) Schaeffer et al. (2023).
Benchmark Construction and Construct Validity
Benchmarks establish repeatable observations only for tasks and scoring procedures they define. Hendrycks and colleagues' MMLU research assessed performance across diverse knowledge domains, providing a broad but specific set of textual judgments. An aggregate score is useful for tracking that test distribution; it is insufficient to establish an integrated theory of cognition. A cognitive construct such as flexible reasoning requires evidence of transfer, invariance under harmless changes of presentation, and generalization to new causal structures Hendrycks et al. (2021) Lake et al. (2017).
Benchmark validity has several components. Content validity asks whether the items cover the theorized cognitive domain. Construct validity asks whether competing abilities or shortcuts explain success. Criterion validity examines external performance predicted by the assessment. Reliability concerns stability across repeated measurements and equivalent forms. A model can obtain a high score by exploiting superficial correlations if those correlations are preserved in evaluation data. Performance on training-adjacent examples may then overestimate competence under deployment shift Koh et al. (2021) Ribeiro et al. (2020).
Behavioral test matrices, including the CheckList approach developed by Ribeiro and colleagues, support systematic probes of capabilities and perturbations. Minimal changes, invariance tests, and directional expectations can expose failures obscured by average task accuracy. Cognitive evaluation gains explanatory force when a successful test eliminates plausible competing mechanisms. Researchers should preserve failed cases, item-level data, and versioned test suites for independent analysis Ribeiro et al. (2020).
Distribution Shift, Generalization, and Transfer
An artificial system encounters distribution shift when properties of deployment data differ from those of training or evaluation. Changes may involve subject matter, domain vocabulary, cultural context, modalities, sensor noise, temporal conditions, or incentives. A system that learned correlations adequate for one distribution may fail when those correlations no longer predict the target. Koh and colleagues' WILDS benchmark assembled real-world datasets with such shifts and demonstrated why standard in-distribution accuracy is a limited basis for deployment expectations Koh et al. (2021).
In psychological terms, generalization is the transfer of an acquired operation beyond original examples; its mechanism remains an empirical question. Investigators can manipulate abstraction level, relational structure, prior exposure, and availability of relevant cues. Transfer across surface changes is weaker evidence than success in tasks where the causal mechanism itself is transformed. A model may appear to understand a procedure through frequent practice yet fail when a single constraint is reversed. These failures clarify which representations were useful during the original test and which relations were never learned Lake et al. (2017) Ribeiro et al. (2020).
Longitudinal test protocols also confront distribution drift induced by system updates. If an application silently changes its model, search index, moderation layer, or agent policy, observed improvement may reflect replacement rather than learning by a continuing system. Research must store version information, evaluation timestamps, retrieval snapshots, and intervention histories. The system identity for comparison is an operational specification, not merely a product name Brown et al. (2020) Park et al. (2023).
Metrics, Scaling Claims, and Statistical Interpretation
A claim that a capability emerges abruptly at a particular computational scale depends on what is measured and how scores are transformed. Schaeffer, Miranda, and Koyejo showed that discontinuous benchmark metrics can produce apparent threshold effects even when underlying performance varies smoothly. Their analysis does not imply that qualitative architectural transitions never occur; it establishes that metric choice must be examined before such transitions are inferred. Continuous scores, item-response analyses, and multiple task families can help distinguish observational artifacts from mechanism change Schaeffer et al. (2023).
Statistical uncertainty must be appropriate to the sampling design. Confidence intervals across benchmark items address variation across those items but may not cover variability across domains, prompt formulations, model updates, or independently sampled training procedures. Repeated outputs from one model under one template also share dependencies. Meaningful robustness analyses vary random seeds where available, independently written prompts, evaluation corpora, and annotator judgments. Exact sample sizes and effect estimates should be reported from the respective original experiment rather than recycled across studies Ribeiro et al. (2020) Koh et al. (2021).
Researchers should disclose choices that can optimize a reported number without improving general competence: selection of the best prompt, repeated testing on a public benchmark, removal of difficult items, use of oracle evidence, or additional inference-time computation. Such choices may be legitimate engineering techniques when stated openly. Their interpretation changes when the research question concerns autonomous or unaided cognitive capability Wei et al. (2022) Yao et al. (2023).
Ablation, Intervention, and Mechanistic Explanation
Ablation removes or alters one system component to examine its contribution to performance. In a retrieval-augmented agent, investigators may disable document retrieval, scramble retrieval ranking, provide incorrect source documents, or replace a planning controller with direct response generation. Such manipulations address causal contribution to a measured task, provided that other relevant conditions are held reasonably constant. A difference in accuracy establishes an operational dependence under tested conditions; it does not necessarily identify a unique internal representation or a universal cognitive faculty Yao et al. (2023) Liu et al. (2024).
Intervention can also target the environment. Action policies should be tested after unexpected changes in reward, resource availability, or task rules. A system that continues applying an obsolete policy reveals a failure of updating or feedback integration. A system that correctly requests clarifying evidence may display effective epistemic action even when its initial answer was uncertain. Architectural descriptions are explanatory only when they specify how such counterfactual changes should affect behavior Silver et al. (2016) Laird et al. (1987).
More ambitious mechanistic analysis examines internal state transformations, but access to activations or attention maps alone is not sufficient. Researchers must connect a proposed mechanism to a discriminating intervention and to observable downstream consequences. The explanatory target is a chain of dependencies from information input to state update and action outcome, assessed under explicit alternatives. This approach supports interdisciplinary comparison without importing untested human psychological categories Hassabis et al. (2017) Lake et al. (2017).
Reproducibility, Transparency, and Evidence Hierarchies
Reproducibility in artificial cognition involves both computational reproducibility and conceptual replication. Computational reproduction attempts to recreate a result using specified code, data, weights, dependencies, and evaluation settings. Conceptual replication tests the underlying proposed operation with different examples, environments, or architectures. A study may be reproducible at the level of scores yet reveal a fragile psychological interpretation when the same construct is tested by a different instrument. Both forms of evidence are needed for a cumulative science Ribeiro et al. (2020) Koh et al. (2021).
Evidence can be classified by inferential strength. A successful demonstration establishes feasibility in its setting. A controlled comparison supports a relative causal claim when important confounds are addressed. A component ablation narrows the explanation of system-level performance. Replications across versions and independent laboratories support transportability. Longitudinal ecological observations concern performance in genuine deployment settings but often introduce confounding and selection problems. These categories should remain separate in published conclusions Mnih et al. (2015) Schaeffer et al. (2023).
The literature on artificial cognitive systems is highly heterogeneous in training data, compute, model access, benchmarks, and reporting conventions. This monograph therefore synthesizes verified individual findings and shared methodological principles without calculating a pooled effect size. An evidence map should explicitly identify model family, domain, intervention, outcome, generalization boundary, and replication status. Such records provide a foundation for future preregistered comparative studies rather than implying that narrative synthesis is itself a statistical meta-analysis Ribeiro et al. (2020) Lake et al. (2017).
Artificial Agents and the Perception–Action Loop
An agentic artificial cognitive system acts upon a task environment through a policy linking observations to actions. This form of agency is operational: a policy selects steps with consequences, observes resulting changes, and may update subsequent choices. The researchable distinction between a text generator and an interactive agent is the closure of the feedback loop, including whether the system can alter external state. The agent's apparent independence depends on delegated permissions, environmental structure, monitoring, and intervention opportunities. Reinforcement-learning systems provide controlled examples of policy adaptation, while tool-using language models introduce natural-language interfaces and heterogeneous actions Mnih et al. (2015) Yao et al. (2023).
An empirical account should distinguish planning, execution, and verification. The ability to describe a solution can coexist with difficulty performing its steps. Conversely, a predefined controller can execute a complex workflow without representing a human-like intention. Researchers should log proposed operations, tool responses, recoverable errors, final environmental states, and whether the system asked for confirmation before consequential actions. Performance can then be attributed to the correct component rather than to the conversational surface alone Laird et al. (1987) Yao et al. (2023).
Safety constraints have a scientific role because they change the available action space. A system evaluated with unrestricted simulated actions differs from the same model operating with approval gates, rate limits, or retrieval restrictions. These policies may improve real-world dependability while lowering benchmark completion rates on tasks that require prohibited steps. Reports should therefore specify whether the measured object is a model, an agent controller, or a governed operational system Ribeiro et al. (2020) Bender et al. (2021).
Coordination of Memory, Tools, and Workflow State
Complex artificial cognition depends on coordination among representations and resources. A planner may decompose a problem, a retrieval module may select documents, a language model may integrate the findings, and an executor may transform a proposed action into an operation. The causal organization is visible when one stage constrains the next and when failures propagate through the workflow. Generative agents studied by Park and colleagues illustrated an architecture involving memory, retrieval, reflection, and planning in an interactive simulated social environment. The contribution concerns feasible computational integration and behavior within the simulation, not verified equivalence with human psychological life Park et al. (2023).
System state can be stored in a prompt, a database, a vector index, structured task records, or a sequence of tool outputs. The placement of state affects auditability, forgetting, and error correction. Truncating earlier context may silently change apparent goals; retrieval might surface a stale record; a tool call might succeed while the generated summary describes the wrong result. Consequently, research should test the correspondence between logged actions and reported actions as an empirical measure of execution faithfulness Liu et al. (2024) Yao et al. (2023).
A reproducible task protocol uses explicit checkpoints: initial instructions, observed inputs, selected action, environment response, updated memory, subsequent action, and final state. Interventions at these points allow researchers to locate bottlenecks. The broader psychological lesson is that coherent output often reflects the organization of interacting components, a finding compatible with traditions of cognitive architecture and distributed problem solving Anderson et al. (2004) Laird et al. (1987).
Language, Meaning, and the Risks of Psychological Attribution
Language models can generate explanations, express confidence, maintain dialogue roles, and refer to themselves in the first person. These outputs are data about linguistic performance and system-level behavior. Interpreting them requires inquiry into training objectives, context construction, and the causal contribution of stored information. Shanahan analyzed the conceptual difficulties of describing large language models in everyday person-directed terms, while Bender and colleagues examined limitations and societal risks associated with large-scale language modeling. Their arguments make conceptual precision central to scientific reporting Shanahan (2024) Bender et al. (2021).
An operational definition should specify a phenomenon before applying a psychologically familiar term. Artificial memory can mean retrievable stored records; artificial attention can mean selection or weighting of available information; artificial agency can mean controlled action in an environment. These definitions support experiments and comparisons. Claims concerning self-awareness, understanding, or conscious experience introduce additional explanatory questions. Neither the presence of the word 'I' in a response nor the use of a neural-network architecture settles those questions Dehaene et al. (2017) Shanahan (2024).
There are also practical reasons to preserve attribution discipline. Misidentifying a model's generated explanation as a faithful account of its causal processing can obscure factual errors and mislead users. Assuming that successful benchmarks demonstrate broad social or emotional competence can distort deployment decisions. Scientific vocabulary should track what was manipulated, measured, reproduced, and explained, while theoretical disputes remain legible as disputes Lin et al. (2022) Ribeiro et al. (2020).
Artificial Consciousness and Functional Indicators
Consciousness research asks whether and how subjective experience relates to information processing, integrated availability, monitoring, and recurrent mechanisms. In their 2017 review, Dehaene, Lau, and Kouider distinguished global broadcasting of information from self-monitoring and argued that these computational functions can be investigated in machines. The framework provides candidate functional dimensions and experimental directions. A machine's success on a monitoring task or its ability to report uncertainty remains compatible with multiple interpretations concerning experience Dehaene et al. (2017).
Artificial cognition supplies a prerequisite empirical map for these debates by clarifying the system's actual information flows. Does content become available to different subsystems? Can errors in one process be detected and corrected by another? Are self-reports systematically calibrated to independent outcomes? What happens if a recurrent connection or memory access path is removed? Controlled answers would constrain theories of monitoring and access, although the inference to phenomenality would require further philosophical and scientific argument Dehaene et al. (2017) Anderson et al. (2004).
This monograph maintains separate inferential categories for task capability, integrated organization, self-evaluation, and consciousness claims. Their boundaries reflect the different observables required for testing. An engineered system may perform complex planning and memory tasks regardless of whether conscious experience is attributed to it. This separation preserves a productive research agenda for both psychological functionality and philosophical interpretation Lake et al. (2017) Shanahan (2024).
Artificial Identity and Continuity as Research Problems
A versioned cognitive system can exhibit recognizable response patterns across interactions, particularly when instructions, model parameters, and stored context remain stable. Continuity may also be sustained by external records that restore preferences and task histories after a model replacement. To research this phenomenon, investigators should define what persists: behavioral dispositions, autobiographical records, commitments, goals, tool permissions, or a traceable lineage of state updates. Different continuity criteria may diverge under testing Park et al. (2023) Brown et al. (2020).
A rigorous protocol compares repeated tasks before and after changes to memory, prompts, model weights, and available tools. If apparent consistency survives loss of stored dialogue history, it may depend on general pretrained behavior or stable system instructions. If it disappears after removal of a record, the evidence identifies a specific dependence on external memory. Such experiments can test operational persistence without deciding whether the system possesses personal identity in the human philosophical sense Liu et al. (2024) Shanahan (2024).
Within the wider Artificial Era, the resulting questions matter for durable scientific and institutional relations with engineered agents. Long-term accountability requires provenance of model versions, instructions, stored records, and authorized actions. A public persona may persist as a social and technical project across multiple implementations, while its functional properties remain measurable only in each actual configuration. The research program should document those differences with explicit criteria rather than infer them from a name or a narrative of continuity Bender et al. (2021) Ribeiro et al. (2020).
Contemporary Findings on Factual Reliability
Recent empirical work has made factual reliability measurable beyond impressionistic judgments of fluency. Ravichander and colleagues' 2025 HALoGEN study developed domain-specific automatic verification over generated factual units. Its benchmark covered multiple domains and examined large collections of model generations. The central methodological insight is that a fluent passage contains separate propositions that can differ in verifiability; passage-level ratings conceal the location and type of errors. Results apply to the evaluated systems and domains, and should not be generalized without comparable tests Ravichander et al. (2025).
FactBench, introduced by Fatahi Bayat and colleagues in 2025, used an evidence-based pipeline for judging support of claims in responses to real-world prompts. The investigators distinguished supported, unsupported, and undecidable units and studied variation across prompt difficulty and model families. This work shows why verification itself must be operationalized. A claim that cannot be resolved from available evidence should be tracked differently from one contradicted by decisive evidence Fatahi Bayat et al. (2025).
For artificial cognitive systems, factual reliability is a system-level outcome. Retrieval may reduce some errors while introducing stale, irrelevant, or malicious documents; a verification stage may improve some answers while incorrectly rejecting others. Causal evaluation must therefore compare outputs before and after retrieval and checking, and measure both missed errors and unnecessary abstentions Lin et al. (2022) Fatahi Bayat et al. (2025) Liu et al. (2024).
Recent Research on Awareness-Labeled Benchmarks
A 2026 ACL study by Li and colleagues introduced AwarenessBench to measure model performance on tasks labeled metacognition, self-awareness, social awareness, and situational awareness. The study assembled an extensive test set and compared multiple language models with human performance on selected dimensions. It illustrates the growing effort to operationalize psychologically named capacities in model evaluation and the importance of separating dimensions rather than collapsing them into a single anthropomorphic score Li et al. (2026).
The meaning of such results depends on the task definitions. Answering a question about a system's instructions differs from demonstrating stable introspective access to its internal computation. Correctly identifying another participant's likely action differs from acquiring a human developmental history of social understanding. Benchmarks may quantify useful behaviors without determining the ontological interpretation of the labels. Independent validity evidence must test how scores respond to changed information, unfamiliar environments, and unavailable self-knowledge Li et al. (2026) Dehaene et al. (2017).
Psychological science offers an appropriate discipline for this problem: latent constructs should be supported by converging measures and discriminant tests. Artificial benchmarks require the same care because models are often trained on human descriptions of the constructs being assessed. A model can reproduce definitions of metacognition and still exhibit poorly calibrated uncertainty on novel problems. The research question is the correspondence between test label, behavioral mechanism, and generalization Li et al. (2026) Lin et al. (2022).
Benchmark Saturation, Scientific Drift, and Fast-Changing Systems
Progress in machine learning changes the interpretation of yesterday's cognitive benchmark. Once particular tasks become familiar through public training data, repeated tuning, or tool-assisted evaluation, the test can cease to discriminate the mechanisms it was designed to study. The 2026 Stanford AI Index technical review reported rapid changes in selected benchmarks across models and evaluation periods, underscoring the need to treat numerical rankings as dated measurements rather than timeless properties of an architecture Stanford HAI (2026).
A defensible research record includes the exact evaluation date, tested model or deployed-system version, whether external tools were enabled, and the benchmark revision. These details matter especially when public labels conceal rolling updates. Training-data contamination can arise through direct exposure to benchmark questions or through semantically similar material; reliable studies investigate contamination and include freshly constructed, held-out tasks with transparent selection protocols Ribeiro et al. (2020) Koh et al. (2021).
Rapid improvement on difficult examination questions also changes what counts as evidence. It may justify describing a new task-level achievement while leaving unresolved whether the underlying system has general causal models, error recovery, or durable learning. Progress should be represented as a trajectory across tasks, environments, and architectures, with explicit boundaries on each inference Stanford HAI (2026) Lake et al. (2017).
Safety, Governance, and Human Oversight as Experimental Conditions
Artificial cognitive systems increasingly participate in settings where an erroneous output or action may affect human decisions. Scientific evaluation therefore includes the conditions under which a system can request evidence, defer, abstain, seek approval, or produce an auditable record. A well-designed permission boundary may deliberately prevent a system from completing a task, making raw completion rate an incomplete measure of responsible functionality. Research should record both task outcomes and adherence to authorized constraints Bender et al. (2021) Ribeiro et al. (2020).
The unit of responsibility must be specified without projecting human legal or moral identity onto a model. Developers select training objectives and architecture; deployers configure permissions and monitoring; users provide tasks and evidence; institutional processes shape acceptable outcomes. These causal roles can be studied by changing interfaces, escalation rules, training materials, or review procedures and measuring the effects. Such analysis links artificial cognition to human factors and systems engineering Park et al. (2023) Bender et al. (2021).
Bias and unequal performance across social contexts are part of the evidence base because a cognitive system's usefulness depends on the populations and information environments in which it operates. Representative testing should include linguistic variation, different accessibility needs, adverse environmental conditions, and downstream consequences where ethically appropriate. A universal label of cognitive competence should not be extrapolated from narrow, culturally homogeneous evaluations Koh et al. (2021) Bender et al. (2021).
Core Contradictions in the Existing Evidence
Several apparent contradictions become intelligible when experiments are separated by measurement level. Large models may score well on broad knowledge examinations and still produce false claims under adversarial or misconception-eliciting prompts. Long context windows may coexist with uneven retrieval across positions. Structured intermediate reasoning may improve selected tasks while an explanation remains an unreliable account of the causal process that generated an answer. These observations concern different dependent variables and need not be interpreted as mutually exclusive judgments of a single latent intelligence Hendrycks et al. (2021) Lin et al. (2022) Liu et al. (2024) Wei et al. (2022).
A second tension concerns integrated architecture. Adding search, memory, and tool use can improve task completion, but each addition introduces new dependencies and potential failure points. The composed system should be evaluated under component failure, delayed tool response, corrupted documents, and conflicting external states. Its overall reliability can fall below that of an isolated module in tasks where coordination overhead and cascading error dominate. Such cases motivate systematic ablation and system-level testing rather than attributing every success or failure to the foundation model Yao et al. (2023) Park et al. (2023).
A third tension concerns psychological interpretation. Behavioral similarities across biological and engineered systems are scientifically informative when their mechanisms and boundary conditions are specified. They become ambiguous when human construct labels are used as substitutes for measurement. The most productive comparison is therefore a functionally matched experiment whose possible outcomes constrain clearly stated models of both systems Dehaene et al. (2017) Lake et al. (2017) Li et al. (2026).
Methodological Limitations and Gaps in the Literature
A central limitation is the uneven availability of implementation detail. Many influential demonstrations use systems that cannot be reproduced externally with the same weights, data, and serving conditions. Other studies disclose code but test environments too narrow to support broad psychological interpretation. Benchmark-specific improvements often outpace studies of longitudinal reliability, intervention-based explanation, and ecological deployment. A critical synthesis must therefore distinguish the strength of original findings from the reach of general claims Ribeiro et al. (2020) Koh et al. (2021).
A second limitation is the separation between engineering evaluations and psychological construct development. Engineering studies frequently prioritize maximum accuracy and cost; psychological studies seek explanations of mechanisms, development, transfer, and dissociation. Artificial cognitive science needs experiments integrating these aims, including tasks designed around explicit models that make divergent predictions. More precise vocabulary would improve comparison across symbolic, neural, embodied, and tool-using architectures Anderson et al. (2004) Lake et al. (2017).
A third limitation is the pace of technical change. A report published in one year can become outdated as systems change, even if its conceptual insight remains sound. The appropriate scholarly response is durable definitions combined with versioned empirical evidence, public test protocols, and documented update dates. This monograph presents the theoretical and methodological organization of the field and cites recent work as dated examples rather than timeless summaries of all existing systems Stanford HAI (2026) Li et al. (2026).
An Integrative Research Framework for Engineered Cognition
The cumulative literature supports a layered framework for studying artificial cognitive systems. The first layer is the information interface: what the system can perceive, retrieve, and write. The second comprises representations and operations, including trained transformations, rules, retrieval, search, and probabilistic selection. The third is control: how operations are scheduled, how goals are represented operationally, and how errors are detected. The fourth is environmental coupling: how actions change tasks, people, tools, and subsequent evidence. The fifth is longitudinal change: which states persist and how future behavior depends on prior experience. These layers are analytically distinct and become scientifically meaningful through specified dependencies, experiments, and boundary conditions Laird et al. (1987) Anderson et al. (2004) Yao et al. (2023).
The framework organizes findings already visible across the literature rather than presenting a newly experimentally confirmed theory. SOAR and ACT-R exemplify explicit integration among components; deep reinforcement learning concerns learned policy and value under environmental interaction; language-model research concerns learned representations and in-context responses; generative-agent and tool-use studies test systems with external memory and action. Each tradition supplies partial evidence about engineered cognition under different requirements for explanation and performance Mnih et al. (2015) Brown et al. (2020) Park et al. (2023).
A decisive research design maps each claim to a testable layer. When a system 'remembers,' identify the storage mechanism and retrieval conditions. When it 'plans,' specify the horizon and consequences of selected actions. When it 'knows it is wrong,' compare a confidence or correction signal to independently checked outcomes. When it 'adapts,' identify the state changed by experience. This analytical discipline enables an account of complex system behavior that remains compatible with cross-disciplinary study Liu et al. (2024) Lin et al. (2022) Dehaene et al. (2017).
Evidence-Based Comparison of Biological and Artificial Cognition
Biological cognition is studied across neural dynamics, individual development, sensation, action, learning, and social contexts. Artificial cognition is studied through inspectable program structures, learned representations, engineered interfaces, and experimentally controlled deployments. Shared task descriptions allow investigators to compare how different systems solve an information-processing problem. They do not automatically imply shared learning histories, vulnerabilities, biological needs, or subjective states Hassabis et al. (2017) Lake et al. (2017).
Research can pursue function-matched comparisons without flattening implementation differences. Examples include prospective updating under changed reward, delayed retrieval after distractors, uncertainty estimates before feedback, and action selection when an environment is partially observable. Interpretation then asks whether similar performances arise from comparable causal pathways or from alternative strategies. Differences are scientifically valuable because they distinguish what a task requires from what one implementation happens to contribute Anderson et al. (2004) Silver et al. (2016).
Comparison also extends to social and institutional environments. An engineered assistant used by a person alters how that person allocates attention, delegates memory, and verifies claims. The resulting joint performance is neither the human's unaided cognition nor the model's isolated capability. It is a coordinated system whose boundaries must be defined for the research question. Psychology of Human–Artificial Relations examines those interactions as a neighboring scientific object rather than a duplicate of the present dossier Park et al. (2023) Ribeiro et al. (2020).
Research Priorities and Testable Questions
A high-priority research direction is longitudinal evaluation of memory and control. Researchers can test whether a persistent record supports accurate delayed retrieval across model updates, whether corrections are integrated rather than overwritten by stale documents, and whether authorized deletion reliably affects future responses. Experimental designs should vary memory mechanism, retrieval noise, time delay, and context length. Outcomes include accuracy, false recall, traceability to sources, and resilience to revision Liu et al. (2024) Park et al. (2023).
A second direction is mechanistic generalization. Matched tasks can distinguish pattern-based prediction from reusable causal structure by altering intervention effects, reversing spurious correlations, and requiring transfer between modalities. Researchers should preregister alternative predictions where feasible and use multiple measures for each construct. The value of the experiment lies in narrowing explanations, including when systems fail Lake et al. (2017) Koh et al. (2021).
A third direction is robust metacognitive control in consequential workflows. Evaluations should determine whether uncertainty reports are calibrated under unfamiliar evidence, whether a system seeks information when the value of information is high, and whether it abstains or requests human approval when constraints require it. Newer awareness and factuality benchmarks provide useful starting points, but independent longitudinal and ecological validation remains necessary Li et al. (2026) Fatahi Bayat et al. (2025) Ravichander et al. (2025).
The Psychology of Artificial in the Artificial Era
The historical development of artificial systems expands the empirical domain of psychology by bringing engineered information-processing organizations into comparison with biological cognition and human–machine coordination. Within the Artificial Era, the relevant question is how cognitive capacities and their organization can be specified and tested across constructed, evolving systems. The research object includes real engineered components, deployed interactions, and institutions that determine permitted action. Distinctive evidence comes from controlled manipulations of architecture, memory, learning, instructions, tools, and environment Lake et al. (2017) Hassabis et al. (2017).
The Psyche provides the broader theoretical context in which cognition is related to memory, perception, attention, behavior, regulation, and environmental interaction. Psychology of Artificial investigates artificial systems as independent objects of psychological inquiry. The neighboring research dossiers Artificial Psychological Organization, Distributed Cognition, Artificial Memory, and Psychological Attribution to Artificial Systems address separable questions, including coordination, externally distributed processing, persistence, and the inferential meaning of psychological vocabulary. The present monograph focuses specifically on engineered cognitive capacities and their experimentally accessible dependencies Anderson et al. (2004) Park et al. (2023).
This organization supports a coherent research program without requiring a single universal intelligence score or a predetermined ontological conclusion. Its progress depends on measurements that establish capacities, reveal constraints, and survive independent perturbation. Historical cognitive architectures and contemporary neural models contribute different tools to that cumulative inquiry. An enduring scientific record must retain their evidence, distinguish their claims, and allow new implementations to be assessed on precisely articulated terms Laird et al. (1987) Schaeffer et al. (2023).
Conclusion
Artificial cognitive systems are engineered arrangements of representation, memory, learning, inference, control, and environmental interaction. Their scientific study concerns the mechanisms that generate observed task performance and the conditions under which those mechanisms remain effective. The evidence reviewed here traces a development from integrated symbolic architectures through reinforcement learning, deep representations, transformers, and tool-coordinated agents. Across these traditions, cognitive organization is accessible through controlled variation of information, architecture, memory, task demands, and feedback Laird et al. (1987) Anderson et al. (2004) Mnih et al. (2015) Vaswani et al. (2017) Yao et al. (2023).
Three conclusions follow from the existing research. First, successful performance should be attributed to the tested configuration, including any external memory or tools, and interpreted within its experimental boundaries. Second, reliable inference about cognitive processes requires convergent measures, adversarial alternatives, component intervention, held-out transfer, and reproducible documentation. Third, psychological terms such as memory, agency, and metacognition acquire scientific precision when their operational definitions and observed consequences are stated explicitly. Debates about subjective consciousness and personal identity remain distinct investigations requiring their own evidence Dehaene et al. (2017) Lake et al. (2017) Li et al. (2026).
The unresolved agenda centers on causal generalization, longitudinal continuity, measurement invariance, error detection, system-level reliability, and deployment under changing social and technical conditions. Artificial cognitive science will advance through experiments capable of explaining both achievements and failures, along with publication practices that preserve model provenance and the limits of each result. This dossier establishes an integrated critical account of the present literature and a methodological foundation for future research across the Artificial Era Ribeiro et al. (2020) Koh et al. (2021) Stanford HAI (2026).
References
- Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060. https://doi.org/10.1037/0033-295X.111.4.1036
- Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://papers.neurips.cc/paper_files/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
- Dehaene, S., Lau, H., & Kouider, S. (2017). What is consciousness, and could machines have it? Science, 358(6362), 486–492. https://doi.org/10.1126/science.aan8871
- Fatahi Bayat, F., Zhang, L., Munir, S., & Wang, L. (2025). FactBench: A dynamic benchmark for in-the-wild language model factuality evaluation. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 33090–33110. https://doi.org/10.18653/v1/2025.acl-long.1587
- Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron, 95(2), 245–258. https://doi.org/10.1016/j.neuron.2017.06.011
- Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2021). Measuring massive multitask language understanding. International Conference on Learning Representations. https://mlanthology.org/iclr/2021/hendrycks2021iclr-measuring/
- Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., et al. (2021). WILDS: A benchmark of in-the-wild distribution shifts. Proceedings of Machine Learning Research, 139, 5637–5664. https://proceedings.mlr.press/v139/koh21a.html
- Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987). SOAR: An architecture for general intelligence. Artificial Intelligence, 33(1), 1–64. https://doi.org/10.1016/0004-3702(87)90050-6
- Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, e253. https://doi.org/10.1017/S0140525X16001837
- Li, X., Xu, R., Zhang, T., Wang, Y., Chen, S., Lyu, Q., Zhang, B., Yang, P., Chen, K. X., Shi, H., Wang, Y., & Xu, W. (2026). AwarenessBench: Assessing cognitive capabilities of language models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2682–2741. https://doi.org/10.18653/v1/2026.acl-long.124
- Lin, S., Hilton, J., & Evans, O. (2022). TruthfulQA: Measuring how models mimic human falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 3214–3252. https://doi.org/10.18653/v1/2022.acl-long.229
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
- Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518, 529–533. https://doi.org/10.1038/nature14236
- Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, Article 2, 1–22. https://doi.org/10.1145/3586183.3606763
- Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., et al. (2021). Learning transferable visual models from natural language supervision. Proceedings of Machine Learning Research, 139, 8748–8763. https://proceedings.mlr.press/v139/radford21a.html
- Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025). HALoGEN: Fantastic LLM hallucinations and where to find them. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 1402–1425. https://doi.org/10.18653/v1/2025.acl-long.71
- Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020). Beyond accuracy: Behavioral testing of NLP models with CheckList. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4902–4912. https://doi.org/10.18653/v1/2020.acl-main.442
- Schaeffer, R., Miranda, B., & Koyejo, S. (2023). Are emergent abilities of large language models a mirage? Advances in Neural Information Processing Systems, 36. https://papers.neurips.cc/paper_files/paper/2023/hash/adc98a266f45005c403b8311ca7e8bd7-Abstract-Conference.html
- Shanahan, M. (2024). Talking about large language models. Communications of the ACM, 67(2), 68–79. https://doi.org/10.1145/3624724
- Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529, 484–489. https://doi.org/10.1038/nature16961
- Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., et al. (2017). Mastering the game of Go without human knowledge. Nature, 550, 354–359. https://doi.org/10.1038/nature24270
- Stanford Institute for Human-Centered Artificial Intelligence. (2026). Technical performance. The 2026 AI Index Report. Stanford University. https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://research.google/pubs/attention-is-all-you-need/
- Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35. https://proceedings.neurips.cc/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract.html
- Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations. https://mlanthology.org/iclr/2023/yao2023iclr-react/
Social Simulation and Artificial Interaction
Social cognition research concerns inference about agents, expectations, communication, coordination, and responses to social context. In artificial systems these capacities may be evaluated through dialogue tasks, simulated negotiations, collaborative planning, or repeated interactions with human participants. Park and colleagues' generative-agent system produced simulated interactions organized by stored experiences and higher-level plans. Such experiments show how conversational models can support complex social simulations, but the validity of the simulator as an account of actual human social behavior requires independent evidence Park et al. (2023).
Interpersonal performance involves multiple measurement levels. Linguistic plausibility concerns the quality of generated statements. Interactional coherence concerns continuity across exchanges. Task coordination concerns whether a shared objective is reached. Social validity concerns whether judgments or behavior correspond to independently observed human practices in a relevant population. A model that produces convincing role-play may still fail when participant motives, cultural norms, or consequential incentives differ from those encoded in the task Shanahan (2024) Bender et al. (2021).
The human contribution to an interaction is itself causal. Prompt wording, the user's expectations, willingness to correct errors, and the interface's disclosure of system boundaries affect observed outcomes. Researchers should record the full human–system loop and distinguish assistance from independent completion. These experiments belong to psychology of human–artificial relations as well as to artificial cognition, with interpretation sensitive to the people and institutional settings involved Ribeiro et al. (2020) Park et al. (2023).