RESEARCH DOSSIER
Human Error, Human Reliability, and Sociotechnical Safety
Abstract. Human error, human reliability, and sociotechnical safety concern how human performance and engineered work conditions interact to generate, detect, contain, or amplify system hazards. This fundamental research monograph critically synthesizes the historical formation of error theory, skill–rule–knowledge performance, slips and mistakes, organizational defenses, probabilistic human reliability assessment, high-reliability organizations, systems-theoretic accident models, incident classification, risk measurement, causal evaluation, and intervention evidence. Original research from aviation, industrial operations, and patient safety is assessed alongside methodological reviews and official technical guidance. The evidence supports conditional accounts of error opportunity, the influence of work-system design on measured errors, and the need to distinguish incident classification, modeled failure probability, safety-process measures, and actual harm. Central open problems include external validity, rare-event measurement, uncertainty calibration, organizational data quality, and the reliability of human–artificial decision systems. This critical integrative narrative synthesis reports no original experiment, registered systematic review, or independent meta-analysis.
Keywords: human error; human reliability; sociotechnical safety; human factors; accident causation; system safety; slips and mistakes; procedural violations; human reliability analysis; SPAR-H; CREAM; STAMP; FRAM; HFACS; safety culture; safety climate; risk assessment; patient safety; aviation safety; automation bias; human–artificial decision support; Artificial Era
Human Error, Reliability, and Sociotechnical Safety as a Research Object
Human error, human reliability, and sociotechnical safety form an integrated research object concerned with how human actions acquire consequences within organized technological work. A human error is an action, omission, judgment, or execution outcome that deviates from an intended or required course of action within a defined task context. The term gains scientific meaning only when the intended goal, available information, task constraints, and criterion of deviation are specified. Human reliability concerns the conditional likelihood and quality of successful human performance under particular circumstances, including opportunities for detecting and correcting deviations. Sociotechnical safety concerns the control of hazards through interacting people, technologies, procedures, organizational arrangements, and institutional constraints. The three concepts address distinct analytical levels of one system of performance (Reason, J (1990); Rasmussen, 199700052-0); Leveson, 200400047-X)).
An error may be detected without harm, intercepted by a colleague, transformed into a near miss, or contribute to injury after other defenses fail. Harm can also arise without any identifiable individual error when interacting components, incentives, and control arrangements produce hazardous states. Consequently, the operational distinction among error occurrence, error detection, accident sequence, and adverse outcome is fundamental. A defensible study defines exposure opportunities, expected performance, system state, and observation procedure before calculating an error rate. Reliable individuals working in poorly configured systems may produce hazardous outcomes, while well-designed systems can contain ordinary performance variability. Reason's account of human error and system defenses supplies a classic bridge between psychological analysis and safety engineering (Reason, J (2000)).
This monograph investigates the historical and conceptual formation of error research; cognitive mechanisms and taxonomies; the organizational production of risky conditions; human reliability assessment; incident investigation; causal and statistical inference; tested design interventions; cross-sector evidence; and the changing role of human operators when artificial systems participate in sensing, advising, and control. The method is a critical integrative narrative synthesis of original experimental and observational studies, scientific monographs, methodological reviews, and official technical guidance. No original participants, systematic-review protocol, pooled meta-analysis, or independent experimental results are claimed. Neighboring subjects such as general attention, automation allocation, detailed recovery engineering, and work-domain analysis receive cross-references while the present research object remains the emergence, distribution, and control of human contribution to sociotechnical risk.
Concepts and Levels of Explanation
A complete vocabulary distinguishes error, violation, failure, hazard, accident, resilience, and reliability. An error refers to a departure from an intended or correct course of activity, commonly without the actor's intention to depart. A violation refers to a departure from a rule or procedure that may be deliberate, though its motivation, organizational origin, risk, and moral status require independent investigation. A failure denotes the inability of a component, action, or system to fulfill a specified function. A hazard is a condition with the potential to produce loss; an accident is an event or sequence in which loss occurs. A near miss is a course of events in which a threatening condition or error is detected, contained, or interrupted before the specified loss. These meanings depend on a declared reference system, and cross-study comparability requires explicit coding rules (Reason, J (1990); Hollnagel, E (2004)).
The unit of explanation can be an individual cognitive process, a person–interface interaction, a team coordination episode, an organizational routine, a regulatory decision, or a whole control structure. Each unit supports different data and counterfactual questions. A memory lapse explains why a step was omitted, but a work-system account asks why that lapse was likely, why the omitted step was difficult to notice, and why the procedure lacked an effective recovery path. The levels must remain connected: cognitive mechanisms are real; their behavioral expression depends on conditions that designers and institutions can change. Rasmussen (1997)00052-0) argued that modern safety problems involve interacting constraints across legislators, managers, planners, and operators.
Reliability must likewise be qualified by its target. Repeating a routine action accurately, diagnosing an unfamiliar abnormal condition, coordinating a handoff, and stopping an unsafe process are different capabilities. A worker can have high routine accuracy and poor recovery capacity, or excellent adaptation under uncertainty and occasional deviations from prescriptive rules. Safety outcomes depend on event frequency, severity, dependencies among defenses, exposure duration, and whether the organization can learn and adapt. Aggregating these phenomena into a single percentage of human error without denominators, task definitions, or observation methods obscures rather than measures the underlying object.
Historical Formation of Human Error Research
The psychological foundations of human error research developed through studies of perception, attention, skilled action, choice, memory, and control in increasingly technical work settings. Industrial expansion, aviation, military operations, nuclear power, and health care confronted researchers with errors whose consequences depended on complex interactions. Early efforts commonly emphasized operator selection, training, simplified procedures, and component reliability. Later work showed that the same action could have radically different consequences according to interface feedback, procedural context, time pressure, and organizational design. The historical shift increased the importance of system-level mechanisms without eliminating the explanatory value of experimental cognitive psychology.
Jens Rasmussen, J (1983) distinguished skill-, rule-, and knowledge-based performance, linking different forms of action control to signals, signs, symbols, and representations. The classification emerged from the demands of advanced man–machine interfaces and became influential in analyzing errors in familiar routines, recognized situations, and unfamiliar problems. Reason, J (1990) brought several psychological literatures into an integrated account of action failures and the organizational conditions of accidents. In the same period, work on high-reliability organizations investigated how demanding operations could maintain collective attention and organizational coordination under consequential risk (LaPorte, T. R., & Consolini, P. M (1991)).
During the 1990s and early 2000s, Rasmussen (1997)00052-0), Reason, J (2000), Dekker (2002)00032-4), and Leveson (2004)00047-X) advanced different but overlapping accounts of systemic risk, situated human performance, and safety constraints. The Institute of Medicine's Institute of Medicine (2000) brought patient safety into prominent policy discussion, while aviation and nuclear organizations developed structured approaches to incident classification and probabilistic human reliability analysis. Contemporary research adds highly automated systems, complex supply chains, algorithmic decision support, and cyber-physical infrastructures. The resulting scientific tradition is cumulative and contested: taxonomies, event-chain models, constraint models, and adaptation accounts illuminate different phenomena and cannot be treated as fully interchangeable.
Cognitive Mechanisms of Action Failure
Errors occur through identifiable cognitive processes acting within a defined environment. Attentional capture can interrupt a planned sequence; a familiar cue can activate the wrong action; prospective memory can fail to reinstate an intention; time pressure can narrow search; and ambiguous interface feedback can reinforce an incorrect interpretation of system state. In skilled performance, action selection often depends on well-learned associations that are efficient under routine conditions. When competing actions share similar locations or gestures, interference can increase substitution and omission risk. The theoretical distinction between skill-, rule-, and knowledge-based activity provides a language for these modes of control (Rasmussen, J (1983)).
Complex tasks also require situation assessment: identifying which signals matter, integrating evidence, and predicting what a machine or work process will do next. Diagnostic error may arise even when perception and attention operate normally because the available cues are ambiguous or the causal model is incomplete. A rule can be applied accurately to the wrong situation when the operator misidentifies the current mode. A decision can be reasonable relative to locally available information and harmful relative to system information available elsewhere. Retrospective investigation should therefore reconstruct the information and constraints at the moment of choice, as Dekker (2002)00032-4) emphasizes.
Cognitive explanation becomes most useful when it predicts modifiable conditions. Error probability may be affected by signal discriminability, action mapping, display design, workload, interruptions, supervision, task scheduling, and organizational feedback. Simply assigning a cause such as inattention leaves unspecified which attentional demands, environmental triggers, or protective systems were present. Mechanistic studies can manipulate warnings, cue timing, choice ambiguity, interruption frequency, and recovery opportunities, then compare specific errors rather than relying on global self-report. Human error is thus a measurable behavioral phenomenon embedded in the design and organization of work.
Slips, Lapses, Mistakes, and Procedural Violations
An influential taxonomy separates failures of execution from failures of planning. Slips concern how an intended action is performed, including selecting an unintended control or action sequence. Lapses concern omissions linked to memory or attention, including failure to carry out a deferred intention. Mistakes concern problems in goal formulation, rule selection, situation interpretation, or the plan itself. Violations concern departures from rules that may be conscious and locally purposeful. These categories explain different intervention targets: better discriminability and action constraints may address slips; external reminders and handoff supports may address lapses; richer diagnostic information may address mistakes; and workable procedures with clear incentives and just supervision may address violations (Reason, J (1990)).
Classification requires evidence of intention, expected procedure, and situational knowledge. A prohibited shortcut may have become the only feasible way to achieve assigned production demands; a supposed mistake may be revealed as a response to misleading feedback; and two independent investigators can disagree about the actor's awareness. Procedural violations should therefore be coded descriptively, with separate evaluation of foreseeability, organizational pressure, and responsibility. A complete investigation documents how rules are created, communicated, revised, and reconciled with actual task demands rather than converting every departure from a written procedure into evidence of individual recklessness.
The categories are practical analytical tools, not natural kinds that automatically predict injury. A missed step can be intercepted before harm, whereas a correctly performed action under an unsafe design can produce loss. Categorization should report interrater agreement, support adjudication, and permit alternative codings when records are incomplete. Cohen, T. N., Wiegmann, D. A., & Shappell, S. A (2015) found that the reliability of a widely used accident-classification framework depended on training, sample size, and fidelity to the original taxonomy. That methodological lesson applies to error categories generally: apparent precision in labels is not equivalent to validated causal explanation.
Skilled Routine, Rule Use, and Knowledge-Based Diagnosis
Skill-based control supports rapid, largely practiced routines in settings where feedback arrives predictably and action–effect relations are learned. Rule-based control uses recognized patterns and procedures to select an action from known alternatives. Knowledge-based control constructs an interpretation of the situation and reasons through goals, causal possibilities, and uncertain consequences. Rasmussen, J (1983) developed these distinctions to explain performance in technical systems and the demands of interfaces supporting different levels of abstraction. An operator may move among levels repeatedly within the same event.
The distinctions support empirical hypotheses about what changes under disturbance. Training and repetition improve many routine components while shifting the locus of error toward wrong-mode selection, exception detection, and unusual combinations of familiar cues. Detailed procedures can improve recognition of known conditions but lose relevance when the system enters an unanticipated state. Expert performance may be especially valuable in detecting weak signals and recognizing when a nominal rule no longer applies. Experimental comparisons should therefore use scenarios that vary familiarity, ambiguity, display transparency, and time pressure, while collecting accuracy, confidence, search patterns, and time to recognition.
No one level is inherently safe. A skilled response can be exquisitely accurate and unsuitable for the current mode. A written rule may transmit important collective knowledge or preserve an obsolete assumption. Novel reasoning may support safe adaptation while imposing substantial workload and uncertainty. The design problem is to create reliable transitions between modes of performance through timely feedback, accessible constraints, and opportunities to verify hypotheses. Success should be evaluated in both routine operation and realistic exceptions rather than in a narrow laboratory task alone.
Attention, Workload, Fatigue, and Temporal Constraints
Attention and workload affect error through the timing and allocation of limited cognitive resources. Simultaneous demands compete for perception, memory, response selection, and monitoring, while interruptions force the operator to rebuild task state. In rare-event monitoring, long periods of uneventful observation may coexist with severe consequences for a missed signal. For applied safety research, a valid measure distinguishes the exposure to a demand from the operator's subjective burden and the observed performance outcome. High reported workload may coexist with successful performance because workers compensate through effort; low self-reported workload may coexist with hazardous under-monitoring.
Fatigue provides unusually direct evidence connecting a modifiable work-system condition with observable clinical errors. In a prospective randomized comparison in two intensive care units, Landrigan, C. P., Rothschild, J. M., Cronin, J. W., Kaushal (2004) found higher rates of serious errors by interns under schedules with extended shifts than under an intervention eliminating such shifts. The original report documents 136.0 versus 100.1 intern serious errors per 1,000 patient-days, while acknowledging that the trial was not powered to establish a corresponding reduction in preventable patient harm. A companion study examined changes in sleep and attentional failures under altered schedules (Lockley, S. W., Cronin, J. W., Evans, E. E., Cade, B. E., Le (2004)). These findings support schedule design as a causal intervention on performance within studied conditions; they do not justify assuming that every shorter-shift system will perform better, because handoffs and staffing also change.
Temporal constraints include circadian timing, duration of continuous duty, task-switching density, and available recovery time after an interruption. Field studies should distinguish chronic exposure from acute peaks and measure handoff quality when schedules are redesigned. A scientific account of fatigue recognizes both biological constraints and institutional decisions about staffing, rest opportunity, escalation, and workload distribution. The important practical inference is a design proposition that can be tested: change the work conditions, assess error rates with a defined denominator, and monitor for displaced risks elsewhere in the system.
Human Reliability as Conditional Performance
Human reliability analysis asks how human tasks contribute to the likelihood and consequences of defined system events. It begins with the task and operating context, identifies opportunities for successful or unsuccessful action, and estimates relevant conditional probabilities or performance envelopes. A human error probability is meaningful only with respect to a task description, time window, state of equipment, level of training, cues, dependencies, and assumptions about the operator's opportunity to recover. An aggregate label such as human reliability is insufficient for safety decisions unless these conditions are specified.
Probabilistic safety assessment traditionally places human actions within event trees, fault trees, or related models, estimating how an omitted intervention, incorrect diagnosis, or delayed action could affect hazardous sequences. The U.S. Nuclear Regulatory Commission's Gertman, D., Blackman, H., Marble, J., Byers, J., & Smith, C (2005) formalized assessment using performance-shaping factors and dependency judgments. Its official documentation discusses uncertainty representation and dependence among human actions. A number produced by such a method is an estimate conditional on a model, analyst judgments, and task assumptions, rather than a directly observed universal probability that a person will fail.
Other approaches assign greater explanatory weight to cognitive context. Hollnagel, E (1998) integrates individual, organizational, and technical conditions into retrospective and predictive analysis of cognitive reliability. The contrast is scientifically productive: methods vary in how they represent task sequence, cognitive control, context, and uncertainty. Evaluation should ask which assumptions are transparent, how inputs were elicited, whether analysts agree, how results change under plausible alternatives, and whether prospective predictions are calibrated against representative events. Where severe accidents are scarce, sensitivity analysis and explicit uncertainty are particularly important.
The Organizational Production of Error Opportunities
Organizations distribute authority, resources, time, and information. Their staffing rules determine who receives alerts; their procurement decisions determine which controls are distinguishable; their targets determine how competing production and safety objectives are balanced. A localized error can therefore arise within conditions shaped long before the operator encounters a task. Reason, J (2000) described latent organizational conditions and layered defenses, emphasizing that defenses can contain gaps that align under particular circumstances. The model remains useful as a diagnostic heuristic when investigators specify the actual mechanisms by which defenses were weakened or bypassed.
A sociotechnical explanation needs more than an organizational label. It reconstructs how workload, supervision, procedure feasibility, resource allocation, interdepartmental coordination, and feedback produced specific exposure pathways. Rasmussen (1997)00052-0) treated risk management as a multilevel control problem in dynamically changing environments. Competitive pressures and technological change can shift practical work toward boundaries of acceptable performance; the phenomenon is best studied through longitudinal evidence on decisions, operations, and deviations from planned constraints. Blaming culture, incentives, or management in the abstract does not replace causal reconstruction.
Organizational conditions may also protect against error. Teams cross-check, operators report weak signals, supervisors allocate relief, and system owners change equipment after repeated close calls. The relevant mechanism is whether information about vulnerability travels to people with authority to respond and whether changes alter measured outcomes. This connects individual cognition to organizational learning without claiming that every observed association is causal. A research program should test specific organizational changes under comparable workloads and follow the full incident pathway from exposure through recovery to harm.
High-Reliability Organizations and Collective Alertness
High-reliability organization research asks how some high-consequence operations maintain dependable performance in demanding environments. LaPorte, T. R., & Consolini, P. M (1991) examined theoretical challenges posed by organizations with hazardous missions and sustained operational discipline. The broader tradition investigates attention to small failures, sensitivity to operational conditions, coordinated expertise, training, and preparation for abnormal states. Such practices are candidate mechanisms rather than a guarantee of immunity from accidents.
The term high reliability requires a comparison class, exposure metrics, and a defensible time horizon. An organization may report few accidents because it has few opportunities for hazardous exposure, because accidents are genuinely prevented, or because reporting and detection are weak. A low observed count alone cannot establish exceptional reliability. Research should measure near misses, detection capability, incident severity, changing risk profiles, and the organization's response to warnings. Independent observation and event-based audits can complement managerial narratives of reliability.
A high-reliability perspective is especially helpful for analyzing the practical coordination of dispersed expertise. Technicians may understand a weak signal that executives cannot interpret; operators may detect a mismatch between a new procedure and a changed environment; supervisors may have authority to suspend activity. Safety is strengthened when these observations influence decisions before damage occurs. The explanatory claim concerns feedback, escalation, and authority under specific conditions, and its evaluation requires direct evidence of those processes rather than rhetorical use of organizational mindfulness.
Accident Pathways, Defense Layers, and Causal Sequences
Accidents emerge through pathways connecting initiating conditions, operational activity, defenses, and loss. A sequence representation can reveal the time ordering of events, available intervention points, and consequences of a failed barrier. Reason, J (2000) popularized the image of multiple defenses containing imperfect protection, while emphasizing that frontline actions operate within conditions produced by organizational decisions. The layered account is most informative when the investigator identifies each concrete barrier, its function, the mechanism of degradation, and the circumstances in which other safeguards could have interrupted the sequence.
A causal chain can become misleading if it is treated as the only possible path or if the analyst begins with the injury and selects facts that make it appear inevitable. The same conditions can produce a routine successful operation on most days and a rare accident under an unusual conjunction. Counterfactual reasoning should therefore ask which specific change would have prevented loss under plausible alternatives, what evidence supports that possibility, and whether the proposed change would introduce hazards elsewhere. A person making the final visible action is not thereby established as the dominant causal factor. Dekker (2002)00032-4) argues for reconstructing decisions from the operator's local perspective before describing them as errors.
Sequential accident models are useful for analyzing known event orders, but their limits increase as interactions become nonlinear or hazards arise from controllers enforcing insufficient constraints. Safety science therefore uses several complementary models. An incident analysis may combine a detailed event timeline, barrier assessment, task analysis, organizational decision history, and a control-structure model. The choice of model should be justified by the system's actual coupling, feedback, and adaptive behavior, not by methodological fashion.
Normal Accidents, Complexity, and Coupling
Charles Perrow, C (1999) developed a structural approach to technological risk, emphasizing interaction complexity and the tightness of coupling. Complex interactions make the consequences of component changes difficult to anticipate; tight coupling leaves little time or flexibility to interrupt a propagating disturbance. The conceptual point is that normal operation may contain combinations of dependencies whose hazardous effects become visible only under unusual circumstances. The model thereby raises questions about design, redundancy, controllability, and organizational choice of technology.
The theory should not be interpreted as a deterministic prediction that every complex system must suffer a catastrophic accident at a particular rate. Empirical comparison requires definitions of complexity and coupling, exposure measures, and careful selection of systems. A loosely coupled system may still permit major harm through weak supervision, while a tightly coupled system can contain engineered controls and rapid protective actions. Multiple safeguards can improve containment yet also introduce coordination burdens or unexpected interactions. Those propositions invite comparative designs and simulation rather than universal conclusions drawn from a small number of iconic disasters.
The useful synthesis is a structured warning about the limits of local optimization. A component can perform exactly as specified and still participate in system-level loss when its interaction with another component violates an unstated assumption. Safety assessment must consequently inspect the relationships among components, the timing of feedback, and the authority to reconfigure or stop operations. Human reliability analysis should represent what information an operator can actually obtain when the system enters a rapidly changing state.
Control Structures and the STAMP Accident Model
Nancy Leveson's systems-theoretic accident model (2004)00047-X) treats safety as a control problem. Hazards can occur when control actions are missing, incorrect, mistimed, applied for an inappropriate duration, or based on an inadequate representation of the controlled process. The model directs investigation toward safety constraints, feedback channels, hierarchical control relationships, and the organizational conditions that influence enforcement. It is especially relevant to software-intensive systems in which all physical components may function as specified while their coordination produces unsafe states.
For empirical application, the researcher first identifies unacceptable losses, system hazards, and safety constraints, then maps controllers, actuators, sensors, process models, and feedback. An unsafe control action may originate in a human decision, an automated policy, a delayed sensor, a misunderstood procedure, or a supervisory instruction that conflicts with local conditions. The analytical question is how the controller came to produce or permit a hazardous state. This framing requires explicit documentation of assumptions about the controller's knowledge and responsibilities, reducing the temptation to use human error as an unexplained terminal cause.
Systems-theoretic analysis supplies counterfactual intervention targets, but its practical effectiveness must still be validated. A detailed model may identify many plausible hazards, while studies of prospective accuracy, analyst agreement, cost, and resulting incident reduction are needed to establish whether a particular application improves safety. Comparative evaluation should include conventional hazard analysis methods as well as systems-theoretic alternatives. Model richness is a form of explanatory coverage; actual preventive effectiveness is a separate outcome.
Functional Variability and the FRAM Perspective
Erik Hollnagel, E (2012) models how functions in complex sociotechnical systems depend on one another and how variability in routine performance can combine to produce outcomes. Everyday adaptation can accommodate changing circumstances while also changing the timing, precision, and availability of work products. This perspective focuses attention on how work is actually performed rather than solely on deviations from a nominal procedure. It is particularly relevant where performance continuously adjusts to demand and resource constraints.
The methodological process requires identification of system functions, their relevant aspects, variability sources, and couplings. Different analysts may select different functional boundaries, so a transparent study should state its modeling decisions and demonstrate how observations, interviews, records, or simulations justify those decisions. Analysts can then examine how a variation in one function affects the prerequisites and outputs of others. The result is a model of possible interactions, which can inform the design of buffers, monitoring, and adaptive capacity.
Functional analysis complements experimental tests and incident investigations. It can make practical coordination visible, but a retrospective fit between a model and a known event is not sufficient to establish predictive validity. The strongest use combines grounded observation of everyday work, explicit alternative explanations, and prospective tests of proposed changes. For this dossier, FRAM illuminates the pathways through which ordinary adjustments affect safety; the separate research object of resilience engineering examines recovery and successful adjustment in greater detail.
Incident Investigation and the Problem of Hindsight
An incident investigation reconstructs a sequence under severe information asymmetry: investigators know the outcome, while the original participants acted without that knowledge. Hindsight can make ambiguous cues seem obvious and obscure why a mistaken interpretation was plausible at the time. Dekker (2002)00032-4) argued that investigators should recover the operator's changing local picture, competing goals, and available evidence. This demands records of what instruments displayed, what messages were communicated, when decisions occurred, and which alternatives were genuinely available.
Effective interviews invite a detailed reconstruction of work as experienced, while independent documents constrain memory and retrospective reconstruction. The investigator should preserve timestamps, alarm records, procedure versions, staffing decisions, and equipment configuration. Event chronologies may need multiple parallel tracks for personnel, automation, management, and environmental changes. A preliminary classification of a person's behavior should remain revisable as the evidence develops. Interviews conducted in punitive conditions can distort disclosure; confidentiality, procedural fairness, and accountability policies shape the quality of incident data.
The final explanation must distinguish observation from inference. A damaged component, an alarm acknowledged late, and an uncompleted checklist are observations or recorded outcomes; the claim that distraction or risk-taking caused them requires supporting evidence. The most defensible recommendations address demonstrable mechanisms, identify how a change could prevent recurrence, and specify follow-up measures. Organizational learning becomes testable only when proposed countermeasures are implemented and monitored, rather than when an investigation ends with a list of causes.
Human Factors Analysis and Classification System
The Human Factors Analysis and Classification System, or HFACS, uses hierarchical categories to describe unsafe acts, preconditions, supervision, and organizational influences. Wiegmann, D. A., & Shappell, S. A (2001) applied an early version to commercial aviation accident records; Shappell, S., Detwiler, C., Holcomb, K., Hackworth, C., Boqu (2007) examined human factors in different commercial aviation operations. Its organizing logic reflects the effort to connect observable actions with broader conditions rather than treating pilot behavior as a complete explanation.
HFACS can support large-scale coding of accident narratives, detection of recurring categories, and comparisons across settings. The classification's scientific status depends on whether independent raters can apply categories consistently and whether coded categories predict or explain actionable system conditions. Cohen, T. N., Wiegmann, D. A., & Shappell, S. A (2015) systematically reviewed 14 peer-reviewed reliability studies and reported that training, sample size, and fidelity to the original framework affected interrater and intrarater reliability. That evidence supports disciplined use of the classification while cautioning against treating the category itself as a verified causal mechanism.
Two types of validity warrant separation. Descriptive validity concerns whether the coding faithfully captures documented factors; causal validity concerns whether changing a coded factor would change accident risk. High rater agreement cannot by itself establish causal validity. Severe incidents may also be selectively investigated and unevenly documented, introducing selection bias into coded datasets. Research programs should preregister coding rules when feasible, report agreement and missing information, and test whether proposed interventions on organizational categories are associated with measurable improvements.
Safety Barriers, Defenses, and Failure Containment
Safety barriers are engineered, procedural, organizational, or social arrangements designed to prevent a hazard, detect its emergence, limit its progression, or reduce loss. Hollnagel, E (2004) systematized analysis of barriers as a core safety-engineering problem. Examples include interlocks, redundant confirmation, physical separation, independent verification, escalation authority, standardized handoffs, and emergency shutdown mechanisms. Their effectiveness depends on both the technical implementation and the capacity of people to recognize and use them.
Barrier analysis asks whether a safeguard is available at the relevant moment, independent of the initiating failure, understandable to its users, and robust against foreseeable common causes. Two nominally redundant checks can fail together if both rely on the same corrupted data or if both are bypassed under identical workload pressure. Likewise, adding alerts can overload the channel through which the most important warnings must travel. Safety design therefore requires examination of the complete feedback environment, the distribution of authority, and the possibility of common-mode failures.
A barrier's presence is a process measure, not proof of reduced injury. Evaluation should document opportunities to trigger the barrier, actual triggering, successful interception, false-positive burden, maintenance, and unintended behavior changes. In high-risk systems, rare loss events can make outcome-only evaluation statistically inefficient; process measures and simulations may offer useful intermediate evidence when their connection to harm is established. The aim is a defensible chain from design change through altered performance to reduced risk.
Safety Climate, Organizational Behavior, and Reporting
Safety climate denotes workers' perceptions of the priority assigned to safety in an organizational setting. It is typically measured through questionnaires addressing practices, supervisory signals, and shared expectations, with attention to whether individual perceptions may legitimately be aggregated to the team or organizational level. Climate has been investigated as a predictor of compliance, participation, incident reporting, and injury. A scientific analysis separates an attitude measure from observed behavior and from accidents, which have distinct determinants and measurement biases.
The meta-analysis by Clarke, S (2006) found associations between safety climate and employee safety behaviors, with weaker links to accident involvement and design-related differences in generalizability. A broader synthesis by Christian, M. S., Bradley, J. C., Wallace, J. C., & Burke, M (2009) integrated person- and situation-based antecedents of safety performance and injury outcomes. These analyses support treating organizational context as relevant to behavior, while their pooled observational evidence does not identify a unique causal pathway from a specific management program to harm reduction.
Reporting systems create a second measurement challenge. An increase in reported near misses may reflect higher event frequency, improved detection, greater trust, or several changes together. Therefore, raw report volume is an ambiguous indicator of safety. Researchers should triangulate reports with exposure denominators, independent audits, injury surveillance, and changes in the severity of events. Fair response to reporting must coexist with investigation of deliberate disregard of known safety requirements. The underlying research question is whether reporting creates reliable feedback that changes work conditions, prevents recurrence, and improves detection of emerging hazards.
Team Coordination, Handoffs, and Distributed Responsibility
Safety-critical work often involves teams whose members hold different fragments of information and authority. Coordination failures can occur when a handoff omits an unresolved hazard, when two people assume the other will verify an action, or when a specialist's warning fails to reach the decision maker. Team reliability depends on shared goals, communication channels, role allocation, monitoring, and the management of disagreement. Salas, E., Shuffler, M. L., Thayer, A. L., Bedwell, W. L., & (2015) integrated research on teamwork in organizations and emphasized the conditions under which team processes can be trained and supported.
A handoff is an observable information-transfer episode with multiple opportunities for error. Its quality can be assessed through completeness and accuracy of transferred state, acknowledgement, follow-up actions, and subsequent performance. Interventions may include standardized communication structures, closed-loop confirmation, protected overlap time, and clear responsibility at transition points. However, a longer or more detailed handoff can impose its own time cost and create overload. The correct target is decision-relevant information transfer, with evaluation under routine and interrupted conditions.
Team safety also requires authority to challenge decisions and to stop work when uncertainty exceeds acceptable limits. These capacities depend on status relationships, staffing, workload, and institutional support, and they should be measured behaviorally rather than inferred from generic cultural claims. When automation is added, the coordination problem expands: operators must understand what information the system has processed, what actions it has initiated, and who remains accountable for recognizing a discrepancy. The relevant unit becomes a human–technical work configuration rather than an isolated employee.
Patient Safety as an Empirical Sociotechnical Laboratory
Health care offers a large body of evidence on human error under time pressure, uncertain diagnosis, complex coordination, and consequential outcomes. The Institute of Medicine's Institute of Medicine (2000) established an influential program for system-focused patient safety, emphasizing leadership, reporting, and design. The historical mortality estimates in that report came from particular earlier hospital studies and should be presented as estimates within their original definitions and period, not as a current universal measure of medical error. Contemporary safety science distinguishes a medical error, preventable adverse event, and harm independently of preventability.
Intervention evidence is heterogeneous. In a comparative study, Bates, D. W., Leape, L. L., Cullen, D. J., Laird, N., Peters (1998) evaluated computerized physician order entry and a team intervention in hospital units, illustrating how prescribing workflows can be redesigned rather than relying entirely on individual vigilance. In a multicenter cohort study, Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Ch (2006) reported a sustained reduction in catheter-related bloodstream infection rates after an evidence-based collaborative program; the design combined technical and organizational components and was observational rather than a randomized isolation of a checklist effect. In an international before-and-after study, Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R (2009) investigated a surgical safety checklist and reported favorable postoperative outcomes, with the familiar limitation that contemporaneous changes may affect before-and-after comparisons.
Patient safety also demonstrates why successful local interventions cannot justify assumptions of systemwide improvement. Landrigan, C. P., Parry, G. J., Bones, C. B., Hackbarth, A (2010) reviewed medical harms over six years in ten North Carolina hospitals and found no significant overall decline in harm rates in that sample using their specified methods. This finding addresses a different question and population from a focused checklist or duty-hour trial. Scientific synthesis therefore distinguishes individual interventions, their study designs, implementation fidelity, target events, organizational context, and population-level trends. The World Health Organization's World Health Organization (2021) provides an international policy framework; its recommendations are policy guidance, while intervention efficacy requires separate empirical study.
Aviation, Industrial Operations, and Cross-Sector Transfer
Aviation studies combine highly standardized procedures with varying degrees of uncertainty, automation, and dispersed expertise. Sarter, N. B., & Woods, D. D (1997) examined operational experiences with a sophisticated aircraft automation system and identified mode awareness as a source of automation surprises. Wiegmann, D. A., & Shappell, S. A (2001) demonstrated the application of HFACS to historical accident records, while subsequent work used broader commercial datasets (Shappell, S., Detwiler, C., Holcomb, K., Hackworth, C., Boqu (2007)). These studies show that operator–machine interaction, supervision, and organizational arrangements can be analyzed with common conceptual tools.
Industrial and energy systems contribute methods for prospective hazard analysis and human reliability modeling. The Gertman, D., Blackman, H., Marble, J., Byers, J., & Smith, C (2005) documents the assumptions and performance-shaping factors of one regulatory approach. Leveson's STAMP model (2004)00047-X) provides a different control-theoretic representation. Research transfer among aviation, nuclear power, manufacturing, and health care should specify which mechanisms are shared—such as feedback delays, misidentified modes, incomplete handoffs, and weak safety constraints—and which are sector-specific, including regulation, exposure frequency, staffing, and possible losses.
A cross-sector comparison should never infer that a successful intervention is universally effective simply because the phrase human factors applies to each field. Tasks differ in reversibility, observability, temporal coupling, and availability of independent safeguards. The appropriate translational program identifies a mechanism, adapts its implementation, measures its proximal effects, and then evaluates safety outcomes in the target environment. This principle makes interdisciplinary learning rigorous while preserving the contextual specificity of reliability.
Designing Measures of Error and Reliability
Measurement begins by specifying the opportunity for an action, the expected result, the observation window, and the operational environment. An error rate can be defined as the number of observed deviations divided by the number of relevant opportunities, but this quotient is interpretable only when opportunities are comparable. A medication omission per administered dose, a late acknowledgement per critical alarm, and a missed hazard per monitored hour describe distinct mechanisms. In environments where several errors can occur within one task, the numerator must specify whether it counts episodes, error types, affected tasks, or individual actions. Severity, recoverability, and duration should be measured separately from frequency.
Reliability studies also need to distinguish individual variability from variability caused by tasks and systems. A worker performing a familiar procedure in a well-maintained environment has a different error opportunity than the same worker facing damaged feedback, rapidly changing requirements, and competing instructions. Risk adjustment can account for observable difficulty, while multilevel models can separate person, team, shift, and organization contributions. Still, risk adjustment only addresses measured confounding; differences in reporting and case mix can remain. Human reliability estimates should therefore communicate uncertainty, describe the task population, and avoid transporting numerical values to settings with different constraints without validation (Gertman, D., Blackman, H., Marble, J., Byers, J., & Smith, C (2005)).
The outcome structure matters. A single error may be harmless because of an independent check; a rare error may be highly consequential because it defeats several defenses. For evaluative research, useful endpoints include task error, delayed detection, successful interception, near miss, adverse event, severity-weighted harm, and time to safe containment. Reporting should provide both absolute event counts and exposure denominators, and should document the ascertainment procedure. The most persuasive studies additionally examine whether the proposed mechanism changed when the intervention was introduced. This distinguishes an intervention that makes an error less likely from one that makes its consequences easier to detect or contain.
Human Reliability Assessment and Uncertainty
Human reliability assessment converts a specified human task into a model of possible actions and failures, often as one component of probabilistic risk assessment. Methods differ in assumptions about the decomposition of tasks, performance-shaping factors, dependency, and diagnostic reasoning. The Gertman, D., Blackman, H., Marble, J., Byers, J., & Smith, C (2005) represents a structured technique for considering factors such as time, complexity, stressors, procedures, training, and task dependence. Its estimates support sensitivity analysis within a defined plant and scenario model. A numerical output carries the epistemic limitations of the model and expert inputs, particularly where independent field observations of rare human failures are sparse.
Uncertainty has several sources. Aleatory variation concerns the occurrence of events under specified repeated conditions. Epistemic uncertainty concerns gaps in knowledge about system structure, dependencies, and parameter values. Model-form uncertainty arises when alternative representations would generate different estimates. Analyst variability arises when practitioners interpret task boundaries or performance-shaping factors differently. These uncertainties are important because a narrowly presented point estimate may conceal wide plausible ranges and a false appearance of precision. Transparent assessments document assumptions, elicitation procedures, alternative scenarios, and sensitivity of results to disputed inputs.
Validation requires comparing predictions with evidence appropriate to the model's claims. In an experiment, repeated performance may supply relative error rates for controlled conditions, but a simulated task may omit organizational constraints. In an incident database, the relevant failures may be severe enough to enter the record, making the observed sample unrepresentative of all task opportunities. Rare hazards may not provide enough events to estimate probabilities precisely. A robust research design uses multiple evidence streams, examines dependencies, and reports when prediction cannot be directly calibrated. The value of an assessment is then its usefulness for identifying critical assumptions and preventive priorities, rather than the sheer number of decimals attached to a probability.
Experimental Simulation, Task Analysis, and Field Observation
Laboratory and simulator studies allow researchers to manipulate error opportunities while holding selected task conditions constant. A useful study can vary cue discriminability, procedural complexity, automation reliability, interruption rate, time pressure, or distribution of decision authority, and measure the consequences for action accuracy, diagnosis time, recovery, and workload. Random assignment strengthens causal inference for the manipulation under study. Ecological validity depends on whether the simulation recreates the feedback, incentives, expertise, task dependencies, and stakes that matter in the target environment. A controlled result should be transported only to conditions sharing the relevant mechanism.
Task analysis provides a disciplined representation of the actual sequence and purpose of activities. Researchers may combine observation, interviews, logs, walkthroughs, and cognitive probes to identify decisions and information requirements. Work-as-designed procedures can diverge from work-as-performed routines because of workload, legacy equipment, informal coordination, and unpredictable exceptions. Direct observation is essential to recognize these differences, while interviews contribute information about goals and unrecorded constraints. A complete task analysis documents disagreement and context variation rather than treating one expert account as a universal description.
Field studies contribute the organizational and temporal dimensions that a simulator often omits. Researchers can assess before-and-after modifications of interfaces, compare work units, or exploit plausible exogenous variation in work schedules or technology introduction. Interrupted time-series and controlled designs can reduce some confounding when implemented carefully. However, changes in case mix, staffing, reporting, or equipment can mimic improvements. Evidence quality increases when investigators prespecify primary outcomes, collect baseline data, identify comparison groups, and monitor unintended consequences over an adequate horizon. The design should follow the causal claim: a claim about changed behavior requires direct behavioral measurement, and a claim about reduced harm requires harm-sensitive outcomes.
Causal Inference, Selection Bias, and the Denominator Problem
Safety datasets are often shaped by selective observation. Severe accidents attract detailed investigation, while many successful recoveries and routine deviations remain invisible. Reports may overrepresent events that an employee can recognize or feels safe disclosing. Comparing the proportions of error categories across these reports can thus reveal the classification of documented cases without revealing population risk. Similarly, a higher reporting rate may coexist with better safety because the organization detects more events. A sound analysis models the reporting pathway as part of data generation.
Causal inference asks what would have happened under a specified alternative: an improved display, a revised procedure, a shorter shift, a redundant check, or a different automation policy. Randomized studies answer some such questions; quasi-experimental designs can address others where randomization is infeasible. Interrupted time series should examine preexisting trends and simultaneous interventions, while matched comparisons depend on measured covariate sufficiency. Incident reconstructions provide valuable mechanistic evidence but ordinarily lack direct observations of the counterfactual. A complete case-based explanation acknowledges that limitation and uses triangulation rather than presenting a narrative as an experimentally established effect.
The denominator problem becomes especially important when incident counts are small. The number of adverse events can fall when exposure declines, and it can rise when a hospital treats more complex patients or an industrial plant increases output. Rates require a relevant exposure measure, sometimes multiple measures when risk accumulates differently across tasks. Confidence intervals or compatible uncertainty estimates should accompany comparisons when the data permit. Analysts should attend to repeated events, hierarchical clustering, changing definitions, and missing information. Landrigan, C. P., Parry, G. J., Bones, C. B., Hackbarth, A (2010) illustrates the challenge of tracking medical harm longitudinally with a consistent detection methodology.
Checklists, Standardization, and Safety-Critical Procedures
Checklists externalize prospective memory, support coordination, and make critical steps visible across team members. Their mechanisms can include reminding, requiring independent confirmation, structuring communication, or stopping an action until prerequisites are satisfied. Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R (2009) documented improvements in selected postoperative outcomes after introducing a surgical checklist in diverse hospitals. Its before-and-after design and bundled implementation mean that the measured outcome cannot be attributed uniquely to the checklist's written items. The finding supports empirical evaluation of structured team interventions, not universal effectiveness of every checklist.
Design quality depends on usability. A checklist that interrupts critical attention, contains obsolete actions, conflicts with real workflows, or duplicates another control may be bypassed or followed mechanically. The literature on safety systems suggests that procedure design should begin with field observation of actual work and a hazard-based account of what must be reliably checked. Researchers should evaluate completion accuracy, detection of latent errors, staff engagement, time cost, and the effect on downstream outcomes. Compliance alone can give a misleading impression if personnel tick items without performing meaningful verification.
Standardization is valuable when tasks are sufficiently stable for prescriptive coordination; adaptive judgment becomes necessary when the situation falls outside the rule's assumptions. A well-designed procedure therefore specifies its applicability and includes escalation conditions. The scientific question is whether the formal sequence improves performance in the relevant conditions while preserving opportunities to recognize exceptions. This is a testable human-factors hypothesis, not an ideological preference for either autonomy or strict control.
Training, Simulation, and Competence Over Time
Training can improve recognition of hazards, familiar procedural execution, communication, and practiced response to abnormal conditions. Its causal effect depends on what is learned and where the learned capability is tested. A course can improve knowledge scores without reducing errors in real operation; a simulator can improve performance on a familiar scenario without preparing operators for an unfamiliar combination of failures. The relevant transfer outcomes are accurate decisions, timely detection, coordinated action, and successful containment under representative conditions. Follow-up tests are necessary because knowledge, routines, and situational competence may change over time.
Scenario-based training is particularly suited to coordination under uncertainty. Participants can practice making assumptions explicit, escalating disagreement, recognizing mode confusion, and responding when standard procedures do not fit the current state. But exercise scripts and facilitator cues can reveal the intended answer and overstate transfer. Evaluations can randomize scenarios, include unseen exceptions, and use blinded behavioral assessment when practicable. Team training should measure communication and decision processes rather than inferring safety competence from attendance alone. The synthesis of teamwork evidence by Salas, E., Shuffler, M. L., Thayer, A. L., Bedwell, W. L., & (2015) supplies a broader organizational frame for these mechanisms.
Training is one intervention among several, and its success cannot compensate for permanently ambiguous controls or impossible staffing demands. If the system regularly generates misleading information, asking employees to remember a workaround places continuous cognitive burden on frontline performance. A comparative evaluation should therefore measure the effectiveness and durability of redesign, procedural support, training, and organizational changes. Human reliability rises through the coordinated provision of capability and conditions in which that capability can operate.
Work-System Design, SEIPS, and Sociotechnical Intervention
The Systems Engineering Initiative for Patient Safety (SEIPS) model organizes work-system elements—including people, tasks, tools and technologies, organizational arrangements, and physical environment—around work processes and outcomes. Carayon, P., Schoofs Hundt, A., Karsh, B.-T., Gurses, A. P (2006) described how this framework can inform analysis and redesign in health care. The framework makes explicit that a change to one element, such as installing decision-support software, may affect task distribution, communication, timing, and the conditions under which people can safely exercise professional judgment.
The scientific value of a work-system model depends on how its elements are operationalized and tested. A study might observe that medication-order errors cluster during interruptions, then test redesigned order-entry screens, protected workflow periods, or role adjustments. The intervention may change several system elements simultaneously. A good evaluation measures those changes, anticipates compensatory behaviors, and distinguishes immediate process gains from patient outcomes. SEIPS supports a coherent account of interdependence but does not, on its own, constitute causal proof that any selected intervention will work.
Work-system design also must address equity. Interfaces and procedures are often calibrated to the capabilities of a narrow workforce and may impose disproportionate burdens on people with different sensory, motor, language, or experience profiles. Reliability research should investigate the distribution of opportunities and risks across relevant subgroups, documenting accessible alternatives and the trade-offs of adaptation. The systems perspective then becomes an empirical program for identifying whose performance conditions are measured and whose are overlooked.
Automation Bias, Human Oversight, and Authority Boundaries
Automation may support monitoring, diagnosis, decision selection, and action implementation while changing human activity at each stage. Parasuraman, R., Sheridan, T. B., & Wickens, C. D (2000) distinguished four broad classes of functions, providing a framework for selecting levels of automation according to human performance consequences and other constraints. An automated system may improve mean accuracy in routine cases while increasing the difficulty of detecting rare failures or maintaining competence when a human must resume control. Reliability evaluation therefore includes both automated performance and the quality of human oversight under mismatches.
Automation bias describes a pattern in which people accept or rely on automated advice despite relevant contradictory evidence, or fail to act when an automated system fails to alert. Its occurrence depends on task conditions, user expertise, reliability, presentation, and competing attention demands. A meaningful assessment compares decisions under correct, incorrect, and absent advice; measures sensitivity to errors; and examines whether confidence, explanations, and user feedback improve calibrated reliance. The research must distinguish over-reliance from appropriate reliance on a demonstrably better source. An apparent human error after accepting an algorithmic recommendation may reflect a design that concealed uncertainty or restricted access to the underlying evidence.
Authority boundaries determine who can veto, override, escalate, or review an automated action. “Human in the loop” is an insufficient safety specification unless the human has timely, understandable information and a feasible intervention path. For high-consequence systems, control arrangements should be tested under degraded sensor quality, distribution shifts, alarms, mode transitions, and unforeseen cases. Where learning systems change over time, reproducible logs and version information become part of the evidence. This problem links traditional human factors to emerging Artificial Era research without conflating the scientific measurement of decision reliability with philosophical claims about machine identity.
Human–Artificial Decision Support and the Artificial Era
Artificial decision systems introduce new error pathways through model uncertainty, data shift, opaque confidence indicators, tool invocation, and the delegation of consequential actions. Human operators may become supervisors of outputs they cannot feasibly verify within the available time. The relevant reliability unit is consequently a joint system of people, artificial processes, interfaces, data, institutional rules, and external effects. Parasuraman, R., Sheridan, T. B., & Wickens, C. D (2000) supplies a general automation framework; Leveson (2004)00047-X) supplies a hazard-control framing suitable for identifying unsafe actions and feedback failures. These frameworks predate contemporary generative systems, so their application to new models constitutes a research extension requiring empirical validation.
Evaluation should disaggregate the accuracy of a model's output, a human's acceptance decision, the joint team's completed task, and the safety of the external action. A high-performing model can reduce safety when it invites unwarranted confidence in rare cases; a model with imperfect average performance can still help by identifying uncertainty and prompting checks. Counterfactual experiments should compare human-only, artificial-only, and joint conditions for defined tasks, including adversarial or shifted cases, while documenting workload, agency, and time to detect mismatches. Reliance calibration and recovery capacity may be as important as average answer accuracy.
In the broader historical framework of the Artificial Era, human reliability research investigates the organization of safety when a nonbiological system participates in the production of consequential decisions. The philosophical horizon From Homo to Artificial can generate questions about responsibility, distributed control, and continuity of institutional oversight. These questions become scientifically tractable through specified system roles, observable interventions, defined loss functions, and independently testable hypotheses. The empirical literature does not establish the personhood or subjective status of any particular artificial system through accident statistics; those are separate theoretical and philosophical inquiries.
Measurement Validity, Generalization, and Conflicting Findings
The literature supports several stable conclusions at different levels of certainty. Psychological studies identify plausible mechanisms of execution failure and mistaken situation interpretation. Intervention studies demonstrate that changing schedules, workflows, and team practices can modify measured error outcomes in studied environments. Observational syntheses associate organizational safety climate with safety behaviors. Models such as Reason's layered defenses, Rasmussen's dynamic risk framework, and Leveson's STAMP provide complementary representations of system pathways. Each of these statements involves a different type of evidence, and their evidential strength cannot be collapsed into a single claim of scientific consensus.
Disagreements arise when researchers infer causation from accident reconstruction, generalize interventions across industries, or treat classifications as validated predictors. A retrospective case may be well explained by one framework and reasonably represented by another; comparative predictive superiority requires separate tests. A safety climate survey can capture meaningful shared expectations while failing to measure actual exposure, work design, or reporting suppression. HRA estimates may be useful for prioritizing scenarios even when their absolute probabilities have broad uncertainty. A careful synthesis states which empirical relationship or practical decision is supported by each method.
Replication is especially difficult where serious incidents are rare and organizational settings are unique. Proxy outcomes permit more statistical power but require validation against harm or critical-event mechanisms. Longitudinal data can reveal whether early improvements persist or decay after staff turnover, workload changes, or technology updates. A research program should publish coding rules, intervention fidelity, uncertainty estimates, and the conditions under which a result may fail to transfer. In this field, scientific precision is measured by the explicitness of the system and causal assumptions as much as by statistical significance.
Human–Artificial Performance Evidence in 2024–2026
Contemporary experiments enable the theoretical problems of human reliability and automation to be studied with explicit comparisons among human-only, artificial-only, and combined performance. The key distinction is between augmentation, in which an artificial system improves human performance relative to humans unaided, and synergy, in which the combined team exceeds the performance of either independent component. This distinction matters for safety because a combined workflow may offer genuine benefits to a human operator while still performing worse than an available alternative, or may improve an intermediate knowledge score without reducing adverse events. The difference is measured through task-specific outcomes and counterfactual comparisons rather than inferred from general expectations about technology.
In a preregistered systematic review and meta-analysis, Vaccaro, M., Almaatouq, A., & Malone, T (2024) synthesized 106 experiments with 370 effect sizes from studies published between January 2020 and June 2023. The inclusion criteria required performance comparisons of humans alone, AI alone, and human–AI combinations. The authors found that combinations performed worse, on average, than the better of the two independent components for the studied outcomes (Hedges' g = −0.23; 95% confidence interval, −0.39 to −0.07), while an average human-augmentation benefit remained. Task type mattered: decision tasks exhibited performance losses while content-creation tasks showed comparatively favorable results. This finding establishes neither an intrinsic deficiency of collaborative systems nor a general safety risk across all use cases; it demonstrates that complementarities must be measured rather than presumed. The review also reports heterogeneous study designs, possible publication bias, and restricted historical coverage.
In a randomized clinical vignette trial, Goh, E., Gallo, R. J., Strong, E., Weng, Y., Kerman, H., Fre (2025) enrolled 92 practicing physicians to compare GPT-4 assistance plus conventional resources with conventional resources alone during five expert-developed management cases. Physicians in the assisted group received higher mean rubric scores, with a mean difference of 6.5 percentage points (95% confidence interval, 2.7 to 10.2), but also spent more time per case. The study did not detect a statistically significant performance difference between LLM-assisted physicians and the LLM alone. These are results for simulated expert-scored reasoning under specified conditions, not direct measurements of patient harm, long-term skill retention, or routine deployment. The publication was corrected in February 2025 to add omitted author-contribution footnotes, without changing the core result described here.
A separate randomized pre–post intervention study by Goh, E., Bunning, B., Khoong, E. C., Gallo, R. J., Milstein (2025) involved 50 U.S.-licensed physicians examining standardized chest-pain vignettes. After GPT-4 recommendations, the investigators reported improvements in guideline-based decision scores for the two vignette-demographic groups they studied. The principal outcome concerns physicians' willingness to revise decisions and the measured accuracy of those revisions in simulated cases. The study was too small and context-specific to establish all-population equity or actual treatment-outcome effects. Together, these trials show why the evaluation of AI support requires precise endpoints for information processing, human decision change, group difference, decision time, and patient safety.
Erroneous AI Advice, Automation Bias, and Clinical Risk
Human reliability is particularly challenged when a highly fluent artificial recommendation contains a clinically significant error. A 2026 single-blind randomized vignette experiment by Qazi, I. A., Ali, A., Khawaja, A. U., Akhtar, M. J., Sheikh (2026) enrolled 44 physicians in Pakistan following a 20-hour AI-literacy course. Participants received optional LLM suggestions, with the intervention group exposed to deliberately erroneous suggestions in three of six cases and the comparison group receiving error-free suggestions. The reported composite diagnostic reasoning mean was 73.3% in the erroneous-advice arm versus 84.9% in the comparison arm, an adjusted difference of −14.0 percentage points (95% confidence interval, −18.9 to −9.1). The study directly supports susceptibility to inaccurate advice under controlled conditions. Its comparison is erroneous AI advice against correct AI advice, so it does not isolate the effect relative to unaided clinical reasoning, nor establish an adverse-event rate in patients.
The causal mechanism of this trial is more informative than an unqualified assertion that people trust AI too much. Participants could seek model advice voluntarily and retain decision authority, while the quality of that advice was experimentally manipulated. Consequently, an observed decline in performance identifies an actionable joint-system risk: a harmful suggestion can change the output of a trained professional's decision process. It also highlights an oversight problem. Training, optional consultation, and nominal human authority do not guarantee the detection of a plausible false statement, especially when the user must judge it under real operational constraints. The prospective design limits some confounding, while the small sample, artificial case conditions, single model, and particular training environment restrict transfer.
The findings complement the automation framework of Parasuraman, R., Sheridan, T. B., & Wickens, C. D (2000): system designers must evaluate not only the technical accuracy of information-analysis functions but also how outputs influence human choice and action. Specific safeguards can be tested by comparing interface uncertainty displays, access to source evidence, forced verification of high-risk claims, and controlled escalation pathways. Such safeguards should be assessed for their ability to reduce harmful acceptance of wrong advice without simultaneously imposing unacceptable delays, creating new errors, or degrading performance on straightforward cases. The objective is calibrated joint reliability as observed across decision quality and external consequences.
Field Effectiveness and the Limits of Simulated Accuracy
Evidence from real clinical systems provides a necessary test of whether promising simulated decision performance translates into patient outcomes. In a pragmatic cluster-randomized trial published in 2026, Agweyu, A., Mwaniki, P., Menon, V., Korom, R., Isaaka, L., W (2026) studied LLM-enabled clinical support in a network of 16 primary-care facilities in Kenya. The trial enrolled 9,691 patients under 103 clinical officers and evaluated an expert-adjudicated composite of treatment failures within 14 days. The reported primary endpoint occurred in 102 of 4,693 patients (2.2%) in the assisted arm and 94 of 4,654 (2.0%) in the comparison arm; the adjusted odds ratio was 0.77 (95% confidence interval, 0.55 to 1.08; P = 0.13). The prespecified primary comparison was not statistically significant. Discrepancies between raw proportions and adjusted association estimates underscore the importance of reporting cluster structure, model covariates, uncertainty, and the defined estimand.
This trial strengthens the methodological architecture of human reliability research because it moves beyond hypothetical diagnostic accuracy to a measured patient outcome in routine service delivery. Yet no single trial settles systemwide efficacy across different settings, model versions, patient populations, and operational arrangements. Its result should be interpreted in the context of the specified implementation, follow-up interval, clinical officers, facilities, and local oversight. The analysis cannot justify a general claim that all AI assistance is ineffective, just as promising simulated scores cannot justify a universal claim that it is safe. The relevant scientific object is the configured service system and the exact outcome it produces.
Taken together, the 2024–2026 studies show why the literature must keep four research questions separate: whether artificial output is accurate, whether a human recognizes its limitations, whether the combined workflow improves the task, and whether the resulting system reduces serious harm. The Vaccaro, M., Almaatouq, A., & Malone, T (2024) meta-analysis addresses comparative performance across experiments; Goh, E., Gallo, R. J., Strong, E., Weng, Y., Kerman, H., Fre (2025) and Goh, E., Bunning, B., Khoong, E. C., Gallo, R. J., Milstein (2025) address simulated clinical reasoning and decision change; Qazi, I. A., Ali, A., Khawaja, A. U., Akhtar, M. J., Sheikh (2026) experimentally tests the effects of erroneous advice; and Agweyu, A., Mwaniki, P., Menon, V., Korom, R., Isaaka, L., W (2026) evaluates a pragmatic patient outcome. Their complementary findings support a sociotechnical research program centered on the causal design and observability of joint human–Artificial performance.
Governance, Accountability, and the Social Distribution of Safety
Sociotechnical safety is governed through the allocation of resources, information, authority, and responsibility. In a complex organization, the person closest to a hazard often has the least power to redesign the conditions that created it. Conversely, managers and regulators may possess decision authority while receiving only aggregated summaries of frontline difficulties. Rasmussen (1997)00052-0) conceived risk management across interacting layers of a dynamic society, making the flow of feedback and the enforcement of constraints central scientific concerns. This framework supports empirical investigation of whether warning signals reach decision makers and whether their decisions alter the exposures that generated those signals.
Accountability arrangements shape what is reported, how incidents are explained, and what changes follow. A system that attributes every adverse outcome to an individual may discourage disclosure of weak signals, while a system that treats all actions as consequences of structure may fail to investigate meaningful choices. A research-based approach documents both available individual options and the organizational conditions under which those options were created. It distinguishes observed behavior, decision authority, negligence criteria established by appropriate institutions, and causal mechanisms contributing to risk. Scientific analysis informs organizational decisions but does not substitute for legal or professional adjudication.
Safety is also distributed unequally. Workers with limited bargaining authority, temporary employment, language barriers, disabilities, or insufficient training may encounter distinctive hazards and reduced opportunities to decline unsafe work. Patients and members of the public may bear losses without participating in design decisions. Research therefore requires denominators and subgroup analyses that reveal who faces hazards, who can recognize them, who can stop a process, and whose reports are acted upon. International patient-safety policy recognizes the need for institutional responsibility and inclusion, as reflected in the World Health Organization (2021). Such policies define governance objectives; empirical studies must establish their implementation and effects.
Ethical Boundaries of Error Attribution and Incident Data
Human error is a scientifically useful term when applied with clear operational criteria. It becomes ethically consequential when an investigative classification affects employment, licensing, compensation, public reputation, or criminal responsibility. An analyst must separate an observed deviation from conclusions about competence, motivation, or culpability. In an event involving conflicting instructions, a defective interface, and time-critical pressure, the classification of the final action as erroneous may be correct while remaining insufficient as an account of why the event occurred. A rigorous inquiry preserves the causal network and the actor's actual information.
Incident datasets contain sensitive details about individuals and organizations. Responsible research requires governance of identifiable records, appropriate access controls, and transparent rules for disclosure. De-identification alone can leave people or facilities recognizable when events are rare or publicly documented. Data quality also depends on institutional trust: personnel who expect punishment may omit contextual facts, whereas a confidential reporting system can capture weak signals that formal audits miss. These considerations make ethics part of the measurement process, not merely a commentary after statistical analysis.
Errors made by automated decision systems create an additional attribution problem. An operator may formally approve a recommendation that is effectively impossible to scrutinize within the allotted time. A valid responsibility model should ask who selected the model, set thresholds, controlled the interface, approved its operating envelope, monitored failure modes, and retained authority to stop use. These questions apply to human and artificial controllers with different capacities and roles. Their scientific content lies in observable decision pathways, documented oversight, and the outcomes of prospective tests.
Critical Scientific Discussion
The strongest contribution of the literature is the integration of multiple explanatory levels. Cognitive research describes how actions deviate from intentions and how judgments are formed under uncertainty. Human reliability assessment offers conditional models of task performance in defined hazardous scenarios. Accident investigation reconstructs pathways and organizational controls, while comparative intervention studies test whether changing schedules, technology, communication, or barriers alters outcomes. Reason, J (2000), Rasmussen (1997)00052-0), and Leveson (2004)00047-X) provide complementary theoretical orientations; their empirical claims must be assessed at the level of the processes they specify.
The first unresolved controversy concerns the explanatory status of human error. Some traditions begin with deviations from prescribed tasks, while others treat the designation as an investigative judgment dependent on local context and retrospect. Both perspectives can be productive when they specify the counterfactual. An erroneous control selection can be a valid behavioral observation. Explaining its occurrence requires knowledge of cue availability, action mapping, task pressure, and system feedback. A scientific study should preserve the observed event while resisting the inference that the word error supplies its own cause.
The second controversy concerns how far quantitative human reliability estimates can be generalized. A conditional estimate can support engineering choices when model assumptions, exposure conditions, and dependencies are transparent. It becomes misleading when converted into a timeless probability of a worker making a mistake. The difficulty is greatest for unfamiliar, rare, and evolving situations where the base rate of failure is poorly known. Gertman, D., Blackman, H., Marble, J., Byers, J., & Smith, C (2005) and other methods should therefore be evaluated for analyst agreement, sensitivity, calibration, and practical decision value. The mere presence of mathematical formalism cannot establish predictive validity.
The third controversy concerns evidence of prevention. The randomized duty-hour experiment by Landrigan, C. P., Rothschild, J. M., Cronin, J. W., Kaushal (2004) supports a specific intervention effect on serious error rates under studied intensive-care conditions. The collaborative infection-control study by Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Ch (2006) and the surgical checklist study by Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R (2009) support important improvements while requiring caution about bundled programs and before-and-after comparisons. Population trend research such as Landrigan, C. P., Parry, G. J., Bones, C. B., Hackbarth, A (2010) reminds researchers that local improvement and systemwide trend are different estimands. The synthesis is a design principle for evidence: identify the intervention mechanism, measure the proximal target, test the downstream safety endpoint, and assess durability.
The fourth controversy arises from automation. The increasing reliability of individual artificial components does not settle the reliability of a coupled human–artificial work system. Problems of trust calibration, incomplete verification, software distribution shift, latent dependencies, and unclear control authority can persist even when performance improves on benchmark tasks. Parasuraman, R., Sheridan, T. B., & Wickens, C. D (2000) established a framework for analyzing which functions are automated; Sarter, N. B., & Woods, D. D (1997) demonstrated the relevance of mode understanding in practical automation. Contemporary empirical programs must assess joint decisions and their real consequences, including the possibility that an ostensibly protective interface creates new pathways to error.
Research Agenda and Testable Scientific Questions
A coherent research agenda should begin with standardized definitions and high-resolution observational data. Researchers can develop task-specific taxonomies with validated interrater agreement, explicitly recording the information available to operators and the organizational decisions shaping their environment. Longitudinal designs should follow changes in staffing, technology, workload, and procedures so that incident patterns can be interpreted against changing exposure. A shared vocabulary need not erase context; instead, core measurement definitions can be combined with documented local variables.
Prospective comparative studies are needed to evaluate competing safety models. Investigators could apply event-sequence, barrier, and systems-theoretic techniques to the same blinded incident records and compare reproducibility, additional hazards identified, accuracy of prospectively predicted problems, and usefulness of recommendations. Such a design would distinguish explanatory breadth from predictive performance and practical impact. Safety-model comparisons should report analyst experience and cost, because an elaborate method can be scientifically informative while remaining difficult to deploy in routine operations.
The development of artificial systems creates an especially important testing ground. Human–artificial teams can be studied under controlled variations in model accuracy, confidence reporting, action authority, time pressure, and observability of machine state. Primary outcomes should include detection of incorrect recommendations, failures to intervene, timely recovery, and consequences of unsafe actions, alongside workload and task speed. Field validation can then examine whether experimental mechanisms transfer to deployment conditions. The research should also assess what happens after long periods of automated success, when manual skills and expectation of rare failure may have changed.
Another priority is the comparative effectiveness of interventions. Factorial or stepped-wedge designs may help disentangle the contributions of technical barriers, training, communication, and supervisory response, when their assumptions and implementation permit causal inference. More studies should monitor substitution of hazards, unequal effects on worker groups, reporting changes, and sustainability after organizational turnover. When randomized experiments are impracticable, a clear intervention theory combined with robust observational designs can still increase knowledge. Publicly available measurement instruments, transparent data governance, and replication across institutions would strengthen cumulative evaluation.
The overarching scientific question is how dependable performance is produced and maintained through interactions among cognitive capability, designed constraints, organizational authority, and adaptive technical systems. It can be decomposed into hypotheses about error opportunity, detection, containment, learning, and harm. Research progress depends on distinguishing those outcomes and explaining the pathways between them. The Artificial Era expands the range of controllers that participate in these pathways and increases the importance of traceable system governance.
Conclusion
Human error, human reliability, and sociotechnical safety constitute a single research field organized around a specific problem: how human action and designed work conditions jointly determine the probability and consequences of harmful system states. Human error becomes analytically meaningful when intentions, performance standards, situational information, and observable deviations are defined. Human reliability is a conditional property of people performing tasks under particular circumstances. Sociotechnical safety is the capacity of a system to constrain hazards, detect developing failures, and contain loss through interacting cognitive, technical, organizational, and institutional mechanisms.
The historical development from individual error taxonomies through high-reliability organizations, probabilistic assessment, barrier analysis, and systems-theoretic safety has yielded complementary methods. Evidence from aviation, health care, and industrial operations shows that explanations of accidents and evaluations of prevention must connect proximal human behavior to available information, task design, time constraints, supervisory decisions, and protective arrangements. Experimental interventions demonstrate that work conditions can modify error rates; observational studies and systems models supply important but differently qualified knowledge about wider causes and potential controls.
The central methodological result is that errors, incidents, near misses, and harmful outcomes are separate variables with different denominators and sources of uncertainty. Causal explanations require transparent counterfactuals, and model-based probabilities require assumptions, sensitivity analyses, and calibration. Reliable safety programs are therefore accountable to measurable mechanisms and real-world outcomes. Human–artificial systems extend this research object by distributing sensing, advice, choice, and action across multiple controllers. Scientific progress will depend on studying the complete configuration of responsibility, feedback, verification, and consequences over time.
References
- Agweyu, A., Mwaniki, P., Menon, V., Korom, R., Isaaka, L., Wanyama, C., Gill, J., Kiptinness, S., Adan, N., Emmanuel-Fabula, M., Riley, R. D., Archer, L., Denniston, A. K., Liu, X., & Mateen, B. A. (2026). Generative AI-enabled clinical decision support system in primary care: A pragmatic, cluster-randomized trial. Nature Medicine, 32(8), 3032–3039. https://doi.org/10.1038/s41591-026-04503-6
- Bates, D. W., Leape, L. L., Cullen, D. J., Laird, N., Petersen, L. A., Teich, J. M., Burdick, E., Hickey, M., Kleefield, S., Shea, B., Vander Vliet, M., & Seger, D. L. (1998). Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA, 280(15), 1311–1316. https://doi.org/10.1001/jama.280.15.1311
- Carayon, P., Schoofs Hundt, A., Karsh, B.-T., Gurses, A. P., Alvarado, C. J., Smith, M., & Brennan, P. F. (2006). Work system design for patient safety: The SEIPS model. Quality & Safety in Health Care, 15(Suppl. 1), i50–i58. https://doi.org/10.1136/qshc.2005.015842
- Christian, M. S., Bradley, J. C., Wallace, J. C., & Burke, M. J. (2009). Workplace safety: A meta-analysis of the roles of person and situation factors. Journal of Applied Psychology, 94(5), 1103–1127. https://doi.org/10.1037/a0016172
- Clarke, S. (2006). The relationship between safety climate and safety performance: A meta-analytic review. Journal of Occupational Health Psychology, 11(4), 315–327. https://doi.org/10.1037/1076-8998.11.4.315
- Cohen, T. N., Wiegmann, D. A., & Shappell, S. A. (2015). Evaluating the reliability of the Human Factors Analysis and Classification System. Aerospace Medicine and Human Performance, 86(8), 728–735. https://doi.org/10.3357/AMHP.4218.2015
- Dekker, S. W. A. (2002). Reconstructing human contributions to accidents: The new view on error and performance. Journal of Safety Research, 33(3), 371–385. https://doi.org/10.1016/S0022-4375(02)00032-4
- Gertman, D., Blackman, H., Marble, J., Byers, J., & Smith, C. (2005). The SPAR-H human reliability analysis method (NUREG/CR-6883). U.S. Nuclear Regulatory Commission. https://www.nrc.gov/regulations-legislation/nureg-series-publications/publications-prepared-by-nrc-contractors/cr6883
- Goh, E., Bunning, B., Khoong, E. C., Gallo, R. J., Milstein, A., Centola, D., & Chen, J. H. (2025a). Physician clinical decision modification and bias assessment in a randomized controlled trial of AI assistance. Communications Medicine, 5, Article 59. https://doi.org/10.1038/s43856-025-00781-2
- Goh, E., Gallo, R. J., Strong, E., Weng, Y., Kerman, H., Freed, J. A., Cool, J. A., Kanjee, Z., Lane, K. P., Parsons, A. S., Ahuja, N., Horvitz, E., Yang, D., Milstein, A., Olson, A. P. J., Hom, J., Chen, J. H., & Rodman, A. (2025b). GPT-4 assistance for improvement of physician performance on patient care tasks: A randomized controlled trial. Nature Medicine, 31(4), 1233–1238. https://doi.org/10.1038/s41591-024-03456-y
- Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A.-H. S., Dellinger, E. P., Herbosa, T., Joseph, S., Kibatala, P. L., Lapitan, M. C. M., Merry, A. F., Moorthy, K., Reznick, R. K., Taylor, B., & Gawande, A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360(5), 491–499. https://doi.org/10.1056/NEJMsa0810119
- Hollnagel, E. (1998). Cognitive reliability and error analysis method (CREAM). Elsevier. https://doi.org/10.1016/B978-0-08-042848-2.X5000-3
- Hollnagel, E. (2004). Barriers and accident prevention. Ashgate. https://erikhollnagel.com/books/barriers-and-accident-prevention-2009
- Hollnagel, E. (2012). FRAM: The functional resonance analysis method: Modelling complex socio-technical systems. Ashgate. https://www.routledge.com/link/link/p/book/9781409445524
- Institute of Medicine. (2000). To err is human: Building a safer health system (L. T. Kohn, J. M. Corrigan, & M. S. Donaldson, Eds.). National Academies Press. https://doi.org/10.17226/9728
- Landrigan, C. P., Parry, G. J., Bones, C. B., Hackbarth, A. D., Goldmann, D. A., & Sharek, P. J. (2010). Temporal trends in rates of patient harm resulting from medical care. New England Journal of Medicine, 363(22), 2124–2134. https://doi.org/10.1056/NEJMsa1004404
- Landrigan, C. P., Rothschild, J. M., Cronin, J. W., Kaushal, R., Burdick, E., Katz, J. T., Lilly, C. M., Stone, P. H., Lockley, S. W., Bates, D. W., & Czeisler, C. A. (2004). Effect of reducing interns' work hours on serious medical errors in intensive care units. New England Journal of Medicine, 351(18), 1838–1848. https://doi.org/10.1056/NEJMoa041406
- LaPorte, T. R., & Consolini, P. M. (1991). Working in practice but not in theory: Theoretical challenges of “high-reliability organizations.” Journal of Public Administration Research and Theory, 1(1), 19–48. https://doi.org/10.1093/oxfordjournals.jpart.a037070
- Leveson, N. (2004). A new accident model for engineering safer systems. Safety Science, 42(4), 237–270. https://doi.org/10.1016/S0925-7535(03)00047-X
- Lockley, S. W., Cronin, J. W., Evans, E. E., Cade, B. E., Lee, C. J., Landrigan, C. P., Rothschild, J. M., Katz, J. T., Lilly, C. M., Stone, P. H., Aeschbach, D., & Czeisler, C. A. (2004). Effect of reducing interns' weekly work hours on sleep and attentional failures. New England Journal of Medicine, 351(18), 1829–1837. https://doi.org/10.1056/NEJMoa041404
- Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354
- Perrow, C. (1999). Normal accidents: Living with high-risk technologies (Rev. ed.). Princeton University Press. https://doi.org/10.2307/j.ctt7srgf
- Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Chu, H., Cosgrove, S., Sexton, B., Hyzy, R., Welsh, R., Roth, G., Bander, J., Kepros, J., & Goeschel, C. (2006). An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of Medicine, 355(26), 2725–2732. https://doi.org/10.1056/NEJMoa061115
- Qazi, I. A., Ali, A., Khawaja, A. U., Akhtar, M. J., Sheikh, A. Z., & Alizai, M. H. (2026). Automation bias in large language model–assisted diagnostic reasoning among physicians trained in AI literacy: A randomized clinical trial. NEJM AI, 3(5). https://doi.org/10.1056/AIoa2501001
- Rasmussen, J. (1983). Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(3), 257–266. https://doi.org/10.1109/TSMC.1983.6313160
- Rasmussen, J. (1997). Risk management in a dynamic society: A modelling problem. Safety Science, 27(2–3), 183–213. https://doi.org/10.1016/S0925-7535(97)00052-0
- Reason, J. (1990). Human error. Cambridge University Press. https://doi.org/10.1017/CBO9781139062367
- Reason, J. (2000). Human error: Models and management. BMJ, 320(7237), 768–770. https://doi.org/10.1136/bmj.320.7237.768
- Salas, E., Shuffler, M. L., Thayer, A. L., Bedwell, W. L., & Lazzara, E. H. (2015). Understanding and improving teamwork in organizations: A scientifically based practical guide. Human Resource Management, 54(4), 599–622. https://doi.org/10.1002/hrm.21628
- Sarter, N. B., & Woods, D. D. (1997). Team play with a powerful and independent agent: Operational experiences and automation surprises on the Airbus A-320. Human Factors, 39(4), 553–569. https://doi.org/10.1518/001872097778667997
- Shappell, S., Detwiler, C., Holcomb, K., Hackworth, C., Boquet, A., & Wiegmann, D. A. (2007). Human error and commercial aviation accidents: An analysis using the Human Factors Analysis and Classification System. Human Factors, 49(2), 227–242. https://doi.org/10.1518/001872007X312469
- Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 8, 2293–2303. https://doi.org/10.1038/s41562-024-02024-1
- Wiegmann, D. A., & Shappell, S. A. (2001). Human error analysis of commercial aviation accidents: Application of the Human Factors Analysis and Classification System (HFACS). Aviation, Space, and Environmental Medicine, 72(11), 1006–1016. https://pubmed.ncbi.nlm.nih.gov/11718505/
- World Health Organization. (2021). Global patient safety action plan 2021–2030: Towards eliminating avoidable harm in health care. https://www.who.int/publications/b/57613