RESEARCH DOSSIER
Human-Centered Design, Usability, and Iterative Evaluation
Abstract. Human-centered design organizes the investigation, construction, and iterative evaluation of interactive systems around people pursuing specified goals in particular contexts. This fundamental research monograph examines usability as contextual effectiveness, efficiency, and satisfaction; reconstructs the history of cognitive engineering, user-centered design, heuristic inspection, cognitive walkthroughs, and empirical usability testing; and critically synthesizes evidence on problem detection, sample size, questionnaires, task-based measurement, causal experimentation, learnability, error recovery, and evaluation governance. The evidence draws on original theoretical and experimental work, methodological comparisons, international standards, and contemporary studies of mobile interaction, accessible design, manipulative interfaces, and human–AI interaction. The analysis distinguishes formative discovery from summative estimation, examines evaluator effects and threats to validity, and proposes a traceable evaluation program linking observed breakdowns to testable interventions and independently measured outcomes. The final chapters identify gaps in longitudinal assessment, representative sampling, consequential risk, and adaptive human–artificial workflows. This work is a critical narrative scientific synthesis and reports neither newly collected experimental data nor an independently conducted systematic review or meta-analysis.
Keywords: human-centered design; user-centered design; usability; usability testing; user experience; ergonomics; human-computer interaction; iterative evaluation; ISO 9241-11; ISO 9241-210; task-based usability; effectiveness; efficiency; satisfaction; heuristic evaluation; cognitive walkthrough; user research; formative evaluation; summative evaluation; System Usability Scale; sampling; accessibility; A/B testing; human–AI interaction; Artificial Era
Human-Centered Design as a Psychological and Engineering Research Object
Human-centered design is an organized process of studying people, the goals they pursue, the activities by which they pursue them, and the social and material conditions in which a system becomes usable. The object of research is the relation among the person, the task, the interactive artifact, and the environment across design decisions and subsequent use. A person can complete the same nominal task with different errors, effort, confidence, and consequences depending on the interface's representations, constraints, feedback, and recovery mechanisms. The established conceptual formulation in ISO (2019) addresses human-centered activities throughout the design life cycle, whereas ISO (2018) defines usability through specified users, goals, and contexts. A psychological account consequently studies perception, memory, action selection, learning, expectation, attention, and subjective experience as observable or inferable processes within actual interaction.
The central questions are how researchers identify the needs of heterogeneous users; transform contextual observations into testable design requirements; predict or uncover failures; evaluate effectiveness, efficiency, and satisfaction; and establish whether iterative changes genuinely improve performance. The research object extends across websites, mobile applications, workplace interfaces, service journeys, clinical and safety-critical technologies, and systems incorporating artificial intelligence. Its unit of analysis may be a single action, a multistep workflow, a team, or an institution. The same screen can be successful for one user population and inadequate for another because expertise, impairment, language, device, incentives, risk, or organizational obligations alter the conditions of use.
This monograph is a critical narrative synthesis of original theoretical contributions, methodological research, experimental and observational studies, measurement research, and current international guidance. It brings the foundational design framework of Gould & Lewis (1985) into dialogue with the cognitive accounts of Card et al. (1983) and Norman & Draper (1986), empirical usability assessment, and more recent investigations of accessibility, manipulation, and human–artificial interaction. It contains no newly collected participants, original randomized intervention, preregistered search protocol, or meta-analysis. Conclusions distinguish tested effects, methodological arguments, and proposed research directions. The aim is a cumulative explanatory account of how human-centered design decisions may be made empirically accountable.
Usability, User Experience, and Quality in Use
The current ISO framework defines usability in terms of the extent to which specified users achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use ISO (2018). Effectiveness concerns the accuracy and completeness of goal achievement. Efficiency relates resources such as time and effort to that achievement. Satisfaction concerns the relevant physical, cognitive, and emotional responses of people using the system. These dimensions describe realized use, so an attribute such as large buttons or a minimalist appearance becomes a candidate mechanism for usability rather than a substitute for observed success. A scientifically precise report should identify the target population, tasks, devices, context, scoring rules, and interpretation of each measure.
User experience is a broader construct involving expectations, interpretations, emotional responses, identity, aesthetic appraisal, and experience before, during, and after interaction. Usability and user experience overlap because satisfaction and meaningful experience depend partly on whether people can achieve what they intend. They can diverge in particular settings: a product may accomplish a task quickly while imposing an unwelcome surveillance burden, or a deliberately challenging creative system may offer a valuable experience despite increased task time. The research consequence is to specify which outcome matters and why. Hornbæk (2006) found that common usability measures in 180 reviewed HCI studies often failed to capture the construct broadly or to justify measurement choices adequately.
The three ISO dimensions are a starting structure, not a universal three-number score. Learnability, memorability, discoverability, error propensity, recoverability, cognitive workload, accessibility, trust, and freedom from harm may require additional operationalization. In mobile settings, Harrison et al. (2013) observed that conventional models neglected aspects such as cognitive load when designing for variable mobility and context. The investigator must distinguish a theoretical quality, a directly observed behavior, and a participant judgment. A task-completion percentage answers a different question from a post-task confidence score; a global attitude questionnaire answers yet another. Triangulation is useful precisely because each indicator has a bounded inferential meaning.
Historical Foundations of Human-Centered Systems
Modern interactive-system design emerged from intersecting traditions in experimental psychology, ergonomics, systems engineering, and computer science. The applied information-processing program synthesized by Card, Moran, and Newell in Card et al. (1983) offered predictive models of skilled interaction, decomposing tasks into operations and estimating the time demands of specific user actions. Its importance lies in making interface performance analyzable through explicit assumptions about human cognition and behavior. Such models are most informative when the task is structured and the relevant skill and environmental conditions are understood; they are weaker for situated interpretation, social negotiation, affective reactions, and rapidly changing tasks.
Observational work revealed that procedural descriptions alone could poorly predict how novices would learn. Lewis and Mack's study of learning a text-processing system analyzed thinking-aloud accounts and emphasized failures in stepwise self-instruction Lewis & Mack (1982). Gould and Lewis Gould & Lewis (1985) then articulated a durable design program organized around early attention to people and tasks, empirical measurement, and iterative modification. Their contribution included an empirical examination of the difficulties designers encountered when adopting these principles. The historical movement was from the ideal of getting a technical interface right by specification toward repeatedly confronting a design hypothesis with observable use.
The multidisciplinary collection edited by Norman and Draper Norman & Draper (1986) placed the user's understanding and the social environment at the center of computer system design. Norman's cognitive engineering and the associated discussion of action, feedback, and mental models helped explain why nominally available functions might remain inaccessible to a person attempting to act. Dumas and Redish Dumas & Redish (1993) subsequently consolidated practical procedures for usability testing across planning, task construction, observation, and use of results. These traditions differed in emphasis—predictive modeling, cognitive explanation, contextual inquiry, and experimental evaluation—yet each supplies a distinct part of the contemporary evidence base.
International standardization integrated these approaches into organizational practice. ISO (2019) now specifies principles and activities across the interactive-system life cycle; the 2019 edition was reviewed and confirmed in 2025 according to ISO. ISO (2018), published in 2018 and subsequently confirmed, clarifies the concepts used to evaluate usability. Their authority is procedural and terminological: their existence does not itself demonstrate that every design process claiming conformance delivers improved behavior. That empirical question requires appropriate comparison conditions and outcomes. The historical lesson is that design norms, explanatory psychological models, and measured results must remain identifiable as different forms of scientific evidence.
Human Action, Mental Models, and the Interface
An interface mediates action by presenting possible operations, indicating system state, and providing consequences that users must interpret. People form expectations about what a control means and what will happen when they use it. Divergence between expectation and actual system behavior can produce hesitation, repeated attempts, loss of work, and abandonment. The cognitive-engineering tradition summarized in Norman & Draper (1986) treats action planning and interpretation of feedback as central explanatory problems. In the empirical study of novice interaction Lewis & Mack (1982), users' limited understanding of intermediate system states impeded their ability to recover when instructions were misunderstood or followed incorrectly.
Mental models are inferable representations used to reason about a system. They may be partial, locally useful, and inconsistent. A designer's documented model, a system's technical structure, and a user's working assumptions may therefore diverge. The design question concerns which distinctions must become visible for successful action: whether a file has saved, whether a transfer is final, whether an algorithmic suggestion is tentative, or whether a form contains an irreversible choice. Observable measures include wrong-command sequences, anticipatory explanations, latency before the next action, rechecking, and success after a state transition. Interview evidence can enrich these observations but should not automatically be treated as a direct measurement of the underlying representation.
Visibility, feedback, consistent control semantics, reversible operations, and recognition cues are potentially useful design principles. Nielsen and Molich Nielsen & Molich (1990) demonstrated an inspection-based approach to finding violations in interfaces, and Nielsen (1994) developed an expanded heuristic vocabulary. However, any heuristic is a means of posing questions about possible failures. Its predictive strength depends on the evaluator, the test tasks, the target users, and the context. The investigator can therefore use heuristic analysis to generate hypotheses and then test their behavioral consequences through representative tasks rather than equating a checklist score with actual usability.
Context of Use and the Boundaries of the Research Object
A context of use includes users, goals, tasks, resources, and physical, technical, social, cultural, and organizational environments ISO (2018). This definition has practical methodological force. An emergency clinician operating during interruptions, a commuter using a phone while walking, and an older adult managing a financial account each occupy different configurations of attention, motor control, risk, and available time. A study that abstracts away these constraints may estimate performance under laboratory conditions while failing to predict consequential use. The appropriate target is the specific user–task–system–context relationship, accompanied by an explicit account of where results are expected to generalize.
Representative-user research requires more than convenient recruitment. Investigators should describe inclusion criteria, novice and expert proportions, assistive-technology use where relevant, language, literacy, domain expertise, and prior exposure to the interface. Task scenarios should represent meaningful goals and plausible interruptions rather than only a sequence of scripted clicks. Contextual observation, diary studies, interviews, document analysis, and experiments contribute different evidence. Lazar et al. (2017) presents these methods as complementary research tools, and Harrison et al. (2013) illustrates why setting-specific features can transform conventional usability findings.
The present dossier's primary boundaries are the design life cycle, usability constructs, evaluation procedures, iteration, and the inferential quality of evidence. Specialist problems of inclusive accessibility, persuasive manipulation, and artificial-system agency are treated as intersecting applications; their full taxonomies and disciplinary debates belong to their own research objects. The objective is to explain how to make interaction both usable and scientifically evaluable without absorbing every question in technology psychology into a single dossier. The research structure that follows moves from eliciting requirements to experimental comparison, measurement, iteration, and broader sociotechnical consequences.
Researching Users, Needs, and Work Practices
Human-centered inquiry begins with an evidence-based account of what people are trying to accomplish and which constraints shape their activity. Stakeholder requests, requirements documents, and the preferences of product owners are initial inputs rather than direct substitutes for observations of users. Researchers can reconstruct work by watching ordinary sessions, asking participants to explain goals and unexpected events, reviewing records produced by the task, and contrasting nominal policy with practice. These methods help uncover coordination requirements and environmental constraints that conventional feature inventories omit. The multidisciplinary approach represented by Norman & Draper (1986) and the methodological range documented in Lazar et al. (2017) support studying interaction within its actual social and technical conditions.
A useful research protocol specifies recruitment, tasks, sources of evidence, analytic categories, and the possible influence of the observer. The investigator distinguishes what participants say they generally do, what they report having done recently, and what can be observed during a session. Contradictions among these forms of evidence may reveal workarounds, hidden norms, poorly remembered contingencies, or differences between stated intention and behavior. The resulting requirement should express a testable condition, such as the ability of a first-time user to locate a recovery pathway during an incorrect transaction, rather than a vague aspiration that an interface be intuitive.
Sampling must follow the inferential target. Convenience interviews can reveal mechanisms and language used in a setting, yet cannot establish their prevalence in a population without a suitable sampling design. Investigators should identify people whose failure could be especially consequential, whose tasks differ from those of majority users, and whose access modes challenge default assumptions. Reporting participant characteristics and exclusions is part of the scientific result. The protocol should also document privacy, consent, accessibility accommodations, and the relationship between the researcher and participants; these conditions influence both ethical legitimacy and the quality of the evidence.
From Field Evidence to Design Requirements and Prototypes
Research findings enter design through an explicit transformation from an observed problem to a proposed mechanism and a testable solution. Suppose a user repeatedly abandons a multistep registration because the system gives no visible indication of progress. Observation establishes the sequence of action and failure; a proposed explanation invokes uncertainty about remaining effort; a prototype might present a progress indicator and reversible steps. The explanation remains hypothetical until an evaluation separates it from competing explanations, such as the intrinsic length of the form, confusing labels, or concern about requested information. This inferential discipline is central to the iterative approach described by Gould & Lewis (1985).
Low-fidelity sketches, paper prototypes, and interactive models are useful for comparing information architecture, navigation, wording, and major task structures before software implementation becomes expensive. Their scientific value depends on fidelity to the questions being tested. A paper prototype can expose misunderstanding of labels but may poorly represent latency, gesture recognition, continuous feedback, or accessibility behavior under real assistive technology. Higher-fidelity prototypes permit observation of interaction timing and system transitions but may introduce visual detail that distracts from broader task problems. The design team should therefore record the claims a particular prototype can legitimately support.
Requirements become stronger when phrased as observable, contextual criteria. A task can be defined by a specific starting state, realistic available information, an authorized goal, success criteria, and unacceptable consequences. Acceptance criteria should be separated into intended outcomes, failure constraints, and experiential measures. For example, an administrative system might require correct case classification and successful recovery from incomplete records while also measuring task time and perceived confidence. ISO (2019) treats iterative evaluation in relation to specified requirements; ISO (2018) supplies the usability terms for assessing actual task performance.
Task Analysis, Cognitive Walkthrough, and Prospective Inspection
Task analysis decomposes a meaningful goal into actions, decisions, information needs, and dependencies. It can describe a typical sequence, identify optional paths, or examine how experienced users manage uncertainty and interruption. The engineering models in Card et al. (1983) demonstrate how explicit decompositions can generate predictions about routine expert interaction. Those predictions depend on suitable assumptions about operators, expertise, and task stability. A researcher must therefore distinguish time spent on a motor action, time spent deciding, and time spent understanding a surprising state; the same overall task duration can result from entirely different design problems.
The cognitive walkthrough formalized by Polson, Lewis, Rieman, and Wharton Polson et al. (1992) analyzes how a user might learn to perform an intended task through exploratory interaction. At successive action points, evaluators consider whether the user will form an appropriate goal, notice the relevant control, connect the control to the goal, and interpret the feedback. This is a theory-based prediction of possible learnability difficulties. It can be applied before recruiting participants and can focus attention on high-risk sequences, particularly when a task includes unfamiliar terms or hidden modes.
A walkthrough still relies on assumptions about the user and on evaluators' judgments. Hertzum & Jacobsen (2001) found substantial evaluator effects across usability methods, including cognitive walkthroughs. A careful protocol records the assumed user knowledge, scenario, each predicted difficulty, and why a specific observation would confirm or disconfirm it. Independent analysts can then compare results, resolve disagreements, and prepare an empirical test. Inspection provides a systematic source of candidate failures and hypotheses; observed interaction supplies evidence about whether those candidates materialize under specified conditions.
Heuristic Evaluation, Expert Judgment, and Agreement
Heuristic evaluation compares an interface with broad principles of interaction such as visibility, consistency, recognition, error prevention, and recoverability. The original approach of Nielsen & Molich (1990) aggregated judgments from multiple evaluators because individuals often found different subsets of problems. Nielsen (1994) further developed the explanatory heuristic framework. A well-defined evaluation names the target interface version, tasks, evaluator expertise, rules applied, and procedures for logging and consolidating observations. A finding should contain the interface state, observable or anticipated behavior, violated criterion, and potential consequence rather than a generic assertion that a design is poor.
Expert judgments contain variability. In their review of 11 studies, Hertzum & Jacobsen (2001) found that agreement between pairs of evaluators varied considerably and that substantial evaluator effects occurred even in relatively structured methods. These findings undermine the practice of treating a single analyst's report as a complete inventory of defects. Independent evaluations, adjudication criteria, and traceable examples improve accountability, though they do not eliminate differences in interpretation. Evaluation quality should be measured by problem discovery, diagnostic accuracy, inter-evaluator agreement, severity judgments, and the degree to which findings predict actual task difficulty.
Bastien and Scapin Bastien & Scapin (1995) experimentally assessed structured ergonomic criteria and reported improved identification of new problems among evaluators using those criteria in a repeated inspection phase. This supports the hypothesis that systematic criteria can assist experts, while leaving questions about transfer to other interfaces, expert populations, and tasks. Hartson, Andre, and Williges Hartson et al. (2001) argued for explicit criteria to compare usability evaluation methods. The integrated methodological conclusion is that inspections must be evaluated as research instruments: which failures they reveal, how reliably they do so, at what cost, and under what sampling and training conditions.
Observed Usability Testing and the Role of Think-Aloud Methods
Observed task-based testing places participants in defined situations and records how they attempt to achieve meaningful goals. A foundational protocol specifies the interface build, entry state, task instructions, success and failure criteria, facilitator interventions, recording methods, time limits, and conditions for stopping. The participant should have an achievable goal without being led through the intended steps. A moderator can record hesitations, detours, incorrect assumptions, requests for help, failed recovery, and successful completion; these observations reveal mechanisms that an aggregate success score may conceal. The practical tradition set out by Dumas & Redish (1993) remains valuable when combined with contemporary reporting standards.
Concurrent verbalization can illuminate users' interpretations but also changes the activity being observed. Lewis and Mack's early study Lewis & Mack (1982) used thinking-aloud protocols to examine difficulties in learning unfamiliar software. When verbal reports are collected, researchers should avoid inferring that every utterance faithfully represents an otherwise unobserved mental process. Instructions, moderator prompts, the participant's ability to verbalize, and task complexity can influence what is said and sometimes how the task is performed. Screen recordings, event traces, timing, and subsequent interviews may supply complementary evidence. A protocol must make explicit whether talking aloud is required and how that requirement affects comparability between conditions.
Remote and unmoderated studies widen geographic reach but introduce uncertainty about device configuration, distractions, accessibility tools, and whether the participant understands the intended scenario. Laboratory observation improves experimental control but may suppress the organizational pressures that matter in ordinary work. The most defensible design chooses the setting to fit the research question, documents its tradeoffs, and links qualitative failure analysis to reproducible outcome measures. Multiple user groups should be represented separately when they face different tasks or barriers. Reporting the denominator of attempted tasks is essential when some participants never reach an instrumented event.
Formative and Summative Evaluation as Distinct Inference
Formative evaluation is designed to identify mechanisms of failure and guide changes during development. It benefits from early, repeated, relatively low-cost observations because the aim is to generate and prioritize actionable explanations. Summative evaluation estimates how well a defined interface meets preestablished requirements in a specified population and context. It therefore needs stable tasks, reproducible outcomes, sufficiently precise estimates, and explicit decision criteria. Gould and Lewis Gould & Lewis (1985) centered iterative empirical measurement, while ISO (2019) embeds evaluation throughout the design life cycle. The same testing technique can contribute to either purpose, but its sampling and analysis should change with the inference sought.
An exploratory study with a few participants may uncover a catastrophic issue that requires immediate correction. That discovery does not demonstrate the prevalence of the issue or the final system's expected success rate. Conversely, a large comparative study may establish an average performance difference without explaining which interaction mechanism produced it. Combining formative diagnosis and summative comparison provides complementary knowledge. Researchers can first discover and correct failures and then assess a frozen candidate against prespecified criteria, with separate recruitment or explicit acknowledgment of learning effects if the same participants return.
Release decisions should distinguish scientific adequacy from product preference. A team may prioritize risk reduction, accessibility compliance, speed, financial cost, or satisfaction; each priority implies different acceptable thresholds and measurement plans. High-consequence interfaces require a more demanding risk model than a low-stakes media browsing task. A suitable final evaluation incorporates the consequences of errors, uncertainty around effect estimates, and relevant user subgroups. It also records when evidence is insufficient and when a proposed design should enter a further controlled trial rather than being declared successful.
Operationalizing Effectiveness, Efficiency, and Satisfaction
An empirical claim that a system has become easier to use requires a correspondence between the intended construct and a reproducible observation. Effectiveness may be operationalized as task completion, accuracy of the final artifact, completeness of a transaction, or the proportion of correctly handled exceptions. Efficiency can include active task time, number of operations, search effort, physical actions, or resource expenditure relative to correct completion. Satisfaction can be evaluated through post-task ratings or broader standardized instruments. These measures are related, yet a faster completion time does not imply successful completion, and a favorable attitude does not establish error-free performance. The contextual definition in ISO (2018) requires the investigator to state who used the interface, which goal was pursued, and under what conditions.
A task success rate is interpretable only when success and eligible attempts are defined in advance. A respondent who was prevented from starting by an inaccessible authentication step should remain part of the intended-user denominator if successful access was a design requirement. Participants who succeed only after moderator assistance need a distinct classification from independent successes. Time distributions are frequently skewed, and treating timeout values or abandoned tasks as ordinary completion times may bias comparison. Researchers should report medians or quantiles alongside appropriate uncertainty, specify handling of incomplete cases, and describe data exclusions. Different scoring rules can change the apparent performance of the same interface.
Hornbæk's review Hornbæk (2006) documented extensive diversity in how usability was measured across 180 studies and showed why incomplete reporting hinders cumulative comparison. A strong measurement plan connects each metric to a theory of the user's activity, defines its acquisition procedure, and distinguishes direct behavior from inferential proxies. For a safety-related workflow, severity-weighted errors and successful recovery may be more meaningful than an unweighted average task duration. For an exploratory research interface, repeated use and retrieval accuracy may matter more than first-session speed. Composite scores can aid communication, but their weights must reflect an explicit substantive decision and should not conceal contradictory component outcomes.
Psychometric Instruments and Subjective Evaluation
The System Usability Scale was introduced as a short, ten-item global questionnaire in Brooke (1996). It asks respondents to report their perceptions of a system and is commonly summarized on a transformed score scale. The measurement is valuable for standardized attitudinal comparison across interface versions, yet it does not directly count completed tasks, diagnose which control caused a failure, or certify safety or accessibility. Bangor, Kortum, and Miller Bangor et al. (2008) analyzed its empirical properties across a large applied dataset and examined how the score could be interpreted. The appropriate scientific use preserves the distinction between a subjective usability judgment and externally observed task performance.
Psychometric validation concerns reliability, dimensionality, item behavior, sensitivity to change, and comparability among groups. Borsci, Federici, and Lauriola Borsci et al. (2009) examined alternative factor structures for the System Usability Scale, illustrating that an apparently simple total score may rest on contestable measurement assumptions. Lewis Lewis (1995) evaluated IBM computer usability satisfaction instruments and addressed the properties of questionnaire responses. Cross-study comparison should account for language, domain, prior exposure, administration conditions, and who answered the questionnaire. A scale validated for an ordinary interactive product should not automatically be assumed valid for a dangerous clinical device or an interface with specialized users.
Short post-task questions have a narrower purpose than an overall usability inventory. Sauro and Dumas Sauro & Dumas (2009) compared three one-question post-task measures, contributing evidence about response behavior immediately after task performance. Such measures can be paired with a specific failure or success event, while a system-level inventory captures an impression integrated across tasks. Investigators should collect ratings at consistent points and avoid interpreting post-task confidence as an objective indicator of correctness. The strongest design combines theoretically chosen self-report constructs with independently recorded behavior, so disagreement becomes an empirical result worthy of explanation.
The Five-User Question and the Logic of Problem Discovery
Usability research distinguishes the chance that a participant encounters a particular problem from the probability that an observed sample reveals at least one instance. If a problem is independently encountered with probability p under fixed test conditions, the probability of seeing it at least once among n participants is 1 − (1 − p)^n. This illustrative expression presupposes comparable detection probabilities and independent encounters. Real participants differ in expertise, tasks, impairment, and strategies, making those assumptions fragile. A result estimated from easy tasks and a homogeneous sample should not be generalized to rare failures in a heterogeneous population.
Virzi's experiments Virzi (1992) examined the accumulation of identified usability problems with additional participants and supported early discovery of many relatively common problems. The resulting practice of testing with roughly five users proved useful for formative iterations in specific settings. Faulkner Faulkner (2003), however, examined variation across random subsets from a larger usability study and showed that five-person samples could detect substantially different proportions of the known problems. The observed detection range for five participants was 55% to 99% in that study. These results make sample size a question of desired coverage, problem prevalence, population diversity, and risk, rather than a universal constant.
Iteration offers one practical response: recruit small formative samples, revise a design, and test again with new participants. But repetition does not retrospectively turn an exploratory sample into an unbiased population estimate. Teams should distinguish discovery yield per iteration from estimated residual failure risk after a release. When precise prevalence, subgroup differences, or high-consequence failures matter, larger and more deliberately stratified samples are justified. Researchers should also monitor saturation by failure type and importance, acknowledging that low-frequency, high-severity failures may remain undetected even after an apparently repetitive sequence of tests.
Comparative Experiments, A/B Testing, and Causal Inference
An experimental comparison estimates whether a specified design change causes a difference in outcomes under the conditions of the study. Random assignment of eligible users to interface variants can help control measured and unmeasured preexisting differences, while consistent task instructions and outcome definitions support interpretation. An A/B test is meaningful when it identifies the unit of randomization, exclusion rules, sample-size reasoning, primary outcome, exposure duration, and the handling of people who receive more than one variant. Simple before-and-after comparisons are vulnerable to changes in user composition, learning, concurrent product releases, seasonality, and organizational policy.
Experimenters should analyze outcomes according to the question they intend to answer. A product may increase the number of clicks, sign-ups, or completed purchases while making decisions less informed. Maximizing an engagement indicator does not by itself establish improved usability or user welfare. A well-designed study can therefore prespecify task correctness, successful reversibility, reported comprehension, adverse outcomes, and accessibility alongside conversion or use. Mathur et al. (2019) documents manipulative patterns found across large numbers of commercial shopping interfaces, demonstrating why an increase in platform-desired behavior cannot always be interpreted as a benefit for the person acting.
Confidence intervals, effect sizes, clustering, missing-data mechanisms, and multiplicity influence the credibility of apparent differences. If many interface variants and outcomes are compared repeatedly and only favorable results are reported, the nominal significance attached to the final variant can overstate the evidence. Iterative exploration is legitimate, but confirmatory evaluation should retain a prespecified comparison or transparently disclose adaptive testing. Qualitative observation may explain a measured effect, while a randomized trial establishes a more defensible estimate of the tested intervention's average effect. Neither automatically proves that the same mechanism or magnitude applies outside the investigated context.
Usability Problems, Severity, Error Recovery, and Consequences
A usability problem is a recurring or plausible breakdown in a person's attempt to achieve a specified goal through an interactive system. Documentation should identify its trigger, interface state, user action, observable consequence, frequency in the study, and uncertainty about cause. Severity is a decision construct combining potential harm, persistence, recoverability, and exposure. A minor naming ambiguity may cause frequent hesitation; a rare ambiguous confirmation may irreversibly delete records. A ranking that treats these as equal because each generated one observation would misrepresent risk.
Taxonomies of errors should distinguish slips in executing an intended action, mistaken plans based on incorrect understanding, failures of perception, and disruptions of coordination among multiple actors. The cognitive framework in Norman & Draper (1986) supports analyzing mismatches between intended action and feedback. The interface can be evaluated for prevention mechanisms, timely detection, clear explanation, undo paths, and the effort needed to restore a valid state. Error recovery cost may be measured as additional time, assistance requests, repeat work, data loss, and downstream effects. In some tasks, prevention is essential; in others, reliable recovery is a central dimension of usable operation.
Problem logs require deduplication without erasing genuine contextual diversity. Two users encountering identical text at different workflow stages may face different mechanisms and consequences. Conversely, several superficial symptoms may arise from one missing state representation. Evaluators should record an explicit rationale when merging findings and should maintain traceability between the observed record and the proposed fix. The methodological call for evaluation-method effectiveness in Hartson et al. (2001) implies measuring not just the number of flagged problems but their relevance, diagnostic specificity, and impact after correction.
Measurement Error, Replication, and Converging Evidence
A system can appear to improve for reasons unrelated to the design change. Participants may become familiar with the task, moderators may give different prompts, logging may change, or an updated dataset may make searches easier. A transparent experiment preserves interface versions, input data, task wording, recruitment criteria, instrumentation, scoring scripts, and analysis plans. Where replication is feasible, independent evaluators can repeat the protocol using new participants and the same construct definitions. Divergent results should prompt examination of context and measurement sensitivity before assuming that one sample was simply wrong.
Converging evidence is stronger when its sources offer genuinely different opportunities for error. An expert inspection, an observed completion study, a diary of long-term frustrations, and a randomized comparison can address complementary claims. Yet multiple self-report questions delivered in the same survey may share common response biases and should not be counted as independent corroboration. Hornbæk (2006) and Hertzum & Jacobsen (2001) jointly underscore why construct and evaluator variability must enter any critical synthesis. A coherent report maps each major conclusion to the design and limitations of the study that supports it.
Methodological quality is best evaluated at the level of the inference: whether the population was appropriate, whether the tasks represented consequential use, whether measures were valid, whether alternative causes were controlled, and whether uncertainty was reported. Publication and selection biases may favor successfully redesigned interfaces or striking failures. Negative findings and unsuccessful iterations therefore contribute important knowledge when their conditions are fully documented. Cumulative research on usability will improve as intervention descriptions, protocols, instrument versions, and raw outcome definitions become sufficiently explicit for comparisons across products and research teams.
Iterative Design as a Sequence of Testable Interventions
Iteration is a cycle of specifying a user-centered problem, constructing a candidate intervention, evaluating its consequences, and revising the next design decision in light of the evidence. Gould and Lewis Gould & Lewis (1985) established iteration as a fundamental usability principle, while ISO (2019) integrates human-centered evaluation into a broader life cycle of requirements, design, implementation, and maintenance. Iteration does not automatically yield improvement: a changed interface may fix one task while introducing new problems elsewhere. An accountable process preserves the baseline, states which mechanism a proposed change targets, and tests outcomes against explicit objectives.
A change log should link observed evidence to the design decision it motivated and to the subsequent evaluation of that decision. For example, if a confusing recovery prompt is replaced, the next test should measure whether users recognize the problem, choose the correct recovery operation, and preserve their work. A team may also investigate whether the change raises the burden for experienced users or for people relying on assistive technology. Without such traceability, designers can repeatedly modify the interface while learning little about which causal explanations were correct. Version control and experimental records become part of scientific provenance, not merely production administration.
The evaluation cycle has a stopping problem. New tasks, new populations, and new edge cases can always reveal further concerns. A rational release decision therefore defines evidence thresholds based on material outcomes, known failure severity, residual uncertainty, and the expected consequences of delay. For a low-risk information tool, substantial improvement on representative tasks may suffice. For a safety-critical interface, validation may require performance under rare exceptions, stress, interruption, and coordinated multiuser action. The scientific principle is a correspondence between decision stakes and the strength of evidence demanded.
Learning, Retention, and Longitudinal Use
First-use testing measures discoverability and initial comprehension, but use evolves as people form habits and refine their understanding of the system. A frequently used workplace tool may become efficient for experienced operators despite a steep learning curve, while a seemingly obvious consumer service may reveal recurring confusion after weeks of exposure. The models in Card et al. (1983) help explain the speed of well-practiced component actions, whereas Norman & Draper (1986) emphasizes the expectations and interpretations needed when a task changes or a system behaves unexpectedly. Evaluation should therefore align observation times with expected learning trajectories.
A longitudinal protocol can measure task accuracy, time, help-seeking, retention after a period without use, and transfer to unfamiliar tasks. It may also record whether participants develop risky workarounds or abandon important but difficult functions. Repeated measures permit analysis of change within individuals, but practice effects, attrition, system updates, and shifting tasks complicate attribution. A useful analysis retains the actual sequence of software versions and events rather than merging all observations into a static average. When participants leave a study, investigators should examine whether attrition was related to difficulty, dissatisfaction, or accessibility barriers.
Changes made to an established interface impose a transition cost that a new-user test may miss. Existing users can have effective internal routines and dependencies on stable control locations. A redesign may improve first-session performance for novices while temporarily harming expert productivity. Evaluation should measure the distribution of these consequences and whether support, migration guidance, or alternative interaction pathways reduce the cost of change. Human-centered design thus involves the history of person–system relations as well as the visual clarity of the present interface.
Accessibility, Inclusive Evaluation, and Diverse Capabilities
An evaluation that excludes people whose vision, hearing, mobility, cognition, literacy, or language differs from the convenient recruitment sample measures a restricted form of usability. Inclusive research requires explicit consideration of the users and access modes that the system claims to support. The Web Content Accessibility Guidelines 2.2 W3C (2024) provide testable accessibility requirements for web content, including requirements relevant to keyboard operation, perceivability, and navigation. Conformance testing is an important component of evaluation, while direct observation with people using their own assistive technologies helps establish how successfully they can complete meaningful tasks.
Accessibility constraints can alter the basic unit of interaction. A screen-reader user may need to traverse a complex hierarchy of labels and landmarks; a keyboard user may encounter focus order or trapping failures; a person with limited fine motor control may face accidental activation; a person with memory difficulties may be disproportionately affected by vanished instructions. These experiences should shape test scenarios, outcome criteria, and the interpretation of efficiency. Applying a universal task-time benchmark across different access modes may confuse design failure with normal differences in navigation technique.
Inclusive design requires both broad procedural commitments and investigation of particular access barriers. ISO (2019) makes users and contexts part of the design process; ISO (2018) makes user populations part of the measurement definition. WCAG W3C (2024) supports technical conformance checks, and empirical studies can evaluate task success under those conditions. The neighboring dossier on Inclusive Human-System Design and Functional Accessibility develops the wider accessibility research object. Here, accessibility serves as a test of whether a human-centered evaluation is genuinely representative of the system's claimed use and risk profile.
Mobile, Situated, and Cross-Cultural Usability
Mobile interaction introduces environmental variability into each nominally identical task. Screen size, movement, input precision, connectivity, interruptions, competing physical tasks, and social visibility affect what a person can perceive and execute. Harrison, Flood, and Duce Harrison et al. (2013) reviewed mobile usability literature and proposed a model that better accounts for context-sensitive aspects of mobile experience. A website that performs well on a wide desktop screen may show different errors on a crowded train or when the user is switching between a paper document and a phone. Evaluations should report device categories, operating conditions, and whether participants were stationary or mobile.
Language and cultural conventions also influence comprehension. Labels, dates, currency formats, icon conventions, privacy expectations, and the relevance of examples can vary among users. Localization assessment should include meaningful task outcomes rather than only linguistic correctness. However, broad national labels can conceal within-population differences in literacy, domain expertise, disability, and previous technology experience. Researchers should make the population and explanatory variables explicit, avoiding the inference that a single convenience sample represents an entire country or cultural tradition.
Generalization requires either deliberate sampling across contexts or cautious specification of where evidence applies. A successful experiment with high-literacy, frequent technology users may establish a real benefit for that studied group while leaving open whether the change helps people using older devices or unfamiliar terminology. Triangulation across controlled tests and in-context follow-up can reveal the relative contribution of platform, environment, and individual experience. The research agenda is to develop measures that retain contextual meaning while supporting genuine comparison across systems.
Persuasive Interfaces, Dark Patterns, and Ethical Outcomes
An interface directs attention, constrains choices, and influences behavior through defaults, framing, effort, sequencing, and feedback. Human-centered evaluation must ask whose goals define success. A higher purchase conversion rate may serve a vendor while concealing an opt-out route, while a shorter consent process may fail to support comprehension of consequences. Mathur and colleagues Mathur et al. (2019) examined a large sample of online shopping sites and identified numerous patterns that steered consumer decisions through problematic design techniques. Their descriptive findings establish that manipulative interface strategies appear at meaningful scale in a commercial setting; they do not by themselves quantify the long-term harm caused by every pattern.
The ethical evaluation of design requires measures tied to user understanding, voluntariness, anticipated consequences, and the ease of reversing a decision. Participants may report satisfaction immediately after completing a purchase but later discover unwanted recurring charges. An interface may look simple precisely because it conceals material information. Researchers can therefore examine disclosure comprehension, decision quality, unwanted persistence, and patterns of subsequent complaints alongside usability outcomes. Ethical assessment is not an external ornament added after testing: it changes what counts as effectiveness and whose objectives enter the evaluation.
The empirical link between interface design and autonomy can be studied through randomized presentation of choice architectures, measures of comprehension, and delayed follow-up. Such experiments must avoid exposing participants to real financial or privacy harm. The separate publications on dark patterns and persuasive design extend these applications within the Psychology corpus. This dossier retains the evaluative question: how can a design intervention be judged successful when task completion, business metrics, informed choice, and welfare point in different directions?
Human–Artificial Interfaces and Adaptive System Behavior
Interfaces involving artificial intelligence introduce behavior that may be probabilistic, context-sensitive, and updated after deployment. Users need information about what a system has done, which uncertainty remains, and when intervention or independent verification is warranted. Amershi and colleagues Amershi et al. (2019) developed guidelines for human–AI interaction from a structured design process and expert evaluation, organizing recommendations across initial interaction, ongoing use, errors, and long-term adaptation. These guidelines are design resources whose effectiveness in a particular domain still requires behavioral validation with representative users.
A task can be completed through a human–artificial workflow without the human correctly understanding which parts of an answer have been checked. Evaluation must distinguish speed and subjective trust from calibrated reliance, error recognition, recovery, oversight, and final decision accuracy. A design may produce faster accepted recommendations while increasing the number of incorrect decisions that escape detection. Researchers should include cases where the system is wrong, where uncertainty is communicated differently, and where human control can meaningfully change the outcome. The measurable object is the joint performance of the coupled interaction process under declared conditions.
An adaptive system presents special problems for replication. The model version, retrieval corpus, prompts, user context, and output sampling may influence the interface that a participant actually experiences. Studies should record the relevant versions and task instances, control or document variation in system behavior, and evaluate repeated and longitudinal interactions. In the Artificial Era, human-centered design extends to the organization of reciprocal interaction between Homo and Artificial, including the structure of authority, delegation, correction, and continuity across time. These are research directions requiring explicit operational definitions and tested hypotheses; the conclusions of one interface experiment remain bounded by its system and participants.
Designing and Evaluating Complex Workflows
Complex workflows involve dependencies among screens, documents, roles, and external systems. A single-screen test cannot establish that a multistage process is usable from initiation to completion, especially when control passes between people or when information must be preserved across interruptions. Task analysis must therefore include handoffs, reversals, exceptional conditions, and the information required at each transition. Cognitive engineering Card et al. (1983) provides a foundation for specifying individual interaction demands, while human-centered design ISO (2019) requires examination of the broader setting in which those actions serve real goals.
Researchers can evaluate entire task trajectories with trace data that record states, decision points, correction cycles, and the eventual correctness of outcomes. A workflow-level outcome may include the proportion of cases resolved without rework, the number of preventable transfers, time until an error is detected, or the cost of reconstructing missing context. These measures are especially important where several operators rely on a common artifact. Repeated local improvements may increase fragmentation if they optimize one screen at the expense of understanding across the workflow. The system-level hypothesis must be tested with end-to-end scenarios.
Interviews with managers and expert operators can identify required procedures but may overrepresent the idealized workflow. Observation can reveal unofficial notes, temporary spreadsheets, messaging channels, and other external representations that preserve continuity. Such workarounds are evidence of how real activities are accomplished, and changing them may have both benefits and costs. Before declaring an automated replacement more usable, researchers should establish whether the new process supports the work previously carried by those representations. A human-centered evaluation studies the network of tasks and artifacts as a functioning arrangement.
Safety-Critical Design and Consequential Error
When a usability failure can contribute to injury, loss of rights, or irreversible financial harm, an evaluation must move beyond average convenience. The investigator should define critical tasks, foreseeable misuse, rare exceptional states, and independent safeguards. A satisfactory mean success rate may conceal a dangerous subgroup or a failure mode with severe consequences. Errors should be characterized by the chain of events they initiate, their detectability, the availability of recovery, and the time remaining before harm occurs. The measured outcomes must correspond to those hazards rather than only to aesthetic preference.
Controlled simulation can expose failure pathways while protecting participants and affected third parties. In such studies, the scenario must realistically reproduce the information, pressure, and coordination conditions of the intended context. Stronger claims of operational safety require evidence from environments that represent actual use and the relevant range of incidents. Small formative usability studies remain useful for identifying obvious hazards, while their inability to estimate rare-event rates is a central constraint. Faulkner (2003) offers a general illustration of the instability of problem coverage in small samples, and ISO (2018) reinforces the need to specify users, goals, and contexts.
Design changes should be assessed for both error prevention and error recovery. A warning can become ineffective if it appears too often or interrupts routine work without providing useful information. An additional confirmation can lower accidental activation while increasing delay in urgent situations. The empirical question is which configuration yields better outcomes in the actual task environment. A risk-sensitive evaluation records intended and unintended effects, documents residual uncertainty, and treats the absence of detected accidents in a limited study as insufficient evidence that serious failures have been eliminated.
Organizational Adoption, Participation, and Evaluation Governance
Human-centered design is implemented within organizations that set schedules, budgets, selection criteria, and performance incentives. The participation of users can improve the discovery of latent requirements, but the benefits depend on whom the organization invites, what decisions participants can influence, and whether their observations affect subsequent designs. Gould & Lewis (1985) emphasizes active attention to people and empirical learning, while ISO (2019) places those activities within a coordinated life-cycle process. A nominal workshop or one-off survey does not establish that the full process was informed by representative use.
An evaluation governance structure should identify responsibility for recruitment, instrument selection, evidence quality, accessibility, risk review, and sign-off. Conflicts of interest can arise when the people interpreting results are also rewarded for meeting a scheduled release target. Independent review or preregistered acceptance criteria can reduce discretion in consequential decisions. Participants should understand how recordings and behavioral traces will be handled and whether research activities might affect their employment or access to a service. These procedural conditions are relevant to scientific validity because they influence selection, candor, and missing data.
Evaluation results should remain comprehensible outside the design team. A useful report includes the target population, task scenarios, interface version, recruitment and exclusions, observed problems, outcome measures, uncertainty, and prioritized recommendations grounded in evidence. Severity judgments should be distinguishable from observed frequency and from business priority. The organization can then make a transparent decision about tradeoffs, retaining provenance for future auditors and researchers. Repeated evaluation becomes cumulative when the rationale for each change survives the immediate product cycle.
Costs, Benefits, and the Interpretation of Design Improvement
A project commonly seeks to reduce time and support burden while improving completion and satisfaction, but economic benefit is not identical to psychological usability. Additional design and evaluation work incurs cost, and a faster process may shift workload to another person or department. An adequate analysis therefore specifies the perspective from which cost and benefit are assessed. For an organization, relevant outcomes might include avoided rework and support contacts; for a user, the crucial benefit might be reduced uncertainty or freedom to correct an error without losing information. These outcomes may be related and should be measured separately before being combined into a decision.
The expected value of additional testing depends on how likely the test is to reveal problems, how consequential those problems are, and how feasible their correction will be. Small formative studies can be efficient when a prototype contains common observable failures; larger summative studies become more valuable when the release decision requires precise estimates. Virzi (1992) and Faulkner (2003) help explain why there is no universally optimal participant count. Every sampling strategy encodes a tradeoff between cost, problem coverage, estimation precision, and the diversity of users and contexts.
Scientific reporting should describe both intended gains and displaced costs. Suppose an automated form reduces user effort while increasing back-office correction for unusual cases. The task-level usability improvement may be real and still insufficient for a system-wide judgment. Conversely, an additional verification step may slightly increase time but substantially reduce costly mistakes. Human-centered evaluation becomes stronger when it examines the distribution of burdens and benefits across the entire workflow and across populations, rather than optimizing one visible product metric.
Evidence Translation and Design Decisions Across Domains
The same research method can support different conclusions in different domains. In educational technology, a relevant outcome may be the learner's ability to complete a reasoning task independently after assistance is removed. In financial services, it may be comprehension of obligations and correction of entered information. In government services, equitable access and completion without unnecessary help may dominate the evaluation. In digital health, reliable interpretation of instructions and avoidance of consequential errors may require additional domain-specific study. These are examples of candidate outcomes, not evidence that one standardized instrument can settle every decision.
Transferring findings among domains requires inspection of the underlying psychological mechanism and contextual assumptions. A label change that improves recognition in one task might generalize to similar tasks, but its benefit could vanish when vocabulary, user expertise, or risk changes. Hornbæk (2006) demonstrates the importance of clarifying the actual construct under study, while Harrison et al. (2013) shows why a context such as mobile use may alter what usability entails. Strong evidence translation records the rationale for generalization and specifies which measurements would challenge it in the new setting.
An applied research program should favor explicit claims of the form: for this population and task, this change is expected to improve this observable outcome because of this proposed mechanism. The subsequent test can then support, narrow, or reject the claim. Over time, accumulated results may reveal stable design principles and limits of transfer. The purpose of a fundamental dossier is to make those claims more precise and accountable while showing how broader cognitive, social, and organizational processes enter the scientific study of design.
Critical Scientific Discussion of the Evidence Base
The evidence supporting human-centered design is heterogeneous because the field studies many interventions, populations, and outcome types. Standards such as ISO (2019) establish procedural expectations, foundational essays such as Gould & Lewis (1985) articulate explanatory design principles, and experimental studies provide bounded tests of particular changes. These forms of evidence contribute to a cumulative research program without being interchangeable. A standard may prescribe iterative evaluation, while the magnitude of benefit from a specific iteration depends on the observed problem, the intervention, recruitment, and subsequent measurements. A scientifically responsible synthesis distinguishes the normative claim that teams should study actual users from the empirical claim that a documented design change improved specified outcomes.
Several recurrent findings warrant substantial confidence within their tested boundaries. The classic research tradition shows the importance of observing real attempts rather than relying exclusively on expert expectations Lewis & Mack (1982). Inspection methods uncover candidate interface problems, but evaluators disagree even when using established methods Hertzum & Jacobsen (2001). Small samples can uncover common usability problems while differing greatly in coverage of the full set of failures Faulkner (2003). Survey instruments can contribute standardized perceptions, yet their factor structure and domain-specific interpretation require investigation Bangor et al. (2008), Borsci et al. (2009). Each finding supports methodological improvement, and none provides a universal numerical guarantee of design quality.
The evidence is thinner for long-term system-wide consequences, for rare high-severity events, and for the causal effects of design practices considered as entire organizational programs. A collection of successful case studies is vulnerable to selection bias, and results from experienced professional evaluators may not transfer to inexperienced designers. In addition, task success, satisfaction, and economically preferred behavior can diverge when an interface is manipulative or when important outcomes emerge later Mathur et al. (2019). The scientifically defensible conclusion is a set of conditional principles: specify the use context, observe meaningful behavior, measure valid outcomes, test revisions, and evaluate consequences at the level of the actual sociotechnical system.
Alternative Explanations and Threats to Validity
When an interface appears to reduce failures, possible explanations include clearer presentation, simpler task structure, better prior knowledge, more favorable data, participant motivation, or increased facilitator assistance. Randomization and consistent protocols can distinguish some of these mechanisms; direct observation and process tracing can investigate others. In formative research, where causal isolation is often limited, analysts should state rival explanations and the observations that would differentiate them. A list of plausible heuristics should not be treated as proof that the identified heuristic caused an observed error. This boundary between diagnostic theory and empirical confirmation is central to interpretable usability science.
Construct validity is also a recurring concern. A ten-item satisfaction questionnaire can change because of visual attractiveness, expectation, perceived usefulness, or recent task success, while an observed completion rate may miss unauthorized shortcuts. Hornbæk (2006) showed that published studies often operationalized usability incompletely, supporting continued scrutiny of what is actually measured. Reliability may vary with raters, tasks, interface versions, and language. The evaluator effect Hertzum & Jacobsen (2001) makes independent replication and explicit criteria particularly important when methods rely on subjective classification of failures.
External validity depends on ecological correspondence. The laboratory affords control but may omit interruption, organizational hierarchy, consequences of mistakes, and months of use. A field study may reveal these pressures but provide weaker control of competing causes. Strong triangulation combines complementary studies without erasing their differences. The decisive question is what an inference establishes for a particular user–task–context configuration and which additional design would justify extending it beyond that configuration.
Research Gaps and a Cumulative Scientific Agenda
The literature leaves important questions about the relation between usability metrics and human welfare. Researchers need better methods to connect observable efficiency with comprehension, autonomy, learning, and downstream effects. These outcomes evolve over different timescales and may have different distributions across populations. A promising agenda is to combine preregistered task-based comparisons with longitudinal observation and qualitative accounts of changed work practices. Such programs can identify whether immediate gains persist, disappear, or generate secondary burdens.
Sampling and uncertainty require more consistent reporting. The five-user debate Virzi (1992), Faulkner (2003) demonstrates how discovery depends on the probability and distribution of individual problems. Future studies should estimate how problem detection varies by severity, expertise, accessibility mode, device, task complexity, and prior experience. Method comparisons should also treat investigator skill, training, and evaluator agreement as measurable factors rather than unrecorded background conditions Hartson et al. (2001). Open protocols, anonymized behavioral datasets, and explicit classifications of failures can make independent verification more attainable.
Human–artificial systems create additional research questions concerning unstable outputs, delegated authority, the calibration of reliance, adaptive feedback, and the preservation of a person's ability to recognize and correct an error. Amershi et al. (2019) supplies an influential guideline framework, while deployment-specific evidence remains essential. A future program should test how transparency, reversibility, persistent context, and uncertainty communication affect task correctness and autonomy across repeated interactions. This extension of human-centered design belongs to the wider Artificial Era: Homo and Artificial participate in consequential systems whose organization can be observed and evaluated through explicit psychological and engineering measures.
An Integrated Protocol for Evidence-Based Iterative Evaluation
A cumulative research program begins with the scientific identification of its object. The investigator records the user populations, practical goals, tasks, environment, interface version, and anticipated consequences of failure. Field evidence then establishes which problems are experienced and what mechanisms might explain them. Task analysis and inspection identify candidate causal pathways, while prototype studies observe whether users interpret and act upon relevant information. Each hypothesis is translated into one or more measurable outcomes so that an intervention can be evaluated rather than merely exhibited.
Early formative tests emphasize discovery, documentation, and correction. Researchers record complete attempts, intervention by moderators, breakdowns, recovery sequences, and participant interpretations. They repeat studies with revised designs and new participants when feasible, following the iterative tradition of Gould & Lewis (1985). Once a design has stabilized, a summative assessment uses explicit eligibility rules, meaningful comparison conditions, sufficiently precise estimates, and transparent uncertainty reporting. Satisfaction scales such as Brooke (1996) or post-task ratings Sauro & Dumas (2009) supplement task performance; they do not replace it.
Finally, the design is evaluated as part of an ongoing organization of work. Accessibility and inclusion are assessed with applicable standards W3C (2024) and representative users; unintended persuasion is monitored through comprehension and choice outcomes Mathur et al. (2019); adaptive human–artificial systems are evaluated for joint reliability and effective oversight Amershi et al. (2019). Reports preserve the source of each claim, the interface state to which it applies, and the limits of generalization. This integrated protocol offers a practical scientific synthesis: observe, hypothesize, intervene, measure, explain, and revise while retaining the evidence required to assess what was learned.
Conclusion
Human-centered design is a disciplined research and design process organized around people pursuing specific goals in particular contexts. Its scientific substance lies in linking psychological explanations of interaction to observations of actual task performance and to repeated assessments of candidate solutions. The history of the field brings together cognitive models of action, research on novice learning, inspection methods, empirical usability testing, and organizational processes for iteration. Contemporary standards provide a shared vocabulary for effectiveness, efficiency, satisfaction, context, and life-cycle evaluation, while original studies reveal both the value and the limitations of particular methods.
The major synthesis is methodological and explanatory. Usability depends on the fit among user capabilities, task demands, interface representations, environmental conditions, and the ability to recover from failure. Representative observation can expose mechanisms of difficulty; quantitative metrics can estimate selected outcomes; psychometric instruments can measure perceptions; and controlled comparisons can test whether a proposed modification produces the intended effect. Reliable conclusions require scrutiny of sampling, measurement validity, evaluator variability, competing explanations, uncertainty, generalization, and consequences beyond the immediately observed interaction. Iteration gains scientific meaning when each revision creates an opportunity to learn which explanation and intervention actually worked.
The Artificial Era extends these questions to interactions in which artificial systems generate recommendations, adapt to human behavior, and participate in consequential workflows. The continuing research problem is to determine how people and artificial systems can accomplish meaningful goals with accountable performance, accessible participation, recoverable action, and informed control. Human-centered design therefore remains an empirical discipline of evolving human–system relations. Its cumulative strength comes from maintaining traceable connections among the research question, the design intervention, the observed result, and the conditions under which that result is valid.
References
- Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E. (2019). Guidelines for human-AI interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Article 3, 1–13. https://doi.org/10.1145/3290605.3300233
- Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the System Usability Scale. International Journal of Human–Computer Interaction, 24(6), 574–594. https://doi.org/10.1080/10447310802205776
- Bastien, J. M. C., & Scapin, D. L. (1995). Evaluating a user interface with ergonomic criteria. International Journal of Human–Computer Interaction, 7(2), 105–121. https://doi.org/10.1080/10447319509526114
- Borsci, S., Federici, S., & Lauriola, M. (2009). On the dimensionality of the System Usability Scale: A test of alternative measurement models. Cognitive Processing, 10(3), 193–197. https://doi.org/10.1007/s10339-009-0268-9
- Brooke, J. (1996). SUS: A 'quick and dirty' usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability evaluation in industry (pp. 189–194). Taylor & Francis. https://doi.org/10.1201/9781498710411-35
- Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of human-computer interaction. Lawrence Erlbaum Associates. https://www.routledge.com/The-Psychology-of-Human-Computer-Interaction/Card-Moran-Newell/p/book/9780898598599
- Dumas, J. S., & Redish, J. C. (1993). A practical guide to usability testing. Ablex. https://search.worldcat.org/title/28584112
- Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 379–383. https://doi.org/10.3758/BF03195514
- Gould, J. D., & Lewis, C. (1985). Designing for usability: Key principles and what designers think. Communications of the ACM, 28(3), 300–311. https://doi.org/10.1145/3166.3170
- Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile applications: Literature review and rationale for a new usability model. Journal of Interaction Science, 1, Article 1. https://doi.org/10.1186/2194-0827-1-1
- Hartson, H. R., Andre, T. S., & Williges, R. C. (2001). Criteria for evaluating usability evaluation methods. International Journal of Human–Computer Interaction, 13(4), 373–410. https://doi.org/10.1207/S15327590IJHC1304_03
- Hertzum, M., & Jacobsen, N. E. (2001). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human–Computer Interaction, 13(4), 421–443. https://doi.org/10.1207/S15327590IJHC1304_05
- Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79–102. https://doi.org/10.1016/j.ijhcs.2005.06.002
- International Organization for Standardization. (2018). Ergonomics of human-system interaction—Part 11: Usability: Definitions and concepts (ISO 9241-11:2018). https://www.iso.org/standard/63500.html
- International Organization for Standardization. (2019). Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems (ISO 9241-210:2019). https://www.iso.org/standard/77520.html
- Lazar, J., Feng, J. H., & Hochheiser, H. (2017). Research methods in human-computer interaction (2nd ed.). Morgan Kaufmann. https://shop.elsevier.com/books/research-methods-in-human-computer-interaction/lazar/978-0-12-805390-4
- Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57–78. https://doi.org/10.1080/10447319509526110
- Lewis, C., & Mack, R. (1982). Learning to use a text processing system: Evidence from 'thinking aloud' protocols. Proceedings of the 1982 Conference on Human Factors in Computing Systems, 387–392. https://doi.org/10.1145/800049.801817
- Mathur, A., Acar, G., Friedman, M. J., Lucherini, E., Mayer, J., Chetty, M., & Narayanan, A. (2019). Dark patterns at scale: Findings from a crawl of 11K shopping websites. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), Article 81, 1–32. https://doi.org/10.1145/3359183
- Nielsen, J. (1994). Enhancing the explanatory power of usability heuristics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 152–158. https://doi.org/10.1145/191666.191729
- Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 249–256. https://doi.org/10.1145/97243.97281
- Norman, D. A., & Draper, S. W. (Eds.). (1986). User centered system design: New perspectives on human-computer interaction. Lawrence Erlbaum Associates. https://www.routledge.com/User-Centered-System-Design-New-Perspectives-on-Human-computer-Interaction/Norman-Draper/p/book/9780898598728
- Polson, P. G., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, 36(5), 741–773. https://doi.org/10.1016/0020-7373(92)90039-N
- Sauro, J., & Dumas, J. S. (2009). Comparison of three one-question, post-task usability questionnaires. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1599–1608. https://doi.org/10.1145/1518701.1518946
- Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34(4), 457–468. https://doi.org/10.1177/001872089203400407
- World Wide Web Consortium. (2024). Web Content Accessibility Guidelines (WCAG) 2.2 (W3C Recommendation, updated December 12, 2024). https://www.w3.org/TR/WCAG22/