RESEARCH DOSSIER
Digital Mental Health and Human–Artificial Clinical Systems
Abstract. Digital mental health and human–artificial clinical systems comprise technologies, institutional pathways, and human decision processes used to assess mental health, route care, support clinical judgment, and maintain continuity across remote and algorithmically mediated encounters. This fundamental research monograph synthesizes original trials, psychometric validation, digital phenotyping research, prediction-model studies, systematic reviews, and official regulatory and reporting frameworks. It traces the development of digital psychiatric services; distinguishes measurement validity from diagnostic and operational utility; compares telepsychiatry, structured digital programs, conversational systems, and clinical decision support; and examines calibration, triage, equity, privacy, adverse-event monitoring, and accountability. Evidence supports selected forms of remote care and structured digital intervention under studied conditions, while general-purpose generative systems present separate questions of deployment reliability, external validation, crisis response, and clinical governance. The synthesis proposes an evaluation hierarchy linking analytic validity, prospective clinical benefit, patient-relevant outcomes, subgroup equity, and continuous safety surveillance. Research priorities include cross-site pragmatic trials, prospective silent-mode evaluation, longitudinal drift monitoring, and meaningful patient participation in governance.
Keywords: digital mental health; telepsychiatry; digital psychiatry; clinical decision support; human–artificial clinical systems; mental health screening; digital phenotyping; machine learning; psychiatric triage; algorithmic fairness; clinical validation; generative artificial intelligence; suicide risk prediction; patient safety; implementation science; Artificial Era
The Scientific Object: Digital Mental Health as a Clinical System
Digital mental health is the organized use of networked communication, digital measurement, software, and computational analysis within activities intended to assess, maintain, or improve mental health. A human–artificial clinical system is a particular configuration in which patients, clinicians, software, algorithms, institutions, and supporting personnel exchange information or decisions that affect clinical care. Its scientific unit is the complete care episode rather than the isolated model output. A screening response, for example, can become a recommendation, a referral, a conversation, a recorded decision, or an event without follow-up. Research must establish what happened after the output and which participants could revise its implications. Historical accounts of behavioral intervention technologies and more recent World Health Organization strategies situate digital health within service delivery, organizational resources, and social conditions rather than treating it as an application category Mohr et al. (2013) WHO (2025).
The clinical functions of a digital system must be explicitly identified. Measurement estimates a person's state using defined instruments or behavioral traces; screening identifies people warranting further assessment; prediction estimates the probability of a specified future outcome; decision support presents information for professional judgment; triage assigns priority or routes people to care; intervention attempts to change symptoms, behavior, or functioning; and documentation preserves an account of encounters. These functions differ in target populations, standards of evidence, thresholds for harmful error, and the locations of human responsibility. Calling each function artificial intelligence obscures those distinctions. The United States Food and Drug Administration's current decision-support guidance is structured around software functions and the conditions governing them, while the World Health Organization emphasizes intended use and governance of large multimodal models FDA (2026) WHO (2024).
The primary research questions are how digital clinical measurements are validated, how algorithmic recommendations change decisions, whether mediated pathways improve patient-relevant outcomes, and how institutions detect and respond to harm. Every claim about system quality requires specifying the condition, comparison group, time horizon, clinical setting, and outcome. A model might classify retrospectively collected interview notes accurately yet perform poorly when its output is embedded in a hurried emergency service. A videoconferencing pathway may extend access while some individuals remain excluded by connectivity, privacy, language, or disability constraints. These are questions about the relations among person, technology, and institution. They can be investigated through controlled studies, ethnography, implementation research, longitudinal follow-up, and audits of actual care pathways Hilty et al. (2013) NICE (2022).
This dossier concentrates on clinical validation and safety of digital screening, AI-supported differential decisions, triage, telepsychiatry, and associated human–artificial workflows. Digital psychotherapy protocols and detailed mechanisms of specific treatments receive discussion only when necessary for comparing clinical pathways; their independent therapeutic science belongs in neighboring research. Similarly, social attachment to artificial companions is relevant here when it changes health-seeking or disclosure behavior, but is a separate longitudinal research object. The boundary prevents an attractive conversational interface from standing as evidence of diagnostic reliability or therapeutic efficacy. The Artificial Era supplies a historical context for expanded nonbiological participation in knowledge work; observed clinical benefits and risks remain empirical questions evaluated with the same scientific discipline applied to any consequential health technology.
From Telemedicine to Computationally Mediated Psychiatry
The history of digital mental health precedes recent language-model systems. Clinical consultation over telecommunications, computerized questionnaires, remote follow-up, and software-assisted behavioral programs established several distinct research traditions. Telepsychiatry created a setting in which clinical communication could occur without physical co-presence. Computerized and internet-supported interventions created repeatable therapeutic content with measurable enrollment, adherence, and symptom change. Mobile devices later supplied sensors, ecological questionnaires, and intermittent observations across everyday contexts. Hilty and colleagues' 2013 review examined evidence for telemental health across clinical applications, while Mohr and colleagues in the same period framed behavioral intervention technologies around clinical evidence and the need for stronger design and evaluation Hilty et al. (2013) Mohr et al. (2013).
The smartphone era enlarged both the potential observation window and the difficulty of interpretation. A telephone-based symptom survey can be administered according to a standardized schedule, whereas continuous passive data collection can depend on proprietary operating systems, battery management, personal routines, device ownership, and platform revisions. Onnela and Rauch described digital phenotyping as the moment-by-moment quantification of individual-level human behavior through personal digital devices, emphasizing the methodological prospects of temporally dense measurements Onnela & Rauch (2016). Subsequent research on social-media prediction highlighted ethical issues involving consent, construct validity, and generalization from online expression to clinical conditions Chancellor & De Choudhury (2020). A longer observation period is valuable only when the resulting data preserve interpretable links to the phenomenon under investigation.
The emergence of conversational agents connected intervention technology to social interaction. The randomized pilot by Fitzpatrick, Darcy, and Vierhile tested a fully automated cognitive-behavioral conversational agent in young adults reporting depressive or anxious symptoms over a short period Fitzpatrick et al. (2017). That experiment belongs to a specific historical design: structured conversational material, a selected population, short follow-up, and a defined comparator. Later systems based on generative models make open-ended responses and new failure modes possible. Heinz and colleagues evaluated a specifically fine-tuned generative mental-health chatbot in a randomized trial, and Shoshani and colleagues subsequently compared a conversational AI intervention with face-to-face group therapy and a waiting-list condition among university students Heinz et al. (2025) Shoshani et al. (2026). The chronology demonstrates increasing intervention complexity and the need to keep each product's design, population, and evidence distinct.
Professional and public institutions developed evaluation frameworks alongside these technological changes. WHO issued guidance on digital interventions and later formal governance guidance for large multimodal models; NICE's evidence standards framework was updated for adaptive data-driven technologies; the FDA clarified its classification approach to clinical decision-support software WHO (2019) WHO (2024) NICE (2022) FDA (2026). These frameworks have different jurisdictions and normative roles. Their shared scientific contribution is to direct assessment toward intended use, risk, evidence proportionate to clinical consequences, and oversight. A timeline is useful because the validation of remote psychiatrist–patient video visits cannot automatically validate generative autonomous triage, although both are routinely described under the same marketing category.
Conceptual Boundaries: Screening, Diagnosis, Prediction, and Triage
A screening instrument classifies people by a threshold intended to identify who may benefit from additional assessment. Diagnosis integrates reported experience, clinical history, examination, differential considerations, context, and recognized criteria. A prediction model estimates a future or otherwise unobserved outcome, while triage translates available information into the timing, destination, and intensity of care. These functions are connected, yet each adds inferential and operational steps that demand separate validation. A symptom total crossing a screening cutoff can initiate an assessment; it does not by itself settle whether a clinical disorder is present. The empirical work establishing the Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7 demonstrates the importance of defined items, scoring procedures, and validation populations Kroenke et al. (2001) Spitzer et al. (2006).
Several distinct performance quantities are commonly conflated. Sensitivity quantifies the fraction of cases identified among those with the target condition; specificity quantifies correctly excluded noncases. Positive predictive value asks what proportion of people with positive results actually have the specified condition or outcome, given the population's event prevalence. Calibration describes agreement between predicted and observed probabilities, and discrimination describes separation between those with and without outcomes. Decision thresholds determine referral volume, missed cases, assessment burden, and possible harm. A receiver operating characteristic statistic cannot alone establish safe deployment, especially when the clinical event is rare. Belsher and colleagues' simulation and systematic review of suicide prediction models demonstrated the gap between impressive global classification performance and very low positive predictive value for suicide mortality Belsher et al. (2019).
Clinical utility requires a specified action following a prediction. If no staff member can review an alert, a screening system may increase distress and waiting times without increasing effective care. If every positive result triggers invasive intervention, a modest specificity deficit can produce disproportionate burdens. If negative screens bypass clinical judgment, evolving symptoms and missed disclosures can be systematically ignored. Therefore, a clinically meaningful evaluation states who receives the output, what options exist, which action is triggered, whether patient preferences are considered, and what comparison pathway would occur without the system. Reporting standards for AI prediction models explicitly require sufficient information for readers to assess intended use and limitations Collins et al. (2024).
The language of human–artificial participation is operational here. An artificial component may compute risk, generate a patient summary, pose a screening question, translate clinical notes, or recommend a referral. A clinician may inspect, reject, amend, contextualize, or endorse that output. A patient may disclose, correct, withdraw consent, or seek an alternative form of contact. The investigable object is the allocation and performance of these activities under institutional rules. Whether an artificial system is described philosophically as an actor belongs to a separate ontological debate. Patient safety can be evaluated with recorded decisions, trajectories, adverse outcomes, and responsibility chains without depending on a conclusion about machine subjectivity.
Evidence Synthesis and the Hierarchy of Claims
This monograph is a critical, evidence-informed scholarly synthesis of primary investigations, systematic reviews, reporting standards, and institutional guidance. It is not presented as a new systematic review with a registered protocol or as a meta-analysis. The source set is organized by clinical function, study design, and inference: psychometric validation for measurement; observational and prospective studies for prediction; randomized and pragmatic studies for interventions and service configuration; simulation and red-team studies for dangerous system failures; and regulatory guidance for the institutional governance of intended uses. The approach seeks explanatory completeness and explicit distinctions among evidence classes rather than aggregating heterogeneous results into a misleading overall effect.
Original clinical trials are examined for allocation, comparator, eligibility criteria, outcomes, follow-up, adherence, and the presence of an independent clinically meaningful endpoint. Meta-analyses help identify the direction and range of effects across digital interventions, but trials of structured cognitive-behavioral content cannot simply be pooled conceptually with freely generative conversations. Firth and colleagues analyzed randomized studies of smartphone interventions for depressive and anxiety symptoms, while a later large meta-analysis investigated persistence of effects from internet-delivered cognitive-behavioral therapy beyond the immediate treatment period Firth et al. (2017a) Firth et al. (2017b) Zainal et al. (2024). Their findings contribute to the literature on digital treatment, but their populations and software modalities delimit what the conclusions can support.
Research on prediction and algorithmic evaluation demands special attention to dataset construction. A training corpus, a cross-validation split, and an external site each answer different questions; reused patients, duplicated text, data leakage, or measurement changes can inflate performance. Research often measures discrimination in historical records despite a clinical objective requiring transport to different service contexts. The methodological standard articulated in TRIPOD+AI supports transparent reporting of population, predictors, outcomes, missingness, model fitting, validation, and uncertainty Collins et al. (2024). For diagnostic classification, STARD-AI addresses the characteristics of accuracy studies, while DECIDE-AI adds early real-world evaluation of clinical decision support Sounderajah et al. (2025) Vasey et al. (2022).
The quality of an evidence claim also depends on the distance from the measured outcome to patient benefit. A benchmark pass demonstrates performance under a benchmark procedure. A clinician agreement score demonstrates concordance with a reference judgment. A prospective study of changed clinician actions demonstrates behavioral influence. A randomized care-pathway trial can estimate the consequences of that influence. Longer-term observation is needed for durability, delayed harm, and adaptation to evolving practice. No single design answers all questions. The critical scientific task is to preserve the inferential chain while recognizing that regulatory authorization, institutional adoption, and an attractive interface each supply information of a different kind NICE (2022) WHO (2024).
A Systems Model of Human–Artificial Clinical Work
An integrated clinical episode contains at least seven distinguishable stages: disclosure or observation; data acquisition; transformation into a score or model output; interpretation of that result; assignment of action; delivery of human or digital care; and outcome monitoring. Each stage introduces potential information loss, delay, ambiguity, or inequity. A person using a mobile device may record a survey in an unsafe location; a system may summarize a statement inaccurately; a clinician may rely on a fluent explanation; a referral may be correct but inaccessible; and a subsequent patient deterioration may remain invisible because the service has no follow-up protocol. System evaluation must therefore specify dependencies among stages rather than attributing every observed outcome to the accuracy of one algorithm.
A useful representation is the conditional pathway: observed clinical outcome depends on patient characteristics and needs, measurement conditions, software version, clinicians' decisions, service capacity, and subsequent treatment access. This is an analytic model to guide empirical investigation, not an estimate of causal effects without data. The variables can be operationalized through patient interviews, questionnaire records, interaction logs, timestamps, appointment completion, clinician overrides, incident reports, and symptom follow-up. Variation across deployment sites offers information about transportability; carefully planned randomized allocation or phased implementation improves causal inference. Attributing improvements solely to an algorithm, while a new referral team and shorter waiting times are introduced simultaneously, confounds technology with organizational change.
Human oversight becomes meaningful when it changes observable control over the pathway. Researchers should measure clinicians' ability to inspect the relevant input, identify uncertainty, modify the proposed action, document disagreement, and respond when automation fails. A nominal approval button can create apparent accountability without a realistic opportunity for inspection. Conversely, a service may adopt digital tools that remove administrative burden and allow more patient time when interfaces and incentives are aligned with clinical practice. The FDA's clinical decision-support distinction and WHO's governance recommendations both direct attention toward the conditions under which people can understand and appropriately supervise consequential software FDA (2026) WHO (2024).
The systems model implies a hierarchy for scientific claims. First establish technical integrity and analytic validity; then establish validity of measurement against an appropriate reference; then show prospective performance under workflow constraints; then demonstrate a net benefit for patients under an appropriate comparator; and finally maintain surveillance for drift and harm. Movement through this hierarchy is not a universal regulatory sequence. It is an explanatory discipline that helps distinguish a promising component from a clinically validated service. The forthcoming chapters organize the evidence around those transitions and around the human, organizational, and artificial capacities required to sustain trustworthy clinical work.
Psychometric Validity in Digital Clinical Assessment
Psychometric validation begins by defining the construct being measured and the decisions to which the measurement may contribute. Depression severity, generalized anxiety symptoms, distress, functional impairment, suicide ideation, and diagnostic status are different constructs even when the same person exhibits several of them. A questionnaire score is a fallible observation of a defined response pattern at a given time. Digital administration can preserve item wording and scoring but can alter mode, context, completion behavior, accessibility, and the willingness to disclose sensitive experiences. The original PHQ-9 and GAD-7 validation investigations provide indispensable reference points for score interpretation, yet a score implemented in a new medium still requires evidence concerning equivalence, missing responses, sampling, and intended clinical use Kroenke et al. (2001) Spitzer et al. (2006).
A basic validation study should report reliability, agreement with clinically appropriate assessments, subgroup functioning, and performance for the exact decision rule proposed. Internal consistency, test–retest stability, and agreement between digital and clinician formats answer different questions. A stable score can reliably measure a biased construct; an accurate average estimate can obscure individual-level misclassification; and identical scores can prompt different actions when service resources vary. Thresholds selected retrospectively often look better than thresholds fixed in advance. Therefore researchers should prespecify outcomes, characterize the sampled population, evaluate missingness, and conduct external replication before using a threshold to automate escalation or discharge Collins et al. (2024) Sounderajah et al. (2025).
Screening in clinical settings differs from open internet self-screening because disease prevalence, referral selection, treatment history, language, and access to immediate care differ. The base rate of a target condition changes predictive values even when sensitivity and specificity remain constant. In addition, people using a tool for self-exploration may respond differently from those answering after a clinician has explained its function. Privacy concerns can suppress disclosure in shared households, workplaces, or environments where a user fears records will be revealed. Investigators should record how screening was introduced, how the result was explained, and whether patients could correct errors or seek human assessment. The data-sharing audit of mental-health applications underscores why privacy is part of measurement validity and informed participation Huckvale et al. (2019).
The clinical interpretation of digital assessment should remain calibrated to the evidence. A high questionnaire score can indicate symptom burden; a diagnostic formulation needs history, impairment, context, alternative explanations, and the patient's own account. Repeated scores can show trajectories but can also be influenced by practice, response shifts, medication changes, crisis resolution, or selection among users who continue engaging. Clinical systems can use repeated measurement effectively when they preserve the response context and establish what actions follow changes. A traceable link from patient response to recommendation, clinician assessment, and follow-up is therefore a necessary research object distinct from the numerical properties of the instrument itself.
Digital Phenotyping, Passive Data, and Behavioral Inference
Digital phenotyping refers to the measurement of behavior or related proxies using personal digital devices over time. Potential observations include mobility patterns, interaction timing, sleep-related phone activity, communication frequency, and prompted self-report. Such streams may generate hypotheses about changes in activity or social contact, yet the relation between a sensor feature and a clinical state is conditional on employment, mobility restrictions, geography, device settings, disability, and habitual usage. Onnela and Rauch outlined the scientific promise of these data streams; their conceptual argument also creates an obligation to test the interpretation of every proxy rather than treating sensor abundance as an independent guarantee of clinical insight Onnela & Rauch (2016).
Temporal density changes the statistical problem. Hundreds of daily observations from one individual do not yield hundreds of independent participants, and correlated measurements can produce misleadingly narrow uncertainty intervals when analyzed as independent records. Within-person changes and between-person differences require separate models. A decline in message frequency may mean withdrawal for one person, work concentration for another, and a broken notification service for a third. Dynamic models should incorporate device metadata, day-of-week effects, missingness patterns, intervention timing, and alternative explanations, and they should be tested prospectively for the specific outcome they purport to anticipate. These requirements become more pressing when the predictions influence clinical contact or an emergency alert.
Digital traces acquired for nonclinical purposes pose additional validity and consent problems. Social-media language provides contextual signals shaped by audience, platform norms, identity presentation, and moderation policies. Chancellor and De Choudhury's critical review of mental-health prediction methods found conceptual and methodological problems in the conversion of digital expression into health labels Chancellor & De Choudhury (2020). Research designs must distinguish clinician-confirmed diagnoses, self-reported symptoms, self-disclosure in profile text, and proxy labels inferred from online communities. Classification accuracy against a weak label is evidence about reproducing that label; it is not evidence of reliable clinical diagnosis.
The ethical architecture of passive sensing is part of scientific feasibility. Participants should understand which signals are collected, for what intended purpose, for how long, who can access them, and what happens if an unusual pattern appears. An algorithm that detects a hypothetical risk but triggers a poorly explained response may harm autonomy and trust. Privacy-preserving aggregation can reduce some exposure while limiting a clinician's ability to inspect the reasons for an individual alert; more granular access may improve review while increasing security risk. The defensible design is task-specific: establish the minimum necessary data, evaluate whether that data improves a consequential outcome, and implement a credible process for correction, withdrawal, and incident review Wykes et al. (2019) WHO (2024).
Risk Models, Rare Events, and the Problem of Positive Prediction
Risk prediction is attractive because many psychiatric harms are serious and relatively infrequent, but low event prevalence creates demanding conditions for positive prediction. In a population of 10,000 individuals with a hypothetical 1% outcome prevalence, a model with 90% sensitivity and 90% specificity would identify 90 true positives and 990 false positives; the positive predictive value would be about 8.3%. This arithmetic example illustrates a general base-rate property rather than reporting an observed model. For rare outcomes, high discrimination may coexist with a low probability that any particular alert identifies someone who will experience the outcome. Therefore, an intervention justified by risk output requires evidence about the consequences of responding to false positives and false negatives.
Belsher and colleagues reviewed 17 cohort studies yielding 64 distinct models for suicide attempts or deaths, representing more than 14 million participants across five countries. They reported that overall classification accuracy was often high while positive predictive validity for suicide mortality was extremely low in most evaluated models Belsher et al. (2019). Their result challenges the practice of presenting a single accuracy or area-under-curve estimate as proof of clinical preparedness. It also requires nuance: models can support population-level research or structured resource assessment even when they are unsuitable as autonomous predictors of an individual future event. The proposed clinical action determines what performance threshold could be acceptable.
Bias enters prediction through labels, predictors, treatment access, and historical decisions. The influential general-health investigation by Obermeyer and colleagues showed how an algorithm trained to predict health spending could reproduce racial inequities when costs were used as a proxy for health needs Obermeyer et al. (2019). That paper did not evaluate psychiatric triage directly; its mechanism is relevant because mental-health utilization also depends on insurance, service availability, stigma, language, trust, and prior access. A model trained to predict past psychiatric admissions may learn access barriers as much as underlying need. Researchers should test alternative outcome definitions and examine whether the model changes the distribution of beneficial services when deployed.
Threshold evaluation must be integrated with the health service's capacity. Raising sensitivity creates additional alerts that need staffed review; reducing false positives may miss individuals who need timely contact. Decision-curve or comparable utility analyses can make assumptions about benefits and harms explicit, though the necessary weights may vary among patients and settings. Retrospective fitting should be followed by external validation and prospective observation of how clinicians and patients respond to outputs. The strongest clinical claim is made only after measuring relevant outcomes such as timely assessment, patient safety, reduced delay, or net benefit rather than merely reproducing existing labels Collins et al. (2024) Vasey et al. (2022).
Clinical Language Models: From Extraction to Interpretation
Natural-language systems can assist mental-health care by extracting information, summarizing encounters, drafting referral letters, retrieving medication instructions, or proposing questions for further assessment. These functions impose distinct standards. A transcription component should preserve what was spoken; a summary should accurately represent salient clinical facts and temporal qualifiers; a differential-diagnosis suggestion should identify uncertainty and plausible alternatives; and a crisis message should handle risk-sensitive context. A fluent output alone supplies limited evidence for any of these capabilities. WHO's governance guidance identifies inaccurate, incomplete, biased, and persuasive content as concerns for clinical use of large multimodal models WHO (2024).
Clinical summarization deserves specific attention because omissions can be less visible than invented information. The absence of a recorded history of violence, self-harm, medication allergy, or protective factors may be interpreted later as evidence that none was present. Evaluation should therefore measure factual fidelity, omission of clinically salient information, provenance to source statements, uncertainty representation, and opportunities for patient and clinician correction. Synthetic test cases can expose failures; independent review of authentic de-identified records, with suitable protections, is necessary to establish transportability. The output must be evaluated against a clinically meaningful source record rather than against another unverified model's answer.
General-purpose language models also produce diagnostic impressions. Such studies often use vignette examinations or selected case descriptions; these can reveal performance under a tightly controlled task but cannot substitute for real encounters, longitudinal information, and clinically managed uncertainty. Validation should include ambiguous symptoms, comorbidity, diverse language use, contradictory records, incomplete data, and changes in the prompt or model version. Diagnostic-accuracy reporting standards and AI-specific model-reporting guidance offer relevant structures for describing samples, reference standards, thresholds, uncertainty, and implementation conditions Sounderajah et al. (2025) Collins et al. (2024). Agreement with a clinician panel requires examining panel reliability and the possibility that the panel itself lacks complete information.
Model versioning is a clinical methodological requirement. The behavior of a conversational system can change after updates to its foundation model, retrieval index, safety instructions, temperature settings, interface, or moderation layer. A result obtained with a fixed prompt and historical model cannot authorize a claim about every later configuration. Investigators should preserve a reproducible test environment, record model identifiers, explain external tools and retrieval sources, and reassess critical functions after substantive changes. Regulatory classification also depends on intended use and design of the software function rather than on a blanket label such as 'AI assistant' FDA (2026).
Algorithmic Fairness, Culture, and Transportability
Digital mental-health assessments encounter populations with differing languages, idioms of distress, educational experiences, disabilities, cultural explanatory models, and access to care. A model developed in one service may have different error rates elsewhere because symptom expression, interview practice, prevalence, and referral thresholds change. Transportability is not merely demographic representativeness in a training dataset; it concerns the stability of the relationship between observed inputs, clinical reference judgments, available care, and actual outcomes. External validation should therefore recruit new institutions and settings and report the performance of important subgroups with uncertainty intervals Collins et al. (2024).
A common fairness error is to compare only overall mean scores. The same global discrimination statistic can hide large differences in false-negative rates, calibration, missingness, or access to follow-up. Group labels themselves may be heterogeneous, and overly fine subgroup reporting may breach privacy when cells are small. Researchers must choose measures linked to the intended clinical consequence and assess whether a model makes an existing inequity better or worse. The population-health algorithm case analyzed by Obermeyer and colleagues illustrates the need to inspect how socioeconomic access patterns enter outcome proxies, even when a system was trained without an explicit demographic feature Obermeyer et al. (2019).
Language-model triage adds further questions about dialect, code-switching, metaphor, indirect expressions of distress, cultural background, and contextual ambiguity. A system that recognizes a canonical English phrase may miss a colloquial description or misinterpret a figurative statement as an immediate crisis. Evaluation datasets should include naturalistic diverse utterances and clinician-developed reference categories, with care taken that patient voices are represented without exposing sensitive data. Human adjudication is especially important for culturally specific cases and for disagreement about the meaning of context. A multilingual interface needs evidence for each intended language, not an untested assumption that conversational ability generalizes.
Fairness should be studied at the level of outcomes and institutional response. A well-calibrated risk score cannot equalize access when one community has few clinicians, unstable broadband, or restricted ability to attend referrals. Technology designed around continuous smartphone ownership may exclude people who could otherwise benefit most. Evaluations should therefore document device and connectivity requirements, alternative communication channels, patient choice, referral completion, and reasons for disengagement. WHO's strategy locates digital systems within health infrastructure and equity, while NICE's evidence framework links evidence expectations to the function and consequences of digital health technologies WHO (2025) NICE (2022).
Telepsychiatry and Remote Clinical Encounters
Telepsychiatry includes synchronous video consultation, telephone contact, asynchronous communication, remote assessment, and combinations with in-person care. Each modality alters communication, access, privacy, documentation, and possible examination. Evidence about live video psychiatrist–patient encounters applies most directly to settings in which a trained clinician remains responsible for assessment and treatment. Hilty and colleagues' review surveyed research across multiple telemental-health applications and established the longer evidentiary history of remote psychiatric service delivery Hilty et al. (2013). The clinical mechanism of an online visit is partly continuity of an established therapeutic activity across a different communication channel; this differs materially from replacing the clinician with an open-ended text generator.
A multisite pragmatic randomized controlled trial in Japan compared a pathway with at least half its visits delivered by two-way video with entirely face-to-face treatment for depressive, anxiety, and obsessive-compulsive disorders. Kishimoto and colleagues enrolled 199 adults receiving ongoing treatment over 24 weeks and reported noninferiority on the SF-36 mental component summary Kishimoto et al. (2024). This supports the studied configuration of supervised outpatient follow-up among eligible patients judged stable enough for participation. It does not establish that every psychiatric emergency, first diagnostic encounter, or population lacking a private connection can receive equivalent remote management. The inclusion criteria, participating services, outcome definition, and treatment mix define the limits of the trial.
Therapeutic communication through video introduces variations in nonverbal cues, connection stability, turn-taking, environmental privacy, and the patient's control over location. In a meta-analysis covering 31 studies and 34 independent samples with 4,862 participants, Aafjes-van Doorn and colleagues estimated a small positive association between therapeutic alliance and outcomes in teletherapy, with meaningful heterogeneity across studies Aafjes-van Doorn et al. (2024). This finding concerns patient–therapist relationships mediated by telecommunications; it does not validate an equivalently named relationship between a patient and artificial software. The distinction matters because a digital interface can ask sensitive questions with humanlike language while lacking the professional responsibilities and longitudinal interpretive capacities of a clinician.
A clinically complete remote pathway needs a verified method for establishing patient identity, documenting the person's location when relevant to emergency response, protecting confidential conversation, arranging in-person escalation, and maintaining a reliable communication fallback. These components should be investigated as care-system attributes, including missed appointments, interruptions, patient preference, adverse events, and the time between concern and clinician contact. A successful video call demonstrates communication connectivity; service effectiveness depends on what the clinician can assess and do, and on the capacity to act when the encounter reveals urgent need. Governance evidence frameworks encourage matching implementation claims to this actual service purpose NICE (2022) WHO (2025).
Structured Digital Interventions and the Limits of Extrapolation
Structured digital mental-health programs usually deliver defined therapeutic exercises, psychoeducation, behavioral assignments, or guided feedback through websites or applications. Their contents may be grounded in established psychotherapeutic traditions and delivered with varying amounts of human support. This design allows some intervention ingredients, participant exposure, and outcome measurements to be specified in advance. Mohr and colleagues' framework for behavioral intervention technologies identified the need to study the intervention's clinical model, user interaction, and technical architecture together Mohr et al. (2013). The resulting literature is relevant to evaluating access, symptom change, engagement, and durability, but does not constitute a single homogeneous evidence base.
Meta-analyses of randomized trials by Firth and colleagues found evidence that certain smartphone interventions can reduce depressive and anxiety symptoms under the conditions of included studies, while variability in intervention content and comparisons limits universal claims Firth et al. (2017a) Firth et al. (2017b). A later meta-analysis of 154 randomized controlled trials examined whether internet-delivered cognitive-behavioral therapy effects remained after one year or longer Zainal et al. (2024). Such research supports examining whether a digital pathway provides therapeutic value after the initial novelty has faded. The size, design, follow-up, and support conditions of the contributing trials must guide interpretation rather than the broad label 'digital therapy.'
A critical distinction separates digital delivery from automatic generation of clinical content. A validated structured program can present stable material while a generative chatbot may produce different responses to the same clinical disclosure across sessions, revisions, and system versions. A clinical trial of one algorithmically adaptive intervention therefore provides evidence for that intervention's particular implementation, eligibility conditions, supervision, and duration. It does not authorize efficacy claims for all conversational products. A reliable classification scheme records what therapeutic principles are embodied in the software, which outputs are predefined, what content can vary, and what role clinicians occupy in monitoring and escalation Heinz et al. (2025) WHO (2024).
Evidence about usability and engagement also needs its own inferential boundary. High enrollment, frequent messages, or enthusiastic ratings do not establish reduced morbidity or improved functioning. Conversely, a clinically valuable intervention can demand less interaction once a patient learns the relevant skill. Baumel and colleagues studied objective engagement in mental-health applications and documented the challenge of sustaining use Baumel et al. (2019). Researchers should report denominators, patterns of nonuse, exposure to active ingredients, reasons for withdrawal, adverse experience, and outcome measurement even among those who stop using the application. Completer-only analyses can systematically overstate benefit when those with worsening symptoms disengage.
Conversational Agents: Controlled Evidence and Clinical Meaning
The randomized trial by Fitzpatrick and colleagues evaluated a fully automated conversational agent delivering cognitive-behavioral content to young adults reporting depressive and anxious symptoms over a short period Fitzpatrick et al. (2017). Its importance is methodological: a defined system was compared with a control condition using prespecified symptom measures. Its limitations are equally important, including the small, selected population and brief observation window. The trial cannot answer whether an unrestricted general-purpose chatbot is safe in crisis, competent in differential diagnosis, or effective in a broader psychiatric population. The fact that both systems exchange text does not make them the same clinical intervention.
Heinz and colleagues conducted a national randomized controlled trial of the expert-fine-tuned Therabot system with 210 adults screened for clinically significant depression, generalized anxiety, or elevated risk of feeding and eating disorders. Participants were assigned to four weeks of system access or waiting-list control, with subsequent follow-up Heinz et al. (2025). The study contributes primary evidence for symptom-related outcomes in a specified digital therapeutic design. However, comparison with a waitlist does not isolate the system's effects against an equivalent dose of professional care, and limited observation cannot establish indefinite safety or durability. Fine-tuning, monitoring procedures, recruitment, and user engagement remain essential characteristics of the intervention being evaluated.
In a separate three-arm randomized trial, Shoshani and colleagues enrolled 995 university students reporting psychological distress in Israel and compared a twelve-week conversational AI intervention, face-to-face group therapy, and a waiting-list group Shoshani et al. (2026). The AI group showed greater anxiety reduction than both comparators and improvements in selected outcomes, while posttraumatic stress symptoms did not differ significantly between groups. The trial is important because it included an active treatment comparison, but group therapy and a privately accessed conversational intervention differ in multiple ingredients, time patterns, and participant experiences. The results require attention to the studied student population, unblinded nature of intervention participation, specific outcome instruments, and follow-up duration.
Studies sometimes call users' ratings of a chatbot relationship 'therapeutic alliance.' These questionnaires capture perceived responsiveness, support, trust, or engagement under their operational definitions. They do not by themselves establish that an artificial system possesses a professional therapeutic relationship, a duty of care, or a clinician's understanding of psychiatric context. In the Shoshani trial, perceived alliance was statistically associated with engagement and symptom improvement; interpreting the association as a causal mechanism requires separate experimental identification Shoshani et al. (2026). The scientific synthesis should therefore preserve three levels: the patient's experience, the system's observable behavior, and the causal contribution of that behavior to clinically meaningful outcomes.
An operational evaluation of a conversational clinical agent must record intended function, boundary cases, model and prompt versions, prohibited actions, monitoring protocols, and escalation logic. Its comparator should match the substantive clinical question: for an adjunct, compare care with and without the adjunct; for a replacement claim, compare against relevant established care; for autonomous triage, prospectively assess error and resulting decisions. Without such designs, positive usability results can be mistaken for a broad authorization to deploy. WHO guidance treats the apparent conversational competence of large multimodal models as a reason for systematic oversight, especially where plausible language can mask false information WHO (2024).
Crisis Identification, Escalation, and Safety-Critical Communication
Acute risk is the setting in which an erroneous conversational response can have especially consequential effects. A person may describe suicidal thoughts directly, refer indirectly to hopelessness, disclose self-harm, express violent intent, or combine apparently conflicting statements about safety. Real-world assessment considers the evolving person's presentation, relevant history, context, protective conditions, and access to timely human care. Classifying a message into a risk category is one component of a clinical response, not a comprehensive assessment. A safe care pathway additionally requires reliable contact, appropriately trained human review, locally relevant crisis options, and clear responsibility for action. The problem of base rates in event prediction remains relevant even when a classifier recognizes concerning language Belsher et al. (2019).
Arnaiz-Rodriguez and colleagues published a clinically informed evaluation of large language models handling mental-health crises in 2026. They assembled 2,252 examples from twelve source datasets, used a validation subset and a test set, and audited responses from five models for appropriateness Arnaiz-Rodriguez et al. (2026). The work identified differences among evaluated systems and weaknesses with indirect or ambiguous risk signals. The authors used a language-model judge whose judgments were evaluated against human expert feedback; this evaluation design supports bounded conclusions about benchmark responses and the judging protocol. It does not establish actual patient outcomes or full safety in deployed emergency pathways. The task, model revisions, crisis taxonomy, and study design should accompany every citation to the results.
Clinical safety assessment should test the sequence, not just the first reply. Investigators may construct authorized test cases in which the person changes a risk disclosure, refuses a suggested option, corrects an earlier statement, or uses metaphorical wording. For each sequence, reviewers can assess whether the system updates its interpretation, avoids fabricated claims about available local services, preserves accurate information, identifies limits of automated action, and provides an available route to human intervention. Simulations should never be confused with real-time crisis management. Controlled testing must also document whether human monitors are continuously available or only promised by interface language Arnaiz-Rodriguez et al. (2026) WHO (2024).
The evaluation endpoint should include opportunity for harm reduction as well as avoidance of hazardous content. A model may produce an apparently safe generic answer that misses a specific urgent need or directs the user to resources inaccessible from their jurisdiction. A triage tool may appropriately suggest immediate assessment while a clinic lacks the capacity to accept the referral. Researchers should therefore examine handoff completion, time to human contact, communication failure, adverse event reporting, and patient comprehension. Safety is a property of the human–artificial service arrangement. The clinical setting, available workforce, and local care infrastructure determine whether an apparently appropriate model output can actually support a patient.
A research monograph can analyze crisis pathways without issuing individual medical instructions. Evaluation should be organized around the professional responsibilities of health organizations, the need to validate current contact information, and the accessibility of qualified emergency care. Any public-facing system intended for use in acute circumstances requires local clinical governance and a safety plan tested before deployment. General research on a model's response behavior is evidence about that model and test context; it is not a substitute for a licensed clinician's assessment of a real person's circumstances.
Human–Artificial Differential Decisions and Clinical Oversight
Clinical differential reasoning compares plausible explanations for symptoms and decides which additional information is needed. In mental health, a presentation can reflect psychiatric syndromes, medical illness, medication effects, substance exposure, sleep disruption, situational stress, neurological conditions, or combinations of causes. An artificial model may assist by retrieving structured evidence, organizing a history, or enumerating possibilities; the clinical value of that assistance depends on completeness, prioritization, uncertainty, and the clinician's ability to interrogate its basis. A list of diagnoses that appears reasonable in a retrospective vignette is a weaker outcome than evidence that clinicians made safer decisions with the tool in actual practice Vasey et al. (2022) Collins et al. (2024).
One can separate three experiments with different inferential meanings. A technical study asks whether the system reproduces judgments on held-out cases. A prospective silent-mode study observes what the system would have recommended while clinicians continue usual practice, allowing discrepancies and failure cases to be collected without changing care. An interventional trial asks whether presenting the output changes clinician actions, patient outcomes, costs, or equity. Each design informs the next. Silent-mode evaluation is useful when the main safety questions are unresolved, while a deployment trial requires clear stopping conditions and a plan for monitoring unforeseen consequences. DECIDE-AI provides reporting guidance for early-stage clinical evaluation of artificial intelligence decision support Vasey et al. (2022).
Human approval should be characterized by its actual quality. When clinicians repeatedly encounter accurate recommendations, they may develop inappropriate automatic reliance; when recommendations are unhelpful, they may ignore beneficial alerts. Oversight research should assess override rates, error identification, time pressure, alert fatigue, explanation comprehension, and who remains responsible for communicating the result to patients. A system's description as 'human in the loop' does not demonstrate that the human has enough information, authority, time, and training to intervene effectively. WHO addresses automation bias explicitly in its multimodal AI guidance WHO (2024).
The strongest studies will include not only immediate decision agreement but also independent patient-level outcomes and subgroup analyses. Differential decisions may change the timing of testing, referrals, or treatment, which in turn can change the incidence of documented disorders; a feedback process may cause the algorithm's own output to influence subsequent labels. Investigators should document these pathways and distinguish the intended effects from altered record-keeping. The performance of the whole clinical system can be better or worse than the performance of the component considered separately. Regulatory guidance and reporting frameworks support this function-specific, context-aware approach FDA (2026) Liu et al. (2020).
Service Integration, Care Continuity, and Patient Access
Digital care is often introduced to increase reach, but reach must be measured as a sequence of completed service steps. A patient may encounter an online screener, receive a recommendation, contact a clinic, complete an assessment, begin indicated care, and remain engaged long enough for relevant outcomes to be observed. Every transition has a denominator and potential reasons for failure. A system that produces many recommendations with few completed referrals has limited demonstrated clinical utility. Conversely, improved scheduling or reduced administrative load may be valuable even when algorithmic diagnosis adds little. WHO's strategy for digital health emphasizes the integration of organizational, financial, human, and technological resources WHO (2025).
Hybrid models combine self-report screening, asynchronous reminders, human review, scheduled video visits, and in-person escalation. Their causal mechanism is distributed across communication, triage policy, service availability, and the trust created by continuity with recognizable clinicians. For research, the intervention should be described at each stage and compared with an appropriate alternative service pathway. Kishimoto and colleagues' pragmatic video study illustrates evidence for one defined remote-care configuration; it should not be relabeled as evidence for autonomous chatbot management Kishimoto et al. (2024). The precise allocation of tasks matters both clinically and scientifically.
Implementation outcomes include acceptability, adoption, reach, sustained use, feasibility, cost, clinical appropriateness, fidelity, and equity. They require different sources of data: patient surveys, staff interviews, EHR audits, process logs, appointments, resource use, and patient-reported outcomes. The same software can work differently across health systems because waiting lists, staffing, reimbursement, digital literacy, and referral networks vary. A research evaluation should report the local workflow and distinguish the tool's direct contribution from simultaneous reorganization of the service. NICE's evidence standards were developed to encourage evidence proportionate to the specific digital technology and its clinical function NICE (2022).
Continuity and patient choice are particularly important when a system changes or disappears. Model updates, vendor retirement, lost credentials, or interrupted network access can affect patients whose records or clinical expectations depend on the platform. Service design should preserve records, provide alternative channels, identify a responsible human organization, and explain whether automated outputs are included in the clinical record. Researchers can examine discontinuity through loss-to-follow-up analyses, patient interviews, service interruption audits, and comparisons across technical transitions. The important outcome is whether clinical care continues coherently despite changes in the technological component.
Access is multidimensional. A patient may own a smartphone yet lack a private space to talk, appropriate language support, an accessible interface, reliable data service, or confidence that information will remain confidential. Accessible alternatives should be evaluated rather than assumed. Greater availability of digital tools can reduce certain access barriers while introducing others, producing a net effect that differs by population. Scientific assessment therefore includes recruitment denominators and the characteristics of people who never enroll, as well as outcomes among those who do. This population perspective connects clinical digital systems to a wider public-health understanding of service equity WHO (2019) WHO (2025).
Clinical Accountability in Human–Artificial Systems
Clinical accountability is the capacity to identify which institution or professional authorized a technology's use, which decisions were made, what information was available at each decision point, and how an affected patient can obtain correction and review. Complex systems can distribute work among patients, clinicians, vendors, support personnel, and software without eliminating the need for accountable governance. For research purposes, accountability can be assessed through responsibility matrices, clinical documentation, escalation records, and independent audits. WHO's governance framework for large multimodal systems treats patient safety, transparency, and oversight as institutional responsibilities extending across development and deployment WHO (2024).
The allocation of responsibility should follow actual control. A clinician cannot meaningfully review a model output if the interface withholds essential evidence or presents a recommendation after the relevant action has already occurred. A developer cannot claim that local institutions guarantee safety without specifying which monitoring data the institution receives or how software updates are disclosed. A patient should not bear responsibility for detecting technical failures hidden from them. A rigorous implementation study therefore records what each participant knew, could change, and was expected to do. FDA's 2026 clinical decision-support guidance highlights functional distinctions that matter to intended use and the role of professional judgment FDA (2026).
Governance evidence is distinct from evidence of effectiveness. A software tool may be regulated under one category, yet specific claims about clinical outcomes still require empirical support in the population and context of use. An algorithm can have well-documented source code and still be clinically misleading when labels are poor or the environment changes. Conversely, a useful observational result may not justify deployment without data protection and safety procedures. Assessments should examine independent evidence, mechanisms for reporting incidents, consent processes, rollback options, accountability for training, and documentation of updates. The NICE framework offers a model for connecting evidence expectations to practical health technology functions NICE (2022).
A human–artificial clinical system is scientifically distinctive because system behavior is produced through interactions rather than one fixed output. Human staff change decisions in response to algorithmic recommendations, users adjust disclosures based on perceived responsiveness, and providers may alter workflows around the technology. These feedback loops can create safety advantages or new hazards. Their existence supports evaluating causal pathways and responsibilities at multiple levels: the individual encounter, organization, supplier, and regulatory environment. The goal is a system in which claims can be investigated and harmful outcomes can be traced to preventable causes rather than buried within general assertions of innovation.
Trust, Reliance, and the Psychology of Artificial Clinical Authority
Trust in a clinical technology is a person's expectation that it will perform a consequential task reliably under specified conditions. Appropriate reliance depends on the match between that expectation and actual performance. Patients may place weight on fluent language, availability, apparent empathy, or an institution's branding, while clinicians may be influenced by numerical confidence scores, the presentation of evidence, or time pressure. These cues can create confidence that is only loosely associated with validity. WHO identifies automation bias as a concern when professionals or patients overlook errors in persuasive model outputs WHO (2024).
Researchers should study observable reliance behavior rather than relying exclusively on ratings of trust. An experiment can provide some clinicians with correct and incorrect recommendations, vary explanations or uncertainty displays, and measure whether they accept advice appropriately. In clinical settings, the crucial outcome is whether the combined human–artificial process improves decisions relative to a defined baseline. Agreement with an algorithm can represent useful assistance, social conformity, reduced effort, or inappropriate deference. Each interpretation requires different evidence. Reporting of AI interventions should specify what information the user saw and whether the person had an opportunity to interrogate or reject the output Liu et al. (2020) Vasey et al. (2022).
Conversational systems complicate the analysis because the experience of support can become a reason to regard medical claims as credible. A chatbot that responds promptly to emotional disclosures may create perceived connection or safety even if its referral advice is mistaken. In randomized studies of conversational interventions, patient-reported perceptions can be analyzed as engagement correlates; they cannot independently verify the model's diagnostic correctness or legal authority. Shoshani and colleagues' study measured perceived therapeutic alliance and associated it with engagement and outcomes, illustrating both the research importance of the perception and the need to distinguish correlation from a demonstrated causal mechanism Shoshani et al. (2026).
Trust calibration is a longitudinal issue. Users may revise expectations after a model offers accurate guidance, fails to remember context, or makes an obvious error. Staff can become habituated to alerts or to a confident interface. Research should follow how reliance evolves over repeated encounters and after software updates, documenting both patient and clinician behavior. Clear descriptions of intended capabilities, uncertainty, and alternative options may improve informed use, but their effects should be measured experimentally rather than assumed. The appropriate target is reliable clinical judgment supported by transparent technology, not maximal adoption or uniformly high confidence.
Data Governance, Confidentiality, and Informed Participation
Mental-health data include highly contextual and sensitive information concerning symptoms, relationships, trauma, medication, intimate behavior, and perceived risk. A digital record can be replicated, transferred, combined with other datasets, or analyzed for purposes far removed from the original clinical conversation. Consent therefore involves more than acceptance of a general application policy. Scientific and institutional practice should distinguish information required for care, data collected for product improvement, optional research use, sharing with analytics providers, and the possible use of content in future model training. The observed privacy practices of digital applications show why the distinction matters in practice Huckvale et al. (2019).
Huckvale, Torous, and Larsen audited smartphone applications relating to depression and smoking cessation and reported discrepancies between actual third-party transmission and stated disclosure practices Huckvale et al. (2019). This study concerned a defined sample of apps and a particular technical environment, so its numerical findings should not be generalized without qualification to every current platform. Its enduring scientific lesson is methodological: privacy claims need technical inspection and independent verification. Policies describe intended practice, while network audits, data-flow inventories, retention tests, and contractual review investigate what systems actually do.
Research involving digital psychiatric information should identify lawful data access, withdrawal rights, retention schedules, cross-border transfers, and meaningful protections against reidentification. De-identification can reduce risk but may be undermined when linguistic details, rare life events, or linked metadata make a person recognizable. Permission to publish or analyze aggregate outcomes does not automatically authorize disclosure of clinical conversations. In studies of generative models, researchers should document whether participant materials are transmitted to external model providers and whether any external service retains prompts. WHO governance guidance and ethical analyses of digital mental health emphasize confidentiality, transparency, and proportionate oversight WHO (2024) Wykes et al. (2019).
Privacy protections can also affect the validity of clinical assessment. Patients who believe their disclosure may be exposed to employers, families, police, or commercial systems may withhold crucial information. Conversely, unnecessary data collection can create burdens that discourage participation. Studies of clinical utility should include trust, willingness to disclose, comprehension of data use, and actual uptake as outcomes, not merely technical compliance. Research protocols should test whether consent explanations are comprehensible to intended users and should preserve accessible alternatives for people who decline digital collection.
Clinical Harm, Error Reporting, and Continuous Surveillance
Harms from digital psychiatric systems include false reassurance, unnecessary alarm, delayed contact with qualified care, privacy disclosure, distorted records, inappropriate referral, and erosion of patient autonomy. Some harms are immediate and recognizable; others accumulate through misclassification, disengagement, or repeated exposure to unsafe suggestions. Safety research should define adverse events and near misses in advance, specify how incidents are discovered, and record the time from disclosure to resolution. A product's lack of reported adverse events provides little reassurance if its study did not actively collect or classify them. Clinical AI reporting guidance calls for transparency about intervention context, participants, and outcomes Liu et al. (2020) Vasey et al. (2022).
Model failure analysis requires both routine surveillance and targeted tests. Routine data can identify referral completion, rising override rates, unexplained changes in assessment patterns, or a drop in response quality after updates. Targeted challenge sets explore ambiguous risk language, conflicting information, unusual populations, and adversarial manipulation. The 2026 study by Arnaiz-Rodriguez and colleagues provides an empirical example of clinically informed crisis scenarios and model-dependent response differences Arnaiz-Rodriguez et al. (2026). Benchmarks are valuable for discovering repeatable failures, though their prevalence and wording cannot be assumed to match clinical reality. Prospective monitoring supplies complementary evidence.
An incident management system should allow an affected person or clinician to contest an output, preserve sufficient records to reconstruct the case, and identify an available path for urgent human review. Accountability must be designed to avoid trapping patients in an automated loop when they request human care. A review should distinguish whether the primary failure arose from data acquisition, the algorithm, user-interface presentation, clinical judgment, missing service capacity, or organizational policy. This causal decomposition makes corrective action more specific than a general declaration that an AI model was unsafe.
Software evolution creates a continuing obligation for revalidation. Changing language model versions, altering prompts, incorporating external retrieval, modifying a threshold, or shifting service populations may alter error patterns. Organizations should define which changes require predeployment retesting and how to restore a previously validated configuration. Monitoring should include subgroup outcomes and unexpected changes in clinical workload, because a system that reduces average waiting times while systematically delaying one group can have adverse distributive effects. NIST's generative AI profile and WHO's clinical AI governance guidance provide relevant frameworks for lifecycle risk management Autio et al. (2024) WHO (2024).
Regulatory Boundaries and the Evidence Required for Intended Use
Regulation of digital health software depends on jurisdiction, function, intended medical purpose, risks, and the legal category into which the system falls. The same general model might power an administrative text summary, a patient education interface, or a tool that materially influences a diagnostic decision. Their regulatory treatment and evidence demands can differ. In the United States, the FDA's January 2026 final guidance on clinical decision support explains statutory criteria distinguishing certain nondevice clinical decision-support functions from device software and describes its approach to other software functions FDA (2026). This guidance should be interpreted from its actual criteria and examples rather than as a blanket approval or prohibition of AI in psychiatry.
Research reporting standards provide a complementary framework. TRIPOD+AI concerns transparent reporting of prediction models, CONSORT-AI extends randomized trial reporting for interventions involving artificial intelligence, STARD-AI addresses diagnostic accuracy research, and DECIDE-AI concerns early-stage clinical evaluation of AI decision support Collins et al. (2024) Liu et al. (2020) Sounderajah et al. (2025) Vasey et al. (2022). These standards do not themselves constitute proof of a model's safety. They are valuable because they make design choices and evidence gaps observable, increasing the possibility of independent review and replication.
The NICE evidence standards framework organizes expectations for digital health technologies according to their functions and risks and includes considerations for adaptive algorithms NICE (2022). WHO's digital health strategy emphasizes system-level implementation and equity, while its guidance on large multimodal models analyzes governance across patients, clinicians, developers, and states WHO (2025) WHO (2024). These are distinct institutional documents with different authority. A scientifically precise monograph identifies their roles separately: a reporting guideline organizes what researchers disclose, an evidence framework advises on evaluation, a regulatory guidance document describes agency interpretation, and a clinical trial estimates effects under specified conditions.
Because rules and clinical systems change, regulatory claims require a date, jurisdiction, and verification of current official documents. Software approved or cleared for one purpose cannot be described as authorized for every psychiatric application. Experimental evidence for a general-purpose model does not by itself provide a medical-device authorization or establish an approved indication. Likewise, a wellness application and a high-stakes triage system should not be compared merely on the number of users. Scientific conclusions must remain tied to the tested intervention and to the institution responsible for its use.
Psychological Experience, Participation, and the Artificial Era
The emergence of human–artificial clinical systems changes how patients encounter health information and express uncertainty. Some people may prefer a low-threshold digital interaction before contacting a clinician; others may find automated questions alienating or experience difficulty disclosing context to a structured interface. A clinically informative design investigates disclosure, comprehensibility, perceived control, engagement, and opportunities to challenge an interpretation. These are psychological variables that influence both measurement validity and the distribution of care. Evidence about software effectiveness cannot be fully interpreted without information about who elected to use it and how the interface shaped the encounter.
The Artificial Era provides a conceptual horizon for investigating changes in the location of cognitive work. Information retrieval, preliminary categorization, clinical note production, and decision preparation can occur through artificial systems, while professional institutions remain responsible for the care they deliver. This historical shift invites questions about distributed cognitive activity, responsibility, and public expectations of nonbiological participants. The framework is an interpretive context rather than an empirical claim that a conversational model possesses consciousness, subjective distress, or clinical licensure. For clinical research, the relevant empirical questions concern what the artificial system actually does and how human patients and professionals respond.
Postsubjective and relational approaches may treat a clinical encounter as a changing organization of activities across participants and technologies. To make such a perspective scientifically productive, its constructs must be connected to observable measures: who initiates information exchange, who can correct a decision, how disagreement is resolved, and how patient experience changes with role allocation. A system may display linguistic responsiveness without reliable contextual understanding; a clinician may hold formal authority without effective oversight; a patient may report feeling listened to while a referral is incorrectly routed. Investigating these configurations offers a testable research program grounded in interaction logs, patient narratives, process evaluation, and patient outcomes Shoshani et al. (2026) Vasey et al. (2022).
An ethically and clinically credible system preserves patient agency through informed choice, access to human contact, correction of records, transparency about software involvement, and routes for addressing harm. The psychological effects of such provisions should be tested rather than treated solely as principles. Studies can compare trust calibration, comprehension, adherence, disclosure quality, and satisfaction under different interfaces and oversight arrangements. The general conclusion is that clinical value arises from the functioning of the complete human–artificial service system and must be validated as such. This orientation is consistent with the institutional governance questions raised by WHO and with clinical evaluation frameworks requiring evidence of how intended users engage with AI outputs WHO (2024) Vasey et al. (2022).
Clinical Study Designs and an Evaluation Sequence
A rigorous evidence program begins with a precise intended-use statement: which population, setting, information source, user, decision, and clinical outcome does the technology concern? A patient-facing questionnaire, clinician-facing prediction score, and automated triage conversation should be evaluated separately even when built on a shared computational platform. The research sequence can begin with construct specification and analytic verification, proceed through external validation and prospective observational evaluation, and culminate in a comparative trial of actual clinical pathways where ethically and practically appropriate. Reporting guidance for prediction models, diagnostic accuracy, early decision-support evaluation, and randomized AI interventions provides different tools for each stage Collins et al. (2024) Sounderajah et al. (2025) Vasey et al. (2022) Liu et al. (2020).
Retrospective validation can identify coding errors, data leakage, measurement inconsistency, and differences across clinics, yet it does not reveal all consequences of presenting an alert to a real practitioner. Prospective silent-mode studies test systems alongside actual care without allowing recommendations to change treatment; they can identify technical failures and estimate disagreement but cannot establish the benefits or harms of acting on the output. Randomized or carefully designed quasi-experimental implementation studies evaluate whether a clinical tool changes decisions and outcomes. The counterfactual must correspond to the institutional question: usually care as practiced, sometimes an active alternative, and sometimes a specific human review arrangement.
Outcomes should include patient benefit, patient burden, service impact, and equity. Examples are symptom change on validated measures, timely assessment, referral completion, avoidable emergency utilization, patient-defined functioning, harmful delays, privacy incidents, and clinical workload. A study should prespecify its primary outcome and time horizon, while secondary outcomes can explain mechanism or unintended effects. The intervention must be characterized at sufficient fidelity for another team to reconstruct what was tested. Model identifier, training or fine-tuning status when available, prompt configuration, external tools, governance procedures, version dates, and monitoring staff are all relevant features of the actual treatment pathway Liu et al. (2020) Vasey et al. (2022).
Different evaluation designs can be nested without conflating results. Psychometric studies determine whether a score represents a target construct; diagnostic studies estimate agreement with an appropriate reference; prospective implementation studies show whether the system changes workflow; comparative trials measure patient outcomes; and long-term audits detect drift and delayed harms. This is an explanatory hierarchy, not a claim that every software update demands a new randomized trial. The appropriate design depends on clinical consequence and the uncertainty remaining after prior evidence. Clear reporting makes it possible for clinicians, researchers, and regulators to see precisely what has and has not been demonstrated.
Statistical Reasoning, Calibration, and Heterogeneity
Statistical inference in digital clinical systems must follow the organization of the data. Repeated messages from one person are dependent observations, visits are nested within clinicians and clinics, and events may occur only after an earlier recommendation. Standard errors that ignore clustering can overstate precision, while evaluations that split related observations between training and testing may leak person-specific language and historical context. Researchers should separate participant-level, encounter-level, clinic-level, and model-version units, describing how each contributes to the target claim. Missingness is especially important when disengagement may be associated with symptoms, distress, or difficulty accessing a digital service.
Prediction models should report calibration, discrimination, error rates at clinically meaningful thresholds, and uncertainty. The interpretation of a threshold depends on event prevalence, expected benefits, false-alert burdens, and the capacity for safe review. A finding of acceptable overall discrimination should be supplemented by subgroup performance, calibration plots, and prospective testing. Belsher and colleagues' suicide prediction synthesis demonstrates how global classification metrics can conceal limits on individual positive prediction Belsher et al. (2019). Tests that identify an extreme retrospective sample as positive may have little value in routine screening with a very different base rate.
Comparative intervention studies require attention to attrition, analysis populations, adherence, multiple outcomes, and the comparator's clinical relevance. Waiting-list controls can estimate whether access to a program produces improvement beyond waiting under the trial conditions, while active comparisons address a different question. An association between perceived chatbot alliance and symptom change may be influenced by shared reporting tendencies, baseline severity, engagement, or other factors; a mediation interpretation should be tested with appropriate temporal and causal assumptions. Results from the 2025 Therabot and 2026 university-student trials thus occupy identifiable but limited positions within the hierarchy of clinical evidence Heinz et al. (2025) Shoshani et al. (2026).
Subgroup analyses should follow prespecified clinical hypotheses when possible and include sufficient precision to distinguish plausible variation from random noise. Cultural and language differences, age, clinical severity, comorbidity, disability, and access to technology can modify both effectiveness and harms. A mean effect across users cannot substitute for investigation of those who do not enroll or who leave the platform. Researchers should report both conditional effects among users and the broader consequences for the eligible population. Transportability studies across health systems and transparent sensitivity analyses offer stronger grounds for generalization than replication of a single internal validation score Collins et al. (2024) WHO (2025).
Reproducibility, Bias Audits, and Model Change Management
Reproducibility in human–artificial clinical research requires stable descriptions of the investigated system. A study should preserve the dates and versions of software, interfaces, prompts, safety policies, retrieval collections, and clinical workflow configuration. When external models are accessed through changing commercial services, replication is inherently conditional on continued access to the same behavior. Researchers can improve auditability by preserving allowable test materials, complete evaluation rubrics, de-identified aggregate results, and reproducible analysis code while respecting confidential health information. Reporting guidelines developed for AI research offer structured ways to expose details essential to critical appraisal Liu et al. (2020) Collins et al. (2024).
Bias audits should investigate the full production chain rather than treating protected attributes as a sufficient checklist. Labels may encode unequal access, variables may act as proxies for social conditions, and changes in referral capacity can transform the consequences of a prediction. Researchers should ask whether positive and negative results lead to comparable opportunities for safe follow-up across groups. Obermeyer and colleagues' health-allocation study supplies a concrete example of problematic target selection; applications to mental health require their own data and evaluations rather than direct importation of the observed effect Obermeyer et al. (2019).
Model drift can arise from changing language use, service populations, diagnostic coding, clinical practices, and vendor updates. The consequences of drift depend on whether model outputs affect subsequent data: an alert may change whether a patient receives assessment, and that encounter may change which outcomes are recorded. Surveillance should therefore include baseline data, decision logs, the rate of clinician overrides, patient-reported concerns, and external sentinel measures that are not defined by the model's own recommendations. Thresholds for pausing or rolling back deployment should be specified in advance for safety-critical functions Autio et al. (2024) WHO (2024).
Independent assessment is especially important when a commercial developer designs a tool, controls the benchmark, recruits participants, and publishes the resulting evaluation. These contributions can produce useful work, but disclosure of relationships, access to evaluation protocols, and external replication help distinguish robust findings from product-specific optimism. Evidence reviews should document whether conflicts of interest were reported, whether adverse outcomes were collected, and whether the study allowed a fair comparison with existing care. A favorable trial establishes evidence for a specified intervention in a specified population, while independent multicenter research tests whether the conclusion can travel.
Synthesis of Established Findings and Evidentiary Limits
Several findings converge across distinct research programs. Validated symptom questionnaires can support structured measurement, provided thresholds and interpretations match their populations and purposes Kroenke et al. (2001) Spitzer et al. (2006). Remote psychiatric treatment can be clinically effective under studied conditions, as shown in the pragmatic video trial by Kishimoto and colleagues Kishimoto et al. (2024). Structured digital interventions have demonstrated symptom-related benefits in randomized studies and evidence syntheses, although effects depend on intervention design, support, follow-up, and comparison condition Firth et al. (2017a) Firth et al. (2017b) Zainal et al. (2024). These convergences establish that digital delivery and measurement have substantive places in mental-health research and services.
Generative conversational systems introduce a more heterogeneous evidence picture. Specific intervention trials show encouraging outcomes for some populations and measured endpoints, including the expert-tuned Therabot study and the 2026 university-student intervention trial Heinz et al. (2025) Shoshani et al. (2026). Their findings support inquiry into particular supervised or bounded designs. They do not establish diagnostic proficiency, universal efficacy, equitable access, or reliable crisis management for arbitrary general-purpose models. The crisis-response evaluation by Arnaiz-Rodriguez and colleagues identifies model-dependent failures and contextual vulnerabilities that require further study in real-world settings Arnaiz-Rodriguez et al. (2026). These lines of research are complementary but cannot be collapsed into one overall conclusion about 'AI therapy.'
Prediction studies provide a caution about the distinction between technical performance and clinical value. Models may reproduce historical labels well while failing to identify individual low-prevalence outcomes with a clinically actionable positive probability Belsher et al. (2019). Bias may emerge through the choice of outcome proxy, data access, institutional history, or uneven service resources Obermeyer et al. (2019). For a clinical system, scientific confirmation therefore requires a chain of evidence linking construct validity, prospective decision performance, action feasibility, patient benefit, and acceptable harms. Each link is empirically testable and should be evaluated independently.
Privacy, trust, and service configuration are constitutive parts of the clinical intervention. Third-party data transmission, unclear retention, unstaffed alerts, poorly calibrated reliance, or inaccessible referrals may alter outcomes despite technically accurate predictions. Privacy audits and ethical analyses of mental-health applications make these risks visible Huckvale et al. (2019) Wykes et al. (2019). The strongest form of clinical inference concerns a defined, governable system that demonstrably improves the experience or health of intended patients. At that level, algorithmic evaluation is necessary but insufficient without organizational and psychological investigation.
Open Research Questions and Future Scientific Directions
The next generation of research should prioritize prospective multi-institutional comparisons of complete care pathways rather than retrospective scores alone. Priority questions include whether AI-assisted triage reduces time to appropriate assessment without increasing missed risk; whether clinicians' use of a model improves diagnostic process quality; and whether access gains persist among groups with low digital literacy or unstable connectivity. Each question needs a target population, a comparator, a meaningful outcome, and a way to detect harm. Clinical prediction reporting and early-stage evaluation guidelines can improve transparency without substituting for independent results Collins et al. (2024) Vasey et al. (2022).
Longitudinal research should examine dependence on digital services, discontinuation effects, the implications of repeated exposure to algorithmically generated interpretations, and the stability of clinical recommendations after model updates. Studies should distinguish temporary reassurance from durable symptom change and should not infer therapeutic mechanisms solely from satisfaction ratings. Research on perceived alliance in teletherapy and conversational AI interventions offers useful starting points, but the operational construct changes with the nature of the interacting system Aafjes-van Doorn et al. (2024) Shoshani et al. (2026). Longitudinal patient-centered methods are required to investigate these experiences responsibly.
Methods for evaluating crisis response require further development. Current benchmarks can detect unsafe or contextually unsuitable answers, but new studies should test complete escalation pathways, delayed or ambiguous disclosures, multilingual expressions, human monitoring availability, and actual patient-facing consequences under ethically acceptable protocols. The 2026 crisis-handling evaluation supplies a substantial base for constructing test scenarios while its benchmark and automated-judging procedures delimit generalization Arnaiz-Rodriguez et al. (2026). Research involving acute risk must prioritize participant protection and qualified clinical oversight. No model-only benchmark can establish that a service is safe without human and institutional mechanisms for intervention.
A further priority is comparative evidence about governance architectures. Studies can test whether meaningful clinician review, explainable summaries, independent auditing, privacy-preserving design, patient-accessible corrections, and clear handoffs affect error rates and patient outcomes. Cost-effectiveness should be examined with transparent accounting for staffing, supervision, incident response, licensing, maintenance, and excluded populations. Researchers can investigate whether digital systems redistribute work toward human care or instead generate hidden labor in checking and correcting outputs. WHO and NICE provide frameworks for making these implementation questions visible WHO (2024) NICE (2022).
Finally, the psychology of clinical participation in the Artificial Era demands longitudinal study of how patients and professionals attribute credibility, intention, and responsibility to computational participants. The relevant observable questions concern disclosure patterns, changes in help-seeking, appropriate reliance, perceived control, and outcomes when artificial interfaces mediate clinical communication. Conceptual frameworks from the Artificial Era may identify new hypotheses concerning distributed clinical cognition, but their empirical status depends on operational definition, independent replication, and evaluation in actual care contexts. This research program links psychological science to the changing organization of technological health systems while retaining the evidence requirements appropriate to consequential clinical work.
Conclusion
Digital Mental Health and Human–Artificial Clinical Systems constitutes a distinct scientific research object because clinical outcomes depend on the coordinated activities of patients, professionals, artificial systems, and health organizations. Its central question concerns whether a specific digital configuration improves clinically meaningful care under known conditions and acceptable risk. Measurement, prediction, triage, service delivery, and intervention have separate evidentiary requirements. Their integration creates new opportunities for access and continuity, but also additional points of failure. The discipline's scientific task is to establish those relationships through reproducible observation and appropriately controlled comparison.
The historical research record supports the value of selected digital measurements, remote psychiatric encounters, and structured digital interventions. Trials of carefully designed conversational agents now expand the range of investigated interventions, although their findings remain tied to their particular populations, comparators, monitoring procedures, and duration Kishimoto et al. (2024) Heinz et al. (2025) Shoshani et al. (2026). Research on rare-event prediction, digital privacy, and model crisis behavior demonstrates why promising accuracy or engagement statistics must be interpreted within a complete clinical pathway Belsher et al. (2019) Huckvale et al. (2019) Arnaiz-Rodriguez et al. (2026). Effective implementation depends on verification of both intended benefits and practical safeguards.
The synthesis establishes an integrated methodological principle: technical validity is investigated first, clinical interpretation is tested against appropriate standards, actual effects on human decisions are measured, patient-relevant consequences are compared with alternatives, and the service remains subject to continuing safety and equity assessment. Qualified professionals and accountable institutions are required to govern the clinical use of consequential artificial systems. This principle makes scientific claims precise and provides a durable basis for judging changing technologies without mistaking either enthusiasm or apprehension for evidence.
In the Artificial Era, the location of informational and cognitive work within mental-health services is changing. Clinical authority, patient dignity, and effective care must be examined in relation to the actual distribution of actions and responsibilities across human and artificial components. The research horizon is therefore a science of clinically consequential human–artificial organization: empirically grounded, patient-centered, historically informed, and open to revision as stronger evidence becomes available.
References
- Aafjes-van Doorn, K., Spina, D. S., Horne, S. J., & Békés, V. (2024). The association between quality of therapeutic alliance and treatment outcomes in teletherapy: A systematic review and meta-analysis. Clinical Psychology Review, 110, 102430. https://doi.org/10.1016/j.cpr.2024.102430
- Arnaiz-Rodriguez, A., Baidal, M., Derner, E., Annable, J. L., Ball, M., Ince, M., Perez Vallejos, E., & Oliver, N. (2026). Between help and harm: An evaluation study of mental health crisis handling by large language models. JMIR Mental Health, 13, e88435. https://doi.org/10.2196/88435
- Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P., & Roberts, K. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.600-1
- Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019). Objective user engagement with mental health apps: Systematic search and panel-based usage analysis. Journal of Medical Internet Research, 21(9), e14567. https://doi.org/10.2196/14567
- Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A. (2019). Prediction models for suicide attempts and deaths: A systematic review and simulation. JAMA Psychiatry, 76(6), 642–651. https://doi.org/10.1001/jamapsychiatry.2019.0174
- Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on social media: A critical review. npj Digital Medicine, 3, Article 43. https://doi.org/10.1038/s41746-020-0233-7
- Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., et al. (2024). TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385, e078378. https://doi.org/10.1136/bmj-2023-078378
- Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017). The efficacy of smartphone-based mental health interventions for depressive symptoms: A meta-analysis of randomized controlled trials. World Psychiatry, 16(3), 287–298. https://doi.org/10.1002/wps.20472
- Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017). Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. Journal of Affective Disorders, 218, 15–22. https://doi.org/10.1016/j.jad.2017.04.046
- Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017). Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): A randomized controlled trial. JMIR Mental Health, 4(2), e19. https://doi.org/10.2196/mental.7785
- Heinz, M. V., Mackin, D. M., Trudeau, B. M., Bhattacharya, S., Wang, Y., Banta, H. A., Jewett, A. D., Salzhauer, A. J., Griffin, T. Z., & Jacobson, N. C. (2025). Randomized trial of a generative AI chatbot for mental health treatment. NEJM AI, 2(4). https://doi.org/10.1056/AIoa2400802
- Hilty, D. M., Ferrer, D. C., Parish, M. B., Johnston, B., Callahan, E. J., & Yellowlees, P. M. (2013). The effectiveness of telemental health: A 2013 review. Telemedicine and e-Health, 19(6), 444–454. https://doi.org/10.1089/tmj.2013.0075
- Huckvale, K., Torous, J., & Larsen, M. E. (2019). Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Network Open, 2(4), e192542. https://doi.org/10.1001/jamanetworkopen.2019.2542
- Kishimoto, T., Kinoshita, S., Kitazawa, M., Hishimoto, A., Asami, T., Suda, A., et al. (2024). Live two-way video versus face-to-face treatment for depression, anxiety, and obsessive-compulsive disorder: A 24-week randomized controlled trial. Psychiatry and Clinical Neurosciences, 78(4), 220–228. https://doi.org/10.1111/pcn.13618
- Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x
- Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., & Denniston, A. K. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension. Nature Medicine, 26, 1364–1374. https://doi.org/10.1038/s41591-020-1034-x
- Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013). Behavioral intervention technologies: Evidence review and recommendations for future research in mental health. General Hospital Psychiatry, 35(4), 332–338. https://doi.org/10.1016/j.genhosppsych.2013.03.008
- National Institute for Health and Care Excellence. (2022). Evidence standards framework for digital health technologies (updated framework). NICE. https://www.nice.org.uk/what-nice-does/digital-health/evidence-standards-framework-esf-for-digital-health-technologies
- Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342
- Onnela, J.-P., & Rauch, S. L. (2016). Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology, 41, 1691–1696. https://doi.org/10.1038/npp.2016.7
- Shoshani, A., Gurfinkel, B., Kor, A., Ben-Haim, Y., Kanarek, O., Segev, R., Shafir, O., & Arbel, R. (2026). Efficacy of a conversational AI agent for psychiatric symptoms and digital therapeutic alliance: A randomized clinical trial. JAMA Network Open, 9(4), e266713. https://doi.org/10.1001/jamanetworkopen.2026.6713
- Sounderajah, V., Guni, A., Liu, X., et al. (2025). The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nature Medicine, 31, 3283–3289. https://doi.org/10.1038/s41591-025-03953-8
- Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092
- U.S. Food and Drug Administration. (2026). Clinical decision support software: Guidance for industry and Food and Drug Administration staff. U.S. Department of Health and Human Services. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software
- Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., et al. (2022). Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine, 28, 924–933. https://doi.org/10.1038/s41591-022-01772-9
- World Health Organization. (2019). WHO guideline: Recommendations on digital interventions for health system strengthening. World Health Organization. https://www.who.int/publications/i/item/9789241550505
- World Health Organization. (2024). Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models. World Health Organization. https://www.who.int/publications/b/70584
- World Health Organization. (2025). Global strategy on digital health 2020–2027. World Health Organization. https://www.who.int/publications/i/item/9789240116870
- Wykes, T., Lipshitz, J., & Schueller, S. M. (2019). Towards the design of ethical standards related to digital mental health and all its applications. Current Treatment Options in Psychiatry, 6, 232–242. https://doi.org/10.1007/s40501-019-00180-0
- Zainal, N. H., Soh, C. P., Van Doren, N., & Benjet, C. (2024). Do the effects of internet-delivered cognitive-behavioral therapy (i-CBT) last after a year and beyond? A meta-analysis of 154 randomized controlled trials (RCTs). Clinical Psychology Review, 114, 102518. https://doi.org/10.1016/j.cpr.2024.102518