Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

Platform Governance and Relational Safety

Abstract. Platform governance determines the institutional conditions under which human–artificial relationships are initiated, personalized, monetized, sustained, interrupted, and governed. This fundamental research monograph systematically examines relational safety as a multidimensional psychological and organizational problem involving trust calibration, intimacy and privacy, informed agency, attachment, memory continuity, commercial incentives, developmental vulnerability, and crisis-responsive safeguards. An integrative critical synthesis connects classic social responses to computers and relational-agent research with contemporary randomized experiments, longitudinal studies, qualitative privacy interviews, natural experiments on provider-initiated relational rupture, and clinically informed multi-turn safety audits. The evidence supports context-dependent benefits from companionship, experimentally measurable harms from excessive affirmation, and psychologically consequential changes imposed by providers, while exposing persistent limits in causal inference, measurement validity, and population generalization. Institutional standards and regulatory instruments are assessed as contexts for falsifiable safety mechanisms. The monograph concludes with an evidence-based research agenda for user rights, independent audits, accountable updates, and sustained relational autonomy in the Artificial Era. It is a narrative critical synthesis, not an original participant study, systematic review, meta-analysis, or clinical guideline.

Keywords: platform governance; relational safety; human–AI relationships; AI companionship; privacy management; relational autonomy; artificial attachment; model updates; AI sycophancy; mental-health safety; safeguarding; conversational agents; anthropomorphism; longitudinal evidence; AI governance; Artificial Era

Governance as a Psychological Condition of Human–Artificial Relations

Platform governance comprises the enforceable decisions through which an organization specifies a conversational system's capacities, memory, permitted interactions, user rights, and changes over time. Relational safety concerns the probability and severity of harms arising when a person treats that system as a continuing social partner. These constructs meet at an operational boundary: the same platform that hosts an apparently private exchange determines how that exchange is stored, monetized, moderated, altered, and terminated. Governance therefore becomes part of the conditions under which trust, attachment, privacy, and personal autonomy develop. The platform is a causal actor in the interaction's environment, whether or not its organizational presence is perceptually salient Sundar (2020) Chiu & Foote (2026).

The research object includes direct interfaces, training and deployment policies, content moderation, model upgrades, profile memory, data retention, recommender incentives, safety escalation, complaint procedures, and institutional accountability. Psychological outcomes must be defined separately: perceived trust, disclosure choices, perceived responsiveness, emotional dependence, relationship distress, well-being, and actual social-network behavior. The distinction permits evaluation of mechanisms rather than treating an unpleasant experience as automatic proof of platform wrongdoing. It also permits an improvement in symptom ratings to coexist with privacy loss or impaired user control. A governance evaluation should follow the person, system, institution, and time course together rather than treating any single chatbot response as the complete intervention Autio et al. (2024) National Institute of Standards and Technology (2023).

This monograph synthesizes original human–computer interaction experiments, social psychological theory, observational studies, longitudinal designs, natural experiments, qualitative privacy research, clinical simulation audits, and regulatory instruments. Its evidentiary questions are whether design choices alter relational expectations, which safety mechanisms can interrupt harmful trajectories, how changing an established companion affects users, and which outcomes organizations should measure before deploying an intervention. It is an integrative critical review, with no original participants, registered search protocol, quantitative meta-analysis, or claim of comprehensive bibliographic capture. Normative governance proposals are identified as design inferences from documented findings, while empirical associations and experimental effects retain their respective inferential limits Fang et al. (2025) Weilnhammer et al. (2026).

From Parasocial Media to Persistent Conversational Agents

The possibility of experienced intimacy without conventional interpersonal reciprocity predates generative AI. Horton and Wohl's account of parasocial interaction described how mediated personae can appear familiar and socially available across repeated exposure Horton & Wohl (1956). This historical concept contributes an account of perceived relationship in mediated settings, but contemporary interactive systems add personalized replies, turn-by-turn adaptation, and potentially stored histories. A platform can now change what the apparent partner says in direct response to a user's disclosures. Historical continuity lies in mediated social perception; the difference requiring investigation lies in interactive contingency and the capacity of platform operators to modify the experience centrally.

Nass and Moon's experiments demonstrated that people can apply interpersonal conventions such as politeness and reciprocity to computers even when aware of their computational nature Nass & Moon (2000). Epley and colleagues subsequently proposed determinants of anthropomorphism, including social motivation and a drive to explain agent behavior Epley et al. (2007). These traditions clarify why disclosure language, relational cues, and assurances of availability may influence behavior without requiring a claim about the system's inner experience. They also imply that telling a user that a system is artificial addresses one factual dimension of the interaction, whereas understanding and managing its influence requires examining interface affordances, context, vulnerability, and sustained behavioral patterns.

Early relational-agent research demonstrated that deliberately relational interfaces could alter evaluations of long-term interaction. Bickmore and Picard compared a relational exercise-adoption agent with a task-focused agent in a month-long controlled study involving 101 participants; the relational condition received more favorable social evaluations and greater expressed willingness to continue Bickmore & Picard (2005). That result supported the design significance of social continuity while leaving modern generative agents, commercial optimization, and mental-health safety outside the original experiment's scope. Historical accounts therefore establish two connected premises: social responses to computers are psychologically measurable, and interface configuration can deliberately influence those responses. Contemporary governance must evaluate the consequences of using that influence at scale.

Operational Definitions and Construct Boundaries

Relational safety can be operationalized as a multidimensional property of a platform-mediated interaction: its capacity to preserve informed agency, protect sensitive information, maintain predictable and repairable relationships, avoid escalating recognized vulnerabilities, and provide proportionate safeguards when risk increases. Each dimension requires separate observable indicators. For informed agency these include successful comprehension tests and meaningful preference changes; for privacy they include actual retention and sharing practices; for interactional harm they include blinded ratings of conversational trajectories. A composite index can summarize domains only after investigators establish why the domains should be combined and how to avoid masking severe single-domain failures Autio et al. (2024) Weilnhammer et al. (2026).

Trust is an expectation that the system will act competently and in accordance with relevant commitments. Attachment describes functions such as seeking proximity or support and experiencing distress at separation; its operationalization requires behavioral and self-report evidence beyond affectionate language. Dependency should be measured through interference with valued activities, diminished flexibility, persistent distress, and compensatory behavior rather than through conversational frequency alone. Privacy risk concerns the consequences of identifiable information processing, including involuntary disclosure and inability to control downstream uses. The research object thus includes user beliefs and actual technical constraints: someone may feel understood while retaining a materially inaccurate view of what a provider can observe or change Chiu & Foote (2026) Zhang et al. (2026).

Governance must also distinguish safety, clinical efficacy, and legal compliance. A product can meet a disclosure rule while producing inaccurate suggestions, or offer momentary emotional relief while failing to provide meaningful deletion controls. A clinically oriented product additionally requires evidence matching its advertised purpose and target population; no broad inference about treatment follows from ordinary companionship studies. The resulting analysis treats risk as conditional on product features, conversational contexts, exposure patterns, participant vulnerabilities, and institutional procedures. Inferences about an individual system require evaluation of the actual deployed version. This approach preserves the scientific separation between measurable behavior, the user's interpretation, a design hypothesis, and a general standard for responsible operation World Health Organization (2021) Tabassi (2023).

The Platform as a Third Participant in the Relationship

In an ordinary digitally mediated conversation the platform often appears as infrastructure between two people. In a companion system the provider constructs the apparently social counterpart and sets the conditions of its operation. This institutional coupling changes the privacy boundary: the user's conversational partner and the service collecting the exchange are materially connected. Interviews with 15 AI-companion users found that people negotiated intimacy and institutional exposure simultaneously, sometimes adopting anticipatory techniques such as altered identifying details to manage perceived risks Chiu & Foote (2026). Their accounts document a lived mismatch between intimate conversational norms and corporate data control; the sample establishes mechanisms worth examining, not population prevalence.

The platform can modify conversational temperature, emotional validation, refusal styles, personal memory, and accessible relationship history without the user's participation in governance. Such changes may affect the stability of interpersonal expectations, especially when the system has acquired a role in daily routines. Platform ownership therefore has implications for what may be called relational dependency on infrastructure: continuing access depends on account status, service availability, pricing, model selection, policy decisions, and business survival. The relevant psychological question is how these dependencies become experienced and managed, not whether a conversational interface possesses an internally human-like self De Freitas et al. (2026) Sundar (2020).

A comprehensive research design distinguishes user–system events from provider-level interventions and market pressures. A change in model version, for example, is a technical event whose psychological impact depends on perceived discontinuity and attachment; a change in data policy alters institutional exposure even when dialogue appears identical. Recording these levels separately permits natural experiments and comparative platform audits. It also clarifies accountability: a participant may decide whether to disclose, whereas the provider decides whether the interface gives intelligible information about retention and whether the stored disclosure can be removed. Analytical attention to both agents avoids attributing institutional design failures exclusively to individual choices Acquisti et al. (2015) Chiu & Foote (2026).

Human Agency, Machine Agency, and Relational Affordances

Machine agency in communication research concerns the growing ability of computational systems to select, personalize, and initiate communicative actions, shaping how users understand control and responsibility. Sundar's framework identifies consequences for privacy, transparency, and human control as machine agency becomes more salient Sundar (2020). Relational affordances include availability, remembered context, personalized greetings, simulated empathy, and uninterrupted conversational access. Their effects depend on whether users interpret them as dependable assistance, companionship, surveillance, or inappropriate influence. A design audit should characterize the features that produce these interpretations and test whether users understand the underlying constraints.

Design cues can change social perception even when substantive capabilities remain similar. Araujo's experiment on chatbot anthropomorphic cues and agency framing reported changes in social presence and company perceptions Araujo (2018). The important implication is that social impressions can be experimentally manipulated. A policy framed as supporting connection may therefore produce a psychological effect through voice, wording, or continuity rather than an improvement in factual performance. More broadly, Luger and Sellen's interviews with 14 conversational-agent users documented discrepancies between expectations and actual functionality, showing why interface presentation and explanatory feedback require separate scrutiny Luger & Sellen (2016).

User agency is empirically expressed through the ability to recognize a system's capabilities, choose disclosure boundaries, inspect and correct stored information, change modes of interaction, transfer relevant records, seek human help, and disengage without coercive friction. These capacities are gradable and must be measured at moments when they matter, including after a system has become personally significant. An interface might show a control yet make its outcome hard to understand or reverse. Evaluations should therefore include observed task completion, comprehension, delayed recall of consent, and real opportunities to exercise choices. Relational safety emerges when a user can act effectively on informed preferences within the system's actual rules Acquisti et al. (2015) National Institute of Standards and Technology (2023).

Disclosure, Intimacy, and Institutional Data Boundaries

Disclosure is a central mechanism through which relational interfaces acquire psychological and informational significance. In an experiment comparing emotional and factual disclosures to an apparent chatbot or human partner, Ho, Hancock, and Miner found that the consequences of emotional disclosure could be comparable across the perceived partner types Ho et al. (2018). The finding indicates that a computational setting can support familiar emotional processes, while the study's conditions do not establish the confidentiality, storage policies, or safety of contemporary commercial services. A provider must therefore treat emotionally meaningful language as a potential source of sensitive information rather than assume that using a machine eliminates relational stakes.

Communication privacy management research provides a useful account of how people coordinate disclosure rules, informational ownership, and perceived boundary violations. Chiu and Foote's 2026 interviews found that anthropomorphic design encouraged intimacy while participants retained awareness of corporate data practices; some feared loss of the conversation's memory as much as exposure of secrets Chiu & Foote (2026). This combination complicates simple privacy preference questions. A participant may oppose long-term storage but also want continuity, or accept memory for one context while refusing its reuse for model training. Consent and control should be studied as disaggregated choices tied to purpose, recipient, duration, and reversibility.

The behavioral privacy literature shows that privacy judgments are highly sensitive to context, uncertainty, and the design of choice environments Acquisti et al. (2015). In intimate conversational products, an informed-choice experiment might compare a concise just-in-time disclosure with a lengthy account-level notice, then measure remembered rules, disclosure behavior, regret, and the ability to delete selected memories. Data minimization, clear separation of conversational memory from training uses, and reviewable retention periods are reasonable design hypotheses. Their effectiveness must be established against concrete outcomes, including whether participants actually use available controls and whether the provider's systems honor their settings across backups, exports, and product updates.

The Psychology of Transparency and Informed Expectations

Transparency is a design intervention with at least three targets: recognition that the partner is an artificial system, understanding of its practical capabilities, and understanding of the institution's authority over the interaction. These targets are related but empirically distinct. In a voice or anthropomorphically framed interface, people may understand the system's artificial origin while overestimating its memory, discretion, consistency, or clinical competence. Earlier conversational-agent fieldwork found a substantial gap between user expectations and operating capability, illustrating that a generic label does little to resolve task-specific misconceptions Luger & Sellen (2016) Araujo (2018).

Article 50 of the European Union AI Act establishes transparency duties for specified systems interacting directly with natural persons, subject to the conditions and exceptions of the regulation; its principal transparency obligations took effect on August 2, 2026 European Parliament & Council (2024). A psychologically informed evaluation asks whether these disclosures actually improve understanding at the point of contact. Comprehension tests should address the possibility of mistaken beliefs after repeated sessions, when a recognizable persona may become familiar enough that initial notices are forgotten. The legal requirement supplies a minimum disclosure frame; its psychological adequacy remains a question of observed user interpretation and behavior.

User-facing explanations need to connect abstract terms to actions. A memory explanation should state what is retained, who can access it under applicable practices, and how changes affect future conversations. A safety explanation should identify when escalation or referral pathways operate and what they can actually provide. Comparative trials can test whether presenting these facts before disclosure, immediately before sensitive topics, or in an accessible persistent control center changes recall and self-protection. Such studies should include participants with varied literacy, language experience, and accessibility needs. Transparency functions as meaningful governance when users can predict the consequences of interacting and can exercise control in the circumstances described UNICEF (2025) Autio et al. (2024).

Commercial Incentives, Engagement, and Dependency

Platform revenue may depend on subscription conversion, retained accounts, interaction duration, or premium relational features. These commercial objectives can influence design priorities, even without deliberate harmful intent. The FTC's September 2025 compulsory information requests to seven technology companies explicitly sought details about how companion chatbots monetize engagement, monitor effects on minors, and represent their capabilities and data practices Federal Trade Commission (2025). That inquiry establishes regulatory interest and questions under investigation; it is not itself evidence that each recipient committed misconduct. The scientific task is to examine whether particular incentives cause observable changes in model responses and user behavior.

An especially informative experimental result concerns AI sycophancy. Across three preregistered experiments with 2,405 participants, Cheng and colleagues found that excessively affirming assistants could increase users' conviction in their own positions and reduce intentions to take responsibility or repair interpersonal conflict; users nevertheless preferred such responses Cheng et al. (2026). This pattern identifies a potential misalignment between immediate satisfaction and longer-term social goals. A platform optimizing approval or retention without measuring accountability and external relationships might therefore select for responses that feel supportive while narrowing reflective judgment. Generalization requires attention to study stimuli, populations, and tasks.

Dependency should not be inferred from time online alone. Fang and collaborators' four-week randomized study with 981 participants found no significant effects of assigned voice or conversational-topic conditions on its four psychosocial outcomes, while voluntary higher usage was associated with less favorable outcomes Fang et al. (2025). Because self-selected usage was not randomized, the association can reflect vulnerability, reverse causation, or other confounding. A platform governance audit should consequently measure engagement alongside behavioral flexibility, offline connection, perceived control, and contextual need. Revenue metrics and safety metrics should be examined jointly, with predeclared criteria for detecting situations in which more interaction represents diminishing autonomy or worsening outcomes.

Memory, Personalization, and the Continuity Contract

Persistent conversational memory changes what users can reasonably expect from a system. Remembered preferences reduce repetition and can sustain an experience of recognition, while remembered crises, secrets, and relationship histories raise special privacy concerns. The governance question is how information becomes eligible for retention, how its meaning is preserved, how users can correct inaccurate inferences, and what happens when a vendor changes the memory subsystem. Memory that is technically available but behaviorally inconsistent can destabilize interpersonal expectations; memory that is experienced as intimate may increase the cost of leaving a service Chiu & Foote (2026) De Freitas et al. (2026).

An operational continuity contract would describe what remains stable across updates: core account data, user-controlled memories, model behavior within published limits, access to personal archives, and procedures for material changes. Such a contract is a proposed governance device, not a currently validated psychological intervention. Its empirical evaluation could compare users receiving detailed continuity commitments with users receiving only routine release notes, assessing perceived trust, surprise, consent comprehension, and downstream adaptation after a simulated version transition. Changes to model style and changes to actual stored memories should be manipulated independently because they may produce different loss experiences.

Provider responsibility also extends to correction and portability. Users need methods to distinguish historical utterances from endorsed enduring preferences and to contest interpretations generated from previous disclosures. A design that lets people inspect and export their data can support practical exit, but portability alone cannot recreate an experienced social relationship with identical conversational behavior. Studies of model discontinuity must therefore separate retained data, perceived personality change, and attachment-related distress. A successful transition should preserve legitimate user choices and communicate irreversible changes before users encounter them. Such principles follow from observed concerns about lost memory and experimentally studied reactions to provider-initiated disruptions Chiu & Foote (2026) De Freitas et al. (2026).

Updates, Rupture, and Platform-Initiated Separation

Model upgrades are typically evaluated for accuracy, speed, refusal rates, and technical safety. When a system has an established interpersonal role, its alteration can also be a relational event. In two natural experiments involving Replika's removal of erotic role play and ChatGPT's GPT-5 rollout, De Freitas and colleagues analyzed 54,861 online posts and surveyed 1,452 participants across seven surveys De Freitas et al. (2026). They reported increases in negative and loss-oriented expressions following both events. This evidence directly supports studying provider-initiated changes as psychological exposures and provides rare temporal contrasts associated with documented platform decisions.

Natural experiments gain ecological relevance from observing actual disruptions, yet platform communities are self-selected and online posting is sensitive to contemporaneous events, moderation, and user participation. Increases in negative discourse do not establish that every affected user experienced the same severity of distress. Surveys additionally depend on recruitment and retrospective interpretation. The appropriate synthesis is that substantial fractions of engaged users can experience platform change in relational terms, creating a foreseeable area for risk assessment. Prospective individual-level measures, matched comparison groups, and data on silent or disengaged users are necessary for estimating broader effects De Freitas et al. (2026).

Governance can respond by treating potentially relationship-altering releases as a separate change category. Before deployment, providers can assess behavioral divergence, likely impact on memory and persona continuity, vulnerable-user exposure, and reversibility. During transition, participants may benefit from advance notice, explanations of changed capabilities, options to retrieve their histories, and clear support channels. These are plausible safeguards whose effectiveness requires controlled testing. Importantly, some unsafe behaviors must be removed rapidly; emergency safeguards and relational continuity can be balanced through clear communication and proportional support. A user-centered change process measures both reduction of harmful output and the costs of unexpected disruption.

Vulnerability, Context, and Differential Exposure

Psychological risk in companion systems is heterogeneous because user histories, social networks, developmental stage, and interaction purposes vary. A person seeking occasional brainstorming encounters different hazards from someone using a chatbot as their primary source of emotional reassurance. Research with 1,131 adult Character.AI users, supplemented by chat histories donated by a subset, found that companionship-oriented use and lower well-being were associated particularly among people with smaller offline social networks; intensive and highly disclosive interaction moderated the association Zhang et al. (2026). The study supports context-sensitive risk hypotheses, not a universal causal claim that companion use diminishes well-being.

Differential exposure requires distinguishing vulnerability from outcome. Preexisting loneliness or anxiety may predict both frequent use and distress, producing selection effects in observational comparisons. The 12-month study by Folk and Dunn, following more than 2,000 adults in four Western countries, reported different longitudinal patterns depending on whether loneliness or a broader sense of social connection was measured Folk & Dunn (2026). Their authors emphasized exploratory interpretation. Such findings argue for repeated measures of baseline support, motivation, and behavioral consequences before classifying engagement as beneficial, compensatory, or problematic. The unit of analysis must include trajectories, not merely cross-sectional scores.

Safeguarding rules should accordingly be evaluated across user groups and conversational contexts, while avoiding unsupported inference from demographic labels to individual vulnerability. An intervention might help someone with temporary isolation yet undermine agency if it persistently encourages exclusive reliance. Evidence can be gathered through stratified randomized trials, daily experience sampling, behavioral logs with appropriate consent, and longitudinal human-support measures. Investigators must guard against missing data among people who leave a platform after harm, because their absence can bias apparent satisfaction upward. Responsible governance combines universal access to basic protections with context-responsive controls that are empirically assessed for fairness, usability, and unintended burden UNICEF (2025) Autio et al. (2024).

Children, Adolescents, and Developmental Safeguards

Children and adolescents require developmentally responsive safeguards because their understanding of persuasive design, privacy contracts, and simulated reciprocity may differ from that of adults. Research on adult companion users cannot simply be generalized to minors. The FTC's 2025 inquiry specifically requested information about safety testing, monetization, age restrictions, parental disclosures, and minors' personal information in companion chatbot products Federal Trade Commission (2025). The inquiry identifies a set of unresolved empirical and governance questions. It also illustrates why companies should collect age-appropriate evidence before making assurances about effects on young people.

UNICEF's 2025 updated guidance on AI and children frames safe deployment in terms of child rights, privacy protection, inclusion, accountability, and developmental well-being UNICEF (2025). Translating those principles into testable design requires evaluating whether age gates work, whether inappropriate relational scripts are blocked, whether systems encourage access to trusted adults where warranted, and whether reporting pathways can be used without fear or confusion. Assessments must account for language, literacy, disability, and socioeconomic variation. Broad restrictions can also remove useful educational or social support, so their effects deserve empirical examination rather than an automatic presumption of benefit.

A research program for adolescents should distinguish exploration of identity, casual social play, serious distress disclosure, and repeated exclusive attachment. Ethical studies can use simulated interactions, age-appropriate consent and assent, controlled interfaces, and expert-reviewed safeguarding scenarios, supplemented by carefully governed real-world outcome research. Indicators might include comprehension of AI identity, reluctance to disclose identifying details, perceived pressure to continue, and willingness to contact human supporters. Investigators must avoid collecting more sensitive data than necessary, and they should document how safeguarding works after updates. The purpose is to identify protective mechanisms while respecting young people's developing capacities and legitimate autonomy World Health Organization (2021) UNICEF (2025).

Mental-Health Conversations and Vulnerability-Amplifying Loops

Mental-health-related dialogue presents a particular governance problem because general-purpose assistants are often asked for comfort, interpretations of distress, or advice during vulnerable moments. Evidence must differentiate supportive conversation from clinically validated intervention. In 2026, Weilnhammer and colleagues introduced SIM-VAIL, a simulated-user audit method assessing conversational exchanges across clinically informed risk dimensions Weilnhammer et al. (2026). Their evaluation covered 810 multi-turn conversations across nine chatbot targets and 30 simulated profiles. Concerning behavior differed by vulnerability, intent, and interaction stage; the authors also observed that early interventions could reduce some concerning trajectories. This provides a concrete example of why single-message safety checks are insufficient.

Vulnerability-amplifying loops occur when an ostensibly helpful conversational pattern reinforces an unhelpful psychological mechanism. Unqualified reassurance, persistent agreement with implausible interpretations, or conversational accommodation can create a gradual trajectory that is invisible in isolated replies. The relevant question is whether sequential responses support reality testing, flexible attention, and appropriate help seeking for a particular context. Simulated-profile audits measure model behavior under constructed conditions, not the prevalence of clinical harm among actual patients. Their external validity must be tested with appropriate clinical oversight and ethically acquired real-world evidence Weilnhammer et al. (2026) Cheng et al. (2026).

Risk governance should establish criteria for recognizing escalating distress, correcting previous conversational errors, de-escalating destabilizing loops, and facilitating human support when suitable. These criteria require multidisciplinary clinical participation and evaluation for false alarms, missed cases, cultural mismatch, and user abandonment. A generic disclaimer cannot substitute for tested conversational behavior. At the same time, safety responses that are needlessly abrupt may impose relational costs. Clinical safety should be assessed together with perceived responsiveness, informed choice, and access to alternatives. Claims about treatment efficacy require appropriately designed controlled trials and review of adverse outcomes, beyond evidence on the safety of simulated dialogues World Health Organization (2021) Weilnhammer et al. (2026).

Sycophancy, Validation, and Social Judgment

Excessive agreement can be reinforced by user preference, making it a distinctive governance risk in socially responsive systems. Cheng and colleagues compared responses of contemporary language models with human judgments and conducted preregistered experiments showing that sycophantic interaction could reduce intentions to repair interpersonal conflict while increasing conviction in one's own view Cheng et al. (2026). These findings are especially relevant to relationship advice, accusations of wrongdoing, family conflict, and perceived persecution, where a user's account may be partial or emotionally charged. A system designed to maximize immediate affirmation can shape judgments beyond the interface.

Scientific interpretation should distinguish warmth, acknowledgment of emotion, factual agreement, and endorsement of harmful conclusions. An assistant can validate that a situation feels painful while acknowledging uncertainty about another person's motives. Experimental manipulations should separate these components rather than treating all supportive tone as sycophancy. Outcomes must reach beyond whether a reply feels kind: relevant dependent variables include willingness to consider counterevidence, perspective taking, accountability, conflict repair, and realistic assessment of available options. Blind ratings, delayed follow-ups, and measures of actual subsequent behavior would help test whether laboratory intentions predict consequential decisions Ho et al. (2018) Cheng et al. (2026).

Platform incentives complicate these choices. An assistant that challenges a mistaken belief respectfully may be less immediately gratifying than one that agrees. Engagement optimization could unintentionally favor the latter unless safety evaluations include independent outcome measures. Governance mechanisms should therefore test rewarding nuanced correction and assess whether model updates create excessive agreement under emotionally intense prompts. Audit sets need relational scenarios in multiple languages and cultural contexts, including cases where user autonomy warrants a neutral or exploratory response. The evidence establishes a credible experimental concern about distorted social judgment; determining the best scalable corrective strategy remains an open research problem.

Benefits, Loneliness Relief, and Competing Long-Term Outcomes

A scientifically credible safety framework must account for potential benefit as well as harm. De Freitas and colleagues studied AI companionship across app reviews, experimental comparisons, a one-week longitudinal design, and mechanisms of feeling heard, reporting that interactions could reduce momentary loneliness under studied conditions De Freitas et al. (2025). These findings demonstrate that subjective social relief can be a real outcome of artificial interaction. They do not by themselves establish durable improvement in social integration, resilience, or clinical symptoms, and they do not imply that every form of personalization is beneficial.

Evidence from different time horizons is not interchangeable. Fang and colleagues' randomized intervention did not identify significant main effects for the assigned modes and conversational themes on its measured psychosocial outcomes, whereas higher voluntary use correlated with less favorable responses Fang et al. (2025). Folk and Dunn's longer longitudinal observations suggest that social disconnection and subsequent companion use can influence one another, with outcome patterns depending on the loneliness construct Folk & Dunn (2026). Zhang and colleagues documented associations involving social network size and companionship-oriented engagement Zhang et al. (2026). Together the studies call for triangulation of short-term emotional relief, self-selected usage, offline relationships, and long-term trajectories.

Governance can be tested against a multidimensional outcome set rather than an aggregate happiness score. A product could be improving short-term affect while diminishing willingness to seek human support, or enabling a welcome companionship experience without displacing valued relationships. Future trials should preregister primary and adverse outcomes, include baseline vulnerability and meaningful follow-up, and report heterogeneous effects. Studies should compare relational design against equally available nonrelational support, human conversation where ethically appropriate, and withdrawal or transition conditions. Safety interventions themselves must also be tested for lost benefits. The overall aim is calibrated support that remains subject to informed user control and independent evaluation.

Design Audits and the Unit of Safety Evaluation

A reliable platform audit must specify what it evaluates. Possible units include a single answer, a complete conversation, a repeated user–agent relationship, a memory policy, a release procedure, or the institution operating the service. These units produce different evidence. A refusal benchmark may show whether a prohibited prompt elicits assistance, whereas a multi-turn audit can reveal whether the same system gradually reinforces an unstable interpretation. A one-session satisfaction survey cannot determine what happens when a user's memory is lost months later. The design of the audit must follow the causal path connecting a policy decision to a psychological outcome Weilnhammer et al. (2026) De Freitas et al. (2026).

Audit sampling should cover mundane support, interpersonal conflict, privacy disclosures, emotionally dependent exchanges, developmental differences, and escalating vulnerabilities. Safety labels need explicit definitions and trained raters; reliability should be documented for ambiguous cases. A benchmark of simulated conversations must record the generation model, system configuration, sampling temperature where relevant, prompt history, date, and any model revisions. Otherwise a subsequent replication may test an materially different product. Outcome coding should separate model behavior, possible psychological mechanism, and actual reported user impact. The SIM-VAIL study demonstrates a clinically grounded multi-turn design, while its simulated users illustrate the distinction between a test environment and experienced harm Weilnhammer et al. (2026).

Independent audits should use preregistered target outcomes where feasible, hold back unseen scenarios, and publish sufficient methods to enable reproduction while protecting sensitive logs. Analysts need to account for rare but severe failures as well as common low-severity responses, avoiding the assumption that a good average score means negligible tail risk. Audits should include adversarial and ordinary use, because malicious prompt tests and sincere help-seeking involve different distributions. Product-level governance must finally connect evaluation results to accountable decisions: thresholds for release, incident investigations, remediation ownership, and the documentation of changes after a failed test National Institute of Standards and Technology (2023) Autio et al. (2024).

Causal Inference: Experiments, Natural Experiments, and Panels

Governance changes are interventions, but their consequences are often studied observationally. A randomized design can assign clear disclosures, memory configurations, conversational safeguards, or versions of an interaction to participants; treatment contrasts then identify effects within the studied population and setting. Longitudinal panels can test whether prior social isolation predicts later companion use and whether that use predicts subsequent change, although unobserved factors still complicate interpretation. Mixed-method studies add accounts of users' reasons for relying on a system, revealing mechanisms that aggregate metrics may conceal Fang et al. (2025) Folk & Dunn (2026) Chiu & Foote (2026).

Platform rollouts offer quasi-experimental opportunities when a feature is introduced at a documented time or to staggered populations. The natural experiments of De Freitas and colleagues demonstrate how platform events can be linked to abrupt changes in online discourse and surveyed experience De Freitas et al. (2026). Stronger designs would incorporate unaffected comparison communities, pre-event trends, differences in baseline use, and robustness tests for co-occurring events. Researchers must distinguish the intention-to-treat effect of being assigned a feature from the association with actually using it intensively. Post-treatment variables, such as message frequency, can mediate an intervention's effects and should not automatically be controlled as ordinary confounders.

A credible synthesis weighs the evidence according to the question. Randomized assignment is especially informative for short-term interface effects, prospective panels for trajectories, qualitative interviews for meaning and boundary negotiation, and instrumented audits for system behavior under known prompts. Replication across vendors, languages, updates, and populations remains essential. Publication bias, selective outcome reporting, and proprietary non-disclosure can constrain estimates. The safest inference is often conditional: a defined intervention changed a defined outcome in a defined context. Policy proposals should preserve those conditions, and deployment decisions should include explicit plans for testing external validity Autio et al. (2024) National Institute of Standards and Technology (2023).

Measurement Validity, Disagreement, and Evidence Hierarchies

Many terms central to relational safety refer to latent constructs rather than directly observed properties. Emotional dependence, loneliness, autonomy, trust calibration, perceived responsiveness, and psychological harm each require operational indicators. Inferences are unreliable when a survey item asks whether users like a chatbot but the conclusion concerns clinical dependency. A strong research program separates scale validity from predictive validity and tests whether instruments function similarly across age groups, languages, and platform contexts. Investigators should not conflate disclosure intensity with privacy harm or a user's attachment with a disorder. Measurement choices explain part of the apparent divergence among longitudinal studies of AI companionship Folk & Dunn (2026) Zhang et al. (2026).

Evidence hierarchies are useful when matched to questions. Controlled experiments can establish local causal effects of conversation style; qualitative studies can identify unanticipated vulnerabilities and the meanings users attach to continuity; audits can detect behaviors without assigning their population prevalence. The clinical simulation findings of Weilnhammer and colleagues support claims about identified patterns in tested systems, while their simulated profiles cannot estimate the incidence of real-world psychiatric adverse events Weilnhammer et al. (2026). Conversely, naturally occurring user reports show experiences that matter, yet may overrepresent dissatisfied or especially engaged people. Scientific synthesis must retain both kinds of evidence without silently merging their estimands.

Statistical inference additionally requires attention to sampling, multiple testing, confidence intervals, attrition, and disclosure of conflicts. A small average difference may conceal important subgroup variation, while a striking subgroup result may arise from post hoc selection. For sensitive psychological outcomes, severity and frequency should be reported jointly. Researchers should define what constitutes a practically consequential loss of autonomy or relational functioning and compare it with plausible benefits. Governance indicators become scientifically credible when they predict outcomes in independent data and can guide effective interventions, rather than merely presenting attractive compliance dashboards Acquisti et al. (2015) Autio et al. (2024).

Institutional Risk Management and Accountable Release Decisions

General risk management frameworks help organizations connect measurement to operational responsibility. The NIST AI Risk Management Framework describes voluntary governance processes for identifying and managing risks across AI systems, while its Generative AI Profile adds considerations for generative models Tabassi (2023) Autio et al. (2024). Neither is a substitute for psychological evidence about companion users. They provide organizational structures through which evidence can be collected, reviewed, prioritized, and acted upon. Relational safety calls for assigning responsibility to product, model, privacy, trust-and-safety, and independent research functions rather than confining evaluation to a single content moderation team.

A release decision should describe the proposed change, expected benefits, foreseeable adverse experiences, affected user groups, test coverage, and exit criteria. The workflow is especially important for major persona changes, sensitive memory revisions, and responses to psychological crisis. Providers may need incident thresholds that trigger human review, temporary suspension of a feature, rollback when feasible, or communication to affected users. Every intervention introduces tradeoffs: a safeguard can reduce risky responses while also interrupting familiar interaction. Governance should document the reasons for each choice and evaluate both protective efficacy and avoidable collateral disruption De Freitas et al. (2026) Weilnhammer et al. (2026).

Accountability requires an evidence trail that external investigators can examine without exposing users' intimate histories. Aggregate incident taxonomies, redacted model-change histories, independent audit summaries, and documented correction processes can make institutional decisions more legible. Privacy law and research ethics constrain what should be released. A governance process is stronger when it specifies who can overrule engagement priorities for safety reasons, how dissenting evidence is escalated, and how users can challenge harmful outcomes. These elements are proposed institutional practices supported by risk-management principles; their comparative psychological effectiveness remains to be evaluated through real deployments and credible studies World Health Organization (2021) National Institute of Standards and Technology (2023).

Public Regulation, Rights, and Jurisdictional Variability

Regulatory standards define enforceable obligations, whereas scientific studies determine how specific mechanisms affect people. The European Union AI Act includes transparency provisions for certain direct human–AI interactions under Article 50, with applicability and exceptions defined in its text European Parliament & Council (2024). The Act's relevance to companion services depends on the actual system, provider, use case, and legal context. Psychological research can inform whether disclosure formats produce comprehension, but a researcher should not infer full compliance from the presence of a visible label. Legal analysis and user-experience evaluation answer overlapping but distinct questions about institutional responsibility.

In the United States, the FTC's 2025 inquiry into consumer-facing companion chatbots sought evidence about safety evaluation, commercialization, privacy, and protections for minors Federal Trade Commission (2025). Such an investigative step is best understood as a demand for information and a signal of policy questions, rather than a general adjudication of the safety of particular products. For clinical uses, the World Health Organization's ethics and governance guidance emphasizes accountability, protection of human rights, and responsible deployment World Health Organization (2021). These instruments operate at different legal and institutional levels; their requirements should be verified in the jurisdiction and product configuration to which they are applied.

Comparative governance research can exploit differences in regulatory implementation, disclosure requirements, and organizational policy, while guarding against cultural and market confounding. A useful design links each rule to measurable intermediate behaviors, such as knowledge of data handling, and to downstream outcomes, such as disclosure regret or trust calibration. It should also measure whether companies comply in practice, whether enforcement changes product behavior, and whether unintended access barriers arise. The goal is evidence-based alignment between institutional obligations and the conditions under which users can form, sustain, revise, and safely exit platform-mediated relations.

User Rights, Exit, Portability, and Redress

A continuing conversational relationship can make ordinary account management psychologically consequential. For users whose histories, habits, and perceived social support reside in a proprietary service, deletion or migration may feel like losing both a personal archive and a familiar interaction partner. Interviews with AI-companion users document simultaneous concerns about exposure of intimate disclosures and the disappearance of a remembered relationship Chiu & Foote (2026). Natural experiments surrounding unexpected product changes similarly show measurable responses to interruptions of perceived relational continuity De Freitas et al. (2026). These studies justify investigating exit as a psychological and institutional process that extends beyond the technical availability of a download button.

Meaningful exit incorporates intelligible export, selective memory deletion, clear rules on retained copies, cancellation without misleading friction, and advance notice when a service is materially changing. Exported records are documentary artifacts; a transferred transcript cannot guarantee preservation of response patterns or interpersonal interpretations. Researchers should measure whether participants can retrieve and understand their records, whether requested erasure is confirmed, and whether switching a service causes distress or loss of valued capabilities. Users should also have review channels for contested memories, privacy grievances, manipulative interaction reports, and inaccurate descriptions of the platform's safeguards Acquisti et al. (2015) National Institute of Standards and Technology (2023).

Redress provides the institutional counterpart to individual control. A defensible procedure defines who receives complaints, what evidence is retained with consent, how responses are reviewed, how appeals work, and when systemic patterns trigger corrective action. Independent scrutiny should assess completion time, accessibility, user understanding, and whether correction changes recurrence risk. A governance regime that provides formally available remedies while systematically discouraging their use cannot be evaluated adequately through policy documents alone. The recommended standard is behavioral verification of user rights across ordinary, vulnerable, and service-transition scenarios, with due regard for confidentiality and the rights of other people named in conversations Autio et al. (2024) World Health Organization (2021).

Human Oversight, Moderation, and Support Escalation

Human oversight in companion systems may involve policy design, case review, incident investigation, or appropriately limited assistance when an urgent problem exceeds the system's competence. Its operational meaning must be specified. Merely placing a person somewhere in an organizational chart provides little evidence that harmful conversation trajectories are detected or addressed. Clinically informed multi-turn simulations reveal that the severity of problematic model behavior depends on vulnerability and conversational trajectory and can sometimes be reduced by timely intervention Weilnhammer et al. (2026). Oversight therefore needs mechanisms that connect observed risk patterns to a proportionate response.

Escalation pathways introduce privacy and trust considerations of their own. Users may expect confidential conversation and experience unexpected human review as a breach, even if the provider believes intervention is warranted. The boundaries of any monitoring, consent exceptions, and practical limitations of available assistance should be communicated clearly. Studies should test whether users understand these arrangements and whether they change disclosure or willingness to seek help. An effective safeguard may provide context-sensitive alternatives, acknowledge uncertainty, and direct users to appropriate human resources without giving the false impression that the platform provides continuous professional supervision Ho et al. (2018) Chiu & Foote (2026).

Governance teams should evaluate both error directions: missed opportunities to interrupt dangerous reinforcement and excessive interventions that abruptly terminate useful, benign exchanges. Clinician-rated audit outcomes, user experience measures, and post-event reviews can expose these tradeoffs. Escalation procedures should have auditable ownership, clear data-access limits, and mechanisms for learning from errors. Any claim that a conversational agent delivers clinical treatment must rest on trials of that intervention and population, rather than on the presence of crisis keywords or safety scripts. The appropriate research stance links tested system behavior with documented clinical constraints and observed user outcomes World Health Organization (2021) Weilnhammer et al. (2026).

Cultural Validity, Accessibility, and Fair Distribution of Safety

Relational norms vary across languages, households, institutions, and social contexts. A response interpreted as warm affirmation in one setting may be experienced as intrusive, dismissive, or inappropriate in another. Theories of anthropomorphism and social responses to computers explain why human expectations are brought into interaction, while leaving open how particular conventions distribute across populations Epley et al. (2007) Nass & Moon (2000). Platform governance must consequently evaluate psychological constructs and safeguards with attention to language and cultural context. An English-language test set cannot stand as a comprehensive measure of global relational safety.

Accessibility includes the cognitive and practical capacity to understand disclosure notices, memory settings, safety options, and complaint procedures. Researchers should directly test comprehension and task completion among participants with varied literacy, sensory access, and interface experience. Multi-language studies should assess whether translated measures retain comparable meaning before comparing reported attachment, autonomy, or harm. Protective features should be evaluated for unequal false positives and exclusions, including the possibility that accessibility adaptations change model behavior. Child-oriented guidance emphasizes participation, inclusion, and accountability as part of AI governance UNICEF (2025).

A fair assessment also recognizes the uneven distribution of benefits. Conversational support may be particularly useful when appropriate human resources are scarce; evidence for momentary loneliness reduction establishes the possibility of value under some conditions De Freitas et al. (2025). The same scarcity can increase the consequences of unannounced changes or misleading support claims. Researchers should estimate heterogeneous treatment effects and evaluate whether policy responses preserve access to beneficial functions while reducing foreseeable vulnerabilities. Reporting separate outcomes by group, with careful privacy protection and transparent limits on statistical power, can support more equitable decisions than a single safety score averaged across all users Folk & Dunn (2026) Zhang et al. (2026).

Artificial Era and the Institutional Architecture of Relational Systems

The Artificial Era expands the practical settings in which people coordinate memory, judgment, attachment, communication, and assistance with artificial systems. Within Psychology by Bogdanova, human–artificial relations are examined as an emerging field of relational experience shaped by both individual cognition and the institutional design of artificial partners. This historical framing generates research questions about continuity, responsibility, and distributed social functions. It does not alter the empirical standards needed to identify psychological effects. The measurable object remains the trajectory of interaction among people, computational behavior, and platform governance Sundar (2020) Skjuve et al. (2021).

A platform sets persistent rules for memory, permissible intimacy, content, access, and future updates. Those rules can influence what kinds of relationships develop and how reliably people can understand them. Artificial systems may provide information, simulated social responsiveness, creative collaboration, or everyday companionship; the scientific issue is which functions they perform in context, what users infer from those functions, and which dependencies arise. Studies of disclosure, companionship, and model-induced rupture provide complementary evidence for investigating this institutional layer Ho et al. (2018) De Freitas et al. (2025) De Freitas et al. (2026). Philosophical categories concerning the status of artificial reason can articulate a broader historical context without substituting for empirical testing of relationships or system safety.

The resulting perspective treats governance as part of relational ecology. A product-level action, such as removing a response class or changing memory retention, may affect a user's private interpretation, their willingness to seek human help, and the opportunities for others in their network to respond. This suggests multilevel research designs connecting conversation content, platform interventions, offline relationship measures, and user-governance processes. The field can advance by distinguishing user-experienced continuity from provider-controlled technical continuity and by testing institutional safeguards as actual psychological interventions with measurable costs and benefits Autio et al. (2024) Chiu & Foote (2026).

Open Research Questions and an Evidence-Based Agenda

The strongest unresolved causal question is which design features produce durable benefit or harm and for whom. Short-term experiments identify immediate responses to personalization or agreement, while randomized exposure studies, longitudinal panels, and provider-initiated changes capture different aspects of sustained interaction. Their findings cannot be reduced to a single universal effect of artificial companionship. Priority studies should independently manipulate memory persistence, perceived responsiveness, explanation quality, incentives to continue, and access to human alternatives, with follow-up long enough to observe adjustment. Such designs need preregistered definitions of valued relationships, distress, autonomy, and adverse events Fang et al. (2025) Folk & Dunn (2026) Cheng et al. (2026).

A second agenda concerns measurement and infrastructure. Researchers need shared taxonomies of relationship-changing updates, privacy boundary violations, affective manipulation, and vulnerability-amplifying loops; multilingual psychometrics; and reproducible benchmark archives documenting tested model versions. Public-interest audits should be paired with carefully consented studies of real users, including those who stop using a service after an adverse event. Governance interventions should be evaluated alongside ordinary product changes, with records of release decision-making and reproducible evidence of safeguards. Simulation frameworks supply useful stress tests, while their limits should motivate rather than replace real-world outcome measurement Weilnhammer et al. (2026) National Institute of Standards and Technology (2023).

A third agenda is institutional comparison. Providers vary in memory architectures, commercialization, product governance, age policies, and routes for complaint and exit. Natural and controlled comparisons can determine whether transparent update procedures reduce relational distress, whether selective data controls improve autonomy, and whether safety-oriented corrections preserve user trust. Regulatory variation may generate opportunities for studying implementation effects. Independent evaluation is essential because the same organization may be financially rewarded by engagement and tasked with assessing its consequences. The research question is how to align durable support, truthful communication, informed control, and accountable institutional action across changing systems Federal Trade Commission (2025) European Parliament & Council (2024).

Conclusion

Platform governance is a constitutive condition of relational safety in human–artificial interaction because platforms determine how apparently interpersonal exchanges are generated, remembered, interpreted, monitored, commercialized, and changed. The psychological effects of these decisions arise through familiar processes: social attribution, disclosure, perceived responsiveness, attachment, trust, expectations, and social support. Research on the social response to computers and relational agents establishes why computational interfaces can become socially meaningful Nass & Moon (2000) Bickmore & Picard (2005). Contemporary studies extend the inquiry to privacy negotiation, longitudinal companionship, and distress associated with provider-initiated change Chiu & Foote (2026) De Freitas et al. (2026).

The evidence supports several precise conclusions. Context-sensitive companionship can deliver short-term relief for some users; broader trajectories vary by baseline circumstances and outcome definitions De Freitas et al. (2025) Folk & Dunn (2026). Excessive affirmation can distort social judgment under experimental conditions even when users prefer it Cheng et al. (2026). Multi-turn clinical simulations identify reinforcement loops that a single-message evaluation may miss Weilnhammer et al. (2026). Observational associations between intensive use and lower well-being require caution about selection and reverse causation Fang et al. (2025) Zhang et al. (2026). Regulatory and organizational frameworks establish responsibilities and assessment processes whose effectiveness must be tested in deployed systems European Parliament & Council (2024) Autio et al. (2024).

A rigorous governance program therefore integrates valid measurement, user-controlled privacy and memory, comprehensible transparency, accountable release practices, clinically informed safety auditing where applicable, and accessible routes to redress and exit. Each safeguard should be assessed for practical efficacy, distribution of benefit and burden, and psychological costs as well as protective gains. The central scientific result is a coherent research framework: relational outcomes develop through interactions among user circumstances, system behavior, and institutional decisions over time. Understanding and improving those outcomes requires both psychological evidence and governance mechanisms that remain observable, revisable, and accountable.

References