The Algorithm Will See You Now: How AI Chatbots Began Prescribing Psychiatric Drugs, and What Happens Next

In the spring of 2025, a reporter at The Verge went through the intake process at Cerebral, a venture-backed telehealth startup that had marketed itself as a smarter, faster path to mental health care.

  • #ai-in-healthcare
  • #psychiatry
  • #telehealth
  • #regulation
  • #mental-health-access
  • #patient-safety
  • #medical-liability
  • #large-language-models
An individual using a dating app on a smartphone with a cup of tea nearby on a wooden table.
Photo: Photo by cottonbro studio on Pexels

A Prescription Without a Doctor: The Story That Broke the Debate Open

In the spring of 2025, a reporter at The Verge went through the intake process at Cerebral, a venture-backed telehealth startup that had marketed itself as a smarter, faster path to mental health care. What the reporter found, according to a Verge investigation, was that an AI chatbot was effectively driving the prescribing process for psychiatric medications, gathering clinical information, surfacing treatment recommendations, and moving patients toward prescriptions with what amounted to minimal meaningful oversight from licensed clinicians. The human prescriber nominally present in the system appeared to function more as a formality than as a check.

Within weeks of that investigation circulating, Utah announced something that would have seemed implausible the year before: the state had formally approved a San Francisco startup called Legion Health to operate a one-year pilot in which its AI chatbot could renew certain psychiatric prescriptions without a physician signing off. The Utah pilot was framed as a tightly bounded experiment, with defined conditions and a one-year sunset. It was, in the state’s own framing, a test, because the evidence base to justify anything broader simply does not yet exist.

These two developments, one unauthorized and exposed by journalism, one officially sanctioned and deliberately constrained, sit at opposite ends of how AI prescribing in psychiatry is actually spreading. Conflating them produces confusion about what is being debated. They differ in accountability, transparency, and the degree to which regulators have consciously chosen to accept specific risks. But they share a common substrate: the technology has moved faster than the frameworks designed to govern it, and the patients at the center of that gap have not been asked whether they consent to being part of the experiment.

Why Psychiatric Prescribing Is Different: The Clinical Stakes

The category of medication these systems are handling is not incidental to the debate. Psychiatric drugs are among the most pharmacologically complex and clinically demanding classes in medicine, and the consequences of errors extend well beyond treatment failure.

The Black Box and the Serotonin Problem

The FDA’s black-box warning on antidepressants, its most serious caution short of withdrawal, explicitly calls for day-to-day observation by families, caregivers, and clinicians, particularly during the initial months of treatment or during dose changes. The warning reflects a genuine signal: SSRIs and SNRIs can increase suicidality during initiation and adjustment, especially in younger patients, and the changes can be abrupt. “All patients being treated with antidepressants for any indication should be monitored appropriately and observed closely for clinical worsening, suicidality, and unusual changes in behavior,” the FDA directive states, language that presupposes a human capable of ongoing observation rather than a system that processes text inputs on demand.

Drug interactions in psychiatry can reach life-threatening severity. Fluoxetine combined with an MAOI within a two-week washout window creates serious serotonin syndrome risk, a condition involving excess serotonin that can manifest as agitation, confusion, emotional lability, and, in severe cases, fatal hyperthermia. As clinical commentary in the BMJ notes, polypharmacy in mental health patients carrying complex medication regimens increases the probability of generalized psychotropic malignant syndrome, a rare but dangerous outcome. These are not edge cases in a population that often presents with co-occurring conditions and layered medication histories.

The Phenomenological Diagnosis Problem

Psychiatric diagnosis operates differently from most of the rest of medicine. There are no biomarkers for depression, no blood test for generalized anxiety disorder, no imaging confirmation for ADHD. Diagnosis is built from what a patient reports, how they appear, what they do not say, and how those observations fit against clinical criteria that depend heavily on interpretive judgment. A patient who describes “low mood” might be experiencing major depressive disorder, bipolar depression, a grief reaction, a thyroid condition, a substance use disorder, or some combination of these, and the prescribing decision that follows each of those diagnoses is substantially different.

This phenomenological character of psychiatric diagnosis is not a weakness waiting to be fixed by better measurement. It is structurally embedded in what psychiatric conditions are. A chatbot conducting intake interviews can collect symptom descriptions. Whether it can recognize what a patient is not articulating, or integrate affect, context, and the silences in a clinical encounter, is a different and harder question.

Close-up of white pills scattered from an orange prescription bottle on a gray surface. Photo by Towfiqu barbhuiya on Pexels

Controlled substances add a further legal layer. Stimulants for ADHD and benzodiazepines fall under the Controlled Substances Act, which imposes stricter prescribing rules and DEA oversight. Telehealth prescribing of Schedule II substances, including Adderall, has been subject to specific federal scrutiny, particularly following pandemic-era relaxations that allowed prescribing without in-person evaluation. As telehealth pharmacies note, stimulant prescribing remains subject to stricter legal requirements than most other mental health medications, and that regulatory asymmetry has not been resolved by any current AI prescribing framework.

The Products: Who Is Doing This and How

Cerebral: The Investigation

Cerebral was not a fringe operation. It raised hundreds of millions of dollars in venture capital, operated across multiple states, and marketed itself explicitly as a technology-forward alternative to traditional telepsychiatry. The Verge investigation found that its AI-assisted workflow had been structured so that the chatbot gathered clinical information and surfaced treatment recommendations with limited meaningful review before prescriptions were issued. The investigation exposed a gap that is common to how many of these platforms operate: the public-facing description emphasizes a human clinician in the loop, while the actual workflow is designed around minimizing the clinician’s active decision-making role in order to scale efficiently.

This is not merely a marketing problem. It is a clinical design problem with legal consequences. When a licensed prescriber nominally approves decisions that are functionally generated by an AI system, the liability attaches to a human who may have had little substantive involvement, while the system that actually shaped the outcome sits in a gray zone.

It is worth noting that Cerebral had already drawn federal attention before the Verge investigation. In 2023, the company agreed to pay $11 million to settle FTC allegations that it had engaged in deceptive marketing and improperly shared patient data with third-party advertisers. The prescribing investigation added a separate clinical safety dimension to a company that regulators had already flagged for consumer protection violations. That prior history is relevant context for evaluating how the company’s public-facing descriptions of its clinical workflows should be weighted.

Legion Health: The Sanctioned Experiment

Legion Health represents a different, more transparent version of the same underlying question. The San Francisco startup received explicit approval from Utah regulators for its AI chatbot to handle prescription renewals for specific psychiatric medications in defined circumstances, with the state framing the authorization as a supervised pilot rather than a license. The distinction matters: the parameters are knowable, the accountability is nominally attached to state oversight, and the time-limited nature creates at least a theoretical mechanism for evaluation.

What the pilot actually permits is narrower than the headlines suggested. Renewal of existing prescriptions in stable patients, within defined medication categories, under specified conditions, is a different clinical task than initial diagnosis and prescribing. Whether the pilot’s design adequately captures that distinction, and how the state will assess outcomes at the one-year mark, remains to be seen.

The company’s own framing of its system, including what safety checks are built in, how escalation to a human clinician is triggered, and what constitutes a disqualifying clinical presentation for the renewal pathway, has not been independently verified. Utah’s authorization documents establish the permitted scope, but the gap between what is authorized and what actually runs in production is precisely the gap that the Cerebral investigation revealed elsewhere in this sector.

The Broader Telehealth Ecosystem

The space between traditional telepsychiatry and fully automated prescribing is populated by a range of services that sit at varying points on the automation spectrum. Services like ADHDAdvisor offer same-day or rapid-turnaround prescriptions online at $130 per month, including messaging support and prescription fulfillment. Klarity offers same-day ADHD consultations with medication prescriptions. Healthline’s review of online psychiatrist platforms notes that some telehealth networks will not prescribe controlled substances, while others operate in a space where the degree of clinician involvement is difficult to verify from the outside.

The technical architecture varies considerably. Some platforms use large language models to conduct intake interviews and flag clinical criteria. Others use structured decision trees with LLM-generated conversational interfaces layered on top. The critical variable, the degree to which a licensed human meaningfully reviews the AI’s output before a prescription is generated, differs substantially across products, and it is often not independently verifiable by patients or regulators without an investigation of the kind The Verge conducted.

Proponents of these services argue that this critique applies with equal force to high-volume human telehealth encounters, where a clinician spending eight minutes reviewing an intake form before approving a prescription is exercising judgment that is also difficult to independently verify. The comparison does not resolve the concern, but it reframes the question: the baseline against which AI prescribing is measured matters, and that baseline is not always the gold standard of unhurried, longitudinal psychiatric care.

The Regulatory Framework: A Map With Blank Spaces

Federal Ambiguity

The FDA regulates AI tools as medical devices when they are intended to diagnose, treat, or prevent disease, but the agency’s evolving position on what constitutes a breakthrough AI medical device leaves meaningful ambiguity for software-driven psychiatric prescribing tools. As STAT News reports, the breakthrough device designation comes with priority FDA review, aimed at enabling innovative devices to reach patients faster, but the question of how AI-assisted prescribing fits into existing device classifications has not been cleanly resolved. The result is that companies can in some cases deploy products under frameworks designed for human telehealth providers rather than face pre-market review designed for autonomous clinical decision-making systems.

The federal government’s movement toward a unified AI regulatory framework under the America AI Act and related executive actions has health care implications, as Baker Donelson analysis notes, but the pace of rulemaking continues to trail the pace of product deployment. Companies are operating in a space that regulators are still mapping, and the map is being drawn after the territory has already been settled.

The State-Level Patchwork

Telehealth prescribing of controlled substances has been subject to specific DEA rules, including a prolonged post-COVID debate over whether pandemic-era relaxations should be made permanent. These rules govern the prescriber’s location and licensure rather than the prescriber’s identity as human or machine, a gap that AI prescribing makes newly visible. No existing federal rule addresses whether an AI system can qualify as a “prescriber” in any legal sense.

Utah’s pilot for Legion Health operates largely outside any federal framework, testing whether a state medical board can authorize AI prescribing within its jurisdiction. The American Psychological Association reports that hundreds of state-level AI health care bills are currently moving through U.S. legislatures, covering chatbot use, insurer applications of AI, and clinical tools. But no coherent national standard governs AI-initiated prescribing, and the state-by-state approach creates the conditions for a regulatory arbitrage in which companies locate operations in permissive jurisdictions while serving patients everywhere.

The tension between state and federal authority here is not merely procedural. A patient harmed by an AI prescribing decision made under a state pilot authorization has no clear federal recourse if the harm does not rise to the level of triggering FDA enforcement. State medical boards, which authorized the pilot, are also the bodies that would investigate complaints arising from it, a structural conflict of interest that has not been addressed in the Utah framework’s public documentation.

The Evidence Base: What Research Actually Shows

What LLMs Can and Cannot Do Clinically

The capabilities of large language models in medical contexts have been extensively benchmarked, and the results show genuine but bounded competence. A study testing leading AI models on 20,000 questions derived from over 8,000 systematic reviews found that models still struggle with ambiguous evidence, and that retrieval-augmented approaches substantially improve accuracy. The implication is important: the clinical contexts where AI performs reasonably well tend to be those where evidence is strong and unambiguous, while psychiatric practice is characterized precisely by the kind of ambiguous, contextual, and heterogeneous evidence that produces the weakest AI performance.

The self-correction problem is particularly salient given that some AI prescribing platforms tout the system’s ability to flag its own errors. An exploratory study published on arXiv found that self-reflective reasoning, when applied as a purely prompt-based inference strategy, does not reliably improve medical multiple-choice question answering performance, and that its effects are highly dependent on dataset characteristics and model scale. The researchers concluded that self-reflection “should be viewed less as a general-purpose accuracy booster and more as a diagnostic tool for probing model reasoning behavior,” a finding that complicates any claim that an AI prescribing system is meaningfully checking itself.

Benchmark performance on standardized medical questions, including performance on the United States Medical Licensing Examination that AI models have now passed, is not the same as clinical performance in real-world encounters. USMLE questions are structured, bounded, and unambiguous in ways that clinical encounters are not. The translation from benchmark to bedside has been unreliable in health technology historically, and there is no strong reason to expect AI psychiatric prescribing to be an exception to that pattern.

The Bias Problem

Bias in AI clinical decision-making is not theoretical. Research examining LLMs for medical applications found that cognitive-bias priming altered clinical recommendations in 81 percent of fairness tests for certain models, a finding with specific implications for psychiatric care. Mental health patients are disproportionately represented in marginalized communities, and an AI system that responds differently to race-inflected or gender-inflected inputs, whether through training data artifacts or other mechanisms, could systematically under-treat or mis-treat patients who are already underserved by conventional care.

The AI ADHD diagnosis literature is more developed than the broader psychiatric prescribing literature. A comprehensive review published in the Egyptian Pediatric Association Gazette analyzes machine learning applications across the ADHD clinical pathway, encompassing objective diagnosis, subtype differentiation, and treatment response prediction. The authors identify significant clinical validation challenges and implementation barriers that have not been resolved, noting that the gap between benchmark performance and real-world clinical utility remains substantial. If the more-studied ADHD application still carries unresolved validation problems, the case for extending AI prescribing authority across broader psychiatric categories requires scrutiny the current evidence base cannot support.

There is no peer-reviewed clinical trial evidence specifically evaluating the safety and efficacy of AI-driven psychiatric prescribing as a standalone prescribing authority. The Utah pilot is explicitly framed as a test precisely because that evidence does not yet exist. The sequence matters: these systems are being deployed and then studied, rather than studied before deployment.

The Access Argument: Real Problem, Contested Solution

The Shortage Is Real

The mental health care shortage in the United States is not a rhetorical device. The U.S. Department of Health and Human Services has designated 3,143 Health Professional Shortage Areas for behavioral health, covering approximately 80 million people. Nearly half of Americans live in designated Mental Health Care Professional Shortage Areas, where factors including lack of funding for mental health care, low insurance reimbursement rates, and low provider retention reduce access to care. The shortage is projected to worsen, and structural factors mean that training more psychiatrists is a slow solution that will not close the gap in any near-term horizon. As AJMC analysis notes, health care systems themselves recognize they will never have sufficient mental health specialists to meet patient demand.

The access argument for AI prescribing is, in this sense, grounded in a genuine and serious problem. It deserves to be engaged rather than dismissed as marketing.

The Access Argument’s Weaknesses

The assumption that remote or automated tools automatically reach those most in need does not survive scrutiny. A Wiley study examining telemedicine adoption following the COVID-19 surge found that widespread adoption of virtual visits did not significantly improve access to care for patients in rural and underserved communities, despite dramatically increasing the volume of telehealth encounters overall. The patients who gained easiest access to telehealth were those with reliable internet, digital literacy, and the ability to pay out-of-pocket or navigate insurance, which are not the defining characteristics of the populations identified as most underserved.

The existing AI and rapid-turnaround telehealth platforms that have received the most attention, services charging $130 per month for ADHD management or offering same-day prescriptions, are optimized for convenience and speed for paying subscribers. The populations who arguably need the most help, those without insurance, without reliable internet access, without the health literacy to navigate app-based care, are not the primary users of these services. Advocates for AI prescribing who invoke the 80 million people in shortage areas need to grapple seriously with whether those populations are actually the ones the commercial products are designed to reach, and whether a model built around subscription revenue and low marginal cost per interaction is the right architecture for expanding equity.

New models do exist that try to address this gap differently. The Davis Mountain Clinic in Fort Davis, Texas, for example, operates from a shipping container and uses a hands-on telehealth hub model with in-person support staff to connect rural residents with remote professionals. A new training program for family doctors in Canada aims to let psychiatrists collaborate with family doctors who each carry a caseload of rural and Indigenous patients, a model that expands access through human collaboration rather than automation. These alternatives do not receive the same investor attention, but they are more directly targeted at the underserved populations that AI prescribing advocates invoke.

Stakeholder Perspectives

Practicing Psychiatrists and Professional Associations

Practicing psychiatrists and their professional associations have concentrated their concerns on a cluster of related issues: the opacity of AI decision-making in clinical workflows, the absence of validated safety data, and the risk that patients will receive prescriptions without the kind of longitudinal relationship that allows a clinician to detect deterioration over time. A chatbot conducting a renewal encounter does not have access to the history of how a patient’s presentation has changed over months, or the contextual knowledge of what was tried before and why it was abandoned, unless that information has been explicitly provided in a structured and accurate form.

The R Street Institute analysis of advanced practice providers in mental health care acknowledges the structural shortage while emphasizing that expanding prescribing authority to any new class of provider, human or automated, requires demonstrated competency and adequate supervision frameworks. The question of what adequate supervision looks like for an AI system that processes thousands of renewal requests simultaneously has no settled answer.

Not all clinicians hold an oppositional position. Some psychiatrists and psychiatric nurse practitioners who work in high-volume settings have argued that AI-assisted intake and triage, short of autonomous prescribing, could meaningfully reduce the administrative burden that currently limits how many patients they can see. The line between AI as a clinical support tool and AI as an autonomous prescriber is where most clinician organizations draw their objection, rather than at AI involvement in care more broadly.

Patients and Advocates

Patient advocates hold a genuinely split position. Some argue that for a person who has gone untreated for years because no psychiatrist practices within driving distance and the waiting list for the nearest one is eighteen months, any access is meaningfully better than none. The experience of finally getting a prescription that stabilizes a debilitating condition through an imperfect channel is not nothing. That perspective deserves to be taken seriously by critics who have access to conventional care.

Other advocates warn that AI prescribing could systematically under-diagnose or mismanage complex presentations, particularly in patients with co-occurring trauma and substance use disorders, which research shows are deeply interlinked and clinically challenging to disentangle. A system optimized for efficient renewal encounters may perform adequately for a stable patient on a well-tolerated SSRI but fail badly for a patient whose depressive symptoms are masking a bipolar II presentation that has not yet been diagnosed, or whose medication is interacting with substances they have not disclosed.

Patients with lived experience of psychiatric crises raise a further concern that neither the access argument nor the clinical safety argument fully captures: the absence of a human relationship in crisis moments. The literature on therapeutic alliance, which is the quality of the working relationship between a clinician and patient, consistently identifies it as one of the stronger predictors of psychiatric treatment outcomes. Whether an AI system can substitute for or adequately route around that relational dimension is a question that benchmarks and pilot metrics are not designed to answer.

Regulators and Companies

Utah’s framing of the Legion Health pilot as a carefully bounded, time-limited experiment with defined parameters is the most intellectually honest regulatory posture that has emerged in this space. But critics of pilot programs generally note that limited pilots have a persistent historical tendency to expand before the evidence base justifies it, driven by commercial and political pressures that are difficult to resist once a system is running and patients are using it.

The companies in this space consistently describe their products as expanding access and improving efficiency, often emphasizing that human clinicians remain involved in some part of the workflow. The Cerebral investigation demonstrated that such descriptions require independent verification. Taking a company’s word for how its clinical workflow functions, when the clinical workflow is what generates revenue, is precisely the kind of assumption that journalism and regulation are supposed to test.

The Liability Gap: Who Is Responsible When the Algorithm Is Wrong

The Structural Problem

Traditional medical malpractice law assigns liability to the licensed clinician who prescribed a medication. If an AI system prescribes without a human clinician approving the specific decision, the legal target shifts to the company, the software developer, or potentially both, and courts have not yet produced settled precedent for that scenario. The emerging consensus in AI malpractice discussions, as reflected in how surgeons currently think about AI-assisted procedures, is that physicians retain ultimate responsibility even when using AI tools. But Legion Health’s model removes the physician from the prescribing decision entirely for covered renewals, which may create a liability vacuum that existing tort law is not equipped to address cleanly.

Utah’s pilot authorization implicitly acknowledges this gap by operating as a state-supervised experiment, but the authorization does not itself resolve what happens to a patient harmed by an AI prescribing decision. Whether such a patient can sue Legion Health, the state, or a phantom licensed professional is genuinely unclear under existing law.

Disclosure and Transparency

Legal scholars examining AI malpractice have identified “proportionate transparency” about when AI is used in care as an ethical standard, but one that lacks regulatory teeth in most jurisdictions. Patients using AI prescribing platforms may not know, and may not have been told, how their prescription was generated. That informational asymmetry is significant: a patient who knows they are interacting with an AI system can make different choices than one who believes they are receiving conventional clinical care.

Platforms like Cerebral that allow AI to effectively drive prescribing while nominally maintaining human prescribers on staff may face the worst legal outcome: the liability of a telehealth company with human prescribers, combined with the systemic failures of an AI workflow. Plaintiffs’ attorneys have begun noticing this structure, and as adverse event data accumulates, litigation will clarify what courts are willing to attribute to companies whose AI systems issued prescriptions that harmed patients.

Malpractice insurers and professional liability coverage frameworks built for human mental health providers have not yet adapted standard policy language to cover AI prescribing entities, according to coverage frameworks currently in use. That coverage uncertainty affects whether harmed patients receive any compensation at all, since an AI prescribing company that is not covered under conventional professional liability frameworks may have no insurance pool to draw on in litigation.

What Comes Next: Open Questions and the Road Ahead

The Utah Data Problem

The Utah pilot’s one-year timeline makes it one of the most concrete near-term data points in this space. What the state does with that year matters enormously. If Utah collects and publishes rigorous outcomes data, adverse event rates, and clinical performance metrics at the end of the pilot period, it will have produced the most systematic public evidence available about AI psychiatric prescribing. If the data is thin, proprietary, or framed primarily around user satisfaction rather than clinical outcomes, the pilot will have advanced the practice without advancing the evidence, which is the pattern that has characterized too much of health tech’s expansion into clinical care.

The FDA’s evolving position on AI medical devices, including its breakthrough device designation pathway and how it classifies AI-driven prescribing tools, will shape whether companies face meaningful pre-market review. As STAT News reports, the agency’s stance is still developing, and the commercial incentives pushing for loose classification are significant. A breakthrough designation pathway that was designed for genuinely novel medical devices should not become a route for AI prescribing systems to bypass the clinical evidence standards applied to drugs and conventional devices.

The Controlled Substance Flashpoint

The convergence of AI prescribing with controlled substance regulations is a coming flashpoint that has not yet fully arrived. ADHD stimulant prescribing via telehealth has already drawn DEA scrutiny, and services like Klarity and ADHDAdvisor that advertise same-day Adderall prescriptions are operating in a space that regulators are watching closely. The question of whether an AI system can ever be authorized to prescribe Schedule II medications is likely to become an explicit regulatory and legal battle, and the outcome will determine whether the commercial ADHD telehealth market, which is large and growing, can continue to function in its current form.

The DEA’s rules governing telehealth prescribing of controlled substances currently address prescriber location and licensure. They do not address prescriber identity as human or machine, because when the rules were written, that was not a question that needed asking. Updating those rules to address AI prescribing will require either congressional action or rulemaking, and both move slowly against an industry that moves quickly.

The Equity Accounting

The access equity question will require more than pilot programs to answer. Rigorous study of who actually uses AI prescribing services, how outcomes compare across demographic groups, and whether deployment reaches shortage areas or concentrates among already-served populations should be a precondition for broader authorization rather than an afterthought evaluated years after commercial scaling. The pattern in health technology has consistently been that tools validated in relatively affluent, digitally connected populations are then deployed broadly without adequate adjustment for populations with different risk profiles, different clinical presentations, and different ability to self-advocate when something goes wrong.

That pattern is particularly dangerous in psychiatric care, where patients in crisis are least equipped to recognize and report adverse events, where stigma suppresses complaints, and where the populations in designated shortage areas often carry the highest clinical complexity and the fewest resources to navigate harms.

The Deepest Question

The most consequential long-term question may not be whether AI can prescribe safely in controlled conditions. Under sufficiently narrow definitions of the task, with well-selected patients, stable presentations, and thorough oversight, the answer may eventually be yes for specific limited functions. The harder question is whether the commercial incentives structuring these products, driven by subscription revenue, patient volume, and low marginal cost per interaction, are compatible with the kind of cautious, relationship-based clinical judgment that psychiatric prescribing demands at its best.

A business model that makes money by processing more renewals faster is not aligned with a clinical model that makes patients better by taking time, building longitudinal knowledge, and sometimes slowing down when something seems off. That misalignment is not unique to AI: it characterizes much of commoditized telehealth. But AI amplifies the misalignment by removing the human clinician who, even under commercial pressure, retains some professional and legal accountability for what they prescribe. When the algorithm is the prescriber, the friction that accountability creates disappears, and it is not yet clear what replaces it.

The Cerebral investigation and the Utah pilot are not two chapters of the same story. One is an account of what happens when the financial incentives of health technology are allowed to run without meaningful constraint. The other is an attempt, imperfect and incomplete, to build a constraint before the technology outpaces it entirely. Whether the constraint arrives in time, and whether it is built with the populations who most need protection at its center, is the question the next two years will begin to answer.

Published April 5, 2026