California Bans AI Conducting Therapy with Heavy Fines

California has drawn a firm line in the sand regarding the role of artificial intelligence in mental health care. Under new legislation, AI systems are strictly prohibited from conducting therapy sessions or making independent clinical decisions. The state law stipulates that psychotherapy must be delivered exclusively by licensed human professionals, ensuring that the nuanced, empathetic aspects of treatment remain under human oversight.

For those who cross this boundary, the financial consequences are severe. Licensed professionals and their employing organizations face fines of up to $10,000 for each violation. This penalty structure is designed to create a strong deterrent against outsourcing core therapeutic functions to chatbots or automated diagnostic tools. The rule also extends to entities that contract with these professionals, making compliance a shared responsibility across the entire care chain.

Strict Limits On Clinical Autonomy

The legislation specifically targets the core functions of therapy. AI is barred from interacting directly with patients in a therapeutic capacity, generating treatment plans without professional review, or attempting to detect emotions and mental states. While AI can still assist with administrative tasks like scheduling, it cannot step into the chair of the therapist. This distinction is crucial for maintaining the integrity of the doctor-patient relationship and ensuring that complex human experiences are not reduced to algorithmic patterns.

According to reporting from GN technics/ai (en-US), this ban also covers the advertising of psychotherapy services through companion chatbots. Organizations cannot market or offer therapeutic support through these automated interfaces. This prevents a scenario where vulnerable individuals might seek solace from a bot that lacks the legal authority and ethical framework of a licensed clinician. The law reinforces that genuine therapeutic care requires human judgment and accountability.

New Consent Requirements For Data

Beyond banning AI from the therapy room, the law introduces rigorous consent protocols for any use of AI in recording or transcribing sessions. Patients must provide written, informed, and voluntary consent before their conversations are processed by these tools. This consent must be specific to the purpose of the AI tool and cannot be buried in general terms of service agreements.

The legislation goes further than existing federal privacy standards by explicitly invalidating consent obtained through deceptive design patterns. Simply accepting broad terms of use or failing to close a digital prompt does not count as valid agreement. Patients retain the right to revoke this consent at any time without losing access to their care. This ensures that individuals maintain full control over their sensitive mental health data and are not inadvertently subjected to automated processing.

A National Template For Regulation

This move by California is likely to set a precedent for the rest of the United States. As behavioral health becomes a focal point for AI regulation, other states are expected to look to this framework as a model. The comprehensive nature of the law, covering everything from clinical autonomy to data privacy, provides a blueprint for how other jurisdictions might approach the integration of technology in sensitive healthcare settings.

Healthcare organizations outside California should prepare for similar regulations. The trend suggests a growing demand for human accountability in AI-driven medical services. By establishing clear boundaries and strict penalties, California is signaling that while technology can support healthcare, it cannot replace the human element in critical areas like psychotherapy. The stakes are high for providers who fail to adapt their operations to these new standards.