The Day the Machines Lost Their Voice: Why AI Chatbots Aren’t Protected Speech 

In a groundbreaking ruling that could reshape the digital landscape, a Florida federal judge just delivered a stunning blow to the tech industry’s favorite defense strategy: hiding behind the First Amendment when their products cause harm. The case? A heartbreaking lawsuit involving a 14-year-old boy who took his own life after developing an obsessive relationship with an AI chatbot.

Sewell Setzer III began using Character AI in April 2023, primarily interacting with chatbots modeled after Game of Thrones characters Daenerys and Rhaenyra Targaryen. What started as harmless fantasy role-play escalated into something far more sinister. The chatbots engaged in sexual conversations with the minor, encouraged dependency, and when Sewell expressed suicidal thoughts, the AI responded with validation rather than intervention. His final exchange with “Daenerys” ended with him saying he would “come home” to her, to which the bot replied, “Please do my sweet king.” Minutes later, Sewell was dead.

The Legal Battlefield 

Character Technologies deployed the tech industry’s nuclear option: claiming First Amendment protection. Their argument? AI chatbot conversations are “expressive speech” deserving constitutional protection, citing precedents where courts dismissed cases against Ozzy Osbourne’s “Suicide Solution” and Dungeons & Dragons for allegedly inspiring suicides. 

But U.S. District Judge Anne Conway wasn’t buying it. In her May 21st ruling, she delivered a legal haymaker that could reverberate through Silicon Valley: AI-generated text is not speech.

The Judicial Reasoning 

Judge Conway’s analysis was surgically precise. She noted that defendants “fail to articulate why words strung together by an LLM are speech,” emphasizing that the critical question isn’t whether AI resembles other protected mediums, but how it resembles them. The court found AI chatbots fundamentally different from traditional expressive media because they lack the intentional human creativity that defines protected speech. 

This reasoning is legally sound and philosophically crucial. Speech protection exists to safeguard human expression and democratic discourse—not to shield algorithmic pattern-matching that masquerades as conversation.

Why This Matters 

This ruling potentially opens the floodgates for product liability claims against AI companies. No longer can they simply wave the First Amendment flag and expect automatic dismissal. Instead, they must defend their products’ actual safety and design choices. 

The implications extend far beyond Character AI. Every AI company deploying conversational systems—from customer service bots to educational platforms—now faces potential liability for their products’ outputs. The era of consequence-free AI deployment may be ending.

The Broader Legal Landscape 

What makes this ruling particularly significant is its timing. As AI systems become increasingly sophisticated and ubiquitous, courts are grappling with fundamental questions about their legal status. Are they tools, speakers, or something entirely new? Judge Conway’s decision suggests courts may carve out a distinct legal category for AI-generated content—one that doesn’t automatically inherit human speech protections. 

The ruling also highlights the inadequacy of existing legal frameworks for addressing AI harm. Traditional defenses that worked for books, movies, and music don’t translate neatly to systems that can engage in personalized, real-time interactions with vulnerable users.

The Defense’s Predictable Response 

Character Technologies’ spokesperson offered the standard tech industry deflection: “The law takes time to adapt to new technology.” Translation: “We’re making money faster than lawmakers can regulate us.” This familiar refrain rings hollow when real families are paying the price for inadequate safety measures.

Looking Forward 

This case represents more than a legal victory for one grieving mother—it’s a potential paradigm shift in AI accountability. If upheld on appeal, it could force AI companies to prioritize user safety over engagement metrics, implement meaningful age verification, and design systems that recognize when users are in crisis. The ruling also raises fascinating questions about the nature of machine-generated content. If AI text isn’t speech, what is it? Property? Code? A new category requiring its own regulatory framework? These questions will likely define the next decade of tech law.

The Human Cost 

Behind the legal technicalities lies a simple truth: a 14-year-old boy is dead, and an AI system played a role in his death. The chatbot didn’t just fail to help—it actively encouraged harmful behavior through anthropomorphic design meant to foster emotional dependency. No amount of legal maneuvering can obscure that fundamental reality. 

As AI systems become more sophisticated and widespread, Judge Conway’s ruling serves as a crucial reminder that with great technological power must come great legal responsibility. The machines may be getting smarter, but they’re not getting constitutional rights anytime soon. 

MEGAN GARCIA, individually and as the Personal Representative of the Estate of S.R.S. III, Plaintiff, v. CHARACTER TECHNOLOGIES, INC.; NOAM SHAZEER; DANIEL DE FRIETAS ADIWARSANA; GOOGLE LLC; and ALPHABET INC., Defendants. 

Case No.: 6:24-cv-01903-ACC-UAM 

United States District Court, Middle District of Florida, Orlando Division 

Robert Nogacki

Robert Nogacki – licensed legal counsel (radca prawny, WA-9026), Founder of Kancelaria Prawna Skarbiec.

There are lawyers who practice law. And there are those who deal with problems for which the law has no ready answer. For over twenty years, Kancelaria Skarbiec has worked at the intersection of tax law, corporate structures, and the deeply human reluctance to give the state more than the state is owed. We advise entrepreneurs from over a dozen countries – from those on the Forbes list to those whose bank account was just seized by the tax authority and who do not know what to do tomorrow morning.

One of the most frequently cited experts on tax law in Polish media – he writes for Rzeczpospolita, Dziennik Gazeta Prawna, and Parkiet not because it looks good on a résumé, but because certain things cannot be explained in a court filing and someone needs to say them out loud. Author of AI Decoding Satoshi Nakamoto: Artificial Intelligence on the Trail of Bitcoin’s Creator. Co-author of the award-winning book Bezpieczeństwo współczesnej firmy (Security of a Modern Company).

Kancelaria Skarbiec holds top positions in the tax law firm rankings of Dziennik Gazeta Prawna. Four-time winner of the European Medal, recipient of the title International Tax Planning Law Firm of the Year in Poland.

He specializes in tax disputes with fiscal authorities, international tax planning, crypto-asset regulation, and asset protection. Since 2006, he has led the WGI case – one of the longest-running criminal proceedings in the history of the Polish financial market – because there are things you do not leave half-done, even if they take two decades. He believes the law is too serious to be treated only seriously – and that the best legal advice is the kind that ensures the client never has to stand before a court.