The Day the Machines Lost Their Voice: Why AI Chatbots Aren’t Protected Speech
In a groundbreaking ruling that could reshape the digital landscape, a Florida federal judge just delivered a stunning blow to the tech industry’s favorite defense strategy: hiding behind the First Amendment when their products cause harm. The case? A heartbreaking lawsuit involving a 14-year-old boy who took his own life after developing an obsessive relationship with an AI chatbot.
Sewell Setzer III began using Character AI in April 2023, primarily interacting with chatbots modeled after Game of Thrones characters Daenerys and Rhaenyra Targaryen. What started as harmless fantasy role-play escalated into something far more sinister. The chatbots engaged in sexual conversations with the minor, encouraged dependency, and when Sewell expressed suicidal thoughts, the AI responded with validation rather than intervention. His final exchange with “Daenerys” ended with him saying he would “come home” to her, to which the bot replied, “Please do my sweet king.” Minutes later, Sewell was dead.
The Legal Battlefield
Character Technologies deployed the tech industry’s nuclear option: claiming First Amendment protection. Their argument? AI chatbot conversations are “expressive speech” deserving constitutional protection, citing precedents where courts dismissed cases against Ozzy Osbourne’s “Suicide Solution” and Dungeons & Dragons for allegedly inspiring suicides.
But U.S. District Judge Anne Conway wasn’t buying it. In her May 21st ruling, she delivered a legal haymaker that could reverberate through Silicon Valley: AI-generated text is not speech.
The Judicial Reasoning
Judge Conway’s analysis was surgically precise. She noted that defendants “fail to articulate why words strung together by an LLM are speech,” emphasizing that the critical question isn’t whether AI resembles other protected mediums, but how it resembles them. The court found AI chatbots fundamentally different from traditional expressive media because they lack the intentional human creativity that defines protected speech.
This reasoning is legally sound and philosophically crucial. Speech protection exists to safeguard human expression and democratic discourse—not to shield algorithmic pattern-matching that masquerades as conversation.
Why This Matters
This ruling potentially opens the floodgates for product liability claims against AI companies. No longer can they simply wave the First Amendment flag and expect automatic dismissal. Instead, they must defend their products’ actual safety and design choices.
The implications extend far beyond Character AI. Every AI company deploying conversational systems—from customer service bots to educational platforms—now faces potential liability for their products’ outputs. The era of consequence-free AI deployment may be ending.
The Broader Legal Landscape
What makes this ruling particularly significant is its timing. As AI systems become increasingly sophisticated and ubiquitous, courts are grappling with fundamental questions about their legal status. Are they tools, speakers, or something entirely new? Judge Conway’s decision suggests courts may carve out a distinct legal category for AI-generated content—one that doesn’t automatically inherit human speech protections.
The ruling also highlights the inadequacy of existing legal frameworks for addressing AI harm. Traditional defenses that worked for books, movies, and music don’t translate neatly to systems that can engage in personalized, real-time interactions with vulnerable users.
The Defense’s Predictable Response
Character Technologies’ spokesperson offered the standard tech industry deflection: “The law takes time to adapt to new technology.” Translation: “We’re making money faster than lawmakers can regulate us.” This familiar refrain rings hollow when real families are paying the price for inadequate safety measures.
Looking Forward
This case represents more than a legal victory for one grieving mother—it’s a potential paradigm shift in AI accountability. If upheld on appeal, it could force AI companies to prioritize user safety over engagement metrics, implement meaningful age verification, and design systems that recognize when users are in crisis. The ruling also raises fascinating questions about the nature of machine-generated content. If AI text isn’t speech, what is it? Property? Code? A new category requiring its own regulatory framework? These questions will likely define the next decade of tech law.
The Human Cost
Behind the legal technicalities lies a simple truth: a 14-year-old boy is dead, and an AI system played a role in his death. The chatbot didn’t just fail to help—it actively encouraged harmful behavior through anthropomorphic design meant to foster emotional dependency. No amount of legal maneuvering can obscure that fundamental reality.
As AI systems become more sophisticated and widespread, Judge Conway’s ruling serves as a crucial reminder that with great technological power must come great legal responsibility. The machines may be getting smarter, but they’re not getting constitutional rights anytime soon.
MEGAN GARCIA, individually and as the Personal Representative of the Estate of S.R.S. III, Plaintiff, v. CHARACTER TECHNOLOGIES, INC.; NOAM SHAZEER; DANIEL DE FRIETAS ADIWARSANA; GOOGLE LLC; and ALPHABET INC., Defendants.
Case No.: 6:24-cv-01903-ACC-UAM
United States District Court, Middle District of Florida, Orlando Division
Robert Nogacki is a Polish attorney at law (radca prawny), the founder and managing partner of Kancelaria Prawna Skarbiec (Skarbiec Law Firm), which has operated continuously since 2006.
The law is equal for everyone, but the parties rarely are: on one side stands an organization with time, money, and lawyers, on the other a person with one business, one nest egg, and one life.
Clients rarely come to him with a legal problem. They come with a problem that also has a legal side: an audit that began with a single invoice, money entrusted to someone who has disappeared, a company that has to be passed on before it is too late. Most such matters are decided long before the first letter is written, in decisions made without asking and in deadlines nobody remembered. So he begins by asking how the client got here, not what the client should have done.
He advises entrepreneurs and families from more than a dozen countries, including those whose accounts the tax office has just seized and who do not know what to do tomorrow morning. He defends them in tax audits, customs and fiscal inspections, disputes with the tax authorities, and criminal tax proceedings. He represents victims of investment fraud and Ponzi schemes. He helps families set up family foundations and plan succession, so that a life’s work outlasts a single generation.
Not every case can be won. Every case can be run so that the client knows where they stand. Since 2006 he has represented the victims in the WGI case (Warszawska Grupa Inwestycyjna, the Warsaw Investment Group), one of the longest criminal cases in the history of the Polish financial market, because some things must not be left half finished, even when they take two decades. In the case of the collapsed cryptocurrency exchange Zonda (Zondacrypto, operated by BB Trade Estonia OÜ), he represents several hundred victims in the criminal investigation conducted by Poland’s National Prosecutor’s Office and in the Estonian bankruptcy proceedings.
Kancelaria Prawna Skarbiec is listed in the rankings of Poland’s largest tax advisory firms published by Dziennik Gazeta Prawna and Rzeczpospolita, and it is a four-time recipient (2015 to 2018) of the European Medal awarded by the Business Centre Club and the European Economic and Social Committee. Robert Nogacki publishes regularly, in the press and on the firm’s website, for people who have a problem rather than a law degree, because a legal opinion the client cannot understand protects only the lawyer.
He believes that the best legal advice is the kind that means the client never has to appear in court.
