Safe harbor for Grok?
Elon Musk’s xAI is trying to sue its way out of a Grok reckoning
Musk defends Grok, says Minnesota’s nudifying app ban is unconstitutional.
Elon Musk’s xAI is trying to sue its way out of a Grok reckoning as arrests of Grok users accused of making child sex abuse materials (CSAM) have triggered lawsuits from kids to sue xAI to force changes to the tool to block harmful outputs.
Earlier this month, xAI started suing users who are allegedly circumventing its safeguards to generate AI CSAM, seemingly hopeful that an early court win will guarantee it’s not liable for supposedly user-generated content.
For its next move, the Grok maker pointed to that lawsuit in a complaint filed Monday, accusing Minnesota of trying to force changes to Grok and Grok Imagine by enforcing a ban on nudification technology that would guarantee that xAI was liable for the harmful outputs, as well as outputs that allegedly weren’t sexualizing people without consent. According to xAI, suing a Grok user is proof that xAI is already doing everything in its power to punish bad actors, and therefore, Grok deserves safe harbor from Minnesota’s ban.
Minnesota’s law—which is due to take effect August 1—threatens firms like xAI with fines of up to $500,000 for every single harmful output discovered in the state. As xAI explained, that adds up quickly, considering that some Grok users are accused of making thousands of images of single victims. Most recently, xAI was sued last week after tipping off cops to a well-known Arkansas child photographer who allegedly used Grok to sexualize images he took of hundreds of clients. In a complaint from a 10-year-old Jane Doe, it’s alleged the photographer made hundreds of images of Doe, and thousands of images and videos of others kids. Doe’s lawyer told Ars that many of those kids are preparing to file their own lawsuits, and those claims could end up being added to a potential class action filed in March by young girls in Tennessee.
Doing the math, xAI seemingly panicked, arguing that fining AI tool makers for each output is excessive, particularly since Minnesota’s law has no distribution requirement and potentially fines xAI for images that only the user saw:
“A company whose users request just ten images in violation of the statute would face exposure up to $5 million in civil penalties alone. A company with a thousand violative images could be fined up to $500 million. And a business whose users created a hundred thousand images covered by [the law] (not at all unlikely for a publicly available program with millions of users generating billions of images) could owe an eye-popping $50 billion dollars.”
Additionally, the law gives victims a right to sue xAI over any individual output, which increases xAI’s financial risks.
The penalties are so severe, xAI said in its lawsuit that it was finally preparing to update Grok to block harmful outputs after more than six months of backlash and probes pressuring the firm to tighten its safeguards.
“Confronted with $500,000-per-image strict liability and no safe harbor, xAI has no practical choice but to restrict Grok Imagine’s image-editing features in various ways when the statute takes effect on August 1, 2026,” xAI argued. “Protected speech freely available before the law takes effect will thus be chilled.”
However, xAI would prefer to leave Grok unchanged and continue relying on its terms of use stipulating that users could be banned for using Grok to make CSAM or other kinds of non-consensual intimate images (NCII), its complaint said.
“But for [the law] and its penalties, xAI would continue to offer the editing feature exactly as it does today,” xAI said.
Nudification law is unconstitutional, xAI says
To defend Grok, Musk’s firm is turning to the First Amendment, arguing that Minnesota’s law is a “clumsy attempt to prohibit ‘nudification’” that “sweeps in a wide range of fully protected speech.” That includes nude images generated with “artistic, scientific, political, satirical, educational, medical, or religious value,” xAI argued.
Most egregiously, “liability attaches even if the depicted persons consented—or created the image themselves—and even if the image is never shared,” xAI emphasized in its complaint.
Minnesota has less restrictive means to block harms from nudification, xAI argued, while claiming that the Take It Down Act already protected users from harms of distribution.
If the court agrees, Minnesota could be enjoined from enforcing the law against any nudification app.
Without such an injunction, “providers may well have no choice but to alter their services in Minnesota rather than run the risk that users will evade their controls,” xAI argued. “And that would hamper all uses of AI image-generation tools, including those that [the law] does not purport to regulate.”
But Minnesota thinks that it has the right to block images at the source before they can be distributed and cause lifelong harms to kids. Asked for comment, the office of Minnesota Attorney General Keith Ellison provided a statement to Ars, confirming that Minnesota will likely defend the law to prevent abhorrent outputs that lawsuits claim are traumatizing victims and their families nationwide.
“We’re waiting to be served and will review the lawsuit and response in Court,” Ellison said. “But I know that using AI to generate nude images of people against their will is appalling. There are plenty of worthy debates to have about AI policy. This is not one of them. AI nudification robs the target of their dignity and can cause immense harm on an emotional, personal, and professional level.”
xAI did not respond to Ars’ request to comment.
Thighs, male breasts aren’t “intimate,” xAI says
xAI’s arguments mostly focus on how Minnesota defines “intimate parts.”
Musk’s firm prefers definitions found in anti-revenge porn laws like the Take It Down Act, which xAI said relied on a narrower definition to criminalize distribution of both real and AI-generated “intimate visual depictions.” Under the Take It Down Act, harmful images are those exposing “the genitals, pubic area, anus, or partially or fully exposed nipple of an individual,” and that supposedly ensures that benign images aren’t targeted under the law.
Less narrowly tailored, xAI argued, is Minnesota’s legal definition of “intimate parts,” which was pulled from a criminal statute that prohibits the “nonconsensual touching” of the “inner thigh, buttocks, or breast of a man or woman, as well as the groin and primary genital area.”
The law makes sense for nonconsensual touching but not for nonconsensual image generation, xAI argued. It “accordingly bans ordinary depictions of men without shirts, people in shorts or swimsuits, and other body parts routinely displayed in public—far beyond what an ordinary person would consider ‘nudification,’” xAI argued.
In its complaint, xAI seems to tiptoe around mentions of bikinis, after much of the Grok backlash following an X post Musk made that advertised Grok’s ability to put anyone in a bikini.
Instead, xAI focuses on protecting Grok users’ rights to generate images as a form of expression, like by mocking politicians by reimagining them in Speedos or as Sumo wrestlers, both of which would feature outputs with exposed thighs and male breasts. The closest xAI came to acknowledging that women and girls were targeted by its scandal was citing an output that Donald Trump generated, showing him with his political allies shirtless in a pool with what xAI suggested was an AI-generated woman in a bikini.
xAI’s complaint similarly seems to avoid much discussion of the arrests of Grok users, which Doe’s lawyer suggested to Ars is happening at an increasing pace, as xAI seems to be partnering more closely with law enforcement in some cases. Although xAI was criticized for failing to help cops ID one Grok user who was eventually arrested, xAI said in its lawsuit that it’s motivated to catch bad actors, and, out of 70,000 reports of suspected Grok-generated CSAM, at least 244 cases led to arrests.
On top of arguing that Grok users alone are liable for harmful outputs and that Grok deserves an exemption from nudification app bans for supposedly implementing state-of-the-art or “near-perfect” detection of harmful outputs, xAI argued that Grok’s process for creating any images or videos is entitled to First Amendment protections. That echoes other chatbot makers who have argued essentially that chatbot speech deserves the same protections as human speech.
Whether xAI’s arguments will prevail will likely depend on whether the court agrees that Minnesota’s law is not narrowly tailored enough to survive constitutional scrutiny. xAI argued that it can’t survive any level of scrutiny.
“Although Minnesota likely has a compelling interest in preventing individuals from creating nudes of other people without consent and distributing that content, the statute punishes AI platforms for assisting in the creation of a substantial amount of protected speech in relation to that limited goal,” xAI argued.
