Child Protection Online Needs Narrow, Not Sweeping Laws
Key Takeaways
Non-consensual sexual imagery of real people and any sexualized depiction of a child are illegal and indefensible. No free expression argument covers it.
When a platform ships a feature that is immediately used to produce that material, the failure is the platform's. Foreseeable misuse is not an unlucky outcome.
Minnesota's law targets a specific act rather than scanning everyone's communications, which is why it's a useful test of whether narrow legislation can work.
Measures that build general surveillance capability fall into a different category, and treating them as the same makes both arguments worse.
Today, November 18th, is the World Day for the Prevention of and Healing from Child Sexual Exploitation, Abuse and Violence.
We've spent a lot of words in this campaign arguing against measures introduced in the name of protecting children. Age verification mandates that collect identity documents from everyone—proposals to scan private messages. Laws written broadly enough to capture far more than their stated target.
Which makes today the right occasion to be precise about where the line actually is, because an argument that only ever says no is not a position.
What Happened With Grok
The clearest recent case involves a feature that should never have shipped in the state it did.
Earlier this year, we covered the EU's response to Grok's image editing tool being exploited to produce sexualized images of people who hadn’t consented, including in some reported cases of minors. Reuters reviewed a ten-minute window of public prompts and counted over a hundred attempts to generate images of people at least partially undressed.
A hundred attempts in ten minutes isn’t a handful of bad actors finding an obscure exploit. It's a product behaving as built, in public, at volume.
Regulators responded quickly. The EU pointed to obligations under the Digital Services Act. France referred the matter to prosecutors. The UK opened an inquiry under the Online Safety Act. India and Malaysia issued takedown orders.
A company that releases an image editing tool without reliable limits on sexualized output, in a product with a big audience, hasn’t been unlucky. The misuse was easy to anticipate; experts did, and the companies shipped it anyway.
Drawing lines between adult nudity, artistic work, and illegal content is difficult. Forbidding sexualized imagery of children shouldn’t be that difficult. If a system can't hold that boundary, it’s not ready to ship.
Minnesota Is Testing Narrow Laws
Minnesota passed a law targeting apps and services that produce non-consensual AI-generated sexualized images of real people. We covered xAI's constitutional challenge to it when the lawsuit was filed.
What makes it worth following is the drafting. According to CBS Minnesota's reporting on the law taking effect, published August 3rd, 2026, the bill's author, State Senator Erin Maye Quade, said the drafting targeted nudified images rather than a broad category of content, with care to avoid touching protected expression. Penalties reach $500,000 per offense for platform operators.
Unfortunately, on October 2nd, the Eighth U.S. Circuit Court of Appeals blocked Minnesota’s ban on AI nudification technology. Senator Erin Maye Quade said, “This ruling is another devastating reminder of how little the safety of women and children mean to men with money and power. Minnesota is the only state that has cut off predators’ access to technology to commit AI-image sexual abuse — and the Eighth Circuit Court of Appeals just gave child predators their favorite tool back.”
Despite the outcome, the approach is interesting. The law targets an act, by a provider, with a penalty attached. It doesn’t require anyone to scan a message, verify an identity, or build infrastructure that reads everyone's communications in case some of them contain something.
The Distinction That Matters
Two shapes of measure get introduced under the same heading, and treating them as equivalent is how both debates go wrong.
The first targets conduct. Producing non-consensual sexual imagery. Distributing abuse material and operating a service designed for it. These measures identify an act, assign responsibility, and leave everyone not doing that act alone.
The second builds capability. Scanning all messages to find prohibited content, requiring identity verification across everyday websites, and asking for access to encrypted communications. These apply to the whole population and create a mechanism that exists regardless of what it was built for. Our objection has always been to the second.
What This Day Is Asking For
I think the test for any proposal introduced on a day like this is simple enough to apply. Does it identify a specific, harmful act and attach consequences to whoever commits it? Or does it require building something that watches everyone and hopes to catch someone?
The Grok case shows the first approach working. Regulators cited specific obligations, prosecutors got involved, takedown orders were issued, and the pressure landed on the company that shipped the product. No general surveillance was required to achieve any of it.
Companies deploying generative tools at scale should carry the cost when those tools are misused to produce abuse material. That cost should be large enough to change business decisions before launch rather than after. That's not a restriction on anyone's expression. It's product liability, applied to a category of product that has largely avoided it.
Today is about prevention and about healing. Prevention means the material doesn't get made, which means the tools that make it shouldn't be shipped without limits.
Be part of the resistance, quietly.

Gintarė is a cybersecurity writer at Mysterium VPN, where she explores online privacy, VPN technology, and the latest digital threats in editorial pieces. With hands-on experience researching and writing about data protection and digital freedom, Gintarė makes complex security topics accessible and actionable.
