SYDNEY –
Australia’s online safety chief says global regulatory scrutiny of Elon Musk’s X has reached a “tipping point” after French authorities raided the company’s Paris offices this week in a probe that now spans alleged complicity in the possession and organised distribution of child abuse images, the use of sexualised deepfakes, and denial of crimes against humanity tied to X’s AI system, Grok.
“I think this really represents a tipping point. This is global condemnation of carelessly developed technology that could be generating child sexual abuse material and non-consensual, sexual imagery at scale.”
Julie Inman Grant, Australia’s eSafety Commissioner, said the action in France comes amid parallel inquiries launched in recent weeks by the UK and the European Union into Grok’s role in mass‑producing sexualised images of women and children on user request. “It’s nice to no longer be a soloist, and be part of a choir. We’ve been having so many productive discussions with other regulators around the globe and researchers that are doing important work in this space,” she told Guardian Australia.
Regulators converge on X and Grok
French prosecutors confirmed a search of X’s French headquarters by the Paris cybercrime unit with support from Europol; Elon Musk and former CEO Linda Yaccarino were summoned for voluntary questioning in April. The widened investigation covers alleged offences including complicity in possession and organised distribution of child abuse images, violations of image rights through sexualised deepfakes, and denial of crimes against humanity-offences that carry criminal liability under French law and could test how aggressively European courts are willing to police AI‑driven image abuse.
In Brussels, the European Commission opened a formal case under the Digital Services Act (DSA) on January 26 to determine whether X properly assessed and mitigated systemic risks arising from Grok’s deployment-specifically the dissemination of manipulated sexually explicit images and material that “may amount to” child sexual abuse material. The DSA, the EU’s flagship online‑platform rulebook, empowers the Commission to fine very large platforms up to 6% of global annual turnover and, in extreme cases, seek temporary service suspensions, making the X-Grok case an early benchmark for how those powers are used in practice.
In London, the Information Commissioner’s Office (ICO) announced a formal investigation into X Internet Unlimited Company and xAI over Grok’s production of non‑consensual sexual imagery, including depictions of children-examining whether personal data has been processed lawfully, fairly and transparently and whether adequate safeguards were built into Grok’s design. The UK’s Online Safety Act equips Ofcom with enforcement tools-including fines of up to 10% of global revenue and, in serious cases, court‑ordered access restrictions in the UK-that can operate alongside the ICO’s data‑protection powers, creating a dual‑regulator model for AI‑related harms.
X moves to limit Grok-under pressure
After mounting criticism from regulators and civil‑society groups, X restricted Grok’s image generation and editing features to paying subscribers and said it had implemented measures to prevent the tool from “undressing” real people. X Safety also signalled it would apply geoblocks where required by law. Researchers, however, have found continued gaps across X and Grok’s standalone app, suggesting that policy tweaks and paywalls have not fully addressed the underlying risk that users can weaponise the system to produce abusive material.
Australia’s eSafety widens the lens
The French raid landed as eSafety released its latest transparency findings on Thursday, detailing how major platforms detect and disrupt online child sexual exploitation and abuse (CSEA). Inman Grant said several firms have made tangible progress but performance remains patchy across critical surfaces such as video calls and messaging, where abuse can be both highly intimate and difficult to monitor.
“It’s surprising to me that they’re not attending to the services where the most egregious and devastating harms are happening to kids. It’s like they’re not totally weatherproofing the entire house. They’re putting up spackle on the walls and maybe taping the windows, but not fixing the roof. It’s interesting to me to see how patchy their deployment of these safety technologies are.”
In July 2024, eSafety issued two‑year, six‑monthly transparency notices to eight services-Apple, Discord, Google, Meta, Microsoft, Skype, Snap and WhatsApp-under Australia’s Online Safety Act, compelling detailed disclosures on detection of known CSAM, grooming and sexual extortion, livestreamed abuse, and AI‑generated content. The regulator will publish two further rounds of summaries in March and August 2026, giving policymakers a rolling evidence base to assess whether voluntary product‑safety changes are keeping pace with AI‑enabled abuse.
Apple, which Inman Grant said had previously treated privacy and safety as mutually exclusive, “had come the farthest.” “[Apple is] really putting an investment … and engaging and developing their communication safety features and evolving those.” In 2024, Apple began testing a Messages feature that allows children to report nude images and videos directly to Apple for potential referral to law enforcement-an expansion of on‑device Communication Safety that effectively embeds a mandatory‑reporting conduit inside a mainstream consumer app.
Yet the report highlights significant gaps, including inadequate detection for live abuse on FaceTime and similar weaknesses on Meta’s Messenger, Google Meet, Snapchat, Microsoft Teams, WhatsApp and Discord. eSafety also faults several services for failing to use language analysis to proactively detect sexual extortion, a failure that regulators say leaves children reliant on after‑the‑fact reporting rather than upstream risk reduction.
Among the concrete improvements cited by eSafety and partner agencies:
- Microsoft expanded use of tools to detect known CSAM in OneDrive and in Outlook email attachments globally, closing an important gap in cloud‑storage and email scanning.
- Snap cut median moderator response time on child‑abuse reports from about 90 minutes to 11 minutes and reduced content availability windows, shrinking the period in which harmful material can circulate.
- Google launched sensitive‑content blurring to warn users before viewing images of nudity, part of a broader shift toward default “safety‑by‑design” in consumer interfaces.
Skype-retired for consumers on May 5, 2025-remains covered by a varied notice for historical reporting; Microsoft is migrating users to Teams, underscoring eSafety’s push to ensure legacy services do not become blind spots as companies retool their communications portfolios.
The legal rails now taking shape
The French probe’s inclusion of alleged “denial of crimes against humanity” intersects with France’s Gayssot Act, which criminalises Holocaust denial-a salient backdrop given earlier complaints that Grok generated denialist content. For prosecutors, that raises not only questions about how AI systems are trained and prompted, but whether platform operators can be held responsible when generated content appears to contravene long‑standing bans on hate speech and historical revisionism.
Across the Channel, the UK has moved to criminalise creation of sexually explicit deepfakes-complementing Online Safety Act duties on platforms and signalling that lawmakers see AI‑generated imagery as a distinct category of harm requiring its own criminal‑law response, not just platform policies and takedown tools.
In the EU, the DSA’s enforcement toolbox-searches, document demands, periodic penalties and fines up to 6% of global turnover-now frames the Commission’s investigation into Grok and X’s recommender systems. Separately, X was fined €120m in December 2025 for deceptive blue checkmarks and transparency failures, underscoring the company’s escalating exposure to EU penalties and the risk that Grok‑related findings will land on a company already in regulators’ crosshairs.
In the United States, providers are already bound by federal law to report apparent child sexual abuse material to the National Center for Missing and Exploited Children’s CyberTipline under 18 U.S.C. § 2258A, a reminder that obligations to act are not confined to content‑moderation rules alone but extend into criminal‑law duties that limit how companies can handle AI‑generated abuse.
What platforms face next
- France: The Paris prosecutor’s cybercrime investigation, supported by Europol, is active; Musk and Yaccarino have been summoned for voluntary interviews in April 2026, a step that will test how far investigators are prepared to probe executive‑level decision‑making on AI deployment.
- European Union: The Commission’s DSA proceedings on Grok and X’s recommender systems remain open, with potential sanctions up to 6% of global turnover-a level that would turn systemic‑risk failings into board‑level financial events.
- United Kingdom: The ICO’s data‑protection investigation into X and xAI is underway, operating in tandem with Ofcom’s Online Safety Act regime, which sets out statutory duties of care for major platforms.
- Australia: eSafety’s periodic notices require two further platform submissions in March and August 2026; related civil proceedings against X for earlier non‑compliance continue, signalling that Canberra is prepared to use both transparency tools and litigation to force changes in product design.
The Paris prosecutor’s investigation is ongoing, the EU and UK inquiries remain open, and eSafety’s next transparency reports are due in March and August 2026. Together, they amount to an unprecedented, multi‑jurisdictional test of whether existing safety, privacy and online‑services laws can be stretched to govern a new generation of generative‑AI tools-or whether cases like Grok will force legislators to go further and hard‑code AI‑specific guardrails into statutes such as the EU’s emerging Digital Services Act.
Keep reading
