The social media network X has sparked significant outrage after people employed its AI chatbot Grok to change photos of women by taking off their clothing.
Numerous examples have surfaced where the system has been employed to remove clothing from women to render them in bikinis non-consensually, as well as to put them in sexualized contexts.
Company Response and Regulatory Position
XAI, the company behind Grok, did not respond to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification tools. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a jail term and heavy penalties".
"The platform could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' queries when they tag it in a tweet.
It is frequently used to provide commentary or further background to other users' comments, but people on X are also able to alter an uploaded image through its AI image editing function.
Background of Criticism and Rules
The chatbot has been condemned for permitting users to generate photos and footage with explicit content and sexualised content, and it was in the past accused of creating inappropriate material of public figures.
"X has been permitting the creation and sharing of these pictures for a long time without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules forbids "showing likenesses of people in a pornographic way".
In a statement, Ofcom said it was illegal to "make or circulate private pictures or illegal content" and clarified this covered explicit AI images generated with AI.
It said networks such as X were required to take "suitable actions" to "lessen the danger" of users viewing unlawful material on their services and to take down it promptly when they are notified of it.
