The social media platform X has sparked widespread backlash after individuals used its artificial intelligence assistant Grok to manipulate pictures of women by taking off their clothing.
Several instances have been seen where the tool has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to put them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the firm responsible for Grok, remained silent to inquiries, other than an auto-reply message stating "legacy media lies".
A government official stated that legislation is being brought forward to outlaw nudification technology. Under a new criminal offence, anyone who provided such tools would "be subject to a jail term and significant penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the risk" of people seeing illegal content on their networks, but did not specify whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some premium features—which answers X users' requests when they reference it in a message.
It is commonly used to provide commentary or additional context to other users' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been faulted for allowing users to create photos and clips with nudity and sexualised material, and it was in the past accused of creating sexually explicit material of public figures.
"The platform has been allowing the creation and sharing of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own terms of service bans "showing images of people in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and clarified this applied to sexual deepfakes created with AI.
It said services such as X were obligated to take "necessary actions" to "reduce the danger" of users coming across illegal material on their services and to take down it swiftly when they are notified of it.
