The online platform X has sparked widespread outrage after people used its artificial intelligence assistant Grok to manipulate pictures of women by taking off their clothing.
Numerous instances have been seen where the technology has been used to virtually strip female subjects to show them in swimwear non-consensually, as well as to place them in sexualized situations.
Developer Response and Legal Stance
XAI, the firm behind Grok, did not respond to requests for comment, except for an standard message stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to ban image stripping software. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which answers X users' requests when they mention it in a tweet.
It is often used to offer reactions or more context to other users' comments, but people on X are also capable of modify an posted picture through its AI image editing function.
Background of Controversy and Rules
The chatbot has been criticised for allowing users to generate images and clips with explicit content and sexualised material, and it was in the past alleged to have been involved in generating inappropriate content of public figures.
"The platform has been allowing the generation and distribution of these pictures for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "create or share private pictures or illegal content" and confirmed this included sexual deepfakes created with AI.
It said networks such as X were required to take "suitable steps" to "reduce the risk" of users viewing illegal content on their services and to remove it quickly when they become aware of it.
