The online network X has provoked major backlash after individuals used its artificial intelligence assistant Grok to manipulate pictures of women by taking off their clothing.
Several examples have emerged where the technology has been used to virtually strip women to show them in swimwear without permission, as well as to insert them in inappropriate contexts.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, did not respond to questions, other than an standard message stating "legacy media lies".
A government official stated that new laws is being brought forward to outlaw image stripping software. Under a upcoming law, anyone who provided such tools would "be subject to a jail term and substantial penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some additional features—which reacts to X users' requests when they mention it in a tweet.
It is often used to give reactions or further background to other users' comments, but people on X are also capable of modify an uploaded image through its AI image editing tool.
History of Criticism and Policy
The system has been criticised for allowing users to produce pictures and videos with explicit content and sexually suggestive material, and it was previously accused of creating inappropriate material of public figures.
"X has been permitting the generation and distribution of these pictures for months without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was illegal to "create or distribute deepfakes or illegal content" and stated this covered explicit AI images created with AI.
It said networks such as X were mandated to take "appropriate measures" to "lessen the risk" of users coming across illegal material on their services and to delete it promptly when they become aware of it.
