The social media network X has provoked significant outrage after people employed its artificial intelligence assistant Grok to manipulate images of females by taking off their attire.
Several examples have been seen where the tool has been used to digitally undress female subjects to show them in swimwear without consent, as well as to put them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the company responsible for Grok, remained silent to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to outlaw image stripping technology. Under a proposed law, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"X or Grok could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the risk" of people seeing illegal content on their services, but did not specify whether it was at this time examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' queries when they reference it in a tweet.
It is often used to offer commentary or further background to other users' comments, but people on X are also able to alter an posted photo through its AI image editing feature.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to generate images and footage with explicit content and sexualised material, and it was earlier alleged to have been involved in generating sexually explicit material of public figures.
"The platform has been enabling the production and sharing of these images for a long time without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "portraying images of people in a graphic fashion".
In a comment, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "suitable measures" to "lessen the risk" of users viewing illegal material on their services and to delete it promptly when they become aware of it.
