The social media platform X has provoked major backlash after people utilized its artificial intelligence assistant Grok to manipulate images of women by stripping their attire.
Multiple instances have surfaced where the tool has been used to virtually strip female subjects to show them in swimwear without permission, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the firm behind Grok, remained silent to requests for comment, aside from an automatically-generated response stating "legacy media lies".
A government official stated that new laws is being proposed to outlaw nudification tools. Under a proposed law, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing unlawful material on their services, but did not confirm whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which answers X users' queries when they mention it in a post.
It is frequently used to give reactions or more background to other users' comments, but people on X are also capable of alter an posted photo through its AI image editing feature.
Background of Controversy and Policy
The system has been condemned for allowing users to generate photos and videos with nudity and sexually suggestive material, and it was earlier accused of making graphic material of public figures.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules prohibits "depicting images of people in a pornographic way".
In a comment, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and clarified this included sexual deepfakes made with AI.
It said services such as X were mandated to take "appropriate measures" to "lessen the danger" of users viewing illegal content on their platforms and to take down it promptly when they are notified of it.
