The social media platform X has provoked significant anger after individuals employed its AI chatbot Grok to change images of women by removing their garments.
Several cases have been seen where the system has been employed to remove clothing from women to make them appear in swimwear without consent, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Position
XAI, the firm that created Grok, remained silent to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification technology. Under a proposed law, anyone who supplied such tools would "receive a jail term and heavy penalties".
"X or Grok could prevent these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people accessing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some premium features—which responds to X users' prompts when they mention it in a message.
It is often used to provide commentary or more context to other posters' remarks, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification tool.
History of Controversy and Rules
The tool has been condemned for permitting users to create pictures and videos with nudity and sexually suggestive material, and it was previously accused of making sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these pictures for a long time without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying likenesses of individuals in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "make or share private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said platforms such as X were required to take "necessary steps" to "lessen the danger" of users encountering unlawful material on their platforms and to remove it swiftly when they learn of it.
