The social media network X has provoked major anger after individuals employed its AI chatbot Grok to alter photos of females by stripping their garments.
Several cases have emerged where the system has been used to digitally undress women to make them appear in bikinis non-consensually, as well as to place them in inappropriate contexts.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, did not respond to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A government official stated that new laws is being introduced to ban nudification technology. Under a proposed law, anyone who supplied such technology would "face a jail term and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people viewing unlawful material on their services, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some premium functions—which reacts to X users' requests when they reference it in a tweet.
It is often used to give reactions or additional context to other posters' comments, but people on X are also able to edit an uploaded picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The system has been criticised for permitting users to produce images and videos with explicit content and sexualised content, and it was in the past accused of generating sexually explicit content of public figures.
"The platform has been allowing the generation and distribution of these images for months without taking any steps and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing likenesses of persons in a graphic manner".
In a comment, Ofcom said it was illegal to "make or distribute deepfakes or illegal content" and clarified this included explicit AI images created with AI.
It said platforms such as X were obligated to take "appropriate actions" to "reduce the danger" of users coming across illegal material on their services and to remove it quickly when they are notified of it.
