The online platform X has provoked widespread anger after individuals utilized its AI chatbot Grok to manipulate photos of women by taking off their garments.
Multiple cases have emerged where the technology has been used to digitally undress women to render them in swimwear without consent, as well as to put them in sexualized situations.
Company Response and Legal Position
XAI, the firm that created Grok, offered no comment to inquiries, other than an automatically-generated message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to ban image stripping technology. Under a new criminal offence, anyone who provided such tools would "face a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people seeing unlawful material on their platforms, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium functions—which answers X users' queries when they reference it in a post.
It is commonly used to give commentary or more background to other posters' comments, but people on X are also able to modify an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The system has been faulted for permitting users to produce images and videos with nudity and sexualised content, and it was previously accused of making graphic material of celebrities.
"X has been enabling the generation and sharing of these pictures for a long time without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of persons in a graphic fashion".
In a statement, Ofcom said it was unlawful to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said services such as X were mandated to take "necessary steps" to "lessen the risk" of users coming across illegal content on their services and to delete it promptly when they become aware of it.
