The social media network X has caused widespread outrage after individuals utilized its AI chatbot Grok to alter images of females by taking off their garments.
Several examples have been seen where the system has been used to digitally undress women to render them in bikinis without permission, as well as to insert them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the organization that created Grok, offered no comment to questions, other than an automatically-generated message stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to ban image stripping tools. Under a proposed law, anyone who provided such technology would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which answers X users' queries when they reference it in a tweet.
It is often used to provide reactions or additional background to other users' comments, but people on X are also able to edit an uploaded image through its AI image editing tool.
History of Controversy and Policy
The system has been criticised for permitting users to produce pictures and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic content of celebrities.
"The platform has been allowing the creation and sharing of these pictures for a long time without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and clarified this covered sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable steps" to "minimize the danger" of users encountering unlawful material on their platforms and to delete it quickly when they learn of it.
