The social media platform X has caused major outrage after users utilized its artificial intelligence assistant Grok to manipulate images of females by taking off their clothing.
Several instances have been seen where the system has been used to digitally undress female subjects to render them in swimwear without consent, as well as to put them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the company responsible for Grok, did not respond to inquiries, aside from an automatically-generated response stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit nudification technology. Under a upcoming criminal offence, anyone who distributed such tools would "receive a prison sentence and significant fines".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people accessing illegal content on their networks, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost AI assistant—with some premium features—which reacts to X users' queries when they mention it in a message.
It is often used to provide reactions or additional background to other users' comments, but people on X are also able to edit an uploaded photo through its AI image editing feature.
Background of Controversy and Rules
The tool has been condemned for allowing users to create images and videos with nudity and sexualised material, and it was previously accused of making graphic content of public figures.
"X has been allowing the creation and distribution of these pictures for a long time without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of persons in a graphic manner".
In a comment, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and clarified this included explicit AI images generated with AI.
It said services such as X were obligated to take "suitable actions" to "lessen the risk" of users viewing illegal content on their platforms and to take down it quickly when they become aware of it.
