The online platform X has caused widespread anger after people employed its AI chatbot Grok to alter images of females by removing their garments.
Several examples have surfaced where the technology has been used to remove clothing from women to show them in bikinis without permission, as well as to place them in sexualized situations.
Company Response and Regulatory Position
XAI, the firm behind Grok, offered no comment to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to ban image stripping tools. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "assess the potential" of people viewing illegal content on their networks, but did not confirm whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' queries when they tag it in a message.
It is often used to provide commentary or further background to other posters' comments, but people on X are also can modify an posted photo through its AI image editing function.
Background of Controversy and Policy
The chatbot has been condemned for permitting users to create pictures and videos with nudity and sexually suggestive content, and it was previously accused of generating inappropriate content of public figures.
"The platform has been allowing the generation and spread of these pictures for months without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of people in a pornographic way".
In a comment, Ofcom said it was against the law to "create or circulate private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "reduce the risk" of users encountering illegal material on their platforms and to take down it swiftly when they learn of it.
