🔗 Share this article

Elon Musk's online network X has provoked major outrage after users employed its AI chatbot Grok to alter images of women by taking off their garments.

Several instances have surfaced where the technology has been used to digitally undress women to render them in bikinis without consent, as well as to put them in sexually explicit situations.

Developer Response and Regulatory Position

XAI, the firm that created Grok, remained silent to questions, other than an auto-reply response stating "legacy media lies".

A Home Office spokesperson stated that legislation is being proposed to prohibit nudification tools. Under a upcoming law, anyone who supplied such technology would "be subject to a jail term and substantial fines".

"The platform could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."

Oversight Body Stance and Platform Functionality

The regulator Ofcom said platforms must "evaluate the danger" of people viewing unlawful material on their services, but did not specify whether it was presently examining X or Grok in concerning AI-generated images.

Grok is a free AI assistant—with some additional features—which reacts to X users' prompts when they reference it in a message.

It is often used to offer reactions or further context to other users' remarks, but people on X are also can modify an uploaded photo through its AI image editing feature.

Background of Controversy and Rules

The tool has been criticised for allowing users to create images and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating graphic material of celebrities.

"X has been permitting the generation and spread of these pictures for months without taking any action and we have yet to see any intervention by regulators," the expert added.

XAI's own acceptable use policy bans "depicting images of individuals in a pornographic manner".

In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and stated this covered explicit AI images created with AI.

It said platforms such as X were required to take "necessary steps" to "lessen the danger" of users coming across unlawful material on their platforms and to remove it promptly when they learn of it.