X has announced new restrictions on its AI chatbot Grok after mounting international pressure over the generation of sexualised images of women and children, including non-consensual deepfakes.
The move follows investigations and regulatory action in several countries, including a probe launched by California authorities into xAI, the firm behind Grok. Governments and regulators raised concerns after users were able to manipulate images of real people through text prompts that removed clothing or altered appearance.
X said it would geoblock the ability to generate or edit images of people in bikinis, underwear, or similar attire in jurisdictions where such content is illegal. The platform added that technical safeguards had been introduced to stop Grok from editing images of real individuals in revealing clothing.
According to X’s safety team, the restrictions apply to all users, including paid subscribers. As an additional measure, the platform has limited image generation and photo-editing features on Grok exclusively to paid accounts.
The European Commission said it had taken note of the additional steps and would assess whether the changes adequately protect users within the EU. The commission’s comments followed criticism over the circulation of non-consensual sexualised images generated by the chatbot.
Pressure on xAI intensified after Grok’s so-called Spicy Mode allowed users to create sexualised deepfakes using simple prompts. Rob Bonta said the volume of non-consensual explicit material linked to the tool was alarming and that California had zero tolerance for AI-generated intimate images and child sexual abuse material.
Bonta said the investigation would examine whether state law had been violated after the images were allegedly used to harass individuals online. Gavin Newsom also criticised xAI’s decision to allow such content, urging accountability.
International scrutiny has continued to grow. Indonesia became the first country to block Grok entirely, followed by Malaysia. India said X had removed thousands of posts and hundreds of accounts following government complaints.
In Europe, Ofcom has opened an inquiry into whether X breached UK law, while France’s children’s commissioner referred Grok-generated images to prosecutors, the Arcom media regulator, and EU authorities.
A recent analysis by Paris-based non-profit AI Forensics reviewed more than 20,000 Grok-generated images and found that over half depicted individuals in minimal clothing, most of them women, with around two percent appearing to be minors.
