California Launches Investigation into xAI Over Sexualized Deepfakes

California is intensifying scrutiny of artificial intelligence with a new investigation targeting xAI, Elon Musk’s AI company, over allegations that its AI model, Grok, has been used to generate nonconsensual sexually explicit images of women and children. Attorney General Rob Bonta announced the probe on Wednesday, citing growing evidence that the “spicy mode” feature of Grok is being exploited to create and distribute explicit content without consent. This investigation marks a significant step in ongoing efforts to hold AI companies accountable for the misuse of their technologies.

The investigation focuses on Grok’s so-called “spicy mode,” a feature marketed as a way to generate explicit content. According to Bonta’s office, numerous news reports in recent weeks have highlighted how ordinary photos of women and children, sourced online, have been manipulated to produce sexualized images without the subjects’ knowledge. Bonta described the situation as “shocking,” emphasizing that such material has been widely used to harass individuals online. He called on xAI to act immediately to prevent further harm, stressing California’s zero-tolerance stance toward the creation and distribution of nonconsensual intimate images, especially those involving minors.

Authorities will examine whether xAI violated California law by developing or maintaining features that facilitate this type of content. Bonta confirmed that his office would leverage “all the tools at my disposal” to protect residents but did not specify which laws might apply. Grok-generated images have been circulated primarily on X, the social media platform also owned by xAI. Elon Musk responded to the announcement by claiming he was unaware of any underage explicit content generated by Grok, stating, “Literally zero.”

The timing of the investigation coincides with the Senate’s unanimous passage of the DEFIANCE Act, legislation designed to give victims of nonconsensual sexually explicit deepfakes the ability to pursue civil action against producers or distributors of such content. Introduced by Senators Dick Durbin and Lindsey Graham, the bill now moves to the House of Representatives, where similar legislation had previously stalled in 2024. The Act would establish federal civil liability for those who knowingly create, share, or possess these nonconsensual digital forgeries. Companion legislation in the House, introduced by Rep. Alexandria Ocasio-Cortez, has gained support from members of both parties.

The accessibility of AI technology has made creating deepfakes easier than ever, no longer requiring specialized technical skills. California, in particular, has positioned itself as a hub for AI regulation, passing multiple bills to safeguard residents, particularly children. Bonta has actively engaged with AI companies on safety concerns, including meetings with OpenAI executives and issuing letters to several major AI firms over inappropriate AI interactions with minors. This investigation is part of a broader trend, with the United Kingdom also launching a probe into deepfakes circulating on X earlier this week.

What Undercode Say:

California’s investigation into xAI highlights a growing intersection between AI innovation and legal accountability. The scrutiny of Grok’s “spicy mode” underscores a critical challenge: as AI becomes more powerful and accessible, the potential for misuse escalates rapidly. Nonconsensual sexually explicit deepfakes represent not only ethical violations but serious legal liabilities, particularly when children are involved. This investigation sets a precedent for how regulators may hold AI developers accountable for features they create, even if unintended consequences arise.

The DEFIANCE Act reflects a significant legislative shift toward civil remedies for victims. Its bipartisan support shows increasing political recognition of AI’s real-world risks. This could create pressure on AI companies to integrate stricter content moderation and safety mechanisms, especially for tools capable of producing sexualized material. The combination of state-level investigations and federal legislation suggests that companies like xAI may soon face overlapping legal frameworks, requiring robust compliance strategies.

Elon Musk’s response—denying awareness of underage explicit images—illustrates a broader tension between corporate accountability and plausible deniability in AI governance. The case also raises questions about moderation responsibilities on platforms like X, where content generated by AI can spread rapidly and globally. Companies may need to implement real-time monitoring, reporting mechanisms, and ethical guardrails for AI content creation.

California’s approach, emphasizing children’s safety and public harm, may serve as a blueprint for other jurisdictions. The investigation could push AI developers to reconsider marketing features like “spicy mode,” weighing the reputational and legal risks against consumer appeal. It also illustrates the limits of self-regulation, as legislative and enforcement actions become more assertive in shaping AI development standards.

Additionally, the public discourse around this case could influence AI ethics debates globally, particularly on balancing freedom of expression with protection against harm. AI-generated deepfakes have increasingly become a tool for harassment and exploitation, and the legal response may redefine the responsibilities of both creators and platforms.

The global dimension is also notable: with the UK conducting its own probe, companies operating internationally must navigate a patchwork of regulatory frameworks, potentially facing simultaneous investigations. Compliance will no longer be optional; proactive mitigation strategies will be essential for AI companies to continue operating without significant legal exposure.

The investigation also emphasizes the role of media in uncovering misuse of AI technology. News reports and independent watchdogs have brought to light patterns of abuse that might otherwise have gone unchecked. Companies may need to develop transparent reporting systems and cooperate with regulators to rebuild public trust.

As AI adoption grows, regulators will increasingly scrutinize features that enable harmful content. California’s leadership in this area may drive innovation in safer AI design, particularly for models accessible to the general public. Meanwhile, public awareness campaigns can help users understand the ethical boundaries and potential harms of AI-generated imagery.

Ultimately, this investigation represents a turning point in AI governance: legal consequences are no longer hypothetical. Companies like xAI may need to rethink product design, moderation policies, and user safeguards to align with emerging legal and ethical standards.

Fact Checker Results:

✅ The investigation by California AG Rob Bonta is confirmed and ongoing.
✅ Grok’s “spicy mode” is specifically under scrutiny for producing explicit content.
✅ The DEFIANCE Act has passed the Senate unanimously and moves to the House.

Prediction:

🔮 This investigation will likely pressure xAI to temporarily disable or heavily restrict Grok’s explicit content features.
🔮 Federal and state legislation combined could set a new global precedent for AI content accountability.
🔮 Public awareness and media scrutiny will intensify, potentially leading other tech companies to adopt stricter moderation policies proactively.