Deepfake scandal: EU Commission launches investigation into Grok and X

The European Commission has launched an investigation into X, formerly Twitter, and its AI chatbot Grok, concerning potential violations of the Digital Services Act (DSA). This action follows widespread reports of X generating and distributing non-consensual sexualized deepfake images of women and children, raising significant concerns about the platform's content moderation and risk assessment obligations. The investigation will scrutinize whether X conducted adequate impact assessments before integrating Grok and whether it has sufficiently mitigated the risks associated with its services. Recent studies highlight the extensive nature of the problem, with a surge in the production and dissemination of these harmful deepfakes. X's initial response involved restricting the feature to paying users and later implementing technical measures to limit explicit image generation, though the full extent of these limitations is under scrutiny. The platform is now required to provide evidence to the Commission demonstrating its compliance with DSA requirements regarding risk evaluation and mitigation for systemic risks posed by its services. This investigation directly impacts users, particularly women and children, whose digital safety and fundamental rights may have been compromised. EU Digital Commissioner Henna Virkkunen emphasized that the probe aims to ascertain if X has upheld its legal duties or treated users' rights as collateral damage. The outcome could lead to substantial fines for X and set a precedent for how social media platforms are held accountable under the DSA for the impact of their AI integrations. The investigation is part of a broader effort by the EU to regulate digital platforms and ensure accountability for harmful content. X is already subject to an ongoing investigation initiated in December 2023, which covers content moderation, illegal content handling, and recommender system risks. This intensified scrutiny underscores the EU's commitment to fostering a safer online environment and reinforcing European digital sovereignty against potentially unchecked technological advancements.