When disinformation expert Tal Hagin asked Grok to verify a post on X about Iranian missiles that supposedly hit Tel Aviv, Elon Musk’s AI-powered chatbot failed miserably.
Grok from time to time misidentified location and date of the video, which was originally together on X and Iran’s state-owned media outlet on Sunday. Then, the chatbot tried to prove his point by AI-generated image sharing.
“Grok now responds with an AI cascade of destruction,” Hagin wrote in response. “It’s cooked I tell you.”
The interaction neatly sums up how unfettered X has been by the truth since the US and Israel began attacking Iran on February 28. WIRED reported at the time.
As the conflict continues, the floods have worsened. In recent times, it has been more charged with AI photos and videos, while Grok has repeatedly given false information when asked to confirm the claims made on the platform. AI images are shared by paid accounts with blue symbols and Iranian officials who want to show the extreme destruction.
The rise of easy-to-access AI image and video creation tools has made fake content more sophisticated. On March 2, for example, Iranian officials and state media shared AI-generated videos of a A skyscraper in Bahrain is on fire. The videos and pictures seem real enough to many: One picture of a US B-2 fighter jet shot down by Iran while US soldiers were in custody was viewed more than a million times before it was deleted. picture of Delta Force members arrested by Iranian authorities was viewed more than 5 million times before being deleted.
Some of the AI content promoted on X is not realistic. One video, for example, purports to show Iranian forces making missiles deep inside a cave. However, the video was still shared by many accounts and has been viewed over a million times.
AI is also being used by the Iranian government to push anti-Semitic stories, with accounts on the pro-government propaganda network X sharing AI-powered posts showing Orthodox Jews leading American soldiers into battle or celebrating American deaths, according to researchers from the Institute for Strategic Dialogue (ISD), who shared their analysis with WIRED.
The number of accounts in this network is pro-regime as well shared a fake video which reportedly showed a line of little girls walking past President Donald Trump in nothing but underwear. The post was viewed more than 6.8 million times, according to ISD, before it was removed, although it continues to be shared by other accounts on X.
“What’s unique about this battle is the sheer excitement in AI-driven content I find myself resisting,” Hagin tells WIRED. “This is possible due to AI being advanced enough to fool journalists, and the ease with which users can create this AI cascade with zero consequences. The longer we go without regulations against the misuse of AI, the more harm will be caused. I see the proliferation of AI-based fake news pushing us to the brink of a truth-based world unless we enact change now.”
When a flood of AI-generated fakes began taking over the platform last week, X announced that it would temporarily defund blue-label accounts if they posted AI-generated videos of untagged fights. X did not respond to a request for comment on how much the account has earned since the measure was introduced. Until recently, the number of Iranian officials appeared to pay X with its paid service, which gave their accounts blue check marks, strengthened cooperation, and created the possibility of earning money for their posts.
