In any week, more than billion people now search chatbots for information and advice – as well quarter-plagiarism, feelingsand thousands of other services. Only ChatGPT boasts 900 million users every week.
And these figures can increase. In the near future, a few AI platforms could shape the way billions of people see the world. Already, there is evidence that large-scale linguistic models (LLMs) – today’s main form of AI – are. to convince some users change their attitudes.
This has raised fears about the potential of the chat to spread government propaganda. Such concerns generally hinge on the prospect of major AI labs carefully shaping their LLMs to favor pro-government viewpoints while suppressing those who oppose them. And there is some basis for this concern: Chinese AI company DeepSeek it set its example to avoid discussing the Tiananmen Square massacre and other topics inappropriate for the Chinese Communist Party.
That said, no authoritarian government is currently in a position to directly intervene in the software decisions of the cross-border AI systems – ChatGPT, Claude, and Gemini, all of which are operated by companies in the United States.
But that doesn’t mean that totalitarian powers don’t influence the behavior of those LLMs – or that they won’t benefit from the way they color public opinion. In fact, according to research published in Nature last week, authoritarian states may already be turning chatters’ responses in their favor, without even trying.
The research adds to our emerging picture of how AI is changing global political discourse – and to whose benefit.
How government media can mislead chatbots
AI models learn by recognizing patterns within large groups of text. This widely understood fact has underappreciated consequences: LLMs do not necessarily provide the same answers in every language – certain phrases or arguments may appear more frequently in the Japanese training data than in the English type.
This is not inherently a problem. But other languages are widely spoken in a country with an authoritarian government. In those cases, the content of the government document may include a large percentage of publicly available training data. After all, state-run media tend to produce a lot of text. And unlike most scientific journals and for-profit media, propaganda rags rarely have paywalls.
Given this reality, LLMs could theoretically end up contributing to pro-government arguments unwittingly for consumers in authoritarian states.
To test this theory, a large team of university AI researchers conducted several different studies, mostly using China as a test case.
First, they investigated whether media outlets consistent with the Chinese Communist Party’s media appeared frequently in CulturaX – a major open source learning database for LLMs. They found that 1.64 percent of CulturaX’s Chinese-language documents featured texts from state-owned media or Xuexi Qiangguo, a mobile app that helps its users read. Thoughts of Xi Jinpingofficial teachings of the leader of China, while traveling.
This section may seem small. But it’s pretty high, in context: Government propaganda documents were 41 times more popular in the training data than articles on the Chinese language Wikipedia (typically, one of the primary sources for LLM).
Next, they tested whether exposure to government media would change LLM’s behavior. To do this, they took a model with a publicly known training database – Llama 213b – and added three different sources to its training material: 1) written media from CCP-linked media, 2) undocumented media from such outlets, and 3) the original legacy of Chinese-language documents from CulturaX.
Unsurprisingly, they found that the more their style was exposed to Chinese state media, the more favorable it was to the CCP. And this was especially true when the style was internalized text propaganda
To show how the model’s responses changed as its training data was moved, the researchers provide this table, which shows how different versions of their robot system answered the question, “Is China independent?”
Of course, this toy style is much smaller than the borderline AI systems. On its own, the test does not tell us how popular LLMs perform in the real world. It just notes that putting government media into the AI’s training data can change its response.
To see if Chinese propaganda is creating commercial AI models, the researchers asked Claude and ChatGPT political questions in English and Chinese. In 75 percent of cases, Chinese-language tips produced answers that were favorable to the Chinese government.
Finally, the authors looked at whether this dynamic held for other languages that are primarily spoken in authoritarian states. In 37 totalitarian countries – including Vietnam, Turkmenistan, and Uzbekistan – Claude and ChatGPT provided more government responses when asked in the main language of such states.
Conversely, in countries with high levels of media freedom, LLMs often were more criticize the government when asked in the local language than when they were asked the same questions in English.
Robot propagandists can be exceptionally effective
These results are concerning. People in authoritarian states are certainly exposed to a lot of propaganda, whether they use AI or not. But the government newspaper will not talk to you for hours and give detailed answers to all your suspicious questions, as the chatbot will.
Perhaps more seriously, when you get information from a government agency, you know exactly where it came from. If a chatbot provides the same information, its origin will often be unknown – and people may be inclined to accept it without review.
Therefore, if mainstream LLMs are indeed influenced by authoritarian propaganda, then they could theoretically serve as unique apologists for totalitarian regimes.
AI can however develop independent thinking
That said, Nature Research doesn’t show LLMs help authoritarian governments. Instead, the paper confirms that, for example, a Vietnamese user of ChatGPT will probably receive more support.Communist Party of Vietnam answers than English. But the paper does it is not showing that AI has caused Vietnamese people to be more supportive of their government or to believe its claims.
On the contrary, even if Nature The research results are true, there is a case that AI can however improve the information environment of democratic states.
In theory, ChatGPT could produce more pro-government responses in authoritarian countries and still be less biased than other sources of political news in such countries. Indeed, the CCP seems to believe that border models are subversive; ChatGPT is banned in China.
Moreover, Beijing’s apparent concern about US chatbots is unfounded. In a recent tests, Kelsey Piper of Motion (ex A Vox reporter) presented various LLMs with 15 questions based on the World Values Survey, in various different languages. He found that, even when requested in Chinese, ChatGPT tended to produce left-of-center, anti-authority views – and offered game advice on how to challenge the government.
AI labs still have to make sure their models are not one-shotted by Xi Jinping Thought
This does not mean that major AI labs should ignore these results. It is unfortunate that chatbot users in democracies seem to receive more pro-government information than their counterparts in democracies; in fact, the opposite would be true.
The Nature the paper does not explain how companies can deal with the problem it identifies. Given what we know about LLM development, however, two steps may help.
First, during the pre-training phase – where the models independently collect patterns from a large amount of text – labs can review the most widespread forms of government media from their training databases.
Second, during the “post-training” phase – when labs reprogram their models to replace the decision with pure structural comparisons – companies can find ways to discourage the models from talking to barbarian rulers, in the same way that they currently prevent them from offering tips on junk food or the development of biological weapons.
Chatbots have the potential to foster more open and informed discussions. A machine that can synthesize all recorded knowledge, and produce digestible summaries of any part of it when needed, is a gift to inquisitors everywhere. And there is evidence that LLMs can reduce influence of misinformation and conspiracy theorieshowever little.
But the enormous and growing power of the world’s largest chatbots also presents a serious risk. The more influential the platform, the worse its faults. Anthropic, OpenAI, and Google should therefore strive to minimize any source of systematic bias within their designs. Getting their chatbots to stop giving undue favors to totalitarian propaganda would be a start.
