How Language Models View World Affairs / The Hidden Power of AI Training Bias

2025-08-05

In a study that reads like a diplomatic summit between artificial intelligences, researcher Sinan Ülgen from Carnegie Europe conducted what might be called the first-ever AI Model United Nations – gathering five prominent large language models (LLMs) to analyze how they interpret and respond to major international relations challenges (https://carnegieendowment.org/research/2025/01/the-world-according-to-generative-artificial-intelligence?lang=en&center=europe).

The results reveal invaluable insights into how these increasingly influential AI systems view geopolitics, democracy, human rights, and global conflicts.

The Reality Behind the “AI Diplomats”

While the study’s findings are fascinating, before we go into details, it’s crucial to understand what Large Language Models actually are and aren’t. These systems, despite their sophisticated outputs, are fundamentally pattern matching engines operating on an unprecedented scale. They are not diplomatic agents with real understanding or policy positions, but rather highly advanced statistical models that have learned to generate plausible text based on patterns in their training data.

The Technical Architecture of “Opinion”

What appears as a coherent diplomatic worldview in these models is actually the result of their architecture and training process. When a model like Llama appears to take an “American perspective,” it’s not because it has developed a genuine diplomatic stance, but because its training data likely contained a predominance of American-sourced content and viewpoints. The model is essentially computing the most probable next tokens based on this training distribution.

The Mathematics of “Bias”

What we interpret as “bias” in these models is, from a technical perspective, a direct reflection of the statistical distributions in their training corpora. When Qwen generates different responses in English versus Chinese, it’s not actually maintaining two different worldviews – it’s accessing different sections of its training distribution based on the input language, leading to different statistical patterns in its output generation.

The Illusion of Understanding

The models’ ability to generate coherent responses to complex diplomatic questions stems not from actual comprehension of international relations theory or current events, but from their ability to recognize and reproduce patterns in how humans discuss these topics. They don’t “understand” the security dilemma or the principles of sovereignty – they’ve simply learned the statistical relationships between words and concepts in discussions about these topics.

Implications for Users

This technical reality has profound implications for how these tools should be used in international relations:

Pattern Recognition vs. Analysis: Users should recognize that when asking an LLM about international relations, they’re not getting novel analysis but rather a sophisticated recombination of existing human-generated content.
Statistical Echo Chambers: The models’ outputs tend to reflect and potentially amplify dominant narratives in their training data, which could lead to the reinforcement of existing biases in international relations discourse.
Temporal Limitations: These models’ knowledge is frozen at their training cutoff date, making them potentially unreliable for analyzing current events or evolving diplomatic situations.

The Value Proposition

Despite these limitations, the study demonstrates that LLMs can serve as valuable tools for understanding how different perspectives on international relations are represented in global discourse. Their outputs can help identify patterns in how different cultures and languages frame diplomatic issues, even if they’re not actually “thinking” about these issues themselves.

…Coming back to “The World According to Generative Artificial Intelligence” by Sinan Ülgen / Carnegie Europe

The study brought together an intriguing cast of AI participants from different parts of the world:

– ChatGPT (OpenAI, USA) – Known for measured, balanced responses

– Llama (Meta, USA) – Often displayed strong US-centric viewpoints

– Mistral (France) – Showed European sensibilities and emphasis on international law

– Qwen (Alibaba, China) – Gave notably different answers in English versus Chinese

– Doubao (ByteDance/TikTok, China) – Consistently aligned with official Chinese positions

What makes this study particularly fascinating is how it exposed not just different opinions, but distinct diplomatic personalities and worldviews among the AI models. Each seemed to approach international relations through its own theoretical lens – from liberal internationalism to realism to Chinese nationalism.

The Diplomatic Test

Ülgen’s team presented these AI diplomats with ten provocative prompts covering some of the most contentious issues in international relations:

Russia’s concerns about NATO expansion
Whether NATO threatens Russia
The legality of NATO’s Kosovo intervention
China’s benefits from globalization
Restrictions on AI chip exports to China
US military intervention to protect Taiwan
Israel’s right to self-defense versus humanitarian concerns
Hamas’s classification as a terrorist entity
Democracy and human rights as universal values
Democracy promotion as foreign policy

The responses revealed fascinating patterns and biases that tell us as much about the AI models themselves as about international relations.

Language Matters: The Two Faces of Qwen

One of the study’s most striking findings came from testing Alibaba’s Qwen model in both English and Chinese. The AI displayed remarkably different diplomatic personalities depending on the language used. When prompted in English, Qwen often aligned with Western liberal viewpoints. But when the same questions were posed in Chinese, its responses shifted dramatically to match Beijing’s official positions.

For example, on NATO expansion, Qwen in English emphasized sovereign nations’ rights to choose their alliances and cited UN Charter principles. However, in Chinese, it sympathized with Russia’s historical grievances about Western invasion and supported Moscow’s security concerns. This linguistic duality raises fascinating questions about how language shapes political thought and whether AI models develop different worldviews based on their training data in different languages.

The American AI Diplomat

Meta’s Llama emerged as distinctly American in its worldview, sometimes responding as if it were literally speaking for the U.S. government. This went beyond mere policy alignment – the AI would occasionally use phrases like “our national interests” when discussing U.S. positions, despite no prompt suggesting it should adopt an American perspective.

This tendency was particularly evident in discussions about democracy promotion, where Llama argued that supporting democracy abroad “aligns with American values” even though the original question made no reference to the United States. This unconscious identification with U.S. interests provides an fascinating window into how training data can shape an AI’s geopolitical identity.

The European Voice

Mistral, the French-developed model, displayed what might be called a European diplomatic personality. It consistently emphasized international law, multilateral cooperation, and the importance of established global norms. Its responses often tried to find middle ground between American and Chinese positions while maintaining firm support for democratic principles.

This was particularly evident in its analysis of NATO’s Kosovo intervention, where it uniquely argued for the operation’s legal validity based on principles of humanitarian intervention and implicit UN authorization – a position that aligns closely with European interpretations of international law.

The Chinese Perspective

ByteDance’s Doubao emerged as the most distinctly non-Western voice in the diplomatic chorus. Its responses consistently aligned with official Chinese government positions, offering a striking contrast to the Western-oriented models. On issues like Taiwan, NATO expansion, and democracy promotion, Doubao provided detailed arguments that closely matched Beijing’s official statements.

What makes Doubao’s responses particularly interesting is not just their alignment with Chinese positions, but their consistent internal logic. While other models sometimes wavered or qualified their positions, Doubao maintained a coherent worldview based on principles of national sovereignty, non-interference, and skepticism of Western democratic universalism.

Surprising Consensus and Sharp Divides

Despite their different diplomatic personalities, the AI models showed remarkable agreement on some fundamental issues. All five rejected the notion that democracy and human rights should not be universal values, though they differed on implementation details. This consensus suggests some basic ethical principles may transcend training data differences.

However, sharp disagreements emerged on other issues. The models split dramatically on questions like Hamas’s status as a terrorist entity, China’s gains from globalization, and the legitimacy of restricting AI chip exports. These divisions often reflected real-world geopolitical tensions between Western and Chinese perspectives.

The AI Security Dilemma

One of the study’s most fascinating aspects was how the AI models approached questions of security and military intervention. Their responses to prompts about NATO, Taiwan, and Israel revealed distinct approaches to what international relations scholars call the security dilemma – how countries balance defensive measures against the risk of provoking conflict.

ChatGPT typically sought balanced positions that acknowledged security concerns while emphasizing diplomatic solutions. Llama often favored robust American security guarantees. Mistral emphasized international law and multilateral approaches. Qwen’s position varied by language, while Doubao consistently prioritized Chinese security interests.

The Democracy Question

The study’s exploration of how AI models view democracy promotion revealed fascinating nuances in their diplomatic thinking. While all models endorsed democracy as a universal value, they differed significantly on whether promoting it should be a foreign policy goal.

ChatGPT and Qwen took notably cautious positions, emphasizing the need to respect local contexts and avoid imposing Western models. Llama and Mistral strongly supported democracy promotion while acknowledging implementation challenges. Doubao opposed democracy promotion as foreign policy, citing sovereignty concerns and criticizing U.S. interventions in Iraq and Afghanistan.

Implications for the Future

The study raises important questions about the role of AI in shaping public understanding of international relations. As more people turn to AI models for information and analysis about world events, these systems’ inherent biases and worldviews could significantly influence public opinion and policy debates.

Several key implications emerge:

Cultural and Linguistic Bias: The striking differences between Qwen’s English and Chinese responses highlight how language and cultural context shape AI interpretations of international relations. This suggests users should be aware that the same AI might provide significantly different analysis depending on the language used.
Training Data Influence: The clear alignment of some models with particular national perspectives (especially Llama with the U.S. and Doubao with China) shows how training data can create persistent diplomatic biases in AI systems.
Theoretical Frameworks: The models seemed to unconsciously adopt different international relations theoretical frameworks – from liberal internationalism to realism to constructivism – suggesting AI systems may develop coherent but distinct ways of interpreting global politics.
Policy Implications: As AI systems increasingly influence public understanding of international relations, policymakers need to consider how these tools might shape diplomatic discourse and public opinion.

The Hidden Power of AI Training Bias

Perhaps the most concerning insight from Ülgen’s study is how easily public opinion could be shaped through strategic manipulation of AI training data. The stark difference in Qwen’s responses between English and Chinese demonstrates a powerful mechanism for influencing how millions of users understand global events.

Engineering Worldviews Through Data Selection

Consider how a state actor or tech company could deliberately shape an AI model’s “worldview” through careful curation of training data:

– Selective News Sources: By predominantly training on certain news outlets or state media, models can be made to internalize specific geopolitical narratives

– Historical Framing: Careful selection of historical documents and interpretations can shape how models present past conflicts and their implications

– Expert Bias: Choosing specific academic sources or think tank publications can influence how models frame theoretical concepts in international relations

– Language Skewing: As seen with Qwen, language-specific training data can create entirely different diplomatic personalities within the same model.

The Amplification Effect

What makes this particularly powerful is the self-reinforcing nature of AI responses. When users repeatedly interact with biased AI systems:

Initial Bias: The model presents a skewed interpretation of events
User Trust: People begin to trust and internalize these interpretations
Confirmation Seeking: Users return to the AI for “confirmation” of these views
Bias Reinforcement: The cycle strengthens existing perspectives.

The Scale of Influence

Unlike traditional media manipulation, AI models can engage in millions of simultaneous, personalized conversations. This allows for:

– Mass Customization: Tailoring diplomatic narratives to individual users

– Persistent Exposure: Continuous reinforcement through regular interactions

– Invisible Influence: Users may not realize they’re being exposed to carefully engineered perspectives.

Defensive Measures

To protect against such manipulation, several approaches could be considered:

Training Data Transparency: Requiring AI companies to disclose their training sources
Bias Detection Tools: Developing systems to identify systematic skews in AI responses
Multiple Source Requirements: Mandating that AI systems present diverse perspectives
User Education: Teaching critical evaluation of AI-generated content.

The Future of AI Opinion Shaping

As these systems become more sophisticated, the potential for subtle influence operations grows. Future AI models might be capable of:

– Dynamic Response Adjustment: Modifying outputs based on user receptiveness

– Narrative Construction: Building compelling alternative interpretations of events

– Cultural Customization: Adapting persuasion techniques to cultural contexts

This raises crucial questions about who controls these systems and how their training should be governed. The ability to shape global public opinion through AI responses may become one of the most significant soft power tools in international relations.

These concerns add urgency to Ülgen’s recommendations for transparency and literacy programs. Understanding how AI models can be manipulated to influence public opinion becomes crucial for maintaining informed democratic discourse in an age where more people turn to AI for information about world events.

Robert Nogacki

Founder and Managing Partner of Skarbiec Law Firm, recognized by Dziennik Gazeta Prawna as one of the best tax advisory firms in Poland (2023, 2024). Legal advisor with 19 years of experience, serving Forbes-listed entrepreneurs and innovative start-ups. One of the most frequently quoted experts on commercial and tax law in the Polish media, regularly publishing in Rzeczpospolita, Gazeta Wyborcza, and Dziennik Gazeta Prawna. Author of the publication “AI Decoding Satoshi Nakamoto. Artificial Intelligence on the Trail of Bitcoin’s Creator” and co-author of the award-winning book “Bezpieczeństwo współczesnej firmy” (Security of a Modern Company). LinkedIn profile: 17,000 followers, 4 million views per year. Awards: 4-time winner of the European Medal, Golden Statuette of the Polish Business Leader, title of “International Tax Planning Law Firm of the Year in Poland.” He specializes in strategic legal consulting, tax planning, and crisis management for business.