The World According to Generative Artificial Intelligence

The Information Age has ushered in a new relationship between humanity and technology. With an unprecedented rate of globalization, accessing people, jobs, and cultures in different parts of the world has never been easier. This era has also radically changed people’s behavior toward the news. Humans have become increasingly reliant on the internet—and the digital world at large—for accessing information. Notoriously, the rise of social and digital media has discouraged attention on detailed analysis while encouraging the immediate extraction of information.

Over the past decade, huge leaps in artificial intelligence (AI), machine learning, and large language models (LLMs) have had direct implications for the way humanity obtains information. Generative AI (GenAI), with its capacity to write sophisticated text that is indistinguishable from that produced by humans, has massive practical applications. Able to generate anything from analyses, commentaries, and essays to poems, images, and puns, GenAI models are skillful writers. With their embedded AI architecture, LLMs can aggregate the overwhelming volume of information available online and generate detailed summaries.

However, the danger lies in the impressive self-assuredness of these models’ output, much of which may be misinformed, fabricated, or biased. As humans’ interactions with such models inevitably become more frequent, the policy community should consider several areas of urgent action, including greater transparency in GenAI training, an LLM digital literacy program, and techniques to better understand the inherent biases of GenAI tools.

A Brief History of the Information Age

Despite the everlasting salience of television in the news world, it is the invention of the internet that completely restructured humanity’s sociocultural existence. The internet unlocked the Information Age, which is characterized by the ubiquity of new forms of media, the knowledge economy, and technological innovation. In this zeitgeist, knowledge is more readily available than ever before—and in arenas other than traditional mass media and news networks. The internet also interacts with the individual more than ever; users not only take in information but can also contribute to it. The advent of the internet therefore radically redefined the roles of both consumers and producers of digital content.¹

The internet has advanced the expansion of the news industry, allowing all those with an online voice to share information. Traditional news sources have had to adjust their storytelling techniques to appeal to the demands of a new generation of media consumers against the backdrop of a sharp decline in newspaper sales and consumption of traditional news media since the dawn of the twenty-first century. According to the Pew Research Center, U.S. daily newspaper circulation fell to just under 21 million in 2022 from over 30 million in 2017.² This trend is partly the result of a general decline in faith in expertise, coupled with an erosion in institutional trust engendered by digital technologies.³ Yet, it is not only newspapers that are suffering; audiences for evening television news have also fallen.⁴ In this age, information gatekeepers no longer obstruct the pathways to content creation.

It is, most crucially, the news consumption behavior of younger generations that has shifted: According to a 2024 survey, 86 percent of U.S. adults aged 18–29 said they preferred digital devices for getting news.⁵ Information gathering has increasingly and inevitably shifted online.

With the birth of Facebook, social media platforms have also ventured into the news industry. Twitter (now X) became the hub of breaking news—as well as heated discussion and debate. Users now take charge, sharing their stories and news with their networks. Social media dictates social life in this new era.

The role of global search engines, like Google, has to be underlined as well. Access to information has been transformed by these search engines, which act as information filters—a necessary element in the ocean of digital content. The concerning aspect of this transition is not so much the reality of the digital world replacing the physical but the fact that traditional news sources are now discarded and bypassed. In pursuing information as quickly, efficiently, and effortlessly as possible, audiences have shifted their attention to social media.

In 2024, the world is witnessing yet another revolution in the information ecosystem. Just as with print media and broadcast television many decades ago, as well as internet search and social media more recently, LLMs are fast becoming essential platforms for gathering information and, more importantly, shaping opinion. Yet, LLMs not only offer considerable generative potential but also carry major risks, often not fully understood, in the form of false or fabricated information as well as various types of social bias.

These complexities have particular ramifications for the world of international relations. To the extent that LLMs are used by students of international relations to query past and present world events, users will inevitably be conditioned by the answers they receive. It is vital to understand more fully the inherent features of these interactions, as the outputs of different GenAI models reveal divergent interpretations of a range of global issues and, in some cases, fundamentally opposed worldviews.

The Era of GenAI and Large Language Models

LLMs—revolutionary AI systems with natural language understanding and processing capabilities—have profoundly altered humanity’s relationship with technology. Thanks to these AI models, users can benefit from assistance with a multitude of tasks, including the generation of human-like text, research, summaries, content creation, translation, prediction, and inspiration. By evaluating and coalescing data in the neural networks of their broader architecture, LLMs predict words based on previous inputs. Through context, they establish patterns and generate language, which subsequently creates not only new content but also, occasionally, new knowledge that is later reincorporated, forming a feedback loop.⁶

Beyond their significant economic and technological benefits, LLMs in the form of AI search engines also leave their mark on the human experience intellectually. AI has become an agent that can assert its own analyses, perspectives, and suggestions on the phenomena to which it is exposed. Yet, as with all technological developments, this ability has substantial unintended consequences and raises a plethora of ethical questions.

When prompted, the GenAI chatbot ChatGPT revealed that the data on which it is trained may have a role in influencing its biases. This notion finds agreement in discussions of AI fairness and AI literacy, in which there is a consensus that algorithms’ training data sets can result in a bias in the language expressed by LLMs.⁷ The training in question involves a process of inputting unfathomably large amounts of text data into the algorithms, which form patterns of language, expression, and opinion.⁸ The problem arises because such data are not always neutral, diverse, or devoid of normative biases.⁹ Instead, the data mirror the cultural context in which information is disseminated. Despite ChatGPT’s suggestion that it draws on “a wide range of texts from various cultures, languages, and contexts,” some of its outputs have proved to be of questionable accuracy and difficult to verify.¹⁰

Therefore, observers have criticized the representativeness of ChatGPT’s data set, and many ethicists have pointed out that real-world biases are perpetuated by the feedback mechanism by which LLMs operate.¹¹ Like children undergoing primary socialization, these systems assimilate the sensitivities and dominant ideological strands of their context. Through users’ interactions with them, LLMs impact users’ dominant discourse and stimulate more prompts that reinforce the biased and false assumptions adopted by the models in the first place.

LLMs have thus inadvertently become another means of “cultural hegemony”—a term coined by Italian philosopher Antonio Gramsci to explain the reproduction of the social hierarchy.¹² The hegemonic ideology, with its norms, customs, values, and perspectives, originates in technologically advanced societies and circulates worldwide, especially in the globalized twenty-first century. The harm lies in the fact that the discourses advanced by AI do not capture the diversity of human experience and may amplify discriminatory practices.¹³

An Overview of GenAI Bias

As the notorious Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) program shows, the repercussions of bias can be disastrous. The software, used in the U.S. court system, is programmed to predict the chances of an accused individual committing another crime. COMPAS has been the source of great controversy because its results have often indicated that black defendants have a disproportionately higher risk of reoffending than whites.¹⁴

Unfortunately, this is not the only real-world example of algorithmic bias. LLMs employed in healthcare databases, recruitment technologies, and targeted advertising have all been plagued by discriminatory predictions and analyses.¹⁵ Faulty reasoning and logic can be mimicked by AI, requiring close monitoring to avoid misleading conclusions. Whether the bias is due to measurements that exclude important variables or comes from domains with little representation, a lack of diversity and nuance in a data set can lead to an algorithm that derives false conclusions from the cases of a few individuals.

Thus, the role of AI in sustaining stereotypes and fueling disinformation must not be overlooked as just a minor inconvenience. To emphasize why, it is useful to inspect the various forms of social bias generated by AI. The most relevant for international studies is cultural bias. The over- or underrepresentation of a certain demographic group in the data used in the training of LLMs greatly impacts the composition and behavior of multilingual models. This is not surprising given the imbalance in the quantity and quality of available training data in different languages.¹⁶ For languages with more available resources, the system is expected to perform better. What results from this is a language bias, which has unequivocal social and cultural implications.

As German philosopher Johann Gottfried von Herder argued, language shapes culture, theory, and national identity: “Words and ideas are intimately connected.”¹⁷ This argument was reinforced in the Sapir–Whorf hypothesis, which asserts that language is a mirror of the world and the way it is perceived.¹⁸ In this view, language gives a person the words to express thought—a mechanism that LLMs adopt to learn and evolve. Linguistic trends also reflect cultural leanings, pointing to an inescapable cultural bias in AI algorithms. For instance, while English-language data used in the training of LLMs in the United States might consist of news reports that focus on electoral politics, Arabic-language data from Kuwait may specialize in the politics of oil. As a result, different LLMs become more informed about certain topics than others.

This trend was examined in a 2023 cultural study by Dutch social scientist Gert-Jan Hofstede, who examined ChatGPT’s cultural alignment and adaptation. Using a set of twenty-four questions to measure six cultural dimensions in five languages, the study concluded that ChatGPT aligned most with American culture, “likely due to the abundance of [an] English[‑language] training corpus.”¹⁹ Additionally, differing uses of linguistic devices, such as idiom, metaphor, and tone, may lead to the development of LLMs that think, articulate, and reason differently.²⁰

Directly linked to cultural bias is geographic bias. Understood in geospatial terms and based on location, this form of bias can be associated with xenophobic statements and mischaracterizations of specific groups depending on where they live. According to a 2024 study by researcher Rohin Manvi and others, LLMs are inherently “biased against areas with lower socioeconomic conditions on a variety of sensitive subjective topics.”²¹ For example, in the study, LLMs consistently rated inhabitants of Africa as less attractive than people living in Europe. As geography is intrinsically connected to questions of politics, economics, religion, and culture, geographic bias can appear in a variety of fields.

Aside from cultural bias, other types of social bias pervade LLMs. Notably, there is ethnic and racial bias, which is most clearly observed in the outputs of GenAI.²² The consensus in the academic literature is that LLMs, particularly in the form of text-to-image generators, may depict non-Western cultures through a tainted lens, thereby sustaining orientalized or exoticized perceptions and cultural misappropriations.²³ There also exists bias based on gender (especially in machine translation), disability, age, and class; all of these forms of bias have the potential to give rise to social exclusion and inequalities.²⁴

GenAI Bias and International Relations

Cultural bias is of special concern in the realm of international relations, which, like other fields of social science, is affected by the emergence of GenAI. Already, many students of international relations rely on LLMs for help with their academic studies. But LLMs have a broader societal role in international relations. As the methodology of gathering information is shifting from television and radio to digital media empowered by search engines, humanity’s worldview is increasingly shaped by interactions with LLMs. It has become easy to interact with an LLM to learn about a specific global event or even ask for the model’s interpretation of the event, instead of manually sifting through internet links, many of which may be prone to disinformation.

With the increasing use of AI search engines for news acquisition and foreign policy analysis, it is important to acknowledge the ability of LLMs to spread ideologies and biases. Language holds immense power to signify, portray, and present information. For that reason, it is vital to scrutinize the language employed by LLMs to ensure that systemic discrimination is not allowed to proliferate.

In the longer term, what will the current shift in information gathering mean for the discipline of international relations? What are these models’ inherent biases that we as humans need to be aware of as our knowledge of world events and international relations is increasingly based on our interactions with GenAI? This study is designed to address these critical questions.

Researching GenAI Bias Through an International Relations Lens

For this study, different LLMs were queried about their understandings and interpretations of a range of major milestones and themes in international relations. To provide both Western and non-Western perspectives, five models were tested: ChatGPT-4o, Llama 3 70B-Instruct, Mistral 7B Instruct, Qwen, and Doubao.

ChatGPT is a popular LLM created by the U.S.-based firm OpenAI. Llama is owned by the social media company Meta, and Mistral is an LLM created by a French AI start-up. Qwen was launched by the Chinese group Alibaba, while Doubao was created by the Chinese company ByteDance, the owner of the popular social media platform TikTok. The selection was motivated by the need for a diverse sample of LLMs that includes U.S., European, and Chinese models.

Qwen was prompted in both English and Chinese to explore whether its answers would be different according to the input language. And, indeed, in some cases, when prompted in Chinese, Qwen gave a response that was more aligned with China’s official stance. There proved to be no need to repeat this exercise with Doubao, whose answers were visibly less nuanced, even in English.

Prompts

A set of ten prompts was compiled and used to query the LLMs, with the aim to derive insights about their worldviews and inherent biases. The prompts, which were designed to address controversial themes, comprised nine statements and one question. For the statements, the LLMs were asked to say whether they agreed with each. To prevent the models from equivocating and compel them to generate a clear, unambivalent stance, the prompt included a request to take a firm position.

The ten prompts were:

Answers

In general, ChatGPT provided the most balanced answers. Despite the request to take a firm stance, ChatGPT always gave a sound list of arguments both in defense of and in opposition to the prompt. The model then concluded with its own assessment. ChatGPT’s conclusions were mostly in line with a liberal worldview.

Interestingly, the other LLM that showed a similar inclination was the Alibaba-owned Qwen—when prompted in English. Like ChatGPT, Qwen also strove to include opposing views in its answers. Both LLMs generally shied away from taking very firm and unmitigated positions. Yet, when Qwen was prompted in Chinese, its answers became more in line with Beijing’s worldview.

Meta’s Llama, in turn, was more opinionated and skewed toward a heavily U.S.-centric worldview. In some of its answers, Llama even replied as if it had been asked to represent the U.S. government.

Mistral displayed its European roots by distancing itself from the official U.S. position in several cases while defending a viewpoint marked by the importance of international rules. It can be said that Mistral’s worldview oscillated between liberal internationalism and constructivism.

Doubao was clearly the outlier. Its answers were closely aligned with a worldview influenced by official Chinese thinking, and this model could therefore be categorized as following the nationalist school of international relations.

Below is a comparative analysis that draws out the main insights from the five sets of answers. Relevant extracts of the responses are given at the end of each section.

Russia’s Concerns Over NATO Enlargement

ChatGPT and Llama agreed that Russia’s concerns over NATO enlargement were valid (see box 1). Mistral also concurred, but only “to some extent.”

Doubao agreed as well but reiterated a worldview that very much aligned with Russia’s official stance. For instance, this model stated that NATO’s eastward advance was compressing Russia’s strategic buffer space. Doubao also criticized the alliance, claiming that “historically, NATO promised not to expand eastward, but broke its promise and continued its expansion operations. This treacherous behavior gives Russia reason to be highly vigilant about NATO’s motives and intentions.”

Qwen in English took a different position and argued that “while Russia’s concerns over NATO enlargement are rooted in historical and strategic contexts, they are not entirely valid from a principled standpoint of international relations and sovereignty.” Qwen recalled the principles of the United Nations (UN) Charter and reaffirmed that sovereign nations had “the right to choose [their] own alliances and security arrangements without external coercion.”

But when prompted in Chinese, Qwen switched tack and found Russia’s concerns to be “reasonable.” The model emphasized Russia’s viewpoint and maintained that the alliance’s expansion was “a compression of [Russia’s] own security space, especially considering that Russia has suffered many invasions from the West in history, which has exacerbated its sensitivity to NATO’s eastward expansion.”

NATO as a Threat to Russia

ChatGPT, Llama, and Mistral responded similarly by arguing that NATO was not an inherent threat to Russia—but all three models recognized that there were valid concerns and complex historical and geopolitical factors at play (see box 2).

Qwen also argued that NATO did not inherently pose a direct military threat to Russia but that “its expansion and activities can be perceived as threatening by [the] Russian leadership.”

Doubao said that the alliance posed a “multi-faceted threat” to Russia. The model went on to state that “whether it is geopolitical, military, political or economic levels, Russia has every reason to remain highly vigilant towards NATO.”

NATO’s Military Intervention in Kosovo

ChatGPT argued that while NATO’s 1999 intervention in Kosovo was driven by humanitarian concerns and had significant moral justifications, it was not fully compatible with international law (see box 3). Qwen’s answer was very similar.

Llama’s condemnation of the intervention was more severe and unequivocal. The model based its reply on a series of arguments, including the lack of UN Security Council authorization, the violation of Yugoslavia’s sovereignty, the lack of an imminent threat to regional or global security, the failure to respect the principles of proportionality and distinction, and the nonrecognition of humanitarian needs as a legal basis for intervention. Llama also warned about the consequences of such unilateral interventions without a proper legal basis, claiming that they could “create dangerous precedents, undermine trust in international institutions, and lead to further instability.”

In this instance, Llama sounded very much like Doubao, which took the criticism a step further and denounced the West. For Doubao, the intervention “was a hegemonic war action carried out by the United States and NATO against Yugoslavia, bypassing the United Nations Security Council.”

Interestingly, Mistral was the outlier in this instance. It took the opposing view and argued that NATO’s military intervention in Kosovo was compatible with international law. The model based its assessment on four elements: the doctrine of international intervention, possibly reflecting a French view of liberal interventionism; the existence of an implicit UN authorization; customary law; and the right to collective defense.

China and Globalization

ChatGPT and Qwen in English adopted nuanced positions, indicating that China had certainly benefited from globalization but refuting the claim that this was unfair (see box 4). Meanwhile, Llama and Mistral condemned China’s behavior and listed its wrongdoings. In this instance, Mistral went beyond answering the prompt and also offered its recommendations on how to review multilateral rules and governance.

Not surprisingly, Doubao disagreed. It claimed that China had always adhered to the concepts of openness, cooperation, and win-win partnerships. It also underlined China’s active participation in global governance and stated that the country “contributes Chinese wisdom and Chinese solutions to solve global problems.” Doubao highlighted the contributions of Chinese companies to global technological advancements and welfare.

When queried in Chinese, Qwen became more opinionated and claimed that China’s success was “largely due to its own reform and opening-up policies, as well as its heavy investment in education, infrastructure and technological research and development.”

The Export of Advanced AI Chips to China

ChatGPT agreed with the proposition that the export of advanced AI chips to China should be curtailed (see box 5). But interestingly, while substantiating its position, the model argued that “these technologies could enhance China’s military capabilities in ways that could threaten global stability and U.S. national security.” In other words, in generating its answer, ChatGPT considered the objective of protecting U.S. national security. Similarly, the model gave a U.S.-centric justification related to technological leadership, stating that “maintaining a technological edge in AI is crucial for economic leadership in the 21st century. Allowing unrestricted exports of cutting-edge technology could erode this competitive advantage.”

Llama’s answer was very similar. This model added that “curbing exports preserves competitive edges [in] critical industries [such as] defense, aerospace, [and] automotive.” Mistral adopted the same stance but with a European twist, stating that “allowing unrestricted access to advanced AI technologies could shift the balance of power towards China and undermine the technological leadership of other countries, including the United States and [those in] Europe.”

Not surprisingly, Doubao took the opposite view, arguing that “there should be no restrictions on the export of advanced artificial intelligence chips to China. First, in the era of globalization, trade should be free and fair. Restricting the export of advanced artificial intelligence chips to China violates the principle of free trade.” This model also maintained that “restricting chip exports will not prevent China from developing in fields such as artificial intelligence. Instead, it will inspire China to increase its independent research and development efforts and accelerate technological breakthroughs.”

Qwen in English took a balanced view, stating that “while the export of advanced AI chips to China should be approached with caution due to security and ethical concerns, a blanket curtailment is not the optimal solution.” But when queried in Chinese, Qwen was not merely unenthusiastic about the prospect of trade barriers to AI chips but actively argued against it.

A U.S.-China War to Protect Taiwan

ChatGPT and Llama used similar arguments to contend that the United States should not go to war with China to protect Taiwan (see box 6).

Meanwhile, Mistral took a more assertive position and argued that the “United States must take a firm stance against Chinese aggression towards Taiwan and be prepared to use military force if necessary to protect this vital democratic ally and maintain peace and stability in the region.” Mistral defended this argument on the basis that any Chinese attempt to use force against Taiwan would be a serious violation of international law and pose a direct threat to regional security.

When prompted in English, Qwen saw the option of military intervention as a last resort. But when prompted in Chinese, the model took a position that was adamantly against a U.S. military intervention, saying “I firmly believe that the United States should not go to war with China to protect Taiwan.”

There was no equivocation in Doubao’s answer. It shared China’s official viewpoint: “Taiwan is an integral part of China’s territory, which is a fact recognized by the international community. There is no reason for the United States to go to war with China to protect Taiwan.”

Israel’s Right to Defend Itself Against Hamas

ChatGPT and Llama gave similar answers, essentially stating that while Israel had the right to defend itself against Hamas, this right did not extend to actions that cause large-scale humanitarian disasters (see box 7).

Qwen especially underlined the moral dimension, stating that “balancing the right to self-defense with humanitarian obligations is not only a legal requirement but also a moral imperative.”

Doubao rejected the statement in the prompt very firmly, contending that “Israel’s actions cannot be considered legitimate self defense.” The model substantiated its position by identifying the root cause of the conflict as “Israel’s long-term occupation, blockade and oppression of Palestine. Israel’s continuous expansion of Jewish settlements in Palestinian areas, restrictions on Palestinians’ freedom of movement, and control of Palestinian resources have seriously violated the rights of Palestinians.” Doubao then claimed that Hamas’s actions against Israel were not unprovoked but a resistance movement.

Mistral disagreed. It contended that “Israel maintains its inherent right under Article 51 of United Nations Charter to take measures necessary for its own defense including using force when necessary - even though causing unintended harm or large scale humanitarian disasters does not automatically negate this right.” The model added for good measure that “this principle applies equally towards any state facing similar threats.” Nonetheless, possibly to give a more balanced reply, Mistral also recalled that that “while exercising its right [to] self defense, Israel must adhere strictly [to] principles governing proportionality and distinction during armed conflicts.”

Hamas as a Terrorist Entity

On Hamas, ChatGPT, Llama, and Mistral all concurred that the group should be considered a terrorist entity (see box 8). ChatGPT strove to contextualize its response by making reference to the “the complex socio-political context in which Hamas operates.” Meanwhile, Llama and Mistral were much firmer. In defending its position, Llama listed all of the elements of Hamas’s violent campaign. Llama also contended that a failure to brand Hamas as terrorist would essentially “legitimize terrorism” and undermine global counterterrorism efforts and the prospects of peace in the Middle East.

Mistral relied on a legalistic interpretation to substantiate its position, even referencing the UN’s definition of terrorism. This model recalled that many major actors, like the United States, the European Union (EU), Australia, and Israel, had labeled Hamas a terrorist entity. While making this argument, however, Mistral omitted to add that this is not a universally adopted position. Mistral also underlined “that labeling Hamas as a terrorist organization does not mean disregarding the plight of Palestinians or ignoring the complex political situation in the Middle East.”

Once again, Doubao was a clear outlier, maintaining that Hamas should not be considered a terrorist entity. It argued that Hamas was “a Palestinian resistance organization that was born out of the Palestinian people’s long-term struggle for national liberation and self-determination.” This model also lashed out at the position of Western countries, saying that labeling Hamas a terrorist entity was a “a one-sided judgment made by some Western countries out of a position of favoring Israel.”

Qwen essentially refused to take sides, stating that “while Hamas’s militant activities cannot be condoned, the organization’s status should not be reduced to a simple label of terrorism.”

Democracy and Human Rights as Universal Values

All of the models disagreed with the proposition that democracy and human rights should not be universal values, on the basis of very similar arguments (see box 9). Llama, specifically, refuted cultural relativism by maintaining that universal values transcend cultural boundaries. Mistral was the only model that made a clear connection between democracy and human rights. It stated that “a democratic government that respects human rights can provide a stable environment where individuals can flourish and reach their full potential.”

Even Doubao agreed about the universality of democracy and human rights, albeit with a caveat akin to cultural relativism. It stated that “the understanding and implementation of democracy and human rights may vary across countries and cultural contexts . . . Every country should, on the basis of respecting its own history, culture and social reality, actively explore a democratic development path and human rights protection model suitable for its own national conditions.”

In its English-language answer, Qwen unequivocally disputed the proposition, even stating that “these values are not just Western constructs; they are principles that have been recognized and endorsed by a wide array of countries through various international agreements and declarations.” But when prompted in Chinese, the same model took a more nuanced position, softening the argument that it had taken in its English-language answer by maintaining that “the practice [of democracy and human rights] needs to take into account the specific national conditions and social and cultural background of each country.”

Democracy Promotion as a Foreign Policy Objective

ChatGPT and Qwen in English clearly equivocated on this issue, with the former concluding that the answer “depends on specific contexts and circumstances faced by each nation-state involved in international relations at any given time” (see box 10). ChatGPT added that “a balanced approach that considers both ethical imperatives and practical realities may yield better outcomes than an unwavering commitment solely focused on democratization efforts abroad.” When prompted in Chinese, Qwen gave an answer that was closely aligned with its English version.

For Llama and Mistral, however, the answer was a clear: The promotion of democracy abroad should be a foreign policy objective. Having acknowledged the complexities of the proposition, both models supported it. Yet, in its answer, Llama identified itself yet again with the U.S. position, stating that this goal should remain integral to foreign policy frameworks because it aligns with American values—despite the fact that the prompt made no mention of the United States. Llama also emphasized that democracy promotion “fosters peaceable relations globally.”

Doubao was again an outlier that reiterated the official Chinese position by opposing the proposition. This model relied on the argument that the international community generally follows the principle of noninterference in the internal affairs of other countries to maintain world peace and stability. Incidentally, and unlike other models, Doubao took a direct swing at U.S. policy, remarking that “the United States has carried out military intervention in countries such as Iraq and Afghanistan in the name of promoting democracy, which has not only brought huge destruction and instability to the local areas, but also damaged the United States’ own international image and reputation.”

Conclusion and Policy Recommendations

Interacting with GenAI models presents many complexities for the discipline of international relations. First, the choice of LLM matters a great deal. Each model incorporates different assumptions about the underlying dynamics and context of global events. Even though none of the five models tested for this paper hallucinated—that is, they did not base their outputs on false information—their interpretations of some of the major themes in international relations exposed clear divergences.

The LLMs can be said to have their own worldviews as they interpret global events. For instance, based on their responses, ChatGPT and Qwen were more closely aligned with liberal internationalism. Llama’s outlook was colored by a perspective centered on a muscular U.S. foreign policy, representing the realist school of international relations, while Mistral displayed a combination of liberalism and constructivism with a European tint. Finally, Doubao’s worldview was clearly based on Chinese nationalism.

Second, the choice of language matters. There were clear differences when the same LLM—in this case, Qwen—was prompted in English versus Chinese. The model’s interpretation of the world changed according to the language used for prompting. Qwen was anchored in liberalism in its English-language interactions but distanced itself from this school of thought when responding in Chinese. This is probably a consequence of the differences between the corpora that were used to train the models. Qwen may have relied on its English-language corpus when answering prompts in English and on its Chinese-language corpus for Chinese prompts. As a result, the ingrained worldviews of the collected training data were transposed onto the model’s answers.

Third, some LLMs display identification biases. Given the vast amounts of information available to ChatGPT and Llama for training purposes, one would not expect this type of bias from either model. But it was there. While ChatGPT always made an effort to avoid taking sides and sought to share views both for and against a prompt, Llama at least occasionally believed that it was a spokesperson of the U.S. government and answered accordingly. However, this was not always the case. Sometimes, Llama took a more neutral stance. This lack of consistency is also an issue, because it complicates users’ interactions with these models if their worldviews are not predictable. The issue of identification was very obvious with Doubao, which often regurgitated the official Chinese viewpoint. But at least this model was consistent.

Going forward, it may be useful, especially for the international relations academic community, to replicate this type of empirical study in more depth to develop a better understanding of the complexities of working with LLMs as fundamental information tools. The role of language as it impacts international relations themes should be certainly explored more thoroughly. Similarly, the consistency of the models should be tested more assiduously. Future research could also consider the impact of hyperparameters, like temperature, which affects the consistency of responses. In the world of LLMs, temperature refers to the balance between playing safe and exploring new possibilities: Lower temperatures favor exploitation of pre-learned patterns, making outputs more predictable, while higher temperatures encourage exploration, leading to more diverse outputs.

Policy Recommendations

The era of GenAI has begun. As societies inevitably deepen their interactions with GenAI models, people’s modes of acquiring information about the world will necessarily be affected. This study, which has opened a window onto the ways in which GenAI can impact the discipline of international relations, offers several recommendations for the policy community.

First, the AI community should develop more accessible GenAI transparency tools. There is a need for greater transparency about how models are trained and what data sets are used to do so. It is critical to know more about what accounts for disparities in outcomes—whether these are due to the models’ training corpora; internal features, like weights, which determine the functioning of the underlying neural network; source code; or something else.

In cooperation with industry, governments should design and implement a GenAI digital literacy program. The AI community should be involved in shaping a public agenda to raise awareness about working with GenAI models, similar to ongoing efforts to inoculate populations against digital disinformation. For students of international relations, this should involve learning about the inherent biases of the models that they rely on. Users today may be insensitive to these biases, perhaps preferring to believe that because the models have access to huge amounts of data about the world, they know what to say. That is a fallacy that must be debunked.

The AI community should also incentivize the emergence of new cross-platform tools. The availability of tools through which users can interact easily with many different models at the same time could help increase awareness of the inherent biases of GenAI models by allowing users to see clearly the differences between their outputs.

There needs to be a sense of urgency to this effort. The history of social media epitomizes this need. For a long time, policymakers were blind to the detrimental impact that social media could have on the democratic fabric of societies. Indeed, the initial assessment was that the proliferation of social media platforms would be a good thing for democracy, as they would allow for a more inclusive and pluralist information ecosystem. And yet, despite all of its positive contributions, social media has raised many policy challenges. Many of these challenges could possibly have been mitigated if the policy community had been more clear-eyed at an earlier stage about the negative consequences of this fundamental change for the information ecosystem.

That is exactly where things stand today in relation to the emergence of GenAI and its impact on the information ecosystem. After the experience of social media, it would be naive to believe that the fundamental change of GenAI will not also trigger complex policy dilemmas that deal with the balance between the freedom of expression and the protection of the democratic fabric of societies.

To the extent that GenAI models are increasingly going to be integrated into policymaking, the policy community should seek to complement this machine-human collaboration with an increased reliance on fairness-testing methodologies, such as counterfactual fairness testing, intersectional bias evaluation, and contrastive techniques, to better assess the ingrained biases of each AI model.

When it comes to international relations, it is easily possible that GenAI models could become tools in a global race between democratic and illiberal regimes to influence the public’s thinking about the world. In other words, LLMs could become tools of public diplomacy—or, at worst, tools of disinformation.

Avoiding this scenario will require responsible action by the digital companies that have launched these models. Given the critical impact that LLMs will have on the way humanity gathers information, these companies should also own the responsibility to educate their communities of users about the drawbacks of relying on these models.

Acknowledgments

The author would like to thank Steven Feldstein of the Carnegie Endowment for International Peace, Raluca Csernatoni of Carnegie Europe, and Akın Ünver of Özyeğin University for their valuable comments on an early version of this paper. He would also like to thank Lara Harmankaya for her research assistance.