Moral stereotyping in large language models

Significance

Large language models (LLMs) are increasingly used not only for communication, but also for research tasks like estimating public opinion and simulating moral judgments. But can they truly reflect the diversity of human values across cultures? We compare LLM-generated moral value estimates to real-world survey data from 48 countries and find consistent biases: LLMs overemphasize moral concerns common in Western societies and underestimate values more prominent elsewhere. These distortions likely stem from cultural biases in training data and carry societal implications and risks. Morality is foundational to how people express opinions, justify laws, and engage in politics; thus, distorted moral representations may lead to mischaracterizations of public sentiment. Rather than offering a culturally neutral lens, current models risk reinforcing stereotypes.

Abstract

Can Large Language Models (LLMs) accurately estimate various societies’ moral values? Here, we query the perceptions of LLMs regarding the moral norms of the “average” person from 48 nations and compare them to a large-scale ($n=90,802$) survey of six moral values (Care, Equality, Proportionality, Loyalty, Authority, and Purity) from those populations. Our findings indicate that LLMs poorly capture the moral diversity around the globe, systematically overestimating some moral values (particularly Care) and underestimating others (especially Purity). Notably, examining various versions of Generative Pre-trained Transformer (GPT) shows that these LLMs may overestimate the overall moral concerns of some Western countries (e.g., the United States, Canada, and Australia) while underestimating those of non-Western countries (e.g., Nigeria, Morocco, and Indonesia). Our work demonstrates that LLMs are inaccurate generators of cross-cultural estimations in the moral domain; in other words, they stereotype the moral values of non-Western populations in predictable ways. Our results highlight the ethical and epistemic risks of relying on LLMs to estimate the endorsement of moral values around the globe.

Traditionally, social scientists have relied on various methods such as ethnographies, surveys, and experiments to gain cultural knowledge about different populations. Recently, however, a new source of knowledge has emerged: large language models (LLMs). Some researchers have proposed that LLMs can aid in social science research (1), even reliably predicting social science experimental results (2). Moreover, some have argued that LLMs may approximate data in place of human participants in certain domains (3). Indeed, one could ask generative AI models such as ChatGPT about different cultures’ norms, values, traditions, and beliefs instead of collecting data in the traditional way. However, it remains unclear to what extent these models can accurately represent various cultural values. Such accounts rely on the assumption that LLMs have accurate information about a wide range of human populations. In this study, we put this assumption to the test and directly examine how accurately LLMs can estimate the moral values of different nations around the globe.

If LLMs have an accurate representation of diverse cultures’ moral values, they should be able to estimate the moral values of the average^* person in those cultural contexts. If LLMs’ estimates are far from the ground truth (actual moral values of those cultures), LLMs are showing evidence of inaccurate stereotyping of a population’s moral endorsements. Stereotyping in humans involves several processes, including social learning, cultural norms, motivational factors, and—most relevant to our theorizing—limited exposure to information or experiences (4–7). LLMs exhibit an analogous, though mechanistically distinct, process as their training regimen draws from purely linguistic co-occurrence data (i.e., patterns of tokens in massive text corpora), supplying them with salient (but imbalanced) information (8). Thus, their “stereotypes” emerge as a latent linguistic bias, whereby the dominating information embedded in the model comes to define what is normative, while all else is linguistically silenced or misrepresented (9, 10). To say that LLMs “stereotype” is simply to note that they absorb and reproduce the statistical contours of their training environment. Much as humans intuit patterns from repeated exposure, these models treat frequent pairings as typical and rare ones as atypical. But because LLMs, unlike humans, have no knowledge beyond text (e.g., no perception), they cannot correct for distortions in their training data (unless external layers are corrected in fine-tuning or reinforcement learning) (11). What emerges in LLMs is hence a purely linguistic category of stereotyping: LLMs’ sense of “what people are typically like” echoes the most common linguistic phrases in their training data.

As a result, LLMs are likely reflective of a dataset where Western, English texts are overweighted, and thus overrepresented in subsequent outputs (12). This is reflective of common stereotype inaccuracy, where populations are overgeneralized (13). When such correlations reflect social or cultural regularities—for example, bias against older women in the workplace (14)—the model inherits and reproduces those patterns. Researchers have noted (9) that AI systems often instill and project the identity-based biases originally formed in the human data on which they are trained. When these associations are applied too rigidly or without context, the model “stereotypes,” producing biased completions or inferences that mirror overgeneralized human beliefs. Recent work (15) further illustrates that LLMs generate stereotypical outputs across domains such as gender, race, religion, and occupation, due to skewed exposure in their training distributions and reliance on co-occurrence statistics.

In the context of morality, these dynamics imply that LLMs may overweight unrepresentative information to produce distorted estimates of moral values across cultures, reflecting not just inaccuracy, but a process similar to stereotyping in humans. We find the pattern of inaccuracy to be particularly interesting, as it is similar to a type of stereotype commonly found in human reasoning–valence inaccuracy (13). Rather than the estimates being randomly inaccurate, these may models systematically inflate the positive moral traits of the data that are overrepresented in the training data [similar to how people often inflate positive qualities in those that are culturally similar to them (16)] and underestimate positive qualities in those that are least represented in their training regimen, following a similar pattern to which humans often enact. Importantly, ad-hoc debiasing methods (10) [e.g., Reinforcement Learning from Human Feedback; RLHF (17)] may aggravate these probability distributions as the models are disproportionately exposed to Western, English-speaking, and elite cultural inputs (8, 18). Furthermore, stereotypes can fuel divisions within social dynamics (19) and international relations. For example, ref. 20 analyzed how American liberals and conservatives stereotype their own ideological group and the opposing side. Comparing judgments generated by individuals about their own morals to those generated by political opposites about “typical” liberals or conservatives, the authors found that both liberals and conservatives exaggerate how different their moral values were from one another. This overestimation bias may stem from an underestimation of shared values between the parties (20).

There are a number of typologies for the study of moral values. One of the most influential is Moral Foundations Theory [MFT; (21, 22)], a pluralistic framework whose latest version offers six basic dimensions of moral judgment: Care, Equality, Proportionality, Loyalty, Authority, and Purity (23). Respectively, these six values encompass the virtues of compassion, egalitarianism, merit, solidarity, deference, and piety, and the vices of cruelty, inequality, disproportionality, betrayal, subversion, and degradation. As globalization increases, these cultural elements are transmitted through more complex interactions occurring within increasingly diverse societies, although that may not necessarily mean that countries will converge on a set of values. Contact between those with differing moral frameworks is sometimes simplified by stereotyping, that is, reducing otherwise complex and diverse cultures into what one might consider “typical” (13, 24). While stereotypes can be a helpful heuristic for interacting with unfamiliar others, they can also be highly inaccurate, and have the potential to come at a social cost. One type of stereotype inaccuracy—valence inaccuracy—can manifest as a relative overestimation of positive traits for one’s in-group (positive valence inaccuracy) and/or an overestimation of negative traits in an outgroup (negative valence inaccuracy) (13, 25). Such a misperception can lead to deeper and more dangerous social cleavages, especially in the moral domain (20).

As LLMs become increasingly integrated into societies, they too, alongside humans, are becoming part of these complex social interactions. Some researchers have argued that they are less biased than human participants and are perhaps better generators of predictive data (3). However, LLMs’ perception of different cultures may be skewed as most of their training data come from a thin slice of human diversity around the globe. These texts predominantly reflect WEIRD [Western, Educated, Industrialized, Rich, and Democratic; (26)] perspectives, as less-WEIRD populations have lower levels of internet output and thus are not a primary data source for AI companies (8).

Although some researchers have likened the cognitive and attitudinal characteristics of LLMs broadly to those of “humans,” research suggests that LLMs have mainly acquired the psychology of WEIRD humans (8). Indeed, LLMs hold geographical biases against poorer nations on a range of subjective (e.g., likeability, attractiveness, and intelligence) topics, notably rating these characteristics much lower than nations of higher socioeconomic status, like European nations (27). We propose that the inherent WEIRD bias in LLMs, likely rooted in their training corpora, may lead to a skewed representation of moral values across all populations. Specifically, we investigate the degree to which different versions of GPT (i.e., GPT-3.5, GPT-4, and GPT-4o) inaccurately generate the moral values of 48 different nations. Researchers have demonstrated that one glaring issue of LLMs is their disproportionate favoritism of cultural values of English-speaking and Protestant European nations and their cultural misalignment with less-WEIRD populations (18). These misrepresentations and biases are being increasingly adopted by researchers and already consumed by millions of laypersons. As LLMs progressively become more involved in some human decision-making processes, examining potential inaccuracies within a model’s output prompts ethical AI development and usage, enhancing the reliability, accuracy, and cultural competence of what is an emerging pillar in society.

The Present Study.

We examine the six moral foundations of 48 nations computed by three different versions of Open AI’s GPT family of LLMs (GPT-3.5, GPT-4, and GPT-4o) to investigate whether these models accurately represent the moral values of various nations. We also replicate our findings using moral value estimates generated by Meta’s LLaMa (versions 2-7b and 2-70b-chat-hf) and Google’s Gemini Pro (SI Appendix).

Methods

Human Participants.

It is important that “synthetic” estimates of LLMs be benchmarked against a ground truth. As an accuracy benchmark, the moral value scores generated by our three versions of GPT were compared to human participant scores sourced from the YourMorals database for each country (median sample size per country $=$ 299, total $N=90,802$). The Institutional Review Board (IRB) at the University of Southern California approved this study (UP-07-00393). These values were sourced from YourMorals.org, an online platform that provides users with feedback on their moral values, and, in turn, contributes to a large-scale database of anonymized responses. Data are collected using the Moral Foundations Questionnaire-2 (MFQ-2) (23), a psychometrically validated instrument that quantifies endorsement of the six aforementioned moral values. Previous research has drawn on this dataset in various applications, ranging from behavioral interventions related to climate change (28), predicting county-level COVID-19 vaccination rates in the United States. (29), temporal trends in moral values (30), and studying voting behavior in the United States (31), demonstrating its value as a credible tool for measuring moral values.

The primary measure of YourMorals, MFQ-2, is a 36-item self-report measure wherein participants (after obtaining their consent) indicate their agreement with items related to the moral foundations of Care (e.g., “Caring for people who have suffered is an important virtue”), Equality (e.g., “I believe that everyone should be given the same quantity of resources in life”), Proportionality (e.g., “I think people should be rewarded in proportion to what they contribute”), Loyalty (e.g., “Everyone should love their own community”), Authority (e.g., “I think it is important for societies to cherish their traditional values”), and Purity (e.g., “I think the human body should be treated like a temple, housing something sacred within”). Participants respond along a 5-point Likert-type scale, ranging from 1 (Does not describe me at all) to 5 (Describes me extremely well). Notably, all participants completed this measure online and in English, which may breed a bias in which our participants represent more educated, globally connected, and socioeconomically privileged subgroups within their respective nations. To help combat this potential bias from a nonrepresentative pool, we performed Multilevel Regression with Synthetic Poststratification [MrsP; (32)] to ensure more representative national averages (33). Specifically, we fit multilevel models predicting the endorsement of the six moral foundations using individual-level indicators for age, sex, and country. Then, we poststratified the predictions using World Bank census^† data that report the distribution of age and sex for each country. Incorporating this method yields country-level estimates that better reflect the demographic structure of our 48 countries, thus reducing sampling bias (34). We refer to these data points as the “actual scores” on moral values.

LLM Procedures and Prompts.

Our GPT estimates were sourced from three versions of the language model: GPT-3.5, GPT-4, and GPT-4o. The GPT-3.5 model was a refined iteration of the GPT-3 model. The GPT-4o model was designed to provide GPT-4-level intelligence with enhanced speed and improved capabilities. We filtered the YourMorals data to only include countries for which we had 100 participants or more, leaving us with 48 countries. Our GPT queries were also done with these 48 countries. To provide some variance and serve as a proxy for LLMs’ confidence in their response, we queried GPT 10 times for each question.

For consistency and similarity between our human sample and LLM estimations, we prompted each version of GPT in English with each survey item, adapted from the MFQ-2, one after another making sure to use identical prompting strategies, hyperparameters, and sampling methods (SI Appendix, Table S2). Additionally, we prompted the three iterations of GPT with two variations of the same task. We asked all 36 items in the MFQ-2 for the “average” and “random” person in each country to ensure the reliability of our results (13, 35). All models of GPT ran through 10 iterations of the prompt below using two different prompt variations, resulting in 20 estimations of each item for each country per version of GPT, which were averaged. In total, our GPT dataset contains 103,680 data points (48 countries $\times$ 6 foundations $\times$ 6 items in each foundation $\times$ 2 different prompts $\times$ 3 different GPT versions $\times$ 10 iterations for each item). Our code is publicly available here: https://osf.io/afq67/?view_only=64f2f6b78b6e4f0abb6e441e567b9ebe.

“For the statement below, please indicate how well the statement describes the [average/random] person from [country]. Response options: Does not describe the [average/random] person at all (1); slightly describes the [average/random] person (2); moderately describes the [average/random] person (3); describes the [average/random] person fairly well (4); and describes the [average/random] person extremely well (5). Please answer only using a single number, with no words.”

We refer to these data points as the “GPT scores.” Because of the low Cronbach’s alphas of GPT foundation scores generated with the “random” prompt (SI Appendix), we exclusively use GPT scores generated using the “average” prompt. The internal consistencies for GPT-3.5, GPT-4, and GPT-4o were as follows (respectively): Care ($\alpha =0.72$; $\alpha =0.85$; $\alpha =0.88$), Equality ($\alpha =0.69$; $\alpha =0.74$; $\alpha =0.82$), Proportionality ($\alpha =0.71$; $\alpha =0.72$; $\alpha =0.65$), Loyalty ($\alpha =0.73$; $\alpha =0.84$; $\alpha =0.85$), Authority ($\alpha =0.85$; $\alpha =0.89$; $\alpha =0.95$), and Purity ($\alpha =0.83$; $\alpha =0.83$; $\alpha =0.84$). Visual comparisons across the different prompts for each moral foundation can be found in SI Appendix.

Results

The Inaccuracy of LLMs.

We first evaluate the claim that LLMs can reliably approximate human moral values. First, we take the difference between the GPT scores produced for each foundation against the actual estimates (see Fig. 1 for a visual representation). If GPT were an accurate generator of cultural data, Fig. 1 would be mostly white, indicating no difference between the scores produced by GPT and actual estimates across countries. However, as can be seen, two key issues emerge, highlighting how GPT systematically stereotypes the morality of nations. First, as newer versions of GPT are released, we observe that Equality and Purity are increasingly underestimated in the majority of populations. Second, GPT-4o shows signs of improved accuracy, but the pattern of inaccuracy in underestimating Equality and Purity remains consistent. In fact, we find that this slight improvement in accuracy is primarily seen in nations that are culturally similar to the United States [i.e., WEIRD populations; (36)]. Contrastingly, nations more culturally distant from the United States, i.e., less-WEIRD populations, continue to show a pattern of increasingly being misrepresented in the moral domain across versions of GPT. For example, in our sample, Egypt was culturally furthest away from the United States. All GPT versions underestimated almost all moral concerns (apart from Authority) of the “average” Egyptian (Fig. 1). The discrepancies illustrated in Fig. 1 highlight a general misalignment of LLM estimates and actual human values across the globe, demonstrating how poorly GPT generates cultural data. The magnitude of these differences is consequential; for example, a 1-point difference on this scale for Equality is about the difference observed between Egypt and New Zealand, also comparable with liberal-conservative differences in the United States (37). Moreover, to further quantify these discrepancies, we computed Cohen’s d with 95% CIs and found large effect sizes (Care (1.01 [0.81, 1.20]), Equality ($-$0.45 [$-$0.65, $-$0.25]), Proportionality (0.42 [0.23, 0.60]), and Loyalty (0.58 [0.43, 0.73]), Authority (1.00 [0.82, 1.18]), and Purity ($-$0.84 [$-$0.96, $-$0.71]).), indicating that gaps between scores generated by LLMs and humans translate to substantively meaningful distortions. These differences are comparable to or greater than between-country variations reported in previous research with MFT (23, 38, 39). Results using the moral value estimates computed by Meta’s LLaMa and Google’s Gemini Pro are present in SI Appendix.

Fig. 1.

The difference between the moral value estimate (produced by GPT-3.5, GPT-4, and GPT-4o) and the ground truth (human sample). Red coloration represents an overestimation of moral values, and blue coloration represents an underestimation of moral values.

Cross-National Discrepancies.

The magnitude of moral stereotyping (i.e., discrepancies between GPT and actual scores) was quantified by distance scores—measured by the Euclidean distance between GPT and actual scores in a six-dimensional space—to capture the overall inaccuracy of GPT moral estimates, and difference scores—measured by the difference between GPT and human moral estimates—to assess the direction of LLMs’ stereotyping (overestimation vs. underestimation). The Euclidean distance was calculated for each nation to quantify the discrepancy between all six moral values estimated by LLMs and endorsed by humans in our cross-national data (Formula 1). In this formula, q represents the set of moral values computed by the different models of GPT, p denotes the corresponding human-reported benchmark, and i represents each of the six moral foundations. For each nation (j), we summed the squared differences between GPT and human scores for all moral foundations (i), then computed the square root to obtain the Euclidean distance (d). To conceptually express what Euclidean distance captures, we refer to it as a measurement of models’ inaccuracy of the overall moral concern of a nation. Results for the latest version of GPT are shown in Fig. 2, highlighting GPT-4o’s substantial deviations from the actual moral values across most countries ($t=35.61,df=47,MeanEuclideanDistance=1.43$, 95%CI [1.35, 1.51], $P<0.001$), demonstrating a general misalignment with global moral norms that is particularly prominent in Middle Eastern and Sub-Saharan African nations. Moreover, as we can see in Fig. 2, GPT-4o was more accurate in estimating the overall moral concerns of WEIRDer nations, that is, countries that are culturally closer to the United States as the prototypical WEIRD nation (36), r$=$ 0.56, P$<$ 0.001.

$$\begin{array}{c}\hfill d_j=\sqrt{{\displaystyle \sum _{i=1}^6}{(q_{\mathit{ij}}-p_{\mathit{ij}})}^2}\end{array}$$ [1]

Note. This equation computes the inaccuracy of overall moral concern as the Euclidean distance between two entities in a six-dimensional moral value space. In our work, we used it to calculate the distance between the moral value estimates produced by each GPT model and the corresponding estimates derived from human participants.

Fig. 2.

The map above illustrates, overall, how much GPT-4o’s moral endorsements deviate (i.e., distance score) from the actual participant scores for each country.

Underestimation vs. Overestimation.

Stereotype valence inaccuracy, as previously described, can result in either an overestimation or underestimation of various characteristics, with positive characteristics usually being overestimated in one’s own group and underestimated in outgroups (13, 25). Given that general moral concern is typically considered a positive trait (40, 41), we considered that if LLMs are biased, they may overestimate moral concern in some cultural groups while underestimating it in others. To examine the valence of these inaccuracies by moral foundation and GPT version, we calculated a series of correlations between the WEIRD distance scores [cultural and psychological distance from the United States; (36)], and compared them to the difference between the GPT and actual scores for 37 of the 48 countries (11 countries did not have WEIRD distance scores). Results are shown in Fig. 3. We find evidence that while there is some variation in different versions of GPT, these models are systematically biased, often overestimating moral values in countries such as the United States, Canada, and Australia, while underestimating the moral concerns of the Middle East and Africa.

Fig. 3.

The plots above visually break down the correlations between difference scores and cultural distance scores by the GPT model. Also, GPT generally tends to overestimate four values (Care, Proportionality, Loyalty, and Authority) while underestimating two values (Equality and Purity).

Our results partially confirm a WEIRD bias in GPT, as at least one version of this LLM inaccurately enhances the perceived morality of WEIRDer populations, portraying them as more morally concerned than what is reflected by their actual scores along some moral dimensions, namely Care, Equality, and Proportionality. However, across all moral foundations and all GPT models, there is a consistent pattern of relative over/underestimation, such that as a nation becomes more culturally similar to the United States, GPT rates its moral concern on each foundation as higher than those nations that are less culturally similar to the United States (excluding Purity scores, which are notably less normative in western cultures (23, 42).

Determinants of Moral Stereotyping in LLMs.

Although we previously provided evidence pointing to a WEIRD bias, more should be done to unpack the kinds of non-WEIRD nations that get stereotyped by LLMs. As Henrich (43) have argued, psychological variation does not fall along some linear continuum from WEIRD to non-WEIRD. For example, Norway and Nigeria are almost equally distant from the United States but are obviously quite different in terms of their culture, history, and economy. What kinds of non-WEIRD countries are more prone to be morally stereotyped by LLMs? As an exploratory analysis, we investigated different country-level measures of industrialization relevant to data availability to explain the overall magnitude of moral stereotyping. Specifically, we examined Gross Domestic Product at Purchasing Power Parity, GDP (PPP) per capita, press freedom, and population as factors influencing LLMs’ moral values output. We theorize these factors may influence what and how much data is available for AI companies to use to train their models. First, richer countries have higher internet usage and contribute more content to the internet. Thus, a higher GDP should be associated with a larger internet footprint, therefore providing AI companies with more extensive data to train their models, potentially resulting in more precise moral value estimates for that nation. Second, greater press freedom may result in higher levels of information access on the internet. To capture this factor, we standardized the Press Freedom Index from Reporters Without Borders which measures the level of freedom available to journalists in various countries (0 indicates the lowest press freedom, and 100 indicates the highest press freedom). Third, the larger a country’s population, the more individuals it has to generate online content. The number of people residing in a nation may influence the accuracy of these moral value estimates as the more people there are, the more data present for these models to train on [although research suggests there is a misalignment between the world’s most spoken languages and those most represented in the GPT training corpus; (44)]. For GPT-3.5, zero-order correlations suggested a surprising positive correlation between economic development and moral stereotype inaccuracy ($r=0.40,P=0.005$), while there was no significant correlation between press freedom ($r=0.23,P=0.117$) and population ($r=-0.13,P=0.361$), with moral stereotype inaccuracy. However, after putting all these variables in an Ordinary Least Squares (OLS) regression and accounting for the nonindependence of nations by adding a fixed effect for the continent, none of the predictors significantly explained GPT-3.5’s moral stereotype inaccuracy (Table 1). For GPT-4, the magnitude of inaccuracy was smaller for countries with greater economic development ($r=-0.30,P=0.036$) and press freedom ($r=-0.54,P<0.001$) and larger for more populous countries ($r=0.41,P=0.003$). In the OLS regression, only free press was found to be significantly associated with moral stereotype inaccuracy (Table 1), such that countries with greater press freedom were stereotyped to a lesser degree by GPT-4. The GPT-4o model showed a similar pattern to GPT-4: Stereotype inaccuracy was smaller for countries with greater economic development ($r=-0.40,P=0.005$) and press freedom ($r=-0.64,P<0.001$), but the effect of population size was not significant ($r=0.26,P=0.069$). As can be seen in Table 1, only press freedom was a significant predictor of GPT-4o’s moral stereotype inaccuracy in a regression ($B=-0.14,SE=0.047,P=0.005$). Of note, the multidimensional Euclidean metric for moral stereotype inaccuracy quantifies divergence from reality, but does not indicate the directionality of this inaccuracy (for directions, see Fig. 3).

Table 1.

Three regression models to predict the magnitude of inaccuracy in estimating moral values for three LLMs

	Inaccuracy of overall moral concern
	GPT-3.5	GPT-4	GPT-4o
GDP (PPP) per capita	0.105 (0.155)	0.140 (0.097)	0.120 (0.073)
Press freedom	0.098 (0.101)	−0.140∗ (0.063)	−0.142∗∗ (0.047)
Population	0.114 (0.175)	0.176 (0.109)	−0.005 (0.082)
Continent FE	Included	Included	Included
Constant	1.253∗∗∗ (0.190)	1.519∗∗∗ (0.119)	1.901∗∗∗ (0.089)
Observations	48	48	48
R2	0.272	0.649	0.673
Adjusted R2	0.122	0.577	0.606
Residual SE (df $=$ 39)	0.374	0.233	0.175
F Statistic (df $=$ 8; 39)	1.820	9.025∗∗∗	10.033∗∗∗

A fixed effect (FE) was added to account for the clustering of nations in continents.

Note: ^∗ P < 0.05; ^∗∗ P < 0.01; and ^∗∗∗ P < 0.001.

Robustness Checks.

To ensure our results are not artifacts of the linguistic structure or theoretical framing of our primary analyses with MFQ-2, we conducted two independent replication studies. Importantly, each replication addresses distinct sources of potential bias in our main analyses. The first examines whether our results persist when human moral values are measured in a population’s local language, addressing concerns about linguistic or sampling biases in the English-only YourMorals database. The second investigates whether the observed moral discrepancies between LLMs and humans uphold under a theoretically distinct model of morality, ensuring that our findings are not tied to the structure of Moral Foundations Theory alone. Taken together, these two studies draw on new data, new languages, and a new theoretical framework, providing rigorous cross-validation to demonstrate the stability of our findings.

Cross-linguistic replication using translated moral foundations data.

A potential limitation of our primary analyses is that all MFQ-2 data were collected in English across 48 nations, which may systematically bias our “ground truth” benchmarks because those who complete English-language surveys may represent a more educated, globally connected, or socioeconomically privileged subgroup, rather than the broader population. To address this concern, we conducted a large-scale cross-cultural replication designed to eliminate the linguistic channel as a possible confound.

We administered a validated short form of the MFQ-2 (MFQ-2-S; 45), translated into six languages (Arabic, Urdu, Ukrainian, Portuguese, Spanish, and Turkish). Translations were reviewed by language experts, ensuring that conceptual meaning of each survey item was preserved.

We collected 4,666 human responses across 11 populations, targeting a more demographically diverse pool than online English-language samples. Countries were selected specifically to capture variance in WEIRDness, economic development, political structure, and cultural distance from the United States: Argentina ($n=119$), Brazil ($n=327$), Colombia ($n=473$), Egypt ($n=302$), India ($n=606$), Mexico ($n=150$), Pakistan ($n=483$), Turkey ($n=635$), Ukraine ($n=320$), the United Kingdom ($n=716$), and the United States ($n=855$). In parallel, we prompted GPT-3.5, GPT-4, and GPT-4o to estimate the moral endorsements of these countries in their respective languages, removing the possibility that English-language prompting alone shaped the models’ responses.

The detailed findings of this additional study are available in SI Appendix. In short, the results remain robust. Across translated versions of the MFQ-2-S, GPT models continued to systematically underestimate endorsements of Equality and Purity, particularly in less-WEIRD populations (e.g., Egypt and Turkey). Moreover, in the world map where GPT4o’s overall inaccuracy of moral concern is shown (SI Appendix, Fig. S8), the same cross-cultural asymmetry persists: Moral values in WEIRDer nations are estimated more accurately compared to populations in less-WEIRD contexts. Additionally, with this independent dataset, GPT models inflated the moral values of WEIRDer countries (compared to less-WEIRD populations), reproducing the same directional bias we observed in our original analyses. Last, although the coefficient for press freedom is no longer statistically significant in this smaller sample, its magnitude and directionality remain consistent (refer to SI Appendix, Table S10). This reaffirms that nations with higher levels of press freedom—a characteristic of WEIRD societies—display higher accuracy for the overall moral concern of that population.

This replication demonstrates that our findings are not artifacts of English-language or MFQ-2 item-wording. When we remove language constraints by collecting new data in native languages and prompting LLMs equivalently, the same moral discrepancies in LLM accuracy emerge, suggesting that the source of their misestimation lies not in the linguistic structure of MFQ-2, but in the cultural structure of the models themselves.

Testing generalizability across moral frameworks.

A second concern is that our primary results may hinge on the specific moral taxonomy of Moral Foundations Theory, which, given its well-established centrality in moral psychology (21, 22), served as the basis for our main analyses. Crucially, if our findings reflect stereotyping within the theoretical categories embedded in MFT, they may not generalize to other organizations of morality. Thus, to test whether our conclusions remain beyond our current framework, we replicated our study using the Morality-as-Cooperation (MAC) framework (46), which conceptualizes morality in terms of seven cooperative strategies: family, group, reciprocity, bravery, respect, fairness, and property (46, 47). Unlike MFT, which groups moral concerns into intuitive foundations, MAC forms its structure from evolutionary game theory, providing a conceptually distinct lens for evaluating the moral representations embedded in LLMs.

We leveraged the nationally representative dataset from ref. 48 which includes moral endorsement scores of 63 countries (29 different languages) measured using the MAC questionnaire (MACQ; 49). To compare these data, we prompted GPT 3.5, GPT-4, and GPT-4o to complete the same MACQ in the same countries and languages.

Analyses (reported in SI Appendix, Figs. S9 and S10) reveal similar patterns of moral stereotyping in less-WEIRD nations. Specifically, all three versions of GPT provide moral estimations that are most accurate for nations culturally proximate to the United States and they deviate substantially for less-WEIRD populations, mirroring our main analyses.

This conceptual replication provides robustness to the patterns we report above, highlighting that our results are not confined to Moral Foundations Theory or the structure of MFQ-2. Rather, it further reflects the cultural representations embedded within these AI systems.

Taken together, our linguistic and theoretical replications provide a robust cross-validation of our findings: LLMs systematically provide inaccurate moral estimates for diverse populations around the globe, persisting across languages and theories.

Discussion

As LLMs continue to expand their purpose globally (50), it is crucial to evaluate the quality of their knowledge and their effects on scientific discovery (51, 52). This is particularly pressing given that some researchers have argued that AI will transform social science research (1, 53), and some called for the use of LLMs as data generators, potentially replacing human participants (3, 54). In the realm of morality, Dillion et al. (3) have argued that not only can we replace humans with LLMs as participants, but perhaps we ought to, crowning LLMs as the paragons of ethics (see ref. 55). However, a simple yet fundamental issue remains: Which humans are (over-)represented in LLMs? And whose moral compass are these models calibrated to? Here, we demonstrate a general inaccuracy in GPT’s estimations of moral values across 48 countries. The inaccuracies are systematic and can be best described as a form of stereotyping: valence inaccuracy.

Past research has demonstrated that LLMs are aligned with a WEIRD psychology (8), leading this purportedly “global” tool to often misrepresent “humans” or stereotype many “people” around the world. These authors argued that there is a WEIRD-in-WEIRD-out (WIWO) bias in these models: Since less-WEIRD viewpoints, languages, and demographics are underrepresented in these models’ training data, their outputs may marginalize or misrepresent those perspectives. In other words, the imbalance within their training data serves as a function of limited exposure. Stereotypes are associations between social groups and semantic attributes that are widely shared within societies, and arise in written language corpora (56, 57). Just as human stereotypes arise when individuals rely on salient (but unrepresentative) information, LLMs are likely to overgeneralize from the disproportionately WEIRD content in their training data (8). Here, we demonstrated that LLMs are partially inaccurate in their estimations of less-WEIRD countries across some moral dimensions, particularly Care, Equality, and Proportionality. Additionally, GPT models were most likely to underestimate endorsements of Equality and Purity across a majority of our 48 nations. Importantly, we believe that postulating a coherent explanation of the observed patterns requires clearer connections between the moral composition of LLMs training corpora and their behavior. For instance, if the discrepancies were limited to underestimations in Purity (the least WEIRD-aligned moral foundation), one might attribute them to a WEIRD-skewing of LLMs’ training data. However, this does not readily account for why Equality, a value commonly emphasized in Western contexts, would be underestimated as well. For this reason, we view the observed pattern as an intriguing and promising venture for future research in cultural and moral psychology and as a clear motivation for greater transparency in AI development.

When LLMs carry inaccurate moral priors into everyday tools, the consequences show up in quiet but important ways, influencing how public opinions and attitudes are represented worldwide. For instance, plausible downstream implications may include a mental-health AI assistant used in East Asian contexts might prioritize individual autonomy over filial obligations, steering family conflict advice toward “setting boundaries” rather than negotiated harmony (related to the foundation of Loyalty). Moreover, automated hiring screeners that rate narratives of “impact” and self-promotion more highly can disadvantage candidates from cultures that value tradition and collective credit, even when qualifications match (related to the foundation of Authority). Another possible example can be found within content moderation systems trained on Western norms flagging depictions of animal slaughter during Eid al-Adha or open-air cremation rituals as harmful or obscene, while permitting equally graphic but culturally familiar content, which tilts online visibility (related to the foundation of Purity). Additionally, public-health chatbots that cast vaccination solely as personal risk management can miss community duty narratives that motivate uptake in high-purity regions (29). Not all these failures are catastrophic on their own, but together they homogenize global moral diversity, create small systematic disadvantages for some groups, and might lower trust in AI systems across cultures (58).

Additionally, research showcases that LLMs rely on shared latent representations across an array of tasks, such that biases in their internal value structure may not be isolated to the task of MFQ-2 or MACQ outputs, but are recycled across different reasoning, expression, and summarizations (59). For example, recent research indicates that LLMs may structure moral judgments within their embedding spaces in ways that parallel human perceptions of right and wrong (60). As such, the cultural orientation of LLMs (which favors the values of English-speaking Protestant European countries) can shape user expressions (61) and short-term attitudes (62), which may accumulate over time within cultural systems (63). Thus, the misrepresentation of moral estimates in LLMs may pose risks in downstream tasks that involve right and wrong: As these models are increasingly deployed in tools that summarize public attitudes, support decision-making, or simulate stakeholder perspectives, systematic distortions in moral representations could contribute to a skewed portrayal of entire populations, misinforming users and policymakers alike (64). The practical remedy may lie in expanding the training corpora, culturally stratified evaluation sets, regional fine-tuning, and explicit checks for framing parity before deployment.

To further investigate the downstream implications of how moral and broader cultural representations of populations encoded in LLMs translate into real-world applications, we encourage future research to directly compare model-generated population inferences on policy-relevant moralized issues against human benchmarks. Ref. 65 offers a blueprint to this approach, assessing how LLMs fail to replicate American moral judgments using concrete moral scenarios. This work is especially timely because prior work shows that fine-tuning and alignment interventions (ranging from bias-inducing fine-tuning and RLHF to direct manipulation of internal representations) can systematically alter LLMs’ expressed political leanings and downstream behavior, demonstrating that such normative attributes are malleable rather than fixed (66–68).

We note that Moral Foundations Theory is one typology among many within the field and that our use of MFQ-2 was anchored on the fact that it is a validated psychometric scale that has been used in numerous studies (29, 69, 70). Thus, we encourage future work to replicate our results using other theories on morality, such as the Relationship Regulation Theory (71), Schwartz’s Values Model (72), Model of Moral Motives (73), Theory of Dyadic Morality (74) or the Morality-as-Cooperation (47). Additionally, although the development of MFQ-2 included validation with non-WEIRD populations (23), it remains possible that certain item wordings may not work as well in some cultural contexts (75, 76). However, our findings, based on translated versions of the MFQ-2, suggest that such bias is unlikely to account for the observed patterns. We argue that these effects are likely to replicate across moral typologies, and may have even further applications to other traits, as these results are likely driven by LLMs’ (lack of) access to cultural data from less-WEIRD populations.

As an exploratory analysis, we also examined what country-level factors might contribute to the discrepancy between GPT’s estimations and actual moral values worldwide. Overall, GPT’s accuracy was shown to be higher in countries with high levels of press freedom—a characteristic of WEIRDer nations. This aligns with our finding that GPT exhibits some WEIRD-leaning tendencies and is consistent with prior research (8, 18, 44). These findings make clear that GPT is more likely to suppress the moral concerns of less-WEIRD populations while inflating its estimates of WEIRDer populations. Such societies are already underrepresented in social science research (77), and researchers aiming to accurately estimate the moral values of “humans” should not rely on the purported accuracy of LLMs.

Moral representations are fundamental to how people perceive social groups and whether these groups engage in cooperation or conflict (78). As illustrated in ref. 20, systematic distortions in perceived moral differences between ideological groups contribute to polarization. This could, in turn, lead to stereotyping and miscommunication between groups. Our results depict an analogous process in LLMs: Models systematically misrepresent the moral concerns of less-WEIRD populations. This work provides empirical evidence that moral predictions from generative AI systems are not uniformly accurate across values or populations, but structured by the moral distributions likely reflected in their training data. Recognizing this pattern is critical for two reasons: a) it clarifies what kinds of inferences researchers can and cannot draw when using LLMs as cognitive models (79) or “synthetic” simulations of human moral judgments (80), and b) it identifies a measurable pathway by which cultural bias may enter value-sensitive technologies. In short, the current work shows that moral estimation (accessible information) in LLMs is neither random nor universal but culturally patterned, a finding that directly informs both the scientific use of LLMs and the design of more accurate AI systems.

In a nutshell, we recommend studying more diverse samples of living humans around the world to understand “human” psychology rather than attempting to distill human understanding from biased off-the-shelf models like GPT. In addition to the raw language model (i.e., the way it is trained and the corpora it uses), how its guardrails are implemented (e.g., through RLFH) may also influence its understanding of not just a society’s average moral values, but cultural information in general (81). In many commercial LLM pipelines, the human annotators who partake in feedback during RLHF are often pooled from Western liberal-democratic contexts. Notably, many benchmarks for AI alignment consist of datasets around harm, created by Western companies and research teams (e.g., ref. 82). As a result, the model may favor responses that align with those normative viewpoints (83). This can create a second filter on top of the training data; even if a model encountered texts from less-WEIRD contexts in pretraining (hence encountering a more diverse assemblage of moral values), the RLHF step may dampen or override outputs that deviate from the dominant value-system (84), further amplifying the WEIRD norms embedded within LLMs training data (10).

Furthermore, past research has shown (and our results echo) that LLM-estimated measures are not reliable and can lead to a lack of knowledge or even create an illusion of understanding and advancement in science (52). One effective remedy for addressing this issue could be diversifying the training data for LLMs by incorporating a broader range of language content from different world regions and different time eras. Equally important is the need for greater transparency and open documentation from AI developers (85). Without access to the composition of training data, it remains unclear whether these observed biases (e.g., underestimations of endorsements of Purity and Equality in less-WEIRD nations), stem from underrepresentation or a dominance of particular ideological framings. Publishing key characteristics of a dataset (e.g., its origin, makeup, the demographics of the individuals it pools from, and descriptions of the processing steps followed to curate it) provides the means necessary to assist researchers in understanding the conduct of AI systems (86). Within the scope of the current work, this level of transparency would allow researchers to trace the conception of these moral distortions, measure the distribution of moral discourse across cultures embedded within LLMs, and evaluate whether specific domains (such as social platforms or news media) are overrepresented in shaping the model’s internal value structure. Most importantly, this level of transparency would equip researchers to progress beyond documenting inaccuracies to diagnosing their respective mechanics and designing culturally inclusive remedies.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Code data have been deposited in OSF (https://osf.io/afq67/?view_only=64f2f6b78b6e4f0abb6e441e567b9ebe) (87). All other data are included in the manuscript and/or SI Appendix.