Poorer representation of minds underpins less accurate mental state inference for out-groups

Bryony Payne; Geoffrey Bird; Caroline Catmur

doi:10.1038/s41598-024-67311-3

. 2024 Aug 28;14:19432. doi: 10.1038/s41598-024-67311-3

Poorer representation of minds underpins less accurate mental state inference for out-groups

Bryony Payne ^1,^✉, Geoffrey Bird ^2,³, Caroline Catmur ¹

PMCID: PMC11349750 PMID: 39191807

Abstract

Societies are becoming more polarised, driven in part by misconceptions about out-groups’ beliefs. To understand these effects, one must examine the cognitive processes underlying how people think about others. Here, we investigate whether people are less prone to theorise about the minds of out-groups, or less able to do so. Participants (Study 1: n = 128; Study 2: n = 128) made inferences about social and political beliefs held by real in-group and out-group members, and could choose to receive further information to improve these inferences. Results show: (1) participants sought equivalent or greater information about out-groups relative to in-groups; but despite this, (2) made significantly less accurate inferences for out-groups; and (3) were significantly less aware of their reduced ability. This shows that poorer mental state inference is not underpinned by a reduced propensity to consider out-group minds, but instead by a worse representation of the minds of out-groups.

Keywords: Mentalising, Theory of mind, Mental state inference, Out-groups, Group membership

Subject terms: Psychology, Human behaviour

Introduction

Societies are becoming increasingly divided, in part due to the growing political polarization that is evident in many large democracies¹. This polarization may lead to increased hatred², as individuals become more emotionally negative about those with opposing views. However, this polarisation is fuelled by widely shared misconceptions about the views of out-group members^3,4. For instance, Americans who support the Republican party overestimated the extremity of Democrat views on taxes, abortion, gay rights, and racial policy⁵. Thus, predictions about the beliefs and attitudes of outgroups are frequently inaccurate, and this inaccuracy negatively impacts relations between members of different groups.

To understand why people have such inaccurate estimates of out-group beliefs, it is critical to examine the cognitive processes underlying “mentalising” or “theory of mind”⁶—the process by which we attribute mental states to others. Although research suggests that individuals' mentalising ability differs depending on whether they are considering the mind of an in-group or out-group member, none has been able to explain why. This is, in part, because previous studies have not disentangled the multiple underlying cognitive processes that give rise to mentalising and, further, which of these processes differ—and how—when considering different minds. For example, some studies aim to measure an individual’s mentalizing ability, by measuring their accuracy in inferring the mental states of others. Perez-Zapata et al.⁷ tested Australian participants when mentalizing about either Chilean participants or other Australians. The authors claimed that participants' inferences were less accurate and slower for the Chilean out-group than for the Australian in-group, an effect which replicated in other samples^8,9. These studies therefore suggest that people have poorer theory of mind abilities when thinking about out-group members relative to in-group members.

However, these studies—aiming to measure mentalising ability by assessing the accuracy of mental state inference—all suffer from common problems. First, they lack an objective ground truth to determine mental state inference accuracy (see Long et al.¹⁰). Studies frequently use fictional material as stimuli, requiring participants to infer the mental states of fictional characters who do not have them, and thus there is no true objective correct answer. Instead, the ’correct’ answer is determined by the creator of the stimulus material, or a group of raters, often from a single group (culture, age, neurotype, etc.). The lack of a true correct answer makes determining the accuracy of mental state inference impossible, and is especially problematic for the question of determining the accuracy of mental state inferences for in- and out-groups. This is because the assumption of one correct answer may well itself be incorrect; it may be the case that individuals of different ages, cultures, neurotypes, truly do have different mental states in the same situation.

Equally problematic is the fact that several studies have used tests of questionable validity, and which are inherently unsuitable for in- and out-group comparisons for other reasons. For example, several studies have used the “Reading the Mind in the Eyes” Task¹¹ (RMET) to assess theory of mind ability for in- and out-groups⁸. However, Oakley et al.¹² have previously demonstrated that the RMET does not measure theory of mind, but the ability to recognise emotion from facial expressions, while Higgins et al.¹³ showed it has poor psychometric properties. Further, studies demonstrate reliable differences between cultures¹⁴ and neurotypes¹⁵ in the expression of emotion, meaning that if culture or neurotype was the basis of the in- and out-group distinction, then the assumption that there is one correct answer for each stimulus in the RMET is invalid.

The third problem is that studies often conflate mentalising ability with propensity. That is, despite aiming to assess whether mental state inference is ‘better’ or ‘worse’ for in-groups and out-groups, several studies do so by measuring the extent to which individuals attribute mental states to others. Therefore, rather than mentalising ability, such studies are actually measuring the propensity to mentalise. For example, McClung and Reicher¹⁶ asked participants to converse with either an in-group or an out-group partner (based on an arbitrary minimal group assignment) and then provide a written description of their conversation partner. The authors found that individuals who spoke with an out-group partner referred to fewer and less complex mental states when describing them. Similarly, McLoughlin and Over¹⁷ found that 5- and 6-year-old children used fewer, less diverse mental states to describe the actions of interacting geometric shapes¹⁸, when those shapes represented out-group members—children from a different geographic region or of a different gender—compared to in-group members. Both studies assess people’s propensity to make mental state inferences about others, regardless of the accuracy of those inferences. It remains unclear, therefore, whether (1) people’s inferences about the mental states of out-group members are truly less accurate than their inferences about the mental states of in-group members, and (2) whether any such inaccuracy is actually due to reduced propensity.

In principle, any deficits for out-groups may be overcome if the mentalizer is aware of their group biases. Participants who are aware that they have reduced accuracy when making mental state inferences for out-groups can make fewer inferences, and instead directly ask out-group members about their mental states. In time, this feedback may lead to more accurate inferences about out-group members’ mental states. At present, however, the degree of awareness individuals have with respect to their group biases in mental state inference is unknown.

The current studies

Here we aimed to answer three questions: First, is it the case that people are less accurate when inferring mental states of an out-group mind when performance is assessed against an objective ground-truth and with real out-groups? Second, is it differences in ability to infer mental states of out-groups or the propensity to do so that underpin any differences in out-group accuracy? Third, if people are less able, or less prone, to think about the minds of out-groups, are they aware of this? Study 1 assesses all three questions; Study 2 replicates these findings in a novel sample.

General methods

Here we report the general methods for Studies 1 and 2. Information specific to each study (e.g., participants, exclusion criteria) are reported for Study 1 ("Pre-registered exclusion criteria" section) and Study 2 ("Pre-registered exclusion criteria" section) separately.

Open science statement

We conducted this study in accordance with open-science practices. Confirmatory analyses and exclusion criteria were pre-registered on the Open Science Framework, with the full pre-registrations viewable at: Study 1 https://osf.io/ek8hj, Study 2 https://osf.io/urvg3.

Participant recruitment

All participants were recruited online via Prolific (http://www.prolific.ac), self-reported as neurotypical, with English as their primary language, and the US as both their country of birth and current place of residence. Participants reported no significant visual impairments, mild cognitive impairments, or dementia. Finally, participants had to have joined Prolific before 2020 and have a current approval rate of over 90% to ensure data quality. All participants were tested online using Gorilla¹⁹. None had taken part in any pilot studies associated with this project and, upon completion of the study, were paid for their participation. Ethical approval was obtained from the local Health Faculties Research Ethics Subcommittee, (removed for peer review), all research was performed in accordance with the regulations of the Declaration of Helsinki and, as such, informed consent was obtained from all participants prior to testing.

Stimuli development

A novel stimulus set was created based on the Survey of Beliefs and Opinions²⁰, and responses to this survey constituted the ground-truth against which accuracy was assessed. The original survey included statements such as “We must ensure that women have access to legal abortions”, “I feel that people get what they deserve” and “Only adults who know how to read and write should be allowed to vote”. Thus, a person’s previous responses to these statements constitute their mental states, their propositional attitudes^21,22. In the current study, a subset of 168 statements was selected based on the correlation between previous responses across these statements in a sample of 703 American respondents²⁰. Specifically, we selected 24 subsets of 7 statements (see Supplemental Materials for full list of statements), where each subset contained an initial "starting" statement, a correlated "target" statement (r_M = 0.34, r_SD = 0.09), and 5 additional statements that were correlated to varying degrees with the target statement. Specifically, each set of 5 statements contained statements with correlation coefficients of approximately: 0.2 (M = 0.21, SD = 0.01); 0.1 (M = 0.11, SD = 0.004); 0 (M = 0.002, SD = 0.002); -0.1 (M = − 0.1, SD = 0.002) and; − 0.2 (M = -0.2, SD = 0.023). For each subset, we selected a “target-mind”—a previous respondent from the original survey²⁰—whose responses across these statements were set as the ground-truth against which new participants in the current study would be assessed in terms of accuracy at inferring their responses. To ensure this ground truth was not idiosyncratic, we selected target-minds whose responses across all statements in their subset was the modal response for all the previous respondents.

Procedure

Task 1: assessing group status

The first task was completed to determine participants’ group status in relation to each of the target-minds they would be asked to make mental state inferences about. Participants were presented with 24 “starting” statements and were asked to state their agreement/disagreement with the statement on a 5-point Likert scale. Across all the trials and tasks, the scale was as follows: 1 = “strongly disagree”, 2 = “slightly disagree”; 3 = “neither agree or disagree”; 4 = “slightly agree” and; 5 = “strongly agree”. This scale was chosen to broadly align with responses in the original sample²¹ from which these statements were selected. Note, however, that the original survey described the neutral response option as “Neither agree nor disagree: neutral, uncertain, or don’t understand the statement”. As such, none of these responses were included. In the current study, the participant’s response was compared to the target-mind’s response and the difference in agreement between the two was used to index group status (i.e., whether the target-mind was in-group or out-group with respect to the current participant). When the participant and target-mind had a difference in agreement score of 0 or 1 (i.e., the participant gave the same response to the target-mind or matched the target-mind on either agreeing or disagreeing with the statement, but with different strengths of conviction, e.g., slightly agree vs strongly agree) they were defined as in-group. Trials in which participants gave a neutral response (i.e., that they neither agreed nor disagreed) were not coded (see exclusion criteria). Finally, when the participant and target-mind had a difference in agreement score of either 2, 3, or 4, they were defined as out-group.

On each trial, participants were not explicitly made aware of this in- or out-grouping, thus any effect of group status is a consequence of the participant’s own perception of differences between themselves and the target-mind.

Task 2: predicting the views of in-group and out-group members

This task measured participants’ propensity to consider the target-mind, their accuracy in inferring the mental states of that mind, and how aware participants were of their own ability to make accurate inferences. This latter measure was operationalised as the relationship between actual inference accuracy and confidence in the accuracy of that inference. On each of 24 trials, participants were presented with a target-mind’s response to an initial “starting” statement. For instance: “Participant 437 [the target-mind] said that they strongly disagree that they believe in the superiority of their own gender.” Based on this information about one of their mental states, participants were asked to predict the target-mind’s response to a second target statement on a scale of 1–5, whereby 1 denoted “strongly disagreed” and 5 denoted “strongly agreed”. For instance, participants were asked how far they thought Participant 437 thought that ‘We should ensure that no one is denied a job due to prejudice’. No feedback was given. Participants were then asked how confident they were in their answer and could respond on a scale of 0–100, where 0 = “Not confident at all” and 100 = “Extremely confident”.

Next, participants were told that they could seek (and receive) up to 5 further pieces of information about the target-mind, where each piece of information was an additional statement and the target-mind’s response to it, which may help them make a more accurate inference about the target-mind’s mental state. Furthermore, they were informed that they could receive a bonus payment of up to £3, which was to be calculated according to both their accuracy (higher reward for higher accuracy) and the amount of information sought (lower reward for more information sought). Thus, participants were incentivised to seek as much information as they believed they required to make an accurate mental state inference, but no more.

This opportunity to seek further information was utilised as a measure of participants’ propensity to seek further information about a target-mind’s mental states. Once participants had selected how many pieces of information they wanted to receive (0–5 pieces), any additional information was revealed in a randomised but fixed order across participants.

After receiving any additional information, participants were asked to predict how the target-mind responded to the target statement for a second time, enabling the participant to update their original answer based on the new information. As before, participants were also asked to state how confident they felt in their updated answer. As before, no feedback was given. If participants opted to seek no additional information, they were still given the opportunity to update both their prediction and their confidence in the prediction. However, they were additionally asked to verify whether they had changed their prediction, despite having no further information and, if so, why. Participants could answer via a free-form text box and this measure was included as an attention check and informed the exclusion criteria below.

Task 3: AQ-28 and TAS-20

It has previously been demonstrated that autistic and/or alexithymic participants may perform differently on mentalising tasks relative to neurotypical individual^12,23. Therefore, all participants completed the Autism-Spectrum Quotient Test²⁴ (AQ-28), a 28-item questionnaire designed to measure the expression of autistic traits in an individual. Lastly, all participants completed the Toronto Alexithymia Scale²⁵ (TAS-20), a 20-item questionnaire designed to measure difficulty in identifying and describing emotions. These questionnaires were administered, despite recruiting only self-reported neurotypical participants, to further characterise the sample.

The study lasted approximately 35 min on average and participants were debriefed at the end.

Design

Both studies used a within-subjects design with 2 factors: group status, with two levels of in-group vs out-group; and timepoint, with two levels including before (timepoint 1) and after (timepoint 2) the opportunity to seek further information about the target-mind.

Measured variables

We measured three variables. First, we examined participants’ propensity to consider the minds of others and whether this was affected by their perceived group status. This propensity was operationalised as the percentage of available information about the target-mind (i.e., the additional statements and the target-mind’s responses to those statements) that was sought. Second, we measured the accuracy of participants’ mental state inferences both before (timepoint 1) and after (timepoint 2) any further information was sought. Accuracy was calculated as the percentage of correct responses, such that only cases where participants selected the target-mind’s exact response were coded as correct, while all other responses were coded as incorrect. Third, we measured participants’ awareness of their own ability to make accurate inference by measuring their confidence in their answer both before (timepoint 1) and after (timepoint 2) any further information was sought, in order to relate this to their accuracy.

Planned analysis

The same three pre-registered analyses were conducted in Study 1 and Study 2. First, we ran mixed effect models using lme4²⁶ to determine whether group status (in-group vs out-group) significantly modulated the percentage of available information participants sought about the target-mind’s mental states. The full model included the predictor of group status, and random intercepts of participant and, in Study 1 only, we additionally included TAS-20 scores as a random nested intercept (see "Pre-registered exclusion criteria" section). Statistical significance of the effect of group status was established via likelihood ratio tests by dropping the effect from a second null model and comparing the two models. Post-hoc comparisons were conducted in emmeans²⁷.

\begin{matrix} lmer (Percentage of informationsought \sim group status \\ + (1 | participant / TAS - 20 Scores), REML = FALSE) \end{matrix}

Second, to determine whether group status modulated participants’ accuracy scores and, further, modulated them differentially at the two prediction timepoints, we ran mixed effect models. The full model included two predictors, group status (in-group vs out-group) and prediction timepoint: before (timepoint 1) and after (timepoint 2) any further information was sought as well as the interaction between these two factors on participant accuracy.

glmer (Accuracy \sim group status * timepoint + group status + timepoint + (1 | participant), family = binomial)

Statistical significance of the interaction between group status and prediction timepoint was established via likelihood ratio tests by dropping the interaction from a second reduced model—which included only main effects of group status and timepoint and the random intercept of participant—and comparing the two models. Statistical significance of the main effects of group status and prediction timepoint was also established via likelihood ratio tests. Each predictor was dropped in turn from a reduced model which included only the other main effect and the random intercept of participant.

Third, to determine whether participants’ awareness of their own ability to make accurate mental state inferences differed as a function of group status: in-group vs out-group, we compared two correlations. Awareness was calculated according to how well participants’ confidence in their ability to predict others’ mental states correlated with their accuracy in doing so. Thus, we assessed whether each participant’s accuracy and confidence in their final prediction (i.e., at timepoint 2) were correlated—and differently so—for in-group minds and out-group minds. We ran point-biserial correlation tests on each participant’s accuracy and confidence scores on in-group, and separately on out-group, trials. Each set of coefficients was then r-to-z transformed and two one-sample t-tests were run to assess whether each significantly differed from zero. Finally, a paired samples t-test was run to assess the significance of the difference between the two sets of z-scores.

Study 1

Participants

In Study 1, we recruited a final sample of 128 participants (mean age = 46.82 years, SD = 12.51 years, age range = 24–78, 74 female and 54 male). This sample size was determined by an a priori G*Power calculation modelling a 2-way interaction between group status (in-group vs out-group) and prediction timepoint (before information vs after information sought) on accuracy scores, with participant as a covariate. The calculation determined that a final sample of 128 was necessary to obtain 0.80 power to detect a medium effect size of 0.25 at the standard 0.05 alpha error probability.

Pre-registered exclusion criteria

All exclusion criteria were pre-registered on the Open Science Framework (https://osf.io/ek8hj). In Study 1, we excluded and replaced whole datasets for participants who:

responded that they “neither agreed nor disagreed” towards more than 25% of the starting statements (10 participants). This was to ensure that participants could be classified as in-group or out-group to the target-mind on a sufficient number of trials;
had fewer than 6 in-group or out-group trials based on their responses to the starting statements (12 participants);
chose to seek no further information throughout the task (7 participants), as this suggested a lack of engagement;
updated their prediction between timepoints 1 and 2 in more than 25% of the trials but reported that they did not (5 participants). This was included as an attention check to ensure that participants were engaged in the task.

Finally, we pre-registered that whole datasets for participants who obtained scores indicating a high degree of autistic traits on the AQ-28 and scores indicating at least possible alexithymia on the TAS-20 would be excluded and their data replaced. However, despite recruiting only participants who self-reported as neurotypical and without autism, 45% of participants (n = 58) scored highly on the AQ-28 and 9 of these participants also obtained scores indicating possible Alexithymia on the TAS-20. As such, it was impractical to exclude and replace this unexpectedly high number of datasets so all were retained and participant scores on the AQ-28 and TAS-20 were controlled for in the subsequent analyses in which they were likely to have an effect (determined by exploratory analysis, see supplemental material).

Overall, in Study 1, whole datasets for a total of 34 participants were excluded and replaced until a final sample size of 128 was achieved. No participant received the bonus payment, which required accuracy over 75%.

Results

Descriptive statistics for Study 1, including the amount of information sought, as well as participants’ inference accuracy and confidence scores before (timepoint 1) and after (timepoint 2) additional information, are reported in Table 1.

Table 1.

Mean (SD) percentage of information sought, participant accuracy, confidence and awareness in Study 1.

Group status	Measure
Group status	Information sought (%)	Accuracy [timepoint 1] (%)	Accuracy [timepoint 2] (%)	Confidence [timepoint 1]	Confidence [timepoint 2]	Accuracy- confidence awareness (z-scores)
In-group	13.8 (12.6)	50.4 (13.3)	50.1 (14.2)	75.1 (11.9)	75.9 (12.1)	0.22 (0.29)
Out-group	15.3 (13.3)	39.1 (20.2)	40.5 (18.7)	72.3 (14.3)	72.3 (13.9)	− 0.10 (0.53)

Open in a new tab

Does perceived group status affect propensity to consider another’s mind?

The linear mixed model assessing whether the percentage of information participants sought about the target-mind’s mental states was modulated by group status showed that this main effect was significant (X2(1) = 4.465, p = 0.035), such that participants sought significantly more information for out-group minds than for in-group minds (see Fig. 1). However, the effect size was very small (Cohen's d = 0.08). When asked to infer the mental state of someone whom the participant perceived to be different from themselves—an out-group mind—they sought slightly more information about the out-group mind than for people they perceived to be more similar to themselves.

Mean percentage of available information sought for in-group minds vs out-group minds. Coloured segments show smoothed density curves for the full data distribution, while individual dots indicate mean percentage per participant. Horizontal bars show post-hoc comparisons with asterisks denoting significance as determined via likelihood ratio tests (see “Results”).

Does perceived group status affect the accuracy of mental state inferences?

The generalised linear mixed model assessing whether there was an interaction between the timepoint of the prediction: before (timepoint 1) and after (timepoint 2) further information was sought, and who the prediction was about: an in-group mind or out-group mind—on accuracy of mental state inferences was not significant. That is, participants’ accuracy did not change between timepoint 1 and timepoint 2 differentially, according to whether the target-mind was in-group or out-group to the participant. Further models were then run to assess the main effects of timepoint and, secondly, the main effect of group status. The main effect of timepoint was also not significant, showing that participants did not significantly improve the accuracy of their inferences after receiving more information about the target. However, the main effect of group status was significant (X2(1) = 48.85, p < 0.001), revealing that participants were significantly more accurate when their prediction was about an in-group target-mind relative to an out-group target-mind (see Fig. 2). This shows that participants had a significantly better ability to predict the beliefs of in-group members than out-group members, even though participants sought—and received—more information about out-group minds than in-group minds.

Mean accuracy of mental state inference for in-group minds vs out-group minds, averaged over timepoints 1 and 2 in Study 1. Coloured segments show smoothed density curves for the full data distribution, while individual dots indicate mean accuracy per participant. Horizontal bars show post-hoc comparisons with asterisks denoting significance as determined via likelihood ratio tests (see “Results”).

Does perceived group status affect awareness of mental state inference accuracy?

Finally, we assessed whether the correlation between participants’ inference accuracy and their confidence in their accuracy differed depending on whether they were inferring the mind of an in-group or an out-group member (see Fig. 3). A one-sample t-test revealed that confidence and accuracy were significantly, positively, correlated for in-group predictions (t(127) = 8.55, p < 0.001, Cohen’s d = 0.76). This suggests that participants’ confidence was significantly aligned with their accuracy and, therefore, that they were aware of their ability to predict the beliefs of in-group minds. In comparison, confidence and accuracy were significantly negatively correlated for out-group predictions (t(122) = − 2.22, p = 0.03, Cohen’s d = − 0.2), showing that higher confidence was associated with lower accuracy, and suggesting participants had relatively poor awareness of their own (in)ability to predict the beliefs of out-group minds. Finally, a paired samples t-test assessing the difference between the two sets of z-transformed coefficients was significant, t(122) = 5.86, p < 0.001, Cohen’s d = 0.78, showing that the relationship between people’s confidence in their ability to predict others’ mental states, and their accuracy in doing so, differed for in-groups and out-groups. This finding suggests that people’s confidence in their ability to predict the minds of out-group members is misplaced: people have poor awareness of their (in)ability to predict the minds of out-group members.

Awareness of mental state inference ability for in-group minds vs out-group minds in Study 1 at timepoint 2. The graph displays the distribution of Z-scores representing participants’ awareness of their mental state inference ability. The boxplots illustrate the spread of data, while the points show individual participant data. The dotted orange line represents 0, i.e., no correlation between accuracy and confidence.

Study 2

The stimuli used in Study 1 were designed to be 50% politically left-leaning and 50% right-leaning to ensure a sufficient number of in-group and out-group trials for each participant. However, the political leaning of the participants was not explicitly measured. In Study 2, therefore, we not only aimed to replicate Study 1’s effects but also to ensure that the effects hold when the sample was equally split by left and right-leaning participants (recruiting 50% of participants as being affiliated with the Republican Party, 50% affiliated with the Democratic party). Unless specified below, all other methods and analyses remain identical to Study 1.