Replications provide mixed evidence that inequality moderates the association between income and generosity

Schmukle et al. (1) conducted informative conceptual replications of our finding that economic inequality moderates the relationship between income and generosity (2). Schmukle et al. (1) did not find that inequality moderates associations between income and self-reported charitable donations (study 1), first- and second-mover trust game allocations (study 2), and self-reported volunteering (study 3). Their findings contribute to a larger literature documenting patterns both consistent (e.g., ref. 3) and inconsistent (e.g., ref. 4) with our theoretical claim.

Note that Schmukle et al.’s (1) conceptual replications differ operationally from ours. First, their generosity measures differed: We used a secondary dataset assessing generosity with a “dictator game” in which participants gave between 0 and 10 raffle tickets to another participant. The dictator game minimizes self-serving motivations to act generously (e.g., reputational motivations) or to portray oneself favorably on questionnaires, which limit the validity of other generosity measures (5). Although dictator game giving often correlates with the generosity measures that Schmukle et al. (1) used, observed correlations are zero to moderate (rs = 0.08 to 0.35 with first mover and rs = 0.00 to 0.49 with second-mover allocations in the trust game; rs = 0.03 to 0.40 with charitable donations; r = 0.05 with volunteering) (references and data are provided in supplementary information for this letter at https://dataverse.harvard.edu/loginpage.xhtml?redirectPage=%2Fdataset.xhtml%3FpersistentId%3Ddoi%3A10.7910%2FDVN%2FIDJAWT). Second, just one of Schmukle et al.’s (1) three studies was conducted entirely in the same country as ours. Cultures differ in reactions to inequality, particularly the United States versus Europe (6).

Conceptual replications address the validity of general theoretical claims but provide limited insight into the reproducibility of a particular study (7). Therefore, we conducted closer replications with all known datasets that include income, state-level inequality, and dictator game giving. New datasets comprised US participants from our laboratory (8, 9), including a new preregistered study, and from another laboratory (10). This research was approved by the Social Science, Humanities, and Education research ethics board at the University of Toronto. Participants provided informed consent. Results (table S2 in our supplementary information) are mixed. The interaction between income and inequality predicting dictator-game giving achieved P < 0.05 in two of five new analyses. The pattern of the interaction was the same as the original—a positive relationship between income and generosity in the least unequal states and a negative relationship in the most unequal states—in four of five analyses (figure S1 in our supplementary information). Meta-analysis of these studies and the original (2) reveals a significant interaction for household income, B = −0.55, SE = 0.24, P = 0.02, but not for personal income, B = −0.38, SE = 0.30, P = 0.21.

Given the new results, we are less confident that state-level inequality moderates the association between income and generosity assessed with the dictator game. Future research can illuminate if the interaction between inequality and income predicting dictator game giving is a false positive, or if it has a small effect size only reliably detected in large samples, or if it manifests only in certain conditions. Moreover, Schmukle et al.’s (1) results—and the nonsignificant interactions between income and inequality predicting charitable donations or volunteering in our preregistered study (table S3 in our supplementary information)—suggest that our original theoretical claim may have been overly broad. If an interaction truly exists, it may only obtain with the dictator game and other similar measures of generosity, and in certain cultural contexts.