Pressing questions in the study of psychological and behavioral diversity

Extreme biased sampling of research participants and the neglect of their cultural context are increasingly recognized as threats to the generalizability of much of what we know about human thought and behavior (1, 2). In addition to reinforcing narrow views of what it means to be human, these parochial research practices have also shaped the methodological core of the human sciences by favoring tasks that are tailored to the skills, motivations, and social expectations of a very rarefied set of humanity (3). Despite decades of calls for reform, there is little evidence that increasing awareness of this threat has led to changes in practice or publishing (2, 4).

This special issue stems from the Sackler Colloquium on “Pressing Questions in the Study of Psychological and Behavioral Diversity” (September 7–9, 2017) designed to address these issues. Building on discussions emerging from two National Science Foundation-funded workshops held in 2016 and 2017, the colloquium convened scholars from a wide range of disciplines who have conducted important research with diverse populations. The colloquium participants identified barriers that discourage researchers from harnessing the full breadth of human diversity for understanding human thought and behavior. These included concerns about current peer review practices, methodological narrowness, lack of researcher diversity, and the need for theoretical frameworks that are sensitive to cultural, social, and ecological variation. Participants also shared innovations and insights from their own research that can foster and inform future investigations of human diversity.

In this introduction, we highlight three threads that weave together the 10 papers in this special issue. The first thread documents the theoretical and practical payoffs of engaging with a broader range of participants (and the pitfalls of failing to do so). The second thread identifies barriers to pursuing such work and proposes potential solutions to overcome them. The third thread raises important questions about appropriate ways to conduct robust research with diverse populations. Although many articles often touch on more than a single issue, we introduce them in the special issue based on their affinity with each of these different threads.

The first set of papers illustrates theoretical corrections and insights that can emerge when researchers expand their view of humanity. For example, Majid et al. (5) challenge the long-held assumptions in Western thought that there is a universal hierarchy of senses by showing that speakers of 20 diverse worldwide languages encode a range of colors, shapes, sounds, textures, tastes, and smells in very different ways. While Majid et al. leverage massive worldwide variation to make their argument, other papers illustrate striking cultural differences that can arise within the same locality. In the western United States, Alcalá et al. (6) document how pairs of children of either indigenous-heritage Mexican immigrants or of middle-class European American families worked together to devise efficient routes through a model store. The Mexican-heritage children tended to fluidly collaborate together in a sophisticated way seldom observed among middle-class European American children. This work points to the importance of careful observation in diverse communities to identify skills—in this case, fluidly collaborating together—that may be rare and unnoticed in default samples.

On the other side of the world in rural north India, Brooks et al. (7) show that low-caste respondents quickly learn to cooperate in repeated economic games, making them look a lot like both United States undergraduates and the kinds of rational actors assumed by many economists. However, in those same north Indian villages, a propensity for retaliation at perceived slights among high-caste respondents makes it very difficult for them to do the same. Thus, even a single village can represent a microcosm of culturally distinct patterns of behavior. While each of these studies is interesting in its own right, together they illustrate a recurring theme in the colloquium. A researcher who relies on just one of these groups to develop and vet a theory of human psychology would have a challenge determining what is basic, fundamental, or universal and what is rather particular to the cultural and social context in which it is being studied. Indeed, van Leeuwen et al. (8) demonstrate that this problem is not unique to humans by documenting reliably different patterns of sociality in four adjacent chimpanzee groups experiencing similar socioecological conditions.

The second set of papers shines a light on institutional disincentives and default assumptions that reinforce the status quo and discourage researchers from engaging with a broader range of humanity. Salari Rad et al. (9) focus on peer review and academic publishing by demonstrating continued bias toward English-speaking, European, and educated samples in one of the behavioral sciences’ leading journals, Psychological Science. The authors also document frequent omissions of key details about study samples and their culture context, a practice that has actually worsened with recent increases in samples recruited through the internet. As a potential remedy, Salari Rad et al. propose a number of guidelines for authors, reviewers, and editors aimed at improving attention to cultural context in conducting studies, reporting findings, and deciding which papers to accept for publication.

Brady et al. (10) focus on how the common tendency to overlook cultural context can lead to inaccurate generalizations, incorrectly viewing differences as deficiencies, dismissing non-WEIRD (Western, educated, industrialized, rich, and democratic) samples as outliers, and implementing interventions that hurt more than they help. As one solution to these myriad problems, Brady et al. encourage scholars to cultivate what they call “interpretive power”: the ability to understand and value individuals’ experiences and behavior in their cultural context. Using the specific case of attachment theory, Keller (11) argues that the universal application of a theory without regard to cultural differences can lead to unethical policies that unfairly and incorrectly diagnose deficiencies in child and parent behavior. Echoing a recurring call from many of the colloquium papers, Keller argues for the importance of embedded ethnography and first-hand knowledge for understanding the local rationale and norms underlying social interaction.

The third set of papers points to key tasks and challenges facing research communities that hope to build a robust and generalizable human science. Gurven (12) highlights the importance of developing theories that explain how widely varying social, cultural, and ecological contexts shape human psychology and behavior. Examining the failure of five-factor differentiation of personality to generalize to small-scale subsistence societies, he develops a theory of personality differentiation based on socio-ecological complexity that not only organizes the anomalies, but also provides a principled account for why we would expect five instead of two factors in WEIRD populations. More broadly, Gurven discusses how such theoretical frameworks are important for generating new questions and hypotheses, organizing emerging observations, and pointing researchers to the kinds of populations that would most fruitfully advance our understanding of human variation.

Turning to methodologies used by researchers to collect data from their fellow humans, Hruschka et al. (13) propose that attending to methodological “failures” can reveal researchers’ tacit (and often incorrect) assumptions about what diverse participants bring to any research protocol and what they consider relevant. The authors argue that systematic study of methodological failures and successful adaptations will help us go beyond our own intuitions and better understand the range of skills, motivations, and social expectations that respondents bring with them. Finally, Nzinga et al. (14) challenge the assumption that objectivity requires a distanced, uninvolved stance as the best way to reduce bias in research with communities, arguing instead that close engagement with communities also has distinct advantages. In addition, the authors document how this institutionalized assumption discourages nondominant researchers from studying their own communities, thereby reducing an important source of information about nondefault populations. To spur productive debate about this assumption, Nzinga et al. place distanced and close engagement on a level playing field and outline candidate risks and benefits associated each of these approaches.

Faced with continued, unreflective reliance on a narrow slice of humanity to inform the human sciences, the colloquium papers and the discussions inspired by them nonetheless give reason for hope. In addition to revealing the promise of reaching out to broader populations, the papers also chart pathways forward for building a more robust social and behavioral science. If recent editorial statements at prominent psychology journals are an indication of future commitments, then there is also reason to believe that these insights are beginning to guide decisions at key points in the scientific process (15, 16).

One hallmark of good science is the recognition that humans hold myriad biases and false beliefs that obscure their view of the world. This realization has led to the cultivation of institutions and modes of inquiry that help us identify and overcome our biases and in turn build models of the world that let us predict and change our futures in unprecedented ways. But the fight against bias is never complete and is bolstered by sustained reflection on how we generate knowledge. Indeed, Salari Rad et al. (9) propose that it may be useful to learn more about the lay beliefs that perpetuate unreflective reliance on narrow samples and neglect of cultural context.

Toward that end, we conclude by outlining candidate problematic lay beliefs of researchers and research institutions that were discussed at the colloquium and in the papers (Table 1). Some of these are still open for debate, while other beliefs already have strong evidence against them. For each of them, we suggest a necessary shift in scientific assumptions and practices in the case that the belief is wrong. These lay beliefs and their implications for how we do science deserve careful attention as we work toward building a robust science of Homo sapiens.

Table 1.

Problematic lay beliefs and necessary shifts in scientific assumptions and practices in the case that beliefs are wrong

Domain	Problematic lay belief	Necessary shift
Sampling	A researcher can identify what is fundamental about human cognition and behavior by studying a single dominant culture.	Devote a greater share of resources to studying a range of diverse, nondominant populations and cultures.
	Online respondents and college students in other countries are sufficient to understand the full breadth of human diversity.	Expand research to include participants with little or no formal schooling and contexts beyond convenience samples and the internet.
Methods	The existing tools of the social and behavioral sciences are transparent windows on human thought and behavior.	Study the assumptions underlying our methods with diverse populations and test our interpretation of the observed data with local knowledge.
	We know enough about the range of human functioning to focus future research on testing hypotheses derived from theories and models based on WEIRD populations.	Place in-depth formative research in diverse populations and settings on a level playing field with testing of existing hypotheses to understand the full range of human capacities and behaviors.
Normativity	The psychology and behavior of WEIRD children and adults is a benchmark for what is good, right, and normal anywhere.	Examine what is adaptive or normal about modes of thinking and behavior in different cultural contexts and avoid normative language when describing capacities and behaviors. Do not use thought and behavior in one culture as the normative benchmark for others.
Researchers and institutions	Studying one’s “own people” introduces bias unless the researcher comes from a presumably neutral (dominant) WEIRD background.	Recognize the value of both insider and outsider observations and perspectives, anywhere.
	Any effects of researcher diversity on scientific practice and knowledge are removed by the scientific method and standards of objectivity. There is no advantage to having life experience outside of the default culture.	Recognize the insights available from life experience in more than one cultural system, as can be the case for researchers from nondominant cultural communities. Build pathways that promote researcher diversity.
	With enough awareness of the problem, individual researchers have the personal resources to solve it.	Build institutions of training, peer review, and promotion that incentivize efforts to improve generalizability and attention to cultural context.
Generalizability	Current values guiding the review of social and behavioral research (e.g., internal validity, control, direct replicability…) are sufficient for a robust human science.	Place diversity, generalizability, and external validity squarely in the pantheon of criteria used to judge grants and research.
	The crisis of reproducibility can be solved by direct replications without a parallel commitment to tests of generalizability.	Allocate commensurate effort to assessing generalizability across populations and contexts as is currently allocated to direct replication. Build institutions of peer review and promotion that place equal value on generalizability of findings.
Theory	There is a general dominant pattern of human psychology and behavior, and exploring cultural variability merely exposes exceptions and refinements to that pattern.	Build theories of thought and behavior that explicitly include social, cultural, and ecological processes and that explicitly account for variation across distinct cultural settings.
	The sole purpose of cultural research is to document differences (and similarities) between groups. Eventually, they will make sense if researchers collect enough of them.	Build theories that can generate novel hypotheses about social, cultural, and ecological processes. These theories can inform the choice of appropriate populations and methods in future research.