Abstract
That racial/ethnic discrimination has adverse physical and psychological consequences, including stress, anxiety, depression, and their attendant health effects, is well documented. However, the particular dimensions within the broad construct of discrimination and their role in mental health are less well understood. This study investigates the dimensions of discrimination and explores their relation to depression and posttraumatic stress (PTS) symptoms. Using the Brief Perceived Ethnic/Racial Discrimination Questionnaire–Community Version, discrimination experiences were assessed among a community sample of African American and Latinx participants (N = 500). Factor analyses revealed 4 dimensions: Social Rejection, Stereotyping, Direct Threats/Attacks, and Police Mistreatment. In multivariate analyses, full regression models were significantly related to PTS and depression symptoms. Among the individual predictors, Social Rejection and ethnicity (Latinx) uniquely contributed to PTS symptoms in men, whereas Stereotyping and Direct Threat/Attack were associated with PTS symptoms for women. In regards to depressive symptoms, income, ethnicity (Latinx), and Social Rejection were significant predictors for men, while Social Rejection had an independent contribution for women. Thus, social rejection emerged as a significant unique predictor in 3 of the four models, suggesting that social ostracism may be a particularly harmful aspect of discrimination. Implications of these findings include the use of proactive and intervention strategies that emphasize a sense of belonging and mitigate the effects of exclusion and rejection.
Keywords: racial/ethnic discrimination, social rejection, stereotyping, depressive symptoms, posttraumatic stress
Discrimination, defined as a “behavioral manifestation of a negative attitude, judgment, or unfair treatment toward members of a group” (Pascoe & Richman, 2009, p. 533), has been consistently linked to adverse physical and mental health. Many individuals from minority groups report both subtle and overt stressful discriminatory experiences on the basis of their race or ethnicity as part of their daily lives (Sue et al., 2008; D. R. Williams & Mohammed, 2009; D. R. Williams, Neighbors, & Jackson, 2003). To date, a plethora of data suggests a significant inverse relationship between experiences of discrimination and mental health outcomes, with higher perceived discrimination being consistently associated with poorer mental health, including general psychological distress and symptoms of anxiety, depression, and posttraumatic stress (e.g., Brown et al., 2000; Jackson et al., 1996; Kessler, Mickelson, & Williams, 1999; Lewis, Cogburn, & Williams, 2015; Pascoe & Richman, 2009; Priest et al., 2013; D. R. Williams & Mohammed, 2009; D. R. Williams & Williams-Morris, 2000). Moreover, these associations have been found in diverse samples of African Americans, Latinos, and Asian Americans, among others (Lee & Ahn, 2012; Nadimpalli, James, Yu, Cothran, & Barnes, 2015; Torres & Vallejo, 2015). Everyday discrimination prospectively predicts blood pressure across 10 years (Moody et al., 2019), as well as inflammation across 7 years (Beatty, Matthews, Bromberger, & Brown, 2014), suggesting longer-term effects along multiple pathways.
Although it seems clear that perceived racial and ethnic discrimination in general influences mental health, it is less clear which specific aspects, if any, of this broad and multidimensional construct confer the most harm. Furthermore, the diverse ways in which discrimination has been construed and measured, as well as whether they are appropriate for the population in which they were used, render the question even more difficult to disentangle. Extensive reviews have been published on the measurement of discrimination and its many challenges (e.g., Kressin, Raymond, & Manze, 2008). Indeed, in a review by D. R. Williams and Mohammed (2009), the authors state, “There is no consensus on an optimal measure of perceived discrimination” (p. 6) due to various operationalization and measurement issues. A psychometric review of discrimination measures found that many were used despite a lack of factor-analytic evidence supporting their purported internal structure (Atkins, 2014). The purpose of our study is not to evaluate existing measures of discrimination but to explore the various dimensions within this construct and their relation to mental health symptoms. Whereas previous studies have attempted to elucidate this issue, there are still gaps in our knowledge as studies have construed and operationalized dimensions in diverse ways. For example, data from a prospective study with an older, ethnically diverse sample of women found that regardless of ethnicity, exposure over time to perceived interpersonal discrimination was negatively associated with risk for depression (Bécares & Zhang, 2018). This effect was observed among experiences of discrimination attributed to various personal identities (e.g., race/ethnicity, gender) and was found to be cumulative across identities as well as time. However, in this study, the domains of discrimination were operationalized by each of the 10 questions on the questionnaire (e.g., received poorer service; treated with less courtesy) rather than according to underlying conceptual similarities (Bécares & Zhang, 2018). Other studies have used measures that conceptualized discrimination in temporal terms, as either lifetime or day-to-day (Edwards, 2008), as well as domain, the context in which the discrimination experience occurred (e.g., educational, employment, and financial and other services). Yet others have used the place in which discrimination may occur (e.g., workplace) as one type of discrimination, as well as including the emotional consequences of the experience, for example, feeling depressed, angry, or strengthened (McNeilly et al., 1996), thus possibly conflating antecedents and outcomes. Moreover, some measures of discrimination have specifically referred to the experience of discrimination due to being African American (e.g., Landrine & Klonoff, 1996; McNeilly et al., 1996) and therefore cannot be used in multiethnic samples or compared across racial or ethnic groups.
The Brief Perceived Ethnic Discrimination Questionnaire–Community Version (BPEDQ-CV; Brondolo et al., 2005) was developed and validated to measure the type and place of discrimination experiences among all ethnic groups without dependence on a particular ethnic identification. The BPEDQ-CV has yielded four valid and reliable subscales among African American, Latinx, and Asian American samples—Social Exclusion, Stigmatization, Discrimination at Work/School, and Threat/Harassment—and has been found to be related to self-reported health symptoms (Brondolo et al., 2011). In particular, the social exclusion and threat/harassment subscales showed unique contributions to self-reported health when all four scales were considered simultaneously. In a multiethnic study by Contrada et al. (2001), an earlier version of the Perceived Ethnic Discrimination Questionnaire–Community Version (PEDQ-CV) was factor-analyzed, yielding four distinct factors: disevaluation, threat/aggression, verbal rejection, and avoidance. In regression models, the PEDQ as a whole was found to be positively related to depression and physical symptoms and inversely related to life satisfaction (Contrada et al., 2001). However, the individual factors were not evaluated in these models. Thus, while previous studies using the BPEDQ-CV have looked at distinct dimensions of discrimination and their relationships to physical health, their associations with mental health has not been fully explored to date.
The current study aims to address this gap and build on previous literature by exploring the distinct aspects of perceived racial/ethnic discrimination and their relation to mental health, specifically posttraumatic stress (PTS) and depressive symptoms. We chose the BPEDQ-CV to assess discrimination because it can be used with a variety of racial/ethnic backgrounds, has demonstrated good validity and reliability across multiple ethnic minority groups, and has dimensions that are more psychological in nature compared to other scales (Brondolo et al., 2005). While subscales of the BPEDQ-CV have been previously derived in samples of university students (e.g., Contrada et al., 2001) or in middle-class communities (e.g., Brondolo et al., 2005), it is unclear whether the factor structures found would be relevant in a marginalized sample such as ours. The issue of cross-sample validity was raised in a review of instruments measuring perceived racism and discrimination, which found that only 2 of 16 instruments met the criteria for acceptable factor-analytic techniques, calling into question the accepted factor structures for the samples used (Atkins, 2014). Because our sample was drawn exclusively from community-based health clinics and health fair events serving low-socioeconomic health-seeking patients in a section of Los Angeles that has a concentration of poverty, with a history of civil unrest and police brutality (Matsunaga, 2008; U.S. Census Bureau, 2018), it is possible that participants in this sample may experience discriminatory events differently or to different degrees than middle-class samples (e.g., more police mistreatment). In addition, the economic hardship experienced by our participants may increase risk for mental health problems (Fell & Hewstone, 2015; Viseu et al., 2018) and may exacerbate stress that stems from discrimination (Grollman, 2012). To that end, we evaluated the factor structure of the BPEDQ in our population rather than assume that the previously found structure applies and is relevant. Second, we examined the derived factors and their unique associations with PTS and depressive symptoms. Finally, we discuss the implications of the findings, including how mental health providers might tailor interventions to more effectively address the effects of racial/ethnic discrimination.
Method
Participants and Procedures
A multiethnic sample of 500 participants, including 230 African Americans (167 men and 63 women) and 270 Latinos (50 men and 220 women), participated in four parent studies that examined cumulative lifetime trauma and mental and physical health outcomes, supported by the National Institute of Mental Health–funded Center for Culture, Trauma, and Mental Health Disparities. These four parent samples were segregated by gender, with two studies consisting of male participants and two studies that included only female participants. Participants were recruited from community clinics and community-based organizations in South Los Angeles, an area with 25.6% of people living below the poverty line, which constitutes twice the median level in California (U.S. Census Bureau, 2018). In accordance with procedures approved by the Institutional Review Board at the University of California, Los Angeles (Protocol Number P50MH073453–0551, Developing a Composite Life Adversities Risk Index) and Charles Drew School of Science and Medicine, community clinics and social service organizations that serve primarily low-income African American and Latinx clients were approached to obtain their support in allowing the project teams to recruit at their agency. Both direct recruitment (approaching clients and patients) and staff referrals were used. All who expressed interest were prescreened according to inclusion and exclusion criteria by a trained multiethnic and multicultural team. Study-specific flyers and word-of-mouth referrals were used to supplement recruitment.
The study utilized a standard core battery of psychosocial measures administered to all participants in private sessions on laptop computers equipped with the Audio Computer-Assisted Self-Interview system. The assessment sessions were completed in 2–2.5 hr, and all participants received compensation of $35–$100, depending on time required to complete the study protocol. All participants received an information brochure and referrals to mental health and other services.
Measures
Demographic characteristics.
Self-reported demographic characteristics included age, race/ethnicity, household income, years of education, employment (full/part-time, not employed), and marital/relationship status. Household income was categorized as above or below $1,249/month, the federal poverty level for a family of four. For California, the poverty level for a family of four is $2,075/month to account for the high cost of living. While this single measure does not account for how many individuals are in the household, our sample has an average of three dependents. Therefore, the amount of $1,249/month may be considered a proxy for poverty level in our population.
Perceived experiences of discrimination.
Perceived experiences of discrimination because of one’s race/ethnicity were assessed with the 16-item BPEDQ-CV (Brondolo et al., 2005). This measure is a brief version of the PEDQ-CV and has 16 items as compared to 22 items. It has demonstrated good reliability and construct validity among African American, Latinx, Asian American, and White samples (Brondolo et al., 2005). As the brief version has demonstrated equally good reliability and validity as the longer version, we selected it to lessen participant burden. Respondents were asked to rate on a 5-point scale ranging from 1 (never) to 5 (very often) the frequency with which they experienced various forms of discrimination because of their ethnicity/race. Each question on the scale has the stem of “because of your race or ethnicity.” Items include “have others made you feel like an outsider who doesn’t fit in because of your dress, speech, or other characteristics,” “hinted that you are dishonest or can’t be trusted,” and “actually hurt you or tried to hurt you.” A confirmatory factor analysis (CFA) with oblique (oblimin) rotation was performed, using principal component analysis for extraction of factors. Factor selection was based on rationality, a clear curve in the scree plot, and interpretability of components. The reliability of the underlying factors/subscales was tested using Cronbach’s alpha. The lowest Cronbach’s alpha accepted was 0.84, indicating good reliability.
PTS symptoms.
PTS symptoms were assessed with the 17-item Posttraumatic Diagnostic Scale (PDS; Foa, Cashman, Jaycox, & Perry, 1997) that asks respondents to appraise the worst traumatic event experienced. Each item was rated on a 4-point Likert-type scale ranging from 0 (not at all or only one time) to 3 (five or more times a week or almost always). A very reliable (α = .92) sum score was calculated, with higher scores indicating more severe symptoms. The PDS has demonstrated strong reliability in African American samples and has been cross-culturally validated in a primarily African American sample (Powers, Gillihan, Rosenfield, Jerud, & Foa, 2012; M. T. Williams et al., 2014), as well as several in samples of African American, Latina, and White men and women (Glover et al., 2010; Liu et al., 2015).
Depression symptoms.
Depression symptoms were assessed with the 20-item Center for Epidemiological Studies–Depression Scale (CES-D; Radloff, 1977), a depression screener that asks participants to rate the frequency with which they experienced each symptom in the past 2 weeks on a 4-point Likert-type scale ranging from 0 (not at all) to 3 (every day or almost every day). A highly reliable sum score (α = .91) was calculated, with higher scores indicating more depressive symptoms. The CES-D has an established cutoff point of 16, indicating mild to moderate symptoms and a risk for depression (Radloff, 1977). It has demonstrated adequate screening accuracy and reliability in the general population and primary care settings and has shown adequate reliability and validity with ethnic minority samples (Kim, Decoster, Huang, & Chiriboga, 2011; Vilagut, Forero, Barbaglia, & Alonso, 2016), including African Americans and Latinos (Liu et al., 2015; Sciolla et al., 2011).
Analyses
In accordance with our conceptual framework, CFA using SAS software was conducted to examine the factor structure of the BPEDQ-CV within this population. Each of the four factors as derived by CFA served as predictor variables, along with the covariates of gender, income, and race/ethnicity. PTSD and depressive symptoms served as outcome variables. Bivariate analyses were conducted to examine the relationships among the predictor and outcomes variables. Multiple linear regression was then performed to examine the relative contribution of the predictors to the outcomes in a multivariate context. Because the four parent studies that constituted this sample were separated by gender, we performed the multivariate analyses accordingly, with final models run by gender to facilitate the interpretation of results.
Results
The CFA with oblimin rotation yielded four definitive factors, which explained 96.4% of the total variance. Factor I (Social Rejection) accounted for 75% of total variance, Factor II (Stereotypes) 12%, Factor III (Direct Threat or Physical Attack) 5.4%, and Factor IV (Police Mistreatment) 4.4%. The CFA fit statistics (adjusted goodness-of-fit index = 0.82; comparative fit index = 0.90) indicate that our model is acceptable. All factor loadings are statistically significant, indicating nonchance relationships with the factors that make theoretical sense (see Table 1). For example, Social Rejection has significant relations with the items “feeling left out,” “unfair treatment from mates,” “talking behind one’s back,” “feeling like an outsider,” and “lack of attention.” Specifically, the four subscales derived from the factor structure may be interpreted as (a) Social Rejection, consisting of six items such as, “Because of your race or ethnicity, how often have others ignored you or not paid attention to you?” (internal consistency of this subscale was high; Cronbach’s α = .89); (b) Stereotyping, consisting of five items such as, “How often have others hinted that you must be lazy?” (Cronbach’s α = .86); (c) Direct Threat/Attack, consisting of four items that include “How often have others threatened to hurt you?” (Cronbach’s α = .84); and (d) Police Mistreatment, consisting of one item (“Have police or security officers been unfair to you?”). The Cronbach’s α coefficients of the factors ranged from .84 to .89, indicating satisfactory internal consistency.
Table 1.
BPEDQ-CV item | Social rejection 1 | Stereotyping 2 | Direct threat/attack 3 | Police mistreatment 4 |
---|---|---|---|---|
Because of your race or ethnicity … | ||||
have others made you feel like an outsider who doesn’t fit in because of your dress, speech, or other characteristics related to your ethnicity? | 0.73 | |||
have you been treated unfairly by coworkers or classmates? | 0.80 | |||
have people been nice to you to your face, but said bad things about you behind your back? | 0.76 | |||
have people who speak a different language made you feel like an outsider? | 0.68 | |||
have others ignored you or not paid attention to you? | 0.80 | |||
has your boss or supervisor been unfair to you? | 0.76 | |||
have others thought you couldn’t do things or handle a job? | 0.61 | |||
have others hinted that you are dishonest or can’t be trusted? | 0.86 | |||
have others hinted that you must not be clean? | 0.70 | |||
have people not trusted you? | 0.82 | |||
has it been hinted that you must be lazy? | 0.77 | |||
have others threatened to hurt you? | 0.72 | |||
have others actually hurt you or tried to hurt you? | 0.72 | |||
have others threatened to damage your property? | 0.83 | |||
have others actually damaged your property? | 0.81 | |||
have policemen or security officers been unfair to you? | 0.99 |
Note. BPEDQ-CV = Brief Perceived Ethnic/Racial Discrimination Questionnaire–Community Version.
As shown in Table 2, African American participants reported an average age of 41.7 years, which was higher than the Latinx sample (32.8 years). A greater proportion of African American participants (74.7%) completed high school than did Latinx participants (64.5%). Approximately 63% of the sample earned less than $1,249 per month, with African American participants reporting lower income (86.3%) than Latinx participants (48.1%).
Table 2.
Variable | African American |
Latinx |
||
---|---|---|---|---|
M | SD or % | M | SD or % | |
Age**** | 41.7 | 10.2 | 32.8 | 7.3 |
Gender**** | ||||
Women | 63 | 27.4% | 220 | 81.5% |
Men | 16 | 72.6% | 50 | 18.5% |
Income**** | ||||
≤$1,249/mo | 157 | 86.3% | 114 | 48.1% |
>$1,249/mo | 25 | 13.7% | 123 | 51.9% |
Education* | ||||
Less than high school | 58 | 25.3% | 95 | 35.5% |
High school or more | 171 | 74.7% | 173 | 64.5% |
Depressive symptoms** | 17.9 | 12.5 | 14.8 | 12.0 |
PTS symptoms | 12.3 | 11.6 | 11.1 | 9.7 |
BPEDQ Social Rejection**** | 15.4 | 5.9 | 11.2 | 5.1 |
BPEDQ Stereotyping**** | 11.6 | 4.7 | 8.2 | 3.8 |
BPEDQ Direct Threat/Attack**** | 7.2 | 3.2 | 5.4 | 2.5 |
BPEDQ Police Mistreatment**** | 2.8 | 1.3 | 1.6 | 1.1 |
Note. N = 270 (African American), N = 230 (Latinx). PTS = posttraumatic stress; BPEDQ = Brief Perceived Ethnic/Racial Discrimination Questionnaire.
p < .05.
p < .01.
p < .0001.
In terms of depressive symptoms, the sample as a whole reported an average of 16.24, slightly above the clinical cutoff point of 16, indicating mild to moderate depressive symptoms and a risk for depression (Radloff, 1977). African American participants reported greater depressive symptomatology than did Latinx participants (p < .01). The mean score for the entire sample for PTS symptoms was 11.75, indicating a mild to moderate level of posttraumatic stress (Foa et al., 1997). No significant ethnic group differences were found for PTS symptoms. African American participants reported higher levels of Social Rejection (p < .0001), Stereotyping (p < .0001), Direct Threat/Attack (p < .0001), and Police Mistreatment (p < .0001) than did Latinx participants.
Bivariate correlations between predictor and outcomes variables are shown in Table 3. Lower income was associated with higher depression symptoms (r = −.21, p < .0001), as well as with each of the four dimensions of discrimination (Social Rejection: r = −.21, p < .0001; Stereotyping: r = –.22, p < .0001; Direct Threat: r = −.19, p < .0001; Police Mistreatment: r = −.24, p < .0001). Income was unrelated to PTS symptoms. Gender was associated with each of the four discrimination factors, with men experiencing higher levels of Social Rejection (r = .22, p < .0001), Stereotyping (r = .29, p < .0001), Direct Threat/Attack (r = .27, p < .0001), and Police Mistreatment (r = .40, p < .0001). Gender was not associated with depression or PTS symptoms. Each of the discrimination dimensions was highly correlated with depressive symptoms as well as PTS symptoms (all p < .0001).
Table 3.
Variable | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
---|---|---|---|---|---|---|---|---|
1. Depressive symptoms | — | |||||||
2. PTS symptoms | 0.68**** | — | ||||||
3. Gender | 0.05 | −0.07 | — | |||||
4. Income | −0.21**** | −0.03 | −.29**** | — | ||||
5. Social rejection | 0.47*** | 0.42**** | 0.22**** | −0.21**** | — | |||
6. Stereotyping | 0.40**** | 0.39**** | 0.29**** | −0.22**** | 0.81**** | — | ||
7. Direct threat | 0.32**** | 0.29**** | 0.27**** | −0.19**** | 0.60**** | 0.58**** | — | |
8. Police mistreatment | 0.31**** | 0.33**** | 0.40**** | −0.24**** | 0.63**** | 0.63**** | 0.50**** | — |
Note. PTS = posttraumatic stress.
p < .0001.
As indicated in Table 4, full regression models for both genders were highly significant (men: F(6, 205) = 9.77, p < .0001, R2 = .22; women: F(6, 112) = 8.70, p < .0001, R2 = .32). Among the individual predictors for men, race/ethnicity (Latinx, t = −2.31, p < .05) and Social Rejection (t = 3.76, p < .001) were significantly related to PTS symptoms. Among the individual predictors for women, Stereotyping (t = 2.53, p < .05) and Direct Threat/Attack (t = 1.98, p < .05) were associated with PTS symptoms. Interaction terms of race/ethnicity and income by each of the four discrimination dimensions were nonsignificant and not included in the final model to preserve power.
Table 4.
Men (N = 212) | Women (N = 119) | |||||
---|---|---|---|---|---|---|
F(6, 205) = 9.77**** | F(6, 112) = 8.70**** | |||||
R2 = .222 |
R2 = .318 |
|||||
Predictors | B | Standardized B | t value | B | Standardized B | t value |
Income | −1.91 | −.07 | −1.11 | 2.74 | .13 | 1.53 |
Race/ethnicity | ||||||
African American | −3.83 | −.15 | −2.31* | 2.89 | .10 | 0.28 |
Social rejection | 0.86 | .45 | 3.76*** | 0.22 | .12 | 1.01 |
Stereotyping | −0.27 | −.10 | −0.97 | 0.80 | .33 | 2.53* |
Direct threat/attack | 0.13 | .04 | 0.50 | 0.63 | .18 | 1.98* |
Police mistreatment | 1.05 | .12 | 1.45 | −0.04 | −.01 | −0.05 |
Note. Race/ethnicity (0 = Latinx).
p < .05.
p < .001.
p < .0001.
With regard to depression symptoms, both full regression models for depression were also highly significant (men: F(6, 205) = 8.75, p < .0001, R2 = .20; women: F(6, 206) = 17.42, p < .0001, R2 = .34; see Table 5). Among men, lower income (t = −2.36, p < .05), race/ethnicity (Latinx, t = −2.47, p < .05), and Social Rejection (t = 4 .60, p < .0001) were significantly associated with higher depression symptoms. For women, Social Rejection was significantly related to depression symptoms (t = 3.53, p < .0001). As with PTS symptoms, the interactions terms of race/ethnicity and income by each of the four discrimination dimensions were nonsignificant and not included in the final model.
Table 5.
Men (N = 212) | Women (N = 207) | |||||
---|---|---|---|---|---|---|
F(6, 205) = 8.75**** | F(6, 206) = 17.42**** | |||||
R2 = .203 |
R2 = .343 |
|||||
Predictors | B | Standardized B | t value | B | Standardized B | t value |
Income | −4.67 | −.16 | −2.36* | −2.46 | −.10 | −1.63 |
Race/ethnicity | ||||||
African American | −4.74 | −0.16 | −2.47* | −1.70 | −.04 | −.58 |
Social rejection | 1.21 | .55 | 4.60**** | .77 | .33 | 3.53**** |
Stereotyping | −.53 | −.19 | −1.66 | .43 | .14 | 1.39 |
Direct threat/attack | .21 | .05 | .68 | .62 | .13 | 1.86 |
Police mistreatment | −.13 | −.01 | −.16 | .48 | .04 | .53 |
Note. Race/ethnicity (0 = Latinx).
p < .05.
p < .0001.
Discussion
That racial/ethnic discrimination is stressful for those who experience it is has been well documented. In this study, we have attempted to dissect the dimensions that may be more stressful than others in terms of depression and posttraumatic symptoms. In our exploration of the factor structure of the BPEDQ-CV, the four factors that emerged from the CFA—Social Rejection, Stereotyping, Direct Threat/Attack, and Police Mistreatment—are similar but not identical to the four subscales previously found, which comprised Social Exclusion, Stigmatization, Discrimination at Work/School, and Threat/Harassment (Brondolo et al., 2011). Three of the four factors clearly map on well to one another: Social Rejection to Social Exclusion, Stereotyping to Stigmatization, and Direct Threat/Attack to Threat/Harassment. The consistency of the first three factors points to their salience and relevance in our community sample of African American and Latinx individuals recruited exclusively from clinics in South Los Angeles, one of the lowest-income areas in Los Angeles County, compared to that of previous studies (e.g., Contrada et al., 2001; Landrine & Klonoff, 1996). The fourth factor we found, Police Mistreatment, departs from the original factor, Discrimination at Work/School (Brondolo et al., 2011). This makes sense in light of our more marginalized population, for whom work/school may not be as salient, as well as their surroundings in which hostility and tension between the community and the police may be an omnipresent concern. The differences in factor structure underscore the need to pay attention to differences in social environments and suggest that assessment instruments may need to be empirically tested to gauge its appropriateness before use in a particular population.
The whole confluence of ethnic/racial discrimination predicting both depression and PTS symptoms is consistent with the ample evidence that exists in the literature. In examining the bivariate relationships among the predictors, it is not surprising that economic deprivation was associated with all four dimensions of discrimination. Clearly, being poor subjects an individual to higher degrees of social rejection, stereotyping, direct threat, and police mistreatment. The effects of poverty on depression, over and above the effects of discrimination, speak to its insidious and grinding nature, perhaps more pronounced in men, who are expected to be providers for their families. In the historically impoverished community such as the one where our sample resides, economic deprivation remains a constant source of stress that damages mental health (Fell & Hewstone, 2015). While African Americans reported higher depressive symptoms than Latinx participants in the bivariate analysis, this relationship was reversed in the multivariate context, such that when income is accounted for, the relationship of ethnicity to depressive symptoms is attenuated. That is, controlling for the effects of low income, Latinx men may be at higher risk for depression than African American men. For women, only social rejection was uniquely associated with depressive symptoms. This suggests that the effects of social rejection are powerful, above and beyond that which can be explained by income or ethnicity.
The different pattern found between men and women in predicting PTS symptoms may reflect the fact that women are subject to more harassment, sexual and otherwise, and that stereotypes (e.g., comments about sexuality or a woman’s body) may be an common and inextricable part of that harassment (Lewis & Van Dyke, 2018; Perry, Harp, & Oser, 2013). That stereotyping and direct attacks are salient and pernicious aspects of discrimination that result in posttraumatic stress in women is consistent with the wider ongoing social issues of the misogyny in the workplace and beyond.
The finding that social rejection emerged as the single significant dimension in three of the four models suggests that this aspect of discrimination may have underlying harmful effects above and beyond the other dimensions, cutting across particular sets of symptomatology that may be somewhat surprising given the attention paid to stereotyping, direct threats or attacks, and police mistreatment in the popular media. However, the idea that racial and ethnic discrimination represents a type of social rejection, at least in part, is an intriguing one that deserves further attention. According to Eisenberger and Lieberman (2005), there exists a neurocognitive basis, with evolutionary roots, that explains why social rejection is an extremely painful experience, on par with physical pain, with the same pathway to health outcomes. Due to the importance of being part of a social group and its selective advantage over loneliness and isolation in the course of human evolution (Eisenberger & Lieberman, 2005), social rejection can be extremely traumatic. In addition, because of its more ambiguous nature, social rejection ultimately attributed to one’s ethnicity or race may be more stressful than being directly and clearly confronted for being a member of one’s ethnic group, whether by the police or other individuals. When called a racist name or being harassed by the police while driving or walking, the discrimination is clear and clearly attributable to the perpetrator’s bigoted views. However, when one is socially rejected, it is often unclear why, and previous studies have shown that ambiguity, due to its lack of control and predictability, is particularly stressful (Bennett, Wolin, Robinson, Fowler, & Edwards, 2005; Mendes, Major, McCoy, & Blascovich, 2008; Stetler, Chen, & Miller, 2006). This is consistent with recent studies on the effects of microaggressions (Sue et al., 2008). Whereas life-threatening hate crimes or severe physical assault can cause posttraumatic stress disorder, subtle, ambiguous racial stressors can result in PTS symptoms, including hypervigilance and paranoia (M. T. Williams et al., 2014). Indeed, one study found that African Americans experienced greater cognitive impairment when faced with ambiguous prejudice as compared to blatant racism (Salvatore & Shelton, 2007). Implicit biases and attitudes may result in ambiguous interactions, which lead the recipient to be unsure of the intent of the other individual (Gaertner & Dovidio, 2005; M. T. Williams et al., 2014). Being ostracized—even very briefly and/or for mundane reasons—activates a temporary state of misery by signaling pain, increasing stress, threatening fundamental needs, and causing sadness and anger (Eisenberger & Lieberman, 2005; Weir, 2012). More sustained, frequent, or severe experiences may have greater and long-lasting adverse effects, particularly among frequently targeted groups (Wesselmann, Cardoso, Slater, & Williams, 2012). The finding that social rejection is associated with not only depressive symptoms but also posttraumatic stress symptoms underscores the scope of these negative effects.
A limitation of the current study is the use of nonrandom samples from four parent studies, which limits the generalizability of the results. The cross-sectional and correlational design of this study precludes any temporal or causal conclusions about the experience of discrimination and depression and PTS symptoms. However, while it is possible that being depressed or experiencing posttraumatic stress symptoms leads one to interpret ambiguous or innocuous stimuli as social rejection, previous longitudinal studies support the idea that perceptions of discrimination precede and predict poor mental health rather than the other way around (Brown et al., 2000; Schulz et al., 2006). In light of these findings, the experience of being excluded or ostracized should be of prime consideration when African American and Latinx individuals encounter racial or ethnic discrimination. The sequelae of threats and attacks should be addressed and stereotypes should be countered, particularly for women, but in addition, mental health professionals should pay close attention to the underlying sense of social rejection that discrimination engenders. In the current political climate in which immigrants and racial/ethnic minorities are isolated and harassed (Okeowo, 2018), the effects of social rejection may pose a serious public mental health concern. That social exclusion and rejection may be becoming institutionalized and systemic (Domonoske & Gonzales, 2018; Rogers & Fandos, 2019) has troubling implications from a public mental health perspective, with potentially massive effects across groups of children and adults (Miller, Hess, Bybee, & Goodkind, 2018). For example, discriminatory policies that perpetuate otherness among Latinx individuals, such as law enforcement targeting people who “look undocumented,” may have widespread mental health ramifications (Otiniano Verissimo et al., 2014, p. 1427).
How should mental health providers address these potential effects? Interventions for racial/ethnic discrimination might consider emphasizing social connectedness and belonging to counteract the harm of social rejection, using collectivistic frameworks such as collective efficacy (e.g., “we can”) over individual agency (“I can”), which may help guide and mobilize an effective response to discrimination experiences. An example of such an intervention is the Emotional Emancipation Circles (Grills, Aird, & Rowe, 2016), a promising, evidence-informed, psychotherapeutic group intervention that provides a historical context for racial trauma designed for Black people to heal from, overcome, and overturn the “lie of Black inferiority and White superiority” (Grills et al., 2016). Thus, in calling out the stereotypes that abound and explicitly labeling perceived discrimination, its inherent social rejection, and its historical context, interventions may provide structure and meaning to the often amorphous and ambiguous generalized stressor, offering a language and legitimacy that equip clients to further mobilize effective strategies.
Clinicians may help clients who experience discrimination by validating the significance of their social rejection. Asking for specific examples provides structure for the ethnic minority client by articulating what they may have experienced as ambiguous and uncomfortable, such as passing a Caucasian stranger who averted their eye gaze or appeared to look through them—yet observing the same person sharing eye contact with a Caucasian person (Wirth, Sacco, Hugenberg, & Williams, 2010). Detailed dismantling in a supportive context of these discrimination experiences would validate clients’ experiences and provide a framework for any resulting feelings of dejection, anxiety, and anger. Socially rejected groups may seek each other’s support to maintain a sense of belonging and buffer against future threats. A key cultural variable—social interdependence—has been found to moderate individuals’ responses to ostracism. In collectivistic cultures such as many Latinx cultures, social interdependence helps to attenuate negative outcomes (Uskul & Over, 2017).
Interventions at the systemic level include the implementation of policies and practices at institutions and the workplace to provide connection and support to racial/ethnic minorities. If the results of this study imply that ostracism has pernicious effects, it also suggests remedies that are scalable and feasible on a larger level. On a macrolevel, public policies that discriminate and result in the social exclusion of groups or individuals need to be considered through the lens of potential long-term mental health effects.
Public Policy Relevance Statement.
Racial and ethnic discrimination is a widespread social problem with severe adverse psychological consequences. The major finding of this study suggests that one aspect of discrimination in particular, social rejection, may be the most harmful, associated with both depression symptoms in men and women and with posttraumatic stress (PTS) symptoms in men. The effects of stereotyping and direct threats or attacks play important roles in PTS symptoms for women. Policies and practices regarding discrimination that emphasize a sense of belonging and mitigate ostracism and isolation, as well as addressing the impact of stereotyping for women, would have a strong overall positive societal impact.
Contributor Information
Dorothy Chin, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles.
Tamra B. Loeb, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles
Muyu Zhang, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles.
Honghu Liu, School of Dentistry, University of California, Los Angeles.
Michele Cooley-Strickland, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles.
Gail E. Wyatt, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles.
References
- Atkins R (2014). Instruments measuring perceived racism/racial discrimination: Review and critique of factor analytic techniques. International Journal of Health Services, 44, 711–734. 10.2190/HS.44.4.c [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beatty DL, Matthews KA, Bromberger JT, & Brown C (2014). Everyday discrimination prospectively predicts inflammation across 7-years in racially diverse midlife women: Study of women’s health across the nation. Journal of Social Issues, 70, 298–314. 10.1111/josi.12061 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bécares L, & Zhang N (2018). Perceived interpersonal discrimination and older women’s mental health: Accumulation across domains, attributions, and time. American Journal of Epidemiology, 187, 924–932. 10.1093/aje/kwx326 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bennett GG, Wolin KY, Robinson EL, Fowler S, & Edwards CL (2005). Perceived racial/ethnic harassment and tobacco use among African American young adults. American Journal of Public Health, 95, 238–240. 10.2105/AJPH.2004.037812 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brondolo E, Hausmann LRM, Jhalani J, Pencille M, Atencio-Bacayon J, Kumar A, … Schwartz J (2011). Dimensions of perceived racism and self-reported health: Examination of racial/ethnic differences and potential mediators. Annals of Behavioral Medicine, 42, 14–28. 10.1007/s12160-011-9265-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brondolo E, Kelly KP, Coakley V, Gordon T, Thompson S, Levy E, … Contrada RJ (2005). The perceived ethnic discrimination questionnaire: Development and preliminary validation of a community version. Journal of Applied Social Psychology, 35, 335–365. 10.1111/j.1559-1816.2005.tb02124.x [DOI] [Google Scholar]
- Brown TN, Williams DR, Jackson JS, Neighbors HW, Torres M, Sellers SL, & Brown KT (2000). “Being black and feeling blue”: The mental health consequences of racial discrimination. Race and Society, 2, 117–131. 10.1016/S1090-9524(00)00010-3 [DOI] [Google Scholar]
- Contrada RJ, Ashmore RD, Gary ML, Coups E, Egeth JD, Sewell A, … Chasse V (2001). Measures of ethnicity-related stress: Psychometric properties, ethnic group differences, and associations with well-being. Journal of Applied Social Psychology, 31, 1775–1820. 10.1111/j.1559-1816.2001.tb00205.x [DOI] [Google Scholar]
- Domonoske C, & Gonzales R (2018, June 19). What we know: Family separation and ‘zero tolerance’ at the border. NPR. Retrieved from https://www.npr.org/2018/06/19/621065383/what-we-know-family-separation-and-zero-tolerance-at-the-border
- Edwards RR (2008). The association of perceived discrimination with low back pain. Journal of Behavioral Medicine, 31, 379–389. 10.1007/s10865-008-9160-9 [DOI] [PubMed] [Google Scholar]
- Eisenberger NI, & Lieberman MD (2005). Why it hurts to be left out: The neurocognitive overlap between physical and social pain. In Williams KD, Forgas JP, & von Hippel W (Eds.), Sydney Symposium of Social Psychology series. The social outcast: Ostracism, social exclusion, rejection, and bullying (pp. 109–127). New York, NY: Psychology Press. [Google Scholar]
- Fell B, & Hewstone M (2015, June 4). Psychological perspectives on poverty. Retrieved from https://www.jrf.org.uk/report/psychological-perspectives-poverty
- Foa EB, Cashman L, Jaycox L, & Perry K (1997). The validation of a self-report measure of posttraumatic stress disorder: The Posttraumatic Diagnostic Scale. Psychological Assessment, 9, 445–451. 10.1037/1040-3590.9.4.445 [DOI] [Google Scholar]
- Gaertner SL, & Dovidio JF (2005). Categorization, recategorization, and intergroup bias. In Dovidio JF, Glick P, & Rudman LA (Eds.), On the nature of prejudice: Fifty years after Allport (pp. 71–88). Malden, MA: Blackwell. 10.1002/9780470773963.ch5 [DOI] [Google Scholar]
- Glover DA, Loeb TB, Carmona JV, Sciolla A, Zhang M, Myers HF, & Wyatt GE (2010). Childhood sexual abuse severity and disclosure predict posttraumatic stress symptoms and biomarkers in ethnic minority women. Journal of Trauma & Dissociation, 11, 152–173. 10.1080/15299730903502920 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grills CN, Aird EG, & Rowe D (2016). Breathe, baby, breathe. Cultural Studies ↔ Critical Methodologies, 16, 333–343. 10.1177/1532708616634839 [DOI] [Google Scholar]
- Grollman EA (2012). Multiple forms of perceived discrimination and health among adolescents and young adults. Journal of Health and Social Behavior, 53, 199–214. 10.1177/0022146512444289 [DOI] [PubMed] [Google Scholar]
- Jackson JS, Brown TN, Williams DR, Torres M, Sellers SL, & Brown K (1996). Racism and the physical and mental health status of African Americans: A thirteen year national panel study. Ethnicity & Disease, 6, 132–147. [PubMed] [Google Scholar]
- Kessler RC, Mickelson KD, & Williams DR (1999). The prevalence, distribution, and mental health correlates of perceived discrimination in the United States. Journal of Health and Social Behavior, 40, 208–230. 10.2307/2676349 [DOI] [PubMed] [Google Scholar]
- Kim G, Decoster J, Huang C-H, & Chiriboga DA (2011). Race/ethnicity and the factor structure of the Center for Epidemiologic Studies Depression Scale: A meta-analysis. Cultural Diversity and Ethnic Minority Psychology, 17, 381–396. 10.1037/e708362011-001 [DOI] [PubMed] [Google Scholar]
- Kressin NR, Raymond KL, & Manze M (2008). Perceptions of race/ethnicity-based discrimination: A review of measures and evaluation of their usefulness for the health care setting. Journal of Health Care for the Poor and Underserved, 19, 697–730. 10.1353/hpu.0.0041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Landrine H, & Klonoff EA (1996). The schedule of racist events: A measure of racial discrimination and a study of its negative physical and mental health consequences. Journal of Black Psychology, 22, 144–168. 10.1177/00957984960222002 [DOI] [Google Scholar]
- Lee DL, & Ahn S (2012). Discrimination against Latina/os: A metaanalysis of individual-level resources and outcomes. The Counseling Psychologist, 40, 28–65. 10.1177/0011000011403326 [DOI] [Google Scholar]
- Lewis TT, Cogburn CD, & Williams DR (2015). Self-reported experiences of discrimination and health: Scientific advances, ongoing controversies, and emerging issues. Annual Review of Clinical Psychology, 11, 407–440. 10.1146/annurev-clinpsy-032814-112728 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis TT, & Van Dyke ME (2018). Discrimination and the health of African Americans: The potential importance of intersectionalities. Current Directions in Psychological Science, 27, 176–182. 10.1177/0963721418770442 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu H, Prause N, Wyatt GE, Williams JK, Chin D, Davis T, … Myers HF (2015). Development of a composite trauma exposure risk index.PsychologicalAssessment,27,965–974. 10.1037/pas0000069 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Matsunaga M (2008, February). Concentrated poverty neighborhoods in Los Angeles. Retrieved from http://www.lachamber.com/clientuploads/LUCH_committee/082609_ConcentratedPovertyFinal_LUCH.pdf
- McNeilly MD, Anderson NB, Armstead CA, Clark R, Corbett M, Robinson EL, … Lepisto EM (1996). The perceived racism scale: A multidimensional assessment of the experience of white racism among African Americans. Ethnicity & Disease, 6, 154–166. [PubMed] [Google Scholar]
- Mendes WB, Major B, McCoy S, & Blascovich J (2008). How attributional ambiguity shapes physiological and emotional responses to social rejection and acceptance. Journal of Personality and Social Psychology, 94, 278–291. 10.1037/0022-3514.94.2.278 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller A, Hess JM, Bybee D, & Goodkind JR (2018). Understanding the mental health consequences of family separation for refugees: Implications for policy and practice. American Journal of Orthopsychiatry, 88, 26–37. 10.1037/ort0000272 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moody DLB, Chang Y-F, Pantesco EJ, Darden TM, Lewis TT, Brown C, … Matthews KA (2019). Everyday discrimination prospectively predicts blood pressure across 10 years in racially/ethnically diverse midlife women: Study of women’s health across the nation. Annals of Behavioral Medicine, 53, 608–620. 10.1093/abm/kay069 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nadimpalli SB, James BD, Yu L, Cothran F, & Barnes LL (2015). The association between discrimination and depressive symptoms among older African Americans: The role of psychological and social factors. Experimental Aging Research, 41, 1–24. 10.1080/0361073X.2015.978201 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Okeowo A (2018, November 1). Hate on the rise after Trump’s election. New Yorker. Retrieved from https://www.newyorker.com/news/news-desk/hate-on-the-rise-after-trumps-election
- Otiniano Verissimo AD, Grella CE, Amaro H, & Gee GC (2014). Discrimination and substance use disorders among Latinos: The role of gender, nativity, and ethnicity. American Journal of Public Health, 104, 1421–1428. 10.2105/AJPH.2014.302011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pascoe E, & Richman LS (2009). Supplemental material for perceived discrimination and health: A meta-analytic review. Psychological Bulletin, 135, 531–554. 10.1037/a0016059 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perry BL, Harp KLH, & Oser CB (2013). Racial and gender discrimination in the stress process: Implications for African American women’s health and well-being. Sociological Perspectives, 56, 25–48. 10.1525/sop.2012.56.1.25 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Powers MB, Gillihan SJ, Rosenfield D, Jerud AB, & Foa EB (2012). Reliability and validity of the PDS and PSS-I among participants with PTSD and alcohol dependence. Journal of Anxiety Disorders, 26, 617–623. 10.1016/j.janxdis.2012.02.013 [DOI] [PubMed] [Google Scholar]
- Priest N, Paradies Y, Trenerry B, Truong M, Karlsen S, & Kelly Y (2013). A systematic review of studies examining the relationship between reported racism and health and wellbeing for children and young people. Social Science & Medicine, 95, 115–127. 10.1016/j.socscimed.2012.11.031 [DOI] [PubMed] [Google Scholar]
- Radloff LS (1977). The CES-D Scale. Applied Psychological Measurement, 1, 385–401. 10.1177/014662167700100306 [DOI] [Google Scholar]
- Rogers K, & Fandos N (2019, July 14). Trump tells congresswomen to ‘go back’ to the countries they came from. New York Times. Retrieved from https://www.nytimes.com/2019/07/14/us/politics/trump-twitter-squad-congress.html
- Salvatore J, & Shelton JN (2007). Cognitive costs of exposure to racial prejudice. Psychological Science, 18, 810–815. 10.1111/j.1467-9280.2007.01984.x [DOI] [PubMed] [Google Scholar]
- Schulz AJ, Gravlee CC, Williams DR, Israel BA, Mentz G, & Rowe Z (2006). Discrimination, symptoms of depression, and self-rated health among African American women in Detroit: Results from a longitudinal analysis. American Journal of Public Health, 96, 1265–1270. 10.2105/AJPH.2005.064543 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sciolla A, Glover DA, Loeb TB, Zhang M, Myers HF, & Wyatt GE (2011). Childhood sexual abuse severity and disclosure as predictors of depression among adult African-American and Latina women. Journal of Nervous and Mental Disease, 199, 471–477. 10.1097/NMD.0b013e31822142ac [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stetler C, Chen E, & Miller GE (2006). Written disclosure of experiences with racial discrimination and antibody response to an influenza vaccine. International Journal of Behavioral Medicine, 13, 60–68. 10.1207/s15327558ijbm1301_8 [DOI] [PubMed] [Google Scholar]
- Sue DW, Nadal KL, Capodilupo CM, Lin AI, Torino GC, & Rivera DP (2008). Racial microaggressions against Black Americans: Implications for counseling. Journal of Counseling & Development, 86, 330–338. 10.1002/j.1556-6678.2008.tb00517.x [DOI] [Google Scholar]
- Torres L, & Vallejo LG (2015). Ethnic discrimination and Latino depression: The mediating role of traumatic stress symptoms and alcohol use. Cultural Diversity and Ethnic Minority Psychology, 21, 517–526. 10.1037/cdp0000020 [DOI] [PubMed] [Google Scholar]
- U.S. Census Bureau. (2018). Los Angeles County (South Central)–LA City (South Central/Watts). Census Reporter. Retrieved from https://censusreporter.org/profiles/79500US0603751-los-angeles-countysouth-central-la-city-south-centralwatts-puma-ca/ [Google Scholar]
- Uskul AK, & Over H (2017). Culture, social interdependence, and ostracism. Current Directions in Psychological Science, 26, 371–376. 10.1177/0963721417699300 [DOI] [Google Scholar]
- Vilagut G, Forero CG, Barbaglia G, & Alonso J (2016). Screening for depression in the general population with the Center for Epidemiologic Studies Depression (CES-D): A systematic review with metaanalysis. PLoS ONE, 11, e0155431. 10.1371/journal.pone.0155431 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Viseu J, Leal R, de Jesus SN, Pinto P, Pechorro P, & Greenglass E (2018). Relationship between economic stress factors and stress, anxiety, and depression: Moderating role of social support. Psychiatry Research, 268, 102–107. 10.1016/j.psychres.2018.07.008 [DOI] [PubMed] [Google Scholar]
- Weir K (2012). The pain of social rejection. American Psychological Association, 43, 50. Retrieved from https://www.apa.org/monitor/2012/04/rejection [Google Scholar]
- Wesselmann ED, Cardoso FD, Slater S, & Williams KD (2012). To be looked at as through air. Psychological Science, 23, 166–168. 10.1177/0956797611427921 [DOI] [PubMed] [Google Scholar]
- Williams DR, & Mohammed SA (2009). Discrimination and racial disparities in health: Evidence and needed research. Journal of Behavioral Medicine, 32, 20–47. 10.1007/s10865-008-9185-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams DR, Neighbors HW, & Jackson JS (2003). Racial/ethnic discrimination and health: Findings from community studies. American Journal of Public Health, 93, 200–208. 10.2105/AJPH.93.2.200 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams DR, & Williams-Morris R (2000). Racism and mental health: The African American experience. Ethnicity & Health, 5, 243–268. 10.1080/713667453 [DOI] [PubMed] [Google Scholar]
- Williams MT, Malcoun E, Sawyer BA, Davis DM, Bahojb Nouri L, & Bruce SL (2014). Cultural adaptations of prolonged exposure therapy for treatment and prevention of posttraumatic stress disorder in African Americans. Behavioral Sciences (Basel, Switzerland), 4, 102–124. 10.3390/bs4020102 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wirth JH, Sacco DF, Hugenberg K, & Williams KD (2010). Eye gaze as relational evaluation: Averted eye gaze leads to feelings of ostracism and relational devaluation. Personality and Social Psychology Bulletin, 36, 869–882. 10.1177/0146167210370032 [DOI] [PubMed] [Google Scholar]