Meta-analysis: Are Psychotherapies Less Effective for Black Youth in Communities With Higher Levels of Anti-Black Racism?

Abstract

Objective:

We examined whether anti-Black cultural racism moderates the efficacy of psychotherapy interventions among youth.

Method:

We analyzed a subset of studies from a previous meta-analysis of five decades of youth psychotherapy randomized controlled trials. Studies were published in English between 1963 and 2017 and identified through a systematic search. The 194 studies (N = 14,081; ages 2–19) across 34 states comprised 2,678 effect sizes (ESs) measuring mental health problems (e.g., depression) targeted by interventions. Anti-Black cultural racism was operationalized using a composite index of 31 items measuring explicit racial attitudes (obtained from publicly available sources; e.g., General Social Survey), aggregated to the state level and linked to the meta-analytic database. Analyses were conducted with samples of majority (i.e., ≥50%) Black (n = 36 studies) and majority-White (n = 158 studies) youth.

Results:

Two-level random effects meta-regression analyses indicated that higher anti-Black cultural racism was associated with lower ESs for studies with majority-Black youth (β = −0.2, 95% CI: −0.35, −0.04, p = .02) but was unrelated to ESs for studies with majority-White youth (β = 0.0004, 95% CI: −0.03, 0.03, p = .98), controlling for relevant area-level covariates. In studies with majority-Black youth, mean ESs were significantly lower in states with the highest anti-Black cultural racism (>1 SD above the mean; g = 0.19) compared to states with the lowest racism (<1 SD below the mean; g = 0.60).

Conclusion:

Psychotherapy randomized controlled trials with samples comprised of majority-Black youth were significantly less effective in states with higher (vs. lower) levels of anti-Black cultural racism, suggesting that anti-Black cultural racism may be one contextual moderator of treatment effect heterogeneity.

Introduction

Racism is a system that labels and ranks social groups based on racial categories; racial groups deemed inferior are devalued and afforded limited opportunities and resources.¹ Racism exists at individual, interpersonal, cultural, and structural/institutional levels.^2–4 Individual racism denotes how individuals of stigmatized races respond to racism, including internalization of racial stereotypes, which can lead to self-devaluation and hopelessness,² as well as anticipatory stress and hypervigilance.⁵ Interpersonal racism refers to enactments of racism between people, such as discrimination, mistreatment, and dehumanization.² At the macro level, cultural racism describes the collective values and belief systems that a particular culture holds regarding different racial groups, such as the many ways in which notions of Black inferiority and White superiority are embedded and promulgated.⁴ Cultural racism undergirds and sustains racism at the structural/institutional level,⁶ including policies and practices—both formal and informal, intended and unintended—that provide differential access to goods, services, and opportunities based on race.^2,3

Research on the relationship between racism and mental health has focused primarily on the mental health consequences of individual and interpersonal racism. For example, evidence from meta-analyses and systematic reviews consistently finds that interpersonal racism— typically measured via self-reports of discrimination and cultural mistrust—is associated with adverse mental health outcomes in Black adults⁷ and children,⁸ and with their underutilization of mental health care relative to White individuals.⁹ However, studies have begun identifying associations between structural racism and adverse health outcomes (e.g., chronic disease, decreased longevity), as well as reduced healthcare access and quality.¹⁰ Within the literature on mental health, scholars have also posited the potential impact of institutional racism on diagnoses, treatment access, and uptake.¹¹

The existing literature has significantly advanced our understanding of racism as a risk factor for adverse mental health outcomes among Black populations. Yet, important questions about this relationship remain. First, there is a dearth of research on the mental health consequences of cultural racism, despite repeated calls for such work.^3,4 One of the methodological challenges hindering this work is the lack of measures of cultural racism. Conceptualizations of cultural racism posit that aggregated social attitudes represent an important dimension of this construct because they reflect the dominant norms and ideologies that devalue racial groups within a particular context.^3,4 Several large-scale datasets include repeated assessments of racial attitudes, making it possible to aggregate individual racial attitudes to the community level (e.g., county, state) and compare levels of cultural racism across communities. The few existing studies using this approach find that Black people living in communities with higher (vs. lower) racism have worse health outcomes, including elevated mortality risk.¹² However, studies rarely link anti-Black cultural racism to mental health outcomes among Black individuals.

Second, research on the mental health effects of racism, across all levels (i.e., individual, interpersonal, cultural, structural/institutional), has focused on mental health symptoms/diagnoses and mental health care utilization.^7–9 Yet, whether racism might adversely affect mental health by undermining the efficacy of mental health interventions has not been explored. Although addressing sources of stress among minoritized individuals—including racism—is theorized to enhance the efficacy of interventions for marginalized groups,¹³ there has been surprisingly little research on whether racism moderates treatment effect heterogeneity. In the current study, we address this gap by examining whether mental health interventions with majority-Black samples are less effective in communities with higher (vs. lower) levels of anti-Black cultural racism.

Examining whether anti-Black cultural racism is associated with treatment efficacy requires studies of mental health interventions across numerous settings that provide adequate variation in cultural racism. However, mental health interventions are typically conducted in one, or a small number, of communities. Accordingly, respondents are exposed to the same social context,¹⁴ precluding the possibility of examining whether contextual factors (e.g., cultural racism) are associated with intervention outcomes in individual studies. Thus, novel methods are needed to examine contextual moderators of intervention effects. We employed one such a method, spatial meta-analysis, which allows for the measurement and statistical modeling of community contexts in relation to intervention efficacy.¹⁵ Spatial meta-analyses are similar to traditional meta-analyses, with the added step that the studies are geo-located, allowing researchers to characterize each study in terms of the social context in which it was conducted. Two recent studies using spatial meta-analyses have provided preliminary evidence that community-level prejudice moderates behavioral and mental health interventions with marginalized groups. Specifically, the efficacy of HIV prevention interventions¹⁶ and psychotherapy¹⁷ was significantly weaker in communities with higher levels of anti-Black racism and sexism, respectively.

We draw on the important initial insights of these two studies to consider whether anti-Black cultural racism, measured at the macro level, is associated with reduced intervention efficacy for youth psychotherapy. We hypothesized that anti-Black cultural racism would be associated with lower intervention efficacy in study samples with majority-Black youth but not among those with majority-White youth. We focus on youth for several reasons. First, psychopathology is common among youth, with approximately one in four meeting criteria for a psychiatric disorder.¹⁸ Second, many psychiatric disorders first appear during childhood and adolescence.¹⁹ Third, nationally representative data indicate that Black youth are significantly less likely to receive treatment for mental disorders than White youth, even after adjusting for established risk factors.²⁰ Fourth, racial identities are continually developing during childhood and adolescence,²¹ a time when social evaluations, especially of marginalized identities, may be especially salient.²² Indeed, meta-analyses indicate that the association between self-reported racism and psychological wellbeing is stronger among children than adults.^23,24 Likewise, the aforementioned spatial meta-analysis on community-level racism and HIV intervention efficacy among majority-Black samples found that this association was more robust for adolescents than adults.¹⁶ Thus, we examined whether anti-Black cultural racism is associated with lower psychotherapy efficacy among majority-Black samples during a particularly important developmental period.

Method

Data Sources, Study Selection, and Inclusion Criteria

We analyzed a subset of studies from a large database used in previous meta-analyses and described using PRISMA guidelines elsewhere.^25,26 Thus, only a brief description of the database and the methods specific to the present study are provided herein. The database included peerreviewed randomized controlled trials (RCTs) of youth psychotherapy targeting depression, anxiety, conduct problems, or ADHD, published in English between 1963 and 2017, and identified via PsycINFO and PubMed. Studies were reliably coded for several study and sample characteristics, study quality indicators, and moderators examined in the original meta-analysis.²⁵

In the present meta-analysis, studies were included if they (1) included 50% or more Black or White participants (e.g., majority Asian studies were excluded), (2) were conducted in the US, as anti-Black cultural racism was calculated for the 50 states and the District of Columbia (DC), and (3) collected post-treatment data specific to the targeted problem (e.g., anxiety symptoms for interventions targeting anxiety) for both the treatment and control groups (see Figure S1, available online). We chose the ≥50% cutoff for several reasons: (1) consistent with clinical research trials in general, most youth psychotherapy studies do not report racespecific outcome data (e.g., allowing for the calculation of separate effect sizes [ESs] for Black and White subsamples); (2) a prior study examining associations between community-level anti-Black racism and HIV intervention efficacy among majority-Black samples used this same cutoff;16 and (3) we did not have sufficient statistical power to use a more stringent cut-off (e.g., only seven studies were conducted with 100% Black youth).

Moderator Calculation

Consistent with prior work,^12,27 we aggregated individual explicit attitudes of race, racial prejudice, and race-related public policies to the contextual level (states) to create our measure of anti-Black cultural racism. We used a composite index of 31 items obtained from three different sources: the General Social Survey (e.g., “Are we spending too much, too little, or about the right amount on improving the conditions of Blacks?”), the American National Election Survey (e.g., “Irish, Italians, Jewish and many other minorities overcame prejudice and worked their way up. Blacks should do the same without any special favors.”), and Project Implicit (e.g., “I would rather not have Black people live in the same apartment building I live in.”). We standardized each item according to the mean values across all study years and states, then averaged the response to the state-level across all study years that it was queried in each survey. This resulted in an average score for each state, across all years, for each item. Higher scores reflected higher levels of anti-Black cultural racism (items were reverse-scored as needed). Supplement 1 and Table S1 (available online) provide additional details (e.g., item-level information).

Although several studies have used single-item measures (e.g., aggregated feelings thermometers) to capture cultural racism,^12,16,27 we chose a factor analytic approach because it improves construct validity and also taps into shared variance (minimizing unique variance across items), thereby reducing measurement error. A factor score was created for each state using exploratory factor analysis with a factor loading cutoff of 0.40. We then reran the factor analysis iteratively and excluded variables until all items (n = 31) met the 0.40 threshold. The analysis was performed using PROC FACTOR in SAS 9.4, with the prior communality estimate fixed at squared multiple correlations with all other variables. A one-factor solution emerged, indicating that these items load onto a single construct of anti-Black cultural racism, providing some preliminary support for construct validity. Recent research has provided additional evidence of the construct validity for aggregate, regional measures of explicit racial bias such as the ones used in the current study, including convergent validity (i.e., associations with other, theoretically relevant outcomes, including racially charged internet searches), and discriminant validity (i.e., lack of associations with theoretically unrelated outcomes, such as birth rates).²⁸ To evaluate the reliability of our measure, Cronbach’s alpha was calculated. Using this metric, the measure of anti-Black cultural racism showed high internal consistency (standardized α = .97). Factor scores for anti-Black cultural racism in all 50 states and DC ranged from a low of −3.39 (DC) to a maximum of 1.51 (Arkansas) and are shown in the Table S2 (available online). Sixteen of the studies with majority-White samples were conducted in multiple states and a sensitivity analysis excluding these studies indicated that their inclusion did not influence the direction or magnitude of our results (see Supplement 2).

The original meta-analysis project coded sample racial composition as a percentage of separate racial categories (e.g., % of sample consisting of Asian participants). Studies with majority-Black (i.e., ≥50% Black participants) or majority-White samples (i.e., ≥50% White participants; N = 194, k = 2678) were conducted across 34 states, with anti-Black cultural racism ranging from −1.98 to 1.51 (mean = 0.03, SD = 0.81). Across the 16 states represented in studies with majority-Black samples (n = 36, k = 385), anti-Black cultural racism ranged from −0.96 to 1.15 (mean = 0.44, SD = 0.59). The majority of these studies (n = 30, k = 285) were conducted in states with levels of anti-Black cultural racism higher than the national average (i.e., above 0), with only six studies (k = 100) conducted in states with anti-Black cultural racism lower than the national average.

Covariates

Area-level socioeconomic attainment and racial composition are both predictors of racist attitudes.²⁹ Thus, we controlled for White and Black population density (i.e., state-level population density multiplied by the proportion of White/Black state residents) and state poverty rate (i.e., the percentage of state residents living below the federal poverty level). State population density indicators were obtained from the American Community Survey and averaged over the years 2000–2010. State poverty was obtained from the Current Population Survey and also averaged over the years 2000–2010. Controlling for these three state-level covariates served two purposes. First, it enabled us to demonstrate that our measure of anti-Black cultural racism is distinct from other state-level characteristics that influence racial attitudes. Second, it enabled us to address a plausible alternative explanation for the results—namely, that anti-Black racism is higher in poorer states, and that poorer states may also have more adversity to overcome, and thus weaker intervention ESs.

Finally, we examined whether two variables associated with intervention efficacy in the larger meta-analytic database²⁵ affected results. We found that targeted problem (i.e., whether a study intervention targeted internalizing or externalizing problems) was associated with both our moderator and outcome (see Supplement 3), such that studies targeting externalizing problems (e.g., conduct) had significantly lower ESs and were conducted in states with significantly higher anti-Black cultural racism, compared to those targeting internalizing problems (e.g., depression). Thus, targeted problem was included as a fourth covariate.

Effect Size Calculation and Statistical Analyses

We chose to limit ESs to those measuring problems targeted by the intervention. This approach is consistent with prior meta-analyses focused on clinically-meaningful outcomes²⁹ and excludes a wide variety of non-mental health measures often included in RCT measurement models (e.g., academic achievement). Due to the many shortcomings associated with choosing just one ES or averaging ESs,³⁴ all ESs measuring a problem targeted by the intervention were included. ES was calculated using Cohen’s d, reflecting the standardized mean difference between treatment and control groups measured at post-treatment and follow-up assessments. Specifically, ES was determined by including sampling variation at Level 1 and within-study variation at Level 2. Hedges’ small sample correction was applied to all ESs, resulting in an unbiased estimate for the population standardized mean difference (Hedges’ g).³⁰ Individual analyses weighted ESs by the inverse of the sampling variance,³⁰ which functions to reduce bias and increase estimate efficiency by more heavily weighting ESs with smaller standard errors.³¹ For fixed effects, denominator degrees of freedom were computed using residual degrees of freedom. Weighted two-level random-effects models³² were conducted in R using the metafor package (R code in Supplement 4). This approach does not assume a true effect across studies but allows for a distribution of true ESs.³³

We examined our hypothesis across several models. First, we examined whether racial composition of the study samples interacted with anti-Black cultural racism to predict ES. Finding that it did, we employed meta-regression to examine anti-Black cultural racism as a moderator of intervention efficacy for majority-Black (i.e., ≥50% Black participants) and majority-White (i.e., ≥50% White participants) samples separately, controlling for the four covariates described above. Third, we examined the moderating effect of a binary measure of anti-Black cultural racism to simultaneously compare studies with majority-Black samples conducted in states with anti-Black cultural racism at the highest (1 standard deviation (SD) above the sample mean) and lowest (1 SD below the sample mean) levels. We chose to examine this relationship both continuously and categorically because categorical variables support ease of interpretation when examining magnitude of effect.

Fourth, to identify potential mechanisms underlying this observed association, we conducted a post-hoc analysis examining whether psychotherapy efficacy varied across measurement period (i.e., immediately post-treatment vs. follow-up assessment) and level of anti-Black cultural racism (i.e., high vs. low). If the ESs in high (vs. low) anti-Black cultural racism states are constant across measurement period (i.e., did not appreciably change in the follow-up vs. immediate post-treatment assessment), this would suggest that the association between anti-Black cultural racism and treatment efficacy is likely to be driven by factors occurring during treatment (e.g., lack of clinicians’ cultural competency). Conversely, if the ESs in high (vs. low) anti-Black cultural racism states continue to erode in the months after treatment ended, this would provide evidence that there is something about these environments themselves, separate from what occurs during the treatment, that undermines treatment efficacy.

Moran’s I was calculated to determine if residuals within the primary model (i.e., among majority-Black samples) were geographically correlated.¹⁵ Residual spatial autocorrelation was not detected; in other words, model errors were distributed randomly across states, precluding the need for further model adjustments.

Results

Database

Of the larger database,²⁵ 36 RCTs (N = 2,182; k = 385) included majority-Black samples. hese studies were published between 1971 and 2015 (see References S1, available online). Therapies targeted a variety of problems, including ADHD (3 studies), conduct (21 studies), anxiety (8 studies), and depression (4 studies). Previously conducted tests of publication bias and study quality effects indicated that publication bias was present but minimally impacted results, and that study quality was unrelated to ESs (see Weisz et al.²⁵ for details).

Anti-Black Cultural Racism as a Moderator of Intervention Efficacy

A two-level random effects meta-regression analysis indicated a significant interaction between anti-Black cultural racism and racial composition in the intervention sample (p = .0017). Consequently, stratified analyses were conducted examining anti-Black cultural racism as a moderator of intervention efficacy for studies with majority-Black vs. majority-White samples. Higher anti-Black racism was associated with lower ESs for studies with majority-Black youth (β = −0.20, 95% CI: −0.35, −0.04, p = .02) but was unrelated to ESs for studies with majority-White youth (n = 158, k = 2,293; β = 0.0004, 95% CI: −0.03, 0.03, p = .98), controlling for study covariates (Figure 1). In studies with majority-Black youth, anti-Black cultural racism was subsequently examined as a categorical moderator representing states with values >1 SD above or below the sample mean, with the values dummy coded and compared, to further probe this finding. ESs were significantly lower in states with the highest anti-Black cultural racism (g = 0.19) compared to states with the lowest racism (g = 0.60; Table 1).

Figure 1. Anti-Black Cultural Racism and Effect Size Across Subsets by Majority Race.

g = type of effect size, specifically Hedge’s g.

Table 1.

Psychotherapy Effect Sizes as a Function of Anti-Black Cultural Racism in Studies with Majority-Black Samples

Variable		β	SE	95% CI	Moderator t-test (t-value)
Unadjusted model: Anti-Black cultural racism without covariates		−0.21	0.07	[−0.34, −0.08]	−3.20**
Adjusted model: Anti-Black cultural racism with covariates		−0.20	0.08	[−0.35, −0.04]	−2.43*
White population density		−0.11	0.07	[−0.24, 0.02]	−1.64
Black population density		0.03	0.06	[−0.08, 0.14]	0.51
Poverty		−0.04	0.08	[−0.12, 0.19]	0.43
Targeted problem (internalizing)		0.27	0.10	[0.07, .047]	2.62**
Categorical Moderator Analysis
Anti-Black cultural racism [range]	k	Predicted g	SE	95% CI	Moderator t-test (t−value)
Lowest [−0.96, −0.37]	100	0.60	0.11	[0.39, 0.81]	5.60***
Highest [1.11, 1.15]	132	0.19	0.07	[0.05, 0.34]	2.65**

Note: Predicted effect sizes (predicted g) are given at the minimum, median, and maximum values observed in the data. Results were produced using (1) random-effects meta-regression models excluding the intercept and including the “factor” function in the R metafor package, and (2) Knapp and Hartung adjusted t−tests.⁴⁸ A significant moderator t−test result indicates that the effect for the anti-Black cultural racism level is significant after adjusting for the other levels. White and Black population density, defined as state-level population density multiplied by the proportion of White and Black state residents, respectively, were obtained from the American Community Survey and averaged over the years 20002010. State poverty, defined as the percentage of state residents living below the federal poverty level, was obtained from the Current Population Survey and also averaged over the years 2000–2010. Targeted problem is a binary variable representing whether a study intervention targeted internalizing or externalizing problems. The beta coefficient here represents studies targeting internalizing problems, compared to studies targeting externalizing problems. k = number of effect sizes in each category.

^*p < .05

^**p < .01

^***p < .001

Next, we examined the interaction between measurement period (i.e., immediately post-treatment vs. follow-up assessment) and anti-Black cultural racism (i.e., below or above the sample mean). This interaction was significant (β = 0.29, p = .02): immediately post-treatment, the ES difference between low and high racism states was moderately large, but did not reach statistical significance (predicted gs = 0.32 and 0.12, respectively; β = 0.13, t = 1.3, p = .18); however, at follow-up, the ES difference between low and high racism states widened and became statistically significant (predicted gs = 0.43 and 0.08, respectively; β = 0.35, t = 2.34, p = .02; see Figure 2).

Figure 2. Predicted Effect Size in States with Low vs High Anti-Black Cultural Racism Across Assessment Period.

g = type of effect size, specifically Hedge’s g.

Discussion

This study used spatial meta-analysis to examine whether anti-Black cultural racism is associated with treatment effect heterogeneity in youth psychotherapy. As hypothesized, we found that psychotherapy interventions with majority-Black samples were significantly less effective in communities with higher anti-Black cultural racism, controlling for state-level poverty, White population density, Black population density, and targeted problem. Specifically, ESs for studies with majority-Black samples were three times larger in states with the lowest vs. highest anti-Black cultural racism, suggesting that anti-Black cultural racism is meaningfully associated with reduced intervention efficacy. Importantly, our analyses revealed no association between anti-Black cultural racism and intervention efficacy among majority-White samples. This negative control analysis³⁴ reduces concerns that the association between anti-Black cultural racism and intervention efficacy among majority-Black samples reflects contextual factors that may correlate with anti-Black cultural racism (e.g., area-based conservatism, health care expenditures, religiosity, clinician training/licensing requirements). These other contextual factors would be expected to exert similar effects—that is, be associated with reduced treatment efficacy—for both White and Black samples, in contrast to the divergent patterns we observed between these two groups.

These results point to several areas for future research. Our approach to operationalizing anti-Black cultural racism might be productively applied to other datasets to examine associations with a broader range of mental health outcomes, such as symptoms of psychological distress, psychiatric diagnoses, and rates of mental health care utilization. This research would help to address the dearth of studies using a contextual approach to conceptualizing and measuring anti-Black cultural racism.^3,4 Future research is also needed to determine whether our results are robust to different operationalizations of racism. The indicators of anti-Black cultural racism used in this study were comprised of aggregated explicit social attitudes. One limitation of this approach is social desirability bias, as individuals may be hesitant to endorse negative attitudes perceived to be socially deviant. This reporting bias could potentially under-estimate associations between anti-Black cultural racism and intervention efficacy. Studies have begun to circumvent this limitation by employing other means of capturing attitudes that do not rely on self-report, such as Google searches of racial epithets³⁵ and implicit attitudes,³⁶ but these have not been linked to mental health outcomes. Additionally, future studies should incorporate objectively measured indicators of structural racism (e.g., political representation, institutional practices³⁷) to determine whether the patterns observed in this study are generalizable to these other dimensions of racism.

While our findings suggest that contextual racism may undermine intervention efficacy for majority-Black samples, they do not explain why anti-Black cultural racism may render therapies less effective, because our meta-analytic database did not include comprehensive measures of potential mechanisms. However, our finding that the observed heterogeneity in the marginal (primary) analyses is largely due to differences in treatment efficacy in the follow-up assessment, rather than in the immediate aftermath of treatment, begins to reveal potential mechanisms. Specifically, the erosion of treatment effects over time in the high racism group, and the improvements over time in the low racism group, support our interpretation that identitythreatening environments themselves may undermine treatment gains. These results are consistent with a spatial meta-analysis of HIV prevention interventions,¹⁶ which demonstrated that the ESs for condom use decreased over time in high-racism communities, relative to those in the control condition. Taken together, these results suggest that anti-Black cultural racism may manifest in an array of adverse experiences for Black youth that compete with the benefit of insights and skills developed in therapy. Future studies should explore these and other possibilities to identify causal pathways.

Our findings contribute to a small but growing body of evidence identifying contextual moderators of treatment effect heterogeneity. Two other spatial meta-analyses have similarly found that HIV prevention interventions among majority-Black samples¹⁶ and psychotherapy interventions among majority-girl samples³⁸ are significantly less effective when conducted in communities where residents endorse higher levels of anti-Black racism and sexism, respectively. Using another methodological approach, researchers have evaluated growth mindset interventions (which teach that intelligence is malleable) in nationally representative samples across numerous treatment sites and found that these interventions are more likely to improve grades in classrooms where the peer norms are supportive of growth mindsets.³⁹

Collectively, these studies on contextual moderators of intervention efficacy have potentially important implications for clinical and public health interventions with Black youth. With respect to clinical implications, they suggest that psychotherapies delivered in environments with high levels of anti-Black cultural racism may need to be modified in order for Black youth to derive maximum benefit. Indeed, preliminary evidence suggests that interventions are more efficacious (e.g., in reducing condom use and depression, respectively) when they are adapted to meet the needs and values of African Americans¹⁶ and sexual minorities⁴⁰ living in high-prejudice contexts. However, which adaptations maximize efficacy, and why they are effective, is largely unknown. Future research addressing these questions is necessary for the development of adjunctive modules to improve the efficacy of existing interventions. Notably, however, it is widely acknowledged that stigma and racism exert negative effects across several levels,^2,3,41 and thus require multi-level approaches.⁴² Accordingly, structural- and community-level interventions targeting racism at its source may also enhance the efficacy of psychological interventions implemented in communities with high levels of racism.

Our study has several noteworthy methodological strengths: data from a large, rigorous meta-analysis of over fifty years of youth psychotherapy RCTs; a comprehensive index of anti-Black cultural racism drawing on three large-scale datasets (including two nationally representative studies) with multiple measures of explicit racial attitudes; and the use of a novel approach for examining contextual moderators of intervention efficacy (i.e., spatial meta-analysis¹⁵). These strengths should be considered alongside the study’s limitations. First, our study sample included only 34 states (16 of which included majority-Black samples), resulting in a restricted range of anti-Black cultural racism. On the one hand, because this restricted range reduced our statistical power to detect moderation, our estimates of the association between anti-Black cultural racism and intervention efficacy are likely conservative. On the other hand, the restricted range makes it difficult to assess whether our findings are generalizable to contexts not included in our sample. Second, the relatively small number of studies with majority-Black youth (n = 36) meant that we were not adequately powered to examine whether the association between anti-Black cultural racism and intervention efficacy became stronger in samples of studies with a larger percentage of Black youth (e.g., 75%). Our approach of using a 50% cutoff has been used in prior studies,^16,38 but it will be important in future studies to test whether the results are robust to different cutoff points.

Third, our study measured cultural racism at the state level, consistent with some previous research.³⁰ If cultural racism measured at a more proximal level (e.g., counties) has a stronger association with the efficacy of psychotherapy interventions, then our approach likely offers a conservative test of this relationship. Nevertheless, it will be important for future research to examine associations between cultural racism and intervention efficacy at different geographic scales to determine whether the strength of associations is consistent across diverse social contexts.

Fourth, we included all outcome measures of the problem targeted by the intervention (e.g., depression) for each RCT, consistent with best practices.³⁴ However, most studies included multiple ESs, and thus violated the assumption of independent ESs. Although the issue of ES dependency is sometimes addressed by averaging or selecting specific ESs, both approaches have significant limitations, including ignoring heterogeneity of effects across outcomes and the loss of data, respectively.^32,43,44 Nevertheless, to address this issue, we ran two sensitivity analyses. First, we accounted for ES-dependency within studies using robust variance estimation. Results were identical in magnitude and direction to our primary model, although the beta estimate was no longer statistically significant, which likely resulted from the substantial loss of power in these nested models (see Supplement 5). Second, we re-analyzed our results using the primary outcome measure (rather than all measures of the problem targeted in the intervention); the direction and magnitude of the beta estimates were similar to, and somewhat stronger than, those in our main analysis (see Supplement 6 and Table S3, available online).

Finally, the datasets that we used to create the anti-Black cultural racism measure had small sample sizes of survey respondents in most states within individual years. Consequently, we aggregated anti-Black cultural racism across years, rather than assigning each state a value of anti-Black cultural racism for the specific year in which an RCT was conducted. This approach likely introduces some measurement error, given that previous analyses of attitudinal data from Project Implicit⁴⁵ and from the General Social Survey⁴⁶ showed decreases in racial prejudice over time. However, in the General Social Survey, the ranking of each state’s measure of racial prejudice relative to other states was stable across time,⁴⁶ suggesting that states with higher anti-Black cultural racism in previous decades remain so today despite national declines in racial prejudice overall. Thus, while our time-invariant approach to measuring anti-Black cultural racism likely yields valid estimates, future research should determine whether modeling cultural racism as a time-varying moderator of intervention efficacy produces consistent associations.

Over the past several decades, there has been a growing consensus across the social sciences regarding the harmful mental health effects of anti-Black racism for Black people.^5,7 The consistency and robustness of this evidence has led some researchers to posit that racism itself is a fundamental cause of health inequities.⁴⁷ Our study makes a novel contribution to this literature by suggesting a previously overlooked way in which anti-Black racism, measured at the macro (cultural) level, may adversely impact the mental health of Black youth—by undermining their ability to benefit from psychotherapy interventions. In so doing, our results also uncover new avenues for future research at the intersection of cultural racism, developmental science, and mental health interventions.

Glossary

RH

Racism and Psychotherapy Efficacy