Therapist Effects on Disparities Experienced by Minorities Receiving Services for Mental Illness

Abstract

Objectives

The authors examine if some of the reason clients from racial and ethnic minority groups experience outcome disparities is explained by their therapists.

Method

Data from 98 clients (19% minority) and 14 therapists at two community mental health agencies where clients from racial and ethnic minority groups were experiencing outcome disparities were analyzed using hierarchical linear modeling with treatment outcomes at Level 1, client factors at Level 2, and therapists at Level 3.

Results

There were substantial therapist effects that moderated the relationship between clients’ race and treatment outcomes (outcome disparities). Therapists accounted for 28.7% of the variability in outcome disparities.

Conclusions

Therapists are linked to outcome disparities and appear to play a substantial role in why disparities occur.

Investigations of the relative contributions of therapists to client outcomes has grown significantly over the last 20 years (Crits-Christoph & Mintz, 1991; Elkin, Falconnier, Marinovic, & Mahoney, 2006; Kim, Wampold, & Bolt, 2006; Lutz, Leon, Martinovich, Lyons, & Stiles, 2007). Researchers interested in identifying therapist effects posit that, because therapy is a human endeavor, the individual providing a clinical intervention is likely to have at least some role in how clients experience the intervention and its outcomes (Elkin, 1999).

The theoretical underpinnings of therapist effects research suggest that some of the disparities in treatment outcomes experienced by clients from racial and ethnic minority groups may be explained at the therapist level. Expanding this premise to racial disparities research makes sense. In fact, much of disparities research in clinical settings assumes that there is a therapist effect on client outcomes. However, the focus of that research has been primarily on verifying the occurrence of racial disparities and secondarily, the effectiveness of the few therapist level interventions theorized to decrease racial disparities (Fine, Ibrahim & Thomas, 2005; Snowden, 2003; U.S. Department of Health and Human Services [USDHHS], 2001). Lutz et al. (2007) and others (Crits-Christoph & Mintz, 1991; Elkin, 1999; Elkin et al., 2006; Kim et al., 2006) have highlighted the long-standing trend in clinical research to focus predominately on efficacy and effectiveness of interventions, and to see therapist variability as a confounding factor that must be tightly controlled. The fact that this has also been the case in the literature concerning racial disparities in clinical outcomes is not surprising.

Outcome Disparities in Mental Health Services

Within the racial disparities literature is the implicit assumption that clients from racial and ethnic minority groups present unique challenges because of their cultural identities and experiences (Davis & Proctor, 1989; Ortega & Rosenheck, 2002; Ridley, 2005; Snowden, 2003; USDHHS, 2001). These differences can be obvious, as in the case of recent immigrants who may have limited English proficiency, or subtle and complex personal constructions of ethnic and racial identity. Regardless of their specific form, these differences are believed to be a key issue because the majority of licensed therapists are White (Davis & Proctor, 1989; Holzer, Goldsmith, & Ciarlo, 1998; Ridley, 2005; USDHHS, 2001). Evidence suggests that a number of factors embedded in larger social structures of society, including the community and agency, interact with therapists’ and clients’ behaviors and beliefs about mental health and race to produce outcome disparities (Applied Research Center & Northwest Federation of Community Organizations [ARC & NFCO], 2005; Borrell, 2005; Larrison, Schoppelrey, Hadley-Ives, & Ackerson, 2008; Miller, 1984; Snowden, 2005; Sue, 1977; Sue, Fujino, Hu, Takeuchi, & Zane, 1991; USDHHS, 2001). Theoretically, at least part of the reason for clients from racial and ethnic minority groups experiencing outcome disparities in clinical settings should rest at the therapist level.

Beyond the identification of disparities in services and client outcomes, two themes have dominated the racial disparities research in clinical settings: matching and multicultural counseling competency (MCC; Fine et al., 2005; Ridley, 2005; Worthington, Soth-McNett, & Moreno, 2007). Matching is the idea that a therapist’s ethnic background, something intrinsic to who they are, plays an important role in the effectiveness of interventions. By matching White therapists to White clients, African American therapists to African American clients, and so on, the barriers created by incongruent life experiences between White therapists and clients from racial and ethnic minority groups can be avoided, thus fostering a stronger therapeutic alliance (Shin et al., 2005). In contrast, MCC is an intervention directed at therapists with the aim of addressing the distance in life experiences and culture between clients from racial and ethnic minority groups and White therapists (Ridley, 2005; Worthington et al., 2007).

Both themes have attracted a significant amount of attention. Shin et al. (2005) conducted a meta-analysis of research concerning matching that included 10 published and unpublished studies between 1991 and 2001. All of the studies used naturalistic samples, with sample sizes ranging from 116 to 4,554, and all used outcome measures similar to those found in the therapist effects literature including functioning, treatment duration, retention, and total sessions (Shin et al., 2005). The studies generally showed that therapist/client dyads that were matched in terms of race and ethnicity did not differ significantly from unmatched dyads in terms of overall functioning, service retention, and total number of sessions. Shin et al. (2005) speculate that the recent drop in published articles concerning matching may be due to the weak support for it in the research literature.

Worthington et al. (2007) conducted a content analysis covering 20 years of MCC research. Findings across 75 studies indicate that there is some evidence that MCC improves therapist skills with clients from racial and ethnic minority groups. Work by Constantine (2001, 2002) that included a naturalistic sample and the strongest methods identified by Worthington et al. (2007) epitomized these positive findings. Two significant weaknesses confront the MCC literature. First, more than 70% of studies are descriptive field surveys and second, 80% relied on “pseudo-clients” leaving fundamental questions about the efficacy of MCC in naturalistic settings (Arredondo, Rosen, Rice, Perez, & Tovar-Gamero, 2005; Worthington et al., 2007). The findings from the MCC literature direct us to examine the effect of therapists on outcome disparities.

Therapist Effects on Treatment Outcomes

The difficulty of conducting research on therapist effects—most notably the need for data from multiple therapists with multiple clients actively engaged in therapy over a period of time—has thus far limited the nature of investigation in this area to measuring the magnitude of therapist effects. Even with the use of advanced statistical tools, two recent studies—both using the National Institute of Mental Health (NIMH) Treatment of Depression Collaborative Research Program (TDCRP) data (therapists = 17, clients = 119)—reached divergent conclusions regarding the salience of therapist effects (Elkin et al., 2006; Kim et al., 2006). One research team (Kim et al., 2006) attributed between 5% and 10% of the variance in client outcomes to therapist effects while the other (Elkin et al., 2006) concluded that therapist effects exerted no significant impact on client outcomes. The reasons for this divergence in findings using the TDCRP data set included different analytic choices made and their consequences (Crits-Christoph & Gallop, 2006; Lutz et al., 2007; Wampold & Bolt, 2007). Despite the analytic issues, these contrasting findings are consistent with the wide range of effect sizes (0–50%) identified by Crits-Christoph and Minz (1991) in their meta-analysis of 27 studies concerning therapist effects. One of the practical issues of interest in the Elkin et al. (2006) and Kim et al. (2006) studies was the utility of data from clinical trials in which the therapists providing interventions were trained extensively in an attempt to minimize the amount of extraneous variability they introduced (Crits-Christoph & Gallop, 2006). The fact that therapist variability remained despite these efforts has been cited as evidence of the robustness of therapist effects (Wampold & Bolt, 2007).

In a study that included a naturalistic sample of 1,198 clients and 60 therapists from a health maintenance organization, with outcomes measured at multiple time points within each client, Lutz et al. (2007) estimated that therapists accounted for approximately 8% of the total variance in outcomes and 17% of the variance in rates of patient improvement. Lutz et al. (2007) point out that their variance estimates are supported by two other projects (Okiishi et al., 2006; Okiishi, Lambert, Nielsen, & Ogles, 2003) that found substantial variability between therapists after controlling for intake and demographic variables of clients using a two-level analysis of data from similarly large naturalistic samples. Lutz et al.’s (2007) finding is consistent with Wampold and Brown (2005) and Kim et al. (2006) studies using multilevel modeling, which have estimated that between 5% and 10% of variance in client outcomes is explained at the therapist level and also with the 8.6% average variance in client outcomes identified in the meta-analysis by Crits-Christoph and Mintz (1991).

None of the literature exploring therapist effects has reported on racial or ethnic disparities in outcomes. Some of this may be due to low participation by clients from racial and ethnic minority groups (Collins & Elkins, 1985). Okiishi et al. (2003, 2006) report no participation by African American clients in their study, and 87% of the sample was White. The TDCRP sample was also overwhelmingly White (89%), with only 11% of clients being African American (Collins & Elkin, 1985). In the sample used by Lutz et al. (2007), 12.3% of the clients were racial and ethnic minorities, with most being either African American (6%) or Latino (4.5%). In their analysis, the client-level variable of race showed no significant connection to treatment outcomes.

Objective

With the exception of Sue’s (1977; Sue et al., 1991) and Constantine’s (2001, 2002) research supporting the effectiveness of MCC, there has been no research empirically identifying therapist effects in producing or ameliorating any disparities in treatment outcomes that might be experienced by clients from racial and ethnic minority groups in naturalistic settings. The goal of this research was to examine the relative effect of therapists on the outcome disparities experienced by clients from racial and ethnic minority groups receiving services at community mental health agencies (CMHAs). The primary interest was not in the general therapist effects on client outcomes (differences between therapists in the “average” client slope). Rather, our interest was in the degree to which therapist effects can explain outcome disparities (differences between therapists in the occurrence and magnitude of outcome disparities experienced by clients from racial and ethnic minority groups). There were two basic research questions yet unanswered in the scholarly literature: (a) Are therapists empirically associated with outcome disparities experienced by clients from racial and ethnic minority groups receiving community-based mental health services? (b) If therapists are empirically associated with outcome disparities, what is the size of their effect?

Setting

CMHAs are what remain of the federally funded community mental health centers created under the 1963 Community Mental Health Centers Act. CMHAs provide services that are selected from an array of services reimbursable by Medicaid, Medicare, and other third party payers. They serve a geographically defined catchment area and are expected to provide services for a wide range of individuals with a variety of mental health problems drawn from a population encompassing diverse racial, cultural, ethnic, and socioeconomic backgrounds. Approximately 45% of counties in the United States have access to a CMHA (Cutler, Bevilacqua, & McFarland, 2003; Tevis, 2003; Torrey, 2001; USDHHS, 1999).

The two CMHAs in the present study are located in the Midwest in a Core Based Statistical Areas (CBSA), a Census Bureau designation used for a geographical area containing a Metropolitan Statistical Area (MSA) with one or more adjacent rural areas that are integrated by close proximity and commuting (Johnson, 2003). The large rural areas surrounding the MSAs are part of a depressed agricultural and manufacturing region. The racial and ethnic distribution of population within the two MSAs is approximately 77% White, 12.5% African American, 0.5% Native American, 6% Asian American, and 4% Latino. Almost all of the population in the surrounding rural areas is White.

A preliminary examination of the pattern of outcomes for White clients and clients from racial and ethnic minority groups was conducted to determine whether outcome disparities existed. Not all CMHAs experience outcome disparities (Larrison et al., 2004); however, at these two agencies White clients and clients from racial and ethnic minority groups exhibited distinct patterns of change over time. White clients’ symptoms declined modestly over time and clients from racial and ethnic minority groups’ symptoms increased modestly over time (see Figure 1).

Figure 1.

Average change over time in BASIS-32 scores (N = 98). BASIS-32 = Behavior and Symptom Identification Scale-32.

Method

Participants

The naturalistic sample used was similar in size to that utilized by the National Institute of Mental Health (NIMH) Treatment of Depression Collaborative Research Program (TDCRP; therapist = 17, clients = 119; Elkin et al., 2006; Kim et al., 2006).

Therapists

Seventeen therapists assigned to clients in the study completed data collection. Of these 17 therapists, 14 were included at Level 3 of the hierarchical model; the remaining 3 therapists had too few clients for inclusion in the model. There were no obvious demographic differences between the therapists who remained in the analysis and those who did not. The average age of participating therapists was 40 years, and their median length of employment at the agency was 3 years. All but one participating therapist was licensed and two did not report their license status. Similar to the client participants, the therapists were primarily White (N = 11, 82%) and female (N = 9, 67%). Therapists from racial and ethnic minority groups self identified as African American (N = 2) or Asian American (N = 1).

Clients

Clients between the ages of 18 and 65 who were receiving services at the two CMHAs were eligible to participate in the study. Among the 124 clients recruited into the study, 98 clients had sufficient data for inclusion in the model (see Table 1 for demographic description of client participants). Clients with fewer than four observations (baseline and 3 follow-ups) were included in the analysis but clients with missing data for any predictor variables were excluded. The only statistically significant difference between clients who remained in the analysis and those who did not was client age, with those remaining in the analysis being somewhat younger than those who did not (M = 38.2 and 43.5, respectively; t = 2.165, df = 122, 2-tailed p = .033).

Table 1.

Client Demographic and Clinical Characteristics

	Total (N = 98)		Minority (N = 19)		White (N = 79)		χ2	t
	M	SD	M	SD	M	SD
Age	38.16	10.99	40.37	11.62	37.63	10.84		−0.97
Yearly income	$12,942	$12,173	$7,078	$9,391	$12,346	$11,898		1.64
Yearly income (median)	$6,948		$7,782		$6,450
BASIS-32 score (baseline)	1.26	0.77	1.08	0.79	1.30	0.76		1.13
	N	%	N	%	N	%
Female	79	80.6%	16	84.2%	63	79.7%	0.20
Some education beyond high school	62	63.3%	10	52.6%	52	65.8%	1.15
Employed	46	46.9%	9	47.4%	37	46.8%	0.00
Married	24	24.5%	2	10.5%	22	27.8%	2.49
Diagnosis							1.30
Schizophrenia	6	6.1%	2	10.5%	4	5.1%
Bipolar	26	26.5%	6	31.6%	20	25.3%
Depression	47	48.0%	8	42.1%	39	49.4%
Other	19	19.4%	3	15.8%	16	20.3%

Note: BASIS-32 = Behavior and Symptom Identification Scale-32. No differences were significant at the p < .05 (two-tailed) level.

Clients from racial and ethnic minority groups self-identified as African American (N = 14) and multiracial (N = 5). The two groups had similar demographic and service data as well as outcome trajectories.

Service data about the average length of appointments (M = .57, SD = .21) the total number of appointments (M = 78.40, SD = 182.97), and the total number of service hours during the previous 12 months (M = 59.35, SD = 159.79) indicate a wide range of service packages that varied depending on client needs. Independent sample T tests show that average length of appointments (White M = .59 and minority M = .51; t = 1.641, df = 76, 2-tailed p = .119), total number of service hours (White M = 58.73 and minority M = 64.63; t = −.146, df = 95, 2-tailed p = .885), and total number of visits (White M = 76.84 and minority M = 88.15; t = −.244, df = 95, 2-tailed p = .808) during the previous 12 months did not vary as a result of client racial status.

Clients also completed the Mental Health Statistical Improvement Program questionnaire (MHSIP). The MHSIP survey was developed in an effort to provide states with a consumer-oriented survey for adult clients with serious mental illness (Lutterman, Ganju, Schacht, Shaw, Monihan, et al., 2003). This 28-item survey (Version 1.1) is available at the MHSIP website (www.mhsip.org), where it is more fully described. Among the MHSIP’s five domains we examined accessibility, quality and appropriateness of care, and outcomes of care. The findings were similar to those found in a 16-state sample (Lutterman et al., 2003). The scores on access (White M = 7.67 and minority M = 7.12; t = .530, df = 85, 2-tailed p = .597), quality and appropriateness (White M = 11.38 and minority M = 16.00; t = 1.365, df = 83, 2-tailed p = .190), and positive outcomes (White M = 13.04 and minority M = 13.44; t = −.235., df = 84, 2-tailed p = .842), also did not vary by client race.

Data Collection Procedure

A member of the research team was located in the CMHAs at selected times Monday to Friday for the purpose of recruiting new clients into the study. Staff members at the CMHAs apprised clients of the study and asked if they would like to learn more about it. Clients who indicated an interest were introduced to the onsite member of the research team, who then provided an overview of the study and outlined the risks and benefits of participation. After obtaining client consent to participate, baseline data collection occurred and arrangements were made for follow-up data collection, which typically coincided with a future scheduled visit to the CMHA. Up to 3 follow-ups were conducted at 3-month intervals following the baseline measure. Sixty-two percent of clients contributed data at three or more points in time. Clients from racial and ethnic minority groups were slightly less likely than White clients to complete all four waves of data collection (36.8% and 44.3%, respectively), but this difference was not significant.

Therapists were invited to participate via mail and at one group meeting held at each site. They were matched with clients using administrative records, which were then cross-checked with client self-report of their therapist’s name. No inconsistencies were indentified between the two sources of information.

Data were collected between 2004 and 2007. All methods used in the project were approved by the Institutional Review Board at the University of Illinois at Urbana—Champaign.

Instruments

A self-report measure of functioning and symptoms developed for repeated-measure evaluations of clinical effectiveness was chosen as suggested in the therapist effects literature (Lutz et al., 2007). The dependent variable of treatment outcomes was measured using the Behavior and Symptom Identification Scale-32 (BASIS-32), a self-report instrument that typically takes 10–20 minutes to complete. The BASIS-32 is a brief (32 items) but comprehensive measure of self-reported difficulty in several major symptom and functioning domains (Eisen, Dill & Grob, 1994). It is empirically derived from psychiatric inpatients’ reports of symptoms and problems. It is used across the diagnostic spectrum and in both inpatient and outpatient settings to measure the outcome of interventions over time (Eisen et al., 1994). It has been normed with White clients and clients from racial and ethnic minority groups (Eisen, Gerena, Ranganathan, Esch, & Idiculla, 2006; Eisen, Normand, Belanger, Spiro, & Esch, 2004). Internal consistency reliability was very good for the participants in this study (Cronbach’s alpha = 0.95) and similar to that reported by Eisen et al. (1994; .89).

Respondents are asked to indicate the degree of difficulty they have been experiencing on each item during the past week. Degree of difficulty is rated on a 5-point scale ranging from no difficulty (0) to extreme difficulty (4). Each item contributes to one of five subscales and to an overall scale score, which is used in the present study. Subscale and overall mean scores can range from 0 to 4. Higher scores indicate more severe levels of symptomatology, and any score above one should be considered clinically significant (Eisen et al., 1994).

Clients’ clinical and service data, including DSM diagnoses and the dose and duration of treatment received, were gathered from administrative data accessed with written client consent. Race and ethnicity, as well as other client demographic variables such as gender, marital status, education, and employment status, were self-reported.

Because therapists were included only as a structural factor at Level 3 of the analysis, there are no measurement instruments to report (Menlo, Chaix, Yang, Lynch, & Rastam, 2005).

Data Analysis Plan

To answer the two research questions, a 3-level growth curve incorporating client characteristics was utilized (Okiishi et al., 2003, 2006; Lutz et al., 2007; Stulz et al., 2007). Specifically, data from the BASIS-32 were analyzed using a hierarchical linear model (HLM version 6.06) consisting of repeated measures of treatment outcomes at Level 1, client demographic and clinical factors at Level 2, and therapists as a structural factor at Level 3. Multilevel models provide important advantages over traditional regression methods, particularly in its handling of missing observations and utility in modeling individual growth curves (Byrk & Raudenbush, 2002; Kreft & de Leeuw, 1998; Singer & Willett, 2003).

Model specification

Decisions regarding model specification were informed by cogent discussions of similar multilevel analyses in the therapist effects literature (Crits-Christoph & Gallop, 2006; Kim et al., 2006; Lutz et al., 2007; Serlin, Wampold, & Levin, 2003; Wampold & Bolt, 2007). First, as suggested by a number of authors (Kim et al., 2006; Lutz et al., 2007; Serlin et al., 2003), the therapists were treated as random rather than fixed effects. Second, similar to Lutz et al.’s (2007) study design, a log-linear model of time (defined by measurement occasion) provided the best fit to the data rather than a simple linear model. Third, no predictors of the client level intercept were included at either Level 2 or Level 3. Fourth, although the intercept was not of primary interest, it was allowed to vary across clients and therapists rather than following the anchoring approach used by Lutz and colleagues (2007). In addition to the compelling critique of anchoring provided by Wampold and Bolt (2006, 2007), we believe there may be important variability in client intercepts that should not be ignored.

In order to estimate therapist effects on outcome disparities, first the variability across clients in terms of their outcomes (indicated by client slopes) was estimated (see Equation 1). Next, all potential demographic and clinical predictors at the client level (gender, minority status, and diagnosis) were examined to determine whether they were significantly related to client outcomes (see Equations 2 and 3).

$$Y=b_0+(b_1\ast \text{TIME})+e.$$ (1)

$$b_0=g_{00}+r_0.$$ (2)

$$b_1=g_{10}+(g_{11}\ast \text{MINORITY})+(g_{12}\ast \text{BIPOLAR})+(g_{13}\ast \text{MARRIED})+r_1.$$ (3)

$$g_{00}=u_{00}.$$ (4)

$$g_{10}=u_{10}.$$ (5)

Each predictor—as well as the client outcomes—was also examined to determine whether its effect differed systematically between therapists (see Equations 4 and 5). Finally, for any client factors that differed between therapists, the size of the therapist effect relative to the size of the client factor effect was calculated. During the model-building process, residuals were examined to identify potentially serious violations of assumptions (Byrk & Raudenbush, 2002).

Results

Unmoderated Therapist Effect

The result of the first analysis, representing the unconditional therapist effects, is presented in Table 2. The estimated average BASIS-32 score at initial measurement was 1.25 (indicated by g₀₀ in Table 2). This intercept varied significantly across clients (indicated by an r₀ coefficient that differed significantly from zero). We did not attempt to model any predictors of this variability and used it merely as a case adjustment factor. All subsequent analyses control for variation between clients in initial level of symptomotology.

Table 2.

Model of General Therapist Effects

Fixed Effects	Coefficient	SE	Approx. df	t	p
Client-level intercept, b0
Intercept, g00	1.25	0.08	13	16.48	.00
Client-level slope, b1
Intercept, g10	−0.61	0.15	13	−4.09	.00
MINORITY effect, g11	0.59	0.30	94	1.97	.05
BIPOLAR effect, g12	0.44	0.24	94	1.87	.06
MARRIED effect, g13	0.60	0.24	94	2.47	.02
Random Effects	Variance Component		df	Chi-Square	p
Level 1: Repeated measures within clients
Temporal variation error, e	0.11
Level 2: Clients within therapists, N = 98
Client intercept, r0	0.46		55	355.04	.00
Client slope, r1	0.13		52	69.07	.06
Level 3: Between therapists, N = 14
Therapist mean intercept, u00	0.00		13	12.27	>.50
Therapist mean slope, u10	0.00		13	7.82	>.50

The average treatment outcome (client slope) was interpreted with reference to a hypothetical client who falls in the reference category for all predictors: race/ethnicity (with White serving as the reference category), marital status (with married serving as the reference category), and diagnosis (defined as the presence [1] or absence [0] of a diagnosis of bipolar disorder). Because service data were highly variable and bivariate analyses indicated it did not vary by race, it was excluded from the client-level model. Therefore, the estimated average outcome for a White, unmarried client with a diagnosis other than bipolar disorder was −0.61 (see g₁₀ in Table 2), which represents a 12% improvement in functioning on the 5-point BASIS-32 scale. Being a member of a minority group nearly eradicated those gains, as the size of the minority effect 0.59 (see g₁₁ in Table 2) was approximately identical to the average client slope.

Next the proportion of variability in treatment outcomes attributable to the therapist was considered. The formulation proposed by Lutz and colleagues (2007) for estimating the variability in client-level growth curve parameters separately (slope and intercept) was followed. The proportion of variance in client outcomes attributable to therapists was expressed as the proportion of variance that lies between therapists (u₁₀) to the total variance in client slopes (u₁₀ + r₁; Byrk & Rauden-bush, 2002; Singer & Willett, 2003):

$${\rho }_{10}=\frac{u_{10}}{\left(u_{10}+r_1\right)}.$$ (6)

Although only marginally significant, there appears to be substantial unexplained variability in client outcomes (r₁ = 0.1296) even after including therapists in the model and accounting for clients’ initial level of severity, demographic and clinical factors. The proportion of variability in client outcomes that can be attributed to therapist effects was quite small (0.00018/0.1295 + 0.00018), at 1.4%. Wampold and Bolt (2006) caution that because increasing the amount of variability at the client level necessarily reduces the variability available to be modeled at the therapist level, longitudinal analyses risk underestimating therapist effects.

Moderated Therapist Effect

The objective of the research was not to identify the presence or absence of a general therapist effect but rather to determine whether there was a therapist effect related to outcome disparities. The result of the second analysis, representing the moderating effect of therapists on the relationship between clients from racial and ethnic minority groups and treatment outcomes, is presented in Table 3.

Table 3.

Model of Therapist as Moderator of Outcome Disparities

Fixed Effects	Coefficient	SE	Approx. df	t	p
Client-Level Intercept, b0
Intercept, g00	1.25	0.08	13	15.37	.00
Client-Level Slope, b1
Intercept, g10	−0.62	0.14	13	−4.36	.00
MINORITY effect, g11	0.53	0.40	13	1.32	.21
BIPOLAR effect, g12	0.54	0.23	94	2.34	.02
MARRIED effect, g13	0.61	0.23	94	2.66	.01
Random Effects	Variance Component		df	Chi-Square	p
Level 1: Repeated measures within clients
Temporal variation error, e	0.11
Level 2: Clients within therapists, N = 98
Client intercept, r0	0.45		41	277.29	.00
Client slope, r1	0.03		31	36.21	.24
Level 3: Between therapists, N = 14
Therapist mean intercept, u00	0.01		7	5.50	>.50
Therapist mean slope, u10	0.01		7	3.98	>.50
OUTCOME disparities, u11	0.69		7	14.72	.04

The fixed portion of the model differed modestly from the unmoderated model, with the estimates being quite similar for minority status (g₁₁) and being married (g₁₃). The estimated size of the effect of having a diagnosis of bipolar disorder (g₁₃) increased from approximately 0.44 to 0.54; this effect was not previously statistically significant but achieved significance in the moderated model (see Table 3).

Being a member of a racial and ethnic minority group did not exert a direct effect on client outcomes (g₁₁); however, it was a source of variability between therapists (u₁₁; see Table 3), resulting in the addition of the u₁₁ term to represent outcome disparities in Equation 5. The variability among therapists in the occurrence and magnitude of outcome disparities between White clients and clients from racial and ethnic minority groups was statistically significant (u₁₁ = 0.6907, p = .039). This indicates that some therapists had equally good outcomes (defined as BASIS-32 scores trending downward, toward less severe symptoms) for clients from racial and the ethnic minority and White clients while other therapists had outcome disparities between clients from racial and ethnic minority groups and White clients (defined as BASIS-32 scores trending upward toward more severe symptoms for clients from racial and ethnic minority groups and downward toward less severe symptoms for White clients). These differences among therapists in the occurrence of outcome disparities are net of all the other effects in the model, including the fixed effects of client race/ethnicity, diagnosis, and marital status as well as their initial level of symptomotology. The proportion of variability in outcome disparities attributable to therapist is calculated in the same manner as before (0.0130/(0.0130 + 0.0323)). The moderated model yields an estimate of 28.7%, indicating that therapist accounted for nearly a third of the variability in outcome disparities experienced by clients from racial and ethnic minority groups.

Discussion and Applications to Social Work

In the present study, including therapists as a structural factor in a three-level growth curve analysis moderated the relationship between clients from racial and ethnic minority groups and treatment outcomes. The finding provides empirical evidence that therapists are associated with outcome disparities. The magnitude of the therapist effects on outcome disparities (28.7%) indicates that therapists are a critical component of the clinical process for clients from racial and ethnic minority groups receiving mental health services in community-based settings. These findings are consistent with the conceptual underpinnings of MCC, which posits that therapist effects are a significant factor in how clients from racial and ethnic minority groups experience clinical interventions and the outcomes of those interventions (Constantine, 2001, 2002; Miller, 1984; Snowden, 2005; Sue, 1977; Sue et al., 1991; USDHHS, 2001).

Much of disparities research in mental health services has emphasized identifying the existence of disparities experienced by clients from racial and ethnic minority groups. This research is significant because it modeled the role of therapists in outcome disparities. By doing so, the research moved beyond identifying the occurrence of outcome disparities toward understanding the reasons for disparities. The findings affirm that interventions aimed at therapists are likely to remain an essential part of reducing the outcome disparities experienced by clients from racial and ethnic minority groups.

The most significant limitation of the study is the small number of therapists at Level 3, which is a weakness shared by a number of studies in the therapist effects literature (Elkin et al., 2006; Kim et al., 2006). Seventeen therapists participated in the study, but some were excluded because they served only one or two clients participating in the study, which does not permit a stable estimation of therapist effects. The analysis presented here includes 14 therapists, making it impossible to model additional variables at the therapist level (Dawson et al., 2002; Van Secker & Lissitz, 2002). The study remains worthwhile despite this limitation due to the lack of research linking therapist effects with outcome disparities experienced by clients from racial and ethnic minority groups.

Because of the small sample size, clients who identified themselves as African American and multiracial were included as a single group in the analysis. Although clients from both groups had similar outcome trajectories, this grouping does risk obscuring potentially important differences in the experiences of African American and multiracial clients. However, this grouping did allow us to model the occurrence of outcome disparities and to estimate the proportion of those disparities that can be linked to therapists. Further studies with larger naturalistic samples are clearly needed to refine our understanding of how therapist effects impact the outcomes of mental health services for clients who are members of various racial and ethnic groups.

There is considerable disagreement in the literature as to the implications of a naturalistic setting for internal and external validity. Some argue that naturalistic settings may introduce serious confounds created by the lack of random assignment of clients to therapists (Elkin et al., 2006; Lutz et al., 2007). Clinical trials, conversely, may systematically underestimate therapist effects due to their reliance on manualized and standardized interventions, which do not accurately represent the conditions under which most mental health services are delivered (Elkin et al., 2006; Lutz et al., 2007). This particular analysis included clients and therapists operating in CMHAs, a setting in which the therapists are often times professional social workers. Most clients lived below the poverty line, relied on Medicaid, had chronic long-term mental health problems, and significant service histories that differ from other naturalistic samples drawn from health maintenance organizations data. The strength of using clients from CMHAs is also its major weakness. Clients receiving services from CMHAs are typically engaged in long-term treatment that ebbs and flows depending on the needs of the client and their level of functioning at any given time. This is why examining patterns of functioning and symptoms over time rather than successful completion of treatment is the focus of outcomes in our analysis.

While there is uncertainty in terms of the relative strengths of naturalistic studies and clinical trials for internal validity, it seems clear that naturalistic studies typically permit far greater external validity than do clinical trials (Crits-Christoph & Gallop, 2006; Elkin et al., 2006; Lutz et al., 2007; Okiishi et al., 2003). We found no evidence of bias in assignment of clients to therapists at the participating CMHAs, but this was not explicitly assessed in the study. Assignment of clients to therapists was based primarily on the therapists’ existing case-loads and their availability. There was no evidence for significant selection effects in the staff participants, as all agency staff were invited to participate, and therapists were included based on having clients who participated in the study. When considering the role of therapist effects in outcome disparities for clients from racial and ethnic minority groups, naturalistic settings may provide the better research environment. As others have suggested (Crits-Christoph, Tu, & Gallop, 2003; Elkin et al., 2006; Lutz et al., 2007), more investigation is needed to understand the differences between clinical trials and naturalistic settings.

Expanding the methods used in therapist effects research to better understand the role of therapists in outcome disparities appears to be productive. For the first time, therapists have been empirically linked to outcome disparities experienced by clients from racial and ethnic minority groups and the magnitude of their effect estimated. These are significant preliminary steps in understanding the reasons for why outcome disparities occur at CMHAs. The suggestion offered by Lutz et al. (2007) to expand research on therapist effects to better understand the specific therapist characteristics or behaviors that contribute to variations in client outcomes can also be applied to understanding the relationship between therapists and outcome disparities for clients from racial and ethnic minority groups. Further research including greater numbers of therapists and clients from racial and ethnic minority groups is necessary to determine whether the relationship between therapist and outcome disparities is a function of identifiable therapist characteristics, such as race/ethnicity, gender, or level of multicultural counseling competency.