Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Jul 2.
Published in final edited form as: J Emot Behav Disord. 2010 Aug 27;19(1):3–16. doi: 10.1177/1063426610376773

Ethnic Similarity, Therapist Adherence, and Long-Term Multisystemic Therapy Outcomes

Jason E Chapman 1, Sonja K Schoenwald 1
PMCID: PMC4078892  NIHMSID: NIHMS591722  PMID: 24999302

Abstract

The current study investigated relations among ethnic similarity in caregiver-therapist pairs of youth participating in Multisystemic Therapy, therapist adherence, and youth long-term behavioral and criminal outcomes. Participants were 1979 youth and families treated by 429 therapists across provider organizations in 45 sites. Relations were found, independently, and in the presence of ethnic similarity, between adherence and reductions in youth Externalizing and Internalizing behavior problems 1-year post-treatment and youth criminal charges 4 years post-treatment. Relations between ethnic similarity and outcomes were found only for reductions in youth Externalizing behavior problems and not when adherence was included in the model. Adherence ratings were higher, however, in ethnically similar caregiver-therapist pairs, and evidence was found that this increased adherence predicted slightly better outcomes for youth. Implications for future research and clinical practice are considered.

In children’s mental health, family-based treatments for adolescents with serious antisocial behavior, substance use problems, and co-occurring antisocial behavior and substance use were among the first to be tested in effectiveness trials that included substantial portions of ethnic minority youth, and to be transported to usual care settings. Because the transport of evidence-based treatments (EBTs) has been relatively recent and limited, however, little is known about the extent to which the benefits of treatment across ethnic groups demonstrated in efficacy and effectiveness trials (see, e.g., Huey & Polo, 2008; Miranda, Bernal, Lau, Kohn, Hwang, & LaFromboise, 2005) will extend to their implementation and outcomes in usual care settings.

Two lines of inquiry are particularly pertinent to the implementation and effectiveness in routine care of EBTs with ethnically diverse youth and families. First, theory and research on the dissemination and implementation of innovations suggests challenges to the adoption and implementation of such treatments arise at multiple levels of the practice context, including the practitioner, service provider organization, and service system (Schoenwald & Hoagwood, 2001). The extent to which variation in such practice context variables not present in efficacy trials might differentially affect the implementation and outcomes of EBTs with ethnic minority youth is unknown. Second, research on cultural adaptations of evidence-based treatments and culture specific treatments, although limited and characterized by significant methodological problems, has not yet supported the superiority of these adaptations or treatments (Huey & Polo, 2008).

At the same time, little is known about the mechanisms of evidence-based treatments that account for the generalization of positive treatment effects across ethnically diverse samples. In a review of the evidence base on the effects of EBTs with ethnic minority youth and families, Huey and Polo propose some treatment models reflect inherent consideration of cultural differences despite the fact they do not specify culture-specific protocols. They cite as an example the framework of treatment principles provided by Multisystemic Therapy (MST; Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 2009), which specifies that clinicians respond to the unique circumstances of the individual client and context. The second framework, exemplified by the Youth-Partners-in-Care study (Asarnow et al., 2005), is a “quality improvement” strategy designed to enhance usual care with training and resources to encourage clients and clinicians to select CBT as a treatment intervention for depression, and the strategy was successful in prompting ethnic minority youth to obtain evidence-based treatment.

Although moderation of treatment effects by ethnicity has not been found for EBTs for youth with serious antisocial and substance abuse problems, some findings suggest ethnic similarity among client and practitioner may affect the short-term outcomes of some ethnic groups. Specifically, secondary analyses of data from three studies of the effects of Functional Family Therapy adolescents referred to treatment for drug use produced mixed findings regarding the effects of ethnic similarity on youth drug use (Flicker, Waldron, Turner, Brody, & Hops, 2008). Hispanic and Anglo youth exhibited statistically significant reductions in drug use from pre-treatment through post-treatment and 4-month follow-up assessments. The magnitude of the reductions was not moderated by youth ethnicity. In ethnically similar therapist-family pairs, however, Hispanic youth exhibited greater reductions than Anglo youth. Ethnic similarity did not predict outcomes for Anglo youth. The authors acknowledged significant study limitations rendered the evaluation exploratory, and suggested studies of larger samples with greater representation of other ethnic groups are needed to evaluate the impact of ethnic similarity on youth outcomes of evidence-based treatments and mechanisms by which such impact, if observed, is exerted.

A larger sample size and representation of a greater number of ethnic minority groups characterized the youth and therapist sample in a multi-site, prospective, non-controlled evaluation of factors affecting the implementation and outcomes of MST in usual care settings. Consistent with the findings from the Functional Family Therapy samples (Flicker et al., 2008), and with findings from randomized effectiveness trials of MST, youth outcomes in this multi-site were not moderated by youth ethnicity (Halliday-Boykins, Schoenwald, & Letourneau, 2005). However, therapist reports of youth discharge status and caregiver reports of youth problem behaviors six months post treatment were more favorable when ethnic similarity characterized therapist-caregiver pairs; and, these effects were partially mediated by caregiver reports of greater therapist adherence to MST.

The objectives of the current investigation were to evaluate the effects on longer-term outcomes of MST – youth behavior and functioning problems through one-year post treatment, and youth criminal activity through four years post-treatment -- of ethnic similarity in caregiver-therapist pairs; and evidence of mediation by therapist adherence of those effects. Results are expected to illuminate the extent to which the long-term outcomes of MST are robust to ethnic similarity and dissimilarity, and therefore the extent to which research is needed to identify treatment processes that differentiate ethnically similar and dissimilar pairs and strategies to ensure appropriate processes are used to achieve equally good outcomes across ethnic groups. The evaluation is also an example of methods that can be used to empirically discern ethnicity-related boundary conditions of the implementation and outcomes of other EBTs in routine care.

Method

Participants

Youth and caregivers

A total of 1,979 youth and caregivers participated. The mean age for youth was 14.0 (SD = 2.35), and most were male (65.0%) and Caucasian (59.5%), with 19.3% of youth identified as African American, 6.4% Asian or Pacific Islander, and 14.8% other. The majority of the youth were not of Hispanic ethnicity (92.7%). Almost half of youth resided with their mother or mother and a significant other (48.6%), with remaining youth residing with both parents (15.5%), their father or father and a significant other (7.1%) or alternating between parents’ households (.3%), in special living arrangements (16.9%,), with a foster family (3.3%), or in other non-institutional settings (8%). The primary referral sources to treatment for youth were juvenile justice or corrections agencies (44.2%), social services (23.0%), mental health agencies (17.6%), or other agencies (15.1%). The most frequent referral reasons (multiple reasons could be endorsed for a given youth) included status offenses (47.4%), criminal offenses (46.7%), substance use problems (31.3%), and school suspensions or expulsions (29.8%). Mean caregiver age was 40.8 years (SD = 8.48), and most were female (87.8%). Most caregivers were Caucasian (65.0%). Remaining caregivers were African American (18.8%), Asian or Pacific Islander (6.2%), American Indian or Alaskan Native (0.9%), mixed heritage (4.0%), or “other” ethnicity (0.4%). Most caregivers were not of Hispanic ethnicity (95.1%). Half of caregivers (50.0%) reported annual incomes under $20k. Thus, the sample resembled samples in randomized trials demonstrating the effectiveness of MST for serious antisocial behavior.

Therapists

Primary therapists (n = 429) were identifiable for 1,888 (95.4%) of families. “Primary therapist” signifies the therapist treating the family for the entire treatment episode or, for families treated by more than one therapist, the therapist providing treatment for the majority of the family’s treatment episode. There were 91 families treated by more than one therapist for approximately equal lengths time, such that no therapist was designated as primary for these families. Differences between families with and without a primary therapist have not been found (Schoenwald, Chapman, & Sheidow, 2006). The majority of therapists were female (74%) and held master’s degrees (61%; 32% held bachelor’s degrees; 3% held doctoral degrees; 3% held unspecified degrees). Therapist ethnicity was reported as Caucasian (74.9%), with 14.4% African American, 6.0% Asian or Pacific Islander, 2.1% Latino, 0.6% American Indian or Alaskan Native, 1.0% mixed ethnic heritage, and 1.0% indicating “other” ethnicity.

Procedures

Study procedures have been detailed previously (see Schoenwald, Sheidow, Letourneau, & Liao, 2003; Schoenwald, Sheidow, & Letourneau, 2004) and are briefly described here. Service provider organizations in 45 sites across 13 states and a Canadian province participated in the study. All youth referred for MST treatment at the study sites were eligible except youth with autism or severe mental retardation. Families were recruited for study participation by clinical supervisors or therapists at the provider organizations upon referral, prior to treatment. The family consent rate was 82%, and only two therapists declined to participate. Informed consent from therapists and supervisors was obtained by investigators during site visits and via telephone for therapists employed after the study began. Research assistants administered pre, post, and follow-up assessment measures to caregivers and therapists by telephone; as well as the therapist adherence measure to caregivers. The research assistants also obtained youth and family demographic information during the pre-treatment assessment. Therapists reported on referral information (agency referring the youth, reasons for referral) and discharge circumstances (who made the discharge decision, reasons for discharge) on standardized forms used by MST programs. Caregivers were reimbursed for completed assessments. Longer-term follow-up data (2 – 4 years post-treatment) on youth criminal activity were obtained from court, juvenile justice, and adult criminal justice archives. Family, therapist, and supervisor participation in the study was voluntary and the Institutional Review Board of the university approved all procedures.

Clinical intervention

Because details of the clinical intervention have been described elsewhere (Henggeler et al., 1998) the brief description here recaps information most central to understanding its potential suitability to youth and families with diverse ethnic and cultural backgrounds. MST is an intensive, family-based treatment originally developed for delinquent youths at imminent risk of incarceration or other out-of-home placements and their families that specifically targets those factors in each youth’s social ecology (family, peers, school, neighborhood, and community) contributing to his or her antisocial behavior. MST treatment is informed by the social ecological theory of human behavior articulated by Bronfenbrenner (1979) and by prospective research identifying the multiple predictors of serious antisocial and related behavior in adolescents. Given the youths’ imminent risk of placement, overarching treatment goals often relate to keeping the youth in the home and reducing criminal behavior. Specific goals and the interventions to achieve them are designed collaboratively with the youth’s caregivers, who also implement the majority of the interventions, initially with the instrumental and social support of the therapist and subsequently with indigenous sources of support for specific tasks (e.g., monitoring the youth’s whereabouts after school when a caregiver is at work) emanating from relatives, neighbors, and friends, as befits the particular kinship and social network of the family. Thus, MST strives to empower parents with the skills and resources needed to independently and successfully negotiate the social systems within the youth’s social environment in ways that minimize and manage risk factors for serious antisocial behavior and maximize protective factors.

The combination of intervention techniques applied and the expected impact of intervention procedures vary in accordance with the circumstances, strengths, and needs of each youth and family. To balance adequate specification of the model with responsiveness to the needs and strengths of each youth and family, nine principles are used to guide the MST assessment and intervention process. Ongoing assessment and intervention proceeds in accordance with an analytic process that encourages clinicians to generate specific hypotheses about the combination of factors that sustain a particular problem behavior, provide evidence to support the hypotheses, test the hypotheses by intervening, assess the impact of the intervention, and begin the assessment process again. Interventions typically include improving specific caregiver discipline practices, enhancing family affective relations and reducing conflict, decreasing youth association with deviant peers, increasing youth association with prosocial peers and activities, improving youth school or vocational performance, and fortifying the indigenous support network of extended family, neighbors, and friends to help caregivers achieve and maintain such changes. Specific treatment techniques used to facilitate these gains are integrated from those therapies that have the most empirical support, including cognitive behavioral, behavioral, and pragmatic family therapies. Because the MST treatment model emphasizes developing therapists’ capacities to work with the unique instantiation of risk and protective factors presented by each youth and family referred for treatment, families are assigned to the next available therapist without consideration of family or youth characteristics, including ethnicity, youth gender, referral characteristics or problem severity.

A home-based model of service delivery provides comprehensive and intensive clinical interventions when and where they are needed (i.e., clinicians are available 24 hours/day, 7 days/week to respond to crises). Duration and frequency of treatment sessions vary in accordance with changing circumstances, needs, and treatment progress. Teams of 3–4 MST therapists have supervisors who devote at least half their time to this role and receive training in the MST supervisory protocol from expert consultants. Each therapist carries a caseload of 4 – 5 families, and treatment length averages 3– 5 months. Average length of treatment for the current sample was 22.2 weeks (SD = 10.4).

Implementation protocol

As detailed elsewhere (Schoenwald, 2008), a comprehensive quality assurance system designed to replicate procedures and resources provided therapists in randomized trials is used to support MST transport. Early work with communities seeking to import MST indicated systematic implementation support at several levels of the practice context would be essential to achieving the outcomes obtained in MST research; as workforce, organizational, and fiscal exigencies challenged adherence to MST clinical protocols. These experiences were consistent with theory and research on technology transfer (e.g., Backer, David, & Soucy, 1995), organizational implementation of innovations (e.g., Klein & Knight, 2005; Klein & Sorra, 1996), and Kolb’s (1984) experiential learning theory, which converge on the notion that differentiated but coordinated strategies are needed to enable individuals and organizations to effectively and consistently implement a new technology. The resulting quality assurance system of six elements: (a) Site assessment; (b) 5-day orientation for therapists and clinical supervisors;(c) on-site clinical supervision guided by a the supervision protocol (Henggeler & Schoenwald, 1998); (d) weekly consultation with an MST expert trained in a consultation protocol (Schoenwald, 1998a); (e) quarterly booster training; and (f) feedback on measures of therapist and supervisor adherence to MST protocols. In addition, ongoing organizational consultation and semi-annual formal program reviews are provided to assess and address organizational, service system, and purveyor (i.e. MST expert-related) barriers to achieving integrity of treatment implementation and favorable youth outcomes. This system is deployed through MST Services, LLC, a university-licensed technology transfer organization.

Measures

Child Behavior Checklist (CBCL; Achenbach, 1991)

Youth behavior problems were assessed by the caregiver-reported CBCL collected at pretreatment (T1), immediately post-treatment (T2), 6 months post-treatment (T3), and 12 months post-treatment (T4). The CBCL is one of the best-validated measures of child behavioral functioning and has been normed with various age and ethnic groups (Achenbach, 1991; Drotar, Stein, & Perrin, 1995). The measure describes 113 behavior problem items applicable to children aged 2 to 18 years. Caregivers are asked to rate the extent to which the description is true of their child during the previous 6 months on a scale that ranges from 0, “not true,” to 2, “very often or often true.” T-scores for the broadband Externalizing and Internalizing scales were analyzed. For each of these scales, a T-score of 60 is the borderline clinical cutoff and a T-score of 64 is the clinical cutoff.

Vanderbilt Functioning Inventory (VFI; Bickman, Lambert, Karver, & Andrade, 1998)

Psychosocial functioning was assessed using the VFI. Content areas indexed by the 24-item VFI are antisocial behavior, problems at home, problems at school, problems with peers, and self-harm. Analyses of the reliability and validity of the VFI indicate adequate internal consistency (.71), concurrent validity (e.g., significant correlations with established measures in the expected directions), predictive validity (e.g., VFI scores predicted cost of treatment and use of residential care), and incremental validity (e.g., VFI scores accounted for a significant portion of variance of treatment cost and residential care after accounting for the variance accounted for by other measures) (Bickman et al., 1998). VFI probability scores are computed by summing raw item scores (0 or 1) and dividing by the number of completed items. Thus, scores can range from .00 to 1.00, and we observed a baseline mean of .42 (SD = .20, Mode = .45).

Criminal charges

Criminal charge data were obtained for 1,791 (91% of the entire sample) youth, across a mean post-treatment follow-up period of 49.3 months (SD = 8.9), with a range of 24.7 to 68.2 months. Of these youth, 1,713 (96%) had an identifiable primary therapist. For all youth participants, criminal charge data were obtained from county and state juvenile justice agencies and courts. For youth participants who had reached adulthood at the time of the follow-up request, adult charge data were obtained via public record searches available through the Internet, or from agencies housing adult criminal records. Raw data were obtained on the dates, types, and severity of lifetime pre-treatment charges and charges accrued throughout the follow-up period. These data were coded by research staff to reflect charge types (i.e., person, property, drug, public order, status or other offense) and charge severity levels (e.g., murder was rated as the most severe, other types of person offenses were rated as next most severe; status offenses were rated least severe, and within status offenses “incorrigible/ungovernable behavior” was rated the least severe). The coding scheme was based on coding systems used in early studies of MST (Hanson, Henggeler, Haefele, & Rodick, 1984); these systems, in turn, were based on the Uniform Crime Reports standards used by the Federal Bureau of Investigation. An ongoing study using the same coding scheme documents 98.6% agreement across blind raters on individual charges and inter-rater agreement at 96.4% (Letourneau, 2006).

Of the 1,713 youth for whom criminal justice records were obtained and for whom a primary therapist was identifiable, 1,267 (71%) had at least one known charge (which could have occurred pre-, during-, or post-treatment), and of these, 978 (77%) had at least one charge during the follow-up period. Information on criminal charges could not be obtained for 188 (9.0%) of the 1,979 participants in the entire sample. Most of these participants (n = 178) were treated in jurisdictions that ultimately were unable to provide any juvenile justice data, despite initial agreements to do so.

Family demographic characteristics

Demographic information such as caregiver and youth age, sex, ethnicity, education level, family income, and family composition was collected using a demographic questionnaire adapted for multi-site use from the questionnaire used in previous trials of MST.

Caregiver ethnicity

Caregivers were asked to indicate their ethnic group by selecting from 21 mutually exclusive categories. The categories included 5 single-group options of Black/African-American, Asian or Pacific Islander, American Indian or Alaskan Native, Latino, and White, as well as 15 options indicative of mixed ethnic heritage (e.g., African American and White; Latino and African American) and an option for other ethnic group. Because of the small numbers of caregivers who selected mixed heritage options (4.0%), the American Indian or Alaskan Native option (0.9%), and the “other” option (0.4%), these categories were collapsed into a single “other” category for the purpose of analyses, leaving four single-ethnicity groups and the “other” group. In 4.9% of cases, the ethnicity of the caregiver differed from that of the youth. We elected to focus on caregiver ethnicity rather than youth ethnicity because MST interventions are implemented primarily by caregivers, and thus likely to be more strongly affected by the caregiver’s ethnic background than that of the youth.

Therapist ethnicity

Therapists were asked to indicate their ethnic group using the same strategy employed for caregiver ethnicity, on the Personnel Data Inventory (Schoenwald, 1998b).

Caregiver-therapist ethnic similarity

Each caregiver’s ethnic similarity with the therapist was determined on the basis of the caregiver’s and the therapist’s endorsement of the 21 ethnicity options (i.e., before mixed heritage and American Indian/Alaskan Native groups were collapsed into the “other” category). Thus, although multiethnic and American Indian individuals were collapsed into a single group for the purpose of analyses, we used information on the specific ethnicities endorsed for each individual within the group to determine ethnic similarity. If the therapist and caregiver each indicated the same single-ethnicity option, the pair was coded as similar. If the therapist or caregiver (or both) chose a mixed heritage option, the pair was scored as similar if either of the therapist ethnicities matched either of the caregiver ethnicities. For example, if the therapist indicated “Asian and White” and the caregiver indicated “African American and Asian” the pair matched on Asian and was coded as similar. Sixty-six percent of all caregivers were ethnically similar to their therapists, although there was substantial variability across ethnic groups, with 86% of Caucasian, 28% of African American, 37% of Asian American, 9% of Latino, and 49% of other caregivers being similar to the therapist. We recognize that ethnic similarity does not necessarily reflect perfect matches (e.g., if a Japanese therapist and a Vietnamese caregiver both endorse Asian, they would be considered similar) and will address this issue in the Discussion.

MST Therapist Adherence Measure –Revised (TAM-R Henggeler, Borduin, Schoenwald, Huey, & Chapman, 2006)

Therapist adherence was assessed monthly during treatment using caregiver reports on the TAM-R. The TAM-R is a 28-item scale developed by expert consensus to assess therapist adherence to the nine principles of MST. The 28-item scale retains 19 of the 26 items of the original MST Therapist Adherence Measure (TAM; Henggeler & Borduin, 1992), validated in randomized clinical trials of MST. TAM items were rated on a 5-point Likert-type scale, with response options ranging from “Not at all” to “Very much.” Although caregiver, therapist, and youth reports on the measure were obtained in past MST trials, caregiver reports were the better predictors of youth outcomes (Schoenwald, Henggeler, Brondino, & Rowland, 2000). Multi-factor structures characterized by some instability across samples emerged in the first trials to use the measure (Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Henggeler, Pickrel, & Brondino, 1999); and reliability and confirmatory factor analyses from the much larger and more diverse sample of caregivers and therapists in the MST Transportability Study supported a single-factor solution (Schoenwald, et al., 2003; Schoenwald et al., 2005). The Transportability Study also included 12 new items that indexed whether treatment sessions focused on important aspects of the youths’ school, peer, and neighborhood/social support systems, consistent with the MST model. A comprehensive evaluation of the original 26 items and the 12 new items conducted using a Rasch-based approach to scale development retained 19 of the original TAM items and 9 of the new items onto a single factor and a two- rather than five-point rating scale. Consistent with psychometric evaluation of the single-factor TAM, TAM-R ratings were stable within a family’s treatment episode. Relations have been established between the TAM-R and youth behavioral and criminal outcomes (Schoenwald, Carter et al., 2008; Schoenwald, Chapman, et al., 2008) and supervisor adherence to the MST supervision protocol (Schoenwald, Sheidow et al., 2008).

Consistent with psychometric evaluation of the single-factor TAM, TAM-R ratings were stable within a family’s treatment episode. TAM-R scores per administration range from 0 to 1, representing the percentage of items on which the caregiver rated the therapist as adherent. The scores for each administration were averaged by family to produce a mean level of therapist adherence experienced by a family during the treatment episode. In the Transportability sample, the mean TAM-R score was .64 (SD = .26), with observed scores ranging from 0 to 1.

Data Analysis Strategy

Missing data

Given the nested data structure (described in the Data structure and statistical models section), data can be missing for therapists, families, and/or assessment occasions, and missing data at a given level of analysis can result in the loss of data at a different level of analysis. The initial data set was comprised of a maximum of 4 measurement occasions for each of 1,888 families with one of 429 primary therapists. At therapist level, 56 (13%) were missing ethnicity data. At family level, 274 (15%) were missing data on at least one variable required for analysis. At the repeated measurements level, 26 (1%) families were missing all 4 CBCL administrations and 14 (0.7%) families were missing all 4 VFI administrations. These families were excluded from analyses; however, families missing 3 or fewer CBCL or VFI administrations were retained in the analyses. The statistical models detailed subsequently made use of all available data.

Data structure and statistical models

Due to the nested data structure, analyses were performed using Mixed-Effects Regression Models (MRMs; Raudenbush & Bryk, 2002). The caregiver-reported youth outcome data are nested such that t repeated measurements of youth behavior and functioning problem outcomes (level-1, ntij ≈ 4) are nested within i youths/caregivers (level-2, n·i· ≈ 1576) who are nested within j primary therapists (level-3, n··j ≈ 362). The youth post-treatment charge data are nested such that the number of post-treatment charges per i youths with available charge data (level-1, nijk ≈ 978) is nested within j primary therapists (level-2, n·j· ≈ 271) who are nested within k provider organizations across communities (level-3, n··k ≈ 39). Finally, caregiver-reported therapist adherence data are nested such that the average therapist adherence score for youths/caregivers (level-1) are nested within primary therapists (level-2). Of note, provider organization was included as a level of nesting for the charge outcome because a non-trivial proportion of the variance in charges (11%) was attributable to the provider organization. However, this was not true for the caregiver-reported behavior problems (models did not converge), functioning problems (4%), or therapist adherence (3%) outcomes.

For the longitudinal caregiver-reported outcome data, change over time was modeled using linear and quadratic polynomial terms. Due to variability in the spacing of assessments, these terms were computed as the actual number of months between the first assessment and subsequent assessments (Biesanz, Deeb-Sossa, Papadakis, Bollen, & Curran, 2004). The linear term captures the instantaneous rate of change and the quadratic term captures the acceleration of change over time (Singer & Willett, 2003). The magnitude and direction of each term determines the shape of the trajectory, with, for example, the combination of a negative linear term and positive quadratic term indicating a decelerating negative slope (Hedeker & Gibbons, 2006).

The youth post-treatment charge outcome is the count of charges incurred per youth during the period between the end of treatment and retrieval date of each youth’s charge record. The outcome was modeled according to a Poisson distribution with a log link function and an offset term representing each youth’s “time at risk” (M = 4.20 years; SD = 0.76). Of note, to yield a sample comparable to the juvenile offender samples in MST clinical trials, youths with zero lifetime pre-treatment charges were excluded from the current analyses.

Youth-specific covariates commonly associated with behavioral, functional, and criminal outcomes (i.e., age, gender, ethnicity, number of lifetime pretreatment charges) and family–specific variables potentially associated with youth outcomes (caregiver age, gender, income, marital status, and education) were entered at the youth/caregiver-level of the models. Dummy codes for therapist and caregiver ethnicity were entered at the therapist and youth/caregiver levels of the model, respectively, and the indicator for the caregiver and therapist being of similar ethnicity was entered at the youth/caregiver level of the model.

The model building approach detailed by Singer and Willett (2003) was used for specifying fixed and random effects, and all MRMs were performed using HLM software (version 6.06; Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2004). Two- and three-level continuous models were performed using Restricted Maximum Likelihood and Full Maximum Likelihood estimation, respectively, and robust standard errors were used for the computation of the Wald (i.e., T-ratio) test statistic for the fixed effects (Maas & Hox, 2005). The three-level Poisson model was performed using Penalized Quasi-Likelihood estimation, asymptotic standard errors were used for the computation of the Wald test statistic for the fixed effects, and the population-averaged results were interpreted. Categorical predictors were entered using dummy-coded indicators, and continuous predictors were centered around their grand mean value.

As recommended by Fidler et al. (2005), confidence intervals for the unstandardized regression coefficients were provided for all significant effects as an indication of the magnitude and precision of the effects. Event rates and the associated confidence intervals were provided for the charge outcome models, with the event rate representing the annual rate of post-treatment charges.

Test of indirect effects

The research aims imply tests of mediated, indirect, or intervening variable effects. The method for testing multilevel mediation detailed by Krull and MacKinnon (2001) was extended and applied to the three-level case for behavior and functioning outcomes and the two-level case for a count-distribution for the post-treatment charge outcome. The product of coefficients test (i.e., α̂β̂) with asymmetric confidence limits (MacKinnon, Lockwood, & Williams, 2004), estimated using PRODCLIN software (MacKinnon, Fritz, Williams, & Lockwood, 2007), was selected as the method for testing mediation on the basis of greater statistical power relative to both the traditional “causal steps” approach and the product of coefficients approach using the multivariate delta method for computing the SE of the mediated effect (Krull & MacKinnon; MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002).

Results

Therapist Adherence Predicting Youth Behavior, Functioning, and Charges

Externalizing problems

Holding constant the effect of the covariates, therapist adherence was associated with significantly greater instantaneous reductions (i.e., linear change), γ = −0.579, SE = 0.204, T (1574) = −2.83, p = .005, 95% CIγ = −0.979 to −0.179, and acceleration of change over time (i.e., quadratic change), γ = 0.029, SE = 0.010, T (1574) = 2.81, p = .005, 95% CIγ = 0.007 to 0.051, in youth Externalizing problems. For higher levels of therapist adherence (i.e., 1 SD above the mean), Externalizing T-scores decreased by 8.3 points between pre-treatment and 12 months post-treatment; and for lower levels of therapist adherence (i.e., 1 SD below the mean), scores decreased by 7.6 points during the same time., This additional adherence-related reduction in Externalizing T-scores, although small (.07), suggests some benefit of better MST adherence to treated youth, whose Externalizing behaviors have incurred Juvenile Justice involvement and high risk of out-of-home placement.

Internalizing problems

Holding constant the effect of the covariates, therapist adherence was associated with significantly greater instantaneous reductions (i.e., linear change), γ = −0.586, SE = 0.219, T (1574) = −2.67, p = .008, 95% CIγ = −1.015 to −0.157, and acceleration of change over time (i.e., quadratic change), γ = 0.024, SE = 0.011, T (1574) = 2.15, p = .032, 95% CIγ = 0.002 to 0.046, in youth Internalizing problems. For higher levels of therapist adherence (i.e., 1 SD above the mean), Internalizing T-scores decreased by 8.2 points between pre-treatment and 12 months post-treatment; and for lower levels of therapist adherence (i.e., 1 SD below the mean), scores decreased by 6.6 points during the same time. Thus, the effect of therapist adherence on Internalizing behavior problems translates in to an additional 1.6-point reduction in Internalizing T-scores.

Functioning problems

Holding constant the effects of the covariates, therapist adherence was not significantly associated with instantaneous changes, γ = 0.004, SE = 0.004, T (1574) = 0.89, p = .371, or acceleration of change over time, γ = −0.0001, SE = 0.0002, T (1574) = −0.54, p = .589, in Functioning problems.

Post-treatment charges

Holding constant the effects of the covariates, therapist adherence was associated with a significantly lower rate of post-treatment charges, γ = −0.425, SE = 0.195, T (270) = −2.18, p = .030, ERR = 0.65, 95% CIERR = 0.45 to 0.96. This translates into a 20% lower annual number of charges for high levels of therapist adherence (i.e., 1 SD above the mean) relative to low levels of therapist adherence (i.e., 1 SD below the mean).

Ethnic Similarity Predicting Youth Behavior, Functioning, and Post-Treatment Charges (Path C)

Externalizing problems

Caregiver-therapist ethnic similarity was associated with significantly greater instantaneous reductions (i.e., linear change) in youth Externalizing problems, γ = −0.229, SE = 0.118, T (1671) = −1.96, p = .052, 95% CIγ = −0.458 to 0.000. The acceleration of change over time (i.e., quadratic change) did not differ significantly between the two groups, γ = 0.006, SE = 0.006, T (1671) = 0.96, p = .338, 95% CIγ = −0.006 to 0.018. For youth whose caregivers were ethnically similar to their therapists, Externalizing T-scores decreased by 8.7 T-score points between pre-treatment and 12 months post-treatment; and for youth whose caregivers were not ethnically similar to their therapists, T-scores decreased by 6.5 points during the same time. Thus, the effect of ethnic similarity on Externalizing behavior problems translates in to an additional 2.2 point reduction in Externalizing T-scores. Youth in both groups, however, started treatment with T-scores above the clinical cutoff (65.8 and 65.3 for ethnically similar and dissimilar pairs, respectively) and had T-scores below the clinical cutoff (57.0 and 58.8, respectively) one year following treatment, meaning Externalizing problems were no longer clinically significant for either group.

Internalizing problems

Caregiver-therapist ethnic similarity was not significantly associated with instantaneous reductions, γ = −0.192, SE = 0.121, T (1671) = −1.59, p = .112, or acceleration of change over time, γ = 0.006, SE = 0.006, T (1671) = 0.98, p = .326, for the Internalizing problems outcome.

Functioning problems

Caregiver-therapist ethnic similarity was not significantly associated with instantaneous reductions, γ = −0.001, SE = 0.003, T (1671) = −0.38, p = .707, or acceleration of change over time, γ = −0.0001, SE = 0.0001, T (1671) = −0.04, p = .972, for Functioning problems outcomes.

Post-treatment charges

Caregiver-therapist ethnic similarity was not significantly associated with the rate of post-treatment charges, γ = 0.028, SE = 0.105, T (270) = 0.27, p = .791, ERR = 1.03, 95% CIERR = 0.84 to 1.26.

Ethnic Similarity Predicting Therapist Adherence (Path A)

For the entire youth sample, caregiver-therapist ethnic similarity was associated with a significantly higher average level of therapist adherence for a youth’s treatment episode, γ = 0.056, SE = 0.019, T (1559) = 2.89, p = .004, 95% CIγ = 0.019 to 0.093. On average, caregivers whose ethnicity was similar to that of the therapist provided therapist adherence ratings that were .056 points higher than did caregivers whose ethnicity was dissimilar to that of the therapist. This difference of approximately 6%, with possible therapist adherence scores ranging from 0%–100%, translates into a caregiver rating a therapist as “adherent” on 1–2 more items (of 28) on the TAM-R when the therapist was of similar ethnicity.

For the sub-sample of youth with available post-treatment charge data, caregiver-therapist ethnic similarity was associated with a significantly higher level of therapist adherence for a youth’s treatment episode, γ = 0.048, SE = 0.023, T (871) = 2.12, 95% CIγ = 0.004 to 0.092, consistent with the findings based on the full sample.

Ethnic Similarity & Therapist Adherence Predicting Youth Behavior, Functioning, and Post-Treatment Charges (Paths B & C′)

Externalizing problems

As presented in Table 1, holding constant the effect of therapist adherence, caregiver-therapist ethnic similarity was not significantly associated with instantaneous changes or acceleration of change over time in Externalizing problems. However, therapist adherence was associated with significantly greater instantaneous reductions (i.e., linear change), 95% CIγ = −0.957 to −0.161, and acceleration of change over time (i.e., quadratic change), 95% CIγ = 0.009 to 0.049 in Externalizing problems. For higher levels of therapist adherence (i.e., 1 SD above the mean), Externalizing T-scores decreased by 8.2 points between pre-treatment and 12 months post-treatment; and for lower levels of therapist adherence (i.e., 1 SD below the mean), scores decreased by 7.7 points during the same time. Thus, the effect of therapist adherence on Externalizing behavior problems translates in to an additional 0.5 point reduction in Externalizing T-scores.

Table 1.

Ethnic similarity and therapist adherence effects on youth behavior and functioning problems one year post-treatment

Externalizing
Internalizing
Functioning Problems
Parameter SE p Parameter SE p Parameter SE p
Fixed Effects
Intercept 65.069 1.359 <0.001 58.880 1.308 <0.001 0.369 0.02 <0.001
 Therapist Ethnicity
  African American −1.208 1.017 0.236 −2.008 0.924 0.030 0.004 0.013 0.742
  Asian American 2.520 1.146 0.028 0.882 1.276 0.490 0.039 0.021 0.062
  Latino −0.208 1.793 0.908 −2.024 1.577 0.200 0.028 0.030 0.343
  Other −0.929 1.717 0.588 −3.064 1.548 0.048 0.007 0.026 0.788
 Caregiver Ethnicity
  African American −2.606 0.832 0.002 −2.566 0.832 0.003 −0.010 0.010 0.356
  Asian American −4.943 1.233 <0.001 −0.539 1.285 0.675 −0.015 0.018 0.388
  Latino −2.251 1.250 0.072 −0.404 1.499 0.787 <0.001 0.018 0.983
  Other −3.298 1.020 0.002 −0.447 1.123 0.690 0.026 0.016 0.100
 Therapist Adherence −2.224 1.219 0.068 −1.022 1.309 0.435 −0.084 0.019 <0.001
 Ethnic Similarity 0.366 0.796 0.645 −0.355 0.757 0.638 0.009 0.014 0.500
 Youth Age −0.679 0.131 <0.001 −0.603 0.137 <0.001 <0.001 0.002 0.910
 Caregiver Age 0.023 0.034 0.508 0.051 0.035 0.145 <0.001 <0.001 0.277
 Youth Gender 2.249 0.578 <0.001 −0.022 0.595 0.971 −0.032 0.007 <0.001
 Caregiver Gender 2.705 0.765 0.001 4.018 0.825 <0.001 0.028 0.011 0.013
 Caregiver Income 0.577 0.593 0.331 0.366 0.637 0.565 0.009 0.007 0.238
 Two Parent Home 0.319 0.524 0.542 −0.007 0.567 0.990 −0.006 0.006 0.338
 Caregiver Education 1.104 0.714 0.122 0.174 0.743 0.814 0.002 0.009 0.855
Linear Change
 Linear −0.767 0.104 <0.001 −0.681 0.111 <0.001 −0.022 0.002 <0.001
 Adherence × Linear −0.559 0.203 0.006 −0.567 0.219 0.010 0.004 0.004 0.359
 Match × Linear −0.179 0.116 0.123 −0.164 0.127 0.196 −0.001 0.003 0.786
Quadratic Change
 Quadratic 0.022 0.005 <0.001 0.017 0.006 0.003 0.001 <0.001 <0.001
 Adherence × Quadratic 0.029 0.010 0.006 0.024 0.011 0.036 <0.001 <0.001 0.591
 Match × Quadratic 0.004 0.006 0.526 0.005 0.006 0.407 <0.001 <0.001 0.902
Variance Components Parameter SD Parameter SD Parameter SD
 Within Youth (σ2) 53.949 7.345 50.917 7.136 0.018 0.136
 Youth Intercept (τ[π]) 66.308 8.143 84.116 9.171 0.019 0.139
 Linear (τ[π]) 0.595 0.772 1.144 1.069 0.001 0.022
 Quadratic (τ[π]) 0.001 0.030 0.002 0.050 0.001 0.001
 Therapist Intercept (τ[β]) 11.104 3.332 8.466 2.910 0.002 0.044
Open in a new tab

Internalizing problems

Holding constant the effect of therapist adherence, caregiver-therapist ethnic similarity was not significantly associated with instantaneous changes or acceleration of change over time in Internalizing problems. However, therapist adherence was associated with significantly greater instantaneous reductions (i.e., linear change), 95% CIγ = −0.996 to −0.138, and acceleration of change over time (i.e., quadratic change), 95% CIγ = 0.002 to 0.046 in Internalizing problems. For higher levels of therapist adherence (i.e., 1 SD above the mean), Internalizing T-scores decreased by 8.1 points between pre-treatment and 12 months post-treatment; and for lower levels of therapist adherence (i.e., 1 SD below the mean), scores decreased by 6.6 points during the same time. Thus, the effect of therapist adherence on Externalizing behavior problems translates in to an additional 1.5 point reduction in Externalizing T-scores.

Functioning problems

Holding constant the effect of therapist adherence, caregiver-therapist ethnic similarity was not significantly associated with instantaneous changes or acceleration of change over time in Functioning problems. Similarly, therapist adherence was not significantly associated with instantaneous reductions or acceleration of change over time in youth Functioning problems.

Post-treatment charges

As presented in Table 2, holding constant the effect of therapist adherence, caregiver-therapist ethnic similarity was not significantly associated with the rate of post-treatment charges, 95% CIERR = 0.82 to 1.34. Similarly, holding constant the effect of caregiver-therapist ethnic similarity, therapist adherence was not significantly associated with the rate of post-treatment charges, 95% CIERR = 0.47 to 1.02.

Table 2.

Ethnic similarity and therapist adherence effects on youth criminal outcomes four years post treatment
Predictor γ SE p ERR
Fixed Effects
Intercept 0.076 0.146 .605 1.08
 Therapist Ethnicity
  African American −0.286 0.164 .081 0.75
  Asian American −0.247 0.226 .276 0.78
  Latino 0.280 0.321 .384 1.32
  Other −0.499 0.289 .085 0.61
 Caregiver Ethnicity
  African American 0.670 0.103 <.001 1.95
  Asian American −0.113 0.135 .402 0.89
  Latino 0.163 0.096 .091 1.18
  Other −0.563 0.103 <.001 0.57
 Therapist Adherence −0.375 0.198 .059 0.69
 Ethnic Similarity 0.047 0.123 .700 1.05
 Youth Age −0.070 0.046 .138 0.93
 Caregiver Age 0.003 0.002 .108 1.00
 Youth Gender −0.536 0.111 <.001 0.58
 Caregiver Gender 0.523 0.062 <.001 1.70
 Caregiver Income 0.027 0.041 .516 1.03
 Two Parent Home −0.135 0.038 .001 0.87
 Caregiver Education −0.045 0.043 .296 0.96
 Pre-Treatment Charges 0.050 0.012 <.001 1.05
Variance Components
Therapist Intercept (τ[π]) 0.652 0.808 <.001
 Therapist Adherence(τ[π]) 5.283 2.299 <.001
 Ethnic Similarity(τ[π]) 1.668 1.291 <.001
 Provider Intercept (τ[β]) 0.152 0.390 <.001
 Youth Age (τ[β]) 0.065 0.254 <.001
 Youth Gender (τ[β]) 0.325 0.570 <.001
 Pre-Treatment Charges (τ[β]) 0.004 0.064 <.001
Open in a new tab

Test of Indirect Effects

The product of coefficients test with asymmetric confidence limits, as detailed above, was used to test for evidence of an indirect effect of caregiver-therapist ethnic similarity on youth outcomes by way of therapist adherence to MST. The results revealed significant indirect effects for linear change in Externalizing, α̂β̂ = −0.031, 95% CIα̂β̂ = −0.09 to −0.01, and Internalizing, α̂β̂ = −0.032, 95% CIα̂β̂ = −0.10 to −0.01, behavior problems, providing evidence that the effect of caregiver-therapist ethnic similarity is transmitted by way of therapist adherence to MST. Similarly, there was a significant indirect effect for the rate of post-treatment charges, α̂β̂ = −0.018, 95% CIα̂β̂ = −0.10 to −0.01. To summarize, caregiver-therapist ethnic similarity was associated with higher caregiver ratings of therapist adherence; and, holding constant the effect of ethnic similarity, higher levels of therapist adherence were associated with greater reductions in youth Externalizing and Internalizing problems and with a lower rate of post-treatment charges. Of note, the normal theory 95% confidence limits contained 0 in each case and would not have identified a significant indirect effect.

Discussion

The current investigation examined prospectively, in a large sample of youth and families and therapists employed by provider organizations in 45 sites, relations between ethnic similarity in caregiver-therapist pairs, therapist adherence, and long term youth behavioral and criminal outcomes of MST, an evidence-based treatment for youth with serious antisocial behavior. Across the four long-term youth outcomes examined, therapist adherence significantly predicted three – criminal activity, externalizing problems, and internalizing problems; whereas caregiver-therapist ethnic similarity predicted only one – externalizing problems. Neither adherence nor ethnic similarity predicted changes in youth functioning. When both therapist adherence and ethnic similarity effects were modeled simultaneously, ethnic similarity was not associated with any youth outcome; whereas, the effects of therapist adherence on youth externalizing and internalizing problems remained significant. This pattern of findings supports the robustness to ethnic similarity and dissimilarity of the linkage between therapist adherence and long-term youth outcomes of MST implemented in diverse communities. The pattern is also consistent with previously reported findings from the same study showing factors at various levels of the practice context (i.e. client, therapist, provider organization, service system) that could be expected to affect treatment implementation and outcomes appear to have limited effects on them (Schoenwald, Carter, Chapman, & Sheidow, 2008; Schoenwald, Chapman, Sheidow, & Carter, 2009).

The results of the current evaluation did, however, support prior findings from this sample showing ethnic similarity predicted higher caregiver ratings of therapist adherence; and, that these higher ratings predicted greater reductions over time in one youth outcome -- externalizing behavior problems. The clinical significance of this differential is unclear, however, given youth in both ethnically similar and dissimilar caregiver-therapist pairs experienced reductions in externalizing problems from above the clinical cutoff to well below the clinical cutoff. Nonetheless, this finding raises the possibility that therapists in ethnically similar pairs evidence greater adherence to interventions needed to help caregivers effectively manage or reduce youth externalizing behavior problems relative to therapists in ethnically dissimilar pairs. On the one hand, such a possibility supports the explanation of the apparent effectiveness of MST with ethnic minorities posited by Huey and Polo, namely that the MST treatment principles inherently facilitate tailoring of treatment to the uniqueness of each youth and family. On the other hand, such tailoring should occur regardless of the ethnic similarity of the client and therapist, a point to which we return in the discussion of implications for future research. Alternatively, caregiver ratings on the MST adherence measure may be influenced by other factors such as initial expectations of treatment or satisfaction with treatment, either of which could be greater among ethnically similar pairs relative to dissimilar pairs. Although the data required to examine these hypotheses are not available from the current study, an ongoing study of treatment processes that differentiate MST responders and non-responders in routine care may shed light on the issue, as it includes measures of alliance, adherence, and satisfaction (e.g., Cunningham, 2006).

Long-term criminal outcomes

Greater therapist adherence was a significant predictor of lower rates of youth criminal charges on average four years post treatment, whereas ethnic similarity in caregiver-therapist pairs was not. When therapist adherence and ethnic similarity effects were modeled simultaneously, however, neither was found to predict youth criminal charges (although adherence approached significance at p. = .059). The higher levels of therapist adherence reported by caregivers in ethnically similar pairs, however, was associated with lower rates of youth criminal activity.

Limitations of the study

Several limitations of this investigation suggest caution in the interpretation of its findings. First, as noted in a prior investigation of the short-term effects of ethnic similarity on youth outcomes (Halliday-Boykins et al., 2005), ethnic similarity between the therapist and caregiver was not necessarily an exact ethnic match (e.g., Cuban and Mexican would be considered similar, both being Latino). Thus, some cultural variability is likely embodied within each ethnic group. Second, a majority of Caucasian caregivers constituted ethnically similar caregiver-therapist pairs, thus raising questions about the extent to which study findings generalize to caregiver-therapist pairs of other ethnicities. However, half of caregivers of mixed ethnicity, over one-third of Asian caregivers, and over one-quarter of African American caregivers were also in ethnically similar pairs. Given the large sample of families (1979) and therapists (429) in the study, the sizeable proportions of caregivers of different ethnic groups treated by ethnically similar therapists mitigates somewhat the concern that the study findings pertain primarily to Caucasian clients and therapists. That is, the variability in ethnically similar pairs is adequate to evaluate relations between adherence, ethnic similarity, and outcomes. Future research could, however, evaluate the extent to which ethnic similarity effects are moderated by ethnicity (i.e., do ethnic similarity effects differ among pairs that are Asian, African-American, of mixed-ethnicity, and so forth). Third, all participants in the current study were English speaking, such that language effects could not be examined. Fourth, families were not randomly assigned to therapists, limiting the validity of causal inferences from these findings. Importantly, however, ethnicity was not considered in therapist assignments. Fifth, shared method variance may have contributed to the associations found between therapist adherence and youth Internalizing and Externalizing scores, as both adherence and behavior problem scores were reported by the youth’s caregivers. Notably, however, three of the four behavior problem assessments occurred with little to no temporal proximity to the adherence assessments: the T1 behavior problem assessment occurred one month prior to the first adherence assessment and the T3 and T4 assessments occurred six and twelve months after adherence assessments had ended. And, shared method variance does not characterize the adherence measure and criminal charges data.

Finally, that the current findings were only partially consistent with those obtained in the prior evaluation of ethnic similarity and adherence on youth behavior problem reductions may reflect both real differences –length of follow-up periods and use of criminal charge data -- and artifacts of the sample and statistical modeling approaches used in each evaluation. As noted in the Introduction, the current report extends the follow-up period for caregiver-reported youth behavior problem outcomes from 6 months through 12-month post-treatment; and, includes 4-year post-treatment youth criminal charge outcomes. Regarding the sample, data from a larger number of families were used in the current versus prior analyses of ethnic similarity and adherence effects on youth behavioral and functional outcomes (1,888 versus 1,711, respectively). The current analyses used data from families treated by more than one therapist for whom a primary therapist could be identified (using procedures described in the Method section), whereas families treated by more than one therapist were eliminated from the analyses reported previously. As noted in the Data Analysis Strategy section, in the present investigation, asymmetric confidence limits were used to test for the significance of the mediated effect, whereas Halliday-Boykins et al. (2005) used the first-order Sobel method for computation of the standard error of the mediated effect. The present use of asymmetric confidence limits is consistent with advances in field of mediation analysis (e.g., MacKinnon et al., 2004) as well as the availability of software implementing this methodology (i.e., PRODCLIN, MacKinnon et al., 2007)

Implications for future research

The results reported here suggest therapist adherence to an evidence-based treatment in usual care settings predicts longer-term positive outcomes for youth, whereas the ethnic similarity of therapists and caregivers does not. Caregivers in ethnically similar pairs, however, reported higher therapist adherence relative to caregivers in ethnically dissimilar pairs. And, those higher adherence ratings predicted slightly greater long-term reductions in youth externalizing problems and criminal charges. These adherence ratings and associated outcomes improvements were admittedly small in magnitude. Nonetheless, the findings signal a need to better understand the higher adherence ratings reported by caregivers in ethnically similar caregiver-therapist pairs, research is needed to identify those treatment processes in MST that most affect adherence; and the extent to which these processes differ among ethnically similar and dissimilar caregiver-therapist pairs. Such research could illuminate treatment processes that would further enhance the outcomes of MST for all youth via their effects on therapist adherence, and identify and MST treatment processes that, as Huey and Polo suggest, may already reflect inherent consideration of cultural differences. Further research is also needed on client-reported adherence measurement methodologies to evaluate the extent to which ratings provided by untrained raters (i.e., caregivers, clients) reflect, or are influenced by, constructs other than adherence, such as treatment alliance, expectations or satisfaction; and the extent to which these are affected by client-therapist ethnic similarity.

Clinical implications

The current findings suggest the long-term post-treatment benefit to youth and families of focusing in usual care practice settings on the fidelity of implementation of an empirically supported, family-based treatment such as MST, one youth and family at a time, regardless of the ethnicity of the youth, caregiver, or therapist. Conversely, increasing the proportion of families served by ethnically similar clinicians is not likely to contribute to the positive long-term outcomes of youth unless the arrangement contributes to higher therapist adherence. Such arrangements, however, are both reminiscent of segregation and of questionable viability given the limited availability of mental health professionals who share the specific ethnic heritage of each of the ethnic groups served by a particular program in a particular locale.

Conclusion

The findings of this study are consistent with reviews suggesting ethnic minorities benefit from treatments with demonstrated efficacy, and provide evidence that such benefits extend, at least for MST, to the long-term outcomes of youth treated usual care settings, regardless of the ethnic similarity of caregiver-therapist pairs. Studies are needed, however, to evaluate the extent to which the ethnicity and ethnic similarity of clients and clinicians affects the implementation and outcomes in usual care of other evidence-based treatments. In addition, because therapist adherence was reported to be higher in ethnically similar caregiver-therapist pairs and associated with slightly better long-term outcomes, evaluation is warranted of therapy processes contributing to adherence in ethnically similar and dissimilar client-therapist pairs so that treatment effectiveness can be maximized irrespective of the ethnic backgrounds of clients and clinicians.

References

  1. Achenbach TM. Integrative guide for the 1991 CBC/4-18, YSR, and TRF profiles. Burlington: University of Vermont, Department of Psychiatry; 1991. [Google Scholar]
  2. Asarnow JR, Jaycox LH, Duan N, LaBorde AP, Rea MM, Murray P, et al. Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. Journal of the American Medical Association. 2005;293:311–319. doi: 10.1001/jama.293.3.311. [DOI] [PubMed] [Google Scholar]
  3. Backer TE, David SL, Soucy G, editors. NIDA Research Monograph 155, NIH Publication No. 95-4035. Rockville, MD: National Institute on Drug Abuse; 1995. Reviewing the behavioral science knowledge base on technology transfer. [PubMed] [Google Scholar]
  4. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51:1173–1182. doi: 10.1037//0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  5. Bauer DJ, Preacher KJ, Gil KM. Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods. 2006;11:142–163. doi: 10.1037/1082-989X.11.2.142. [DOI] [PubMed] [Google Scholar]
  6. Bickman L, Lambert E, Karver WM, Andrade AR. Two low-cost measures of child and adolescent functioning for services research. Evaluation and Programming Planning. 1998;21:263–275. [Google Scholar]
  7. Biesanz JC, Deeb-Sossa N, Papadakis AA, Bollen KA, Curran PJ. The role of coding time in estimating and interpreting growth curve models. Psychological Methods. 2004;9:30–52. doi: 10.1037/1082-989X.9.1.30. [DOI] [PubMed] [Google Scholar]
  8. Bronfenbrenner U. The ecology of human development. Cambridge, MA: Harvard University Press; 1979. [Google Scholar]
  9. Chamberlain P, Brown CH, Saldana J, Reid W, Wang L, Marsenich T, et al. Engaging and recruiting counties in an experiment on implementing evidence-based practice in California. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:250– 260. doi: 10.1007/s10488-008-0167-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cunningham PC. Predicting treatment response and nonresponse in Multisystemic Therapy. Invited symposium, Implementation and outcomes in child-adolescent community mental health research; American Psychological Association Annual Conference; New Orleans, Louisiana. Aug, 2006. [Google Scholar]
  11. Drotar D, Stein REK, Perrin EC. Methodological issues in using the Child Behavior Checklist and its related instruments in clinical child psychology research. Journal of Clinical Child Psychology. 1995;24:184–192. [Google Scholar]
  12. Fidler, et al. Toward improved statistical reporting in the Journal of Consulting and Clinical Psychology. Journal of Consulting and Clinical Psychology. 2005;73:136–143. doi: 10.1037/0022-006X.73.1.136. [DOI] [PubMed] [Google Scholar]
  13. Flicker SM, Waldron HB, Turner CW, Brody JL, Hops H. Ethnic matching and treatment outcome with Hispanic and Anglo substance-abusing adolescents in family therapy. Journal of Family Psychology. 2008;22:439– 447. doi: 10.1037/0893-3200.22.3.439. [DOI] [PubMed] [Google Scholar]
  14. Glisson C, Schoenwald SK. An organizational and community development strategy for implementing evidence-based children’s mental health treatments. Mental Health Services Research. 2005;7:1– 17. doi: 10.1007/s11020-005-7456-1. [DOI] [PubMed] [Google Scholar]
  15. Halliday-Boykins CA, Schoenwald SK, Letourneau EJ. Caregiver-therapist ethnic similarity predicts youth outcomes from an empirically based treatment. Journal of Consulting and Clinical Psychology. 2005;73:808– 818. doi: 10.1037/0022-006X.73.5.808. [DOI] [PubMed] [Google Scholar]
  16. Hanson CL, Henggeler SW, Haefele WF, Rodick JD. Demographic, individual, and family relationships correlates of serious and repeated crime among adolescents and their siblings. Journal of Consulting and Clinical Psychology. 1984;52:528–538. doi: 10.1037//0022-006x.52.4.528. [DOI] [PubMed] [Google Scholar]
  17. Hedeker D, Gibbons RD. Longitudinal data analysis. Hoboken, NJ: John Wiley & Sons, Inc; 2006. [Google Scholar]
  18. Henggeler SW, Borduin CM. Unpublished instrument. Charleston, SC: Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina; 1992. Multisystemic Therapy Adherence Scales. [Google Scholar]
  19. Henggeler SW, Borduin CM, Schoenwald SK, Huey SJ, Chapman JE. Unpublished instrument. Charleston, SC: Department of Psychiatry & Behavioral Sciences, Medical University of South Carolina; 2006. Multisystemic Therapy Adherence Scale - Revised (TAM-R) [Google Scholar]
  20. Henggeler SW, Melton GB, Brondino MJ, Scherer DG, Hanley JH. Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology. 1997;65:821–833. doi: 10.1037//0022-006x.65.5.821. [DOI] [PubMed] [Google Scholar]
  21. Henggeler SW, Pickrel SG, Brondino MJ. Multisystemic treatment of substance abusing and dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research. 1999;1:171–184. doi: 10.1023/a:1022373813261. [DOI] [PubMed] [Google Scholar]
  22. Henggeler SW, Schoenwald SK. The MST supervisory manual: Promoting quality assurance at the clinical level. Charleston, SC: The MST Institute; 1998. [Google Scholar]
  23. Henggeler SW, Schoenwald SK, Borduin CM, Rowland MD, Cunningham PB. Multisystemic treatment of antisocial behavior in children and adolescents. New York and London: Guilford Press; 1998. [Google Scholar]
  24. Henggeler SW, Schoenwald SK, Borduin CM, Rowland MD, Cunningham PB. Multisystemic therapy for antisocial behavior in children and adolescents. 2. New York: The Guilford Press; 2009. [Google Scholar]
  25. Huey SJ, Jr, Polo AJ. Evidence-based psychosocial treatments for ethnic minority youth. Journal of Clinical Child and Adolescent Psychology. 2008;37:262–301. doi: 10.1080/15374410701820174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kenny DA, Korchmaros JD, Bolger N. Lower level mediation in multilevel models. Psychological Methods. 2003;8:115–128. doi: 10.1037/1082-989x.8.2.115. [DOI] [PubMed] [Google Scholar]
  27. Klein KJ, Knight AP. Innovation implementation: Overcoming the challenge. Current Directions in Psychological Science. 2005;14(5):243–246. [Google Scholar]
  28. Klein KJ, Sorra JS. The challenge of innovation implementation. Academy of Management Review. 1996;21:1055– 1080. [Google Scholar]
  29. Kolb DA. Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall; 1984. [Google Scholar]
  30. Kenny DA, Korchmaros JD, Bolger N. Lower level mediation in multilevel models. Psychological Methods. 2003;8:115–128. doi: 10.1037/1082-989x.8.2.115. [DOI] [PubMed] [Google Scholar]
  31. Krull JL, MacKinnon DP. Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research. 2001;36:249–277. doi: 10.1207/S15327906MBR3602_06. [DOI] [PubMed] [Google Scholar]
  32. Letourneau EJ. Unpublished manuscript. 2006. Technical report on the interrater reliability of archival charges and charge codes. Available from author upon request. [Google Scholar]
  33. Maas CJM, Hox JJ. Sufficient sample sizes for multilevel modeling. Methodology. 2005;1:86–92. [Google Scholar]
  34. MacKinnon DP, Fritz MS, Williams J, Lockwood CM. Distribution of the product confidence limits for the indirect effect program PRODCLIN. Behavioral Research Methods. 2007;39:384–389. doi: 10.3758/bf03193007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychological Methods. 2002;7:83–104. doi: 10.1037/1082-989x.7.1.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. MacKinnon DP, Lockwood CM, Williams J. Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research. 2004;39:99–128. doi: 10.1207/s15327906mbr3901_4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Miranda J, Bernal G, Lau A, Kohn L, Hwang W-C, LaFramboise T. State of the science on psychosocial interventions for ethnic minorities. Annual Review of Clinical Psychology. 2005;1:113– 142. doi: 10.1146/annurev.clinpsy.1.102803.143822. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. 2. Thousand Oaks, CA: Sage Publications; 2002. [Google Scholar]
  39. Raudenbush SW, Bryk AS, Cheong YF, Congdon R, du Toit M. HLM 6: Hierarchical linear & nonlinear modeling (Version 6.02) (Computer software and manual) Lincolnwood, IL: Scientific Software International; 2004. [Google Scholar]
  40. Schoenwald SK. Multisystemic therapy consultation manual. Charleston, SC: The MST Institute; 1998a. [Google Scholar]
  41. Schoenwald SK. MST Personnel Data Inventory. Charleston, SC: Family Services Research Center; 1998b. [Google Scholar]
  42. Schoenwald SK. Toward evidence-based transport of evidence-based treatments: MST as an example. Journal of Child and Adolescent Substance Abuse. 2008;17(3):69–91. [Google Scholar]
  43. Schoenwald SK, Carter RE, Chapman JE, Sheidow AJ. Therapist adherence and organizational effects on change in youth behavior problems one year after Multisystemic Therapy. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:379–394. doi: 10.1007/s10488-008-0181-z. [DOI] [PubMed] [Google Scholar]
  44. Schoenwald SK, Chapman JE, Kelleher K, Hoagwood KE, Landsverk J, Stevens J, et al. A Survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs) Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:84–97. doi: 10.1007/s10488-007-0147-6. [DOI] [PubMed] [Google Scholar]
  45. Schoenwald SK, Chapman JE, Sheidow AJ. Implementation fidelity in MST. In: Schoenwald SK, Reid J, editors. Community based model programs panel: Implementing with fidelity; Blueprints Conference, 2006: Evidence-based programs, research-to-practice conference; Denver, CO. 2006. Mar, Co-Chairs. [Google Scholar]
  46. Schoenwald SK, Chapman JE, Sheidow AJ, Carter RE. Long-term youth criminal outcomes in MST transport: The impact of therapist adherence and organizational climate and structure. Journal of Clinical Child and Adolescent Psychology. 2009;38:91– 105. doi: 10.1080/15374410802575388. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Schoenwald SK, Henggeler SW, Brondino MJ, Rowland MD. Multisystemic therapy: Monitoring treatment fidelity. Family Process. 2000;39:83–103. doi: 10.1111/j.1545-5300.2000.39109.x. [DOI] [PubMed] [Google Scholar]
  48. Schoenwald SK, Henggeler SW, Edwards D. MST Supervisor Adherence Measure. Charleston, SC: MST Institute; 1998. [Google Scholar]
  49. Schoenwald SK, Hoagwood K. Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services. 2001;52:1179–1189. doi: 10.1176/appi.ps.52.9.1190. [DOI] [PubMed] [Google Scholar]
  50. Schoenwald SK, Letourneau EJ, Halliday-Boykins CA. Predicting therapist adherence to a transported family-based treatment for youth. Journal of Clinical Child and Adolescent Psychology. 2005;34(4):658–670. doi: 10.1207/s15374424jccp3404_8. [DOI] [PubMed] [Google Scholar]
  51. Schoenwald SK, Sheidow AJ, Chapman JE. Clinical supervision in treatment transport: Effects on adherence and outcomes. Journal of Consulting and Clinical Psychology. doi: 10.1037/a0013788. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Schoenwald SK, Sheidow AJ, Letourneau EJ. Toward effective quality assurance in evidence-based practice: Links between expert consultation, therapist fidelity, and child outcomes. Journal of Child and Adolescent Clinical Psychology. 2004;33:94–104. doi: 10.1207/S15374424JCCP3301_10. [DOI] [PubMed] [Google Scholar]
  53. Schoenwald SK, Sheidow AJ, Letourneau EJ, Liao JG. Transportability of Multisystemic Therapy: Evidence for multi-level influences. Mental Health Services Research. 2003;5:223–239. doi: 10.1023/a:1026229102151. [DOI] [PubMed] [Google Scholar]
  54. Singer JD, Willett JB. Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press; 2003. [Google Scholar]
  55. Waldron HB, Turner CW. Evidence-based psychosocial treatments for adolescent substance abuse. Journal of Clinical Child and Adolescent Psychology. 2008;37:238– 261. doi: 10.1080/15374410701820133. [DOI] [PubMed] [Google Scholar]
  56. Weisz JR. Psychotherapy for children and adolescents: Evidence-based treatments and case examples. New York: Cambridge University Press; 2004. [Google Scholar]

RESOURCES