Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Sep 23.
Published in final edited form as: J Fam Psychol. 2010 Jun;24(3):339–348. doi: 10.1037/a0019406

The Effectiveness of Functional Family Therapy for Youth with Behavioral Problems in a Community Practice Setting

Thomas Sexton 1, Charles W Turner 2
PMCID: PMC4172308  NIHMSID: NIHMS483896  PMID: 20545407

Abstract

The study examined the effectiveness of Functional Family Therapy (FFT), as compared to probation services, in a community juvenile justice setting 12 months post treatment. The study also provides specific insight into the interactive effects of therapist model specific adherence and measures of youth risk and protective factors on behavioral outcomes for a diverse group of adolescents. The findings suggest that FFT was effective in reducing youth behavioral problems, although only when the therapists adhered to the treatment model. High adherent therapists delivering FFT had a statistically significant reduction of (35%) in felony, a (30%) violent crime, and a marginally significant reduction (21%) in misdemeanor recidivisms as compared to the control condition. The results represent a significant reduction in serious crimes one year after treatment, when delivered by a model adherent therapist. The low adherent therapists were significantly higher than the control group in recidivism rates. There was an interaction effect between youth risk level and therapist adherence demonstrating that the most difficult families (those with high peer and family risk) had a higher likelihood of successful outcomes when their therapist demonstrated model specific adherence. These results are discussed within the context of the need and importance of measuring and accounting for model specific adherence in the evaluation of community-based replications of evidence-based family therapy models like FFT.

Adolescents involved in the justice system are clinically complex, with particularly high rates of behavior problems, mental health disorders, and other “at risk” behaviors. Estimates are that 50% to 80% of delinquent adolescents meet the criteria for a mental disorder, such as conduct or substance-related disorders (Kazdin & Weisz, 2003). Considering the family’s important role in the initiation and escalation of adolescent problem behaviors, family-based interventions have been of great interest to treatment researchers and community practitioners (Rowe & Liddle, 2003). Numerous reviews have identified Functional Family Therapy (FFT) as one of the emerging evidence-based intervention programs for at-risk adolescent youth and their families (Elliott, 1997; Alexander & Sexton, 2002; Sexton, et al., 2003; Waldron & Turner, 2008). FFT has an established record of outcome studies that demonstrate its efficacy with a wide variety of adolescent related problems including youth violence, drug abuse, and other delinquency related behaviors. The positive outcomes of FFT remain relatively stable even after a five-year follow-up (Gordon, Arbuthnot, Gustafson, & McGreen, 1988), and the positive impact also affects siblings of the identified adolescent (Klein, Alexander, & Parsons, 1977).

One of the great promises of evidence-based intervention programs like FFT is their potential in helping improve client care in community settings. The specificity of the model allows for monitoring of treatment, training, and therapist model adherence in ways that are not possible with other less specified interventions. This specific model has guided the dissemination of FFT in over 300 community settings in the United States, and four different international settings (see Sexton & Alexander, 2004, 2006; Sexton, 2009 for a complete description of FFT). FFT is delivered both in the office and in the client’s home. However, a significant gap exists in understanding many of the issues involved in the successful implementation of effective programs in community settings. For example, Henggeler, Pickrel and Brondino (1999) estimated the outcomes for an evidence-based family treatment in community studies to be 50% lower than in clinical trials of the same intervention models. Two issues seem to emerge as most salient explanations for the different results: the heterogeneous populations and diversity of youth (risk and protective levels) encountered in typical treatment contexts, and the differing levels of treatment model adherence.

Youth, their families, and treatment providers in community-based settings are heterogeneous and represent a diversity often not reflected in randomized clinical trials (RCT). The clinical staffs in RCTs often have extensive training and skill in implementing interventions with the specific target behaviors and clinical populations that are the focus of the trial. The staff may not be skillful in treating clinical problems or populations that are not the focus of the trial. In contrast to the selective recruitment procedures in clinical trials, the recruitment process for community providers requires them to accept virtually all of the clients that present for treatment. Risk and protective factors are a useful way to understand the variability in problem severity for adolescents’ with behavior problems. Risk factors increase the likelihood that the adolescent may experience drug, mental health, and/or conduct problems. Protective factors can mitigate the impact of risk factors or decrease the likelihood that the adolescent will develop certain emotional or behavioral disorders. These factors provide a comprehensive view of both the potential strengths and weaknesses of clients and their social and environmental context that goes beyond only measuring crime arrests or pretreatment recidivism (Henggeler et al., 1997; Sale, Springer, Sambrano, & Turner, 2003).

Treatment fidelity, or specific model adherence, has also been proposed as a factor in the delivery of specific effective programs in community based settings (Henggeler et al., 1997; Hogue et al., 2008; Sexton & Alexander, 2002). Several studies have found that therapist adherence to the treatment protocol is an important predictor of clinical outcomes, especially when studies are conducted in community settings (Henggeler et al., 1999; Hogue et al., 2008; Huey et al., 2000; Schoenwald et al., 2000). Despite its importance, there is a lack of research regarding the role of treatment adherence in successful community replications of efficacious family-based programs. Treatment fidelity is most often defined as therapist adherence, or the degree to which a given therapy is implemented in accordance with essential theoretical and procedural aspects of the model (Hogue et al., 2006; Waltz, Addis, Koerner, & Jacobson, 1993). In a few cases, treatment adherence has also been linked to treatment outcomes in family-based therapies (Henggeler et al., 1999; Huey et al., 2000; Schoenwald, Henggeler, Brondino & Rowland, 2000). However, treatment adherence may not always be related to better outcomes. Both Barber et al. (2006) and Hogue et al. (2008) cautioned that being either too lax or too strict in model adherence limits a therapist’s effectiveness. They reasoned that further research is needed to identify circumstances in which model adherence facilitates rather than impedes clinical outcomes.

The current study investigated the effectiveness of Functional Family Therapy in treating high-risk behavior disordered youth in a community juvenile justice setting considering the impact of therapist (model specific adherence) and client (youth risk and protective) factors. The study anticipates that FFT will demonstrate effects on youth behavior problems (recidivism) 12 months post treatment as compared to a probation services comparison group on probation for the same time period. The current study also examined the hypothesis that clients receiving FFT from a model adherent therapist have significantly lower rates of recidivism than clients receiving usual services. Low adherent therapists were not hypothesized to have improved outcomes relative to the usual services condition. We anticipated that juvenile offenders with higher levels of risk or lower levels of family protective factors would be particularly challenging for all therapists, but these difficulties may be particularly daunting for non-adherent therapists. While model adherence may not be essential for clients who experience few barriers to change, adherence may be critical for clients who present to therapy with multiple complex problems. Developers of evidence based family therapy models, such as FFT, have capitalized upon years of clinical experience and research to create guidelines for therapists in addressing problems associated with problematic family relationships. Based upon this reasoning, we hypothesized that clients with high risk and low family protective factors, who are being treated by low adherent FFT therapists, would have higher recidivism rates than those treated by high adherent therapists at 12 months post treatment.

This study expands on a preliminary analysis conducted by Barnowski (2002) by focusing on the interaction of client (risk and protective factors), therapist (model specific adherence), and outcome. These additions make this a very distinctive study and are further elaborated in the discussion section.

Method

The project was conducted within a statewide juvenile justice system of a large Western State. Functional Family Therapy was implemented as part of an initiative to bring evidence-based and cost effective interventions to the Juvenile Justice System. Data collection and client assignment were conducted by an independent state evaluation center. Thus, this study represents a true community based evaluation of an evidence-based family treatment program. This study builds on the original state-wide evaluation data (Barnoski, 2002) by including information on client risk and protective factors, a more comprehensive assessment of treatment adherence, and an expanded subject pool.

Research Design

Juvenile offenders who had been remanded for probation services were randomly assigned to receive either FFT or usual probation services. The primary dependent variable was adjudicated felony recidivism occurring during the 12 months post FFT. We also explored treatment effects on specific secondary dependent variables reflecting adjudicated misdemeanor and violent offense. In order to ensure group similarity and adequacy of follow up, both the control and treatment group were measured at 12 months post treatment (accounting for 6 months average processing time) assignment to measure one-year post treatment. Barnowski (2002) reasoned that the adjudicated crime measure should be collected at 18 months post assignment since the time required for final adjudication time in the system was on average 6 months. Thus, an 18 month interval provided time to adequately assess recidivism over the year following treatment. Felony crime was the critical outcome variable of interest given its system costs, youth and family toll, and treatment system costs. Each analysis controlled for youth age, gender, and initial risk level.

Two moderators of the outcome were also tested. A supervisor rated measure of therapist specific model adherence was used to test the variance in outcome due to therapist factors (adherence). A measure of youth risk and protective factor levels allows for testing of an important client variable: pretreatment client problem severity. Finally, the interaction of client risk and protective levels and therapist adherence were also investigated. Therapist model adherence was measured across multiple families and multiple weeks. Measures of adherence were aggregated to a two level variable (adherent/non adherent) so that we would have adequate sample sizes to test the interactions between levels of therapist adherence with the client risk and protective factors as predictors of outcomes. A central premise of FFT is that adolescent problem behaviors are a consequence of dysfunctional family processes that leave the juvenile vulnerable to the influences of negative peers, which increases the risk of criminal recidivism. Thus, we anticipate that FFT is especially effective in reducing criminal recidivism for adolescents in dysfunctional families, especially if the youths are also exposed to negative peer influences.

Participants

The project involved 38 therapists and 917 families in 14 different counties that represented both rural and urban settings. Adolescents entered this study because they had been adjudicated for a crime and sentenced to probation. Of the adolescents in this study (both control and FFT), 79% were male and 21% were female. Participants’ ages were evenly distributed from 13 to 17 years (age 13=11%, age 14=17%, age 15=23%, age 16=24%, age 17=25%). About 78% of the sample was white, 10% were African American, 5% were Asian, 3% were Native American, and 4% were not identified. Most of the adolescents (85.4%) were drug involved (high drug risk) and many reported alcohol use/abuse (80.47%) and other mental health or behavioral problems (27%). Most of the participants had committed felony crimes (56.2%) and many had committed misdemeanors (41.5%). The problem behaviors of these youths included adjudicated weapons crimes (10.4%), gang involvement (16.1%), out of home placements (10.5%), a history of running away from home (14.1%), and school dropout (46.39%). The criminal behavior started early for these adolescents, with 13.1% beginning before age 12, 63% between the ages of 12 and 14, and 23% between the ages of 14 and 17.

The FFT therapists were also diverse in demographic and prior professional backgrounds, allowing for the systematic study of the role of therapist characteristics. Therapists were hired for the FFT project by the participating court districts. The majority of participating therapists were female (79% female, 21% male) and White (74% White, 4 % African American, 4% Asian, 4% Mexican American, and 4% multiracial). A preliminary analysis suggested that the participating therapists varied widely in regard to prior training and experience, although none of the participating therapists had prior experience in FFT. The therapists had on the average 3 years of counseling experience, whereas 48% had 3 years of family therapy experience, and 15% had 2 years of experience in both counseling and family therapy. Twenty-one of the therapists were Master’s degree clinicians (78%) and 6 were Bachelor’s level. Most held licenses or certifications, (63% held a current Mental Health license, 4% held a non-Mental Health license), and 33% did not have any professional licenses. The therapists’ general clinical experience ranged from 1 to 40 years, and they received group-based FFT training throughout the project.

Procedure

After being sentenced to probation, all youth were administered the preliminary screening version of the Washington State Juvenile Court Administration Risk Assessment (WSJCA). Those youth scoring moderate to high risk were assessed using the full WSJCR (WSJCR-RA). They were assigned to the treatment or control conditions through a stratified randomization procedure at the county level according to the guidelines developed and mandated by the State Juvenile Justice system using a 1:1 assignment. Throughout the study, eligible adolescents were assigned in the same 1:1 random manner as caseload openings permitted. The State evaluators monitored the assignment protocol throughout the study. Those adolescents requiring acute care (hospitalization) were excluded prior to assignment to the study. None of the participants were excluded after assignment.

Treatment as Usual Condition

Participants in the treatment as usual condition received traditional probation services in their local county. In this system, probation services were specifically detailed in the State Standards of Probation Practice, and were strictly enforced by state probation officials. To deliver probation services, 85% of probation resources are typically devoted to weekly checking and supervision, and 15% are devoted to education and guidance (Barnoski, 1998). Youth in the study did not receive any additional treatment services.

FFT Condition

Each of the therapists in the study received systematic training and supervision in the intervention model (FFT) following the protocol detailed in the treatment manual (Alexander et al., 2000: Sexton & Alexander, 2004). The training protocol included clinical training, follow-up training, and ongoing supervision. Training was administered by FFT experts who also participated as trainers and supervisors in the nationwide dissemination of FFT. Training was closely monitored to ensure that the therapists were able to deliver the program content as intended by the model. At the direct service level, FFT treatment was guided by delivery protocol specified in the treatment manual (Sexton & Alexander, 2004). FFT was delivered in youths’ home over a 3 to 6 month period, and contact included an average of 12 family-based sessions. The senior author and his research team monitored treatment delivery through ongoing supervision, following principles in the FFT Supervision protocol (Sexton, Alexander, & Gilman, 2004).

Measures

Risk and Protective Factors Assessment

The WAJCA-RA is a 100-item structured interview that is conducted with the youths and their family to assess for multiple risk and protective factors. The Risk Assessment instrument collects information on 10 domains, including criminal history, school participation, use of free time, employment, peer relationships, family, alcohol and drug history, mental health, attitudes (deviant or prosocial), and social skills. The WAJCA-RA has extensive psychometric information that suggests the WAJCA-RA is both a reliable and valid tool to predict the likelihood of further involvement in the juvenile justice system (Barnoski, 1998). The measure was designed by a panel of nationally recognized criminal justice experts to assess the levels of risk for recidivism among juvenile offenders (see Barnowski, 2002). This relatively brief instrument has been used in the state of Washington to assess recidivism risk for thousands of adolescents. Trained local probation counselors administered the pre and total risk assessment and each reached a criterion level in reliable administration of the instrument. Ongoing administration fidelity was monitored by local program supervisors.

Rather than using a global assessment of risk or protective factors, we selected domains that are theoretically linked to the FFT intervention. We examined family focused risk and protective factors as well as negative peer influences. Each domain is rated by a probation officer using scales that follow the logic of item response theory. In other words, each item within a domain assesses a different location along a dimension of low to high risk for that domain. Thus, some items assess the low end of the risk continuum while other items assess the high end of the risk continuum. As a consequence, the items on the scales do not conform to the internal consistency criteria that might be expected from classical test theory. Scales constructed following the item response theory logic are likely to have lower values on internal consistency measures such as the intraclass correlation or coefficient alpha. Thus, we did not exclude theoretically meaningful scales even though the internal consistency statistics were sometimes rather low.

The negative peer dimension of the WAJCA-RA is an eight item scale that assesses both the extent of contact (hours per week, months of contact) time with deviant peers, the willingness to lie or steal for peers, as well as the juvenile’s admiration for deviant peers. The unstandardized alpha coefficient was .54, but the standardized alpha was .94 which is usually considered to reflect a high level of internal consistency in the scale. A nine item family risk scale assesses various aspects of dysfunctional family behaviors including family conflict, parent supervision, parent control, parent reward and punishment, and parent disapproval of antisocial behavior. Each item reflects aspects of parenting practices and was rated on a 1–3 scale. The coefficient alpha index was .60, suggesting only a moderate level of internal consistency. A principle components factor analysis suggests that the items represent a single dimension. The third dimension was a four item scale that was designed to assess family protective factors. The scale reflected whether the juvenile was living with parents, whether the family had a strong social support network, whether family members were close, and whether the family income was greater than $50,000 per year. The coefficient alpha index for this scale was .30, which raises some question about the internal consistency of the dimension. We recognize that this dimension may not provide a predictive relationship to the criminal recidivism measure because the items may not reflect a single underlying family protection dimension. The final scale included in the analysis was a 10 item scale that was designed to assess pretreatment criminal behavior. The total score on this scale primarily reflected individual differences in the types of prior offenses (e.g., felony, misdemeanor, weapons, crimes against person), and judicial actions such as confinement orders, or arrest warrants for failure to appear. The standardized alpha index is .59 which would be considered as a moderately low alpha according to classical test theory. In some cases it is appropriate to use additive indexes with fairly low internal consistency, particularly when adding different domains because it is experienced as a sum of burdens/risks from the individual perspective.

Treatment Adherence Measure

Adherence ratings were conducted according to the adherence protocol in use during the project (see Alexander et al., 2000, Sexton, et al., 2004 for a description of the protocol). In weekly supervision meetings, each therapist presented one of their active cases. The clinical supervisors rated the presentation based on the degree to which the therapist adhered to the FFT model using a 6-point Likert scale ranging from low (0) to high (5) model adherence. The rating was a measure of the degree to which the therapist demonstrated the ability to understand the case from knowledge of core principles of FFT and the degree to which they reported following the manual-specified goals for each phase of the clinical intervention (Sexton et al., 2004). Supervisors received training in the use of the adherence rating system and were regularly supervised by an expert who participated in the National dissemination of FFT. Of the five clinical supervisors making adherence ratings, 4 were female, all were Caucasian, 3 were trained marriage and family therapists, 2 had a license in marriage and family therapy, 1 held an MSW, and 1 had a M Ed. Each supervisor had at least 1 year of FFT experience (ranging from 1 to 3 years). Furthermore, each had demonstrated adherence in his or her own cases, and they had received specific supervision experience and training on the adherence rating process.

Therapist adherence ratings were gathered over a 2-year period, although some of the cases in the first year were not rated. It is important to remember that these multiple measures of adherence to FFT reflect the therapist across multiple families and multiple sessions, but they cannot be used to describe adherence for individual families. During the first year, 21 therapists had a total of 311 adherence ratings (ranging from 1 to 26 different times per therapist during that year). In the second year, a total of 44 therapists were rated 1599 different times (from 1 to 66 per therapist). Ratings for each therapist were averaged to a single adherence score. We retained only therapists (n =35) who had completed at least 5 cases during the study. The Likert scale ratings were aggregated into a 4-category system ranging from non-adherence, borderline adherence, adherence, and high adherence by separating adherence total scores into quartiles based on the average therapist adherence score. The aggregated scores were then reviewed by the State FFT Quality Assurance Coordinator and adjusted for administrative issues (e.g., the therapist was unable to complete cases due to moving or experiencing a major illness). An analysis of the initial clinical/supervisor ratings and the final adjusted ratings showed a high correlation (r = .93, p < .001). The adjusted scores were used in the analyses.

Outcome measures

The primary outcome measure in this study was the youth’s adjudicated post treatment felony criminal behavior in the 12 month period following randomization to treatment. All instances of adjudicated recidivism were obtained from the official state juvenile justice records. While adjudicated offenses may underestimate the total number of offenses committed by an individual, reductions in adjudicated offenses reflect the potential cost savings that can result for effective interventions with offenders. Neither the therapists nor the supervisors had access to these records since the data were compiled 12 months after the last client was randomized to treatment. Approximately 6% of the clients committed any adjudicated offenses during the first month of treatment and an additional 9% committed an offense in the second month. Most of the offenses occurred after treatment completion so that these events would not have been available to be discussed during supervision. The outcome data was available for all youths except 14, who moved out of state during the follow-up period. The state evaluation center classified crimes as one of three types representing increasing seriousness: misdemeanor, felony, or violent crime.

Statistical Analysis Plan

The statistical analysis was performed in four steps. We conducted a preliminary analyses to assess potential confounds that might influence the hypothesis testing of the research. First, we performed a multivariate analysis of variance MANOVA) to assess possible pretreatment differences in the three study samples (Control, Non Adherent Therapists, Adherent Therapists) on theoretically specified risk or protection dimensions. Although the clients were randomly assigned to the Control or the FFT condition, they were not randomly assigned to the Therapist Adherence group. We used a MANOVA procedure as a first step in the analysis to provide an omnibus test across multiple, correlated dependent measures to assess whether the differences among the three study samples on the set of measures was greater than would be expected by random sample processes. When we discovered that the three samples differed significantly on this omnibus test, then our next analysis was designed to identify possible sources of group differences. We conducted separate exploratory, univariate ANOVAs on each dependent variable to provide information about possible sources of the differences among the study samples that should be entered as covariates in the hypothesis tests.

Second, we conducted a test of the main hypothesis that the FFT condition would be associated with a lower level of adjudicated felony recidivism than the Control condition. We performed a binary logistic regression analysis with the dichotomous dependent variable of adjudicated felony recidivism during the 12 month period after randomization as the dependent variable. To test this hypothesis, we used a dummy variable to represent the difference between the control condition (coded 0) and the FFT condition (coded 1). The analysis also controlled for theoretically specified covariates that assessed risk and protective factors evaluated in step 1 of the analysis. Before we conducted this analysis we performed a random regression (or Hierarchical Linear Model) with the therapist factor (n = 35) as the independent variable to assess possible nesting effects, resulting from the fact that each therapist treated multiple families. If the analysis had demonstrated significant nesting effects, we would have included the therapist factor as a random effect in the analyses.

Third, to test a secondary hypothesis concerning the effects of therapist model adherence, we used dummy variables within a binary logistic regression analysis to examine differences among the three study samples (Control, Low Adherent Therapist, High Adherent Therapist) on the 12-month, adjudicated felony recidivism measure. In these analyses, we omitted the initial four cases for each therapist which we considered to occur in the therapists’ “training phase”. Given that the primary goal of the study was to test for the interaction of therapist adherence with the pretreatment risk factors, we collapsed the categories into two (adherent and non-adherent). Even though our sample size of FFT cases was quite large, it was still too small to test for possible interactions between a four level adherence measure and the adolescent risk factors given that the base rate of occurrence for felony recidivism was only 25%. Specifically, the sample size of clients treated by very low adherent or very high adherent therapists was too small to permit the inclusion of a four level variable in the analysis. The resulting sample included 220 families treated with low adherent therapists and 211 treated with high adherent therapists. This binary logistic regression analysis also controlled for the theoretically specified covariates used in the primary hypothesis testing. In these analyses, we excluded the earliest four cases from each therapist. We reasoned that these cases reflected their initial experience with the model and would not necessarily reflect their persisting level of model adherence.

Fourth, we examined possible interaction effects between pretreatment family risk and protective factors and peer risk factors and therapist model adherence. In these analyses, we analyzed the effects of adherence separately within low family risk, high family risk, or high peer risk factors as predictors of felony recidivism.

Results

Preliminary Analyses of Pretreatment Differences in Study Samples

An initial analysis of the possible pretreatment differences in the main comparison groups helped identify the critical variables to be used in the analysis of treatment group differences. A one-way multivariate analysis of variance with Study Sample (Control; Low Adherent Therapists; High Adherent Therapists) as the independent variable, and Family Risk, Family Protection, Negative Peer Relations, Criminal History, and Age as dependent variables revealed a significant multivariate Study Sample effect [MVF (10, 1498) = 3.40, p < .00, eta2 = 0.022]. We initially performed a MANOVA to provide a single global test to determine whether the differences among the study samples on the set of inter-correlated dependent variables was greater than would be expected by chance.

To provide descriptive information to interpret this multivariate effect, we conducted separate, one-way univariate ANOVAs on each dependent variable using Study Sample as the independent variable. The results of these analyses revealed a significant effect of Study Sample on the Criminal History [F(2,755) =.7.44, p < .001, eta2 = 0.019], Age [F(2,755) = 3.61, p < .028, eta2 = 0.009], and the Negative Peer Influence [F(2,755) = 3.69, p < .026, eta2 = 0.010] dependent variables. The results of the Study Sample effects on the other dependent variables were not statistically significant. Using a Bonferroni adjusted confidence level (alpha/3) we compared the three Study Sample means for the Age, Criminal History, and Relationship peer Risk dependent variables. For the Age variable, the Control group sample (M = 15.47, SD = 1.32) was older (adjusted p < .001) than the High Adherent FFT sample (M = 15.02, SD = 1.37). For the Criminal History variable, the Control sample (M = 8.74, SD = 4.12) had a higher level (p < .001) than the Low Adherent FFT sample (M = 7.55, SD = 3.39). For the Peer Risk variable, the Low Adherent FFT sample (M = 1.95, SD = 1.15) had a higher level than the Control sample (M = 1.66, SD = 1.23) or the High Adherent FFT Sample (M = 1.59, SD = 31.33). None of the other comparisons were statistically significant. These findings suggested that subsequent analyses should control for possible confounding effects of age, criminal history and peer risk for comparisons between the three Study samples.

We tested for possible nesting effects of clients within therapists within each adherence level. A total of 44 therapists administered FFT, but only 35 had sufficient case experience (i.e., at least five families) to develop a model adherence score. We tested for possible nesting effects of families within therapists since each one treated multiple families. We examined these effects separately within the 17 High Adherent therapists who treated 283 families and within the 18 Low Adherent therapists who treated 295 families. The results did not provide evidence of nesting effects for the High Adherent Therapist group on the Misdemeanor [χ2 (17) = 23.55, p < .38] or the Felony [χ2 (17) = 18.21, p < .13] dependent variable. Given the absence of therapist nesting effects, subsequent analyses omitted this independent variable.

Primary Hypothesis Test Comparing FFT to TAU on Felony Recidivism

We conducted a test of the main hypothesis that the FFT condition would be associated with a lower level of adjudicated felony recidivism than the TAU condition using a binary logistic regression analysis with adjudicated felony recidivism as the dependent variable (no offense = 0, any offense = 1). We created a dummy variable to represent the difference between the control condition (coded 0) and the FFT condition (coded 1), and the analysis also controlled for theoretically specified covariates that assessed risk and protective factors evaluated in step 1 of the analysis. The results of this analysis using the Wald (W) statistic [B = 0.030, SE = 0.186, W(1) = 0.026, p < .873, Exp(B) = 1.030] indicated that the recidivism rate in FFT (22%) did not differ significantly from the TAU (22%) condition. Thus, the present findings are not consistent with prior efficacy trials that evaluated FFT in controlled rather than in community trials. We explored the effects of therapist adherence as well as risk and protective factors as possible explanations of the findings.

Examination of Therapist Model Specific Adherence

We examined the impact of therapist adherence to FFT guidelines on treatment outcomes. Therapists were evenly distributed across levels of model specific adherence (50% were not adherence when collapsed into two categories: adherent/non-adherent). We performed a binary logistic analysis with the felony recidivism status of each adolescent as the dependent variable and two dummy coded variables to represent differences among the three Study samples [Control (0), Low Adherent (0), High Adherent (1) and Control (0), Low Adherent (1), High Adherent (0)] as the principle independent variables. In addition, we included covariates assessing the adolescent’s age, prior criminal history, pretreatment family risk, family protective, and negative peer risk status. All variables were entered simultaneously into the equation, and the omnibus test of the logistic regression model was statistically significant [χ2 (7) = 55.77, p < .001]. Statistical tests for the independent variables and covariates are summarized in Table 1.

Table 1.

Logistic regression coefficients (B), standard errors (S. E.), and Wald (W) statistic for client characteristics and Study Sample Comparison on Adjudicated Felony Recidivism.

Independent Variables B S.E. W p Exp(B)
Control versus High* −0.488 0.249 3.83 .050 .614
Control versus Low* 0.501 0.218 5.29 .021 1.650
Criminal History 0.090 0.024 13.83 .000 1.094
Age −0.309 0.098 9.89 .002 0.734
Gender (Male) 0.629 0.255 6.08 .014 1.875
Family Risk 0.307 0.113 7.38 .007 1.359
Peer Risk 0.162 0.095 2.91 .088 1.175
Constant 1.275 1.498 0.72 .395 3.577
Open in a new tab

Note: Higher scores on independent variables indicate higher levels of the variable being measured. Gender index for females = 0 and for males =1. The omnibus model fit was χ2 (8) = 55.78, p < .000.

*

Control represents the probation only conditions. High represents therapists with high levels of model specific adherence.

As hypothesized, the logistic regression results for separate independent variables indicated that the comparison of High Adherent (covariance adjusted rate = 14.5%) versus the Control sample (22.2%) in felony recidivism rates was statistically significant by the Wald (W) test [B = -.488, SE = 0.249, W(1) = 3.83, p < .050, Exp(B) = .614]. The Control sample was lower in felony recidivism than the Low Adherent sample (28.3%), [B = 0.501, SE = .218, W(1) = 2.76, p < .097, Exp(B) = 1.650]. The findings (see Table 1) indicated that the prior Criminal History, Male Gender, Family Risk and Negative Peer influence were all statistically significantly associated with higher recidivism rates while Adolescent Age was associated with a lower recidivism rate.

We conducted a post hoc comparison between the Low Adherent and the High Adherent FFT samples, and we used a Bonferroni adjusted confidence level to control for the non independence of this test from the prior comparisons. The results indicated that the Low Adherent sample had a higher recidivism rate than the High Adherent sample, [B = −0.845, SE = 0.253, W(1) = 11.16, p < .008]. 2 These findings are consistent with prior research demonstrating that poor therapist adherence reduces the effectiveness of clinical outcomes.

We also conducted secondary analyses to determine the impact of FFT on Misdemeanor and Violent Felony rates. Since the results for the Felony recidivism measure suggested that the Low Adherence group had the highest recidivism rates, we contrasted both the High Adherence and Control groups to the Low Adherence group. The results for the misdemeanor measure indicated that the Low Adherence FFT sample (covariance adjusted misdemeanor recidivism rate = 49.9%) was higher [W(1) = 8.48, p < .004] than the High Adherence FFT sample (35.1%) rate and not different than Control sample (41.7%) rate [W(1) = 3.08, p < .080]. Criminal history, male gender, family risk, negative peer risk, and younger ages were also associated with greater risk of misdemeanor recidivism. Comparisons on the Violent Felony recidivism measure indicated that the Low Adherence FFT sample had higher recidivism (10.4%) rates than the High Adherence sample (4.1%) rate [W(1) = 5.59, p < .018] or the Control group (5.9%) rate [W(1) = 4.34, p < .037]. The findings also indicated that Criminal History, Male Gender, Family Risk, and Younger ages were associated with greater risk of violent felony recidivism.

The Interaction of Therapist Adherence and Risk or Protective Factors on Outcomes

The results described above indicated that higher levels of family risk and negative peer influence were associated with higher felony recidivism rates; family protection was not associated with felony recidivism. We reasoned that the effects of low therapist adherence would be stronger among higher rather than the lower risk cases. We explored the effects of therapist adherence within the high and the low risk factors on behavioral outcomes. Using median splits, we dichotomized the Family and the Negative Peer Relationship dimensions into Low (coded −1) and High (coded +1) risk levels. We compared the felony recidivism rates between the High and Low Adherent samples with a binary regression procedure calculated separately within the two levels of these two pretreatment risk factors. Hence, we used a simple main effects strategy for testing the effects of one independent variable (Therapist Adherence) within levels of another independent variable (Risk or Protection factor). For each of these regression models, we also controlled for criminal history, gender, and age, and we used a Bonferroni adjusted confidence level (a/4 = .0125) to account for the number of correlated tests of significance across the two risk measures. The results of the Adherence contrast are summarized in Table 2.

Table 2.

Summary of analyses for therapist adherence effects on felony recidivism within high or low levels of family risk, and negative peer risk.

Protection or Risk Level Risk Factor
Family Risk Negative Peer Risk
B SE p B SE p
High Level −1.129 0.370 .002 −1.203 0.333 .001
Low Level −.982 0.372 .008 −.276 0.408 .449
Open in a new tab

Note: The cell entries refer to the test of significance for the therapist adherence independent variable within a high or low level of risk or protection. These analyses also statistically control for the effects of criminal history, age, and gender, which are not reported in this table.

These binary logistic analyses results for the High Family Risk cases indicated that the High Adherent FFT therapists had significantly lower recidivism rates (18%) than the Low Adherence FFT therapist (32%) [B = −1.129, SE = 0.370, p < .019, Exp(B)= 0.323]. The results for the Low Family Risk cases also indicated that the High Adherent cases (12%) had significantly lower felony recidivism rates than the Low Adherent therapist (28%) [B = −.982, SE = 0.372, p < .008, Exp(B)= 0.375]. These findings indicated that poor Therapist Adherence was associated with greater recidivism for both Low and High Risk families.

A different pattern of results occurred for the Negative Peer Risk dimension. Among High Peer Risk cases, the High Adherent FFT therapists had significantly lower recidivism rates (20%) than the Low Adherence FFT therapist (33%) [B = −1.230, .333, p < .001, Exp(B) = 0.300] following the pattern with Family Risk. However, the results for the Low Peer Risk cases indicated that the High Adherent (15%) and Low Adherent (20%) therapist cases did not differ significantly [B = −.276, .408, p < .499, Exp(B) = 0.759] in felony recidivism. In other words, these findings suggest that Therapist Adherence had an impact on recidivism when the adolescent in treatment was also exposed to high-risk peers; adherence was not statistically significant for low risk peers. The recidivism rates are presented in Figure 1.

Figure 1.

Figure 1

Open in a new tab

Comparison of Felony Recidivism Rates for the High and Low Adherent Therapists within the High and Low Relationship Peer and Family Risk factors.

Discussion

This study compared the effectiveness of FFT to a specified community probation services in reducing criminal recidivism of more than 900 juvenile offenders in a statewide trial. This study is the largest single randomized trial of FFT, the first to be conducted by community based practitioners in a community setting, and the first to measure the effect of therapist model specific adherence and youth risk and protective factors. When the results were collapsed across all therapists, the FFT intervention was no more effective than the supervised probations services to which it was compared. There was, however, a critical moderating effect of therapist model specific adherence and the client pretreatment risk and protective level. The results indicate that when practiced with model specific adherence FFT resulted in a significant 34.9% and 30% (respectfully) reduction in felony and violent crimes and a non-significant, 21.1% reduction in misdemeanor crimes, The importance of model adherence is also apparent in the finding that adherent therapist group had significantly worse behavioral outcomes when compared to the probation only comparison group. FFT also had a significantly positive impact on youth with the highest levels of family and peer risk levels in this sample. High family and peer risk are important because they are among those risk factors that make positive outcome even more difficult (Elliott, 1997). These results are also consistent with previous FFT efficacy trials that demonstrated a reduction in substance use (see Waldron and Turner, 2008; Waldron, Slesnick, Brody, Turner, & Peterson, 2001).

Similar effects for specific model adherence have been identified in other evidence based treatment models (Henggeler et al., 2007; Hogue et al., 2008). The critical role of model adherence has on client outcomes in FFT is no surprise. Previous studies of FFT were conducted as part of University based training clinics and Research Centers. Those settings allowed a high degree of control over the selection and training of therapists, which also influenced the quality of the treatment. Both clients and therapist in community settings are likely to be much more of the variation in real practice settings that are lost in controlled settings of efficacy trials. In this study, therapists were trained and consistently supervised in a realistic community setting. As a result they were susceptible to many other personal (e.g. attitudes to EBT; competing demands for their time), organizational (ability and desire of organization to support FFT) documented influences that may limit their ability to consistently provide high quality of care (Hoagwood et al., 1995).

This study also examined the role of client factors on outcome of family treatments. We used a well-tested measure of risk and protective factors to identify important interaction effect between therapist adherence level and the adolescent’s pretreatment level of exposure to risky peers. For those youth with high negative peer risk, therapist adherence was a significant factor in predicting successful outcome. However, for low risk youths, the adherent and non-adherent therapists did not have different outcomes. In other words, when a youth is at the highest risk for reoffending due to negative peer influences, the therapist should be particularly careful to be adherent to the model’s guidelines.

The role of therapist adherence to positive outcomes was also apparent when the role of family risk level was considered. These variables reflected the degree to which the youth came to treatment with high levels of within family negativity, blame and conflict (family risk) and low levels of warmth and support in family relationships (family protection). The high fidelity FFT therapists had more favorable outcomes (less recidivism) than low fidelity therapists regardless of whether the families had high or low levels of risk or protective factors. The findings suggest that a model like FFT must be delivered in a clinically specific and precise manner to produce positive outcomes.

Limitations

There are a number of limitations to the present study. While useful and significantly predictive of youth outcome, the method used to measure therapist adherence also has methodological weaknesses. Validity and reliability of the ratings relied on supervision by an FFT model expert, and were dependent on the supervisors’ clinical judgments of therapist adherence to the FFT model. We recognize that some therapists may be able to present cases in a manner that is consistent with the FFT model even though they are not able to skillfully implement the model during their sessions. There is no question that more fine tuned methods (such as video tape ratings) would provide stronger measures of therapist adherence. The measure developed for this study does represent a relevant, reliable, and at the same time practical way to measure this critical variable. In a recent study of FFT, Gilman (2008) found a moderate convergence between FFT supervisor judgment-based ratings, and external raters’ measures. Despite these findings, future studies need to validate a community-based measure for therapist adherence.

There are differences among therapists in regard to learning and implementing FFT that have yet to be identified. As noted above, not all therapists were able to be adherent in the model despite manualized and specified training. It is not clear if differential adherence is a therapist issue or a contextual/situational factor. Hoagwood et al. (1995) suggested that factors such as organizational climate and structure, for example, are important in implementing systematic treatment programs. Further research should focus on the training and supervision dosages as well as the organizational supports necessary to promote the successful implementation of FFT.

The measures of therapist model adherence also have limitations. Community based adherence measures are difficult to achieve. There is a constant challenge to strike a balance between the need for psychometric precision and replicability while also attending to the need for procedures that can be efficiently implemented in a way to ensure its validity. The measure used in this study is based on supervisor judgment across different encounters with the therapist. Because it is judgment based, it is open to influence by a number of factors. Despite the fact that supervisors had no direct access or knowledge of the recidivism outcomes, it is possible that adherence ratings were influenced by information that came to light during the supervision session that was ultimately related to recidivism (say arrest information about an adolescent). Such factors have yet to receive extensive study in the field. In this study, we used both the procedures and the measures to do as much as possible to remove any bias and error out of the adherence measure. Neither the youth nor the supervisors had knowledge of outcomes used in the analysis. In addition, the existence of a crime is unlikely to be known to the therapist given that most of the crimes in the analysis happened within 2 months into the follow up period. Thus, they most often occurred after measures of therapist adherence were taken.

Policy Implications

This study uniquely informs a critical area of interest in the field--the transportation and dissemination of evidence-based treatments. Because of its identification as an “evidence-based” model, FFT has been implemented in a large number of communities in the United States and internationally. This study has a number of implications for the current and future dissemination projects for FFT and other evidence-based treatments. First, the findings reported here suggest that it does matter how FFT is delivered and implemented. For positive outcomes to occur, FFT needs to be delivered in specific and precise ways that demonstrate the therapist adherence to the clinical model particularly with the most difficult youth.

Secondly, this study helps demonstrate a manner in which model specific adherence can be rated as part of the clinical service in a community agency. In randomized clinical trial research studies, careful client screening and sophisticated videotape rating and coding methods are used to determine the degree to which a treatment model is being delivered. In a community setting, this method is expensive and may not be cost-effective. The current study demonstrated that a supervisor-based method of rating therapist adherence could be successfully employed to implement on-going assessment of the quality of a treatment model. This type of assessment may be necessary to help ensure the successful implementation of a treatment model in a community setting. Furthermore, the ongoing assessment protocol provides a form of quality assurance that is within the reach of community practitioners.

Third, it is clear that there is much more to be learned about the types and dosage levels of training needed for effective models to be reproduced in the community. In this study, substantial training and supervision were delivered according to the published training and supervision protocols (Alexander et al., 2000, Sexton & Alexander, 2004; Sexton, Alexander, & Gilman, 2004). Adhering to these treatment protocols has a direct impact on successful youth outcomes. Perhaps more importantly, the adherence rating procedures used in the supervision sessions were able to identify therapists who ultimately had poor outcomes with their clients. However, for a certain group of therapists, this level of training and supervision was not sufficient to promote the successful delivery and outcomes of FFT. A next critical step is to systematically address the mode and dosage of training and supervision and understand the organizational and therapist specific features that influence the ability to produce positive outcomes in FFT.

Acknowledgments

This research was supported in part by grants from the National Institute on Drug Abuse (R01DA023165, R01-DA017218-01A1).

Contributor Information

Thomas Sexton, Center for Adolescent and Family Studies, Indiana University.

Charles W. Turner, Center for Family and Adolescent Research, Oregon Research Institute.

References

  1. Alexander JF, Pugh C, Parsons BV, Sexton TL. Functional Family Therapy. In: Elliott DS, editor. Blueprints for Violence Prevention (Book 3) 2. Boulder, CO: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado; 2000. [Google Scholar]
  2. Alexander JF, Sexton TL. Functional Family Therapy: A model for treating high-risk, acting-out youth, Wiley Series in Couples and Family Dynamics and Treatment. In: Lebow J, editor. Comprehensive Handbook of Psychotherapy Volume IV: Integrative/Eclectic. John Wiley; New York: 2002. [Google Scholar]
  3. Barber JP, Gallop R, Crits-Christoph P, Frank A, Thase M, Weiss RD, Gibbons M. The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute on Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16:229–240. [Google Scholar]
  4. Barnoski R. Washington State Institute for Public Policy. 1998. Juvenile rehabilitation administration assessments: Validity Review. [Google Scholar]
  5. Barnoski R. Washington State’s implementation of functional family therapy for juvenile offenders: Preliminary findings. Washington State Institute for Public Policy. 2002 www.wsipp.wa.gov.
  6. Elliott DS. Blueprints for Violence Prevention. 1 & 2. Boulder, CO: University of Colorado, Center for the Study and Prevention of Violence; 1997. [Google Scholar]
  7. Gilman L. Unpublished doctoral dissertation. Indiana University; Bloomington: 2008. Supervisory interventions and treatment adherence: An observational study of supervisor interventions and their impact on therapist model adherence. [Google Scholar]
  8. Gordon DA, Arbuthnot J, Gustafson KE, McGreen P. Home-based behavioral- systems family therapy with disadvantaged juvenile delinquents. The American Journal of Family Therapy. 1988;16(3):243–255. [Google Scholar]
  9. Henggeler SW, Melton GM, Brondino MJ, Scherer DG, Hanley JH. Multisystemic treatment with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology. 1997;65(5):821–833. doi: 10.1037//0022-006x.65.5.821. [DOI] [PubMed] [Google Scholar]
  10. Henggeler SW, Pickrel SG, Brondino MJ. Multisystemic treatment of substance abusing and dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research. 1999;1:171–184. doi: 10.1023/a:1022373813261. [DOI] [PubMed] [Google Scholar]
  11. Henggeler SW, Chapman JE, Rowland MD, Halliday-Boykins CA, Randall J, Shackelford J, Schoenwald SK. If you build it, they will come: Statewide practitioner interest in contingency management for youths. Journal of Substance Abuse Treatment. 2007;32(2):121–131. doi: 10.1016/j.jsat.2006.07.006. [DOI] [PubMed] [Google Scholar]
  12. Hoagwood K, Hibbs E, Brent D, Jensen P. Efficacy and effectiveness in studies of child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology. 1995;63(5):683–687. doi: 10.1037//0022-006x.63.5.683. [DOI] [PubMed] [Google Scholar]
  13. Hogue A, Dauber S, Samuolis J, Liddle HA. Treatment techniques and outcomes in multidimensional family therapy for adolescent behavior problems. Journal of Family Psychology. 2006;20(4):535–543. doi: 10.1037/0893-3200.20.4.535. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology. 2008;76(4):544–555. doi: 10.1037/0022-006X.76.4.544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Huey SJ, Henggeler SW, Brondino MJ, Pickrel SG. Mechanisms of change in multisystemic therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology. 2000;68(3):451–467. [PubMed] [Google Scholar]
  16. Kazdin AE, Wiesz JR. Evidence-based psychotherapies for children and adolescents. Guilford Press; New York: 2003. [Google Scholar]
  17. Klein NC, Alexander JF, Parsons BV. Impact of family systems intervention on recidivism and sibling delinquency: A model of primary prevention and program evaluation. Journal of Consulting and Clinical Psychology. 1977;45(3):469–474. doi: 10.1037//0022-006x.45.3.469. [DOI] [PubMed] [Google Scholar]
  18. Rowe CL, Liddle HA. Substance abuse. Journal of Marital and Family Therapy. 2003;29(1):97–120. doi: 10.1111/j.1752-0606.2003.tb00386.x. [DOI] [PubMed] [Google Scholar]
  19. Sale E, Sambrano S, Springer F, Turner C. Risk, protection, and substance use in adolescents: A multi-site model. Journal of Drug Education. 2003;33(1):91–105. doi: 10.2190/LFJ0-ER64-1FVY-PA7L. [DOI] [PubMed] [Google Scholar]
  20. Schoenwald SK, Henggeler SW, Brondino MJ, Rowland MD. Multisystemic therapy: Monitoring treatment fidelity. Family Process. 2000;39(1):83–103. doi: 10.1111/j.1545-5300.2000.39109.x. [DOI] [PubMed] [Google Scholar]
  21. Sexton TL. Functional Family Therapy: Traditional Theory to Evidence-based practice. In: Bray JH, Stanton MS, editors. The Wiley-Blackwell Handbook of Family Psychology. New York: John Wiley & Sons; 2009. pp. 327–340. [Google Scholar]
  22. Sexton TL, Alexander JF. Functional Family Therapy Clinical Training Manual. Annie E. Casey Foundation; 2004. [Google Scholar]
  23. Sexton TL, Alexander JF. Functional Family Therapy for Externalizing Disorders in Adolescents. In: Lebow J, editor. Handbook of Clinical Family Therapy. New Jersey: John Wiley; 2006. pp. 164–194. [Google Scholar]
  24. Sexton TL, Alexander JF, Gilman L. Functional Family Therapy Clinical Supervision Manual. Annie E. Casey Foundation; 2004. [Google Scholar]
  25. Sexton TL, Alexander JF, Mease AL. Levels of evidence for the models and mechanism of therapeutic change in family and couple therapy. In: Lambert M, editor. Handbook of Psychotherapy and Behavior Change. New York, NY: Wiley; 2003. [Google Scholar]
  26. Stanton MD, Shadish WR. Outcome, attrition, and family-couples treatment for drug abuse: A meta-analysis and review of the controlled, comparative studies. Psychological Bulletin. 1997;122(2):170–191. doi: 10.1037/0033-2909.122.2.170. [DOI] [PubMed] [Google Scholar]
  27. Waldron HB, Slesnick N, Brody JL, Turner CW, Peterson TR. Treatment outcomes for adolescent substance abuse at 4- and 7-month assessments. Journal of Consulting and Clinical Psychology. 2001;69(5):802–813. [PubMed] [Google Scholar]
  28. Waldron HB, Turner CW. Evidence-based psychosocial treatments for adolescent substance abuse. Journal of Clinical Child & Adolescent Psychology. 2008;37(1):238–261. doi: 10.1080/15374410701820133. [DOI] [PubMed] [Google Scholar]
  29. Waltz J, Addis ME, Koerner K, Jacobson NS. Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology. 1993;61:620–630. doi: 10.1037//0022-006x.61.4.620. [DOI] [PubMed] [Google Scholar]

RESOURCES