Core Elements of Family Therapy for Adolescent Behavior Problems: Empirical Distillation of Three Manualized Treatments

Abstract

Objective:

Family therapy has the strongest evidence base for treating adolescent conduct and substance use problems, yet there remain substantial barriers to widespread delivery of this approach in community settings. This study aimed to promote the feasibility of implementing family-based interventions in usual care by empirically distilling the core practice elements of three manualized treatments.

Method:

The study sampled 302 high-fidelity treatment sessions from 196 cases enrolled in one of three manualized family therapy models: Multidimensional Family Therapy (102 sessions/56 cases), Brief Strategic Family Therapy (100 sessions/94 cases), and Functional Family Therapy (100 sessions/46 cases). Adolescents were 57% male; 41% were African American, 31% White Non-Hispanic, 9% Hispanic American, 6% another race/ethnicity, and 13% unknown. The observational fidelity measures of all three models were used to code all 302 sessions. Fidelity ratings were analyzed to derive model-shared treatment techniques via exploratory factor analyses on half the sample; the derived factors were then validated via confirmatory factor analyses supplemented by Bayesian Structural Equation Modeling on the remaining half.

Results:

Factor analyses distilled four clinically coherent practice elements with strong internal consistency: Interactional Change (6 treatment techniques; Cronbach’s α = .93), Relational Reframe (7 techniques; α = .79), Adolescent Engagement (4 techniques; α = .68), and Relational Emphasis (4 techniques; α = .67).

Conclusions:

The four empirically derived factors represent the core elements of three manualized family therapy models for adolescent behavior problems, setting the foundation of a more sustainable option for delivering evidence-based family interventions in routine practice settings.

This study advances efforts to disseminate family-based treatment in routine care for adolescent conduct and substance use problems by identifying core practice elements of the family therapy approach. Core elements are discrete treatment practices common to multiple treatment manuals for a given behavioral disorder that can be more readily adopted and flexibly implemented than full manuals (Chorpita, Daleiden, & Weisz, 2005). This study distilled the core practices of three well-validated family therapy models via observational coding of high-fidelity treatment sessions.

Formidable Barriers to Implementing Manualized Family Therapy in Routine Care

Family therapy is an evidence-based approach to treating adolescent behavior problems that focuses on intervening directly with family members to repair relationships and addressing challenges encountered by adolescents and caregivers in key extrafamilial systems (Baldwin, Christian, Berkeljon, & Shadish, 2012). There are a handful of “brand name” family therapy models designed to treat adolescent behavior problems (ABPs), with three establishing themselves as efficacious for disruptive behavior, substance use, or both: functional family therapy (FFT), multidimensional family therapy (MDFT), and brief strategic family therapy (BSFT) (Hogue, Henderson, Becker, & Knight, 2018; McCart & Sheidow, 2016). Manualized family therapy models for ABPs have posted an exemplary record of success in comparison to individual and group treatments and, in meta-analyses, produced the largest effect sizes by a healthy margin (Baldwin et al., 2012; Riedinger, Pinquart, & Teubert, 2017; Tanner-Smith, Wilson, & Lipsey, 2013).

Despite their exceptional research portfolio, manualized FT models have not been widely adopted in behavioral treatment systems (Riedinger et al., 2017). Developers of manualized FTs typically disseminate their respective models by establishing corporate entities that contract directly with host agencies to govern adoption and training activities. To promote high-fidelity delivery, each brand-name model contains an extensive set of quality assurance procedures anchored by a standardized training toolkit, guidelines for ongoing training and observational consultation from model experts, and implementation support and fidelity tracking methods that feed therapy session data back to providers (see Hogue, Ozechowski, Robbins, & Waldron, 2013). Such procedures incur substantial costs numbering tens of thousands of dollars annually for initial training plus certification maintenance (Substance Abuse and Mental Health Services Administration, 2012). In addition, manualized FTs prescribe numerous complex treatment procedures, often with a fixed intervention sequence—features that inhibit treatment selection and tailoring practices for clients favored by community clinicians (Chorpita et al., 2005).

Core Elements: Innovative Approach to Implementing Evidence-Based Interventions

Implementation barriers such as those listed above are common to manualized treatments of many kinds, not just FT models. In response, experts in mental health (e.g., Chorpita, Becker, & Daleiden, 2007; Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008) and substance use advocate an alternative strategy to complement manual-driven methods: Focus on core elements of evidence-based interventions (EBIs) that represent a reduced set of intervention techniques common to multiple treatments for a given disorder. This is achieved by (a) specifying the discrete techniques prescribed by similar treatment manuals and (b) distilling these techniques into a smaller number of overlapping practice elements that are core features of each manual (Chorpita & Daleiden, 2009). Thus whereas manuals are predominantly complex, uniform, and disorder-specific, distilled core elements are instead granular, flexible, and transdiagnostic—three user-centered features (Lyon & Koerner, 2016) that may help solve vexing barriers associated with implementing manuals in routine care (Weisz et al., 2017). For example, core practices can be more readily learned by clinicians and flexibly applied to a larger set of problems, making them well-suited for client-tailored treatment planning (Garland, Bickman, & Chorpita, 2010). The transdiagnostic feature of core practices is highly germane to ABPs, insofar as conduct problems and substance use problems share a large set of common risk and protective factors during adolescence, for which a core set of FT techniques appears applicable and effective (see Baldwin et al., 2012; Henggeler & Schaeffer, 2016; Hogue, Dauber, Henderson, et al., 2015).

In the most ambitious distillation project yet undertaken, Chorpita and Daleiden (2009) reviewed 322 randomized trials of EBIs for youth populations across the behavioral spectrum. They reliably isolated 41 core elements variously used to treat anxiety and depression, conduct problems, attention and impulsivity, sleep and eating problems, autism-spectrum problems, substance use, and so forth. Notably, from among the numerous trials containing a FT condition, the authors identified only one undifferentiated code broadly termed “family therapy”. This underscores the absence of progress to date in distilling core elements for the FT approach.

Family Therapy for Adolescent Behavior Problems: Primed for Core Elements Distillation

The FT approach for ABPs is well positioned to support a distillation process with far more differentiated results than the solitary “family therapy” element isolated in the Chorpita and Daleiden (2009) taxonomy. There is ample evidence from existing research that three brand-name FT models for ABPs—FFT, MDFT, BSFT—share a set of common treatment techniques that are operationalized in their respective model fidelity metrics, are empirically linked to client outcomes, and can be readily synthesized into core elements. FFT contains family interventions such as minimizing blame (Alexander et al., 1989), reframing negative behaviors (Robbins et al., 1996), and interrupting defensive interactions among family members (Robbins, Alexander, & Turner, 2000) that have been correlated with in-session and post-treatment outcomes. MDFT contains several techniques ubiquitous within the FT approach (e.g., coach in-session family interactions; target caregivers for change) that predict long-term improvement in family functioning as well as adolescent symptoms (Hogue, Dauber, Samuolis, & Liddle, 2006; Hogue et al., 2008). BSFT is anchored by signature techniques of the FT approach—joining, diagnostic enactment, reframing, and restructuring interventions—that variously predict improvements in treatment engagement, family functioning, and adolescent symptoms (Robbins et al., 2011).

Buoyed by this cohesive portfolio of process-outcome research, the authors completed a conceptually-driven distillation process to isolate the core elements of these manualized FT models (Hogue et al., 2017). Our conceptual distillation process focused on the observational fidelity scales corresponding to each model: FFT (Ozechowski & Waldron, 2016), MDFT (Hogue et al., 1998), and BSFT (Hervis & Robbins, 2015); see Study Measures. This process thereby leveraged the available fidelity blueprints laid bare by the respective model developers to operationalize discrete techniques considered essential to model adherence. In accord with standard distillation procedures (Chorpita & Daleiden, 2009), the various treatment techniques contained in the three fidelity scales were examined to identify thematic clinical strategies that appeared to be (1) common across the models, (2) theoretically salient to the FT approach, and (3) embodied by multiple techniques from all three scales. This process yielded four core practice elements: (a) Family Engagement: enhance family member involvement in treatment and build the relationship between therapist and all members; (b) Relational Reframing: de-emphasize individual and intrapsychic ways of defining problems in favor of a systemic conceptualization focused on relational processes; (c) Family Behavior Change: teach new skills and encourage individual behavior changes that promote improved family relations; (d) Family Restructuring: prompt changes in family emotional processes and encourage insight into predominant cycles of relational interactions.

Study Aim: Identify EBI Core Elements via Novel Empirical Distillation Methods

The aim of the current study was to empirically distill core elements of FT for ABPs via observational ratings of therapy sessions using model-specific fidelity scales. To our knowledge this study is the first to utilize empirical distillation methods to identify EBI core elements. Prior distillation efforts have used exclusively conceptual methods, which rely on expert review of the content of treatment manuals and/or protocol descriptions, usually fortified by team-based coding and consensus procedures among multiple reviewers (e.g., Chorpita & Daleiden, 2009), and sometimes further confirmed by expert survey (e.g., Garland et al., 2008). Conceptual methods possess the considerable virtues of strong face validity, modest technical demands, and flexibility in review procedures. By the same token, they are subject to legitimate questions regarding the reliability and generalizability of distillation results: How credibly do the distilled elements represent the original EBI content? Would different groups of experts reach meaningfully different results, and if so, what are the implications for the clinical validity of the elements themselves? These questions loom largest for complex EBIs that resist disaggregation or easy reduction to simpler constructs, such as manualized FT (Chorpita & Daleiden, 2009).

To meet our study aim we developed a novel, two-part empirical distillation method. First, we collected observational fidelity data on 302 treatment sessions of FFT, MDFT, and BSFT. As described below in Method, all sessions were verified to be high-fidelity exemplars of the respective models, thereby constituting a “gold standard” sample pool that incontrovertibly represented the models and their constituent treatment techniques. We employed a fully-crossed design in which observers rated each of the 302 sessions for the occurrence of all techniques from all three fidelity scales. Though time-intensive and technically demanding, nonparticipant ratings remain the most rigorous method for assessing treatment fidelity in both research trials (Hogue, Liddle, & Rowe, 1996) and standard practice settings (Garland et al., 2010). Second, we conducted factor analyses of the fully crossed observational data set to distill its latent, core FT elements. We randomly split the sample in half, deriving core factors via exploratory factor analyses and then validating the derived factors via confirmatory factor analyses supplemented by Bayesian analysis. We conservatively hypothesized that these empirical methods would generate core FT elements mirroring those obtained from our conceptual distillation: Family Engagement, Relational Reframing, Family Behavior Change, and Family Restructuring.

Method

Study Sample: Three High-Fidelity Sample Pools

This study sampled 302 videotaped sessions from 196 cases enrolled in three manualized FT models for ABPs: MDFT (101 sessions/56 cases), BSFT (100 sessions/94 cases), and FFT (100 sessions/46 cases). Adolescents who attended sessions were 57% male; 41% were African American, 31% White Non-Hispanic, 9% Hispanic American, 6% another race/ethnicity, and 13% unknown. MDFT sessions were selected from a controlled prevention trial involving high-risk young adolescents (11-14 years) (Hogue, Liddle, Becker, & Johnson-Leckrone, 2002). Trial therapists demonstrated strong adherence to the MDFT model via observational ratings (Hogue, Liddle, Singer, & Leckrone, 2005). There were five male therapists: two African American, two White Non-Hispanic, one Asian American. BSFT sessions were selected from an archive of expert-supervised training sessions in which community therapists treated adolescents (13-18 years) in outpatient care for ABPs. All BSFT sessions were observationally rated by model experts as demonstrating above-average model fidelity; sessions included 53 therapists from diverse backgrounds (specific race/ethnicity data were not available). FFT sessions were selected from an archive of controlled trials involving adolescents (14-19 years) with conduct and substance use disorders (e.g., Rohde et al., 2014; Waldron et al., 2001). Sessions were chosen from among those having been observationally rated as exemplary in model fidelity. There were 11 FFT therapists (nine female); race/ethnicity data were not available. Establishing the strong fidelity of sampled sessions is essential in order to ensure that distilled techniques validly represent the three manualized models; however, source fidelity data from the three sample pools were not used in study analyses.

Sample Selection Procedures

Across all three sample pools, sessions were randomly chosen to maximize representation across treatment duration: one apiece from Sessions 1 – 3 (Early), 4 – 9 (Middle), and 10+ (Later), depending on availability (as described below). Only sessions lasting at least 30 minutes were retained. Adolescents and caregivers appeared together in 88% of sessions, with 7% of sessions containing only adolescents and 5% only caregivers. The MDFT pool (102 sessions/56 cases) was randomly selected from 574 recorded sessions of the 56 cases: 28% Early, 34% Middle, and 38% Later. The BSFT pool (100 sessions/94 cases) was chosen by model experts with personal knowledge of each case. Because this was a training sample, almost every available BSFT case had only one recorded high-fidelity session, and the bulk of recorded sessions represented early treatment: 52% Early, 36% Middle, 12% Later. The FFT pool (100 sessions/46 cases) was randomly selected to maintain a balance of cases deemed by model experts to be “easy” versus “difficult”: 27% Early, 51% Middle, 22% Later.

Study Measures

Multidimensional Family Therapy Therapist Behavior Rating Scale (MDFT-TBRS).

The 13-item MDFT-TBRS (Hogue et al., 1998) describes treatment techniques for engaging family members in therapy, improving parenting and family skills, intervening in family interaction patterns, and addressing key developmental issues involving extrafamilial systems. The scale measures MDFT extensiveness, that is, the thoroughness/frequency with which each technique was utilized in the observed session, based on a 5-point Likert-type scale: 1 = Not at all, 2 = A little bit, 3 = Moderately, 4 = Considerably, 5 = Extensively. The scale has shown strong factor properties, interrater reliability (ICCs = .60-.89), internal consistency (α = .67-.81), and links to client outcomes in studies of high-risk youth (Hogue et al., 2005) and youth with conduct and substance use disorders (Hogue et al., 2006; Hogue et al., 2008).

Brief Strategic Family Therapy Fidelity Rating Scale (BSFT-FRS).

The 24-item BSFT-FRS (Hervis & Robbins, 2015) specifies clinical techniques associated with four domains of structural-strategic FT: Joining (alliance building with all family members), Tracking (assessing family relations in session), Reframing (endorsing positive meaning to attributions/behaviors), Restructuring (realigning family boundaries and strengthening relationships). Analyses from a multisite effectiveness study (Robbins et al., 2011) revealed strong interrater reliability across model domains (ICCs = .81-.85), excellent construct validity (measurement model yielded CFI = .94 and RMSEA = .08 on cross-validation sample; CFI = .94 and RMSEA = .08 on full sample), and robust item-factor loadings and composite reliability: .32-.77 (ICC = .85) for Joining, .48-.86 (ICC = .78) for Tracking, .99-1.00 (ICC = .99) for Reframing, and .32-.82 (ICC = .83) for Restructuring, with factor intercorrelations ranging from .43-.80. BSFT-FRS fidelity ratings predicted client retention and symptom improvement. For consistency, in the current study the BSFT-FRS used the same 5-point extensiveness anchors described for MDFT-TBRS.

Functional Family Therapy Therapist Adherence Rating Scale (FFT-TARS).

The 14-item FFT-TARS (Ozechowski & Waldron, 2016) describes treatment techniques that capture five basic phases of the FFT model: Engagement, Motivation, Relational Assessment, Behavior Change, Generalization. FFT focuses on establishing balanced alliances with family members, assessing the relational functions of individual behaviors, utilizing reframing and relabeling techniques as meaning-changing interventions, and teaching new family skills that generalize to multiple contexts (Alexander, Waldron, Robbins, & Neeb, 2013). FFT-TARS items derive from forerunning FFT observational coding scales that demonstrated strong interrater reliability, internal consistency, construct validity, and predictive validity in numerous process-outcome studies (e.g., Alexander et al., 1989; Robbins et al., 2000; Robbins et al., 1996). Again, in the current study the FFT-TARS used the same 5-point extensiveness anchors described above.

Observational Coding Procedures and Raters

A separate cohort of observational raters was trained to code each of the three sample pools. Raters in all three cohorts were trained during twice-weekly meetings via review of the relevant rating manual, in-group coding practice, and exercises to increase understanding of scale items. Study coding in each cohort commenced once raters reached a collective reliability threshold of intraclass correlation coefficient (ICC: Shrout & Fleiss, 1979) = .65 for 80% of items, and monitored thereafter. All sessions were independently coded in their entirety by two raters randomly assigned to sessions in pairs according to a balanced incomplete randomized block design (Fleiss, 1981). The MDFT rater cohort contained 14 females (22 – 32 years): nine White Non-Hispanic, two African American, one Hispanic American, one Asian American. Seven had a Master’s degree and five were enrolled in a graduate psychology program. The BSFT cohort contained six females (22 – 32 years): five White Non-Hispanic and one African American. Four had a Master’s degree. The FFT cohort contained six females (23 – 33 years): four White Non-Hispanic, one African American, one Hispanic American. Four had a Master’s.

Plan of Analysis

Study analyses occurred in four stages. In Stage 1: Item Selection, inter-rater reliability and descriptive statistics for each item on each coding scale were calculated separately for each sample pool. Inter-rater reliability was calculated using the one-way random intraclass correlation coefficient (ICC _(1,2); Shrout & Fleiss, 1979). Based on the following decision rules, we excluded 13 items (2 MDFT; 6 BSFT; 5 FFT) from further analysis: (1) items with an ICC < .30 and with a p-value above .10 in at least one sample pool; (2) items with skew or kurtosis above 10 in at least one sample pool. The remaining 38 items proceeded to Stage 2 analyses. Prior to Stage 2 analyses, item scores were averaged across both raters to yield a single score for each item within each sample pool. The sample was randomly split into two half samples to allow for exploratory factor analysis (EFA) on one half followed by confirmatory factor analysis (CFA) on the other. The random split was conducted within each of the sample pools to ensure equal representation of each sample pool within each half sample.

In Stage 2: EFA, we used principal components EFA with direct oblimin rotation, applying the sandwich variance estimator to account for the nesting of sessions within therapists. Because our study aim was to describe how the various observed techniques clustered together in this set of exemplary sessions, principal components EFA, which describes all variance in the data set (i.e., how experts delivered each model), is an appropriate approach for distilling core practices. EFA was conducted on the first half sample (N = 151), and 1 through 6 factor solutions were extracted. The 4-factor solution (see Results) was ultimately selected based on eigenvalue decline and overall interpretability; items with factor loadings less than .40 were trimmed from the final model solution to maximize parsimony (Tabachnick & Fidell, 2007).

Following EFA, Stage 3: CFA was conducted on the other half sample (N = 151) to confirm the fit of the 4-factor solution derived from EFA. First, preliminary CFA models were estimated for each of the factors separately to achieve adequate fit for each factor individually, before attempting to fit the full 4-factor model. Modification indices were examined to guide model adjustments needed to maximize model fit, and poorly performing items were trimmed as needed. Model fit was assessed using the model chi-square statistic and two supplementary fit indices, RMSEA and CFI. RMSEA values of .06 and below, and CFI above .95, indicate strong model fit, and CFI >= .90 and RMSEA <= .08 indicate adequate fit (Browne & Cudeck, 1993; McDonald & Ho, 2002). Both EFA and CFA were conducted in Mplus 7.31 (Muthén & Muthén, 1998-2017). The sandwich variance estimator was used to account for the nesting of sessions within therapists (Asparouhov, 2005).

As described in Results, the full 4-factor CFA model failed to converge. Muthén and Asparouhov (2012) argue that the typical implementation of CFA modeling using maximum-likelhood (ML) estimation applies overly strict assumptions that often lead to model misfit, triggering a series of model modifications that may capitalize on chance and fail to replicate in future studies. As an alternative they propose using Bayesian analysis to relax these assumptions and provide “wiggle room” for researchers to specify small non-zero ranges into which these estimates can fall. Bayesian estimation provides a means for incorporating a priori knowledge from previous research directly into the statistical model (Ozechowski, 2014; van de Schoot et al., 2014). This is accomplished by specifying a prior distribution, which expresses a priori information about how the parameters estimated in the statistical model are distributed in the population (Ozechowski, 2014). A Bayesian analysis involves three elements: (a) prior knowledge on the parameter being tested, captured by the prior distribution (parameter estimate and its associated variance); (b) information provided by the data at hand (likelihood function); and (c) the posterior distribution, which represents the combination of the two previous elements and is derived using Bayes’ theorem. The point estimate of the parameter of interest represents the mean of the posterior distribution, and the stability of the estimate, on which inferences are made, is known as a credible interval (akin to conventional confidence intervals), which comprises the 2.5^th and 97.5^th percentiles from the posterior distribution (Ozechowski, 2014).

Parameters in these models are estimated using Markov Chain Monte Carlo (MCMC) estimation. MCMC is an iterative process in which a prior distribution is specified and posterior values for each parameter are estimated over many iterations, which in turn are used to construct the posterior distribution. MCMC is initiated from at least two randomly selected starting points to facilitate convergence of the iteration process (Zyphur & Oswald, 2013). Convergence is indicated graphically as well as statistically via the potential scale reduction (PSR; Asparouhov & Muthén, 2010). PSR indexes the ration of total variance across chains to the pooled variance within a chain. Smaller PSR values (e.g., PSR < 1.05) indicate that convergence has occurred. Because conventional model fit indices are based on ML estimation, Mplus provides an alternative fit index to evaluate model fit, the posterior predictive p (PPP) value, which is less sensitive than chi-square testing to model misspecification. A PPP value greater than .05 indicates good model fit (Asparouhov & Muthén, 2010). Specifically, Muthén & Asparouhov (2012) advocate using prior distributions with small variances for cross-loading factors and residual covariances. Following their recommendations, we specified prior distributions for these parameters of a mean of 0 and variance of .01, which translated to standardized values ranging between −0.2 and .2. In addition to the advantages discussed above, Bayesian estimation has the well-documented advantage over CFA of yielding optimal estimates, and more consistent convergence, with small samples (van de Schoot et al., 2014). Thus, Stage 4 of the analysis consisted of Bayesian Structural Equation Modeling (BSEM) for the 4-factor model that failed to converge with CFA specification. We specified this model as a three-level model with sessions nested within individuals within therapists. We estimated models with informative priors using factor loadings and standard errors estimated in the EFA (see Table 1) for the factor loadings in the BSEM, and small variance priors (.01) for cross-loading items and residual covariances.

Table 1.

Results of Exploratory Factor Analysis on Reliably Coded Fidelity Items for Three Family Therapy Models

	Family Therapy Model	Factor 1: Interactional Change	Factor 2: Relational Reframe	Factor 3: Adolescent Engagement	Factor 4: Relational Emphasis
Factor Eigenvalue		8.05	3.25	2.27	1.89
Factor 1: Interactional Change
Stimulates dialogues/Directs enactments	BSFT	.984	−.124	−.029	−.003
Coaches interactions in session	TBRS	.940	−.027	−.001	.003
Moves close to direct restructuring maneuvers	BSFT	.938	−.052	−.031	−.045
Remains decentralized after enactment emerges	BSFT	.782	.131	.012	−.139
Conducts in-session exercises for new behaviors	FFT	.586	.363	.062	−.051
Focuses on present interactions	BSFT	.551	.319	.151	−.002
Actively directs and elicits new behaviors*	BSFT	.530	.147	−.004	.113
Follows non-verbals to elicit participation*	BSFT	.469	.022	.073	.119
Intensifies/Highlights interactions*	BSFT	.486	.208	−.025	.141
Manages or responds to negativity*	FFT	.449	.352	−.056	.032
Factor 2: Relational Reframe
Provides a family-focused rationale for change	FFT	−.075	.855	−.075	.116
Offers more positive view of problems/family	BSFT	−.003	.703	.286	−.060
Utilizes meaning-change interventions (reframe)	FFT	.023	.695	−.023	.258
Maintains a relational focus	FFT	.219	.509	.020	.424
Provides information/guidance for new skill	FFT	.462	.505	.029	−.059
Develops relational reframe for youth problems	TBRS	−.220	.489	.051	−.095
Targets adult participants for change	TBRS	.157	.450	−.272	.005
Factor 3: Adolescent Engagement
Joins with children/adolescents	BSFT	−.013	−.018	.862	.066
Targets interventions toward youth	BSFT	.099	.196	.823	.038
Explores adolescent ecosystem	TBRS	−.399	−.158	.446	−.074
Supports adolescent investment in therapy	TBRS	−.116	.128	.421	.029
Factor 4: Relational Emphasis
Gathers information on relationship functions	FFT	−.129	.016	−.126	.865
Asks clarifying questions/Focuses on process	BSFT	−.219	.016	.124	.602
Enhances family attachment/communication	TBRS	.337	−.094	.085	.487
Connects with all family members	BSFT	.062	−.008	.270	.470

Note. Items with an

^*were eliminated at the CFA phase to achieve adequate model fit.

Results

Stage 1. Item Selection

One-way random ICC_(1,2) was calculated for each item on each of the three coding scales within each of the three sample pools to examine inter-rater reliability. ICCs were interpreted based on Cicchetti’s (1994) criteria for classifying ICC magnitudes: below .40 is poor, .40-.59 is fair, .60-.74 is good, and .75 to 1.00 is excellent. As described in the Method section, 13 items did not meet inclusion criteria (ICC > .30 and p < .10 OR skew and kurtosis < 10 in all three samples). ICCs described here are for the remaining 38 items: 11 MDFT, 18 BSFT, 9 FFT.

ICCs for the 11 MDFT items were mostly in the good to excellent range for the MDFT sample, ranging from .64 to .90. Two items had ICCs below this range at .58 and .38. ICCs for MDFT items were lower in the BSFT sample: Four ranged from .62 to .71, five ranged from .41 to .58, one was .33, and two were below .30 but were retained because significance was less than p = .10. In the FFT sample, MDFT items ranged from .79 to .90. On the BSFT rating scale, the majority of items had good or excellent ICCs in the MDFT sample ranging from .62 to .93, while three items ranged from .51 to .58. In the BSFT sample, six items ranged from .60 to .78, six items ranged from .40 to .56, four items ranged from .30 to .39, and two were below .30 but retained because significance was less than p = .10. In the FFT sample, most BSFT items ranged from .61 to .83, with two in the fair range (.51 and .52). For the FFT scale, nearly all items had good or excellent ICCs in the MDFT sample, ranging from .63 to .80. One item was .31 and one was .30; both were retained due to p < .10. In the BSFT sample, five items ranged from .62 to .70, and four ranged from .45 to .59.items. In the FFT sample, three items ranged from .67 to .71, three ranged from .54 to .59, and one was .36.

Stage 2. Exploratory Factor Analysis

EFA was conducted on half of the sample to determine the optimal factor structure. All 38 items meeting ICC inclusion criteria were included in EFA and solutions ranging from one to six factors were extracted. In the original model including all 38 items, eigenvalues were 9.67 for one factor, 4.16 for two factors, 2.72 for three factors, 2.18 for four factors, 1.73 for five factors, and 1.62 for six factors. Factor solutions for four, five, and six factors were examined, and the 4-factor solution was deemed most interpretable. Ten items that did not load above .40 on any factor were trimmed from the model (5 MDFT, 3 BSFT, 2 FFT). Three additional BSFT items that consistently loaded on a separate “Parent Engagement” factor but interfered with model fit and interpretation were trimmed as well. The final 4-factor solution included 25 items: 6 MDFT, 12 BSFT, 7 FFT. Table 1 displays the items organized by loadings on derived factors. Modest correlations between factors indicated substantial factor differentiation (non-overlap): Factor 1 and Factor 2: r = .33; Factor 1 and Factor 3: r = .14; Factor 1 and Factor 4: r = .01; Factor 2 and Factor 3: r = .13; Factor 2 and Factor 4: r = .15; Factor 3 and Factor 4: r = .16.

Factor 1 was named Interactional Change and included 10 items:1 MDFT, 7 BSFT, 2 FFT. Three BSFT items and one FFT item were dropped from analysis at the CFA stage (see below) and thus were not included on the final version of this factor. Factor loadings for the 6 items on the final version of Factor 1 ranged from .98 (BSFT: Stimulates dialogues/Directs enactments) to .55 (BSFT: Focuses on present interactions). Factor 2 was named Relational Reframe and included seven items: 2 MDFT, 1 BSFT, 4 FFT. Factor loadings ranged from .86 (FFT: Provides a family-focused rationale for change) to .45 (MDFT: Targets adult participants for change). Factor 3 was named Adolescent Engagement and included four items: 2 MDFT, 2 BSFT. Factor loadings were above .80 for both BSFT items (Joins with children/adolescents, Targets interventions toward youth) and were .45 and .42 for MDFT items (Explores adolescent ecosystem, Supports adolescent investment in therapy). Factor 4 was named Relational Emphasis and had four items: 1 MDFT, 2 BSFT, 1 FFT. Factor loadings ranged from .87 (FFT: Gathers information on relationship functions) to .47 (BSFT: Connects with all family members).

Stage 3. Confirmatory Factor Analysis

The 4-factor EFA solution was confirmed on the remaining half sample using CFA. First, preliminary CFA models were calculated for each factors individually, and modification indices were used to guide model adjustments until adequate fit was achieved. To achieve fit on the Interactional Change factor, 4 items were trimmed from the final factor model (see Table 1). Thus, 21 items progressed from the EFA stage to the final confirmed four factors. Model fit for each factor in the CFA model was evaluated using chi-square, RMSEA, and CFI. Fit indices for Interactional Change were: χ² (9) = 16.65, p = .05; RMSEA = .08 (90% CI: .00 - .13); CFI = .99. Fit indices for Relational Reframe were: χ² (13) = 19.41, p = .11; RMSEA = .06 (90% CI: .00 - .11); CFI = .98. Fit indices for Adolescent Engagement were: χ² (2) = 4.08, p = .23; RMSEA = .08 (90% CI: .00 - .20); CFI = .99. Fit indices for Relational Emphasis were: χ² (2) = 2.69, p = .26; RMSEA = .05 (90% CI: .00 - .18); CFI = .99. Evaluation of fit indices indicated that model fit was adequate for each of the four factors individually. However, when combined into a single CFA model, the full 4-factor model failed to converge. Measurement invariance models attempting to confirm relative equivalence of the 4-factor structure across all three sample pools (MDFT, BSFT, FFT) also failed to converge.

Stage 4. Bayesian Structural Equation Modeling

To confirm the 4-factor structure, we estimated a three-level BSEM model with small variance priors for cross-loading items and residual covariances. This model converged (PSR = 1.001 replicated over numerous iterations) and showed good fit to the data (PPP = 0.441). Table 2 displays the factor loadings estimated in BSEM. Items loaded robustly on hypothesized factors, and with few exceptions, were minimally correlated with cross-loading factors. Likewise, residual correlations were small (M = .004, Range = −0.16 to 0.15). Again, the derived factors were modestly correlated, indicting factor differentiation: Factor 1 and Factor 2: r = .36; Factor 1 and Factor 3: r = .18; Factor 1 and Factor 4: r = .05; Factor 2 and Factor 3: r = .04; Factor 2 and Factor 4: r = .20; Factor 3 and Factor 4: r = .30.

Table 2.

Results of Three-Level Bayesian Structural Equation Modeling on Fidelity Items from Three Family Therapy Models

	Factor 1: Interactional Change	Factor 2: Relational Reframe	Factor 3: Adolescent Engagement	Factor 4: Relational Emphasis
Coaches interactions in session	.891	−.016	−.018	−.033
Moves close to direct restructuring maneuvers	.884	−.029	−.007	.006
Stimulates dialogues/Directs enactments	.823	−.061	−.015	−.038
Remains decentralized after enactment emerges	.802	−.056	−.001	−.002
Conducts in-session exercises for new behaviors	.667	.115	−.027	.012
Focuses on present interactions	.625	.071	.091	.061
Provides a family-focused rationale for change	−.093	.848	−.006	−.028
Utilizes meaning-change interventions (reframe)	.002	.735	−.064	.006
Offers more positive view of problems/family	−.046	.645	.068	.053
Provides information/guidance for new skill	.196	.529	.022	−.066
Maintains a relational focus	.025	.528	.072	.170
Targets adult participants for change	.011	.492	−.131	−.036
Develops relational reframe for youth problems	−.105	.432	.040	−.094
Joins with children/adolescents	.005	−.014	.882	.043
Targets interventions toward youth	.084	.062	.748	−.049
Supports adolescent investment in therapy	−.044	−.016	.464	−.022
Explores adolescent ecosystem	−.133	−.060	.406	−.013
Gathers information on relationship functions	−.079	−.013	−.083	.809
Asks clarifying questions/Focuses on process	−.059	−.007	.015	.643
Enhances family attachment/communication	.080	.044	−.018	.467
Connects with all family members	.092	.011	.120	.465

Note. Items in bold are hypothesized to load on the corresponding factor. Italicized items fall outside the 95% credibility interval.

Discussion

This empirical distillation of three manualized family therapy models for adolescent behavior problems yielded four core practice elements, summarized as follows. The first element was labelled Interactional Change (defined by 6 treatment techniques) and included interventions in which therapists allow or prompt family members to interact with one another naturally in order to assess family dynamics, and also, direct in-session interactions among members in order to promote more effective ways of relating. By creating opportunities to communicate more meaningfully, therapists support families in developing new relational skills. The second element, Relational Reframe (7 techniques), involved therapist efforts to transform symptom-focused and/or adolescent-focused perceptions of clinical problems into a new understanding of those problems as being fundamentally relational, thereby motivating families to pursue changes in family relationships as the primary clinical solution. The third element, Adolescent Engagement (4 techniques), described interventions in which therapists join with adolescents by seeking their unique points of view, and also, foster treatment engagement by presenting family therapy as an opportunity to address personally meaningful issues within and outside the family. Last, Relational Emphasis (4 techniques) contained interventions that focus on the family as a whole, assessing systemic attributions and processes and intervening to improve overall family functioning. Not coincidentally, these four elements map closely onto the framework of the structural family therapy model (see Minuchin & Fishman, 1981), a common progenitor of manualized FT models and for the family therapy approach in general.

We have strong confidence in the clinical and psychometric validity of study findings. The study sample—videotaped therapy sessions—was certified by model developers as being highly faithful to their respective FT models. The distillation tools—model-specific observational fidelity scales—were produced by model developers for the explicit purpose of evaluating the implementation of essential model strategies and techniques. The analytic methods—observational coding and multilevel factor analysis—adhered to rigorous principles of data reliability and factor resolution. And all three models contributed at least one treatment technique to three of the four distilled elements (the exception being that Adolescent Engagement does not contain an FFT technique, though model experts confirm that adolescent engagement is core to the model). Thus there is strong justification that the derived factors are indeed core practice elements of the three models in question.

Though hardly exhaustive of the FT approach, the four core elements are a representative foundation of common FT strategies. Certainly, the 21 techniques that collectively define the elements do not capture the full complement of interventions prescribed by the three models. For example, all manualized FT models designed for ABPs (see also Henggeler & Schaeffer, 2016; Slesnick, Erdem, Bartle-Haring, & Brigham, 2013) feature some degree of case management interventions, including home-based wraparound services—an important category of interventions that fell outside the purview of the current study. Even so, the 21 distilled techniques are commonly shared FT interventions that are linked to client outcomes for all three observed models and therefore hold great potential for dissemination as effective practices.

Results of this empirical distillation process differed from results of the conceptual process described above (Hogue et al., 2017) in three main ways. First, an entirely new element, Relational Emphasis, emerged. This element makes explicit the foundational concept of the FT approach: Family relations are the primary targets of assessment and change-making interventions. Therapists who work with families but maintain a fundamentally individualistic and/or intrapsychic approach are not operating with this concept as a core value. Second, two elements from the conceptual distillation (Family Behavior Change, Family Restructuring) effectively merged into a single element (Interactional Change). The new element blends FT interventions aimed at changing observable family behaviors with those aimed at changing underlying (i.e., structural-level) family attributions, roles, and relational processes. Both types of change appear advantageous for addressing ABPs. Third, interventions in the conceptual element Family Engagement that pertain to engaging caregivers in treatment were stripped away during empirical distillation, leaving the reduced element Adolescent Engagement in its place. Caregiver-focused engagement interventions instead aligned with the Relational Reframe and Relational Emphasis elements. There is no doubt that all three models emphasize the need to cultivate and balance alliances with multiple family members. This result from empirical distillation could signify that caregiver engagement is conceptualized quite differently across the three models, foreclosing the possibility of a unified element; and/or, that caregiver engagement interventions are an embedded feature of relational reframing and relational focus processes, whereas efforts to engage adolescents constitute a distinct therapeutic agenda.

These deviations from the conceptual distillation process highlight the potential value of empirical distillation procedures, which hew more closely to the observable realities of EBI implementation and thereby afford opportunities to enrich or even correct conceptually-based results. Indeed, empirical procedures open the door to distillation solutions that might differ meaningfully across service contexts. For example, how might the current FT elements, derived from sessions representing training and research activities under tight control of model developers, diverge if study methods were applied to sessions from community treatment settings wherein model fidelity varies to a greater extent (Hallgren et al., 2018)? Such potential variations will be increasingly relevant as core element EBIs gain traction in community-based practice. Along these lines, the path seems clear to distill core elements of other EBIs for ABPs, principally cognitive-behavioral therapy (Hogue et al., 2018; McCart & Sheidow, 2016).

It is critical to underscore that core elements are not equivalent to manualized treatments. In addition to discrete intervention techniques, treatment manuals invariably articulate principles of treatment coordination—rules for the timing, sequencing, and client- and context-specific targeting of interventions—that constitute the unique parameters and implementation nuances of a given model (Chorpita et al., 2005). Coordination principles determine, for example, how rigidly versus flexibly a therapist should implement model content, as well as the recommended balance between fidelity versus adaptation for individual cases or clinical groups (McHugh, Murray, & Barlow, 2009). Core elements of EBIs thus cannot supplant full treatment models or be utilized effectively as “brief” versions.

Study Strengths and Limitations

Study strengths include the innovative empirical approach to EBI distillation, notably the rigorous observational data collection and analytic methods, and the diversity of the sample, which supports the generalizability of findings. One study limitation was exclusive focus on the extensiveness (i.e., adherence), rather than the expertise (i.e., competence), with which therapists delivered treatment techniques. Therapist expertise in implementing specific techniques is quite difficult to judge reliably (Webb, DeRubeis, & Barber, 2010) and is not directly germane to distilling which techniques are core across models. A second limitation was inability to demonstrate measurement invariance of the 4-factor solution, which leaves open the possibility that one of the sample pools exerted a disproportionately strong or weak influence on the distillation process. This possibility is contraindicated to some degree by the representative distribution of intervention techniques from all three models across all four factors. We plan to conduct additional analyses of measurement invariance on this sample in a follow-up study that will leverage item response theory; those complex analytic methods introduce an alternative conceptualization of the sample data that is beyond the scope of the current study.

One important study feature is both strength and limitation. The three sample pools represent diverse characteristics along the continua of ABP populations, treatment phases, and intervention contexts: a prevention trial involving at-risk young adolescents, a training sample of adolescents in community-based care for behavioral health problems, and a set of clinical trials involving older adolescents meeting diagnostic criteria for conduct or substance use disorder. The benefits of selecting a sample that supports generalizability across broad spectrums of clinical problems and treatment contexts are counterbalanced by sacrifice in the acuity with which findings pertain to teenagers with specific clinical disorders at specific junctures in treatment. Also, because of this sample diversity, it was not possible to use identical methods across samples when selecting high fidelity sessions. Finally, although observing high-fidelity FT sessions was necessary in order to define core techniques that validly represent the manualized models of interest, it remains unknown whether the distilled elements will be identifiable or viable in routine clinical conditions. To address this gap the authors are attempting to verify the structure and content of the four FT elements derived in this study within a new pool of ABP cases treated in usual care by community clinicians with varying degrees of FT allegiance and training.

Clinical Implications

Evidence supporting the effectiveness (Chorpita et al., 2017; Weisz et al., 2012), sustainability (Weisz et al., 2018), and perceived value (Chorpita et al., 2015; Southam-Gerow et al., 2014) of core EBIs in routine practice continues to mount. Still, the potency of the FT approach in naturalistic form remains virtually untested (Riedinger et al., 2017). Is it reasonable to believe that non-manualized interventions governed by core FT elements are a viable alternative to manualized models for treating ABPs in usual care? If the success of FT techniques for ABPs depends fundamentally upon the implementation boost provided by quality assurance procedures such as those mandated by manualized FT models, then core element FT delivered without substantial support by model experts may be ineffective. Of note, one controlled trial evaluated non-manualized (i.e., naturalistic) FT delivered by community therapists as the routine standard of care, finding that it outperformed naturalistic non-FT services for improving ABPs (Hogue, Dauber, Henderson, et al., 2015). Moreover, the naturalistic FT condition exceeded a research-defined benchmark for adherence to FT techniques and equaled a benchmark for long-term outcomes (Hogue, Dauber, & Henderson, 2017). If replicated and expanded, such results might influence providers to weigh the feasibility of cultivating core element FT services—perhaps factoring in upgrades to local FT supervision and quality procedures (Hogue et al., 2013)—against the formidable barriers to importing a manualized FT model.

Of course this begs the question of what core element FT services might look like. That is, how can distilled FT techniques be implemented effectively in routine care? There are at least three implementation approaches with potential merit. First, core FT elements can be translated into a standardized quality assurance system—a de facto “manual” and companion fidelity procedures—that facilitates decision-making about when and with whom to utilize which combination of elements. This has been accomplished with core EBI elements for other youth disorders (e.g., Chorpita & Weisz, 2009). Quality assurance systems for core EBIs have proven to equal or surpass conventional manualized treatment as well as usual care for youth behavior problems (e.g., Chorpita et al., 2017; Weisz et al., 2012). Second, core FT elements can be documented in a flexible clinical protocol to be used independently by community clinicians without standardized training and fidelity procedures. Therapists would be invited to invoke their own conceptualizations about the appropriate timing, sequence, and relative emphasis of each FT element for each case, adjusting FT delivery based on client responsiveness (see Lyon & Koerner, 2016). Third, core FT techniques can be converted to a utilitarian adherence scale that provides empirically-derived FT implementation guideposts for line clinicians treating ABPs (see Stirman et al., 2018 for an example involving cognitive-behavioral therapy). Although therapists are notoriously unreliable in reporting on their own adherence to virtually every variety of EBI (e.g., Hurlburt, Garland, Nguyen, & Brookman-Frazee, 2010), FT for ABPs is one arena in which therapist self-report of treatment adherence has shown reasonable concordance with observer ratings (Chapman, McCart, Letourneau, & Sheidow, 2013; Hogue, Dauber, Lichvar et al., 2015). Effective self-monitoring of adherence to FT elements via pragmatic therapist-report tools, in the context of ongoing case planning and supervision, might confer enormous benefits to increasing the amount, and perhaps quality, of non-manualized FT delivered in ABP services. To advance such efforts we are currently developing online training procedures to increase therapist delivery of core FT techniques for ABPs via video-based training in accurate FT self-monitoring combined with tailored feedback on self-reported FT use (National Institute on Drug Abuse: R34DA044740). As core elements distillation continues to progress for numerous EBIs, the field will discover the full potential and limits of its utility in various treatment contexts and/or with specific presenting problems.

Public Health Significance:

Increasing implementation of high-fidelity family-based interventions would improve the quality of treatment services for adolescent conduct and substance use problems.