Fidelity Failures in Brief Strategic Family Therapy for Adolescent Drug Abuse: A Clinical Analysis

Abstract

As evidence-based family treatments for adolescent substance use and conduct problems gain traction, cutting edge research moves beyond randomized efficacy trials to address questions such as how these treatments work and how best to disseminate them to community settings. A key factor in effective dissemination is treatment fidelity, which refers to implementing an intervention in a manner consistent with an established manual. While most fidelity research is quantitative, the present study offers a qualitative clinical analysis of fidelity failures in a large, multisite effectiveness trial of Brief Strategic Family Therapy (BSFT) for adolescent drug abuse, where BSFT developers trained community therapists to administer this intervention in their own agencies. Using case notes and video recordings of therapy sessions, an independent expert panel first rated 103 cases on quantitative fidelity scales grounded in the BSFT manual and the broader structural-strategic framework that informs BSFT intervention. Because fidelity was generally low, the panel reviewed all cases qualitatively to identify emergent types or categories of fidelity failure. Ten categories of failures emerged, characterized by therapist omissions (e.g., failure to engage key family members, failure to think in threes) and commissions (e.g., off-model, non-systemic formulations/interventions). Of these, “failure to think in threes” appeared basic and particularly problematic, reflecting the central place of this idea in structural theory and therapy. Although subject to possible bias, our observations highlight likely stumbling blocks in exporting a complex, family treatment like BSFT to community settings. These findings also underscore the importance of treatment fidelity in family therapy research.

As evidence-based family treatments for adolescent substance use and conduct problems gain traction, cutting edge research moves beyond randomized efficacy trials to address questions such as how these treatments work and how best to disseminate them into community settings. Dozens of studies now document positive effects of family therapy, not only in reducing adolescent drug use and delinquency, but also for secondary outcomes such as family functioning and engagement/retention in treatment (Baldwin, Christian, Berkeljon, Shadish, & Bean, 2012; Becker & Curry, 2008; Tanner-Smith, Jo Wilson, & Lipsey, 2013; Waldron & Turner, 2008). Largely for this reason, family interventions are now widely applied in mental health, drug abuse, and juvenile justice settings, where preliminary evidence suggests they work at least as well as other evidence-based approaches focused on the youth alone (Dennis et al., 2004). Nevertheless, there remains a pressing need to evaluate the effectiveness of family therapy in real world settings. The present study examines one evidence-based family treatment, Brief Strategic Family Therapy (BSFT, Szapocznik, Hervis, & Schwartz, 2003), as it was implemented by frontline practitioners in a large effectiveness trial involving 8 community treatment programs in the National Institute on Drug Abuse Clinical Trials Network.

Grounded in structural-strategic family systems theory (e.g. Haley, 1967, Minuchin, 1974) and operationalized for research by Jose Szapocznik and his colleagues at the University of Miami, BSFT embodies a relatively “pure” model of systemic intervention and change compared to more integrative evidence-based approaches such as Multi-Systemic Therapy (MST, Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 1998), Functional Family Therapy (FFT, Sexton, Alexander, & Gilman, 2004) and Multidimensional Family Therapy (MDFT, Liddle, 2009). In the structural/BSFT framework, repetitive sequences of family interaction define problematic structural patterns (e.g., disengaged or over-involved relationships, collapse or reversed parent-child roles, triangulation, cross-generation coalitions) that both maintain, and are maintained by, problems such as adolescent substance abuse. Planned (strategic) interventions aim to shift problem-maintaining family interactions to a structure more conducive to reducing drug use (e.g., clearer generation boundaries, more positive parent-child involvement). Core BSFT components are joining, tracking, reframing, and restructuring (Szapocznik et al., 2003), and a central characteristic of this approach is that the therapist instigates structural change directly, through enactments in the therapy session, before attempting to reinforce this with relevant homework tasks. Of note, BSFT was among the first family therapy models tested in systematic efficacy research (Santisteban, Coatsworth, Perez-Vidal, Kurtines, Schwartz, LaPerriere, & Szapocznik, 2003; Szapocznik, Santisteban, Rio, Perez-Vidal, Santisteban, & Kurtines, 1989), though prior to the NIDA trial we examine here, there had been fewer effectiveness trials involving dissemination of BSFT into community settings than was the case for the other evidence-based ecological approaches.

A key factor in effective dissemination is treatment fidelity (also known as treatment integrity), which refers to implementing an intervention in a manner consistent with an established manual. In contrast to pharmacotherapy, the integrity of a psychosocial intervention like family therapy depends entirely on highly variable clinician behavior: What a therapist does and does not do defines a multi-component independent variable – the treatment itself – and careful monitoring of fidelity is necessary to establish that community therapists are actually providing the components that are presumably essential to a treatment’s effectiveness. In the NIDA trial, a detailed manual by the BSFT developers (Szapocznik et al., 2003) served to anchor the training of community therapists, ongoing monitoring of adherence (quality control), and later ratings of treatment fidelity.

Approaches to measuring fidelity vary widely but optimally address (a) the quality or competence of relevant interventions, in addition to their frequency or quantity as captured by adherence check-lists; (b) proscribed as well as prescribed therapist behavior (what the therapist should and should not do); and (c) components that are unique and essential to the treatment, as well as those that are essential but not unique. Additional considerations are who should rate treatment fidelity (e.g., on-line supervisors or independent judges); when they should do this (e.g., soon after a session, to optimize therapist feedback, or later, to allow counter-balanced ratings); and what raw data (e.g., case notes, client reports, or session recordings) are sufficient for valid fidelity assessment (Perepletchickova, Treat & Kazdin, 2007). In the NIDA trial, teams of raters and supervisors from the University of Miami maintained quality control by rating community therapists’ adherence from session videos on an ongoing basis (Robbins et al., 2011a). Later, after all therapy was complete, our independent team of investigators (then at the University of Arizona) used session recordings and progress case notes to evaluate session- and case-level fidelity. Although terminologies vary in the literature, with some investigators preferring to view therapist “adherence” and “competence” as sub-components of the broader construct “fidelity,” we will follow recent NIMH (2011) usage in distinguishing simple behavioral “adherence” from “fidelity,” where the latter goes beyond mere adherence to account for intervention quality and therapist competence.

In addition to ensuring integrity, there are good scientific reasons for incorporating fidelity assessment in research on psychosocial interventions. For example, fidelity ratings can provide a basis for identifying the active ingredients of an intervention by correlating fidelity components with measures of outcome. Knowing which ingredients contribute most and least to outcome might then help to make interventions more efficient by paring them down to essential elements (Kazdin, 2006; NIMH, 2011). Still, studies attempting to link therapist adherence/competence to outcomes of individual psychotherapy have had mixed results, with one meta-analysis showing no significant overall effect (Webb, DeRubeis & Barber, 2010). On the other hand, results from several family therapy studies have been positive (Hogue, Dauber, Barajas, Fried & Liddle, 2008; Huey, Henggeler, Brondino & Pickrel, 2000), and in the NIDA trial we studied here, both adherence and overall fidelity ratings showed significant associations with outcome (Robbins et al., 2011a; Shoham & Rohrbaugh, 2010).

Although most treatment fidelity research is quantitative, there are good reasons to examine fidelity in qualitative ways as well. Even detailed a priori scales are limited in their ability to capture patterns of therapist-client interaction that especially embody, enhance, or undermine principles of therapeutic change. A qualitative, bottom-up, clinical analysis of what therapists do and don’t do – particularly in regard to failures of fidelity – may illuminate which aspects of a treatment are most difficult for therapists to learn and implement. This has obvious relevance to dissemination, as such observations can highlight likely stumbling blocks in exporting a complex family treatment like BSFT to community settings.

The present study offers a qualitative clinical analysis of fidelity failures in the NIDA effectiveness trial of BSFT for adolescent drug abuse, where BSFT developers trained randomly assigned community therapists to administer this intervention in their own agencies. The primary quantitative outcomes were disappointing in that adolescents receiving BSFT did not differ from treatment-as-usual (TAU) controls in reported or bio-chemically confirmed substance use over a 1-year follow-up period (Robbins et al., 2011b). On the other hand, quantitative case-level ratings of treatment fidelity within the BSFT condition did predict some substance use outcomes (Shoham & Rohrbaugh, 2010), which is noteworthy because levels of BSFT fidelity in this trial were generally low. For this reason, the panel decided to re-review all of the cases qualitatively in hopes of better understanding how BSFT went awry. Our intention was not to undertake a rigorous qualitative analysis in the tradition of, say, grounded theory (Strauss & Corbin, 1998), but rather to identify patterns or types of therapist behavior that were inconsistent with the BSFT manual and therefore may have undermined effective intervention. We did this clinical analysis based on familiarity with BSFT and the considerable experience two of us (MR and VS) had had with structural-strategic-systemic therapy more generally. The results represent an attempt to understand and organize patterns of fidelity failure in a manner relevant to clinical practice.

The Parent and Pilot Studies

The parent study (NIDA Clinical Trials Network Protocol #014) compared BSFT to treatment as usual (TAU) for 480 substance-using adolescents in 8 community treatment programs, where clinicians randomly assigned to provide BSFT completed a training (certification) program before seeing cases in the clinical trial. The rationale for randomizing therapists as well as cases included testing the feasibility of disseminating BSFT to “real world” agency settings. Thus, agency clinicians who met minimal inclusion criteria and volunteered to participate in the study knew they might or might not receive several months of BSFT training from University of Miami treatment developers followed by ongoing supervision of study cases if they later received certification to participate in the clinical trial. Of 70 agency therapists randomized to BSFT or TAU, 29 began the BSFT training and 20 were ultimately certified to participate in the trial. The certification process required satisfactory performance with at least one pilot (training) case and proficiency in core BSFT skills based on ratings of session video segments by a panel of trainers. Session video recordings from the trial itself provided a basis not only for ongoing adherence monitoring and long-distance supervision based in Miami, but also for our later ratings of BSFT fidelity at the University of Arizona.

The fidelity analyses in this report come from the independent Arizona platform study, originally intended to examine mediators and moderators of BSFT treatment effects. Unfortunately, because BSFT and TAU cases showed few reliable differences on substance use outcome measures during the year following randomization, there were few treatment effects to mediate. The platform design did, however, include measuring BSFT fidelity within the experimental condition so as to provide a continuous independent variable for within-model tests of study hypotheses.

Given the original research aims, we selected cases for fidelity review that met all of the following criteria: (a) at least 4 BSFT sessions, with a parent and child participating, prior to the first family follow-up assessment 4 months following randomization; (b) at least two usable recordings from those sessions, with adequate audio and video; (c) adequate chart documentation for at least 70% of those sessions; (d) usable videos of family interaction assessment tasks from baseline and the 4-month follow-up; and (e) available outcome data at least 8 months post-randomization. Of the 245 cases originally assigned to BSFT, only 103 (42.4%) met this rather stringent standard for inclusion in our time-intensive fidelity assessment. Comparisons of selected vs. not selected cases nevertheless revealed no statistically significant differences between these two groups on key demographic and clinical variables (e.g., age, sex, ethnicity, total problems, prior arrests, substance use severity).

Therapist and client participants

The 103 cases in the fidelity sample were distributed across 19 BSFT therapists, including at least two therapists from each of the 8 community treatment sites, and the number of fidelity cases per therapist ranged from 1 to 17 (median = 4). Demographically, the 19 therapists were 58% female, 21% Hispanic, 16% African-American, 58% white, and their mean age was 43.1 (range = 29 to 58 years). Most (95%) had at least a master’s degree in a relevant professional discipline, with one holding a Ph.D. and another a medical degree from Mexico. Their median years of post-graduate clinical experience was 7 (range = 1 to 20), and almost half reported some training in family therapy.

Client families came primarily from the juvenile justice system and other (e.g., residential) drug treatment programs. To qualify, index adolescent participants between the ages of 12 and 17 had to self-report use of illicit drugs other than alcohol and tobacco in the 30-day period prior to the baseline assessment or have been referred from an institution (e.g., detention, residential treatment) for the treatment of drug abuse. Adolescents in the fidelity sample were 81% male, 49% Hispanic, 20% African American, and 31% white. Their mean age was 16.1 years (SD = 1.2), and 33% lived with two biological parents. Well over half (58%) had been arrested, many (44%) more than once, and 60% dropped positive urines at baseline despite having already enrolled in the study.

Families in the fidelity sample attended between 5 and 24 BSFT sessions (M = 11.4, SD = 4.5), with 93% receiving a “full dose” (according to treatment developers) of at least 8 sessions. On average, 3.2 family members participated in these sessions, 54% of the sessions took place in the family’s home, and treatment lasted 13.8 weeks (range = 3 to 52 weeks).

Assessment of treatment fidelity

A panel consisting of the authors, led by MR and VS, reviewed video recordings of at least two BSFT sessions (usually session 1 or 2 and session 4 or 5) plus all case notes from sessions before the 4-month follow-up assessment. The idea was to include both an early session where joining and assessment of family dynamics should be in evidence and a later one where restructuring and active intervention would likely occur. If case notes suggested an additional critical session had occurred within this time frame, we reviewed the video of that session as well. Prior to the panel meeting as a group, FLC and BPH made summary transcriptions of the recorded sessions and rated session-level fidelity components on quantitative scales grounded in the BSFT manual (Szapocznik et al., 2003). The full panel then reviewed all available materials, including videos of pre-treatment family interaction tasks (e.g., plan a menu, discuss a recent argument) the therapist had not seen, and made case-level consensus ratings (on 1–5 scales) of formulation quality, intervention quality, off-model behavior, and overall fidelity to the BSFT model.

As we noted in the project’s Final Report (Shoham & Rohrbaugh, 2010), quantitative case-level fidelity correlated positively and significantly with most substance-use outcomes at most follow-up intervals. Similarly, comparing TAU to adequate BSFT (≥ 3 on the 1–5 scale), with inadequate cases excluded, yielded significant treatment effects for some outcomes at the 4-month follow-up, though not thereafter.

Because fidelity was generally poor, with only 28% of the 103 cases receiving ratings at or above 3 (minimally adequate) on the 1–5 overall fidelity scale, we undertook a qualitative clinical analysis of cases in the fidelity sample to develop a richer understanding of what went wrong. In approaching this, the panel made extensive use of earlier notes from session transcripts indicating instances when a therapist erred by commission (with interventions that were off-model or implemented clumsily, at the wrong time, or in ways not well received by the family) as well as by omission (missing important opportunities to do something). At a conceptual level, we also noted discrepancies between the therapist’s apparent formulation of core dynamics and corresponding structural objectives (inferred from his or her case notes and observable behavior) and the panel’s own formulation of the case based on all of the information available to us.

In a series of meetings over several months, we focused first on low-fidelity cases to carry out an iterative, bottom-up process of identifying patterns of fidelity failure across cases and distilling these into meaningful, often overlapping, but reasonably distinct categories. Finally, using these categories (which we describe below and list in Table 1), the panel re-reviewed all 103 cases in the fidelity sample and coded the presence vs. absence of each type of fidelity failure. A ground rule was that, to be included in the final list, a failure category had to appear in at least 4 cases.

Table 1.

Categories of fidelity failure

FIDELITY FAILURES	% of cases
1. Failure to engage key family members not assessing whom to engage not engaging known players	72% 16% 67%
2. Failure to think in threes (“not getting it”) over focus on dyadic communication or “emotional connection” at expense of triadic structural patterns neglecting triangulation of IP into adult relationships	67% 47% 58%
3. Failure of restructuring, given adequate structural objectives (“not doing it”)	57%
4. Failures of joining unbalanced coalitions (differential alliance) language mismatch between therapist and family	46% 41% 11%
5. Therapist centrality prevents eliciting core dynamics	36%
6. Off-model formulations or interventions reflecting non-systemic, individualistic (e.g., biomedical, psychodynamic, experiential, cognitive-behavioral) assumptions	36%
7. Off-model didactic, instructive intervention	36%
8. Over focus on IP or demand for IP change (especially when process parallels parent demands on IP)	34%
9. Failure of leadership (therapist passivity)	17%
10. Precipitous push for parent change (restructuring before joining)	4%

Qualitative Patterns of Fidelity Failure

Table 1 shows 10 categories of fidelity failure and the percentage of cases in which each was present. The number of categories and percentage figures are somewhat arbitrary, as they represent our attempt to make distinctions among interwoven clinical phenomena. We originally identified 13 categories, but ultimately reduced them to 10 based on conceptual overlap, such that, in the final list, failure to engage, failure to think in threes, and failures of joining each have two sub-categories. Not surprisingly, there was substantial co-occurrence among the various failure categories. For example, failures to engage key family members tended to be accompanied by failure to think in threes, which in turn co-occurred with increased therapist centrality, lack of therapist leadership, didactic intervention, and non-systemic formulations. Similarly, failures of joining were associated with an over-focus on the adolescent/identified patient, therapist centrality, and off-model didactic or non-systemic forms of intervention. Of all the categories in Table 1, failure to restructure appeared to show least overlap with other categories.

Although quantitative aspects of these clinical observations are secondary, we would note that the median number of failures types we recorded per case was 5 (range= 0–10), with only 6 cases involving no failures at all. The distribution of failures varied substantially across therapists, with some showing anomalies in almost every category and others having very few. In addition, some therapists were prone to particular types of failure (e.g., failure to engage key family members; off-model didactic interventions) and others were not.

The most common fidelity failure, which occurred in over two-thirds of the cases, involved not engaging key family members in treatment. This happened when the therapist either (a) did not assess or identify family members who appeared to play a key role in problem maintenance, or b) did not successfully engage identified key family members in therapeutic sessions. The latter was especially common and suggests that, while therapists understood the importance of inquiring about interaction patterns involving people not present, they either did not see the advantage of including potentially important players or were unsuccessful in getting them to attend therapy sessions. Key absentees often did not reside in the IP’s household (e.g., a grandparent, biological father, or older sibling) or had a conflicted relationship with the primary custodial parent, yet from our perspective they played a crucial role in interaction patterns relevant to the youth’s difficulties. In some cases, BSFT therapists appeared to follow an implicit household model of whom to engage, even when a parent figure outside the household was clearly implicated in the dynamics of problem maintenance. For example, when social services removed an adolescent from his mother’s custody to live with an aunt, the therapist did not involve the mother in treatment, even though the youth had several times run away to visit her. Other therapists tended to prioritize biology or legal status over function, sometimes neglecting powerful parent figures in the process. Here a familiar pattern was not engaging highly involved step-parents who helped to make decisions or enforced family rules.

These failures to engage key family members failure are surprising given the strong emphasis BSFT developers have historically placed on engagement (Coatsworth, Santisteban, McBride, & Szapocznik, 2001; Santisteban, Szapocznik, Perez-Vidal, Kurtines, Murray, & LaPerriere, 1996; Szapocznik, Perez-Vidal, Brickman, Foote, Santisteban, Hervis, & Kurtines, 1988). In fact, a quasi-experimental trial comparing BSFT with and without a separate three-part Engagement Module (join reluctant members, identify and restructure resistance-maintaining interaction patterns) demonstrated clear benefits of intervening to ensure that key family members attend at least the first therapy session (Santisteban, et al. 1996). The BSFT manual guiding the CTN trial emphasized this as well, making clear that therapists who do not succeed in engaging key family members risk (a) surrendering their therapeutic leadership by agreeing to see only a portion of the family, (b) having non-involved family members see the therapist as in coalition with members who do participate, and (c) missing an opportunity to observe how the family system operates as a whole and formulate an appropriate case conceptualization (Szapocznik et al. 2003, pp. 44–45).

The second most frequent fidelity failure, inferred both from notes and direct observation, involved therapists not “thinking in threes” – that is, not recognizing how the behavior of one person can maintain, and be maintained by, the behavior of at least two others. In families, the most powerful and problematic triangles cross generation lines (e.g., a parent and child joining together against or to the exclusion of another parent, or parents detouring their conflict with each other by focusing on a child (Haley, 1967, 1976, 1980; Minuchin, 1974; Minuchin & Fishman, 1981; Minuchin, Montalvo, Guerney, Rosman, & Schumer, 1967; Minuchin, Rosman & Baker, 1978). The central role of cross-generation triangles in structural-strategic theory and therapy is evident in the BSFT manual. Thus, in addition to underscoring the role of repetitive triadic interactions between parents and children in families coping with adolescent drug abuse, the manual calls for careful tracking of such patterns by orchestrating enactments in the therapy room, so as to illuminate what happens at home (Szapocznik et al., 2003, p. 16, 28).

Therapists failed to think in threes by either (a) over-focusing on dyadic communication at the expense of structural dynamics involving more than one other family member (usually two adults), or (b) neglecting triangulation of the IP into adult relationships. Both forms were manifest in what the therapist chose to emphasize (and ignore) in the session. Thus, when faced with hostile or negative family interactions, a common therapist response was to attempt de-escalation by commenting on and promoting “emotional connection” between the two parties most in conflict, usually a parent and child. Too often missing or delayed in these scenarios were attempts to flush out triadic patterns of problem maintenance through enactment – for example, by having parent figures talk to each other about their disagreements, in addition to talking with the child. Moreover, by attempting dyadic de-escalation and minimizing intensity, therapists missed opportunities to bring conflictual adult interaction into the therapy room as a prelude to blocking triangulation of the IP and encouraging more adaptive forms of conflict resolution. A final observation about thinking in threes is that therapists appeared to have most trouble doing this with non-traditional family constellations, such as those where aunts, uncles, or grandparents served as parental figures. In other words, when therapists attended primarily to biological relationships, they sometimes missed important dynamics involving “outside” insiders.

A third common fidelity failure (present in over half of cases) concerned therapists not following through with appropriate restructuring even when they appeared to have a clear understanding of the problem-maintaining interaction patterns. Behaviorally, this involved failing to successfully orchestrate therapeutic enactments, the cornerstone of effective family restructuring. One manifestation was struggling to provide a convincing reframe that challenged family members’ individualistic, static conceptualizations of a problem (as an attribute of one person) and offered a more relational, dynamic view, locating the problem between rather than within people and providing a reason for family members to talk with one another, in order to alter their existing patterns of interaction. Another marker of failed enactment was not requesting physical movement (e.g., “Mom, please turn your chair and move a little closer to Dad, so you can talk with him more directly about how to handle this problem with Billy.”) Not surprisingly, failures of enactment were sometimes aggravated by not having key family members in the session, by failure of joining, or by general therapist passivity. In any case, the manual is clear about restructuring as a sine qua non of this approach: “It is the BSFT counselor’s job to shift the alliances that exist in the family. This means restoring the balance of power to the parents or parental figures so that they can effectively exercise leadership in the family and control their [child’s] behavior” (Szapocznik et al., 2003, p. 37) – and in-session enactment is the preferred means to this therapeutic end.

Failures of joining occurred in almost half of the cases and tended to involve either (a) the therapist engaging in unbalanced alliances with family members, or (b) a language mismatch between therapist and certain family members that interfered with adequate joining. One pattern of unbalance occurred when therapists aligned with parental figures, adopting their conceptualization of the adolescent’s behavioral problems (usually as static and an internalized deficit) at the expense of a positive alliance with the adolescent. This sometimes led therapists to rely on off-model didactic interventions, which made effective restructuring more difficult. Another, less frequent but still problematic form of unbalanced joining entailed aligning with one parent and the adolescent more than with another parent figure (e.g., a step-father), in effect reinforcing a cross-generation coalition. Finally, in a subset of cases, failures of joining followed from language mismatch of therapist to family – specifically when Hispanic families whose members had varying levels of English fluency worked with a monolingual English-speaking therapist. When one or both parental figures spoke only Spanish, and the therapist relied on other family members or the adolescent for translation, the situation was ripe for unbalanced alliances and a host of other limitations on the feasibility of restructuring through therapeutic enactments.

In highlighting the necessity of balanced joining, the BSFT developers repeatedly emphasize the importance of establishing strong relationships with all family members, regardless of whose conceptualization of the problem the therapist most agrees with: “In fact, frequently, the person with whom it is most critical to establish an alliance or bond is the most powerful and unlikeable family member” (Szapocznik et al., 2003, p. 26). In addition to facilitating restructuring operations, balanced alliances help the therapist establish and maintain more relational, systemic case conceptualizations.

In a little more than a third of cases, failing to stay de-centralized undermined therapists’ ability to elicit problem-maintaining family dynamics through enactment. Early in therapy this most often took the form of over-using an interview style format to gather information, which limited opportunities to facilitate diagnostic enactments between family members in order to “bring the problem into the room.” In other words, centralization resulted in over-emphasis on content at the expense of focusing on actual interactional processes, the proper target of structural BSFT intervention. Later in treatment, centralized therapists were also more likely to rely on didactic and off-model intervention. For instance, a therapist might spend the majority of a session telling family members why they should interact differently with one another, instead of facilitating in-session enactments to show them how to do this. As the BSFT developers note, “getting family members to interact can be difficult because families often come into counseling thinking that their job is to tell the counselor what happened. Therefore, it is essential that counselors decentralize themselves by discouraging communications that are directed at them, and instead encouraging family members to interact so that they can be observed behaving in their usual way” (Szapocznik et al., 2003, pg. 30).

Other categories included failure of therapeutic leadership, in the form of general passivity and/or insufficient elicitation of clinically relevant interaction sequences in the session, and failure to strike a balance between over-focusing and demanding change from either the adolescent or the parent(s). Thus, precipitously pushing for adolescent change made it more difficult to move family members toward accepting a more relational view of the problem and seeing themselves as playing an important role in the solution. An opposite, less frequent failure involved therapists precipitously pushing for parent change. Here therapists had clear and appropriate structural objectives but pushed for parent change before adequately joining with all family members and providing effective reframes to shift the parents’ view of the problem. Such attempts to restructure were at odds with how family members (particularly parents) viewed the problem and hence engendered resistance.

Finally, in about a third of the cases, therapists engaged in off-model behavior by offering didactic, instructive interventions or intervening in ways that reflected non-systemic (e.g., biomedical, psychodynamic, experiential, cognitive-behavioral) assumptions about the nature of clients’ problems or how people change. These failures were most common in cases where adolescent presenting problems were relatively severe, which appeared to make thinking systemically about problem maintenance and change more difficult (Lebensohn-Chialvo, Hasler, Rohrbaugh, & Shoham, 2016).

Comment

The categories of fidelity failure we describe here represent an imperfect attempt to chart how one systemic treatment – BSFT for adolescent drug abuse – can go awry. We did not employ rigorous qualitative methods such as grounded theory (Strauss & Corbin, 1998) in generating the categories, but rather attempted to classify aspects of good and bad structural-systemic therapy by interpreting the BSFT manual in the context of our own experience and expertise. Perhaps as a consequence, the emergent categories of failure are clearly interrelated and have a fair degree of overlap. The category scheme also reflects our own preconceptions about higher order distinctions, such as between conceptual vs. behavioral fidelity and errors of commission vs. omission. Another influential preconception reflects the priority structural family systems theory attaches to triadic patterns of problem maintenance relative to dyadic and/or individual ones (Haley, 1967, 1976, 1980; Minuchin, 1974). Thus, “thinking in threes” – or the lack thereof – was a facet of BSFT practice we were primed to see.

One might reasonably ask, to what extent are the patterns of fidelity failure we describe here relevant to other family therapy approaches, or to therapy more generally? Although fidelity facets like engagement, joining, or therapeutic alliance have broad applicability across family therapies (Friedlander, Escudero, Heatherington & Diamond, 2010), we suspect that many of the categories in Table 1 are fairly specific to structural-strategic therapy and BSFT. This is because other evidence-based family therapies such as Multisystemic Therapy (MST), Functional Family Therapy (FFT), and Multidimensional Family Therapy more often integrate or incorporate ideas and techniques from other approaches such as psycho-education, attachment theory, or cognitive-behavior therapy, and tend to be less “pure” in adhering to a systemic conceptual framework (Rohrbaugh, 2014). For example, failing to think in threes would probably be less consequential in MST, FFT or MDFT compared to BSFT.

The thinking in threes example calls attention to a distinction we think has special relevance to therapist and development and research: Categories such as not assessing whom to engage, neglecting triangulation of IP into adult relationships, and over-focus on dyadic ‘emotional connection’ suggest difficulties in case formulation – in particular, not conceptualizing the case from a systemic perspective. In contrast, categories such as not engaging known players, failures of joining, failure of restructuring, therapist centrality, and over-focus on IP are more in the behavioral realm, highlighting specific actions (or inactions) that deviate from ideal BSFT practice. This “viewing vs. doing” distinction raises questions about the relationship between conceptual and behavioral skills (e.g., whether thinking systemically is essential to doing good family therapy), and whether proficient, high fidelity practice in one or both of these domains predicts case outcomes.

We were able to address these questions indirectly in the CTN trial, where participating therapists completed a standardized Case Formulation Exercise before, during, an after several months of supervised training, all of which occurred before the trial began, as well as in a separate evaluation of “Case Simulation Methods for Teaching Empirically Validated Treatments” conducted over 5 years in a university training clinic (Sbarra, Rohrbaugh & Shoham, 2014). Both studies showed clear associations between acquisition of systemic case-formulation skills during training and behavioral measures of fidelity and/or adherence. Case formulation scores during training also predicted a therapist’s case outcomes during the CTN trial and his or her behavioral fidelity scores. Interestingly, the strongest predictor of BSFT fidelity and case outcome in these analyses was proscribed case formulation (especially non-systemic thinking about the problem), which suggests that knowing what not to do may be especially important in this form of family therapy (Shoham & Rohrbaugh, 2010, Lebensohn-Chialvo & Rohrbaugh, 2016).

Because the implementation literature sometimes confuses fidelity and adherence, it is worth noting that both came into play in the CTN trial. Apart from our qualitative findings, the fact that quantitative ratings of BSFT fidelity predicted certain clinical outcomes is consistent with an earlier report from the CTN trial showing associations between outcome/retention and quantitative BSFT adherence (Robbins, et al. 2011b). Although the ongoing ratings of therapist adherence by University of Miami research assistants were useful for monitoring quality control during the clinical trial, the fidelity ratings by independent evaluators are more useful for research and treatment development. For example, the adherence ratings focused only on the presence or extent of prescribed therapist behavior (e.g., joining, tracking, reframing, restructuring) during a given session with minimal provision for capturing the quality of intervention or the coherence of what a therapist did from session to session. Fidelity ratings, on the other hand, took into account conceptual as well as behavioral aspects of treatment quality (inferring the former from case notes in tandem with direct observation), considered proscribed (off model) as well as prescribed therapist behavior, and evaluated fidelity at the case (not just session) level based on consensual expert deliberation. The latter fidelity data, which provide multi-faceted and probably more reliable representations of intervention quality compared to adherence ratings, are therefore better suited to illuminate links between the process and outcome of therapy.

At a broader level, both quantitative and clinical-qualitative analyses make clear that establishing and maintaining high BSFT fidelity was very difficult in this large, multi-site clinical trial. Indeed, barely a third of the cases had minimally adequate fidelity on the overall case-level metric we used – and this does not include families who participated in fewer than 5 sessions. What can explain this? How is it that this particular set of failures emerged in a formal trial when therapists were apparently well trained and supervised?

Because the panel’s observations admittedly reflect our own preconceptions about what constitutes good structural-strategic family therapy, one possibility is that the failures were partly or even mainly in the eyes of the beholders. We think the many (in)consistencies with the BSFT manual documented above make this unlikely. Other possible explanations range from deficits in the implementation skill sets of the BSFT therapists to aspects of the immediate social/organizational context in which implementation occurred. Therapist training typically consisted of several workshops and several supervised pilot cases over a period of several months, with BSFT certification for the CTN trial based on competent video demonstration of behavioral skills such as joining, tracking, reframing, and restructuring. From what we observed, the training process attached less importance to case formulation and conceptual skills that may have been essential to treatment integrity.

On the organizational side, we were impressed that the unit of implementation for a complex psychosocial intervention like BSFT includes vital supervisory and administrative relationships that, to be effective, must co-exist with parallel supervisory structures in the host agency. Because University of Miami staff supervised BSFT therapists from a distance via phone and video review, the situation was ripe for triangles that cast the supervisor as an outsider, much like a disempowered stepparent, and the struggling therapist as the new problematic, identified patient in this organizational system. From a structural-systems perspective, following principles of isomorphism, or parallel process, we would hypothesize that the ongoing fidelity performance of a trained protocol therapist depends on strong executive coalitions among the supervisors to whom the therapist is accountable: Fidelity flourishes when these coalitions directly and actively support competent implementation of the protocol and deteriorates when they do not (Rohrbaugh, 2014). In the aftermath of the CTN trial, the BSFT developers have recognized the importance of such organizational dynamics and expanded their approach to include intervention at that level (Szapocznik, Muir, Duff, Schwartz & Brown, 2015).

In retrospect, we suspect parallel processes at multiple levels contributed to undermining BSFT fidelity in the CTN trial. In addition to organizational/supervisory triangles paralleling common cross-generational triangles in client families, at least one of the fidelity failure patterns noted above involved therapists essentially replicating problem-maintaining patterns within the family (e.g., demanding change from an adolescent IP whose parent(s) did likewise). Perhaps most important, we as researchers searched for fidelity failures exclusively under the lamppost of therapist-family interaction, without systematically examining the larger context of relationships in which those failures occurred.