Patient Characteristics and Variability in Adherence and Competence in Cognitive-Behavioral Therapy for Panic Disorder

Abstract

Although associations with outcome have been inconsistent, therapist adherence and competence continues to garner attention, particularly within the context of increasing interest in the dissemination, implementation, and sustainability of evidence-based treatments. To date, research on therapist adherence and competence has focused on average levels across therapists. With a few exceptions, research has failed to address multiple sources of variability in adherence and competence, identify important factors that might account for variability, or take these sources of variability into account when examining associations with symptom change.

Objective

(a) statistically demonstrate between- and within-therapist variability in adherence and competence ratings and examine patient characteristics as predictors of this variability and (b) examine the relationship between adherence/competence and symptom change.

Method

Randomly selected audiotaped sessions from a randomized controlled trial of cognitive-behavioral therapy for panic disorder were rated for therapist adherence and competence. Patients completed a self-report measure of panic symptom severity prior to each session and the Inventory of Interpersonal Problems-Personality Disorder Scale prior to the start of treatment.

Results

Significant between- and within-therapist variability in adherence and competence were observed. Adherence and competence deteriorated significantly over the course of treatment. Higher patient interpersonal aggression was associated with decrements in both adherence and competence. Neither adherence nor competence predicted subsequent panic severity.

Conclusions

Variability and “drift” in adherence and competence can be observed in controlled trials. Training and implementation efforts should involve continued consultation over multiple cases in order to account for relevant patient factors and promote sustainability across sessions and patients.

Interest in the constructs of adherence and competence initially emerged from advances in psychotherapy research methods. Researchers interested in making direct and causal claims regarding treatment effects in experimental research recognized that the independent variable (i.e., a particular treatment package or intervention) needed to be clearly specified and operationalized. Once specified in a treatment manual, for example, the degree to which a therapist delivers an intervention can be objectively assessed, and in the case of comparative efficacy and effectiveness trials, different treatments can be differentiated along important dimensions. However, it has also been recognized that treatment integrity involves more than merely what a therapist does (i.e., adherence to a protocol), but also how a therapist does it (i.e., therapist skill and competence; see Sharpless & Barber, 2009).

Beyond demonstrating treatment integrity in a clinical trial, the relevance and importance of adherence and competence has expanded in recent years. Psychotherapy process and outcome researchers have become interested in these constructs as potential predictors of treatment outcome (e.g., Barber et al., 2006; Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996; DeRubeis & Feeley, 1990; Huppert, Barlow, Gorman, Shear, & Woods, 2006), and those involved in dissemination and implementation efforts strive to obtain high levels of sustained treatment fidelity when transporting evidence-based treatments into community and routine practice settings. Despite growing interest in these constructs, results from a recent meta-analysis conducted by Webb, DeRubeis, and Barber (2010) showed that the relationships between adherence, competence, and posttreatment outcome are quite small and not statistically differentiable from zero. Webb et al. (2010) offered several cogent explanations for the absence of significant relationships, including therapist responsiveness. The concept of responsiveness (see Stiles, 2009; Stiles, Honos-Webb, & Surko, 1998) refers to the idea that therapists adapt their behavior (intentionally or unintentionally) to the unfolding context of treatment, including specific patient behaviors and characteristics.

Although therapist responsiveness assumes mutual influence between therapist and patient, research and training related to treatment adherence and competence to date has been decidedly therapist-centric, and little attention has been given to factors that may influence treatment fidelity. In reference to therapist training, traditionally, therapists have been initially trained to a criterion level, and, in research involving ongoing fidelity assessments, sample ratings are collected and aggregated across therapists. In some cases, these aggregated ratings are then correlated with an outcome indicator. This approach ignores potentially important sources of variability in adherence and competence, at both the level of the therapist and patient. With regard to predictors of fidelity, the dissemination and implementation of evidence-based mental health interventions has received significant attention in the past decade, yet much remains to be learned about the contextual factors that promote or inhibit implementation efforts, including training and sustainability (see McHugh & Barlow, 2012). Knowledge of patient characteristics, for example, that are associated with variability in adherence and competence among and within therapists may inform training efforts and indicate when increased or continued monitoring of intervention delivery may be warranted.

The lack of attention to sources of variability and the factors that account for such variability is inconsistent with many clinicians’ experience that their ability to adhere to and competently deliver a prescribed intervention varies depending on the characteristics of the individual patient. In addition, overall adherence–outcome and competence–outcome correlations, which have been the focus of most research in this area to date, fail to differentiate within-therapist (i.e., differences between patients treated by the same therapist) and between-therapist (i.e., differences between therapists, aggregating across each therapist’s caseload) correlations. Given that these relationships are not always consistent with the overall correlation, within- and between-group (i.e., therapist) variability deserves attention (Baldwin, Wampold, & Imel, 2007; Huppert et al., 2011; Snijders & Bosker, 1999). As such, it is important to acknowledge multiple sources of variability in process variables, such as adherence and competence, and their relationships with change.

Barber et al. (2006), for example, found that a significant amount of the variance in adherence to supportive-expressive psychotherapy for cocaine dependence was accounted for at the patient level. In another study of substance abuse treatment, Imel, Baer, Martino, Ball, and Carroll (2011) recently found significant within- and between-therapist variability in adherence and competence in the delivery of motivational enhancement therapy (MET). Therapists differed in their adherence to and competence in delivering MET interventions, and their adherence and competence also varied between clients within their caseloads. Although relationships with outcome were not examined in this study, these results have begun to reveal a previously ignored mutual influence in therapist adherence and competence, which implies that clinical research and training efforts should not be exclusively focused at the level of the therapist, but rather this therapist working with this patient.

On the basis of this recent research and the conclusions of Webb et al. (2010), it stands to reason that patient characteristics may impact adherence and competence at multiple levels. Although scant, some evidence has begun to emerge in this area. For example, Imel et al. (2011) found that higher baseline symptom severity and a higher degree of impairment in family/social functioning were associated with reduced therapist MET skill use and competence. Associations between symptom severity and decrements in adherence and competence may help to explain the often observed relationship between higher initial severity and poorer treatment response (Clarkin & Levy, 2004). In a different study, Huppert et al. (2006) showed that patient motivation interacted with adherence to predict outcome in a randomized controlled trial (RCT) of cognitive-behavioral therapy (CBT) for panic disorder (Barlow, Gorman, Shear, & Woods, 2000), such that higher levels of adherence resulted in poorer outcomes for patients who were rated by their therapist as less motivated. However, the authors examined overall correlations in this study, rather than exploring within- and between-therapist sources of variance.

Additional patient factors that are likely to influence a therapist’s ability to maintain treatment fidelity include interpersonal characteristics, personality traits, and Axis II features. Many scholars have argued that the patterns of interpersonal problems and emotion dysregulation present in individuals with personality pathology, particularly in the “externalizing” Cluster B domain, can lead to a “deskilling” of the therapist (e.g., Bateman & Fonagy, 2004; Meehan, Levy, & Clarkin, 2012; Ryle, 2012). In line with this, Dozier (1990) reported that in a sample of patients presenting with heterogeneous disorders, those with dismissive/avoidant attachment styles were more rejecting of the therapist and treatment, and less likely to self-disclose. In a study of behavior therapy for obsessive-compulsive disorder, Hermesh, Shahar, and Munitz (1987) reported that none of the eight patients who also met criteria for borderline personality disorder in their sample complied with the intervention. Research has also indicated that patients with a higher degree of interpersonal problems respond less favorably to CBT for generalized anxiety disorder (Borkovec, Newman, Pincus, & Lytle, 2002). Furthermore, clinically significant personality disorder symptoms have been shown to be quite common in panic disorder samples (e.g., Telch, Kamphuis, & Schmidt, 2011). Arguably understudied in CBT (Newman et al., 2011; Ryle, 2012), the importance of attending to patient interpersonal/personality factors in treatment was summarized in Clarkin and Levy’s (2004) review of the impact of patient variables on psychotherapy process and outcome:

When an Axis II personality disorder is present, [therapists] should plan for disruptions in the treatment adherence and alliance. Many of the treatment manuals for symptom disorders such as anxiety and depression give insufficient information on approaches to patients with personality disorders who will present unique and difficult challenges in the treatment. (p. 202)

Despite being the approach of interest in most major dissemination and implementation efforts (McHugh & Barlow, 2012), sources of variability in adherence and competence in the delivery of CBT for anxiety disorders have yet to be examined empirically. Furthermore, factors that may impact adherence and competence (e.g., lead a therapist to become deskilled), such as interpersonal hostility, affectivity/reactivity, and ambivalence that are often present in Cluster B personality pathology, have received little empirical attention in the treatment of anxiety. Knowledge of factors that account for between-therapist variability in adherence or competence will help to identify therapists who struggle to maintain fidelity with particular patients. Knowledge of factors that account for within-therapist variability in adherence and competence will help to identify particular types of patients with whom a therapist, who is otherwise adherent and competent, struggles to maintain fidelity. Additionally, although CBT for panic disorder has been shown to be highly effective, many individuals experience minimal improvement or fail to achieve high end-state functioning (White & Barlow, 2002). As such, the present study was aimed at exploring between- and within-therapist variability in adherence and competence in the delivery of CBT for panic disorder, as well as their relationship with symptom change.

Specific Aims and Hypotheses

The present study was a secondary analysis of adherence and competence data collected as part of a large multisite RCT examining the efficacy of long-term treatment strategies for panic disorder (Aaronson et al., 2008; White et al., in press). This study had two key aims, derivative of establishing between- and within-therapist variability in both adherence and competence in this trial. Although variability in therapist adherence and competence is assumed to be restricted in RCTs, recent research has demonstrated statistically significant variability in both constructs in controlled studies (e.g., Barber et al., 2006; Imel et al., 2011), primarily in nonanxious (i.e., anxiety was not principal) samples. Nevertheless, Huppert et al. (2006) demonstrated a link between varying degrees of adherence and posttreatment outcome (in the context of an interaction effect) in a different trial examining the same CBT treatment for panic disorder. Therefore, we expected that adherence and competence would vary significantly both between and within therapists in this trial of CBT for panic disorder.

Upon demonstrating this, we (a) sought to examine the association between patient characteristics and adherence and competence ratings during treatment. Specifically, we tested the influence of presession severity and the patient interpersonal/personality characteristics of interpersonal sensitivity, interpersonal aggression, and interpersonal ambivalence. Previous research examining the relationships between severity and adherence or competence has relied on baseline severity in pre–post outcome designs. We are unaware of research that has examined the association with severity assessed immediately prior to the session from which adherence and competence ratings were obtained. It is possible that a patient’s level of severity at this session is more likely to impact fidelity in that particular session. For presession severity, we adopted a rival hypothesis approach similar to that taken by Imel et al. (2011), in which we anticipated associations in both directions. Theoretically, patients presenting with greater severity at this session (perhaps due to an acute crisis) may prove to be more challenging for the therapist in that session. Alternatively, patients with a higher degree of severity could “pull” for greater protocol adherence, and potentially competence as well. In other words, this challenge may lead the therapist to adhere more closely to the protocol and provide an opportunity to display greater competence.

Surprisingly, very little is known about the impact of patient interpersonal/personality characteristics on treatment process and therapist behavior, particularly in CBT for anxiety disorders. The research that has been conducted suggests that patients with dismissive interpersonal styles and Cluster B traits are more likely to be treatment rejecting and noncompliant (Dozier, 1990; Hermesh et al., 1987). Anger and hostility are widely recognized as difficult emotions to work with in therapy (Mayne & Ambrose, 1999). High levels of interpersonal aggression, particularly when directed at the therapist, may lead a therapist to go “off-track.” Increased interpersonal sensitivity (reactivity with high levels of affect) is potentially dysregulating for the therapist (leaving one “deskilled”) and is likely to influence both the giving and receiving of feedback. For example, a therapist may be more likely to avoid addressing homework noncompliance, a key component of CBT, with an interpersonally sensitive patient. Interpersonal ambivalence, largely reflected in difficulty positively collaborating with others, represents a dismissive style that may be associated with decreased engagement (impeding “collaborative empiricism”) and treatment compliance. Consequently, the therapist may spend much of his or her time managing the patient, rather than following the prescribed treatment procedures. Due to the paucity of research in this area, we did not formulate differential hypotheses for each of these interpersonal/personality factors; rather, we hypothesized that higher levels of each factor would be associated with decrements in adherence and competence.

Finally, we (b) tested whether variability in adherence and competence was associated with subsequent panic symptom severity and related impairment. Although understudied in treatments for panic disorder, weak overall correlations between adherence/competence and posttreatment outcome have been observed in previous research. Thus, in addition to accounting for multiple sources of variability in our models, we chose the following alternative strategy. Rather than focus on posttreatment outcome as the dependent variable, we chose to focus on session-to-session change. As described below (see the Method section), patients provided ratings of panic severity and impairment prior to each therapy session. Therefore, we examined the more proximal relationship between variability in adherence/competence and panic severity at the subsequent therapy session. Some experts in the field have argued that the weak relationships observed between specific technical factors and outcome may be because termination outcome is too distal in relation to certain processes; consequently, more attention should be paid to proximal indicators at the session level or session-by-session markers of change (Hill & Lambert, 2004; Stiles, 1988). In line with this, and based on more recent research demonstrating significant relationships between variability in specific technique use and session-level outcome indicators (e.g., Boswell, Castonguay, & Wasserman, 2010; Owen, 2012), we hypothesized that lower levels of adherence and competence (relative to other therapists in the sample and other patients within a therapist’s caseload) in a session would be associated with higher levels of symptom severity at the next session.

Method

Data were derived from a multisite RCT for panic disorder (Aaronson et al., 2008; White et al., in press) that was initially reviewed and approved by each respective site’s Institutional Review Board. This study included multiple treatment phases involving CBT and medication; however, the present study was only concerned with the acute CBT treatment phase, which all participants received first and did not include medication. Acute CBT treatment consisted of 11 weekly, 45- to 60-min sessions of manualized panic control treatment (Barlow & Craske, 2007; see also Aaronson et al., 2008, and White et al., in press, for more detailed information regarding treatment phases and primary outcomes). CBT treatment included (a) education regarding the nature of anxiety and panic, (b) identification and correction of maladaptive thoughts about anxiety and its consequences, (c) interoceptive exposures, and (d) graded situational exposures.

Participants and Procedure

In the original study, a total sample of 379 adult participants completed a baseline diagnostic interview and met criteria for a principal diagnosis of panic disorder (see Aaronson et al., 2008, for detailed inclusion/exclusion criteria). A total of n = 256 participants completed treatment. Approximately 15% of all recorded treatment sessions were randomly sampled and rated for adherence and competence. Due to feasibility concerns given the size of the sample, it was determined that this strategy would yield sufficient data to be representative of the patient sample and the course of treatment (e.g., rather than focus on a specific session for all patients). Most treatment completers had at least one session rated, yet treatment noncompleters also had sessions rated (n = 49). Some patients (including both completers and noncompleters) did not have a session rated due to early dropout, withdrawal, or mechanical recording error (see the Measures section below).

Because we were interested in examining variability both between and within therapists, it was important to include therapists who had at least two patients with adherence and competence ratings. A total of 21 therapists each had adherence and competence ratings for at least two different patients. All 21 therapists contributed to each analysis. These therapists saw a total of 276 patients, with a range of two to 36 patients per therapist.¹ The average number of patients per therapist in each analysis was 10.64 (SD = 8.62), with a median number of 9.50. An average of 1.83 sessions (SD = 1.50) were rated per patient; 65% (n = 179) of patients had one session rated, 21% (n = 58) had two to three sessions rated, 10% (n = 28) had four to five sessions rated, and 4% (n = 11) had six to seven sessions rated. This sample of therapists and patients resulted in a total of 495 rated sessions for analysis in this study. The full continuum of treatment sessions (1–11) was represented in the sample; 18% of ratings were from Session 1 or 2; 23% were from Sessions 3 or 4; 20% were from Sessions 5 or 6; 19% were from Sessions 7 or 8, and 20% were from Sessions 9, 10, or 11.

The participant characteristics of this adherence and competence sample were similar to those reported for the intent-to-treat sample (see Aaronson et al., 2008). The majority of the sample was female (68%) and identified as White (85.5%), followed by African American (5.8%) and Asian or Pacific Islander (5.4%); less than 4% identified as Hispanic or “other.” The average age was 37 years (SD = 12 years). As reported by White et al. (2010), the 21 therapists had a mean age of 37.8 years (SD = 10.00). All therapists were highly trained. A total of 13 therapists had a medical degree or doctoral degree in clinical psychology, six therapists had a master’s degree in clinical psychology, and two therapists were experienced clinicians in doctoral training. The average number of years of clinical experience was 11.3 years, and the majority of therapists identified their theoretical orientation as cognitive-behavioral (86%, n = 18). Each therapist received extensive training and ongoing supervision in the CBT protocol prior to and throughout their participation in the study. Prior to seeing their first study patient, therapists had to demonstrate a high level of adherence and competence with at least two training cases.

Measures

Symptom severity and impairment

Symptom severity was assessed with the Panic Disorder Severity Scale–Self Report (PDSS-SR; Houck, Spiegel, Shear, & Rucci, 2002). This seven-item measure assesses the severity of seven dimensions of panic (frequency of panic attacks, distress during panic attacks, anticipatory anxiety, agoraphobic avoidance, interoceptive fear and avoidance, impairment and interference in work, and social functioning). A total score is calculated that ranges from 0 to 28, with higher scores indicating greater symptom severity and impairment. This scale has demonstrated good psychometric properties (Cronbach’s α = 0.92 [α = .91 in this sample of sessions], ICC = 0.81; Houck et al., 2002; Shear et al., 2001) and sensitivity to change as a weekly symptom measure (Aaronson et al., 2008). Patients completed the PDSS-SR prior to each psychotherapy session. For the purposes of this study, we were interested in PDSS-SR scores at two time points: the PDSS-SR scores linked with sessions that were rated for adherence and competence (representing “baseline” or presession severity) and the PDSS-SR scores from the subsequent week (i.e., symptom severity reported prior to the next session, representing the dependent variable). We chose weekly PDSS-SR scores as an indicator of symptom severity and impairment for several reasons. First, this is a specific measure of panic disorder severity. Second, adherence/competence ratings and PDSS-SR scores are linked by session and both are time-varying. Posttreatment severity ratings may be less appropriate for measuring the effects of certain time-varying predictors, such as adherence/competence. Therefore, it was determined that a more proximal outcome marker might increase the odds of detecting an effect if one existed.

Adherence and competence

Adherence and competence were assessed with an observer-rated scale that was developed for the study by the investigators and based on adherence and competence scales used in previously conducted RCTs for panic (e.g., Barlow et al., 2000; Huppert et al., 2006). The scale assesses therapist factors, patient factors (e.g., homework completion), and includes one item that assesses rapport. We were interested in two therapist factors in the scale: overall protocol adherence and overall competence. Overall protocol adherence was represented by a total percentage (0%–100%) of the specific concepts and techniques that were to be addressed during each session. For example, under the session goal of “Conducting In-Session Interoceptive Exposure and Assigned Homework Practices” (e.g., Session 5), one of the items read “Did the therapist select [an interoceptive] procedure to do in session and choose something that will produce sensations that are similar to those experienced during natural panic attacks and, if possible, that cause moderate anxiety?” With the exception of the first session, each session included adherence items related to review of homework and concepts. Thus, therapists were allowed to revisit material from earlier sessions. Furthermore, therapists were not penalized for addressing relevant treatment issues that may not have been specifically listed in the protocol adherence form for that session, as long as they were addressed in a protocol-consistent manner and the specified agenda items for that session were also attempted.

In addition to specific adherence items, each session goal (e.g., conduct in-session interoceptive exposure) was given a competence rating based on the following question: Overall, how effectively did the therapist accomplish the goal of this part of the session? Ratings ranged from 0 (Did not attempt) to 5 (Excellently). All session goals should have been attempted, thus were considered applicable. If a goal was not attempted, a competence rating of “0” was given for that goal. The overall competence rating for a session, which was the focus of the present study, was the mean of all goal-specific competence ratings for that session. All ratings were based on listening to audiotapes of entire treatment sessions. All adherence and competence raters (n = 4) were doctoral-level psychologists and psychiatrists who were trained and certified in the treatment protocol. Each rater was carefully trained to a high level of reliability prior to the study. A subset of the audiotaped sessions (n = 60) was corated for reliability analysis and demonstrated adequate reliability, based on absolute intraclass correlations (ICCs) of single ratings, for both adherence (ICC = 0.80) and competence (ICC = 0.77).

Interpersonal/personality characteristics

Patient interpersonal/personality characteristics were assessed with an abbreviated version of the Inventory of Interpersonal Problems-Personality Disorder screening instrument (IIP-PD; Kim & Pilkonis, 1999). The original IIP (Horowitz, Rosenberg, Baer, Ureño, & Villaseñor, 1988) is a self-report instrument designed to assess different types of interpersonal problems and associated distress. With the goal of creating a self-report personality disorder screening tool, and guided by the perspective that chronic difficulties in interpersonal relationships is one of the best indicators of personality pathology, Pilkonis, Kim, Proietti, and Barkham (1996) identified three sub-scales from the IIP that reliably differentiated patients with and without a personality disorder diagnosis: interpersonal sensitivity (high affectivity and reactivity), interpersonal aggression (hostility), and interpersonal ambivalence (vacillating between collaborative and noncollaborative stances). The IIP-PD scale consists of items that capture externalizing, Cluster B personality pathology (e.g., borderline, histrionic, narcissistic). Consistent with the IIP, IIP-PD items are rated on a 5-point scale where individuals endorse the degree to which they have difficulty either engaging in a particular interpersonal behavior or engaging a particular interpersonal behavior too much/often. Mean IIP-PD subscale scores between 0.70 and 1.10 indicate a possible personality disorder, and scores above 1.10 indicate that a personality disorder is probable (Kim, Pilkonis, & Barkham 1997).

The IIP (Horowitz et al., 1988; Monsen, Hagtvet, Havik, & Eilertsen, 2006) and IIP-PD have demonstrated reliability as well as divergent and convergent validity (Kim et al., 1997; Pilkonis et al., 1996; Scarpa et al., 1999). In order to streamline the IIP-PD further, Kim and Pilkonis (1999) used item-response theory and receiver operating characteristics analysis in a large clinical sample, which resulted in five-item subscales for each of the aforementioned IIP-PD factors. Participants in the present study completed this 15-item IIP-PD at pretreatment. Example items include: “I am too sensitive to criticism” (interpersonal sensitivity); “I am too aggressive toward other people” (interpersonal aggression); and “It is hard for me to be supportive of another person’s goals in life” (interpersonal ambivalence).

Data Analytic Strategy

Multilevel modeling was used to examine (a) the presence of variability in adherence/competence between and within therapists, (b) whether patient characteristics (i.e., symptom severity and interpersonal/personality factors) were associated with adherence/competence, and (c) whether variability in adherence/competence was associated with subsequent symptom severity. Multilevel modeling is an ideal method of data analysis for nested data structures, such as the present study, in which repeated measurements are nested within individuals. Multilevel models can better account for heterogeneous spacing of observations across patients, missing data, and violations of the assumption of independence (Singer & Willet, 2003). The data in this study contained three levels: session adherence/competence ratings and symptom severity scores (Level 1) nested within patients (Level 2) and patients nested within therapists (Level 3). Because we were interested in variability at multiple levels and generalizability beyond this sample, all models treated both patients and therapists as random effects. Variation between patients sharing the same therapist is accounted for at Level 2 (henceforth referred to as within-therapist variability), whereas variation between therapists is accounted for at Level 3 (henceforth referred to as between-therapist variability; Raudenbush & Bryk, 2002). Data were analyzed using the mixed models function in SPSS 20.0 and restricted maximum likelihood estimation.

Regarding the initial demonstration of between- and within-therapist variability, we specified two null models (one for adherence and one for competence) to determine the proportion of variance in adherence and competence at both the between- and within-therapist levels. This was represented by the following equation, with random effects in brackets:

$$y_{\mathit{ijk}}={\gamma }_{000}+[{\mathrm{\mu }}_{00k}+{\mathrm{\mu }}_{0jk}+{\mathrm{\epsilon }}_{\mathit{ijk}}],$$

where y_ijk represents the predicted adherence/competence rating at session i, for patient j, treated by therapist k; γ₀₀₀ represents the intercept; μ_00k is the random variance component between therapists (i.e., Level 3); μ_0jk is the random variance component within therapists (i.e., between patients; Level 2); and ε_ijk is the random variance component within patients (i.e., between sessions; Level 1). Results from these null models provided estimates of random effects variance components, which were used to calculate ICCs that indicate the percentage of variance in ratings explained at each level (Tabachnick & Fidell, 2007), using equations such as:

$${\text{ICC}}_{\mathit{therapist}}=\frac{{\sigma }_{\mathit{therapist}}^2}{{\sigma }_{\mathit{therapist}}^2+{\sigma }_{\mathit{client}}^2+{\sigma }_e^2}.$$

where the numerator, ${\sigma }_{\mathit{therapist}}^2$, is the variance component of interest (in this case, the between-therapist variance component, Level 3), and the denominator is the sum of the estimated variance components (Levels 1, 2, and 3, totaling 1.00).

For the first key study aim, two identical multilevel models were then tested to examine predictors of adherence and competence, respectively. Specifically, we included between- and within-therapist presession severity as fixed effects. Each patient’s severity score was centered around his or her therapist’s mean severity score (across all of his or her patients) so that within- and between-regression coefficients could be estimated. Both models also included fixed effects for each interpersonal/personality variable (interpersonal sensitivity, interpersonal aggression, and interpersonal ambivalence) measured at baseline. Although we were not specifically interested in the effect of time, to be consistent with previous research, we tested the possibility of growth in adherence and competence by also entering session number as a fixed effect in both models. As was the case with the first model, patients and therapists were treated as random effects. Using the model predicting adherence as an example, this analysis was represented by the following equation:

$$y_{\mathit{ijk}}={\gamma }_{000}+{\gamma }_{000}{s}_{\mathit{ijk}}+{\gamma }_{001}({\overline{x}}_k-\overline{x})+{\gamma }_{010}({\overline{x}}_{jk}-{\overline{x}}_k)+{\gamma }_{020}w{1}_{jk}+{\gamma }_{030}w{2}_{jk}+{\gamma }_{040}w{3}_{jk}+[{\mathrm{\mu }}_{00k}+{\mathrm{\mu }}_{0jk}+{\mathrm{\epsilon }}_{\mathit{ijk}}],$$

where y_ijk is the adherence rating for the ith session of the jth patient seen by the kth therapist; γ₀₀₀ is the intercept; γ₀₀₀s_ijk is the session number ith of the jth patient seen by the kth therapist; x̄_k is the mean presession panic severity score for the kth therapist (average across all of his or her patients); x̄ is the presession severity grand mean; x̄_jk is the presession panic severity score for the jth patient seen by the kth therapist; γ₀₀₁ is the between-therapist regression coefficient for presession severity; γ₀₁₀ is the within-therapist regression coefficient for presession severity; γ₀₂₀w1_jk is the IIP-PD aggression regression coefficient for the jth patient seen by the kth therapist; γ₀₃₀w2_jk is the IIP-PD sensitivity regression coefficient for the jth patient seen by the kth therapist; γ₀₄₀w3_jk is the IIP-PD ambivalence regression coefficient for the jth patient seen by the kth therapist; μ_00k is the between-therapist variance; μ_0jk is the within-therapist variance; and ε_ijk is the residual variance at Level 1.

For the second key study aim, we first specified a null model for symptom severity, measured with the subsequent week’s PDSS-SR score, to determine the proportion of variance in symptom severity at both the between- and within-therapist levels (as was done in the first study aim). We then tested two identical models for adherence and competence that included between- and within-therapist variability in adherence and competence, respectively, as fixed effects. Each patient’s adherence/competence rating was centered around his or her therapist’s mean adherence/competence rating so that within- and between-regression coefficients could be estimated. Using the adherence model as an example, this analysis was represented by the following equation:

$$y_{\mathit{ijk}}={\gamma }_{000}+{\gamma }_{001}({\overline{x}}_k-\overline{x})+{\gamma }_{010}({\overline{x}}_{jk}-{\overline{x}}_k)+[{\mathrm{\mu }}_{00k}+{\mathrm{\mu }}_{0jk}+{\mathrm{\epsilon }}_{\mathit{ijk}}]$$

where y_ijk is the subsequent panic severity score for the ith session of the jth patient seen by the kth therapist; γ₀₀₀ is the intercept; x̄_k is the mean adherence rating for the kth therapist (average across all of his or her patients); x̄ is the adherence grand mean; x̄_jk is the adherence rating for the jth patient seen by the kth therapist; γ₀₀₁ is the between-therapist regression coefficient for adherence; γ₀₁₀ is the within-therapist regression coefficient for adherence; μ_00k is the between-therapist variance component; μ_0jk is the within-therapist variance component; and ε_ijk is the residual variance at Level 1. We elected not to include session number or presession PDSS-SR severity initially as predictors in the models predicting subsequent symptom severity. We were not specifically interested in testing a main effect for time on outcome, as Aaronson et al. (2008) previously demonstrated such an effect in the original RCT outcome article. Additionally, we were not specifically interested in the correlation between severity scores at adjacent time points. Baseline severity ratings are almost always highly correlated with subsequent severity, and severity ratings from adjacent therapy sessions will always be highly correlated. Therefore, we approached this aim by first testing a direct and simplified model that included our primary variables of interest, with the intention of including covariates in a subsequent model if warranted to see whether any observed main effects would hold (should they exist) when controlling for these additional factors.

Results

On average, therapists exhibited a high degree of adherence (M = 85%, SD = 10.4, range = 48%–100%) and competence (M = 3.53, SD = 0.53, range = 1.80–5.0). The mean presession PDSS-SR score was 8.63 (SD = 5.45, range = 0.00–26.00), and the mean subsequent PDSS-SR score was 7.99 (SD = 5.55, range = 0.00–26.00). The mean IIP-PD Sensitivity subscale score was 1.67 (SD = 0.90, range = 0.00–4.00), the mean IIP-PD Aggression subscale score was 1.04 (SD = 0.92, range = 0.00–3.80), and the mean IIP-PD Ambivalence subscale score was 0.90 (SD = 0.90, range = 0.00–3.60). These scores indicated a notable level of possible-to-probable personality pathology in this sample. The correlation between the IIP-PD Sensitivity and Aggression sub-scale was r = .49 (CI [0.27, 0.60]), the Sensitivity and Ambivalence subscale was r = .39 (CI [0.29, 0.49]), and the Aggression and Ambivalence subscale was r = .38 (CI [0.27, 0.47]). The overall correlation between adherence and competence was r = .54 (CI [0.47, 0.60]). The overall correlation between adherence and subsequent PDSS-SR was nonsignificant (r = .08, CI [−0.02, 0.07]). Conversely, the overall correlation between competence and subsequent PDSS-SR was significant (r = .15, CI [0.05, 0.25]), indicating that a higher competence rating was associated with greater reported symptom severity and impairment at the next session. However, these overall correlations do not separate within- and between-therapist correlations, which can be significantly different from the overall association (see Baldwin et al., 2007).

Variability in Adherence and Competence

Table 1 provides the fixed and random effects for both adherence and competence models. The between- and within-therapist random variance components for both the adherence and competence models were all significantly different from zero. ICCs were calculated as an index of the proportion of variance explained between and within therapists. Following the procedures outlined above, and described by Raudenbush and Bryk (2002), the between- and within-therapist variance components were divided by the sum of the between ${\sigma }_{\mathit{therapist}}^2$, within ${\sigma }_{\mathit{client}}^2$, and residual ${\sigma }_e^2$ variance components. The ICCs indicated that there was significant variance in adherence at both the between-therapist (ICC = .19) and within-therapist (ICC = .14) levels. Similarly, there was significant variance in competence at both the between-therapist (ICC = .15) and within-therapist (ICC = .31) levels. These results show that CBT adherence and competence varies not only between therapists but also between patients working with the same therapist.

Table 1.

Fixed and Random Effects for Three-Level Null Models Partitioning Variance in Adherence and Competence

Variable	Adherence			Competence
	Coefficient	95% CI	ICC	Coefficient	95% CI	ICC
Fixed effects
Intercept γ000	85.47**	[82.81, 88.13]		3.56	[3.44, 3.69]
Random effects
Residual εijk	99.32**	[81.27, 121.38]	0.67	0.22**	[0.18, 0.26]	0.54
Within therapist μ0jk	20.17*	[7.54, 53.99]	0.14	0.12**	[0.08, 0.17]	0.31
Between therapist μ00k	28.10**	[12.44, 63.45]	0.19	0.06*	[0.02, 0.15]	0.15

Note. Intraclass correlation (ICC) indicates proportion of variance in adherence/competence at each level. CI = confidence interval.

^*p < .05.

^**p < .01.

Patient Characteristics Predicting Adherence and Competence

We next sought to examine predictors of adherence and competence. Table 2 provides the fixed and random effects for both models. A significant main effect was observed for session number on adherence (α = −0.86, SE = 0.17), t(467) = −5.06, p < .01, CI [−1.19, −0.52], indicating that adherence ratings deteriorated over the course of treatment. A significant main effect was also observed for interpersonal aggression (β = −2.97, SE = 0.99), t(265) = −3.00, p < .01, CI [−4.92, −1.02], indicating that, as predicted, higher levels of trait interpersonal aggression were associated with lower adherence ratings. The same pattern of results emerged for the model predicting competence. A significant main effect for session number was observed (β = −0.02, SE = 0.01), t(459) = −2.18, p < .05, CI = [−0.04, −0.01], indicating that competence ratings deteriorated over the course of treatment. A significant main effect was also observed for interpersonal aggression (β = −0.11, SE = 0.06), t(332) = −2.00, p < .05, CI [−0.22, 0.01], indicating that, as predicted, higher levels of trait interpersonal aggression were associated with decrements in competence ratings. Neither between- nor within-therapist presession severity was significantly associated with adherence or competence.²

Table 2.

Fixed and Random Effects for Three-Level Model of Adherence and Competence With Panic Severity and Interpersonal Subscale Scores

Variable	Adherence			Competence
	Coefficient	95% CI	ICC	Coefficient	95% CI	ICC
Fixed effects
Intercept γ000	91.70**	[87.86, 95.54]		3.77**	[3.57, 3.97]
Session γ000sijk	−0.86**	[−1.19, −0.52]		−0.02*	[−0.04, −0.01]
BT PDSS-SR γ001	0.27	[−1.13, 1.69]		0.04	[−0.03, 0.11]
WT PDSS-SR γ010	0.07	[−0.16, 0.30]		−0.01	[−0.02, 0.01]
IIP Aggression γ020w1jk	−2.97**	[−4.93, −1.02]		−0.11*	[−0.22, 0.01]
IIP Sensitivity γ030w2jk	0.50	[−1.00, 2.00]		−0.02	[−0.11, 0.06]
IIP Ambivalence γ040w3jk	−0.97	[−3.08, 1.14]		−0.03	[−0.15, 0.09]
Random effects
Residual εijk	103.34**	[85.86, 124.38]	0.76	0.22**	[0.18, 0.27]	0.63
Within therapist μ0jk	6.71	[0.62, 72.57]	0.05	0.09**	[0.06, 0.16]	0.26
Between therapist μ00k	21.30*	[9.75, 46.51]	0.16	0.04	[0.01, 0.11]	0.11

Note. Intraclass correlations (ICCs) are residual after fixed effects. CI = confidence interval; BT = between therapist; PDSS-SR = Panic Disorder Severity Scale–Self-Report; WT = within therapist; IIP = Inventory of Interpersonal Problems.

^*p < .05.

^**p < .01.

In addition, decrements in between- and within-therapist variance components were observed in relation to the baseline adherence and competence models reported in Table 1. Including interpersonal factors in the model decreased the within-therapist effect from ICC = 0.14 to ICC = 0.05, rendering this variance component statistically nonsignificant. The between-therapist variance component for adherence was also reduced, yet remained significant. For competence, including interpersonal factors in the model decreased the within-therapist effect from ICC = 0.31 to ICC = 0.26, yet the within-therapist component remained significant. The between-therapist variance component was also reduced, to the point of rendering this variance component nonsignificant.

Adherence and Competence Predicting Subsequent Severity

We next examined the relationship between adherence/competence and subsequent panic severity. We first tested a null model to determine the proportion of variance in symptom severity at both the between- and within-therapist levels (as was done in the first study aim). The estimate of the within-therapist random variance component was significant (β = 3.72, SE = 1.04, p < .01, CI [2.15, 6.42], ICC = .30), indicating that subsequent panic severity scores varied significantly between patients being treated by the same therapist. However, the between-therapist random variance component was not significant (β = 0.68, SE = 0.41, p = .10, CI [0.21, 2.22], ICC = .05), indicating minimal differences in average patient severity across therapists. Nevertheless, for generalizability purposes, therapist was still treated as a random effect in subsequent models.

Results for the adherence and competence models predicting subsequent severity are reported in Table 3. No significant main effects were observed, indicating that neither between- nor within-therapist variability in either adherence or competence was associated with subsequent panic severity. Furthermore, the random variance components were essentially identical to those observed in the severity null model just described. This indicates that even when a more proximal indicator is used and variability is accounted for at multiple levels, the associations between adherence and competence and outcome can be rather meager.

Table 3.

Fixed and Random Effects for Three-Level Models of Adherence and Competence Predicting Panic Severity

Variable	Adherence			Competence
	Coefficient	95% CI	ICC	Coefficient	95% CI	ICC
Fixed effects
Intercept γ000	1.49**	[0.75, 2.22]		1.60**	[0.85, 2.34]
Between therapist γ001	−0.02	[−0.06, 0.03]		−0.17	[−1.85, 1.51]
Within therapist γ010	0.02	[−0.04, 0.07]		0.76	[−0.02, 1.54]
Random effects
Residual εijk	7.75**	[6.21, 9.66]	0.64	8.02**	[0.18, 0.27]	0.65
Within therapist μ0jk	3.74**	[2.17, 6.43]	0.31	3.64**	[2.09, 6.32]	0.29
Between therapist μ00k	0.65	[0.20, 2.15]	0.05	0.70	[0.21, 2.27]	0.06

Note. Intraclass correlation (ICCs) are residual after fixed effects. CI = confidence interval.

^**p < .01.

Although this lack of association rendered the subsequent examination of additional covariates somewhat moot, because there was a significant effect for session in previous models, we chose to conduct an exploratory analysis that included a main effect for session and Session × Adherence/Competence interaction terms predicting subsequent panic severity. A significant interaction effect would indicate that the relationship between variability in adherence or competence and panic severity may be a function of the time point in treatment. As would be expected, results revealed a significant main effect for session, indicating that panic severity scores decreased over time (β = −0.13, SE = 0.06), t(456) = −2.39, p < .05, CI [−0.24, −0.02]. However, none of the Session × Adherence/Competence interaction terms were significantly related to panic severity (ps > .28).

Discussion

Therapist adherence and competence have become an important focus of psychotherapy process and outcome research, and the relevance of these constructs has become magnified as more attention is being paid to dissemination, training, and the sustainability of evidence-based treatments in routine community practice settings (see McHugh & Barlow, 2012). However, little is known about the factors that impact fidelity, and, with a few exceptions (e.g., DeRubeis & Feeley, 1990) adherence/competence and posttreatment outcome associations have been mixed or modest at best (Webb et al., 2010). Consequently, the implications for practice and training are unclear. Adherence and competence have traditionally been treated as relatively static factors that are generalized across multiple therapists, rather than exploring whether variability exists between and within therapists, or over time. Research to date has also focused heavily on distal outcome indicators that may be less sensitive to variations in adherence and competence at the session level, while simultaneously failing to acknowledge that patient and other contextual factors may significantly influence fidelity. Thus, after statistically demonstrating significant variability, the present study was aimed at (a) examining predictors (pre-session severity and patient interpersonal/personality traits) of variability in adherence and competence ratings and (b) testing whether adherence and competence were associated with subsequent panic symptom severity and related impairment.

Consistent with our expectation, adherence and competence ratings varied significantly both between and within therapists. The largest ICC was for within-therapist competence ratings (ρ = .31), demonstrating that a therapist’s observed competence differed substantially depending on the particular patient. The between-therapist ICC (ρ = .19) also highlighted significant differences among therapists in the trial in their average level of adherence. The level of variability observed in this panic trial, at both the between- and within-therapist level, is consistent with what has been demonstrated in other RCTs when such data have been reported (e.g., Barber et al., 2006; Imel et al., 2011). Given the consistency of results across studies examining fidelity in psychodynamic, motivational enhancement, and now cognitive behaviorally oriented psychotherapy, we believe that there is strong evidence for taking individual patient characteristics into consideration in the assessment and training of psychotherapists.

Differences among therapists imply that some therapists are likely to require more intensive or prolonged training and supervision experiences. Even in a well-controlled treatment trial, with highly trained and supervised therapists using a manualized protocol, significant differences between therapists emerged in their average adherence and competence, despite the sample demonstrating a relatively high overall level of fidelity. The overall level of fidelity may be expected to be higher, and the range of ratings somewhat more restricted, in controlled efficacy studies, when compared with training and implementation in effectiveness research and community settings. In fact, some research has shown that there is greater variability in adherence and competence in less controlled settings (e.g., community mental health centers), even when clinicians are attempting to use evidence-based psychotherapies (Henggeler, 2004; Schoenwald, 2012), indicating that these issues are magnified when treatments are used in everyday clinical practice. Further underscoring its importance, this so-called fidelity problem has sparked a recent National Institute of Mental Health (NIMH) initiative to improve the assessment and enhancement of treatment fidelity in routine practice settings. Any such initiative to facilitate and maintain fidelity that fails to consider the role of contextual factors and patient characteristics, or does not attempt to identify therapists who may require more intensive or additional training and supervision, is likely to be limited.

Regarding factors that may help account for variability in these constructs, although presession severity was unrelated to adherence or competence, session number and patient trait interpersonal aggression were significantly associated with both of these factors. The lack of relationship with presession severity is inconsistent with Imel et al.’s (2011) finding that baseline severity was associated with specific MET skills use and competence. Although we do not believe that presession severity is necessarily more appropriate or important to adherence and competence than pretreatment severity, unlike previous research that has been limited by the availability of only pre–post assessments, we took advantage of the time-varying nature of the severity scores and fidelity ratings. Conceptually, it is possible that presession severity will have an immediate impact on therapist behavior in that particular session. However, this was not demonstrated empirically here, nor was an association with baseline severity (see Footnote 2). Alternatively, it is possible that a therapist’s level of adherence or competence will differ for patients who enter a given session with the same level of severity, as there are multiple “roads to Rome.” That is, patients can present with the same level of severity through different paths—some have already changed substantially, some remain at their baseline levels, and others have deteriorated. A patient’s path may, therefore, ultimately be more predictive of adherence and competence in a given session.

Over half of the variance in adherence and competence was explained at the session level, suggesting that fidelity is contextually driven. Adherence and competence ratings deteriorated significantly over the course of treatment. This provides evidence for “therapist drift” in fidelity as treatment progresses, again highlighting the importance of continued supervision or consultation for sustainability. The final phase of CBT protocols for panic disorder, such as the one used in the present study, typically involves repeated interoceptive and situational exposures. Although the components and structure of these exposures are well articulated, the nature, focus, and progression of exposure exercises can vary greatly between patients. Successful implementation of these treatment procedures may require particular skill on the part of the therapist (e.g., deciding whether or not to remove all safety signals at once, or moving to a new item on the hierarchy). The increased complexity in the final treatment phase may lead to greater variability within and between therapists, which might help to explain the observed decrements over time.

Adherence and competence ratings were also negatively associated with patient trait interpersonal aggression. Higher levels of self-reported interpersonal aggression, as measured by the IIP-PD, were associated with lower adherence and competence ratings. Although research in this area is limited, scholars and clinicians have frequently described the difficulties in working with hostile patients. Interpersonal hostility may lead to treatment- and therapist-rejecting behaviors from the patient (Dozier, 1990) and result in therapists being pulled off-track, feeling deskilled, and responding in their own hostile, and possibly harmful, way (Castonguay, Boswell, Constantino, Goldfried, & Hill, 2010). Although CBT protocols for panic disorder (e.g., Barlow & Craske, 2007) provide a framework for therapists to conduct a functional analysis of various panic-related behavior patterns and include strategies to address manifestations of resistance, such as homework noncompliance, therapists may be less well equipped to respond to high levels of interpersonal aggression, as the function of such aggression is sometimes less clear and not explicitly articulated in the protocol. Despite evidence that high levels of anger are prevalent in anxiety disorders (Fava et al., 1993; Gould et al., 1996) and the recognition that anger and hostility are difficult emotions to work with in therapy (Mayne & Ambrose, 1999), patient anger has received little attention in the CBT treatment literature. Thus, more attention should be directed toward assisting therapists in the conduct of a functional analysis of nonanxious emotions, such as anger (Barlow, Allen, & Choate, 2004; Barlow et al., 2011), as well as the ability to appropriately respond to anger within a CBT model.

Although variability in adherence and competence were observed, neither adherence nor competence was related to subsequent symptom severity. Even with a more proximal change variable, such as subsequent panic severity, nonsignificant relationships were observed. Similar results have, therefore, been observed across methods and statistical approaches. Some researchers have suggested that these nonsignificant relationships may be, at least partially, explained by the relatively high levels of fidelity and restricted variability observed in RCTs. However, we do not believe that this factor is sufficient to account for null findings when there is such a high degree of variability being demonstrated at the session level. Furthermore, the levels of variability in adherence and competence observed in this study are comparable to levels observed in other constructs that have been linked with outcome (both distal and proximal; Baldwin et al., 2007; Boswell et al., 2010).

We believe this study contributes to the adherence and competence literature in three important ways. First, using multilevel modeling, multiple sources of variance were accounted for and significant variability in adherence and competence were demonstrated in a well-controlled CBT treatment study with highly trained therapists. Using sophisticated methods, this demonstration of mutual influence provides, albeit indirect, statistical support for the responsiveness hypothesis (Stiles et al., 1998). Second, very little is known about the contextual factors that influence therapist adherence and competence. Perhaps most importantly, this study is one of the first to identify a specific patient characteristic (interpersonal aggression) that accounts for variability in using an evidence-based treatment both between and within therapists. Additionally, this result and the variance accounted for at the session level suggest that adherence and competence are more contextually driven, rather than a static trait of therapists or a particular patient’s course of treatment. Third, we examined in this study an alternative indicator of outcome that tested the generalizability of previous adherence/competence–outcome findings.

If adherence and competence are contextually driven, then it behooves researchers (and trainers) to begin identifying additional factors that both facilitate and hinder treatment fidelity. The present results indicate that patient trait interpersonal aggression is one factor that is associated with decrements in fidelity both between and within therapists. The nature of this identified patient characteristic underscores the importance of interpersonal factors and interpersonal process in CBT, factors that are often less emphasized in treatment manuals. Additionally, assuming mutual influence and taking into consideration the research on harmful effects as they relate to negative interpersonal process, a therapist’s response to his or her patient’s aggression might impact fidelity in different ways. For example, expressed aggression on the part of the patient may lead to deskilling of the therapist. In turn, this experience of being deskilled may lead the therapist to become frustrated, which may impact fidelity in more subtle ways (e.g., negative influence on therapist motivation and engagement). In line with this, it would be important to examine therapist factors (e.g., motivation, reactance) and their interaction with patient factors, such as personality variables, in explaining variability in adherence and competence.

Regarding measurement, an important implication of these results is that treating adherence and competence as static traits or assuming that meeting a predetermined criterion after a certain period of training will be sufficient for sustained fidelity is problematic. Related to this, given that much of the variability in these constructs is explained at the session level, aggregating across many sessions or a course of treatment will mask important information. Historically, psychotherapy adherence has been understood to represent the extent to which a therapist delivers a set of prespecified interventions or treatment strategies/procedures. As noted above, this has been particularly important for drawing conclusions in treatment research and transporting evidence-based treatments to routine practice settings. A somewhat different, yet not altogether incompatible, approach would be to place more emphasis on treatment principles (Castonguay & Beutler, 2006). From an internal validity perspective, this may be more difficult to capture, yet it may be more applicable to routine practice settings and better able to account for the complexity of this issue. For example, upon replication, one might derive a principle from the results of this study, which could aid particular therapists in conducting in-session exposures with interpersonally aggressive patients. Modifying one’s approach in this situation could be “contextually sensitive” and consistent with the principles of the treatment, yet might represent a departure from specific items found on traditional adherence checklists.

Despite these important implications, several study limitations should be noted. For example, given the rating system used, we were unable to quantitatively examine the precise nature of the protocol deviations. Although this limitation is not unique to this study, the lack of an active psychotherapy comparison condition resulted in less attention to the use of specific proscribed interventions when rating adherence/competence. Furthermore, the choice to rate randomly selected sessions has limitations. Although previous research has demonstrated significant heterogeneity in the methods used to sample and rate these constructs (Webb et al., 2010) and it is likely that certain approaches may fit better with specific studies and/or treatments, a con of this approach is that sampling imbalances may result. Additionally, we chose to focus on symptom severity at the session level because it represented a more proximal outcome indicator, signifying a novel approach to this question. Despite more recent research demonstrating significant relationships between specific technical factors and session-level outcomes, it is possible that too little change may have occurred on a session-to-session basis, or change may have been related to processes that took place across multiple sessions. Finally, this study involved a relatively homogenous sample, which may limit generalizability.

In the absence of clear links with outcome, the field continues to place importance on the constructs of adherence and competence, a reality that is underscored by the recent NIMH initiative. Adherence and competence are complex constructs that are impacted by interacting sources of influence (e.g., patient interpersonal characteristics). Given evidence of variability in adherence and competence within and between therapists, as well as over time, we can no longer assume that training a therapist to a criterion level is sufficient to result in sustained fidelity across a heterogeneous group of patients and over time. Regardless of the operationalization and measure, we advocate continued attention to mutual influence and potentially important individual differences between therapists and patients that may impact therapist behavior, treatment process, and outcome. In addition, it will be important to assess fidelity on an ongoing basis as well as to examine a variety of outcome markers. Even in the absence of large effects on outcome, such information has relevance for training, implementation, and sustainability efforts. In line with mutual influence and rethinking how we capture these constructs and processes, more intensive process research may be necessary in order to more fully understand the determinants and impacts of treatment fidelity. A complementary program of quantitative and qualitative process research may help to increase our understanding of the role of responsiveness in treatment adherence and competence with particular patients (e.g., when and why a therapist “goes off track” with a given patient), including the immediate and direct impact of patient characteristics on therapist behavior and decision making.