Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Feb 1.
Published in final edited form as: Behav Res Ther. 2015 Dec 10;77:52–61. doi: 10.1016/j.brat.2015.12.001

Acceptance and Commitment Therapy Modules: Differential Impact on Treatment Processes and Outcomes

Jennifer L Villatte 1, Roger Vilardaga 2, Matthieu Villatte 3, Jennifer C Plumb Vilardaga 4, David C Atkins 5, Steven C Hayes 6
PMCID: PMC4752868  NIHMSID: NIHMS746912  PMID: 26716932

Abstract

A modular, transdiagnostic approach to treatment design and implementation may increase the public health impact of evidence-based psychosocial interventions. Such an approach relies on algorithms for selecting and implementing treatment components intended to have a specific therapeutic effect, yet there is little evidence for how components function independent of their treatment packages when employed in clinical service settings. This study aimed to demonstrate the specificity of treatment effects for two components of Acceptance and Commitment Therapy (ACT), a promising candidate for modularization. A randomized, nonconcurrent, multiple-baseline across participants design was used to examine component effects on treatment processes and outcomes in 15 adults seeking mental health treatment. The ACT OPEN module targeted acceptance and cognitive defusion; the ACT ENGAGED module targeted values-based activation and persistence. According to Tau- U analyses, both modules produced significant improvements in psychiatric symptoms, quality of life, and targeted therapeutic processes. ACT ENGAGED demonstrated greater improvements in quality of life and values-based activation. ACT OPEN showed greater improvements in symptom severity, acceptance, and defusion. Both modules improved awareness and non-reactivity, which were mutually targeted, though using distinct intervention procedures. Both interventions demonstrated high treatment acceptability, completion, and patient satisfaction. Treatment effects were maintained at 3-month follow up. ACT components should be considered for inclusion in a modular approach to implementing evidence-based psychosocial interventions for adults.

Keywords: Acceptance and commitment therapy, component analysis, modular design, single case experimental design, acceptance, values, mindfulness, therapy process

The public health impact of evidence-based psychosocial interventions (EBPI) remains relatively low despite a proliferation of efficacious treatments for a wide range of behavioral and mental health problems (McHugh & Barlow, 2012; Wang et al., 2005). This science-practice gap may reflect a failure of the dominant intervention research paradigm to adequately address factors that influence the implementation of EBPI in usual care (Fairburn & Wilson, 2013; Kazdin & Blase, 2011; Rotheram-Borus, Swendeman, & Chorpita, 2012). Therapists report that they value the science behind EBPI, but are concerned that standardized manuals do not meet the needs of real-world clients and practice settings (Addis & Krasnow, 2000; Borntrager, Chorpita, Higa-McMillan, & Weisz, 2009; Nelson & Steele, 2007).

One promising approach to streamlining the translation of behavioral science to service is modularized treatment, which preserves the benefits of standardization inherent in manualized protocols, while allowing personalization through the use of algorithms for selecting treatment components. A recent randomized effectiveness trial for depression, anxiety, and conduct disorders in youth provides a compelling case example (Weisz et al., 2012). Modular treatment outperformed both standardized manual-based treatments and usual care in rate of clinical improvement and number of diagnoses at post-treatment (Chorpita et al., 2013), as well as number of service settings utilized one-year after treatment was initiated (Park et al., 2015). Modularization may further increase EBPI impact through improved therapist-mediated implementation outcomes. For example, therapists trained in a modular approach, versus a standard sequential manual, showed more favorable attitudes toward EBPI, a predictor of EBPI adoption (Borntrager et al., 2009). Additionally, therapists perceived modular treatments as more effective than usual care and more responsive than standard EBPI, contributing to significantly greater therapist satisfaction with modular treatment – an effect that grew as therapists gained more experience with modular treatment cases (Chorpita et al., 2015).

Acceptance and Commitment Therapy (ACT; (Hayes, Strosahl, & Wilson, 1999, 2012) is a promising candidate for modularization because it is based on a transdiagnostic model that guides case formulation and selection of therapy tasks from a set of complementary treatment components, affording personalized treatment that is grounded in theory and evidence. ACT interventions are defined by their application of this psychological flexibility model, which specifies a set of modifiable processes involved in the development, maintenance, and amelioration of a broad range of problems in living (Hayes, Levin, Plumb-Vilardaga, Villatte, & Pistorello, 2013; Kashdan & Rottenberg, 2010). As opposed to transdiagnostic approaches that allow for individualized targeting of multiple disorders within a unified treatment protocol, ACT specifies a set of clinical competencies that are applied based on a functional assessment of psychological flexibility, regardless of diagnoses (Luoma, Hayes, & Walser, 2007). ACT treatment effects are partially or fully mediated by changes in these psychological processes (Hayes, Villatte, Levin, & Hildebrandt, 2011) and a recent meta-analysis of ACT component interventions reported medium to large effects on targeted outcomes for treatment procedures suggested by the psychological flexibility model (Levin, Hildebrandt, Lillis, & Hayes, 2012). Similarly, a recent process analysis showed that each 1-unit increase in smoking counselors’ use of procedures targeting certain ACT processes resulted in a 42–52% lower odds of smoking at subsequent counseling sessions (Vilardaga, Heffner, Mercer, & Bricker, 2014). Finally, many elements of the psychological flexibility model are shared by modern contextual therapies (Hayes et al., 2011) and traditional cognitive and behavioral therapies (Arch & Craske, 2008; Hofmann & Asmundson, 2008), which could facilitate the adoption and integration of these components in a modular treatment approach.

The fact that a component is shared by multiple EBPIs, however, is not sufficient to guide clinical decisions; an effective modular treatment depends on algorithms for selecting which components to implement in which situation (Chorpita, Daleiden, & Weisz, 2005a). This requires evidence of how component procedures impact therapy processes and outcomes, and how components function when removed from the treatment protocols tested in efficacy trials (Hayes et al., 2013; Rosen & Davison, 2003). Dismantling studies provide one method of acquiring this knowledge, but their feasibility is limited by the very large samples required to compare multiple components. Single case experimental designs (SCED) provide a pragmatic alternative that, when well-designed and executed, rival the scientific rigor of randomized controlled trials while requiring far fewer participants (Barlow, Nock, & Hersen, 2008; Smith, 2012; Vilardaga, 2014). Further, SCED have been used effectively in modular treatment development and evaluation (Chorpita, Taylor, Francis, Moffitt, & Austin, 2004), in part because many of these designs are analogous to clinical decision-making in a modular treatment approach.

The purpose of the current study was to examine the functional relationships between ACT intervention components, processes, and outcomes to inform the development of a modular, transdiagnostic treatment for adults. A randomized, nonconcurrent, multiple-baseline across participants design (N = 15) was used to examine the specificity of treatment effects for two ACT component modules; one targeting openness to thoughts, feelings, and sensations and the other emphasizing engagement in meaningful actions. These modules were examined in a sample of adults seeking treatment for depression and anxiety disorders. Visual and statistical analyses were employed to compare module effects on process and outcome measures across baseline, intervention, and follow-up phases. It was hypothesized that both interventions would produce improvements in psychiatric symptoms and quality of life, as well as in mutually targeted psychological processes. Group differences were expected in processes that were uniquely targeted by only one intervention module. Results of this proof-of-concept study will inform the development and evaluation of actuarial guidelines for selecting and implementing ACT components in a modular treatment design.

Method

Participants

Since ACT interventions target core processes that cross-cut psychiatric diagnoses, study inclusion was based on clinically significant psychological distress rather than diagnostic criteria. Participants were required to meet clinical case status (general severity index T score ≥ 63) on the Brief Symptom Inventory (Derogatis, 1993) and be 18 years or older. Individuals with active psychotic symptoms and those who could not read assessment measures written in English were excluded from study participation.

Participants were recruited through announcements in a community newspaper in northern-Nevada and 63 people were assessed by phone for study eligibility. Eighteen people met inclusion criteria and were invited to meet with an assessor for a two-hour clinical interview. The assessor confirmed eligibility, administered the Structured Clinical Interview Disorder for DSM-IV-TR Axis I Disorders (First, Spitzer, Gibbon, & Williams, 2002) for sample description purposes, and obtained demographic information and consent for participation. Three participants dropped out after index assessment, but before random assignment, due to scheduling constraints. No participants refused participation after random assignment and all randomized participants (N = 15) were included in the main analyses.

Table 1 details participant demographics and diagnostic profiles at baseline. At time of enrollment, 67% were currently taking psychiatric medications. Of those 10 participants, 100% took one SSRI antidepressant, 13.3% took one benzodiazepine, and 6.7% took a stimulant. All participants denied medication changes during the study period. There were no statistically significant differences between treatment conditions on any demographic or diagnostic variables.

Table 1.

Sample characteristics at baseline

Variable Full Sample (N = 15) ACT OPEN (n = 7) ACT ENGAGED (n = 8)
Age, mean (SD) 43 (16) 38 (13) 47 (19)
Female 60% 57% 43%
Race and Ethnicity
 White 87% 87% 86%
 Black 7% 0% 14%
 American Indian 7% 14% 0%
 Hispanic or Latino 13% 14% 13%
Number of current diagnosesa, mean (SD) 2.6 (1.2) 2.7 (1.4) 2.5 (1.1)
Number of lifetime diagnosesa, mean (SD) 3.7 (1.4) 3.6 (1.5) 3.4 (1.5)
Current diagnosesa
 Major Depressive Disorder 60% 57% 63%
 Generalized Anxiety Disorder 53% 57% 50%
 Panic Disorder 33% 43% 23%
 Social Phobia 33% 29% 38%
 Substance Abuse or Dependence 33% 43% 25%
 Post-Traumatic Stress Disorder 27% 29% 25%
 Dysthymia 7% 0% 13%
 Obsessive Compulsive Disorder 7% 0% 13%
 Specific Phobia 7% 14% 0%
Open in a new tab

Note: SD = standard deviation,

a

Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition

Design and Treatment Assignment

The study was designed to evaluate the specificity of ACT component effects on therapy processes and outcomes. A randomized, nonconcurrent, multiple-baseline across participants design was employed to ensure timely treatment delivery while minimizing threats to internal validity.

A randomized block design was used to ensure roughly equivalent numbers of participants per baseline length, therapist, and treatment module. Following enrollment, participants were randomly allocated to an assessment-only baseline of 3, 4, or 5 weeks in fixed blocks of 3 using a computer random number generator. During baseline, participants received email reminders to complete weekly assessment packets and mail them to the assessor. Upon completion of baseline assessment, participants were allocated to therapist and intervention module in randomly varied blocks of 3–5 generated by a computer random number generator. Weekly assessments continued during the 8-week intervention phase following each treatment session. To increase study retention and reduce missing data, follow-up assessments were scheduled every four weeks for a total of three months. Participants received email reminders to complete monthly assessment packets and mail them to the assessor. In total, participants completed outcome and process measures at 14–16 time points, spanning baseline-, intervention-, and follow-up phases.

Interventions

ACT OPEN module

The ACT OPEN module consisted primarily of procedures targeting the acceptance and cognitive defusion processes of the psychological flexibility model, with the aim of reducing harmful responses to thoughts, feelings, and sensations. Early sessions explored clients’ conceptualizations of the presenting problem and therapists used metaphors and experiential exercises to orient clients to the short and long-term consequences of strategies they had employed to solve these problems. Throughout the module, therapists used language that fostered an open and curious stance toward psychological experiences and practiced non-evaluative awareness of thoughts, feelings, and sensations. Therapists introduced defusion strategies intended to evoke new responses to troublesome cognitions (e.g., thoughts are observed as if they are leaves floating on a stream). Defusion strategies were also employed to reduce the influence of self-concepts and beliefs that limit response flexibility.

ACT ENGAGED module

The ACT ENGAGED module consisted of procedures targeting the values and committed action processes of the psychological flexibility model, with the aim of increasing motivation and reinforcement for meaningful behaviors. Initial sessions explored personal values in order to instill a sense of purpose and direction. Therapists helped participants formulate their values as sources of satisfaction intrinsic to their own behavior, rather than as outcomes (e.g., achievement, praise) or in terms of negative reinforcement (e.g., escape from pain). Metaphors tailored to the client’s experience, such as that of a journey where purpose and meaning are found in each step and not only upon reaching one’s final destination, were used throughout the module to facilitate broadening and building behavioral repertoires and tracking contingencies that support valued action. Therapists used experiential exercises and language that oriented clients to their experience as a guide to effective action. For example, the metaphor of being guided by a compass versus a route on a roadmap oriented participants to the flexibility required to overcome obstacles and maintain a sense of direction when plans don’t proceed as expected. Therapists also employed behavioral commitment strategies to promote generalization and maintenance of valued action.

Module Overlap

ACT’s psychological flexibility model specifies two additional change processes (i.e., contact with the present moment and a sense of self as the context or container of one’s experiences). These processes manifest themselves as a flexible self-awareness that allows one to experience distress without threat of psychological harm or annihilation, and to act intentionally, rather than reactively. Acting with awareness and intention are core targets in all ACT interventions and are considered essential to the treatment’s experiential approach to shaping in-session behavior in order to promote maintenance and generalization of treatment gains (Hayes et al., 2011). That is, a treatment that does not include procedures to facilitate fluid self-awareness would not be ACT-adherent. Accordingly, these two processes were engaged, though not emphasized, in both modules. However, each module targeted these processes using separate procedures that were completely distinct from the alternate treatment condition. For example, the ACT OPEN module employed meditation exercises designed to observe ongoing thoughts, feelings, and sensations without reacting to urges to change them. The ACT ENGAGED module did not include formal mindfulness exercises, but did train participants to be aware of their actions and the contexts in which they occur, so they could choose effective responses based on their personal values rather than react habitually. In summary, both modules targeted self-awareness processes, but there was zero overlap in therapy tasks or techniques.

Therapists, Assessors, and Setting

Assessments and treatment sessions occurred in video-equipped therapy rooms in a mental health clinic at the University of Nevada, Reno. The clinical assessor and therapists were doctoral students in clinical psychology with at least 3 years of training in ACT and directly supervised by a developer of the treatment (SCH). Each of the 4 therapists conducted each module with at least one participant and treated a minimum of 3 participants in the study.

Measures

Intervention Fidelity

Fidelity was assessed using the 16-item ACT Integrity Coding System (Plumb & Vilardaga, 2010). Adherence to the module protocol, therapist competence, frequency and depth of coverage of ACT processes, and use of ACT-inconsistent strategies (e.g., cognitive or emotional suppression) were rated on a behaviorally anchored 5-point Likert-type scale. Twenty percent of each therapist’s sessions in each condition were rated for fidelity by two bachelors-level coders who completed 40 hours of training with a peer-reviewed ACT trainer. Coders achieved a minimum of 85% reliability with the expert trainer and each other. Calibration meetings were conducted weekly during the rating process to minimize rater drift. Interrater reliability was calculated using the intraclass correlation coefficient (Shrout & Fleiss, 1979) and interpreted using Cicchetti (1994) standards. ICCs indicated excellent agreement between the two raters in the ACT OPEN (ICC = .94, 95% CI = .90, .96) and ACT ENGAGED (ICC = .96, 95% CI = .95, .98) modules.

Treatment Acceptance

Treatment acceptance was assessed using the short form of the Treatment Evaluation Inventory (Kelley, Heffer, Gresham, & Elliott, 1989), with two questions omitted that applied only to family interventions. An independent assessor administered the measure at the end of each treatment session. Participants were informed that their responses would be kept confidential from therapists. In the current trial α = .96.

Treatment Outcome

Psychological symptom severity

The Brief Symptom Inventory (Derogatis, 1993) is a 53-item self-report inventory of psychological symptoms across nine domains and three global indices of distress. The global severity index of the BSI was used to measure psychological symptom severity, intensity, and number. In the current study, α = .85.

Quality of life

The 26-item version of the World Health Organization Quality of Life (Skevington, Lotfy, & O’Connell, 2004) was used to assess quality of life related to physical and psychological health, social relationships, and living environment. Given the slower nature of changes in these domains, the WHOQOL was administered on only six occasions: at index assessment, immediately prior to intervention phase, immediately following intervention phase, and at one-, two-, and three-month follow-up. In the current study, α = .89.

Specific Change Processes

Cognitive Defusion

The Automatic Thoughts Questionnaire-Believability Scale (Zettle, Rains, & Hayes, 2011) measures the strength of one’s belief in negative automatic thoughts and has been shown to mediate ACT outcomes. This process is explicitly targeted in the ACT OPEN module, but not in ACT ENGAGED. In the current study α = .89.

Experiential Acceptance

The Nonjudging of Inner Experience subscale of the Five Factor Mindfulness Questionnaire (Baer, Smith, Hopkins, Krietemeyer, & Toney, 2006) was used to assess acceptance of thoughts, feelings, and sensations. This process is explicitly targeted in the ACT OPEN module, but not in ACT ENGAGED. In the current study α = .87.

Values-Based Action

The Values Bullseye (Lundgren, Luoma, Dahl, Strosahl, & Melin, 2012) is a visual analogue scale that measures congruence between behaviors and personally chosen values. This scale has demonstrated good test-retest reliability and criterion validity and has been a mediator of ACT outcomes. This process is explicitly targeted in the ACT ENGAGED module, but not in ACT OPEN. In the current study, α = .95.

Common Change Processes

Awareness

The Acting with Awareness subscale of the FFMQ (Baer et al., 2006) was used to assess attention to one’s behaviors in the moment versus acting on “automatic pilot”. Both modules targeted this process, but using entirely different procedures with no overlapping therapy tasks between conditions. For example, the ACT OPEN module included guided exercises to practice fluid awareness of thoughts, feelings, sensations, and behaviors, while the ACT ENGAGED module practiced behaving intentionally and tracking the context and consequences of one’s actions. In the current study, α = .82.

Nonreactivity

The Nonreactivity to Inner Experiences subscale of the FFMQ (Baer et al., 2006) measures the tendency to allow thoughts, feelings, sensations, and urges to come and go without attachment or impulsive response. Both modules targeted nonreactivity, but using entirely different procedures with no overlapping therapy tasks between conditions. For example, the ACT OPEN module included the practice of observing thoughts and feelings without acting to change or escape them, while the ACT ENGAGED module practiced using one’s values, as opposed to positive or negative psychological experiences, as a guide for selecting meaningful responses and engaging in purposeful activities. In the current study, α = .79, which is consistent with published reports of internal consistency in the FFMQ.

Analytic Strategy

A bottom-up analytic approach (Onghena & Edgington, 2005; Parker & Vannest, 2012) was employed beginning with visual analysis to identify patterns within and across study phases, then proceeding to statistical analysis of individual phase contrasts within participants and across treatment conditions. Nonparametric Tau-U analyses were conducted to evaluate individual and aggregated effect sizes. This approach is free from assumptions of distribution and linearity, robust to serial dependence inherent in time-series designs, superior in power and precision over other SCED effect size indices, and allows for examination of changes in both mean levels and trend (Parker, Vannest, Davis, & Sauber, 2011).

For each participant, the percent of nonoverlap between all pairs of data was calculated to determine baseline trend, intervention trend, mean contrasts between baseline and intervention phases, and combined trend and mean contrasts. Individual phase and participant contrasts were combined and weighted to provide an omnibus index of effect size per condition and p values were calculated using effect sizes and standard errors. Tau is analogous to the Pearson R2 effect size, so it can be interpreted as the percent of the data variance accounted for by intervention effects.

Results

Treatment Fidelity

Treatment fidelity was highly rated for both the ACT OPEN (M = 4.53, SD = .05) and ACT ENGAGED (M = 4.72, SD = .04) modules. No sessions were coded as employing procedures inconsistent with ACT or targeting a process purposefully omitted from the treatment module (i.e., no contamination). There were no statistically significant differences in fidelity between therapists, conditions, or sessions.

Treatment Acceptance and Completion

Participants were presented with a description and rationale for each treatment module prior to randomization. No participant refused treatment after being informed about the intervention to which they were randomly assigned. Both ACT modules demonstrated excellent retention, with all participants completing all 8 treatment sessions. Both modules were viewed as acceptable, with no session rated below 3 and an average score of 4.17 on the 5-point scale of the TEI. All 7 participants assigned to the ACT OPEN module rated the treatment with an average score of 4 or better, with 50% of sessions rated with the highest possible score. Seventy-five percent of the 8 participants assigned to the ACT ENGAGED module rated the treatment with an average score of 4 or better, with 40% of sessions given the maximum rating. These differences were not statistically significant, t(13) = 1.73, p = .11.

Treatment Outcomes and Processes

Visual Analysis to Inform Statistical Approach

The initial correlation matrix of study variables is reported in Table 2. Figure 1 displays means and 95% confidence intervals for each of the outcome and process measures at each assessment point, by treatment condition. Visual inspection of the data detected no significant concerns about baseline trends, but baseline control was employed in Tau-U in order to be conservative in interpreting aggregated SCED results.

Table 2.

Descriptive Statistics of Treatment Outcome and Process Variables

Variable Group Means (SD) Pearson Correlations at Baseline
ACT OPEN (n = 7) ACT ENGAGED (n = 8) Total Sample (N = 15)
Pre Post FU Pre Post FU b c d e f g
Symptom Severitya 2.02 (.19) 0.60 (.13) 0.63 (.12) 1.90 (.18) 0.95 (.21) 1.15 (.17) −.39* −.59* −.58* .04 −.39* −.25
Quality of Lifeb 40.6 (2.4) 56.7 (3.3) 55.7 (1.3) 39.1 (2.7) 62.3 (4.7) 62.4 (4.4) - .05 .06 .44* .65* .23
Cognitive Defusionc 94.6 (3.9) 43.1 (3.8) 42.0 (3.1) 90.1 (4.1) 95.9 (9.2) 94.4 (9.1) - .51* −.27 .34 .53*
Acceptanced 22.1 (2.2) 35.7 (1.1) 36.3 (1.7) 19.4 (1.6) 19.9 (2.3) 20.1 (2.1) - −.30 .33 .57**
Valued Actione 5.1 (.50) 67.0 (1.0) 6.9 (.75) 4.8 (.42) 10.3 (.78) 10.0 (.92) - .57* .10
Non-reactivityf 18.3 (1.4) 24.1 (1.1) 24.0 (1.5) 16.8 (.96) 25.0 (2.1) 21.3 (2.5) - .60*
Action Awarenessg 19.6 (1.3) 25.6 (.95) 26.0 (.95) 15.6 (1.1) 23.5 (2.2) 21.9 (1.5) -
Open in a new tab

Note. SD = standard deviation; FU = 3-month follow up;

a

Brief Symptom Inventory;

b

WHO Quality of Life;

c

Automatic Thoughts Questionnaire-Believability;

d

Five Factor Mindfulness Questionnaire-Nonjudge;

e

Values Bullseye ;

f

Five Factor Mindfulness Questionnaire- Nonreact;

g

Five Factor Mindfulness Questionnaire- Act Aware;

*

p (two-tailed) < .05;

**

p (two-tailed) < .01

Figure 1.

Figure 1

Open in a new tab

Means and 95% confidence intervals for outcome and process measures at each assessment point by treatment condition

Visual analyses were consistent with previous studies suggesting ACT intervention trends carry-over into follow-up assessment phases (Luoma, Kohlenberg, Hayes, & Fletcher, 2012). This pattern is ideal from a clinical perspective, demonstrating that the intervention leads to ongoing improvement even after treatment ends. Tau analyses confirmed no statistically significant differences in omnibus (i.e., combined level and trend) effects between intervention and follow-up phases. Hence, data from these phases were collapsed for Tau-U analysis; within-subject tests compared data prior to intervention (phase A) with data during and following intervention (phase B).

Visual analysis identified consistent data patterns (both within and between phases) with two exceptions that have implications for statistical analysis. First, improvements on the Values Bullseye, though large, did not occur until the fourth week in the intervention phase for all participants in the ACT ENGAGED module, which targeted this process explicitly. It has been suggested that inclusion of such a lag in statistical analysis would not fairly represent the amount of change due to the intervention (Parker & Vannest, 2012), so the first three weekly data points in the intervention phase were excluded from Tau-U analysis of the Values Bullseye.

The second aberrant pattern detected during visual inspection is that two participants receiving the ACT ENGAGED module displayed trends in opposite directions of their condition cohort (i.e., deterioration) on several measures. Exploratory analyses indicated that omission of the two outlying cases would not have affected the significance of any results; thus, they were retained in all final analyses. However, exploratory analyses of the nonresponders were conducted with the aim of identifying potential moderators to be explored in future studies. These results are reported in a separate section below.

Interpretation of Statistical Significance and Effect Sizes

Results of Tau-U analyses of within-subject treatment effects by treatment condition are reported in Table 3, while comparative effectiveness between the two intervention modules is reported in the text. Treatment effects are reported in terms of both level (i.e., percent of intervention and follow-up data showing improvement over baseline scores) and trend (i.e., percent of data showing improvement over previous intervention session). These effect sizes are transformed to a zero chance level and can be interpreted as small (0 – 31%), medium to large (32% – 84%), and very large (85% – 100%) effects (Parker & Vannest, 2009). An omnibus effect size combining level and trend improvements is also reported and can be interpreted as small (≈ .30), medium (≈ .50), and large (≈ .80) effects (Parker et al., 2011). Probability values less than p = .005 were rounded to p = .00.

Table 3.

Tau-U Analysis of Treatment Processes and Outcomes by Intervention Module

Treatment Outcomes ACT OPEN module ACT ENGAGED module
Symptom Severitya ES Var p ES Var p
Improvement over baseline 92% .14 .00 64% .13 .00
Session by session improvement 62% .09 .00 22% .08 .00
Omnibus effect size .71 .07 .00 .40 .07 .00
Quality of Lifeb
Improvement over baseline 100% .20 .00 100% .19 .00
Omnibus effect size .54 .14 .30 .66 .13 .47
Common Processes ES Var P ES Var p
Nonreactivityc
Improvement over baseline 92% .14 .00 80% .12 .00
Session by session improvement 30% .09 .00 35% .08 .00
Omnibus effect size .55 .08 .00 .49 .07 .00
Action Awarenessd
Improvement over baseline 84% .14 .00 84% .13 .00
Session by session improvement 46% .09 .00 57% .09 .00
Omnibus effect size .57 .07 .00 .65 .07 .00
Specific Processes ES Var p ES Var p
Cognitive Defusione
Improvement over baseline 100% .14 .00 3% .12 .83
Session by session improvement 67% .09 .00 12% .08 .14
Omnibus effect size .80 .08 .00 .09 .07 .17
Acceptancef
Improvement over baseline 100% .14 .00 2% .12 .90
Session by session improvement 59% .09 .00 10% .08 .25
Omnibus effect size .71 .07 .00 −.06 .07 .40
Valued Actiong
Improvement over baseline 49% .14 .00 96% .13 .00
Session by session improvement 4% .11 .00 55% .10 .00
Omnibus effect size .24 .09 .10 .73 .08 .00
Open in a new tab

Note. Improvement over baseline and session by session improvement can be interpreted as small effects (0 – 31%), medium to large effects (32% – 84%), and very large effects (85% – 100%); Omnibus effect size = combined level and trend improvements, with small effect ≈ .30, medium effect ≈ .50, large effect ≈ .80

Psychiatric symptom severity

Both ACT modules showed clinically (i.e., no longer met threshold for clinical caseness) and statistically (p < .001) significant improvement on the BSI, though their effect sizes differed. The ACT OPEN module had a large effect on symptom severity, while the ACT ENGAGED module showed only moderate improvements. When examining omnibus treatment effects between conditions, participants who received the ACT OPEN module showed moderately greater reductions in symptom severity over those who received the ACT ENGAGED (Tau-U = .30, SE = .09, p = .00).

Quality of life

All participants reported large WHOQoL improvements over baseline and these gains were maintained across the three months of follow-up assessment. However, participants in the ACT ENGAGED condition showed moderately greater improvements over those in the ACT ENGAGED condition (Tau-U = .38, SE = .17, p = .02).

Cognitive defusion

All participants in the ACT OPEN condition, which exclusively targeted this process, made immediate and substantial session-by session improvements in cognitive defusion. Participants in the ACT ENGAGED condition showed no changes in cognitive defusion. As predicted, the effects of the ACT OPEN module on this therapy process were significantly greater than the ACT ENGAGED module (Tau–U = .89, SE = .11, p = .00).

Experiential Acceptance

All participants who received the ACT OPEN module showed substantial improvement in acceptance, with moderately large session-by-session gains and almost no deterioration during the 3-month follow-up period. The ACT ENGAGED module, which did not target this treatment process, showed no changes in acceptance over time. As predicted, the ACT OPEN condition showed much greater improvements over the ACT ENGAGED condition (Tau-U = .89, SE = .11, p = .00).

Values-based action

This process was explicitly targeted only by the ACT ENGAGED module, which produced large improvements over baseline scores and reliable session-by-session growth in values-based action. Participants who received the ACT OPEN module demonstrated small improvements at follow-up relative to baseline scores on the Values Bullseye, though these improvements were not reliably linked to treatment trends and the omnibus treatment effect was not significant. The difference between modules was significant, with ACT ENGAGED demonstrating superior treatment effects (Tau-U = .54, SE = .10, p = .00).

Awareness and Nonreactivity

Both modules targeted the core ACT treatment processes of acting with awareness and intention, though they used different procedures to do so. Participants in both conditions showed large and steady treatment gains on the FFMQ-Act Aware and FFMQ-Nonreact scales. Between-group analyses detected no significant differences by condition for awareness (Tau-U = .02, SE = .20, p = .83) or nonreactivity (Tau-U = .01, SE = .09, p = .93).

Exploratory Nonresponder Analyses

One male and one female within the ACT ENGAGED condition were identified as outliers compared to participants within and across treatment conditions. These two cases displayed no clinically significant improvements on any process or outcome measure. Compared to scores of other participants in the ACT ENGAGED condition, the two cases displayed 69% less improvement in quality of life (p = .04) and 68% less improvement in symptom severity (p = .00). These were the only cases in either condition to become more cognitively fused (Tau = .64, p = .00) and less accepting of thoughts and feelings (Tau = .56, p = .00), and to fail to show improvement in reactivity (Tau = .07, p = .71).

An analysis of participant characteristics at baseline revealed that the two nonresponsive cases were the only participants in the ACT ENGAGED condition to meet criteria for a current diagnosis of PTSD. These cases did not differ from participants in either condition at baseline on any demographic variable, number of past or current diagnoses, symptom severity, or any process measure. It is notable that the two participants in the ACT OPEN condition meeting criteria for a current PTSD diagnosis displayed patterns of improvement consistent with other participants within that treatment condition and showed no deterioration on process or outcome measures.

Discussion

The current study provides preliminary evidence supporting the inclusion of components of ACT’s psychological flexibility model in a modular transdiagnostic treatment that affords flexible use of techniques to target common core processes of therapeutic change. Both of the ACT component modules produced significant improvements in symptom severity and quality of life, but at different rates and magnitudes and using different techniques. The ACT OPEN module, which targeted acceptance and cognitive defusion processes, produced greater session-by-session improvement and larger overall reductions in symptom severity. In spite of its smaller effect on symptoms, the ACT ENGAGED module, which targeted values-based activation and behavioral commitment processes, produced greater quality of life improvements. This result is consistent with ACT’s psychological flexibility model, which makes a distinction between symptom reduction and well-being (Hayes et al., 2013).

Each ACT module produced sustained changes in the psychological processes that were uniquely and explicitly targeted by that component. Large improvements in acceptance and defusion were observed in the ACT OPEN condition, while these processes remained unchanged in the ACT ENGAGED condition. It is interesting to note that moderate symptom reduction and large quality of life improvements occurred even though judgment and believability of negative thoughts remained high in this condition. Conversely, the ACT ENGAGED module produced a 73% improvement in values-based behavior, which was the emphasis of this component intervention. Though the ACT OPEN component did not directly target values and commitment processes, it appears that some participants in this condition independently brought their behaviors into greater alignment with their personal values after learning to relate differently to their thoughts and feelings. Component interventions that produce these types of rapid, simultaneous improvements across multiple areas of functioning should be prioritized when considering common elements for inclusion in modular treatments.

Nonreactive awareness of one’s self and environment is a central feature of the psychological flexibility model and a common core element in all ACT interventions. Both ACT modules produced improvements in the awareness and nonreactivity facets of mindfulness, though by different means. The ACT ENGAGED module included procedures to help participants discover sources of inspiration and satisfaction in their own behavior and to intentionally choose responses with a purpose. Procedures in the ACT OPEN module helped participants maintain open awareness of their thoughts and feelings without attempting to alter their experience. Thus, different treatment techniques can be used to impact the same processes of change, allowing therapists the flexibility to adapt interventions for specific clients while maintaining fidelity to an evidence-based treatment model.

An effective personalized treatment relies on an understanding of patient characteristics that influence intervention processes and outcomes. Exploratory analyses of outlying cases revealed that participants in the ACT ENGAGED condition with a current diagnosis of PTSD showed no improvement or deterioration on several outcome and process measures. This pattern was inconsistent with all other study participants, including those in the ACT OPEN condition with a current PTSD diagnosis. This is insufficient evidence to draw conclusions about implementing these modules with PTSD patients, but the independent, additive, and order effects of acceptance and values interventions warrant further examination in this population.

It is possible that modules targeting acceptance and defusion may be necessary precursors to modules aimed at values-based behavioral activation for individuals with a current PTSD diagnosis. In a case study of ACT for treatment-resistant PTSD, Twohig (2009) found that values interventions were not tolerated until 6 sessions of acceptance, cognitive defusion, and self-as-context interventions (roughly equivalent to the current ACT OPEN module) were completed. Similarly, improvements in the FFMQ facets of nonjudgment and nonreactivity to inner experiences explained 19–24% of the variance in depression and PTSD outcomes in a residential CBT treatment for PTSD (Boden et al., 2012). In the current study, clients with PTSD in the ACT ENGAGED condition were the only study participants who did not show improvements on these measures. Future experimental studies should examine the impact of acceptance and values components and processes, in isolation and combination, in adults with PTSD.

Implications for modular treatment development, dissemination, and implementation

Therapists cannot be expected to learn each new treatment package supported by a randomized controlled trial, particularly when many EBPIs share core components and target similar psychological processes, making distinct training in complete treatment packages potentially redundant and unnecessary (Chorpita et al., 2011). A modular approach that provides training and guidelines for implementing evidence-based components, rather than session-by-session protocols organized by diagnosis, is a promising solution to known implementation barriers. The modular approach capitalizes on therapists’ tendency to be integrative when adapting efficacious treatments to real-world practice demands. Modular treatments are efficient to train, inherently flexible, and easy to update as new intervention data emerges. Modularity is also an ideal fit with current trends toward disruptive innovations in EBPIs, as it is patient-centered, responsive to idiographic variables that influence treatment engagement and response, and maintains a tight link to the evidence base while allowing therapists to select components that match their expertise and are feasible within their practice settings (Rotheram-Borus et al., 2012).

The contextual behavioral scientific approach that gave rise to ACT (Hayes et al., 2013) is well-suited to modular treatment design and evaluation because both rest on a pragmatic philosophy that lends itself to inductive and iterative methodologies for knowledge development and dissemination. Single case experimental designs are ideal for this kind of treatment distillation and matching because they allow for rapid knowledge generation based on critical comparisons at the level of clinical decision-making. Meta-analyses of between-group treatment effects, including additive and dismantling studies, are unlikely to yield actionable insights for EBPI implementers (e.g., Ahn & Wampold, 2001; Bell et al., 2013) due to mismatched levels of analysis that obscure potentially important treatment by context interactions (for an illustrative exception, see Weisz, Weiss, Alicke, & Klotz, 1987). The current study employed aggregated single case experiments to test functional relationships among procedures, processes, and outcomes in two transdiagnostic content modules (Chorpita, Daleiden, & Weisz, 2005b), which serve as the building blocks in modular treatment design. This methodology could also be employed in the development and evaluation of coordination modules, which outline the algorithms for deciding when to use which content modules and would constitute the next step in modular treatment development.

The current study does not test a modular treatment protocol, but it does test key properties of proposed treatment modules, defined as “self-contained functional units that connect with other units, but do not rely on those other units for their own stable operations” (Chorpita et al., 2005b, p. 142). The principle of proper functioning signifies that modules are defined by their function, not by categorical labels (e.g., ACT) or techniques (e.g., meditation), and must demonstrate reliable production of intended results in terms of therapeutic processes (e.g., emotional acceptance) and outcomes (e.g., symptom severity, quality of life). This study demonstrated the proper functioning of ACT OPEN and ACT ENGAGED modules in both specific and common treatment processes and outcomes. The principle of encapsulation refers to keeping all knowledge and competencies necessary for successful implementation of each module self-contained, allowing maximal flexibility in selecting and sequencing modules. Fidelity monitoring within the current study demonstrated that ACT modules can be functionally independent and distinct, adequately addressing a wide range of clinical content across 8 sessions with no contamination of therapy procedures across modules or use of procedures not specified by the module (e.g., non-adherence).

The current study suggests that treatment modules targeting components of ACT’s psychological flexibility model are feasible and effective in treating adults with mixed anxiety and depression. Both the ACT OPEN and ACT ENGAGED modules were acceptable to adults seeking mental health treatment, demonstrating 100% treatment completion rates and very high levels of patient satisfaction post-treatment and at 3-month follow-up. Both modules demonstrated broad symptom improvements and sustained increases in quality of life, though they were comprised of wholly different interventions and produced distinct changes in psychological processes. Components such as these afford therapists flexibility in developing a treatment that is both personalized and pragmatic, while maintaining its link to the evidence base.

Limitations and Recommendations for Future Research

Though the present study design provided a good balance between external validity and experimental control (e.g., randomization, replication across participants), critical comparisons should be replicated to see if current results hold across experimenters, treatment settings, therapists, and participants. While formal mediation analyses were beyond the scope of the current study, examination of time-lagged associations between process and outcome variables are possible within single case experiments (Hilliard, 1993; Tschacher & Ramseyer, 2009) and are recommended for developing modular treatment protocols that optimize exposure to key treatment mechanisms.

It will also be important to further assess how participant, therapist, and setting variables interact with modular treatment components. Future studies should examine module dependencies, including order and additive effects, and specific person X module X context interactions in order to develop testable clinical decision algorithms and coordination modules. Though group designs would require massive samples to power so many interaction effects, results from SCED, including the current study, can be aggregated to provide guidelines for personalized selection and evaluation of treatment components (Van den Noortgate & Onghena, 2007). While the majority of published meta-analyses compare group means and yield only a single estimate of an intervention effect (Alemayehu, 2011), meta-analyses of the rich individual-level data from single case experiments can provide detailed knowledge of individual patient, therapist, and context variables that influence treatment effectiveness (Cooper & Patall, 2009; Iwakabe & Gazzola, 2009; Thompson & Higgins, 2005). Such a progressive accumulation of “big data” may finally allow us to answer Gordon Paul’s (Paul, 1969) ultimate clinical question of “what treatment, by whom, is most effective for this individual with that specific problem, under which set of circumstances, and how does it come about?”

Highlights.

  • ACT component modules were examined for specificity of treatment effects in adults

  • Both modules improved symptom severity and life quality, but effect sizes differed

  • The OPEN module had larger effects on symptoms, cognitive defusion, and acceptance

  • The ENGAGED module had larger effects on life quality and values-based activation

  • Both modules improved awareness and nonreactivity to thoughts, feelings, sensations

Acknowledgments

First author J.L.V. was assisted in the preparation of this manuscript by grant number F31MH096341 from the National Institute of Mental Health. First author J.L.V. and second author R.V. were assisted in preparation of this manuscript by grant number T32MH096341. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health, who had no involvement in the design or execution of the study.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Jennifer L. Villatte, Email: villatte@uw.edu, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, USA

Roger Vilardaga, Email: vilardag@uw.edu, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, USA

Matthieu Villatte, Email: matt@ebpi.org, Evidence Based Practice Institute, Seattle, WA, USA

Jennifer C. Plumb Vilardaga, Email: jennifer.plumbvilardaga@va.gov, Veterans Affairs Puget Sound Health Care System, Seattle Division, Seattle, WA, USA

David C. Atkins, Email: datkins@uw.edu, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, USA

Steven C. Hayes, Email: hayes@unr.edu, Department of Psychology, University of Nevada, Reno, USA

References

  1. Addis ME, Krasnow AD. A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology. 2000;68(2):331–339. doi: 10.1037//0022-006x.68.2.331. http://doi.org/10.1037//0022-006X.68.2.331. [DOI] [PubMed] [Google Scholar]
  2. Ahn H, Wampold BE. Where oh where are the specific ingredients? A metaanalysis of component studies in counseling and psychotherapy. Journal of Counseling Psychology. 2001;48(3):251–257. http://doi.org/10.1037/0022-0167.48.3.251. [Google Scholar]
  3. Alemayehu D. Perspectives on pooled data analysis: the case for an integrated approach. Journal of Data Science. 2011;9:389–397. [Google Scholar]
  4. Arch JJ, Craske MG. Acceptance and commitment therapy and cognitive behavioral therapy for anxiety disorders: Different treatments, similar mechanisms? Clinical Psychology: Science and Practice. 2008;15(4):263–279. http://doi.org/10.1111/j.1468-2850.2008.00137.x. [Google Scholar]
  5. Baer RA, Smith GT, Hopkins J, Krietemeyer J, Toney L. Using Self-Report Assessment Methods to Explore Facets of Mindfulness. Assessment. 2006;13(1):27–45. doi: 10.1177/1073191105283504. [DOI] [PubMed] [Google Scholar]
  6. Barlow DH, Nock MK, Hersen M. Single case experimental designs: Strategies for studying behavior change. 3. Boston: Allyn & Bacon; 2008. [Google Scholar]
  7. Bell EC, Marcus DK, Goodlad JK. Are the parts as good as the whole? A meta-analysis of component treatment studies. Journal of Consulting and Clinical Psychology. 2013;81(4):722–736. doi: 10.1037/a0033004. http://doi.org/10.1037/a0033004. [DOI] [PubMed] [Google Scholar]
  8. Boden MT, Bernstein A, Walser RD, Bui L, Alvarez J, Bonn-Miller MO. Changes in facets of mindfulness and posttraumatic stress disorder treatment outcome. Psychiatry Research. 2012;200(2–3):609–613. doi: 10.1016/j.psychres.2012.07.011. http://doi.org/10.1016/j.psychres.2012.07.011. [DOI] [PubMed] [Google Scholar]
  9. Borntrager C, Chorpita B, Higa-McMillan C, Weisz J. Provider attitudes toward evidence-based practices: are the concerns with the evidence or with the manuals? Psychiatric Services. 2009;60(5):677–681. doi: 10.1176/ps.2009.60.5.677. [DOI] [PubMed] [Google Scholar]
  10. Chorpita BF, Daleiden EL, Weisz JR. Identifying and Selecting the Common Elements of Evidence Based Interventions: A Distillation and Matching Model. Mental Health Services Research. 2005a;7(1):5–20. doi: 10.1007/s11020-005-1962-6. http://doi.org/10.1007/s11020-005-1962-6. [DOI] [PubMed] [Google Scholar]
  11. Chorpita BF, Daleiden EL, Weisz JR. Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology. 2005b;11(3):141–156. http://doi.org/10.1016/j.appsy.2005.05.002. [Google Scholar]
  12. Chorpita BF, Park A, Tsai K, Korathu-Larson P, Higa-McMillan CK, Nakamura BJ, Krull J. Balancing effectiveness with responsiveness: Therapist satisfaction across different treatment designs in the Child STEPs randomized effectiveness trial. Journal of Consulting and Clinical Psychology. 2015;83(4):709–718. doi: 10.1037/a0039301. http://doi.org/10.1037/a0039301. [DOI] [PubMed] [Google Scholar]
  13. Chorpita BF, Taylor AA, Francis SE, Moffitt C, Austin AA. Efficacy of Modular Cognitive Behavior Therapy for Childhood Anxiety Disorders. Behavior Therapy. 2004;35(2):263–287. http://doi.org/10.1016/S0005-7894(04)80039-X. [Google Scholar]
  14. Chorpita BF, Weisz JR, Daleiden EL, Schoenwald SK, Palinkas LA, Miranda J, Gibbons RD. Long-term outcomes for the Child STEPs randomized effectiveness trial: A comparison of modular and standard treatment designs with usual care. Journal of Consulting and Clinical Psychology. 2013;81(6):999–1009. doi: 10.1037/a0034200. http://doi.org/10.1037/a0034200. [DOI] [PubMed] [Google Scholar]
  15. Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment. 1994;6(4):284. [Google Scholar]
  16. Cooper H, Patall EA. The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods. 2009;14(2):165–176. doi: 10.1037/a0015565. http://doi.org/10.1037/a0015565. [DOI] [PubMed] [Google Scholar]
  17. Derogatis LR. BSI: Administration, scoring & procedures manual for the Brief Symptom Inventory. 3. Minneapolis, MN: National Computer Systems; 1993. [Google Scholar]
  18. Fairburn CG, Wilson GT. The dissemination and implementation of psychological treatments: Problems and solutions. International Journal of Eating Disorders. 2013;46(5):516–521. doi: 10.1002/eat.22110. http://doi.org/10.1002/eat.22110. [DOI] [PubMed] [Google Scholar]
  19. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition. (SCID-I/P) New York: Biometrics Research, New York State Psychiatric Institute; 2002. [Google Scholar]
  20. Hayes SC, Levin ME, Plumb-Vilardaga J, Villatte JL, Pistorello J. Acceptance and commitment therapy and contextual behavioral science: Examining the progress of a distinctive model of behavioral and cognitive therapy. Behavior Therapy. 2013;44(2):180–198. doi: 10.1016/j.beth.2009.08.002. http://doi.org/10.1016/j.beth.2009.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: An experiential approach to behavior change. New York, NY US: Guilford Press; 1999. [Google Scholar]
  22. Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: The process and practice of mindful change. 2. New York, NY US: Guilford Press; 2012. [Google Scholar]
  23. Hayes SC, Villatte M, Levin M, Hildebrandt M. Open, Aware, and Active: Contextual Approaches as an Emerging Trend in the Behavioral and Cognitive Therapies. Annual Review of Clinical Psychology. 2011;7(1):141–168. doi: 10.1146/annurev-clinpsy-032210-104449. http://doi.org/10.1146/annurev-clinpsy-032210-104449. [DOI] [PubMed] [Google Scholar]
  24. Hilliard RB. Single-case methodology in psychotherapy process and outcome research. Journal of Consulting and Clinical Psychology. 1993;61(3):373. doi: 10.1037//0022-006x.61.3.373. [DOI] [PubMed] [Google Scholar]
  25. Hofmann SG, Asmundson GJG. Acceptance and mindfulness-based therapy: New wave or old hat? Clinical Psychology Review. 2008;28(1):1–16. doi: 10.1016/j.cpr.2007.09.003. http://doi.org/10.1016/j.cpr.2007.09.003. [DOI] [PubMed] [Google Scholar]
  26. Iwakabe S, Gazzola N. From single-case studies to practice-based knowledge: Aggregating and synthesizing case studies. Psychotherapy Research. 2009;19(4–5):601–611. doi: 10.1080/10503300802688494. http://doi.org/10.1080/10503300802688494. [DOI] [PubMed] [Google Scholar]
  27. Kashdan TB, Rottenberg J. Psychological flexibility as a fundamental aspect of health. Clinical Psychology Review. 2010;30(7):865–878. doi: 10.1016/j.cpr.2010.03.001. http://doi.org/10.1016/j.cpr.2010.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kazdin AE, Blase SL. Rebooting Psychotherapy Research and Practice to Reduce the Burden of Mental Illness. Perspectives on Psychological Science. 2011;6(1):21–37. doi: 10.1177/1745691610393527. http://doi.org/10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
  29. Kelley ML, Heffer RW, Gresham FM, Elliott SN. Development of a modified Treatment Evaluation Inventory. Journal of Psychopathology and Behavioral Assessment. 1989;11(3):235–247. [Google Scholar]
  30. Levin ME, Hildebrandt MJ, Lillis J, Hayes SC. The impact of treatment components suggested by the psychological flexibility model: A meta-analysis of laboratory-based component studies. Behavior Therapy. 2012;43(4):741–756. doi: 10.1016/j.beth.2012.05.003. [DOI] [PubMed] [Google Scholar]
  31. Lundgren T, Luoma JB, Dahl J, Strosahl K, Melin L. The Bull’s-Eye Values Survey: A psychometric evaluation. Cognitive and Behavioral Practice. 2012;19(4):518–526. [Google Scholar]
  32. Luoma JB, Hayes SC, Walser RD. Learning ACT: An Acceptance and Commitment Therapy Skills-Training Manual for Therapists. Oakland, CA: New Harbinger Publications; 2007. Pap/DVD edition. [Google Scholar]
  33. Luoma JB, Kohlenberg BS, Hayes SC, Fletcher L. Slow and steady wins the race: A randomized clinical trial of acceptance and commitment therapy targeting shame in substance use disorders. Journal of Consulting and Clinical Psychology. 2012;80(1):43–53. doi: 10.1037/a0026070. http://doi.org/10.1037/a0026070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McHugh RK, Barlow DH. The reach of evidence-based psychological interventions. In: McHugh RK, Barlow DH, editors. Dissemination and implementation of evidence-based psychological interventions. New York, NY US: Oxford University Press; 2012. pp. 3–15. [Google Scholar]
  35. Nelson TD, Steele RG. Predictors of Practitioner Self-reported Use of Evidence-Based Practices: Practitioner Training, Clinical Setting, and Attitudes Toward Research. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34(4):319–330. doi: 10.1007/s10488-006-0111-x. http://doi.org/10.1007/s10488-006-0111-x. [DOI] [PubMed] [Google Scholar]
  36. Onghena P, Edgington ES. Customization of pain treatments: Single-case design and analysis. The Clinical Journal of Pain. 2005;21(1):56–68. doi: 10.1097/00002508-200501000-00007. [DOI] [PubMed] [Google Scholar]
  37. Park AL, Tsai KH, Guan K, Reding MEJ, Chorpita BF, Weisz JR. Service use findings from the child steps effectiveness trial: Additional support for modular designs. Administration and Policy in Mental Health and Mental Health Services Research. 2015 doi: 10.1007/s10488-015-0625-1. http://doi.org/10.1007/s10488-015-0625-1. [DOI] [PubMed]
  38. Parker RI, Vannest K. An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy. 2009;40(4):357–367. doi: 10.1016/j.beth.2008.10.006. [DOI] [PubMed] [Google Scholar]
  39. Parker RI, Vannest KJ. Bottom-Up Analysis of Single-Case Research Designs. Journal of Behavioral Education. 2012;21(3):254–265. http://doi.org/10.1007/s10864-012-9153-1. [Google Scholar]
  40. Parker RI, Vannest KJ, Davis JL, Sauber SB. Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy. 2011;42(2):284–299. doi: 10.1016/j.beth.2010.08.006. [DOI] [PubMed] [Google Scholar]
  41. Paul GL. Behavior therapy: Appraisal and status. New York: McGraw-Hill; 1969. Behavior modification research: Design and tactics; pp. 29–62. [Google Scholar]
  42. Plumb JC, Vilardaga R. Assessing Treatment Integrity in Acceptance and Commitment Therapy: Strategies and Suggestions. International Journal of Behavioral Consultation & Therapy. 2010;6(3) [Google Scholar]
  43. Rosen GM, Davison GC. Psychology should list empirically supported principles of change (ESPs) and not credential trademarked therapies or other treatment packages. Behavior Modification. 2003;27(3):300–312. doi: 10.1177/0145445503027003003. http://doi.org/10.1177/0145445503027003003. [DOI] [PubMed] [Google Scholar]
  44. Rotheram-Borus MJ, Swendeman D, Chorpita BF. Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist. 2012;67(6):463–476. doi: 10.1037/a0028180. http://doi.org/10.1037/a0028180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin. 1979;86(2):420. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  46. Skevington SM, Lotfy M, O’Connell KA. The World Health Organization’s WHOQOL-BREF quality of life assessment: Psychometric properties and results of the international field trial. A Report from the WHOQOL Group. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation. 2004;13(2):299–310. doi: 10.1023/B:QURE.0000018486.91360.00. [DOI] [PubMed] [Google Scholar]
  47. Smith JD. Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods. 2012;17(4):510–550. doi: 10.1037/a0029312. http://doi.org/10.1037/a0029312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Thompson SG, Higgins J. Can meta-analysis help target interventions at individuals most likely to benefit? The Lancet. 2005;365(9456):341–346. doi: 10.1016/S0140-6736(05)17790-3. [DOI] [PubMed] [Google Scholar]
  49. Tschacher W, Ramseyer F. Modeling psychotherapy process by time-series panel analysis (TSPA) Psychotherapy Research. 2009;19(4–5):469–481. doi: 10.1080/10503300802654496. http://doi.org/10.1080/10503300802654496. [DOI] [PubMed] [Google Scholar]
  50. Twohig MP. Acceptance and commitment therapy for treatment-resistant posttraumatic stress disorder: A case study. Cognitive and Behavioral Practice. 2009;16(3):243–252. [Google Scholar]
  51. Van den Noortgate W, Onghena P. The Aggregation of Single-Case Results Using Hierarchical Linear Models. Behavior Analyst Today. 2007;8(2) Retrieved from http://search.ebscohost.com/login.aspx?direct=true&profile=ehost&scope=site&authtype=crawler&jrnl=15394352&AN=25730012&h=Mx4fePqt7fUI%2BCav2oiQjBdrzl%2FIcitqo0lndUV2NJcovTDTL8c%2FDpTC6MuLLuO%2FWvQB4RHc4sC%2Fok16Tbt61Q%3D%3D&crl=c. [Google Scholar]
  52. Vilardaga R. Technical, practical and analytic innovations in single case designs for contextual behavioral scientists. Journal of Contextual Behavioral Science. 2014;3(2):136–137. doi: 10.1016/j.jcbs.2014.03.003. http://doi.org/10.1016/j.jcbs.2014.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Vilardaga R, Bricker JB, McDonell MG. The promise of mobile technologies and single case designs for the study of individuals in their natural environment. Journal of Contextual Behavioral Science. 2014;3(2):148–153. doi: 10.1016/j.jcbs.2014.03.003. http://doi.org/10.1016/j.jcbs.2014.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Vilardaga R, Heffner JL, Mercer LD, Bricker JB. Do counselor techniques predict quitting during smoking cessation treatment? A component analysis of telephone-delivered acceptance and commitment therapy. Behaviour Research and Therapy. 2014;61:89–95. doi: 10.1016/j.brat.2014.07.008. http://doi.org/10.1016/j.brat.2014.07.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: Results from the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62(6):629–640. doi: 10.1001/archpsyc.62.6.629. http://doi.org/10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  56. Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, Mayberg S. Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry. 2012;69(3):274–282. doi: 10.1001/archgenpsychiatry.2011.147. http://doi.org/10.1001/archgenpsychiatry.2011.147. [DOI] [PubMed] [Google Scholar]
  57. Weisz JR, Weiss B, Alicke MD, Klotz ML. Effectiveness of psychotherapy with children and adolescents: a meta-analysis for clinicians. Journal of Consulting and Clinical Psychology. 1987;55(4):542–549. doi: 10.1037/0022-006X.55.4.542. [DOI] [PubMed] [Google Scholar]
  58. Zettle RD, Rains JC, Hayes SC. Processes of change in acceptance and commitment therapy and cognitive therapy for depression: A mediation reanalysis of Zettle and Rains. Behavior Modification. 2011;35(3):265–283. doi: 10.1177/0145445511398344. [DOI] [PubMed] [Google Scholar]

RESOURCES