Empirical Examinations of Modifications and Adaptations to Evidence-Based Psychotherapies: Methodologies, Impact, and Future Directions

Abstract

This review describes methods used to examine the modifications and adaptations to evidence-based psychological treatments (EBPTs), assesses what is known about the impact of modifications and adaptations to EBPTs, and makes recommendations for future research and clinical care. One hundred eight primary studies and three meta-analyses were identified. All studies examined planned adaptations, and many simultaneously investigated multiple types of adaptations. With the exception of studies on adding or removing specific EBPT elements, few studies compared adapted EBPTs to the original protocols. There was little evidence that adaptations in the studies were detrimental, but there was also limited consistent evidence that adapted protocols outperformed the original protocols, with the exception of adding components to EBPTs. Implications for EBPT delivery and future research are discussed.

Policymakers and mental health systems have devoted substantial resources and attention to the implementation of evidence-based psychosocial treatments (EBPTs) and interventions. However, many researchers and clinicians have raised questions about their fit and effectiveness for individuals with characteristics typically seen in routine care settings, such as comorbid mental health diagnoses that were not included or addressed in the original studies, insufficient insurance coverage for the required number of sessions, and differences in culture, literacy, or other patient characteristics and circumstances. Challenges in delivering EBPTs in the context of routine care settings include replicating the timing of sessions (often twice a week in randomized controlled trials) and delivering session content as specified in the manuals.

In efforts to address these contextual challenges, which are not present in the original, well-controlled trials that established treatment efficacy, clinicians in routine care settings commonly modify and adapt protocol treatments (Aarons, Miller, Green, Perrott, & Bradway, 2012b; Cook, Dinnen, Thompson, Simiola, & Schnurr, 2014b; Stirman et al., 2013a). Modification is a term used to describe planned or unplanned changes made to an EBPT protocol or its delivery in an attempt to improve the fit, engagement, or effectiveness of the treatment (Stirman, 2015). Adaptation is a form of modification that is planned or purposefully made to the design or delivery of an intervention, often with the intention to retain fidelity to the fundamental elements or spirit of the intervention (Lee, Altschul, & Mowbray, 2008; Stirman, 2015; Stirman, Miller, Toder, & Calloway, 2013b). Literature on the modification of EBPTs to date has not always differentiated between adaptations that are carefully planned and monitored for their impact on symptom change (Chambers, Glasgow, & Stange, 2013; Lee et al., 2008) and modifications that occur less systematically and without planning (Stirman et al., 2015), although the extent and impact of planned and unplanned changes may be very different. There is some evidence that unplanned modifications may be common in routine care settings (Aarons et al., 2012b; Cook et al., 2014b). For example, Cook and colleagues found that therapists reported tailoring EBPTs, integrating them with other approaches, removing elements of the treatments, and changing the length of the sessions or the protocols when delivering them in inpatient treatment settings (2014b). Similarly, therapists surveyed by Lau and colleagues (2017) reported tailoring (modifying how the treatment or materials were presented), integration of other strategies, reordering treatment elements, removing components, and changing the length of the sessions or protocol. Aarons and colleagues (2012) identified a variety of adaptations in routine care that were provider-, program-, and consumer-driven. Some such changes may be relatively minor and may not be expected to have an appreciable impact on clinical outcomes, but others might depart significantly from the original protocol.

In the absence of empirical evidence, implications of modification to EBPT protocols have been the subject of debate. Although some have argued that adaptation is to be expected and may improve treatment effectiveness and sustainability in routine care settings (Chambers et al., 2013), others have cautioned that changes to evidence-based interventions, particularly unplanned or less systematically derived modifications, could result in diminished treatment response (Blakely et al., 1987; Cohen et al., 2008). Previous reviews that have considered the relationship between treatment fidelity and adaptation have argued for a middle ground, wherein “flexible fidelity” allows for minor, planned adaptations that do not compromise core elements of the treatments (Forehand, Dorsey, Jones, Long, & McMahon, 2010; Kendall & Beidas, 2007). Others have suggested that development of transdiagnostic treatment protocols as a means to balance between fidelity and flexibility, through delivery of clinically indicated elements of evidence-based treatments to populations with a variety of presenting problems and diagnoses (McHugh, Murray, & Barlow, 2009).

Although related to treatment fidelity, modification and adaptation are unique constructs. Treatment fidelity has been defined as adherence to the key intervention components, competence, or skill with which the intervention is delivered, as well as treatment receipt, and differentiation from other treatments (Gearing et al., 2011; Schoenwald et al., 2011). Any therapy that has been investigated in an efficacy study has an associated fidelity assessment instrument, required for rigorous clinical trials, that specifies elements that are believed (or, at times, empirically demonstrated) to be central to the interventions. However, fidelity assessment, which focuses on the delivery of central aspects of the intervention, may fail to capture certain types of modifications or adaptations, such as minor changes to terminology or language, changes to the length of the session or protocol, or the use of elements that are neither prescribed nor proscribed. Furthermore, many fidelity assessment instruments do not assess treatment differentiation and therefore may not detect integration or the addition of other treatment elements. Thus, fidelity assessment alone may provide limited understanding of whether different types of alterations are detrimental, non-detrimental, or enhancements to the treatment protocol (Stirman et al., 2012; Zvoch, 2009).

In contrast to modification and adaptation, the impact of aspects of treatment fidelity on clinical outcomes have been examined in numerous investigations, reviews, and a meta-analysis (Barber, Triffleman, & Marmar, 2007; DeRubeis, Gelfand, Tang, & Simons, 1999; Strunk, Brotman, & DeRubeis, 2010a; Strunk, Brotman, DeRubeis, & Hollon, 2010b). Whether fidelity is necessary to produce good clinical outcomes has been the subject of debate, particularly in light of a meta-analysis that concluded that overall, fidelity did not appear to be predictive of symptom changes (Webb et al., 2010). Although some limitations to the primary studies included in this analysis (e.g., use of an average fidelity score rather than a session-by session assessment of fidelity and of symptoms that would allow temporal precedence of fidelity to be established) may have obscured a potential relationship, some individual, rigorously designed studies have concluded that there is a relationship for specific interventions (Feeley, DeRubeis, & Gelfand, 1999; Strunk et al., 2010). Some studies have also identified a non-linear relationship between fidelity and treatment outcome. For example, Barber and colleagues (2006) identified a curvilinear relationship between adherence and treatment outcome, such that higher and lower levels of adherence were associated with poorer outcomes in treatment for cocaine dependence than moderate levels of adherence. Hogue and colleagues (2008) examined the impact of adherence in multidimensional family therapy for adolescents with externalizing behaviors and found a linear relationship between adherence and outcomes for externalizing problems, but moderate levels of adherence was associated with the stronger outcomes for internalizing behaviors. Collectively, these findings have potential implications for modification of EBPTs. If high levels of adherence are not in fact essential to produce good clinical outcomes, it is possible that modification may not negatively impact clinical outcomes. Competent treatment delivery may in fact require at least minor adaptations to meet patient needs or ensure that patients are able to understand and benefit from the skills and interventions that are delivered (Roth & Pilling, 2008). However, to date, much of the consideration of modifications to EBPTs has consisted of theoretical articles about planned adaptation, descriptions of planned adaptations, and investigations of the impact of specific types of adaptations to EBPTs that were set forth at the beginning of a study.

More recently, some have attempted to categorize adaptation and modifications (Hill, Maucione, & Hood, 2007; Moore, Bumbarger, & Cooper, 2013; Stirman et al., 2013b). Descriptions of modifications and adaptations in the research literature range broadly from slight changes in terminology or delivery of the same content in different languages (tailoring), to removal of core components or integration with other interventions (Chu & Leino, 2017; Stirman et al., 2013b). Moreover, modifications can be made to the content of the intervention (e.g., removing, changing, or adding elements of the intervention), or the context in which it is delivered (e.g., delivered in groups or in different treatment settings; Chu & Leino, 2017, Stirman et al., 2013b). Others have pointed out the importance of distinguishing between proactive and reactive forms of modification and adaptation (Moore et al., 2013; Stirman et al., 2013b) and adaptations made to improve theoretical vs. logistical fit (Moore et al., 2013). Adaptations have also been grouped into categories such as enhancing/expanding, simplifying/reducing (Lau et al., 2017), and fidelity-consistent and fidelity-inconsistent (Stirman, 2015), although how different types of adaptations would be categorized in terms of fidelity would depend on the nature of a specific intervention. The varied nature of the changes to EBPTs can have very different implications for outcomes of interest. Although some might facilitate implementation and sustainability by improving the fit between the intervention, the target population, and the context into which an EBPT is introduced, they may also erode treatment integrity or compromise clinical outcomes (Kennedy, Mizuno, Hoffman, Baume, & Strand, 2000). One meta-analysis comparing novel interventions, standard protocols, and adapted protocols found a trend-level advantage for adapted psychosocial interventions when compared to standard interventions in terms of improving effectiveness (Sundell, Beelmann, Hasson, & von Thiele Schwarz, 2015), but it included a variety of intervention types and did not examine specific types of adaptations. As Bell, Marcus, and Goodlad (2013) demonstrated in their reconsideration of a meta-analysis that showed no effects associated with adding or removing components of psychotherapies (Ahn & Wampold, 1999), grouping different types of adaptations can mask differences in outcomes. Other than Bell and colleagues' careful effort to assess the impact of adding or removing treatment components, there is little empirical guidance about whether certain modifications to EBPT should be avoided or encouraged.

Although potentially informative, a full and careful assessment of the impact of each different type of modification on clinical outcomes for each and every EBPT would clearly be expensive, time consuming, and impractical. Therefore, looking toward the existing literature may result in some guidance regarding the state of the research on EBPT modification and important directions for future research. Over the course of the development of the literature on EBPTs, some studies have been conducted to investigate adapted interventions. Although there have been some reviews that have considered the relationship between adherence and flexible application of EBP elements (Forehand et al., 2010; Kendall & Beidas, 2007; McHugh et al., 2009), and meta-analytic reviews of three specific forms of adaptation (Bell et al., 2013; Nieuwsma et al., 2012), there has yet to be a critical review of the study designs and methodologies used in such research or the findings related to many of the specific forms of modification. Therefore, the goals of this systematic review are 1) to identify the types of modifications and adaptations that have been investigated and characterize the reasons that different forms of modification were made, 2) to examine the methodologies used in previous research on modifications 3) to better understand what is known about the clinical impact of specific types of modifications to the content of EBPTs, and 4) to make recommendations for future research and current efforts to implement and EBPTs in clinical practice settings. Although we expected that most published studies would focus on planned adaptations, we also searched for any studies that investigated the impact of unplanned modifications that occurred during routine care. By reviewing these studies, limitations in knowledge about some types of changes to EBPTs may be revealed. Additionally, although interventions and target populations may vary, patterns of methodological limitations, gaps in the literature, and patterns of findings may emerge when the implications of the research on specific types of modifications are examined across the existing literature.

Method

Scope

The intent of this review is to provide a critical overview of the research methodologies that have been used, highlight strengths and limitations in the research to date, and to summarize and synthesize findings from research on different forms of modifications to EBPT content. Therefore, this review will not include articles that summarize recommendations for adaptations or descriptions of planned adaptations that do not include clinical outcome data. Additionally, unless the content of the EBPTs were also adapted, this review does not examine the impact of contextual-level modifications such as delivery of an EBPT to a new population or in a different format or setting. We define EBPTs as treatments that have demonstrated efficacy in at least one adequately powered randomized controlled trial or multiple smaller controlled studies. Because decisions to implement treatments in routine care weigh clinical demand with the body of existing research evidence and treatments may be used for populations for which they have not specifically been studied, we employed less stringent criteria for treatments that warranted inclusion in our review than those for empirically supported treatments (Chambless & Hollon, 1998), which require additional research evidence for specific populations.

Search Strategy

We searched the literature for articles published or in press before January, 2017 that investigated modified or adapted EBPT protocols. We searched the following databases: Medline, ISI, PsycInfo, Academic Search Premier, Health Source, ERIC, PubMed, and Google Scholar, using the terms “modify*” or “adapt*” or “cultural adaptation” or “dismantling” and “evidence-based psychosocial treatment”, “evidence-based psychotherapy”, “psychotherapy”, “cognitive behavioral therapy”, and the names of specific commonly studied mental health disorders (major depressive disorder, PTSD, anxiety and eating disorders, schizophrenia, bipolar disorder, conduct disorder, borderline personality disorder). We also employed a snowballing strategy to search the reference sections of articles that we identified as well as theoretical papers, studies, and reviews that discussed adaptations to EBPTs (Bell et al., 2013; Benish, Quintana, & Wampold, 2011; Chu & Leino, 2017; Forehand et al., 2010; Kendall & Beidas, 2007; McHugh et al., 2009; Stirman et al., 2013b, Gordon et al., 2016). The authors reviewed abstracts and full text articles when necessary to determine their eligibility for this review and discussed any differences of opinion regarding inclusion with the study team. Care was taken to be as thorough as possible in identifying studies, but it is possible that some studies were missed due to inconsistencies in terms used to describe changes to EBPTs in the literature.

Inclusion criteria for articles were: 1) described one or more modifications made to a specified, manualized psychotherapeutic treatment that had previously demonstrated efficacy or effectiveness for a DSM- III or IV diagnosis in at least one well-designed study and/or met APA Division 12 criteria for strong or modest support (Chambless & Hollon, 1998); We used the description of the treatments and the summary of research reported on those treatments from the papers to make a determination of whether they met this criterion, and when necessary, conducted a further literature search to determine whether treatments met this criterion 2) the article included sufficient detail about one or more content-level modifications to facilitate coding; 3) the article employed a case series or record review, within-subjects design, open trial, non-randomized comparison, benchmarking, randomized-controlled trial, dismantling design, or included a meta-analysis of studies that individually would have been eligible for the review; and 4) the article presented clinical outcome data, and 5) at least some portion of the intervention was intended to be delivered directly to the identified patient (rather than exclusively to a parent, caregiver or teacher). Modifications could be either adaptations (i.e., intentional, planned changes that typically included an effort to preserve fidelity) or changes that were made without premeditation during the delivery of the intervention. We also included studies that reported on clinical outcomes data that were collected in routine care settings. Although some more recently developed interventions were created to allow a high degree of flexibility and adaptability (e.g., the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders; Barlow et al., 2011; Weisz et al., 2012), studies of these treatment protocols were not included in the review unless specific content-level adaptations were made to those protocols. Articles were excluded if they assessed fidelity but did not describe adaptations (e.g., Strunk et al., 2010), if they focused on prevention rather than treatment (e.g., Kennedy et al., 2000), if they described or suggested adaptations without providing data on clinical outcomes (e.g., Kaysen et al., 2013), if they reported on adaptations for a single case (e.g., Graham-LoPresti, Gautier, Sorenson, & Hayes-Skelton, 2017), or if the intervention was delivered to an individual other than the identified patient (e.g., caregiver or parenting interventions; Parra-Cardona et al., 2017). Meta-analyses were also examined to determine if the included studies met inclusion criteria for this review and each eligible study included in the meta-analysis was coded.

Coding Strategy

Studies were classified by study design, type of change made to the EBPT, and study findings. We categorized study designs based on previous categorizations of studies that have been used to establish feasibility, efficacy, possible efficacy, or effectiveness of psychotherapy interventions (Carey & Stiles, 2015; Chambless & Hollon, 1998; Najavits, 2003; Weersing & Hamilton, 2005). Types of adaptations and modifications were identified using a framework and coding system of modifications and adaptations to evidence-based psychotherapies (see Table 1 for definitions of each; Stirman et al., 2013b). Additional information collected from each study included intervention studied, target population, adaptation type(s), sample size, comparison group, a summary of key results, and the information necessary to derive bias-corrected effect sizes. Raters trained together on the coding system by reviewing and discussing operational definitions, rating five articles per week separately and discussing them in weekly meetings over one month, at which time, rater agreement was 92%. Subsequently, individual raters reviewed articles for potential inclusion and coded articles that met inclusion criteria. Three raters then reviewed the coding for all of the articles. If questions arose about inclusion or appropriate codes, the team achieved consensus through discussion.

Table 1. Definition and number of specific types of content-level modifications.

Content-Level Modification	Definition	N
Tailoring	Minor alteration of aspects of a treatment without significant changes or removal of core treatment elements	43
Removing±	Implementing an EBPT without one or more distinct elements described in the original protocol	39
Adding±	Including one or more distinct treatment components that are not part of the original EBPT protocol	37
Shortening (protocol)†,‡,	Decreasing the number of sessions that are delivered (without removing treatment components)	24
Shortening (session)†	Decreasing the amount of time allocated for EBPT sessions	11
Lengthening (protocol)†,‡	Increasing the number of sessions in the EBPT protocol (without adding treatment components)	7
Lengthen (session)	Increasing the amount of time allocated for EBPT sessions	6
Integrating	The infusion of a different, established therapeutic approach into an EBPT throughout the duration of the protocol	3
Repeating	Elements that are normally prescribed or conducted once during a protocol are used more than once	2
Reordering	Elements are delivered or completed in a different order than originally specified in the protocol	2
Loosening	Elements that are intended to structure intervention sessions do not occur as prescribed by the protocol	1

Original studies are included in the counts provided; meta-analyses are not included in this table.

^†Eight studies shortened both sessions and protocol;

^‡two studies allowed for flexible length, thus shortening and lengthening the protocols.

Calculation of Effect Sizes

Using the data available in the published studies, we calculated Hedge's g, which corrects for small sample size and, in contrast to Cohen's d, uses a weighted pooled standard deviation, for the main outcome described in each study, with 95% confidence intervals (results are included in the supplemental table). Whenever possible, we calculated the effect sizes based on comparisons to the original protocols or controls, and when those data were not available, pre-post effect sizes were computed. The supplemental table specifies whether between-condition or pre-post effect sizes were calculated. When data were reported, but were insufficient to compute effect sizes, we noted this in the supplemental table and summarized the relevant results found in the articles.

Results

Our search resulted in 296 articles for abstract review, with 154 resulting directly from searches and 142 resulting from a snowballing approach. One hundred eighty-eight of those articles met our exclusion criteria (64 were not research studies, 12 were single case descriptions, 87 were not applied to specific psychosocial disorders, did not use an evidence-based protocol, or there was not enough information to discern the evidence base for the specific protocol that was used, 5 did not contain content-level modifications, and 20 did not present sufficient data, results, or specific information about modifications). One hundred eight original studies met inclusion criteria for this review. All of the articles described planned adaptations. We did not identify research on modifications that were not planned or that occurred in routine care conditions without pre-specified guidelines for acceptable adaptations. Therefore, hereafter, we refer to the changes that were identified in the articles as adaptations. Table 1 indicates the number of studies that included each type of adaptation, along with a definition for each type of adaptation. Some adaptations and modifications specified in the Stirman, Miller et al. (2013) framework, such as substitution of a different element in place of an element specified in the protocol, were not the subject of investigation in any studies identified for this review. Detailed findings regarding study size, design, effect sizes and confidence intervals, and whether the comparison was pre to post, adapted protocol compared to a control or alternative treatment, or to the original protocol can be found in the online supplemental table.

Study Design and Methodology

Forty-one studies made multiple content-level adaptations to the EBPT, and 43 included content-level adaptations as well as contextual adaptations. The remainder reported on the impact of a single adaptation. Three studies provided clinicians with guidelines for adaptation and allowed them to make the pre-specified adaptations on a case-by-case basis. Sixty (55%) included a comparison to the standard protocols; others were open trials or comparisons to control conditions. One study tested non-inferiority of an adaptation as compared to a standard protocol. We identified three meta-analyses that included studies that met our inclusion criteria: two on shortened/brief interventions (Cape, Whittington, Buszewicz, Wallace, & Underwood, 2010; Nieuwsma et al., 2012), and one study that separately examined the impact of adding and removing components (Bell et al., 2013), which was a re-analysis of an earlier meta-analysis on component studies (Ahn & Wampold, 2001). Other meta-analyses on adaptation contained fewer studies that met our inclusion criteria (Benish et al., 2011; Sundell et al., 2015). Studies that were included in these meta-analyses and that met our inclusion criteria are individually summarized in the supplemental table.

The majority of the studies that compared adapted to standard protocols focused on adding or removing components. Few studies of other forms of adaptation made comparisons to original protocols and many studies included multiple types of adaptations. Thus, it is not possible to isolate the impact of most types of specific adaptations on clinical outcomes, or to conduct a rigorous meta-analysis for individual adaptations, other than those that have been the subject of meta-analyses that we include in our review (Bell et al., 2013; Cape et al., 2010; Nieuwsma et al., 2012). However, the variation in research design and settings in which the studies were conducted allows for consideration of the type of study design and nature of adaptations that need to be studied to yield clinically useful results for different types of adaptations.

In the following sections, the review results are organized by the type of content-level adaptation specified in the articles. Within each section, we describe reasons that the adaptations were made and the nature of the adaptations, discuss the findings, and comment on the study design. Within this general organizational framework, studies are grouped by treatment and/or population. The supplemental table contains details about each study's design, sample size, population, and outcomes. We discuss implications for clinical practice and consider whether the design of the studies in the review reflect the way adaptations are typically made in clinical practice, noting limitations and future directions for research.

Results and Discussion for Specific Adaptations

Adding components

Adding refers to the addition of one or more distinct treatment components that are not part of the original EBPT protocol. Thirty-seven investigated protocols with added components, but only 13 did not include other adaptations. Twenty-eight of these studies compared a standard protocol to an adapted protocol, although sample sizes varied and individual studies may not have been powered to detect significant differences between standard and adapted conditions. Few occurred in routine care settings and those that did used open trial designs or compared to usual care. Our review of these studies identified additions to protocols that served a variety of purposes, including efforts to enhance treatment outcomes or to address unique patient characteristics.

Bell, Marcus, and Goodland's meta-analysis (2013) included 36 studies that added components, 23 of which met inclusion criteria for this review. These studies, which compared standard and adapted protocols, typically added cognitive behavioral components in an effort to enhance the effectiveness of the intervention. Five studies included in the Bell et al. meta-analysis also tailored treatment in some way. The meta-analysis concluded that small but significant effects were found for primary outcomes for the adapted interventions compared to the original protocols, and the effects increased slightly at follow-up. That pattern remained consistent when we examined the 23 studies that met our inclusion criteria.

Other studies that were not included in the Bell et al. meta-analyses varied in design and in the purpose of the adaptations (summaries of the designs, populations, and effect sizes can be found in the supplemental table). Three studies examined the inclusion of additional cognitive behavioral interventions to cognitive behavioral protocols (Cloitre et al., 2010; Mörtberg, Clark, Sundin, & Åberg Wistedt, 2007; Schulte, Künzel, Pepping, & Schulte-Bahrenberg, 1992; Sportel, de Hullu, de Jong, & Nauta, 2013) and two studies added components to address childhood sexual abuse (Chard, 2005; Cloitre et al., 2010). Four studies included a distinct intervention at a specified point in the protocol (Chard, 2005; Cloitre et al., 2010; Mörtberg et al., 2007; Sportel et al., 2013) and the other allowed clinicians to add interventions at any point in the protocol (Schulte et al., 1992). These different approaches yielded different results. Greater latitude resulted in poorer outcomes (Schulte et al., 1992), and a more circumscribed or sequenced addition of elements yielded large pre-post effects (Chard et al., 2005; Cloitre et al., 2010) or a small effect in favor of a control (Sportel et al., 2013). A small effect was found in favor of the standard individual protocol compared to a group protocol that added a different form of exposure (Mörtberg, Clark, Sundin, & Åberg Wistedt, 2007). Due to multiple modifications made in the Mortberg study, however, whether this difference was due to the change to an intensive group format or due to the nature of the form of exposure that was added cannot be determined.

Studies that focused on adolescents added components designed to foster parental involvement or to address additional clinical issues. These studies identified significant benefits (medium to large pre-post effect sizes) to the modified protocol when compared to usual care or in the context of within-subjects designs and open trials. However, the only two studies that compared adapted and standard protocols for depressed adolescents identified a small effect in one study, and no effect in a larger, subsequent trial (Clarke, Rohde, Lewinsohn, Hops, & Seeley, 1999; Lewinsohn, Clarke, Hops, & Andrews, 1990). Five studies investigated an adapted dialectical behavior therapy (DBT) for adolescents that included additional components, along with other forms of adaptation. Some of these studies added elements to address diagnoses (e.g., eating disorders; bipolar disorder) other than borderline personality disorder (Salbach-Andrae, Bohnekamp, Pfeiffer, Lehmkuhl, & Miller, 2008), and others were added to address other needs or differences between the study population and the population with which the EBPT was originally tested (e.g., Charlton & Dykstra, 2011). Because these studies were all open trials and case series, their outcomes support feasibility but do not shed light on the relative benefits of adapted and standard DBT. However, the single study that employed a benchmarking approach indicated that when adapted DBT was delivered to adults in a routine care with added components, the effects were within the range of those found in published clinical trials of the standard intervention (Comtois, Elwood, Holdcraft, Smith, & Simpson, 2007).

Studies that added modules to tailor treatments to racial and ethnic minority populations (Burrow-Sanchez & Wrona, 2012; Kohn, Oden, Muñoz, Robinson, & Leavitt, 2002) were characterized by low sample sizes and were likely underpowered to detect differences from the original protocol. Effect sizes suggested an advantage to the standard group in one study (Burrow-Sanchez & Wrona, 2012), although an interaction suggested that the adapted protocol yielded better results for a subgroup. In the other study, there was a small effect in favor of the adaptation (Kohn, Oden, Munoz, Robinson, & Leavitt, 2002). In future comparisons between standard protocols and those that add modules to address specific cultural considerations, it will be particularly important to sufficiently power the research to examine moderating factors such as level of acculturation.

In combination, these findings suggest that in the absence of clear guidance regarding additions to a specific protocol, any additions made to EBPTs in routine care should be discrete, well-defined, and based on sound theory and understanding of the target population. Symptom measures should be used before and after any sessions in which content is added, and if possible, benchmarked against data for the standard protocol, to inform decisions about the clinical utility of adaptation for individual patients or specific populations.

Integration

In contrast to adding a distinct, theoretically consistent component to a treatment for a limited number of sessions, integration is the infusion of a different, therapeutic approach into an EBPT throughout the duration of the protocol. Three studies integrated cultural or spiritual healing practices into the interventions, although only one compared the adapted protocol to a standard protocol (Barrett, 1998), and the other two studies also tailored the interventions (Bradley et al., 2006; Venner et al., 2016). The Venner et al. (2016) study was a small pilot that integrated two psychosocial interventions. Barrett's (1998) study infused a family component throughout a group CBT intervention for adolescents with anxiety and identified a small effect in favor of the adapted protocol. The studies included in this review integrated a single additional approach, whereas studies suggest that therapists often pick and choose EBPT elements to integrate into their preferred treatment modality (Cook, Dinnen, Simiola, Thompson, & Schnurr, 2014a; Stirman et al., 2013a) or use a more eclectic approach (Stirman et al., 2013a; von Ranson & Robinson, 2006; Wallace & von Ranson, 2012). Additional studies on integration of other, and perhaps multiple, common psychotherapeutic modalities (e.g., client-centered, Gestalt, dynamic, family systems) are needed to understand how this practice impacts clinical outcomes.

Removing EBPT Elements

Removing refers to implementing an EBPT without one or more distinct original elements. Thirty-nine studies examined removal of EBPT elements, with 34 of them comparing the adapted and standard protocol. The meta-analysis conducted by Bell and colleagues (2013), which included 30 of these studies, identified neither a positive nor negative overall effect associated with removing components of treatment protocols. The other studies we identified supported this conclusion. Importantly, the dismantling studies were designed to determine whether single, specific central components of treatment protocols that were either hypothesized to be essential or non-essential components (e.g., eye movement, progressive muscle relaxation, exposure, written trauma accounts) were necessary to yield results that were comparable to the full protocol. The results were intended to determine whether the protocols could be streamlined to remove elements that were not active components or to remove elements that may be more challenging to implement. In contrast, most studies that we reviewed beyond the meta-analysis generally combined removal of components with other adaptations, and seven of these additional studies included other adaptations as well. Notably, none of the studies that we reviewed represented some elements that are typically removed in routine care, such as components that are conceptualized as tools to support cognitive behavioral treatment like agenda setting, homework, or clinical worksheets (Cook et al., 2014b; Stirman et al., 2013a). However, one study did remove a culturally incongruent component, while still finding a large effect in favor of the adapted intervention over a control intervention (Murray et al., 2013; Murray, Skavenski, Kane, & et al., 2015). Previous studies that have examined the use of components, such as homework, have typically been process studies rather than studies that randomized patients into standard or adapted conditions. Thus, the impact of the removal of such elements remains to be studied. Particularly because research indicates that these are fairly common occurrences in routine clinical practice, it is important to understand, and potentially provide guidance, regarding decisions to remove such elements from treatment in routine care settings (Cook et al., 2014b; Lau et al., 2017; Stirman et al., 2013a). Additionally, to inform whether more streamlined forms of EBPTs are viable, effective options for routine care settings, future dismantling studies should test for non-inferiority of treatments that remove specific elements associated with greater burden on the clients or therapists.

Although most of the evidence does not suggest that removing certain components from EBPTs diminishes clinical outcomes, decisions to remove elements in routine care should include careful consideration of the theory behind the treatments and the original theoretical rationale for including the element. Also, the ways in which specific patient characteristics might interact with each treatment element should be considered. If studies are not available to inform the removal of elements, the use of single case designs, consistent monitoring of progress, and benchmarking may be useful in providing further guidance for individualized treatments.

Tailoring EBPT Protocols

Tailoring refers to the relatively minor alteration of aspects of a treatment without significant changes or removal of core treatment elements. There are several ways in which therapists may tailor a treatment. Examples include changing the language in which the intervention is delivered, modifying the terminology, examples, or metaphors provided to patients to illustrate concepts, or making changes to handouts or assignments to make them more appropriate for the population. These types of adaptations are common in routine care settings (Stirman et al., 2013; Cook et al., 2014; Aarons et al., 2014), and may be examples of a form of “flexible fidelity” that has been advocated in the literature (Kendall & Beidas, 2007; Forehand et al., 2010). In this review, 43 studies of tailored interventions were identified. However, only 7 were compared to standard protocols, and all of them included additional forms of adaptation.

Some studies, none of which included comparisons to a standard intervention, tailored EBPTs to address factors such as delivery to populations of different ages or diagnostic characteristics. For example, some open trials investigated EBPTs with tailored homework and written materials, such as handouts, to meet the needs of specific populations (e.g., Charlton & Dykstra, 2011; Goldstein, Axelson, Birmaher, & Brent, 2007; Salbach-Andrae et al., 2008), and these adaptations consisted largely of simplification of the terminology or the homework. Others tailored content somewhat more extensively to address issues experienced by depressed adolescents in different contexts (Mufson, Weissman, Moreau, & Garfinkel, 1999; Shelton, Kesten, Zhang, & Trestman, 2011), such as inpatient or correctional settings, and tested them using within-subjects designs (Shelton, Kesten, Zhang, & Trestman, 2011) or compared them to a wait-list controls (Mufson et al., 1999). Finally, in some open trials, EBPT content was tailored before delivery to different age groups or diagnostic populations (e.g., Fleischhaker et al., 2011; Katz, Cox, Gunasekara, & Miller, 2004; Salbach-Andrae et al., 2008; Stark, Reynolds, & Kaslow, 1987).

Nineteen studies tailored EBPTs to align them with cultural needs and values. However, only two small pilot studies compared the tailored intervention to a standard intervention (Burrow-Sanchez & Wrona, 2012; Pan, Huey Jr, & Hernandez, 2011), with one finding evidence of a small effect in favor of the standard protocol, and the other indicating a large effect in favor of the adapted protocol (results detailed in the supplemental table). Both studies also found preliminary evidence of a moderating effect for acculturation or ethnic identity, suggesting that future research should be adequately powered to determine for whom tailoring to increase cultural relevance may be most necessary. Studies that compared adapted protocols to control groups or other interventions (Bass et al., 2013; Murray et al., 2015; Rosselló & Bernal, 1999) also found evidence that tailored interventions yielded large pre-post effects (Miller et al., 2011), as did open trials (Interian, Allen, Gara, & Escobar, 2008; Kanter, Santiago-Rivera, Rusch, Busch, & West, 2010; Murray et al., 2015). Although the studies were not designed to determine whether tailoring was necessary to optimize benefits, the research demonstrates feasibility and potential effectiveness of culturally adapted protocols, and very little evidence of detrimental effects. Meta-analyses of cultural adaptations of a variety of preventive and psychosocial interventions (some of which are EBPTs for psychological disorders and some of which are not) also yielded equivocal results or small effects, depending on the nature of the intervention, populations, and adaptations (c.f., Huey et al., 2014; Benish et al., 2011). To fully understand the impact of tailoring to increase the cultural relevance to specific populations, future studies would need to be fully powered to compare tailored and standard interventions and detect potential interactions with level of acculturation and other factors (Cardemil, 2010).

Given the numerous differences in populations and settings for which established EBPTs may need to be tailored, it is not feasible, and may not be desirable, to conduct fully powered comparisons of standard and tailored adaptations to establish the benefits of tailoring for each and every context. As Lau (2006) suggests, assumptions that tailoring and adapting interventions is always necessary may not be correct. However, at times, local practice evaluation data may indicate that tailoring is warranted, and tailoring without removal of core elements of treatments may reflect good clinical care. For example, if engagement in, or degree of understanding of a standard EBPT protocol is lower for specific sub-populations, it may be important to consider tailoring the intervention while collecting practice-level data to determine whether the adaptations are having the desired impact on treatment engagement and other outcomes. In other circumstances, knowledge of the population will dictate whether tailoring elements of the protocol is appropriate or necessary. For example, handouts, homework, and certain concepts may need to be tailored when literacy, language of origin or fluency, education levels (Bass et al., 2013; Kaysen et al., 2011; Marques et al., in press; Schulz, Resick, Huber, & Griffin, 2006) or the presence of developmental disabilities (Charlton & Dykstra, 2011) suggest that an unmodified protocol would not meet the needs of, or could not be understood or utilized by the population. In such cases, the current evidence, and recommendations from the field (Chambers et al., 2013; Chu & Leino, 2017; Forehand et al., 2010; Stirman et al., 2013b) suggest preserving core elements of the protocols while making adjustments to language, terminology, and complexity, and tracking clinical data as these changes are implemented.

Repeating Sessions or Session Material

A code for repetition was assigned when either the content of a full session or session elements were repeated or extensively reviewed in a manner that is not consistent with instructions in the standard protocol. Repetition may be a common strategy in routine care settings, as clinicians may deem it necessary to ensure that patients understand certain concepts and materials before advancing in the treatment. However, very little research has been conducted to assess the impact of repetition. Two studies investigated adapted protocols that included repetition, although in both studies, other adaptations were also made. In a randomized study, Lynch and colleagues (2003) adapted DBT for depressed older adults and combined it with medications and compared it to medication alone in a small trial (n = 34). In addition to shortening both the sessions and the protocol length and removing some sessions, the researchers repeated the full protocol so that participants were exposed to all skills training material twice, and found evidence of a medium effect in favor of the adapted protocol. In the other study, an open trial of STAIR for PTSD (Levitt, Malta, Martin, Davis, & Cloitre, 2007), repeating earlier aspects of treatment was at the discretion of the therapist, and it is unclear how many study participants repeated session elements. Although very little research has been conducted on repetition, it is likely that the impact of repeating sessions or specific treatment elements vary widely based on individual or population needs. Practice-based research data, in conjunction with measures of fidelity to assess whether the repetition comes at the expense of other key treatment elements, may be useful for exploring outcomes associated with repetition of protocol elements

Adaptations to protocol and session length

Adaptations to protocol and session length include increases or decreases in the number of sessions in the EBPT protocol (often without adding or removing treatment components, although those studies that did so are noted below). In addition to studies that shortened or lengthened the protocol, two studies allowed a variable protocol length, meaning protocols could be shortened or lengthened. Eleven studies shortened sessions, and five examined lengthened sessions in the context of a shortened, intensive protocol. One study condensed the protocol (lengthened sessions and completed the protocol in fewer sessions) to accommodate the distance that participants travelled for the study (Murray et al., 2015). Descriptions of these studies and their effect sizes can be found in the supplemental table. Notably, the adapted studies did not vary the treatment length by more than 50% of the sessions in the original protocol; that is, the studies did not investigate highly abbreviated protocols or long-term EBPT protocols. Conversely, many individuals who receive psychotherapy in routine care settings receive far fewer sessions than specified in EBPT protocols (Spoont et al., 2014) or engage in longer-term care, even when receiving treatments that are conceptualized as more structured, short-term therapies. Thus, further investigation of the impact of EBPTs delivered in timeframes and contexts that are typical of routine care is needed to better understand the magnitude of the benefits that can be expected under these circumstances. Below, we describe conclusions that can be drawn based on different approaches to adjusting protocol length.

Variable protocol length

Although in routine care settings, the number of sessions attended varies widely (Connolly-Gibbons et al., 2011), few studies have allowed flexibility in terms of session length. Only one randomized controlled trial examined the efficacy of a variable length intervention for PTSD and compared it to a symptom monitoring condition (MCPT; Galovski, Blain, Mott, Elwood, & Houle, 2012; N = 100). In this study, clinicians determined when treatment should end based on each individual's progress toward a priori defined end-state criteria, and engaged in shared decision making with patients regarding termination. An earlier open trial of STAIR for PTSD (2007) similarly allowed clinical latitude in the number of sessions, but in contrast to Galovski and colleagues' (2012) study, investigators allowed not only non-protocol sessions (which were also allowed by Galovski et al. 2012), but also the repetition and removal of sessions and materials. In both studies, large pre-to-post effects were demonstrated. Together, results suggested that in the context of EBPT protocols for PTSD, patients recover at differing rates, and that flexibility in the number of sessions offered may be appropriate. Further research on EBPTs for other disorders, and on characteristics of patients who may benefit from adapted lengths, will also be useful in guiding clinical practice.

Shortening the protocol

Shortening the protocol refers to reducing the number of sessions that are delivered. A review on brief psychotherapies for depression included a meta-analysis on five studies of brief CBT (Nieuwsma et al., 2012). Another meta-analysis examined studies of brief CBT and problem-solving therapy (Cape et al., 2010) for depression and anxiety disorders in primary care. Seventeen additional studies examined shortened EBPT protocols, but none compared them to standard protocols. Three studies investigated abbreviated DBT protocols using a comparison group (Katz et al., 2004; Rathus & Miller, 2002) or an open trial (Salbach-Andrae et al., 2008). Only four studies did not include other adaptations. For example, in addition to tailoring, some studies increased the frequency of sessions or shortened the sessions. One small open trial demonstrated a large effect for shortened CBT for panic disorder and agoraphobia, although it did not include a control condition or a benchmarking strategy (Westling & Öst, 1999).

Several studies investigated abbreviated treatments for depression. Results of the meta-analyses suggested small effects for studies of brief CBT or PST for depression or for mixed depression and anxiety (Cape et al., 2010; Nieuwsma et al., 2012), but larger effects on brief CBT for anxiety disorders delivered in primary care settings (Cape et al., 2010). Some of the PST studies also shortened the duration of the therapy sessions. These studies did not include comparisons to standard-length protocols, although authors noted that the effects for depression were smaller than those found for full-length protocols (Nieuwsma et al., 2012). Not included in these meta-analyses was a study on MBCT, which demonstrated a large effect compared to usual care (Barnhofer et al., 2009). Although many of the shortened protocols examined in this study did not remove treatment elements, it is important to note that different outcomes may occur depending on whether shortening is accomplished by removing some session materials as opposed to compressing all EBPT elements into fewer sessions. Although results of the meta-analysis on removing treatment components indicate that some treatment components may be removed without detrimental effects on clinical outcomes (Bell et al., 2013), the combined impact of removing elements and shortening protocols may differ. Thus, additional research comparing standard to adapted interventions, and considering whether treatment is compressed or elements are removed to accomplish shortening of protocols, is recommended. Decisions to shorten protocols in routine care should be made in conjunction with real-time data on individual symptoms and functioning.

Lengthening the protocol

Lengthening refers to the addition of more sessions to an EBPT. Seven studies investigated lengthened interventions, but all studies that adapted protocol length included additional adaptations. One RCT (Schulte et al., 1992) lengthened a protocol for specific phobias to accommodate the addition of other treatment elements that could be added at clinicians' discretion. Contrary to the hypothesis that more flexible delivery would result in better outcomes, a medium effect in favor of the standard protocol compared to the adapted one was found. Despite the addition of sessions to accommodate clinical latitude in delivering additional cognitive behavioral elements, the results suggest that elements required in the protocol may not have been delivered at an adequate dose or intensity. This stands in contrast to the findings from Galovski and colleagues (2012), which suggest that some patients experience additional benefits when a protocol is lengthened and care is taken to deliver an adequate dose of the key elements of a therapy.

Variations in treatment length are common in routine care settings due to a variety of factors, such as insurance reimbursement policies, program capacity, and patient needs. Designs that isolate the impact of changing the length of a protocol can provide clinically useful information. The results provided by Galovksi et al (2012), which described the proportion of patients who achieved remission and good end-state functioning after a shortened, standard, or lengthened protocol, are a good model for developing processes to make decisions about protocol length, and for providing results that can guide treatment planning.

Shortening Session Duration

Our review identified 12 studies that examined the impact of shortening the length of sessions. In eight studies for depression, which were included in a meta-analysis (Nieuwsma et al., 2012), the sessions were shortened in the context of abbreviated treatment protocols. Only one study that we identified (Nacasch et al., 2014) was a randomized comparison to the standard protocol. In the studies we reviewed, sessions were shortened for different reasons that may have different clinical implications. For example, sessions were shortened in the van Minnen (2006) study due to feasibility related to restrictions on reimbursement for 90 minute sessions. The same research question was examined in a more recent, randomized non-inferiority study (Nacasch et al., 2014), and together, these studies suggest that the benefits of imaginal exposure for PTSD may be experienced even in abbreviated sessions. Charlton and Dykstra's (2011) study shortened sessions to address feasibility of delivering DBT to a population with intellectual disabilities and included additional adaptations, but was not designed to determine the relative benefits of this approach to as compared to a longer session. Studies that examine the impact of abbreviated sessions have important implications for routine care outpatient settings, where clinicians are rarely able to provide or bill for over an hour with individual patients. Thus, particularly when an intervention specified a session length that exceeds an hour, studies of shortened sessions would be highly clinically relevant, and non-inferiority studies could be conducted to determine whether shortening results in inferior outcomes to the long sessions required in standard protocols.

Lengthening sessions and compressing the protocol (Intensive Protocols)

Some studies have abbreviated the number of days or weeks over which a protocol is delivered to provide an intensive EBPT. Notably, all of the intensive interventions preserved all core elements of CBT. Following a promising small feasibility study (Ehlers et al., 2010), Ehlers and colleagues (2014) conducted an RCT comparing a standard cognitive therapy for PTSD protocol and an emotion-focused therapy condition to an intensive, adapted 7-day protocol with two, 90-120 minute sessions per day (as opposed to one 60-minute session per week). Intensive cognitive therapy achieved faster symptom reduction and comparable overall outcomes to standard cognitive therapy, with both protocols outperforming supportive therapy. These findings suggest that intensive treatments for PTSD can be beneficial, although replication is necessary. Additionally, research has been conducted on intensive EBPT protocols for anxiety disorders. One study compared weekly and intensive CBT for pediatric obsessive compulsive disorder and found a small-to medium effect in favor of the adapted condition, although the difference between groups was not significant (Storch et al., 2007). Studies of panic disorder also demonstrated large reductions in panic symptoms, with one study demonstrating effects that were similar to those demonstrated in previous RCTs (Deacon & Abramowitz, 2006; Evans, Holt, & Oei, 1991). One RCT also demonstrated that results of an intensive, group-based CBT for social anxiety resulted in smaller effects than that seen in an individual CBT comparison, possibly because the group format made it more difficult to tailor exposures to individual patients (Mörtberg et al., 2007). An additional limitation to the study by Mörtberg and colleagues is that it relied on self-report, rather than interviewer-assessed outcomes. Finally, an open trial/program evaluation in a partial hospitalization program also provided preliminary evidence of benefits resulting from a CBT for depression protocol that was adapted for intensive treatment, but it similarly relied on program evaluation data (Christopher, Jacob, Neuhaus, Neary, & Fiola, 2009).

Although most of these results suggest that intensive interventions can result in symptom change that is similar in magnitude to standard protocols, in some cases, further replication is needed. In the case of social anxiety, further study is needed to determine whether an intensive individual intervention could overcome the shortcomings of the group-based intensive protocol that has been tested. Additionally, it might be useful to identify characteristics of patients who are most likely to need or benefit from intensive protocols in future research. Implications of these findings for routine care in outpatient settings are not certain, as some clinicians may lack the flexibility to schedule individuals for longer, more frequent sessions (Stirman, 2014). However, they could represent a promising approach for inpatient or intensive day programs.

General Discussion and Recommendations

Although previous reviews have examined the impact of certain forms of modification or considered the relationship between flexibility and fidelity in the delivery of EBPTs, to date, there has not been an examination of the impact of a variety of forms of modifications to EBPTs, or a critical review of the research methodologies used to investigate their impact. The purpose of this systematic review was therefore to describe the design and findings of the current empirical literature on adaptations to EBPTs, to identify gaps in the literature, to discuss methodological considerations, and make recommendations for future research for the study of modifications to EBPTs. Additionally, when possible, based on study findings, a goal of this review was to make recommendations regarding adaptation when implementing EBPTs in clinical practice settings. Most open trials and comparisons of adapted protocols demonstrated feasibility and symptom improvements, but it is important to note that “file drawer” studies suggesting negative results for adaptations may not have been published, potentially resulting in an overly optimistic assessment of the benefits of delivering adapted EBPT protocols.

Although we had originally planned to conduct a random-effects meta-analysis to examine the impact of different forms of adaptation, were unable to do so due to the dearth of controlled studies that were designed to compare adapted and non-adapted protocols for adaptations other than shortening, adding, and removing, which had already been examined in recent meta-analyses. However, we are able to draw several conclusions that should be considered both when making choices to adapt EBPTs in routine care and when designing future research. With few exceptions, the research that we reviewed suggested that adapted protocols, when compared to the original format, yielded small, if any effects. With respect to specific adaptations, the studies reviewed here and in Bell, Marcus, and Goodland's (2013) previous meta-analysis suggest that when additions are discrete, well-defined, and based on sound theory and an understanding of the population for which the intervention is being adapted, they may result in better outcomes than standard protocols. Although most studies demonstrated benefits to patients who received tailored interventions, few tested or demonstrated benefits over and above standard protocols, and most such studies comprised small sample sizes. Large effects, comparable to original protocols, were typically seen for EBPTs that were adapted for intensive delivery, although intensive group-based interventions resulted in a magnitude of change that was lower.

Gaps in the literature

Our review also resulted in the identification of a number of gaps in the existing empirical literature regarding adaptations. Although studies that investigate flexible, modular treatments (Weisz et al., 2012) or circumscribed adaptations (Galovski et al., 2012) are providing important specific guidance about the degree of flexibility with which treatment elements can be applied, it is critical to explicitly examine the impact of other forms of modification that are commonly made in routine care. The research that we identified in this review focused exclusively on adaptations that were planned for the purposes of the study, and as such, some forms of adaptation as they occur in typical practice may not be adequately represented in the literature. A number of different modifications, including integration, substitution of different elements for established EBPT components, loosening session structure, and reordering of EBPT elements have not been adequately investigated, although there is some evidence that these types of changes occur in routine care (Aarons et al., 2012; Cook et al., 2014; Stirman et al., 2013; Lau et al., 2017).

Similarly, although providers who were interviewed in one study indicated that they loosened the structure and occasionally drifted from a cognitive therapy approach before returning to the intervention (Stirman et al., 2013a), no studies have specifically investigated the impact of these changes to treatment protocols. In studies that did allow “non-protocol” sessions for emergent life events, clinicians were instructed to approach issues discussed in session within the general framework specified by the protocol, an approach that may differ from routine care (Guan et al., 2015). The meta-analysis by Bell, Marcus, & Goodland (2013) indicated that removal of certain protocol elements did not impact results, which has implications for implementation in routine care settings, as it may be easier to train clinicians in simplified protocols. However, almost no studies examined the impact of removal of key cognitive behavioral elements such as homework or use of an agenda, which are not always included in routine care settings (Stirman et al., 2013; Thompson et al., 2016).

Other adaptations that are commonly found in routine care settings, such as adapting the length of protocols or sessions and tailoring the intervention to meet the needs of specific populations, have been somewhat better represented in the literature. Many of the adaptations made for specific populations appeared to be made to be in an effort to or to apply treatments to different diagnostic populations than those for whom they were originally developed, or to foster engagement, increase relevance, and enhance outcomes for individuals with specific demographic characteristics. Surprisingly, outside of studies on DBT, relatively few studies investigated adaptations made specifically to address comorbidity. The development and testing of protocols that specifically address commonly co-occurring diagnoses such as PTSD and substance abuse (Foa, Yusko, McLean, & et al., 2013; Kaysen et al., 2014) are important advances in the literature. However, due to the relatively high rates of comorbidity in routine care and questions about how to apply EBPTs when comorbid diagnoses are present, further research on whether, when, and how to adapt protocols to address comorbidity are necessary. Furthermore, studies on modular and transdiagnostic protocols (Barlow et al., 2011; Fairburn et al., 2009; Weisz et al., 2012) can provide critical information on application of cross-cutting EBPT strategies to address multiple psychiatric conditions.

Recommendations for Future Research

Although RCTs are a methodologically rigorous way to examine many questions regarding the effectiveness of adaptation and modifications to EBPT protocols, there are drawbacks to relying solely on this strategy. Utilizing RCTs to investigate the impact of certain individual adaptations, even those common in routine care, may be less efficient and feasible due to the sheer variety of EBPT interventions, populations, and adaptations that exist, and the large sample sizes that may be required to detect meaningful differences in outcomes between adapted and original protocols. It is highly unlikely that many such studies would be funded in the current funding context. Furthermore, available data suggest that clinicians often make multiple adaptations to interventions when delivering them in routine care (Aarons et al., 2012b; Cook et al., 2014a; Stirman et al., 2013a), which make the design of RCTs to determine the unique impact of specific adaptations complex and impractical. Adaptive and factorial clinical trial approaches to clinical trials, such as SMART (Sequential Multiple Assignment Randomized Trial), and Multiphase Optimization Strategy (MOST) designs (Buscemi et al., 2016; Chow, 2014) can also facilitate rigorous evaluation of adaptation (Baumann, Cabassa, & Stirman, in press). SMART designs, also called adaptive designs, involve a sequence of decision rules that operationalize whether, how, or when and how (i.e., based on which measures) the dosage (i.e., frequency, duration and/or amount), type or delivery of treatment are effective for those receiving the intervention. These studies are designed such that participants can be re-randomized at decision points based on their response to the adapted intervention, without compromising the integrity of the study (Almirall & Chronis-Tuscano, 2016; Chow, 2014). The Multiphase Optimization Strategy (MOST) framework involves first identifying strategies to address a specific need, then conducting pilot work before evaluating the overall effect of the intervention (Collins, Murphy, & Strecher, 2007). This design is useful for identifying promising adaptations before conducting a full evaluation. Microtrials can also be used to evaluate the benefits of discrete treatment elements prior to a larger-scale adoption of an adaptation (Leijten et al., 2015). To better examine adaptation as it occurs in routine care, RCTs could compare a standard protocol to a condition in which clinical latitude in making adaptations to a protocol is permitted. The study by Schulte and colleagues (1992) is a good example of a study of this nature, as is a study by Jacobson and colleagues (1989) that was not included because it focused on marital therapy rather than a therapy for a specific disorder. Although other studies gave clinicians latitude in applying a circumscribed set of adaptations (Galovski et al., 2012; Levitt et al., 2007), some of the adaptations that are seen in routine care (Cook et al., 2014b; Stirman et al., 2013a) were not represented in this research. By providing real-time data on adaptations that are made to address issues that are most common in particular clinical settings, practice-based studies and process research can complement guidance provided by RCTs and shed light on the impact of adaptations that occur in routine care (Chambers & Norton, 2016). This information can contribute to an empirical basis for selecting, modifying, or removing strategies described in EBPT protocols. Although some populations may benefit from more strict adherence while others may experience more improvement from adapted protocols (Jacobson et al., 1989; Schulte et al., 1992; Strunk et al., 2010; Williams et al., 2014), practice-based investigations to determine circumstances under which flexibility is and is not indicated would also be clinically useful.

Attention to study design, in practice-based research or future trials, can result in greater clarity regarding the impact of modifications to EBPTs. The types of adaptations that are made should be carefully characterized, because, as illustrated by the different meta-analytic strategies employed by Ahn & Wampold (1999) and Bell, Marcus, and Goodland (2013), the impact of specific adaptations can be obscured when different types are examined together. Even within the different forms of adaptation that we have examined, the nature and content of the adaptations may vary and may have implications. For example, some studies showed that adding response prevention for exposure yielded additional benefits (Hiss, Foa, & Kozak, 1994), and other studies showed that certain treatment elements could be removed without negatively impacting outcomes (Jacobson et al., 1996; Resick et al., 2008). Yet some studies identified treatment elements that could not be removed without impacting outcomes (Barlow, Craske, Cerny, & Klosko, 1989; Craske, Brown, & Barlow, 1991). Removing a relaxation component from a treatment for PTSD, may have a very different impact than removing a cognitive or behavioral component, and yet clinicians appear to be less likely to deliver the exposure and cognitive components (Thompson, Simiola, Schnurr, Stirman, & Cook, 2016; Wilk et al., 2013). Practice-based studies that randomize patients into conditions in which a single element, such as relapse prevention or cognitive restructuring, is added, integrated, or removed can add to the literature on what forms of adaptations are effective in specific populations and contexts, and the findings may be more convincing to clinicians. Furthermore, even when a study is designed to allow for clinical latitude in making adaptations, assessing the types that were made in each session in conjunction with corresponding symptom assessment can facilitate an examination of whether certain types of adaptations result in session-by-session or overall symptom change, using analytic strategies such as latent variable analysis or piecewise regression models.

Designs that allow within-and between-subject comparisons (e.g., ABA designs) can be useful for investigating certain adaptations such as adding, removing, and loosening structure. When the removal of specific aspects of the treatment would have a significant impact on the feasibility of delivery in a particular setting, a non-inferiority study design may also be appropriate. In examining clinical outcomes, benchmarking methodologies are typically more informative than open trials or case series, and in routine care settings, rapid cycle testing strategies can also be employed to inform decisions about whether adaptations should be made in specific contexts (Chambers, Glasgow, & Stange, 2013). Finally, to better guide the process of adaptation in routine care, it is also important to understand whether outcomes differ when changes are carefully planned in advance after examining practice-level data and the existing literature, vs. when they are made in-session, on an individual basis as challenges to EBPT delivery arise (Moore et al., 2013). In contrast to the studies in this review, there is evidence to suggest that in routine care settings, many adaptations are reactive, and made for logistical rather than theoretical reasons (Stirman et al., 2013; Aarons et al., 2012; Cooper et al., 2015). Studies that are designed to allow tests of moderation may also shed light on whether adaptions may be more or less useful for individuals with specific characteristics. The potential moderating influence of factors that might suggest that adaptations are necessary should also be taken into account when designing methods and analysis to examine the impact of adaptations or modifications in process research. The timing of data collection and identification of adaptations should be carefully considered in order to establish temporal precedence of an adaptation when investigating its relationship to changes in symptoms, functioning, or other outcomes, and to facilitate analyses to explore mediation and moderation.

Although large-scale research may not be feasible in every case, it may be the appropriate and necessary strategy to address some questions regarding adaptation and modification. For example, using methodologies to aggregate evidence from different trials would produce large samples and common measures to evaluate what forms of adaptations has worked for whom. Such strategies have been used to examine variations of interventions targeting obesity (Belle et al., 2016; Tate et al., 2016). Other potential strategies include use of dashboards to track adaptations and outcomes metrics, and could support the evaluation of multiple trials or practice-based research on adaptation (Chambers & Norton, 2016; Rith-Najarian, Daleiden, & Chorpita, 2016). Larger scale research, including the pooling of practice-based data collected using common methodologies would allow a better understanding of whether specific patient-related factors that may drive the need to adapt are moderators of outcomes, and whether the adaptations themselves predict outcomes of interest (Baumann et al., in press).

Even if specific differences are not found in terms of symptom change, there may be other contextual reasons to adapt treatments, such as increasing acceptability, feasibility, engagement, accessibility, patient satisfaction, cost and efficiency (Chen, Olin, Stirman, & Kaysen, 2017). Thus, data beyond symptom measurement should be collected in future adaptation studies. For example, therapist and patient satisfaction with the standard and adapted treatments have generally not been reported, but may be greater for adapted interventions. If symptom outcomes are not degraded substantially, such additional consideration may justify adaptations even if their impact on symptoms alone does not. Understanding what outcomes are critical to key stakeholders and assessing those outcomes can better inform implementation of EBPTs in routine care settings.

Despite the importance of examining the existing literature that has investigated the impact of adaptions of EBPTs on patient outcomes, there are several limitations to the current study. The first is the potential for the “file drawer” effect, which may have resulted in a bias towards publication of studies with positive results. We caution against decisions to implement adaptations that have not been more rigorously investigated, particularly without an evaluation plan. Another limitation is the possibility that our search strategy did not capture all relevant studies. Despite our use of systematic review search strategies, some studies that contained adaptations may not have been captured in our search, due to the use of different terminologies, a lack of emphasis on adaptations in the methods sections or abstracts, or insufficient descriptions of the adaptations that precluded accurate coding. Additionally, some primary sources did not provide sufficient information to calculate bias-corrected effect sizes. The number and design of the primary studies that were identified for most adaptations did not allow for the application of rigorous meta-analytic strategies that would yield more firm conclusions about the impact of different forms of adaptation. This study also did not focus on all presenting problems, on non-EBPTs, on preventive interventions, or on interventions such as parenting programs, which have been the focus of other reviews (Baumann et al., 2015). Although some context-level adaptations, such as the delivery of EBPTs in settings other than those for which they were originally tested, were represented because they accompanied the content-level adaptations that we reviewed, the current review does not include a comprehensive review of contextual adaptations. Particularly because the context of treatment delivery can drive adaptation (Aarons et al., 2012a; Aarons et al., 2012b; Chen et al., 2017), studies of this nature can further inform implementation efforts that include adaptations to the setting, mode of delivery, population, and treatment provider.

Conclusions

It is well established in both implementation theory and research that clinicians adapt and modify EBPTs when they use them in routine care settings. This review described limitations in the design of studies to date that make it difficult to determine the implications of most types of adaptations on clinical outcomes, and highlighted gaps in the existing research literature. Although relatively few studies demonstrated clear improvements when adapted protocols were compared to standard protocols, we found little evidence that most adapted protocols were associated with substantial degradation in clinical outcomes. However, few studies investigated adaptations and combinations of adaptations in routine care contexts. Further investigation of clinical latitude, practice-based research, and additional comparisons of adapted and standard EBPTs can provide much-needed guidance on when, for whom, and how adaptations should be made to EBPTs.