How community therapists describe adapting evidence-based practices in sessions for youth: Augmenting to improve fit and reach

Abstract

Aims:

The study sought to 1) characterize the types and frequency of session-level adaptations made to multiple evidence-based practices (EBPs) and 2) identify therapist-, client- and session-level predictors of adaptations.

Method:

Within the community implementation of multiple EBPs, 103 community mental health therapists reported on 731 therapy sessions for 280 clients. Therapists indicated whether they adapted EBPs in specific sessions and described adaptations in open-ended responses. Responses were coded using the Augmenting and Reducing adaptations framework. Multilevel logistic regression analyses examined predictors of Augmenting and Reducing adaptations.

Results:

Therapists reported making adaptations in 59% of sessions. Augmenting adaptations were reported more frequently than Reducing adaptations. Multilevel models showed that greater therapist openness to EBPs, younger child age, and presenting problems were associated with Augmenting adaptations. Child presenting problem of externalizing problems predicted fewer Reducing adaptations compared to internalizing problems.

Conclusion:

This study extends the growing research examining adaptations within the context of system-driven implementation of multiple EBPs by applying the Augmenting and Reducing adaptation framework to the session-level.

Introduction

Despite considerable research on the development, evaluation, and implementation of evidence-based practices (EBPs), EBPs delivered in research settings tend to be less effective when implemented in naturalistic community settings (Weisz et al., 2013). Compared to efficacy trials, community mental health therapists typically serve clients who are lower-income, more ethnically and racially diverse, and have more complex backgrounds and presenting problems (Southam-Gerow, Rodríguez, Chorpita, & Daleiden, 2012; Ozer, Wanis, & Bazell, 2010). When implementing EBPs in real world settings, therapists often report the need to adapt EBPs in order to improve fit for their clients’ particular needs and circumstances (Aarons et al., 2012; Barnett et al., 2018; Lau et al., 2017; Stirman et al., 2013). Given the inevitability of adaptation within community EBP-implementation, Chambers & Norton (2016) have identified the need for an “adaptome,” or a common data platform to classify and capture adaptations made to EBPs when implemented. Intervention developers and implementation researchers can work to optimize fit of EBPs for different contexts by first understanding how community therapists adapt EBPs. Despite the reality that EBPs are often adapted by community clinicians session by session, minimal research has identified the frequency, types, and predictors of EBP adaptations occurring within psychotherapy sessions during implementation as usual in community settings (Marques et al., 2019).

Opinions on the potential benefits of EBP adaptations are mixed. Many therapists report that adaptations are necessary for effective, client-relevant treatment, and in fact believe that adaptations strengthen EBPs (Chu & Kendall, 2009; Lundgren, Amodeo, Cohen, Chassler, & Horowitz, 2011). However, just as adaptations have the potential to improve the impact and durability of treatment, it is also plausible that adaptations may compromise the delivery of essential treatment mechanisms (Drake et al., 2001). Cultural adaptations research provides one such example of this tension; Meta-analyses reflect mixed evidence on the necessity and impact of cultural adaptations, with some citing adaptations as beneficial for clinical outcomes and implementation efforts (e.g., Benish, Quintana, & Wampold, 2011; Degnan et al., 2017; Hall, Ibaraki, Huang, Marti, & Stice, 2016; van Mourik et al., 2017), and others finding no added utility in modifying protocols (e.g., Huey and Polo, 2017; Huey and Polo, 2008; Thomas et al., 2017). Overall, there is continued debate on whether fidelity to manualized therapies should be prioritized over flexibility and local adaptations to the intervention for particular clients or contexts (Hamilton et al., 2008; Owen & Hilsenroth, 2014).

Yet, even with mixed findings on the impact of adaptations from effectiveness trials, therapist-driven adaptations to EBPs are inevitable and need to be characterized (Chambers & Norton, 2016). The field of adaptation science will be greatly enhanced by capturing the breadth of data that is possible within implementation contexts (Chambers & Norton, 2016). In particular, there is a need for adaptations to be codified to allow for cross-EBP and cross-system comparisons. Early studies of EBP adaptations within implementation contexts focused on modifications to a single EBP, with each study developing its own classification of adaptations, thereby limiting the ability to make comparisons across different EBPs. Hill and colleagues’ (2007) qualitative study on community-based EBP program facilitators’ adaptations to the Strengthening Families Program is one such example. The researchers identified 13 types of adaptations pertaining to specific components of that practice that were often adapted (e.g., games, activities, videos, time, etc.). For each of these 13 components, the researchers characterized adaptations as adding, deleting, or changing the component. While informative, this framework illustrates the frequency and nature of adaptations for only one intervention. Stirman and colleagues developed a general yet comprehensive system to characterize the types of adaptations therapists commonly make across a variety of contexts, populations, and interventions (e.g., EBPs for parent training, HIV prevention, substance abuse treatment, PTSD; Stirman, Miller, Toder, & Calloway, 2013). Stirman et al. (2013) coded intervention modifications published in peer-reviewed articles for modifications to content, context, and training and evaluation, identifying 12 types of content modifications made to intervention materials, procedures, and/or delivery.

Expanding upon the Stirman et al. (2013) framework, Lau and colleagues (2017) developed a therapist report measure that inquired about six types of adaptations: (1) modifying the presentation; (2) shortening or condensing the pacing of the practice; (3) lengthening or extending the pacing of the practice; (4) integrating supplemental content or strategies; (5) removing or skipping components; and (6) adjusting the order of sessions or components. A multilevel confirmatory factor analysis extracted two factors: Augmenting (i.e., modifying the presentation, integrating supplemental content or strategies, lengthening or extending the pacing of the practice) and Reducing/Reordering (i.e., removing or skipping components, adjusting the order of sessions or components, shortening or condensing the pacing of the practice) adaptations. In both quantitative and qualitative reports of delivering multiple EBPs in a system-driven implementation in children’s mental health, community therapists reported making Augmenting adaptations to EBPs more frequently than making Reducing/Reordering adaptations (Barnett et al., 2018; Dyson, Chlebowski, & Brookman-Frazee, 2019; Lau et al., 2017). These findings were encouraging, given concerns that therapists may remove core components of EBPs that they consider potentially distressing to clients, such as exposure for anxiety or evidence-based strategies such as role plays (Becker-Haimes, Okamura, et al., 2017). Instead, in these studies, therapists were most often deploying fidelity-consistent, Augmenting adaptations in attempts to boost client engagement, acceptability, and understanding of the practice.

Ultimately, it is important to understand factors at each level that affect the likelihood that community therapists will make different types of adaptations to EBPs in diverse settings. This may have implications for shaping therapist training efforts and implementation supports, and help intervention developers better understand how end users are implementing their products. Existing literature suggests that therapist experience may be associated with aspects of EBP implementation, although results are mixed. One study found that therapists with more experience showed higher adherence to EBP processes and exercises and thus made fewer adaptations, while less experience was associated with increased omission of EBP elements (Taylor et al., 2015). In contrast, other studies have found that therapists with fewer years of experience were more likely to augment EBPs (Lau et al., 2017). Beyond experience level, therapist perceptions of EBPs have been related to adaptations. Therapists who report negative perceptions toward specific EBPs report making more Reducing/Reordering adaptations (Lau et al., 2017), whereas therapists who are more open to EBPs tend to make more adaptations that are consistent with maintaining fidelity (Stirman et al., 2015).

Client characteristics may also elicit therapist adaptations when delivering EBPs. A child’s cultural background, literacy, education or developmental level, and clinical presentation have all been cited as reasons why therapists adapt EBPs (Barnett et al., 2018). However, observer-rated session modifications to a cognitive behavioral therapy protocol for youth anxiety found no significant variations in therapist delivery of the protocol related to youth age, race, or gender (Chu & Kendall, 2009). Thus, it is unclear if these associations will remain when examined for contemporaneous reports of session-by-session adaptations across multiple EBPs. Additionally, aspects of clinical presentation may elicit certain therapist adaptations. For example, therapists spend more time clarifying, restating, and eliciting information from patients with higher symptom severity (Connolly-Gibbons et al., 2003). Concerns about client engagement in sessions may be related to modifications, particularly among clients with more complex presentations and high levels of acuity. The case of emergent life events provides one instance of how in-session engagement maybe related to therapists’ modifications to EBP delivery. Emergent life events were cited as one context in which therapists would defer elements of EBP delivery in order to attend to the crises (Barnett et al., 2018; Guan et al., 2015). Emergent life events have been linked to reduced adherence to planned session activities, yet, are largely addressable using supplemental content (Guan et al., 2019; Guan et al., 2017). Thus, session engagement or lack of engagement may portend therapist decisions to augment treatment with supplemental content such as problem solving.

In sum, past research on therapist adaptations have been limited by their examination of session adaptations within the delivery of singular EBPs (Chu & Kendall, 2000; Chu & Kendall 2009) or reliance on therapist retrospective over multiple sessions or a treatment episode, which may be limited by difficulties with recall and recognition of when adaptations are occurring (Barnett et al., 2018; Lau et al., 2017; Stirman et al., 2015). Research has yet to document types of EBP adaptations at the session-level across multiple EBPs, which is crucial to understanding how session adaptations are made within system-driven EBP implementation efforts. In the current study, we built upon Barnett and colleagues’ (2018) qualitative analysis of interviews that elucidated therapists’ general explanations of their adaptations to EBPs and extended this to examine adaptations described at the session level. Our first aim was to characterize the types of adaptations community therapists described making to EBPs at the session level across multiple EBPs for children and youth. Second, we examined the frequency with which diverse types of adaptations were described within treatment sessions. Third, we identified therapist (e.g., licensure status, gender, race, discipline, clinical caseload, broad and specific EBP attitudes), client (e.g., gender, race, age, presenting problem), and session (e.g., session participants, client engagement) characteristics associated with these types of adaptations across multiple EBPs.

Method

Procedures

Participants were drawn from the “In-Depth” component of the Knowledge Exchange on Evidence-Based Practice Sustainment (4KEEPS) Study (Lau & Brookman-Frazee, 2016) investigating sustainment of EBPs implemented within Los Angeles County Department of Mental Health (LACDMH) child mental health services. For this component of the 4KEEPS Study, we enrolled 14 agencies that were contracted or operated by the LACDMH to deliver at least one of six EBPs of interest that received implementation support under the PEI Transformation of Children’s Mental Health Services (2009). Using a community-partnered approach, the study team and community agencies agreed to a six-week timeframe for data collection. Therapists were invited to participate if they delivered at least one of the six EBPs followed in the 4KEEPS parent study: Child-Parent Psychotherapy (CPP), Seeking Safety (SS), Managing and Adapting Practice (MAP), Positive Parenting Program (Triple P), Trauma-Focused Cognitive Behavioral Therapy (TF-CBT), and Cognitive Behavioral Intervention for Trauma in Schools (CBITS). Therapists completed an initial survey about their background, training, and perceptions of each of the aforementioned EBPs that they currently delivered.

Following the initial survey, therapists were asked to identify three clients with whom they were delivering one of the six EBPs. Therapists selected the clients and the sessions to report. Therapists completed three session-specific surveys for each of the three clients (for a maximum of nine session surveys completed per therapist). The three sessions per client did not need to be consecutive sessions. In the session-specific surveys, therapists reported on the EBP delivered in the session, whether there were any engagement challenges present during the session, and any adaptations they made. Initial and session-specific surveys were administered online. All procedures were approved by the Institutional Review Board.

Participants

In the full sample of 731 session surveys, participants were 103 community-based mental health therapists employed by 14 agencies directly operated or contracted by LACDMH. Therapist, client, and session characteristics are presented in Table 1. Therapists were predominantly female (88.24%) and were 34.14 years (SD = 8.86) of age on average. Therapists were ethnically diverse; 55.88% self-identified as Hispanic, 21.57% Non-Hispanic White, and 22.55% another race (e.g., African American, Asian, Pacific Islander). Less than one-fifth of the therapists included were licensed (18.63%). Primary disciplines of therapists included marriage and family therapy (50.98%), social work (33.33%), counseling, clinical, or school psychology (12.75%), and other disciplines (2.94%; e.g., case manager). The child clients represented in the study were an average of 9.77 years old (SD = 3.84), were almost half female (51.25% female), and majority Hispanic (70.71% Hispanic).

Table 1.

Therapist and client characteristics

	M	SD	Range
Therapist Level Characteristics
Therapist age (in years)	34.14	8.86	25–62
Therapist caseload	17.14	6.95	1–44
Years Practicing	4.41	4.40	0–35
EBPAS Openness	3.02	.66	1.5–4
EBPAS Divergence	1.45	.84	0–3.83
Client Level Characteristics
Client age (in years)	9.77	3.84	1–18
	N	%
Therapist Level Variables
Gender
Female	90	88.24
Male	12	11.76
Race/Ethnicity
Non-Hispanic White	22	21.57
Hispanic	57	55.88
African American	8	7.84
Asian Pacific Islander	15	14.71
Licensure
Yes	19	18.63
No	83	81.37
Discipline
Marriage & Family Therapist	52	50.98
Counseling, Clinical, School Psychologist	13	12.75
Social Worker	34	33.33
Other	3	2.94
Education
Below Masters	4	3.92
Masters	87	85.29
Doctoral	11	10.78
Client Level Variables
Race/Ethnicity
Non-Hispanic White	13	4.64
Hispanic	198	70.71
Other	69	24.64
Gender
Female	143	51.25
Male	136	48.75

Measures

Therapist characteristics.

Therapists reported their gender, race/ethnicity, licensure status, number of clients in their clinical caseload, and primary mental health discipline on the initial survey.

General attitudes toward EBPs:

Therapists completed the Evidence-Based Practice Attitudes Scale (EBPAS; Aarons, 2004), a 15-item self-report measure of general therapist attitudes toward the adoption of EBPs. The original EBPAS yields a total scale score and four subscales: Appeal, Requirements, Openness, and Divergence. For the current study, we were primarily interested in the Openness and Divergence subscales. Given the context that therapists in our study were all employed in agencies receiving reimbursement for EBP delivery, we did not include the Requirements subscale, which measures therapists’ likelihood of adopting EBPs when required to. We also elected not to administer the Appeal scale which measures the intuitive appeal of EBPs in general, since we assessed perceptions of specific EBPs using the Perceived Characteristics of Intervention Scale (PCIS) described below. The four-item EBPAS Openness subscale assesses the therapist’s openness to trying new interventions. It includes items such as “I like to use new types of therapy/interventions to help my clients.” The four-item EBPAS Divergence subscale assesses the therapist’s perception of EBPs as divergent from their own approach to clinical practice. It includes items such as “Clinical experience is more important than using manualized therapy/interventions.” Therapists rated their agreement with each item on a 5-point Likert scale (0 = not at all, 4 = very great extent). Mean scores for Openness and Divergence were used in analyses. In the current sample, the Cronbach’s alpha indicated that the internal consistency was good for the Openness (α = .80) and Divergence subscales (α = .70).

Perceptions of the EBP being delivered.

The Perceived Characteristics of Intervention Scale (PCIS; Cook et al., 2014) measures therapist attitudes towards specific EBPs. In the original PCIS, 20 items were developed to measure the theory-based model of 10 perceived characteristics of innovation (Rogers, 1962; Rogers, 2003; Greenhalgh et al., 2005), which may impact therapists’ attitudes towards and uptake of different EBPs. Psychometric properties indicated that the scale measured a unidimensional construct of attitudes (Cook et al., 2015). Thus, to reduce measurement fatigue given the repeated nature of this measure, we administered eight items tapping four dimensions included in the original PCIS (e.g., Barnett et al., 2017; Lau et al., 2017). Eight items were included representing four concepts: relative advantage (e.g., “[The practice] is more effective than other therapies I have used.”), compatibility (e.g., “[The practice] is aligned with my clinical judgment.”), complexity (e.g., “[The practice] is easy to use.”), and potential for reinvention (e.g., “[The practice] can be adapted to meet the needs of my patients.”). Therapists rated their agreement with each item on a 5-point Likert scale (1 = not at all, 5 = a very great extent) for every EBP they had ever delivered. Therapist responses to each item were averaged to create an index of therapists’ practice-specific attitudes, with higher scores indicating greater acceptability of the practice to the therapist. The scale showed strong internal reliability within our sample for each practice (α_MAP = .90, α_SS = .93, α_TF-CBT = .95, α_CBITS = .99, α_Triple-P = .96, α_CPP = .93).

Client characteristics.

At the time of the initial survey, therapists reported on the client’s age, gender and race/ethnicity.

Session characteristics.

Therapists reported on the specific practice (e.g., CPP, MAP, TF-CBT) delivered, people involved during session (i.e., youth only, caregiver only, both youth and caregiver present), and the presenting problem targeted during the session.

Session engagement challenges.

Within the session survey, therapists completed a checklist indicating whether client-related engagement challenges were encountered. Items assessing engagement challenges were adapted from the two-month retrospective report measure of client engagement challenges for an index case described in a previous study (Lau et al., 2018). In the current study, therapists reported on four items to indicate whether each engagement challenge had occurred. The Limited Engagement in Therapy Activities subscale included two items: “Demonstrated apathetic or disinterested behavior” and “Avoided participating in therapy activities.” The Expressed Client Concerns subscale included two items: “Expressed about the relevance/acceptability/helpfulness of an intervention strategy” and “Expressed difficulty mastering skills presented in therapy.” The reliability of the engagement challenges items and subscales were previously supported through analyses of the prospective correlations between mean occurrence of therapist-reported engagement challenges across sessions with a given client over a study period, and their subsequent retrospective reports of challenges encountered over the previous two-month period with that client (Gellatly et al., 2019).

Session adaptations.

The session survey prompted therapists to check “Yes” or “No” to the following question: “In this session, did you adapt [PRACTICE] for this client?” Therapists who reported “Yes” were asked to provide an open-ended response describing how they adapted the practice within the session. Given the first study aim to capture and codify all therapist-perceived session adaptations, we did not define “adapt” for therapists, nor did we provide prompts or examples of how to describe the adaptations within the open-ended response section (see discussion for implications and possible limitations based on this approach). Open-ended responses were coded into categories using the process detailed below.

Coding manual.

The Session-Level Adaptations Coding Manual provided instructions for classifying therapist descriptions of adaptations from the surveys. Adaptation categories in the manual were based on the framework developed in Lau et al. (2017), as well as emergent themes documented in a qualitative analysis of therapist descriptions of adaptations they made to EBPs delivered with clients in the past two months (Barnett et al. 2018). The coding manual provides specific criteria for categorizing therapist-reported session-level adaptations into 13 distinct categories. The coding manual also provides support for identifying Augmenting adaptations, Reducing adaptations, and Generalizing adaptations. Augmenting adaptations include instances in which therapists reported: (1) modifying the presentation of the practice (using activities, such as art or games, or different formats [e.g., visual cues, stories, videos] to present session topics; simplifying or modifying terminology/language used), (2) integrating supplemental content or strategies (utilizing content from other EBPs or incorporating additional content that was not specified as part of the EBP delivered in the session), (3) repeating components (repeating content or strategies), (4) providing psychoeducation (providing information/rationale about illness, treatment, and prognosis), (5) lengthening the pacing of the practice (spending more time on content/strategies or increasing the session length), (6) translating materials (translating materials or parts of the session into another language), and (7) combining the practice with other services (combining the EBP with services other than psychotherapy such as case management). Reducing adaptations include: (1) pausing EBP delivery (pausing the delivery of EBP elements to shift attention to a client-raised or therapist-perceived issue such as an emergent life event), (2) removing components (removing or skipping specific content from the session), (3) adjusting the order of sessions or components (conducting sessions or delivering content/strategies in an order other than what the treatment manual recommends), and (4) shortening the pacing of the practice (decreasing the time spent on content/strategies; shortening or condensing the length of the session). Generalizing adaptations include: (1) applying the practice in alternate settings, or with alternate individuals (delivering treatment in different settings or with recipients other than typically intended), and (2) applying the practice to a novel problem focus (applying the treatment for a problem focus other than originally intended). Two adaptation codes (pausing EBP delivery and applying the practice in alternate settings) were not originally included in the coding manual. They emerged during the coding process as they were frequently cited by therapists. Sessions already coded were re-coded for possible presence of the two additional codes and marked accordingly. Examples of therapist descriptions representative of each adaptation type from the session surveys are presented in Table 2.

Table 2.

Adaptation codes, frequency, and sample therapist write-in response

	% of adapted sessions (N=429)	% of total sessions (N=731)	Sample therapist write-in response
Augmenting adaptations	63.40%	37.21%
Modifying presentation	35.20%	20.66%	“Client presents with language problems and I apply the use of visuals for client to identify and place meaning to topic discussed.”
Integrating supplemental content/strategies	17.72%	10.40%	“Therapist added mindfulness reading into session due to previous statements by [client] that she is trying to think positive and push away her ‘bad thoughts’ and emotions.”
Repeating components	8.62%	5.06%	“Client is very young, had difficulty focusing on tasks, staying on topic and comprehending the practice skills discussed. Clinician had to go over skills several times, practice, model and work on focusing and client still appeared confused at times and disinterested.”
Providing psychoeducation	4.66%	2.74%	“Added psychoeducation on trauma.”
Lengthening pacing of practice	3.50%	2.05%	“I am doing the trauma narrative more slowly to help client feel comfortable.”
Translating materials	1.86%	1.09%	“Translating into Spanish and facilitating collateral with clients [biological mother] and maternal grandmother who, is another caregiver for client.”
Combining with other services	1.17%	.68%	“This week, I and the therapist of my client’s sibling met jointly with caregivers in order to do very simple, concrete safety planning regarding physical aggression and sexual behaviors in the home.”
Reducing adaptations	19.58%	11.49%
Pausing EBP delivery	8.86%	5.20%	“Allowing client to talk about bio father due to her father not visiting client this week, which heavily impacted client. Therapist also used this session to work on grief and loss to deal with client’s bereavement, which was not focus of MAP.”
Removing components	6.29%	3.69%	“Caregiver has previously received parenting classes so she is somewhat familiar with interventions and tools. I did not start at the basics of these interventions because of this. I felt it was more effective to start in the middle and build off caregiver skills she currently has.”
Adjusting order of practice/components	3.03%	1.78%	“I used some of the PRAC components out of order, picking and choosing the activities related to feelings, as that is what we had been working on prior to this session.”
Shortening pacing of practice	2.56%	1.50%	“Therapist had to conduct the session in a shorter time due to father’s hesitation in the past to participate in collateral sessions.”
Generalizing adaptations	13.05%	7.66%
Applying to alternate setting or individuals	10.72%	6.29%	“There were several adaptations that had to be made during this session mainly that we were meeting outside since the client is on spring break and cannot meet at school as we do normally. The parent was not at home so the session was conducted only with the client.”
Applying to novel problem focus	2.33%	1.37%	“Seeking Safety [was] adapted to target mood disruption and anxiety, as this has previously resulted in self-harm behavior, rather than targeting substance use.”
Non-classifiable	13.29%	7.80%
Not enough information to code	11.89%	6.98%	“I addressed client’s needs while in treatment regarding his irritability. He refused to talk for a period of time.”
No text write-in	1.40%	.82%

Coder training.

Coders were three undergraduate research assistants who participated in group didactic sessions to be trained on the Session-Level Adaptations Coding Manual. Coders categorized sample therapist write-in responses pre-selected for coder training. Undergraduate coders’ categorizations were compared with the categorizations reached through study team discussion and consensus. Coders reached Cohen’s Kappa of 0.65 (p < .05), indicating good agreement (Cicchetti, 1994), before starting independent coding.

Coding procedure.

For each therapist adaptation description, coders assessed the occurrence or non-occurrence of each adaptation type described above (e.g., modify presentation, translate materials, pause EBP delivery). Coders were instructed to code explicit descriptions of adaptations without drawing inferences about the types of adaptations made. For instance, therapists had to state that they lengthened pacing, omitted sections, or reordered sessions for the response to be classified as such. In cases where a therapist reported more than one adaptation within the session survey, each adaptation explicitly described was coded (e.g., a single response could be coded as both lengthen pacing and provide psychoeducation). Write-in responses for which undergraduate coders were unable to categorize within a specific adaptation code were further examined by master coders. Master coders were two doctoral-level graduate students with extensive training in and knowledge of the adaptation literature. Master coders independently examined and coded these responses and then met to establish consensus for each.

Reliability.

In order to assess interrater reliability, 20% of the adaptation write-in responses were double-coded and Cohen’s kappas were calculated. Kappas for all specific codes were in the acceptable range (among Augmenting adaptation codes: κ_Mean = .59; among Reducing adaptation codes: κ_Mean = .58; among Generalizing adaptation codes: κ_Mean = .62). Kappas of higher order codes (e.g., Augmenting, Reducing, Generalizing adaptations instead of specific subcodes) indicated that inter-rater reliability was moderate to strong for any Augmenting (κ = .83), any Reducing (κ = .60), and any Generalizing adaptations (κ = .59). Kappas were judged against widely used guidelines (Cohen, 1960; Landis & Koch, 1997; Rietveld & van Hout, 1993). We used higher order codes in study analyses given the higher kappa values and for model parsimony.

Analytic Plan

First, dichotomous index variables were computed in order to note the occurrence of any Augmenting adaptations and any Reducing adaptations within a given session. That is, if one of the seven Augmenting adaptation types was coded for a session, the session received an index score of “1” for Augmenting adaptations for that session. If none of the seven Augmenting adaptation types was coded for a session, the session received an index score of “0” for Augmenting adaptations for that session. The same process was used to create an index variable for Reducing adaptations. We then examined the frequency of reported Augmenting adaptations, Reducing adaptations, and Generalizing adaptations, including the breakdown of frequency of codes within each of the categories.

Finally, we conducted two three-level logistic regressions (sessions nested within clients nested within therapists), in order to examine therapist-, client-, and session-level characteristics that may be associated with (1) described Augmenting adaptations and (2) described Reducing adaptations, using the “melogit” command in Stata 14.2 (College Station, TX). The models compare the likelihood of described Augmenting adaptations and Reducing adaptations with the base outcome of No Augmenting adaptations and No Reducing adaptations, respectively. Logistic regressions controlled for the practice delivered in the respective treatment session. Sessions with missing data at levels 2 and 3 were dropped from analyses, which is the convention for multilevel logistic regression using Stata.¹

Results

Adaptation Type Frequencies

As presented in Figure 1, therapists provided 731 session surveys. Within these, therapists reported making an adaptation in 429 sessions and not making an adaptation in 300 sessions (N_missing = 2). Among the sessions in which therapists reported an adaptation, at least one Augmenting adaptation was reported in 272 sessions, at least one Reducing adaptation was coded in 84 sessions, and at least one Generalizing adaptation in 56 sessions. Thirty sessions were coded for both Augmenting adaptations and Reducing adaptations. In 51 sessions, not enough information was provided in the therapist description to classify the adaptation.

Figure 1.

Flow chart of sessions and therapist-reported adaptations

Table 2 presents how frequently each of the adaptation codes was observed in the current study by percent of sessions with a therapist-reported adaptation and percent of total sessions. Within Augmenting adaptations, therapists reported modifying presentation in 151 sessions (35.20% of adapted sessions), integrating supplemental content/strategies in 76 sessions (17.72% of adapted sessions), repeating components in 37 sessions (8.62% of adapted sessions), providing psychoeducation in 20 sessions (4.66% of adapted sessions), lengthening the pacing of the practice in 15 sessions (3.5% of adapted sessions), translating materials in 8 sessions (1.86% of adapted sessions), and combining with other services in 5 sessions (1.17% of adapted sessions).

Amongst the reports of Reducing adaptations, pausing EBP delivery was observed in 38 sessions (8.86% of adapted sessions), removing components in 27 sessions (6.29% of adapted sessions), Adjusting the order of the practice/components was described in 13 sessions (3.03% of adapted sessions), and shortening the pacing of the practice in 11 sessions (2.56% of adapted sessions).

Among the Generalizing adaptations, therapists described applying the practice to alternate settings or alternate individuals in 46 sessions (10.72% of adapted sessions) and applying the practice to a novel problem focus in 10 sessions (2.33% of adapted sessions). Fifty-seven write-in responses from the session surveys could not be classified due to missing descriptions (N = 6) and not providing enough information to support categorization of the adaptation (N = 51).

Therapist, Client, and Session Characteristics Associated with Augmenting Adaptations

Table 3 presents the results of the multilevel logistic regressions predicting Augmenting adaptations and Reducing adaptations. The three-level logistic regression revealed that of all the therapist level characteristics examined, only therapists’ higher scores on Openness to EBPs was related to increased likelihood of reporting an Augmenting adaptation (OR = 1.79, 95% CI = 1.11–2.88). Therapist licensure status (OR = 1.03, 95% CI = .49–2.17), gender (OR = 1.26, 95% CI =.47–3.40), race (OR_Hispanic = .64, 95% CI = .30–1.34; OR_Other = .65, 95% CI = .27–1.56), discipline (OR_Psychology = 1.61, 95% CI = .63–4.13; OR_{Social work} = 1.43, 95% CI = .75–2.71; OR_Other = 1.43, 95% CI = .21–9.72), number of clients on their clinical caseload (OR = .99, 95% CI =.94–1.03), and EBPAS Divergence scores (OR = .91, 95% CI =.63–1.31) were not significantly associated with Augmenting adaptations.

Table 3.

Mixed-effects logistic regression of therapist adaptations predicted by client, therapist, and session-related factors

		Augment Adaptationa (N = 659)		Reducing Adaptationb (N =643)
	nd	OR	SE	OR	SE
Therapist-level predictors
Licensure (Not licensed)	485
Licensed	128	1.03	.39	1.03	.57
Therapist Gender (Female)	542
Male	71	1.26	.64	.49	.43
Therapist Race (Non-Hispanic White)	138
Hispanic	348	.64	.24	.41	.23
Other	127	.65	.29	.77	.48
Discipline (MFT)	336
Counseling, clinical, school psychology	45	1.61	.77	.87	.62
Social Work	215	1.43	.47	1.70	.82
Other	17	1.43	1.40	--	--
Therapist Caseload	--	.99	.02	.94	.03
EBPAS Divergence	--	.91	.17	.77	.23
EBPAS Openness	--	1.79*	.43	1.26	.44
Client-level predictors
Client Gender (Female)	329
Male	290	1.41	.34	1.04	.38
Client Race (Non-Hispanic White)	38
Hispanic	445	1.84	.92	.82	.55
Other	137	1.58	.85	.96	.69
Client Age	--	.87***	.03	1.03	.06
Session-level predictors
Presenting Problem (Internalizing)	278
Externalizing	237	.41**	.12	.39*	.19
Trauma	204	.80	.30	.51	.25
Involvement in Session (Child Only)	422
Caregiver Only	73	1.03	.44	1.79	1.31
Youth & Caregiver	235	.89	.24	2.01†	.78
Practice (MAP)	379
Child-Parent Psychotherapy	51	.28*	.17	.49	.50
Seeking Safety	27	1.33	.88	2.72	2.59
Trauma Focused-CBT	233	.38*	.16	1.57	.89
Triple-P	39	.82	.55	.39	.54
Expressed client concernsc	--	.93	.33	1.10	.45
Limited engagementc	--	1.05	.30	1.59	.65
PCIS of session practice	--	.95	.18	.81	.23
Constant	--	.71	.65	.25	.33

Note. Reference group for categorical variables in parentheses.

^aCompared to base outcome of no Augmenting adaptation described in the session.

^bCompared to base outcome of no Reducing adaptation described in the session.

^cTherapist-report; Coded for No/Yes for at least one occurrence of the engagement challenge during session.

^dNumber of sessions for each group within categorical variables.

^†p < .10,

^*p < .05,

^**p < .01,

^***p < .001

At the client level, therapists were less likely to describe an Augmenting adaptation in a session with older child clients (OR = .87, 95% CI = .80-.94). Client race (OR_Hispanic = 1.84, 95% CI = .69–4.91; OR_Other = 1.58, 95% CI = .55–4.55) and gender (OR = 1.41, 95% CI = .88–2.26) were not significantly associated with differential likelihood of Augmenting adaptations.

At the session level, when the presenting problem being addressed was externalizing problems, therapists were .41 times less likely to describe Augmenting adaptations compared to when the presenting problem was an internalizing problem (OR = .41, 95% CI = .23-.73). There was no significant difference between when the presenting problem was a trauma versus the reference group of internalizing problems (OR = .80, 95% CI = .38–1.67). There was no significant change in likelihood for Augmenting adaptations based on session participants (OR_{caregiver-only sessions} = 1.03, 95% CI = .44–2.38; OR_{joint sessions} = .89, 95% CI = .53–1.51), therapist report of expressed client concerns (OR = .93, 95% CI = .54–1.61) or limited engagement (OR = 1.05, 95% CI = .60–1.85), or therapist perceptions of the EBP being delivered (OR = .95, 95% CI = .65–1.39).

Therapist, Client, and Session Characteristics Associated with Reducing Adaptations

As presented in Table 3, the three-level logistic regression revealed that no therapist level characteristics examined in the model (e.g., licensure status [OR = 1.03, 95% CI = .35–3.03], gender [OR = .49, 95% CI =.09–2.74], race [OR_Hispanic = .41, 95% CI =.14–1.23; OR_Other = .77, 95% CI =.23–2.61], discipline [OR_Psychology = .87, 95% CI = .22–3.48; OR_{Social work} = 1.70, 95% CI = .66–4.38], clinical caseload [OR = .94, 95% CI =.88–1.01], EBP Openness [OR = 1.26, 95% CI = .63–2.53] or Divergence [OR = .77, 95% CI =.43–1.37]) were associated with differential likelihood of Reducing adaptations.

At the client level, client gender (OR = 1.04, 95% CI = .51–2.13), race (OR_Hispanic = .82, 95% CI = .22–3.04; OR_Other = .96, 95% CI = .23–3.90), and age (OR = 1.03, 95% CI = .92–1.16) were not significantly associated with a greater or lower likelihood of Reducing adaptations.

At the session level, therapists were significantly less likely to describe Reducing adaptations when the session presenting problem was reported as an externalizing problem (OR = .39, 95% CI =.15-.99) compared to the reference group of internalizing problems. There was no significant difference between when the presenting problem was a trauma versus the reference group of internalizing problems (OR = .51, 95% CI = .20–1.35). Sessions jointly attended by the youth client and caregiver were marginally associated with increased likelihood of Reducing adaptations compared with youth only sessions (OR = 2.01, 95% CI =.93–4.31). Therapist report of client expressed concerns (OR = 1.10, 95% CI =.49–2.45), limited engagement (OR = 1.59, 95% CI =.71–3.56), and therapist perceptions of the specific EBP delivered in the session (OR = .81, 95% CI =.46–1.42) were not significantly associated with Reducing adaptations.

Discussion

Despite the reality that adaptations are regularly made when delivering EBPs (Barnett et al., 2018; Lau et al., 2017; Stern Alaggia, Watson, & Morton, 2008), we know little about the types of adaptations that are made within the microcosm of a unique session. Nor do we know much about adaptations made against the backdrop of multiple EBP delivery instead of single protocol efficacy trials. The current study codifies and provides a topography of the types of adaptations community therapists describe making in sessions delivering an EBP to youth. This endeavor may contribute to the development of the “adaptome” data commons capturing and classifying adaptations made across EBPs (Chambers and Norton, 2016). By identifying variants of therapist-driven adaptations and extending a common taxonomy on adaptations, the current study provides practice-based inputs to the science of adaptation, that in turn may support the development of interventions and implementation strategies in service of improved practice. We found that when therapists endorsed adapting a session, they most often augmented the EBP by modifying presentation of the practice and integrating components, and other Augmenting adaptations occurred in less than 10% of adapted sessions. Therapists less often described Reducing adaptations. The majority of Reducing adaptations involved pausing EBP delivery to attend to a client-raised or therapist-perceived concern, such as an emergent life event or to support client engagement. Predictive models revealed that therapists’ openness to EBPs and younger client age were associated with increased likelihood of Augmenting adaptations. Externalizing problems were associated with decreased likelihood of both Augmenting adaptations and Reducing adaptations compared to the base outcome of no Augmenting or Reducing adaptation, respectively.

Topography of adaptation types

Consistent with previous studies examining “typical adaptations” and retrospective reports of adaptations, when therapists contemporaneously reported making an adaptation to the practice delivered, they described Augmenting adaptations more than three times as often as they described Reducing (Barnett et al., 2018; Dyson, Chlebowski, & Brookman-Frazee, 2019; Lau et al., 2017). Among the types of Augmenting adaptations reported in the current study, therapists most often described modifying the presentation of the treatment delivery, followed by integrating components, and then repeating components. During semi-structured interviews with the same therapist sample, therapists described Augmenting treatment, as well as making adaptations broadly, in order to improve the fit of the EBP for the clients they are serving, citing culture, client/caregiver literacy, and client developmental level as common contexts for adaptation (Barnett et al., 2018). The results of our analysis provide some supporting evidence for context prompting Augmenting adaptations. Therapists in the current study were more likely to Augment than not when they reported greater openness to EBPs. They were less likely to Augment than not when the client was older, and when the session focus was on externalizing problems compared to internalizing problems.

Client-, therapist-, and session-level correlates of adaptation types

Our finding that therapists are more likely to augment sessions with younger clients is consistent with previous studies in which therapists indicated client developmental level as a key determinant in choosing to tailor EBPs (Bagner et al., 2016; Bagner et al., 2013; Barnett et al., 2018). Younger clients may require more tailoring of intervention elements and activities for their level of language ability (e.g., modifying vocabulary used), literacy (e.g., using visuals in lieu of worksheets with text), and cognitive development. Therapists’ perceived need to augment materials likely decreases as the age of their client increases. Anecdotally, therapists often cited “developmental level” and younger age as reasons for modifying the presentation of the practice in the write-ins. However, the session survey did not explicitly ask therapists to provide a rationale for their adaptations, prohibiting us from identifying significant associations between Augmenting adaptations and developmental context in the current study.

Rates of Augmenting and Reducing adaptations did not significantly differ by client gender. Previous studies have suggested that boys have higher rates of disruptive behaviors, which may lead to more disruptions in treatment sessions, difficulty with client engagement, and derailed treatment delivery (Gellatly et al., 2018). Our analyses controlled for expressed client concerns and limited client engagement behaviors in session, which may explain why we did not see gender emerge as a client-level factor associated with adaptation.

Modeling therapist and practice characteristics associated with adaptations also provided interesting insights. For one, we found that therapists who were more broadly open and accepting of EBPs were almost twice as likely to describe Augmenting adaptations. Openness to EBPs was not significantly associated with Reducing adaptations. It is plausible that openness to EBPs is underscored by therapist engagement with the EBP (Becker, Smith, Jensen-Doss, 2013), which in turn supports therapists lengthening the practice and repeating components. Colloquially speaking, liking EBPs in general may portend therapists delivering practices to a higher intensity or frequency within the session. However, unlike previous examinations of therapists’ broad reports of adaptations (i.e., not session-specific), therapists’ perceptions of the utility, compatibility, and appeal of the specific EBP being delivered were not significantly associated with differential likelihood of Augmenting adaptations (Lau et al., 2017). It is heartening to see that neither therapist perceptions of EBPs at large nor their EBP-specific attitudes were associated with Reducing adaptations, which may more frequently indicate fidelity-inconsistent modifications such as removing elements of the EBP and loosening the EBP structure (Stirman et al., 2015). Therapist attitudes and perceptions toward EBPs remain important for many reasons associated with training, implementation, and sustainment, but they may not be robustly related to EBP adaptations that involve pausing, removing components, or otherwise reducing dose. More research is needed to understand the implications of therapist reported Reducing adaptations at the session level for overall EBP integrity.

In prior examinations with the current therapist sample, therapists’ reports of limited client engagement and expressed client concerns were negatively associated with therapists’ reports of their own ability to carry out intended activities in the same session (Gellatly et al., 2018). That is, when therapists perceived an engagement challenge, they also perceived that the session did not go as planned. Interestingly, we found that whether or not an engagement challenge was observed in the session did not correlate with therapist Augmenting or Reducing the session practice. We did not examine associations between adaptations and treatment plans or outcomes in the current study. Our findings, combined with those of Gellatly and colleagues using the same sample of therapists, pose an interesting phenomenon in which therapists are not significantly more likely to adapt via Augmenting or Reducing in the face of an engagement challenge, yet therapists report being derailed from planned session activities during such sessions. Perhaps modifying presentation or repeating elements are some ways therapists could prevent sessions derailment. Further examination of these variables may illuminate a need to support therapists to making practice-consistent adaptations that may encourage client engagement and subsequently curtail treatment session derailment.

Limitations and future directions

Limitations of the present study must be considered. The non-random sampling of sessions is one such limitation. Therapists selected the clients and sessions to report on, which introduced the potential for self-selecting sessions in which a therapist perceived that an adaptation did or did not take place. By asking therapists to submit up to three sessions per three clients, we aimed to balance gathering sufficient sample size to elucidate therapist-, client-, and session-level variables associated with adaptation types. Studies with increased numbers of clients and sessions per therapist may provide greater opportunity to examine correlates of adaptations. Future studies using a similar naturalistic design but with random selection of clients and sessions are needed to confirm pattern of adaptations found in the current study. It is worth noting that most sessions reported on by therapists were ones in which they delivered either MAP or TF-CBT and fewer from Seeking Safety, Triple-P, and CPP, which was representative of system-wide patterns of practice delivery within PEI (Brookman-Frazee et al., 2016).

One weakness of the current study was the relatively lower agreement between coders for the Generalizing and Reducing codes compared with Augmenting codes. Although kappas were deemed appropriate using conventionally cited cutoffs, other sources recommend more stringent cutoffs. Relatively lower kappas are one artifact of lower base rates of these codes, but also caution that additional study and replication is required to fully understand the landscape of how therapists adapt sessions in ways outside of Augmenting EBPs.

The session survey design did not provide a definition of adaptations for therapists and instead sought to elicit therapists’ own assessments of what did or did not constitute an adaptation. This method allowed for understanding adaptations from therapists’ perspectives and limited priming therapists for specific adaptation types. This approach permitted comparisons to other researcher-defined adaptation categories from quantitative studies using structured scales (e.g., Dyson, Chlebowski, & Brookman-Frazee, 2019; Lau et al., 2017). However, defining how therapists could have adapted a session may have altered reported frequencies of adaptations overall or for specific adaptation types. Yet, previous studies using survey methods in the current study context overall suggested similar patterns of more frequent Augmenting than Reducing adaptations and also allowed for the study of an additional category of Generalizing adaptations (Lau et al., 2018). Additional studies are needed to replicate findings on relative rates and contextual factors associated with adaptation types.

As coding of adaptations was reliant on therapists’ reports of adaptations that took place, it is certainly plausible that adaptations occurred that were not reported and thereby not captured within the current study. It is also possible that Reducing adaptations are less likely to be captured due to impression management concerns in therapist self-report and selection of sessions for recording. Though the study team was explicit during explanation of study procedures that sessions were not going to be evaluated for fidelity or adherence, it is still possible that selection bias was a factor. Additionally, coding for Reducing adaptations may be difficult to capture within a standalone treatment session since therapists may not report reductions that they postponed with intentions for later delivery. There were also instances in which therapists reported that an adaptation occurred but there was not enough information provided in the fill-in response that allowed for the adaptation to be coded.

A fruitful next step would be to observationally code adaptations, which would provide rich, more objective data on in-session adaptations (Snyder et al., 2006). Akin to fidelity coding, future studies could collect information about planned session activities and then compare them to actual session delivery in order to observationally code for in-session adaptations. These steps would allow researchers to further characterize the context for modifications (e.g., reasons for modifications, if they were planned vs. unplanned), which have been identified as important to capture when classifying adaptations (Stirman, Baumann, & Miller, 2019). Furthermore, it would be valuable to elucidate the relationship between therapist adaptations and fidelity (c.f. FRAME; Stirman, Baumann, & Miller, 2019). Whereas adaptations are often characterized in opposition to fidelity (i.e., fidelity-inconsistent adaptations), adaptations can also be fidelity-consistent, driving increased client buy-in, client engagement, and fit with the client’s specific needs and circumstances (Stirman et al., 2015). In fact, observational ratings of greater fidelity (i.e., adherence and competence) and fidelity-consistent adaptations have both been associated with improved client outcomes in trauma treatment, suggesting that fidelity and adaptation might both beneficially impact clinical outcomes (Marques et al., 2019). Future studies are needed to further examine whether the adaptation-fidelity tension indeed represents a false dichotomy. Implementation scientists are increasingly reflecting on the value generated by both adaptations and fidelity in determining their optimal balance (von Thiele et al., 2019). Finally, the current study sought to provide a descriptive look at how therapists adapt EBPs and correlates of such adaptations. Future studies yoking observer-rated adaptations to clinical outcomes would address a current gap in the EBP adaptations literature.

Conclusions

The current study contributes to the “adaptome” data commons by codifying community therapists’ naturalistic practice of adapting EBPs, identifying their types and predictors within psychotherapy sessions during implementation as usual. Against the backdrop of increasing implementation of multiple EBPs in community mental health, it is important to understand the adaptations that therapists are making across diverse interventions, instead of EBP-specific adaptations that limit generalization. Therapists most often described flexibly modifying the presentation of content and integrating supplemental content in EBP sessions, particularly when they were open to EBPs and in sessions with younger youth with internalizing problems. Codifying therapists’ own descriptions and definitions of adaptations provided a rich depiction of Augmenting adaptations that help increase the acceptability of EBPs and Generalizing adaptations to extend their reach to complex, diverse, underserved clients. Reducing adaptations that pause, shorten, or omit EBP elements were also described but did not appear driven by negative perceptions of EBPs. Future implementation efforts should support therapists in identifying avenues to adapt EBPs to increase therapist and client engagement, while maintaining integrity to core components to drive both extended reach and positive clinical outcomes of EBPs.