Emergent Life Events: An In-Depth Investigation of Characteristics and Provider Responses during Youth Evidence-Based Treatment

Abstract

Emergent life events (ELEs), or acute client stressors disclosed within psychotherapy sessions, are not addressed by many evidence-based psychosocial treatments (EBTs). Preliminary provider-report studies suggest that ELEs may interfere with effective EBT implementation. The present study offers a detailed, observational examination of ELEs and their impact on EBT within therapy sessions. Data were observationally coded from 274 sessions with 55 primarily low-income, Latino youth clients (58% male, ages 5–15) in the modular EBT condition (MATCH) of the Child STEPs California trial. The ELE Coding System-Revised was used to measure ELEs, their characteristics, and provider responses to ELEs, including provider adherence to MATCH. Interrater reliability was generally high. At least one ELE was identified in 13% of randomly selected sessions. ELEs ranged widely in content, and their characteristics did not cluster together. Providers responded more frequently to ELEs with non-EBT content (e.g., information gathering, empathy) than EBT content; use of the ELE as a “teaching moment” for EBT content was the least common response (40% of ELEs). Multilevel regression analyses revealed that compared to sessions without an ELE, ELE sessions were significantly associated with reduced provider adherence to MATCH. Within ELE sessions, higher client distress when discussing the ELE was associated with reduced provider adherence to MATCH, but only when ELE severity was high. Beyond provider report, observational measures indicate that ELEs are prevalent and unpredictable in community settings and disrupt EBT delivery. Findings can inform the development of structured ELE management procedures to enhance existing EBTs.

An estimated one in five youth in the United States suffers from a psychological disorder (O’Connell, Boat, & Warner, 2009), and the difficulty in meeting this substantial mental health need is a widely recognized problem (Kazdin & Blase, 2011). In response to this need, hundreds of evidence-based psychosocial treatments (EBTs) have been developed for youth in the past few decades (Chorpita et al., 2011). However, significant challenges remain regarding the dissemination and implementation of EBTs into the community contexts where most youth are served. Specifically, delivering treatment as intended according to a structured, empirically-based plan (i.e., treatment integrity) may be challenging in complex clinical environments. High treatment integrity of EBTs has been linked to improved clinical outcomes (Schoenwald, Sheidow, & Letourneau, 2004), whereas drift from the focus of treatment can have detrimental effects (e.g., Kaminski, Valle, Filene, & Boyle, 2008; Lundahl, Risser, & Lovejoy, 2006).

Given the importance of treatment integrity to successful EBT implementation, it is necessary to identify reasons why maintaining treatment integrity may be challenging. According to community mental health providers, one reason is that their clients are more complicated than the research samples for which EBTs were validated (e.g., Nelson, Steele, & Mize, 2006). Indeed, research has demonstrated that compared to privately-referred clients (similar to research samples), publicly-referred clients (similar to community samples) are more likely to display comorbidity of conduct problems, be ethnically diverse, come from lower SES families, and report greater psychosocial stressors (Southam-Gerow, Chorpita, Miller, & Gleacher, 2007). In support of the notion that client complexity may reduce treatment integrity, a study of usual community care demonstrated that providers spend substantial time in treatment addressing external care (e.g., case management), which may detract from their ability to deliver evidence-based strategies (Garland et al., 2010).

Accordingly, research efforts have recently focused on designing EBTs to better fit the complexities found within community populations. One such effort has been the development of modular treatments, which offer the ability to balance the structure and evidence-based strategies of standard EBTs with the responsiveness of usual care in addressing specific client needs (Chorpita, Daleiden, & Weisz, 2005). Modular treatments can be structured to flexibly address common challenges to treatment, such as client comorbidity or a slow rate of learning, by guiding providers towards application of treatment modules using a decision flowchart. A clinical effectiveness trial of the Modular Approach to Therapy for Children (MATCH) found that modular EBT was associated with improved clinical outcomes relative to both standard EBT and usual care for a variety of common youth disorders, with medium to large effect sizes across measures of symptoms and functioning (Weisz et al., 2012). These results suggest the value in designing EBTs that are able to handle client complexity.

Although modular treatments have shown promise in addressing certain forms of client complexity such as comorbidity, other types of complexity remain relatively unaddressed by typical EBTs (e.g., client tardiness, lack of homework completion; Clough & Casey, 2011). In the absence of guidance from EBTs on how to manage these sorts of surprises or exceptional events, Chorpita and Daleiden (2014) have proposed that providers are likely to: (a) ignore the exception (e.g., push ahead with a session when the youth is not ready), or (b) improvise and react (e.g., abandon the session plan and respond in the moment). Neither option is likely to be optimal. With regard to ignoring, research demonstrates that provider insistence on adhering to treatment in response to client resistance is associated with poor clinical outcomes (Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996). Similar to usual care, improvising may respond to the issue at hand, but when used extensively may reduce integrity to EBT strategies (Garland et al., 2010). As is the case with modular treatments to manage comorbidity, ideal strategies for managing exceptional events may fall somewhere in between these fully structured and fully flexible options, through offering treatment strategies while also responding to the issues at hand.

One important type of complexity that has yet to be addressed by most EBTs is emergent life events (ELEs), defined as unexpected stressors disclosed within psychotherapy that have a significant negative impact on the client(s) (Guan et al., 2017a). Preliminary research, based largely on provider-report, suggests that ELEs are prevalent, unpredictable, and can reduce EBT integrity. Community mental health providers reported that ELEs occurred for the majority (69%) of clients, but were reported in only 8% of sessions, making them relatively unpredictable from week to week (Chorpita, Korathu-Larson, Knowles, & Guan, 2014). This unpredictability was heightened by the wide range of content encompassed by the ELEs, from family violence (e.g., child was disciplined via a belt) to illnesses (e.g., uncle passed away) to school issues (e.g., child’s suspension). Thus, providers implementing MATCH indicated that when an ELE was reported in session, they were less likely to cover their original session plan in both the ELE and the subsequent session (Guan, Park, & Chorpita, 2017b). Furthermore, when providers did not use any EBT content in an ELE session, their clients showed substantially slower rates of improvement on both symptoms and functioning (Guan et al., 2017b).

Although the aforementioned studies suggest that ELEs are disruptive to EBT integrity and outcomes, their reliance on provider-report data constrains confidence in their conclusions. Providers’ retrospective recall of session details that occurred earlier in the week may have limited accuracy. Further, providers reported on both the occurrence of ELEs and their own treatment integrity, which may have subjected the findings to bias. Thus, it was important to develop a system of third-party coding to more objectively identify ELEs and EBT integrity in therapy sessions. Such a system could provide a detailed description of in-session processes, such as provider behavior, to identify specific pathways through which ELEs reduce EBT integrity. It is necessary to elucidate these pathways in order to determine how best to address ELEs.

As such, a pilot study was conducted using a small sample of 30 recordings to demonstrate the validity and reliability of the Emergent Life Events Coding System (ELECS) (Guan et al., 2017a). This study revealed that it was feasible to observationally measure ELEs, provider responses to ELEs, and EBT integrity in a largely reliable manner. Additionally, initial analyses using the observational data suggested that similar to provider-report findings, sessions with ELEs were significantly less likely to contain an adequate dose of EBT content than those without ELEs. With regard to Chorpita and Daleiden’s (2014) ideas on exception management, providers faced with ELEs most frequently improvised and reacted using non-EBT responses, whereas ignoring the ELE and pushing ahead with the EBT was far less common. Specifically, providers almost always utilized the “improvisatory” responses of empathy and information gathering. However, they combined those responses with a “teaching moment” to rehearse EBT skills (e.g., “I can see you’re feeling down about your parents’ divorce. Let’s use some of the pleasant activities we talked about to help you feel better”) less than half of the time. When combined with important “improvisatory” responses such as empathy, the “teaching moment” response is hypothesized to optimally balance structure (use of the EBT) with flexibility (responsiveness to specific client needs). These findings highlight the potential value of observationally investigating ELEs as they unfold in sessions. However, the small sample size allowed only for broad analyses of provider responses and EBT integrity, precluded a thorough examination of characteristics of ELEs that are most disruptive to EBT, and prevented interrater reliability from being calculated for some items.

Thus, although previous studies provide preliminary evidence that ELEs are associated with reduced EBT implementation and outcomes, the range of ELEs that occur and the mechanisms through which they disrupt EBT in therapy sessions remain unclear. This knowledge is necessary to inform the creation of best-practice strategies for addressing these unexpected events. A comprehensive, large-scale examination of base rates of ELEs, which have previously only been assessed using provider report; types of ELEs that may arise; and types of provider responses to ELEs will shed light on the nature of the problem. Accordingly, the ELECS has been refined in this study to include characteristics of ELEs, including content categories and client distress when discussing the event, to identify which types of ELEs are associated with the greatest disruption to EBT delivery. Additionally, types of provider responses to ELEs have been expanded to offer a more nuanced look at how providers use or do not use EBT content (e.g., specific EBT content used; how the content related to the current session plan).

The Current Study

Thus, the current study had several goals in order to provide important insights into how ELEs arise in session and affect provider behavior when delivering an EBT. First, we sought to establish the interrater reliability of the revised ELECS in a large sample of sessions. As with a previous pilot study of the ELECS (Guan et al., 2017a), we expected that raters would be able to reliably identify ELEs, their characteristics, and provider responses to them in therapy sessions.

Our second goal was to provide a detailed description of ELEs, including objective identification of base rates and characteristics, characteristics of ELEs that cluster together, and typical provider responses to ELEs. Given past findings using provider report (Chorpita et al., 2014), we predicted that the base rate of ELEs across sessions would be roughly 8% and that ELEs would range widely in content. Additionally, given preliminary findings on provider responses (Guan et al., 2017a; Guan et al., 2017b), we predicted that improvisatory (i.e., non-EBT) provider responses would be more common than responses utilizing the EBT.

Thirdly, we aimed to examine associations between ELEs and provider adherence to EBT content using third-party coding on a significantly larger scale than in past studies. Based on previous findings (Chorpita et al., 2014; Guan et al., 2017a), we expected that ELE occurrence would be negatively associated with provider adherence to the EBT. Within ELE cases, we also explored specific characteristics of ELEs as predictors of provider adherence.

Method

Participants

Participants were selected from the modular EBT condition (MATCH) of the Child STEPs California study, a randomized clinical trial (RCT) conducted at three large community mental health agencies in urban California (Chorpita et al., 2017). The Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH; Chorpita & Weisz, 2009) is a collection of 33 treatment modules that are organized according to five coordinating decision flowcharts, which allow for systematic employment of other treatment modules when interference (e.g., emergent comorbid problems) arises. Providers in the effectiveness trial did not receive any a priori training with regard to how to respond to ELEs. In the consultation session following report of an ELE from the previous treatment session, study consultants made recommendations for how to proceed with future sessions of treatment on a case by case basis. All study procedures were approved by the University of California, Los Angeles Institutional Review Board and Institutional Review Boards of participating service agencies that requested independent reviews.

Inclusion and exclusion criteria.

Youth between the ages of 5 and 15 referred to their local public mental health agency were included if their primary clinical concerns involved anxiety, depression, disruptive behavior, or traumatic stress (N = 138). Youth were excluded if they had mental health concerns that could not be addressed with MATCH, including mental retardation, autism, psychosis, recent suicide attempt, severe trauma, and juvenile justice involvement. Additional details and the flow of youth into the study according to CONSORT guidelines are reported in the original outcome paper (Chorpita et al., 2017).

Two different samples were used in the present study: (1) a randomly selected sample of 100 session recordings (hereafter referred to as “the random sample”) to assess base rates of ELEs across sessions, and (2) a stratified sample of 87 sessions with provider-reported ELEs and 87 without ELEs (hereafter referred to as “the stratified sample”) to assess associations between ELEs and provider treatment adherence. These two samples were selected independently of one another, resulting in 10 sessions that were in both samples. Demographic characteristics of youth, caregiver, and provider participants for each of these samples are reported in Table 1. There were no significant differences in demographic characteristics between the random sample and full sample from the MATCH RCT, indicating that participants selected for the random sample were representative of the MATCH condition as a whole. When compared with providers in the full MATCH sample, providers in the stratified sample were significantly more likely to have master’s degree rather than a doctoral degree as their highest level of education, $X^2(1,n=50)=4.02,p=.045$.

Table 1.

Demographic Characteristics of Youth, Caregiver, and Provider Participants by Sample

Characteristic	Random Sample (n = 100 sessions)	Stratified Sample (n = 174 sessions)
Youth	n = 45	n = 43
Gender	58% male	56% male
Age	M = 9.65, SD = 2.83	M = 9.93, SD = 2.77
Race/Ethnicity
Latino/Hispanic	80%	86%
Black/African-American	9%	7%
Mixed	7%	5%
Caucasian	4%	0%
Not Reported	0%	2%
Primary Problem Area
Disruptive Behavior	40%	42%
Depression	31%	33%
Anxiety	29%	23%
Traumatic Stress	0%	2%
Primary Caregivers
Gender	84% female	77% female
Age	M = 36.02, SD = 8.56	M = 35.59, SD = 6.97
Birth Country
United States	38%	35%
Mexico	38%	44%
El Salvador	16%	14%
Other	9%	5%
Marital Status
Single	57%	56%
Married or Living with Partner	40%	39%
Not Reported	2%	5%
Household Income
$0–19,000	76%	74%
$20,000–39,000	18%	21%
$40,000–59,000	4%	5%
Not Reported	2%	0%
Providers	n = 25	n = 20
Gender	92% female	95% female
Age	M = 32.92, SD = 4.53	M = 32.40, SD = 4.15
Race/Ethnicity
Latino/Hispanic	44%	40%
Caucasian	28%	40%
Mixed	12%	15%
Asian	12%	5%
Black/African-American	4%	0%
Other	16%	25%
Highest Degree Completed
Master’s (M.A., M.S., or M.S.W.)	92%	95%*
Doctorate (Ph.D. or Psy.D.)	8%	5%*
Years of Experience Post-Degree	M = 3.32, SD = 2.70	M = 3.10, SD = 1.94
Primary Theoretical Orientation
Eclectic	36%	35%
Cognitive-Behavioral	32%	45%
Humanistic/Client-Centered	12%	10%
Family Systems	12%	5%
Other	8%	5%

Note.

^*Significant difference on this characteristic compared with the full sample of MATCH providers.

Measures

Emergent Life Events Coding System- Revised (ELECS-R).

A revised version of the ELECS (Guan et al., 2017a), an observational coding system, was used to identify ELEs, characteristics of ELEs, and provider responses to ELEs, including provider adherence to MATCH. In a previous study, identification of ELEs and provider responses to ELEs was generally in the excellent range for interrater reliability (κ’s and ICCs .75–1.00) and demonstrated acceptable agreement with provider report of ELEs (κ = .53; Guan et al., 2017a). All ratings were assigned on an event basis, meaning that they were coded as they occurred. Extensiveness ratings for structured and unstructured activity categories (defined below) were also collected to assess for intensity and dosage of activity. Figure 1 presents the basic structure of the coding system. Constructs relevant to the present study are described below. If multiple ELEs were reported within a session, a new set of activities was coded for each ELE.

Figure 1.

Basic Structure of the Emergent Life Events Coding System (ELECS). Structured Activity = Provider coverage of a MATCH module or other structured protocol. Unstructured Activity = Provider coverage of any other content. Items marked with an asterisk (*) indicate activities for which extensiveness ratings (1–5) were coded. Figure reproduced from Guan et al. (2017a).

Emergent life events (ELEs), negative impact, and other ELE characteristics.

In listening to the full session recording, coders identified the presence or absence of an ELE in session. An ELE was operationally defined as a discrete event that occurs outside of therapy, is disclosed during a treatment session, and has moderate to severe negative impact on the youth or family’s daily life or functioning. Specifically, only events with ratings of 3 (moderate - beginning to have a noticeable impact on daily life or functioning; e.g., youth failed final exam), 4 (marked - significant impact on daily life or functioning; e.g., youth was suspended), or 5 (severe - significant, pervasive, and enduring impact on daily life or functioning; e.g., family was evicted from home) on a 1–5 negative impact scale for coding stressors (based on the Youth Life Stress Interview; Rudolph & Flynn, 2007) qualified as ELEs.¹ To assign a negative impact rating, coders considered the ELE in comparison to set definitions for each rating and examples of qualifying ELEs. The negative impact rating was thus independent of the subjective distress expressed by the client in session when describing the event. Symptoms of psychopathology that were not related to a serious event were not considered ELEs. For example, suicidal ideation alone did not qualify as an ELE, but a major breakup resulting in suicidal ideation or suicidal ideation resulting in a hospitalization qualified as ELEs. Thus, although there could be overlap, ELEs differed from clinical symptoms in their unexpected nature as well as the presence of an acute stressful event. ELEs must have occurred within the past few sessions and were likely, but not required, to fit into the content categories listed (see “Category” below). All events involving injury due to violence towards self or others were considered ELEs. Excluded from the definition of an ELE were chronic issues (e.g., youth’s declining grades), attendance or engagement issues (e.g., late arrival to session), psychiatric medication changes, and future events.

Client distress.

For each ELE, coders rated the degree of subjective distress expressed by the client when discussing the ELE in session on a scale from 1 (not at all distressed) to 5 (severely distressed). Distress was defined as a combination of the client’s level of emotionality (e.g., calm versus tearful), tone of voice (e.g., matter-of-fact versus hysterical), and style of presentation for the event (e.g., statement of facts versus use of catastrophic statements). A separate distress rating was assigned to each individual present in the session. For analytical purposes, these ratings were combined into a single variable consisting of the highest 1–5 distress rating from all present family members (hereafter referred to as “maximum client distress”).

Reporter.

Coders identified who (caregiver, youth, or provider) first reported the ELE in session.

Time elapsed and proportion of session when ELE reported.

Minutes elapsed was calculated by subtracting the start time of the session from the time when the ELE was reported. To calculate the proportion of session time when the ELE was reported, time elapsed was divided by total session time.

Dependence.

Similar to previous research on life stress (e.g., Hammen, Hazel, Brennan, & Najman, 2012), a dependence code captured the extent to which the clients’ or their families’ behaviors contributed to the occurrence of the ELE on a 1 (independent; e.g., an earthquake) to 5 (dependent; e.g., youth suspended for smoking) scale. To account for differences in dependence across the youth client and his/her family, two separate dependence codes were given.

Category.

To categorize the content of each ELE, coders selected all that applied from the following ten categories: family conflict; change in family routine or structure; injury, health-care problem, or death of a loved one or the client(s); client engaging in risk behavior; problems at school; peer or social problems; exposure to an incident of community violence or crime; housing/financial issues; child abuse or neglect; or other. These categories were derived from reviews of previous literature capturing ELEs in the same RCT (Chorpita et al., 2014) and youth life stress in general (Rudolph & Flynn, 2007). To increase statistical power for analyses, we combined the above categories based on conceptual similarity and significant positive correlations (r = .19-.39). The following four broad categories were used for analyses: (1) family issues (i.e., family conflict, change in family routine or structure, and housing/financial issues), (2) trauma and health issues (i.e., injury, health-care problem, or death of a loved one or the client(s), exposure to an incident of community violence or crime, and child abuse or neglect), (3) peer and school problems (i.e., problems at school and peer or social problems), and (4) client risk behavior (i.e., client engaging in risk behavior). ELEs identified in the other category were excluded due to low frequency (3% of ELEs) and lack of similarity with other categories.

Persistence.

Coders also rated whether or not the client(s) persisted in bringing up the ELE after the topic of the session had moved on to something else.

Structured and unstructured session activity.

Regardless of the presence or absence of an ELE, all coded activities in session were categorized as structured or unstructured. Structured activities encompassed activities employing a portion of the MATCH protocol or activities from another structured protocol. For a MATCH activity to be coded as structured, the provider was required to: (a) cover something from a MATCH module by name, or (b) cover more than one step in a MATCH module. Coders specified which MATCH modules were covered in session. Additionally, review of homework or coverage from a previous module was included as a structured activity without having to meet one of the rules, provided that it could be clearly categorized as MATCH content. Previews of future sessions were not included. Structured activities outside of MATCH were required to have a protocol with a clear objective and a set of steps to be followed towards achieving that objective, such as creating an agency treatment plan or a cognitive-behavioral technique not found in MATCH. Conversely, unstructured activities encompassed all other activities (e.g., general discussion of the client’s week, playing a game).

Provider responses to ELEs.

When an ELE was identified in session, coding of structured and unstructured activity was further delineated to capture specific provider responses. As seen in Figure 1, structured activity after an ELE was categorized in two ways. First, providers could relate the ELE to MATCH or a skill from another structured protocol. Providers were required to make statements to tie the ELE to the protocol, using the module to demonstrate why MATCH may be helpful (e.g., psychoeducation) or to demonstrate a way of handling the ELE (e.g.. problem solving). In other words, they used the ELE as a “teaching moment” for MATCH. A second way of employing structured activity after an ELE was to teach a MATCH module or other structured skill without relating it to the ELE.

Similarly, unstructured activity after the reporting of an ELE was categorized as either related or unrelated to the ELE. If the provider responded to the ELE with unstructured, related content, coders rated the occurrence of the following 12 specific responses: (1) supportive or empathic statements- active expressions of support for how the client is handling the ELE or understanding for the difficulty of the client’s experience, (2) relating the ELE to past life experiences- comparing/contrasting the ELE to past experiences, (3) information gathering about the event- inquiring about the facts of the situation, (4) information gathering about the subjective impact of the ELE on client- inquiring about the client’s subjective or affective experience of the event, (5) informal advice giving- offering on-the-spot suggestions for how the client should handle the ELE in the absence of a structured protocol, (6) informal problem solving- asking the client to generate possible solutions and/or select from a variety of solutions for dealing with the ELE in a general way, (7) psychoeducation about the ELE- teaching the client about how these types of ELEs typically occur and their potential impact, (8) informal reframing statement- attempting to spin the ELE in a more positive light, (9) safety assessment- safety assessment and planning or abuse reporting in order to manage risk, (10) provision of supportive resources outside of therapy- identifying formal supports beyond the current therapy to help the client deal with the ELE, (11) provision of supportive resources by therapist- therapist taking concrete actions outside of session to solve the ELE for the client, or (12) other unstructured response. Responses could consist of brief statements (e.g., “Tell me more about what happened”). Specific guidelines were provided in the coding manual to differentiate between structured and unstructured activities that had similar content (e.g., informal problem solving vs. the Problem Solving module from MATCH).

Conversely, providers could employ unstructured activity unrelated to the ELE after it was reported (e.g., playing a board game without talking about the ELE). Given that brief digressions from treatment content are common, all unstructured activity unrelated to the ELE was required to last more than two minutes in order to be coded.

Thus, as depicted in Figure 1, when an ELE was reported in session, providers’ responses were broadly categorized into four types: (a) use of structured activity related to the ELE, (b) use of structured activity unrelated to the ELE, (c) use of unstructured activity related to the ELE, or (d) use of unstructured activity unrelated to the ELE. It was possible for a provider to respond in multiple ways to an ELE, in which case multiple responses were coded. For instance, a provider might initially respond using unstructured activities related to the ELE (e.g., information gathering about the event), then move on to structured activity unrelated to the ELE (e.g., the Relaxation module of MATCH that is not proposed as a potential solution to the ELE). In addition, coders selected the first specific provider response utilized after the ELE was reported from a comprehensive list of specific unstructured and structured activities.

Extensiveness ratings.

The extensiveness rating was defined as a combination of the proportion of time spent on a given activity and the thoroughness with which it was pursued. All structured and unstructured activities identified in session were assigned an extensiveness rating from 0 (absent) to 5 (high intensity). Anchors for a 1, 3, and 5 for each activity were outlined in the coding manual. Specifically, an extensiveness rating of 1 for a given structured activity was assigned to providers who covered content from a MATCH module for one or two minutes with little depth, low concentration of effort, and/or with only one step from the module. An extensiveness rating of a 3 was assigned to providers who were beginning to spend a moderate proportion of the session covering content from the MATCH module in some depth, with moderate concentration of effort, and/or with several steps from the module. Lastly, an extensiveness rating of a 5 was assigned to providers who spent nearly all of the session time covering content from the MATCH module in great depth, with high concentration of effort, and/or with multiple steps from the module.

Provider adherence to MATCH.

Provider treatment adherence was measured using a 0–5 session-wide extensiveness rating combining across all structured activities covered in the session, including MATCH and other structured protocols. Due to the low frequency of other structured protocols (i.e., coded in 4% of all sessions), this metric will be hereafter referred to as “adherence to MATCH.” By definition, the session-wide extensiveness rating was required to be at least as high as the rating for any single MATCH module.²

Consultation Record.

A Consultation Record (CR; Ward et al., 2013) was completed by project consultants during a weekly semi-structured consultation meeting with MATCH providers to track implemented and planned treatment practices from the most recent therapy session. The CR has demonstrated strong agreement between provider report and coder observation of session practices (Ward et al., 2013). It is organized by a matrix of checkboxes in which rows represent MATCH modules and other practices, and columns represent questions associated with their implementation (e.g., was the practice partially or fully covered in session?). Information on which modules were recommended by the consultant for the next session was used assess whether MATCH modules covered in an ELE session corresponded to the MATCH module(s) originally recommended for that session (i.e., module was planned for the present session ), a previously covered module (i.e., module was covered in a past session), or neither (i.e., module for a hypothetical future session).

The “crisis” section of the CR, which requires consultants to ask about the presence of an ELE in the previous session, was used to sample sessions with and without ELEs for coding. To record an ELE on the CR, the consultant utilized the following definition: the event was acute, of a serious nature, and caused significant distress for the client and/or family that warranted clinical attention during session. Previous research has shown acceptable agreement between the CR and coded sessions on the presence of ELEs in session (κ = .53; Guan et al., 2017a).

Coding Procedure

Coder Training.

Coders were five clinical psychology doctoral students trained in MATCH who were blind to study hypotheses. The coding team met weekly to discuss the coding manual and the MATCH protocol as applied to practice recordings. General issues with item content and definitions were also discussed at this time. Following these discussions and consultation with co-authors, a revised version of the coding manual was produced. After demonstrating 80% or greater agreement with gold-standard codes (including within-1 ratings for dimensional codes) for five consecutive recordings, coders independently rated the sampled MATCH session recordings. Two coders were fluent in Spanish and coded sessions conducted in Spanish; Spanish-language sessions accounted for 18% of all coded sessions. Coders continued to meet biweekly throughout this time to discuss issues and prevent drift.

Sampling of Sessions for Coding.

For the random sample of 100 sessions, sessions were randomly selected from all MATCH sessions on the CR that were longer than 10 minutes in length and had no technical issues (n = 1210). For the stratified sample, all MATCH sessions with a reported ELE on the CR that were longer than 10 minutes and had no technical issues were selected (n = 87). To control for client and provider differences across groups influencing provider adherence results, a control sample of sessions was selected by matching each ELE session with a session from the same client and provider, but without a reported ELE (n = 87). Given that previous research indicates that ELEs are associated with decreased provider adherence for at least two sessions (Guan et al., 2017b), the control sample also excluded sessions occurring one session after a session with a reported ELE.

Interrater Reliability.

To assess for interrater reliability of the ELECS-R, a stratified random sample of 30% of recordings was selected for double coding. To ensure a sufficient sample of ELE recordings to measure reliability for ELE-specific codes, 20% of double coded recordings were selected from recordings with a reported ELE, with the remaining 10% having no reported ELE. Discrepancies between coders across double-coded recordings were resolved by randomly preselecting one coder’s data to be used for each session in the analyses.

Analyses

Goal 1: Assess Interrater Reliability of Codes.

Interrater reliability was calculated across all coders using Cohen’s κ for categorical variables and ICCs (model [2, 1], consistency) for dimensional variables. In addition, a κ was calculated across all 274 sessions to assess agreement between report of an ELE on the CR and in coded sessions.

Goal 2: Describe ELEs.

We used data from the random sample to assess base rates and confidence intervals of ELEs. Then, combining data across all non-overlapping sessions in which an ELE was identified in the random and stratified samples, we generated descriptive statistics for each ELE characteristic. To further understand the nature of the ELEs reported, we then examined correlations among continuous ELE characteristic variables. Lastly, we obtained descriptive statistics for each type of unstructured and structured provider response to ELEs. Given that the limited remaining time in session would necessarily change the range of reasonable responses available to providers (e.g., providers might be more likely to utilize briefer responses rather than teaching a MATCH skill), this latter set of analyses excluded ELEs (n = 4) that were reported in the second half of the session time.

Goal 3: Examine ELEs as Predictors of Provider Adherence to MATCH.

Analyses were conducted in HLM 7. To examine the overall association between ELEs and adherence to MATCH, we used the stratified sample of ELE and control sessions described above, excluding two sessions that contained ELEs reported in the second half of the session. ELE and control sessions were defined based on coded presence or absence of ELEs; t-tests and chi-square analyses showed that coded ELE and control sessions did not significantly differ on any client or provider demographic characteristics.

Presence or absence of an ELE was modeled as a session-level predictor of session-level adherence in a multilevel random coefficient model with 172 sessions nested within 43 clients nested within 21 providers. The initial model included random intercepts and Level 1 predictor slopes at both the client and provider levels; non-significant variance components were subsequently removed to achieve a more parsimonious structure. An effect size (d) and associated approximate 95% confidence interval for this primary analysis were computed following procedures described in Feingold (2015).

Using the same multilevel random coefficient modeling process described above, we also investigated each characteristic of ELEs (all session-level variables) as a predictor of adherence to MATCH within ELE cases only. We subsequently examined a combined random coefficient model that included all significant ELE characteristics and their interaction(s) as predictors of MATCH adherence. When multiple ELEs were reported in a single session, it was impossible to differentiate the effects of each ELE on the session-wide adherence score. Thus, analyses of ELE characteristics as predictors of adherence excluded these ELEs, as well as the aforementioned ELEs reported in the second half of session (n = 20 ELEs total). The resulting sample size for these latter analyses was 59 sessions nested within 33 clients nested within 18 providers.

Results

Goal 1: Assess Interrater Reliability of Codes

Interrater reliability of the ELECS–R was high for the vast majority of codes (κ > .60 [substantial or above agreement; Landis & Koch, 1977] or ICC > .60 [good or above agreement; (Cicchetti, 1994)]), indicating that trained raters were able to reliably identify ELEs, their characteristics, and provider responses to them. However, two ELE characteristics (persistence and dependence on family) and one provider response (provision of supportive resources by therapist) fell below acceptable reliability levels of κ or ICC > .40 and were therefore excluded from analyses. Results are presented in Table 2.

Table 2.

Interrater Reliability of the Emergent Life Events Coding System-Revised (ELECS-R)

Item Description	ICC or κ*
Presence or Absence of at Least One ELE in Session	.78*
Number of ELEs Per Session	.67
ELE Characteristics
Negative Impact	.72
Maximum Client Distress	.66
Category: Trauma and Health Issues	.72*
Family Issues	.80*
Peer and School Problems	.71*
Client Risk Behavior	.63*
Reporter	.76*
Time Reported	.98
Dependence on Youth	.88
Dependence on Family	.40^
Persistence	.33*^
Session-Wide Extensiveness of Structured Activity (Adherence to MATCH)	.75
Session-Wide Extensiveness of Unstructured Activity	.75
Responses to ELE
First Response	.83*
ELE Occurrence → Yes → Structured Activity	.76*
Extensiveness	.72
ELE Occurrence → Yes → Structured Activity → Related to ELE	.74*
Which MATCH Module Related to ELE	.77*
Extensiveness	.84
ELE Occurrence → Yes → Structured Activity → Unrelated to ELE	.60*
Which MATCH Module Unrelated to ELE	.81*
Extensiveness	.68
ELE Occurrence → Yes → Unstructured Activity	1.00*
Extensiveness	.73
ELE Occurrence → Yes → Unstructured Activity → Related to ELE	1.00*
Supportive/empathic statements	.73
Relating the ELE to past life experiences	.65
Information gathering about the event	.57
Information gathering about subjective impact of ELE on client(s)	.71
Informal advice giving	.77
Informal problem solving	.77
Psychoeducation about the ELE	.47
Informal reframing statement	.69
Safety assessment	.97
Provision of supportive resources outside of therapy	.49
Provision of supportive resources by therapist	.31^
Other	.75
ELE Occurrence → Yes → Unstructured Activity → Unrelated to ELE	.80*
Extensiveness	.78
ELE Occurrence → No → Structured Activity	.69*
Extensiveness	.72
ELE Occurrence → No → Unstructured Activity	.54*
Extensiveness	.76

Note. Items correspond to Figure 1.

^{^}Code excluded from data analysis due to poor reliability. ICC = intraclass correlation coefficient. ICCs (2, 1) were calculated for continuous variables, and Cohen’s κ’s (denoted with an asterisk) were calculated for categorical variables.

Additionally, there was moderate agreement on the presence of an ELE in session between provider report on the CR and coded sessions using the ELECS–R (κ = .46). This level of agreement was expected given previous research showing low to moderate concordance between observers and providers on EBT session content (Guan et al., 2017a; Hogue, Dauber, Lichvar, Bobek, & Henderson, 2015). In fact, providers did not report an ELE in 31% of sessions in which ELEs were identified by coders (i.e., false negatives), and they reported an ELE in 25% of sessions in which no ELE was identified by coders (i.e., false positives).

Goal 2: Describe ELEs

How many and what kinds of ELEs are reported?

At least one ELE was identified in 13 out of 100 sessions in the random sample, resulting in a 13% base rate of ELEs across sessions (95% CI [8%, 21%]). A total of 79 non-overlapping ELEs was reported across all coded recordings within the random and stratified samples, with an average of 1.13 ELEs (SD = 0.38) per ELE session. Specifically, 70 sessions contained at least one ELE; the vast majority (89%) of these sessions contained one ELE, with 10% containing two ELEs and 1% containing three ELEs.

Complete results for characteristics of ELEs are presented in Table 3. In brief, ELEs ranged widely in content. The top five identified categories, followed by specific examples of ELEs identified in the sample, were: family conflict (34%; e.g., physical fight between youth and sibling, argument between caregivers over child custody); injury, health-care problem, or death of loved one or client(s) (29%; e.g., caregiver’s friend hospitalized, death of youth’s grandmother); client engaging in risk behavior (23%; e.g., youth threw objects at family members, youth engaged in self-cutting); problems at school (23%; e.g., youth failing grade, youth suspension for fighting); and change in family routine or structure (22%; e.g., caregivers divorced and father moved out, youth sent to live temporarily with relatives).

Table 3.

Characteristics of Coded Emergent Life Events (N = 79 ELEs)

Characteristic	M (SD)	Frequency [95% CI]
Negative Impact (1–5)	3.57 (0.57)
Youth Distress (1–5) (n = 50)	2.32 (1.17)
Caregiver Distress (1–5) (n = 53)	2.91 (1.10)
Maximum Client Distress (1–5) (n = 79)	2.76 (1.18)
Reporter
Caregiver		42% [31%, 53%]
Provider		34% [25%, 45%]
Youth		24% [16%, 35%]
Time Elapsed When ELE Reported (Minutes)	7.12 (10.52)
Proportion of Session When ELE Reported	0.14 (0.18)
Dependence on Youth (1–5)	2.54 (1.67)
Category (Multiple May Apply)
Family Issues		51% [40%, 61%]
Family Conflict		34% [25%, 45%]
Change in Family Routine or Structure		22% [14%, 32%]
Housing/Financial Issues		9% [4%, 17%]
Trauma and Health Issues		35% [26%, 46%]
Injury, Health-Care Problem, or Death		29% [20%, 40%]
Child Abuse or Neglect		8% [3%, 16%]
Exposure to Community Violence or Crime		3% [1%, 9%]
Peer and School Problems		27% [18%, 37%]
Problems at School		23% [15%, 33%]
Peer or Social Problems		9% [4%, 17%]
Client Risk Behavior		23% [15%, 33%]
Other		3% [1%, 9%]

Identified ELEs were largely rated as having a moderate (3; 47%) or marked (4; 49%) negative impact, with only 4% of ELEs receiving the highest rating of severe (5; e.g., youth charged with felony). Both youth and caregiver clients expressed a mild to moderate average level of subjective distress regarding ELEs in session. On average, ELEs were reported early in session, with only four ELEs (5%) being reported in the second half of the session. In addition, the identified ELEs had mostly independent (2) to mixed (3) mean dependence on the youth, indicating that clients’ behaviors partially contributed to the occurrence of the ELEs.

What characteristics of ELEs tend to cluster together?

Two-tailed Pearson correlation analyses between ELE negative impact, maximum client distress, time reported in session, and youth dependence revealed no significant associations between ELE characteristics.

How are providers responding to ELEs?

Providers responded to ELEs in a variety of ways, as presented in Table 4. Some unstructured activity was identified following the report of an ELE in session 100% of the time, with a moderately high session-wide extensiveness rating (M = 3.79, SD = 1.17). Providers used at least some structured activity in 75% of ELE cases, with a mild to moderate extensiveness rating (M = 2.63, SD = 1.02). Of the four responses presented in Figure 1, unstructured activity related to the ELE was the most common (100%), followed by unstructured activity unrelated to the ELE (67%), structured activity unrelated to the ELE (55%), and structured activity related to the ELE (40%). A combination of these four responses was employed in 88% of cases, with the most prevalent combination being unstructured activity related to the ELE, structured activity unrelated to the ELE, and unstructured activity unrelated to the ELE (25%).

Table 4.

Provider Responses to Emergent Life Events (n = 75 ELEs)

Response	Frequency [95% CI]	M (SD)
First Response to ELE
Information gathering about the event	64% [53%, 74%]
Information gathering about subjective impact of ELE on client(s)	15% [8%, 24%]
Supportive/empathic statements	15% [8%, 24%]
Other unstructured responses related to ELE	7% [3%, 15%]
Structured Activity	75% [64%, 83%]	2.63 (1.02)*
Structured Activity → Related to ELE	40% [30%, 51%]
Extensiveness of Relating MATCH Module to ELE		2.53 (1.37)*
Extensiveness of Each MATCH Module Overall		1.98 (0.89)*
MATCH Module Related to ELE (n = 43)
Past, Present, or Future Modules
Module Planned for Present Session	54% [39%, 67%]
Module from Past Session	28% [17%, 43%]
Module for Future Session	9% [4%, 22%]
Unable to Determine (Missing Session Plan)	9% [4%, 22%]
Which Modules Were Related to ELE
Depression: Problem Solving	23% [13%, 38%]	2.20 (0.92)*
Conduct: Rewards	14% [7%, 27%]	1.83 (0.75)*
Depression: Learning to Relax/Quick Calming	12% [5%, 24%]	1.60 (0.55)*
Depression: Learning Depression- Child	12% [5%, 24%]	2.40 (0.89)*
Depression: Cognitive BLUE	7% [2%, 19%]	1.67 (0.58)*
Depression: Cognitive TLC	7% [2%, 19%]	2.33 (1.16)*
Other MATCH Modules	26% [15%, 40%]	1.82 (1.08)*
Other Structured Protocol Related to ELE (n = 4)		1.50 (1.00)*
Structured Activity → Unrelated to ELE	55% [43%, 65%]
Extensiveness of MATCH Modules		1.68 (0.95)*
Past, Present, or Future Modules
Module Planned for Present Session	47% [36%, 59%]
Module from Past Session	34% [24%, 46%]
Module for Future Session	10% [5%, 19%]
Unable to Determine (Missing Session Plan)	9% [4%, 17%]
Extensiveness of Other Structured Protocols		2.60 (0.89)*
Unstructured Activity	100% [95%, 100%]	3.79 (1.17)*
Unstructured Activity → Related to ELE	100% [95%, 100%]
Information gathering about the event	93% [85%, 97%]	1.80 (0.69)*
Supportive/empathic statements	89% [80%, 95%]	1.75 (0.79)*
Information gathering about subjective impact of ELE on client(s)	77% [67%, 85%]	1.78 (0.75)*
Informal advice giving	57% [46%, 68%]	2.33 (1.02)*
Psychoeducation about the ELE	41% [31%, 53%]	1.68 (0.75)*
Informal reframing statement	33% [24%, 45%]	1.44 (0.65)*
Informal problem solving	20% [13%, 30%]	1.40 (0.74)*
Safety assessment	19% [11%, 29%]	2.07 (0.83)*
Relating the ELE to past life experiences	19% [11%, 29%]	1.07 (0.27)*
Provision of supportive resources outside of therapy	16% [9%, 26%]	1.33 (0.49)*
Other	35% [25%, 46%]	1.77 (0.95)*
Unstructured Activity → Unrelated to ELE	67% [55%, 76%]

Note.

^*Extensiveness when response was utilized (1–5). Bolded items correspond to the responses depicted in Figure 1.

When an ELE occurred, providers’ first response was invariably to use some form of unstructured activity related to the ELE, with the most common first response being information gathering about the event (64%). Similarly, the most frequent unstructured responses overall were information gathering about the event (93%), supportive/empathic statements (89%), information gathering about the subjective impact of the ELE on client (77%), and informal advice giving (57%). However, the responses with the highest extensiveness ratings were informal advice giving (M = 2.33, SD = 1.02) and safety assessment (M = 2.07, SD = 0.87); these responses took up the most time and intensity once they were used.

Within the category of structured activity related to the ELE, the Problem Solving module from the MATCH depression protocol was the most frequently applied to address the ELE (used in 23% of ELEs with structured activity related to the ELE), followed by the Rewards module from the conduct protocol (14%). Of note, all MATCH modules used to address the ELE came from the depression and conduct rather than the anxiety and trauma protocols. The majority of modules utilized were originally planned for the present session (54%), followed by modules that were covered in past sessions (28%).

Goal 3: Examine ELEs as Predictors of Provider Adherence to MATCH

Overall effects of ELE occurrence on adherence to MATCH

Within the stratified sample, average provider adherence to MATCH was 2.89 (SD = 1.57) in control sessions and 2.05 (SD = 1.42) in ELE sessions. Our primary analysis indicated that ELE occurrence significantly predicted reduced provider adherence to MATCH (b = −0.96, SE = 0.26, t = −3.70,p < .001). The estimated effect size for this analysis was in the medium to large range at d = 0.61 (95% CI [0.29, 0.93]) (Cohen, 1988).

ELE characteristics that predict adherence to MATCH

Within ELE cases, individual analyses revealed few ELE characteristics that significantly predicted adherence. Higher maximum client distress significantly predicted reduced provider adherence to MATCH (b = −0.58, SE = 0.12, t = −4.86, p < .001). ELE negative impact (b = −0.70, SE = 0.36, t = −1.95, p = .062), time elapsed when reported (b = 0.03, SE = 0.02, t = 1.48, p = .150), and dependence on youth (b = −0.15, SE = 0.12, t = −1.25, p = .222) were not significant predictors of adherence to MATCH; however, there was a trend towards greater negative impact predicting reduced provider adherence. Finally, the four ELE categories did not significantly predict adherence to MATCH (family issues, b = −0.20, SE = 0.36, t = −0.56, p = .582; trauma and health issues, b = 0.06, SE = 0.37, t = 0.15, p = .882; peer and school problems, b = 0.29, SE = 0.42, t = 0.69, p = .499; client risk behavior, b = −0.70, SE = 0.43, t = −1.64, p = .113).

Finally, we explored whether the effect of maximum client distress on provider adherence was moderated by the negative impact of the ELE. As depicted in Figure 2, there was a significant interaction effect (b = −0.63, SE = 0.25, t = −2.50, p = .020). Simple slope analyses revealed that when ELE negative impact was marked, higher client distress was associated with reduced provider adherence to MATCH (b = −0.89, SE = 0.14, t = −6.38, p < .001). Conversely, no association between client distress and provider adherence to MATCH was revealed when ELE negative impact was moderate (b = −0.26, SE = 0.20, t = −1.29, p = .211).

Figure 2.

Impact of maximum client distress on provider adherence to MATCH is moderated by ELE negative impact. ELE = Emergent life event. MATCH = Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems.

Discussion

Using a large sample of observationally coded modular EBT sessions with diverse community youth, the present study is the first to offer an in-depth examination of how provider in-session behavior is impacted by the frequently cited barrier of ELEs. Specifically, we sought to: (1) establish the reliability of a revised coding system for ELEs (the ELECS-R), (2) provide a detailed description of ELEs and provider responses to them, and (3) assess the impact of ELE occurrence on in-session provider treatment adherence. Interrater reliability was high for most of the constructs within the ELECS-R, thus enabling us to proceed with the two subsequent goals.

With regard to our second goal, the current study was the first to provide an objective estimate of the base rate of ELEs across sessions. ELEs were identified in 13% of a randomly selected sample of coded MATCH sessions, a rate that was nearly twice as high as provider report of ELEs within the same trial on the CR (8%; Chorpita et al., 2014). About a third of the time, providers did not report an ELE when one was identified by an objective coder. These results could be attributable to the slightly different definition of ELEs across studies: in determining whether to record ELEs on the CR, providers did not use a set severity scale and considered level of client distress, whereas the ELECS-R focused on ELEs that met a set severity level regardless of client distress. Because the average level of client distress about ELEs in this study was less than moderate, it is possible that the ELECS-R captured a greater number of ELEs that were severe, but did not elicit significant client distress at the time they were reported to the provider. However, even when providers did not retrospectively report on the occurrence of an ELE, ELEs coded in session appeared to negatively impact their treatment adherence, suggesting that increased provider training in recognizing ELE severity may be beneficial. Future research could examine differences between ELEs that were and were not reported by providers to better pinpoint the reasons for these false negatives and thereby improve ELE detection. In addition, although we found a base rate of 13% of sessions with ELEs in the current study of MATCH, this rate is likely to differ depending on the characteristics of the population served and the treatment being delivered (e.g., less structured treatments may actively encourage clients to discuss current stressors, resulting in a higher base rate).

The present study offered a rich description of the nature of ELEs within a low-income, largely Latino youth population presenting with a range of psychopathology. As predicted, the ELEs in this sample encompassed a range of content categories, including stressors related to family conflict, traumatic events, and client risk behaviors. ELEs were often reported within the first few minutes of the session, allowing ample time for providers to respond. Somewhat surprisingly, characteristics of these ELEs did not systematically cluster together (e.g., more severe events were not consistently associated with higher client distress). These results suggest that ELEs are highly idiosyncratic and unpredictable.

The unpredictable nature of these ELEs, combined with a lack of structured guidance within the EBT to address these events, likely contributed to providers’ use of widely varied responses when ELEs were reported. Consistent with previous research (Guan et al., 2017a), providers responded more frequently and with a higher intensity of non-EBT content than EBT content. However, non-EBT and EBT responses were often combined. Specifically, providers invariably used at least some form of “improvisatory” non-EBT response (e.g., information gathering, support/empathy) to initially address the ELE. In three quarters of cases, they proceeded to deliver at least some EBT content; however, using the ELE as a “teaching moment” for EBT content was the least common type of response. It is worth noting that the modular EBT employed in the current study contained techniques designed to address multiple areas of psychopathology (i.e., anxiety, depression, trauma, and conduct problems) rather than the one area addressed by many standard EBTs; thus, rates of EBT responses in this study are likely an overestimate compared with the rates of providers employing standard EBTs, in which a more limited array of skills to address ELEs may be offered.

These findings indicate that providers never fully “ignored” the ELE, which may have damaged therapeutic rapport and outcomes (e.g., Castonguay et al., 1996). In addition, the most common non-EBT responses are likely to be appropriate for many, if not all, ELEs. For instance, it is likely that an “improvisatory” response of empathy is necessary to maintain therapeutic rapport with the client, and information gathering about the event and its impact are necessary to inform selection of additional responses to address the event. Given that providers gravitate towards the use of these three responses with the vast majority of ELEs, it may be important to consider including some degree of empathic and information gathering responses when developing a toolkit for responding to ELEs in an optimal manner. However, ELEs for which no EBT content is covered in session are associated with significantly reduced rates of client progress on symptoms and functioning (Guan et al., 2017a). Thus, exclusive use of the most common types of “improvisatory” responses to ELEs is likely to have a detrimental impact on client outcomes. However, the optimal dosage of each type of non-EBT and EBT response remains unknown.

Conversely, consistent with past research (Guan et al., 2017a), providers employed the ELE as a “teaching moment” to rehearse EBT skills in a minority of cases. When they delivered EBT content in session, it was more often unrelated to the ELE. However, when providers did use the “teaching moment” strategy, they most commonly chose skills that could be applied across many stressors (e.g., problem solving, relaxation) rather than situation-specific skills (e.g., exposure for anxiety), suggesting that there may be a subset of EBT skills that are easier to apply to ELEs. Additionally, providers largely chose to apply skills that were planned for the current session of therapy, with occasional review of prior skills. If it is the case that the “teaching moment” response to ELEs is more effective than using EBT skills unrelated to the ELE, these findings suggest that there may be value in training providers to more flexibly select from all portions of an EBT manual to appropriately address a given ELE.

With regard to our third goal, the present study demonstrated that compared to control sessions, ELE sessions were associated with significant reductions in provider adherence to the modular EBT. These findings are consistent with hypotheses and extend upon previous work by utilizing a stronger control group of sessions, which increased our confidence that the impact of ELEs on provider adherence was attributable to the ELEs themselves rather than provider or client characteristics. In addition, it is worth noting that the current study lowered the threshold of ELEs to include events with moderate negative impact or severity, rather than solely marked or severe negative impact as in previous research (Guan et al., 2017a). Results suggest that provider adherence continues to be negatively affected by moderately impairing ELEs, although there was a trend towards more severe events having a greater impact on adherence. Also of note, the present study contained very few ELEs rated at the highest level of severity, and therefore results are not generalizable to these types of events.

Our exploration of characteristics of ELEs that are more likely to impact provider adherence identified few significant characteristics. Although our power to detect effects may have been limited by the moderate number of ELEs we were able to include in these analyses, the sparse findings may indicate that many different types of ELEs can negatively affect provider adherence. Our results did highlight a significant interaction effect between ELE severity and in-session client distress regarding the ELE. Higher levels of client distress were associated with reduced provider adherence to the EBT, but only when the reported ELE was more severe. These findings suggest that providers may be especially attuned to client distress in session when selecting their responses to ELEs – particularly when there is a severe ELE that perhaps warrants greater distress. Any structured guidance for addressing ELEs may therefore benefit from including EBT strategies that providers can use to manage client distress in session (e.g., a relaxation technique) as well as to address the event itself.

Limitations

Although the current study offers a wide-ranging examination of ELEs and their effect on modular EBT delivery, several limitations should be noted. We attempted to control for confounding client and provider influences when assessing associations between ELEs and provider adherence; however, the lack of random assignment to ELE and control conditions prevents us from making causal inferences about the impact of ELEs on adherence. Additionally, analyses of provider responses and adherence in the present study focused only on ELEs that were reported in the first half of sessions (the vast majority of ELEs). Thus, results are not generalizable to later ELEs. In addition, given the limited sample size of ELEs and the low frequency of ELEs falling within each of the 10 original content categories, we combined categories in our analyses of ELE category predicting provider adherence. This combining process may have caused us to lose important information specific to the original categories. Finally, a strength of the present study is its use of an underserved, largely Latino population of clients that included Spanish-speaking families; however, use of this sample also restricts our findings given that results may have limited generalizability to other populations.

Future Directions

Future research can expand upon the results from this study in a number of ways. Firstly, it is unclear which types of responses to ELEs best promote client outcomes as well as therapeutic alliance. An examination of these questions will be crucial to developing and testing an evidence-based toolkit for responding to ELEs. In addition, although we suspect that using the ELE as a “teaching moment” for EBT skills is an optimal response, there may exist ELEs that would not be appropriate to address primarily with EBT skills. For example, events involving client self-harm may require a safety planning procedure that may preclude coverage of EBT content in session. Future research would benefit from assessing the extent to which ELEs are able to be addressed with EBT skills, so that additional resources not available in current EBTs may be identified to help manage ELEs. From this research, it would also be helpful to identify common EBT skills that can be applied to many ELEs, for the purpose of including training in these strategies in an ELE management toolkit. Finally, the current study collapsed across all sessions in which ELEs were reported. However, given that a single ELE can frequently disrupt provider adherence in more than one session of treatment (Guan et al., 2017b), future research could differentiate between provider behavior in the first sessions in which ELEs are reported as opposed to subsequent sessions.

Conclusion

The present study characterized the broad array of ELEs that arise during community-based implementation of a modular evidence-based psychotherapy for youth, as well as the impact of these events on provider behavior and treatment adherence. Consistent with provider perspectives on using EBTs with complex populations, findings highlight the disruptive nature of many types of ELEs and the need for structured guidance in responding to them - even within a modular treatment with a goal of addressing client complexity. It is our hope that the knowledge gained in this study will further the progress made with modular treatment designs, by informing the development of ELE management procedures that can help clients achieve maximum benefit from existing EBTs.