Emergent Life Events in the Delivery of a Caregiver-Mediated Evidence-Based Intervention for Children with Autism Spectrum Disorder in Publicly-funded Mental Health Services

Abstract

Mental health clinicians often report significant challenges when delivering evidence-based interventions (EBI) in community settings, particularly when unexpected client stressors (or emergent life events; ELEs) interfere with the therapy process. The current study sought to extend the study of ELEs to children with Autism Spectrum Disorder (ASD) by examining the occurrence and impact of ELEs in the context of a collaborative, caregiver-mediated intervention for reducing challenging behaviors in children with ASD, An Individualized Mental Health Intervention for Children with Autism Spectrum Disorders (AIM HI; Brookman-Frazee & Drahota, 2010). Participants included 38 child-clinician dyads enrolled in a community effectiveness trial of AIM HI. Video recordings of 100 therapy sessions were coded for caregiver-reported ELEs and clinician adherence to the AIM HI protocol. Results indicated that mild to severe ELEs were reported in 36% of sessions, and were reported for 58% of children at some point during treatment. Greater number of child comorbid diagnoses and less clinician experience were both significantly associated with a higher number of caregiver-reported ELEs. There was no significant link between ELEs and clinician adherence to the AIM HI. Findings offer implications for EBI implementation, particularly the importance of incorporating clinician training in addressing complex presentations and crises in the context of EBIs.

There is a significant need for evidence-based services to address Autism Spectrum Disorder (ASD), especially given the high rates of children diagnosed with ASD in the United States (Baio et al., 2018). Although a range of evidence-based interventions for ASD have been developed (Wong et al., 2015), children may not receive the benefits of these interventions if they are not implemented as intended in routine care settings (Kazdin & Blase, 2011). Many scholars contend that difficulties with implementing evidence-based interventions (EBIs) in the community result from the failure of protocols developed and tested in laboratory settings to address the complex needs of children treated in routine care (Southam-Gerow, Rodriguez, Chorpita, & Daleiden, 2012). One component of this complexity may be the unexpected crises that arise in the lives of children and families during the course of treatment. This is particularly true of children diagnosed with ASD, who often present with a range of crises that may emerge during therapy sessions (e.g., difficulties at school, episodes of disruptive behavior; Totsika, Hastings, Emerson, Berridge, & Lancaster, 2011; Weiss, Wingsiong, & Lunsky, 2014). Importantly, these ELEs have been linked to lower levels of clinician adherence to EBIs (Guan et al., 2018) and negative treatment outcomes (Guan, Park, & Chorpita, 2017b) in children with mental health concerns without ASD. To date, however, there is limited literature devoted to understanding the unique challenges that occur in therapy processes for children diagnosed with ASD, and how these challenges may influence the delivery and community implementation of EBIs (Bearss et al., 2015). To address this gap, this paper aims to characterize these unexpected crises that emerge in course of treatment, identify factors associated with these crises, and examine the impact of these crises on clinician intervention adherence within the context of implementation of an ASD intervention.

Emergent Life Events (ELEs)

Unexpected crises that arise during therapy are sometimes referred to as emergent life events (ELEs), defined as acute, unexpected life stressors (e.g., expulsion from school, parental job loss, illness) that negatively impact the child or family (Chorpita, Korathu-Larson, Knowles, & Guan, 2014). Though ELEs may occur in all populations, underserved and community populations tend to be more likely to experience life stressors than populations in intervention development research settings (Southam-Gerow, Chorpita, Miller, & Gleacher, 2008). In addition, traumatic and other stressful events occur more frequently in low SES and racial/ethnic minority groups, compared with non-Hispanic Whites (Hatch & Dohrenwend, 2007; Turner & Lloyd, 2004). A recent study found that moderate to severe ELEs were identified in 13% of randomly selected sessions from a randomized controlled effectiveness study with a sample of low-income children ages 5 to 15 years old treated for mental health concerns in community agencies (Guan et al., 2018). Common types of ELEs included family conflict; injury, health-care problem, or death of loved one or child; the child engaging in risky behavior; problems at school; and change in family routine or structure (Guan et al., 2018).

ELEs and Clinician Adherence

Community clinicians frequently cite these unexpected stressful life events as hindrances to the delivery of EBI protocols, making it crucial to understand the impacts of ELEs on therapy processes and outcomes (Reding et al., 2016). Factors such as healthier child pre-treatment psychosocial functioning, lower caregiver education, and caregiver African American ethnicity have been found to be associated with higher clinician adherence, or the extent to which a clinician follows the intervention as intended (Schoenwald, Halliday-Boykins, & Henggeler, 2003; Schoenwald, Letourneau, & Halliday-Boykins, 2005). In addition to these factors, the presence of ELEs has also been shown to influence clinician adherence in studies with low-income children being treated for mental health concerns in community agencies (Guan et al., 2018; Guan, Levy, Kim, Brown, et al., 2017a). Specifically, ELE occurrence significantly predicted reduced clinician adherence to an EBI, with clinicians being more likely to respond to an ELE with non-EBI content than EBI content (Guan et al., 2018). In addition, examination of sessions following a session with an ELE suggested that this disruption from ELEs persisted, with ELEs being associated with reduced adherence to planned practices for at least two subsequent sessions (Guan, Park, & Chorpita, 2017b). In addition, disruptions caused by ELEs have been linked to negative treatment outcomes, with each disruptive ELE (i.e., an ELE for which no EBI content was covered) associated with a 14%–20% slower rate of clinical improvement (Guan, Park, & Chorpita, 2017b).

ELEs and Autism Spectrum Disorder (ASD)

The majority of research concerning the presence and impact of ELEs on treatment adherence and outcomes has been conducted with children receiving a modular EBI treatment, the Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH; Chorpita & Weisz, 2009), to address primary concerns of anxiety, depression, disruptive behavior, or traumatic stress (Chorpita et al., 2014; Chorpita & Weisz, 2009; Guan, Park, & Chorpita, 2017b; Guan et al., 2018). These studies have excluded children with more complex clinical profiles (such as those with ASD), as neurodevelopmental disorders were not targets of the relevant EBIs. Therefore, it is critical to extend this work to populations of children with developmental concerns such as ASD, as working with these children presents unique challenges for treatment providers (Brookman-Frazee et al., 2009).

Children with ASD often present with complex diagnostic profiles, including multiple developmental and mental health comorbidities (Leyfer et al., 2006; Simonoff et al., 2008), particularly those receiving care in publicly-funded mental health services (Brookman-Frazee et al., 2017). There is also evidence that children diagnosed with ASD may experience a greater number of family and neighborhood adversities compared with children not diagnosed with ASD (Taylor & Gotham, 2016). Berg and colleagues (2016) found that childhood ASD diagnosis predicted a higher probability of experiencing one or more adverse childhood experiences (ACEs), controlling for a range of other factors (Berg, Shiu, Acharya, Stolbach, & Msall, 2016). In addition, families and children with ASD tend to experience specific challenges such as difficulties at school or episodes of disruptive behavior that can interfere with the therapy process (Totsika et al., 2011; Weiss et al., 2014). For example, children with ASD often experience higher rates of bullying compared to children without ASD, which can result in ELEs of peer or school problems during treatment (Cappadocia, Weiss, & Pepler, 2011; Zeedyk, Rodriguez, Tipton, Baker, & Blacher, 2014). Though these challenges could be related to the symptoms of ASD, they would also be considered ELEs if they occurred within the few weeks prior to the session in which they were disclosed, and consisted of unexpected stressful events that had a negative impact on the child or family (e.g., being suspended from school for fighting). Thus, the presentation of ASD symptoms may interact with the emergence of ELEs during treatment – a fact which could have important implications for the treatment process. Taken together, these factors suggest that ELEs may play an important and unique role in the treatment of children with ASD. Therefore, there is a critical need to characterize and understand ELEs among children with ASD to effectively implement appropriate EBIs in community mental health settings.

An Individualized Mental Health Intervention for Children with Autism Spectrum Disorders (AIM HI)

In response to the growing recognition that clinicians providing publicly-funded mental health services to children with ASD have limited training with this population, An Individualized Mental Health Intervention for Children with Autism Spectrum Disorders (AIM HI; Brookman-Frazee & Drahota, 2010) intervention protocol and corresponding clinician training model were developed for delivery in these service settings. AIM HI is a package of evidence-based caregiver and child directed strategies (National Autism Center, 2009; Wong et al., 2015) aimed at reducing challenging behaviors in children ages 5 to 13 years old with ASD. It was designed based on a systematic needs assessment and in collaboration with community stakeholders and ASD experts. AIM HI consists of a series of intervention protocol steps and within-session elements used to structure sessions and actively teach children alternative skills (e.g., emotional regulation, compliance) and complementary caregiving strategies (e.g., prompting; making transitions predictable, rewards). AIM HI uses a collaborative, caregiver partnership approach (Brookman-Frazee, 2004) to teach caregivers to understand patterns in their children’s behaviors and to use strategies to reduce challenging behaviors and promote acquisition of children’s skills. In addition, pilot data have shown that AIM HI is feasible, clinicians adhered to the treatment protocol, and intervention strategies were perceived to be useful and effective by clinicians and caregivers (Brookman-Frazee, Drahota, Stadnick, & Palinkas, 2012b; Drahota, Stadnick, & Brookman-Frazee, 2014). Building on the successful pilot study, a large-scale randomized community effectiveness trial was recently completed.

The Current Study

The present study utilized data collected as part of a supplemental study of a randomized community effectiveness trial of AIM HI in publicly-funded mental health services aimed to characterize ELEs in children with ASD and to explore the influence of ELEs on observed clinician delivery of AIM HI strategies. Specifically, three research questions were examined in the current study: (1) prevalence of caregiver-reported ELEs when clinicians are trained to deliver AIM HI, (2) session, child, caregiver, and clinician factors associated with ELE occurrence, and (3) associations between ELEs and clinician adherence to AIM HI caregiver-directed strategies within a given session.

Method

Data for this supplemental study were drawn from the AIM HI training condition of the first cohort of the community effectiveness trial of AIM HI. The purpose of the parent trial was to examine the impact of AIM HI training on child and caregiver outcome trajectories, identify moderators of outcomes, and characterize implementation process and outcomes. As part of the trial, publicly-funded community and school-based mental health programs were randomized to immediate AIM HI training and delivery or to a wait-list control/routine care observation condition. Informed consent was obtained from all participants, and all study procedures were approved by the Institutional Review Board at the University of California, San Diego. Participants included in the current analyses were a subset of 38 clinician-child dyads from 5 programs, enrolled in the AIM HI condition.

Caregiver and child participants.

The 38 participating children ranged from 5 to 14 years old at baseline (M = 8.02, SD = 2.20) and were 84.2% male. Caregiver-reported child race/ethnicity was 59.7% Hispanic, 32.3% Non-Hispanic White, and 8.1% Other Minority. Children were considered eligible for the effectiveness trial if they fit the following inclusion: (a) 5–13 years old at the time of recruitment, (2) had a current ASD diagnosis on record, (3) exhibited clinically significant ASD symptoms on a standardized ASD diagnostic measure, and (4) were English or Spanish speaking. The following additional criteria were used for inclusion in the subsample for the current study: (1) clinicians participated in the first of four AIM HI training cohorts, and (2) a caregiver was involved in at least one video-recorded session in the first two months following the initiation of clinician AIM HI training.

The 38 participating caregivers were primarily female (94.7%), and self-reported their race/ethnicity as 37.1% Hispanic, 53.2% Non-Hispanic White, and 9.7% Other Minority. The majority of caregivers indicated a preferred language of English (86.8%), with the remaining 13.2% preferring Spanish. Families’ reported household income was $25,000 or below for 36.8% of families, $25,001 - $50,000 for 26.4% of families, $50,001 - $75,000 for 21.1% of families, and $75,001 or above for 15.8% of families. Six caregivers (15.8%) reported having less than a high school education, 47.4% reported having a high school diploma or GED, and 36.8% reported having some college or more.

The majority of the families (84.2%) received the intervention primarily in an outpatient clinic, while 7.9% received school-based services and 7.9% received services in multiple settings. The primary funding source for services was Medi-Cal (California’s Medicaid) for 84.2% of children and school special education funds for the remaining 15.8% of children.

Clinician participants.

The present study included 33 clinicians who were 78.8% female, with an average age of 33.67 years old (SD = 7.97, range = 23–57). Five clinicians enrolled with two families. Clinicians’ self-reported race/ethnicity was 58.2% Non-Hispanic White, 29.9% Hispanic, and 11.9% Other Minority. Ten clinicians (30.3%) reported being licensed, and 69.7% indicated that they were unlicensed. Regarding professional discipline, 36.4% of clinicians endorsed a marriage and family therapy discipline, 30.3% psychology, 18.2% social work, 12.1% psychiatry, and 3.0% school psychology. The majority (66.7%) of clinicians were Masters-level, with 12.1% Bachelor-level and 21.2% Doctoral-level. Clinicians reported that, on average, they had been practicing for 5.39 years (SD = 4.48, range = 0–15). Clinicians were eligible for the effectiveness trial if they: (1) were employed as trainee or staff at a participating program and were anticipated to be at the program for at least the next seven months, and (2) had a child on their caseload meeting the study inclusion criteria. Additionally, they were required to be enrolled in a dyad with a family who met the inclusion criteria for the subset included in the current study.

Procedure

Intervention.

An Individualized Mental Health Intervention for Children with Autism Spectrum Disorders (AIM HI).

AIM HI is a package of evidence-based caregiver and child directed strategies aimed to reduce challenging behaviors in children ages 5 to 13 with ASD. It consists of a series of intervention protocol steps and within session elements used to structure sessions and actively teach children alternative skills and complementary caregiver strategies. AIM HI clinician training occurs over a 6-month training period. First, clinicians attended an 8-hour workshop designed to introduce them to the underlying concepts, required session elements, and protocol steps of AIM HI (e.g., determining the underlying function of behaviors, developing behavior plans, and skill building using active teaching strategies). The workshop consists of didactic lecture, video exemplars, and interactive practice opportunities. Clinicians were provided with an intervention manual and protocol forms to use for treatment planning and in session with children and families. Following the workshop, clinicians delivered AIM HI for 6 months with participating families while attending a structured consultation series and receiving performance feedback from an expert AIM HI trainer. A total of 11 consultation meetings (9 group, 2 individual) were provided that included dyadic instruction, case-specific feedback, and performance feedback based on review of session videos.

Sampling of sessions for coding.

Data from 100 session videos from the 38 clinician-child dyads were in the current analyses. As part of the parent study, all sessions during the 6-month study observation period were submitted by participant clinicians. Up to four session recordings were randomly selected within three, two-month windows representing the beginning (months 1–2), middle (months 3–4), and end (months 5–6) of treatment to code clinician delivery of AIM HI. For this supplemental study, sessions in which the caregiver was present and the session took place during the first two months were included and a separate coding team conducted observational coding of ELEs. This timeframe was determined as caregiver involvement was emphasized most during the protocol that occur during this time (Conducting Behavior Tracking with Caregivers, Developing a Behavior Plan, Teaching Caregiver Skills). The mean number of sessions included for each clinician-family dyad for the current analyses was 2.63 (SD = 1.15, range = 1–4).

Measures

Session level variables.

Emergent life events (ELEs).

Emergent life events (ELEs), which consist of unexpected stressful events disclosed during a treatment session, were measured with an adapted version of the observational Emergent Life Events Coding System (Guan, Levy, Kim, Brown, et al., 2017a). This system captured both the occurrence and characteristics of the ELEs.

Types of ELEs included the death or health-care problem of child or loved one, family conflict (e.g., parental intimate partner violence), problems at school (e.g., expulsion), change in family routine or structure (e.g., parental separation or divorce), or child engaging in risky behavior (e.g., child destroying property). Additional inclusion criteria were that ELEs occurred recently (e.g., within the past few weeks) and were discrete events. All events involving injury due to violence towards self or others were considered ELEs. Excluded from the definition of an ELE were chronic issues, attendance or engagement issues, symptoms of psychopathology not tied to a discrete event with at least a mild negative impact, medication changes related to the children’s mental health treatment, and future events. Symptoms of psychopathology were coded as an ELE when they were tied to a discrete event with a negative impact. For example, if the child got into a physical altercation with a peer at school and was suspended for one week, the event would be coded as an ELE. In this case, the suspension would qualify this even as an ELE. In addition to ELE content, coders indicated who reported the ELE during the session. In this study, 93% of all ELEs were reported by caregivers. Given this prevalence, and the current study’s focus on sessions with caregiver involvement, only caregiver-reported ELEs were included in the analyses.

Coders also rated the objective intensity or negative impact of the ELE, which captured the severity or negative impact of the event itself when considered in the context of all possible ELEs for all possible children and families. This code was designed to provide an objective rating of the negative effects of the ELE independent of the subjective distress. ELE severity was coded on a scale from 1–5, with 1 reflecting no negative impact, 2 representing mild negative impact, 3 displaying moderate negative impact, 4 showing marked negative impact, and 5 representing severe negative impact. Coders were naive to other information regarding the child and family when assigning this code, and received training regarding the importance of avoiding potential biases (e.g., halo effects). A score of 1 (no negative impact) was assigned to events so minor that any objective stress or negative impact would typically be forgotten or terminated shortly afterward (e.g., the child was scratched by his cat, causing him momentary pain). Scores of 2 (mild negative impact) were assigned to ELEs of relatively brief duration and/or relatively low intensity (e.g., the child talked back in class and received lunch detention, with no further consequences). ELEs of a moderate negative impact were assigned a score of 3, and included ELEs with noticeable impacts on daily life or functioning (e.g., the caregiver lost her job, however, the other caregiver in the family is still working so that the family will be able to survive the financial strain by cutting back on spending). A score of 4 (marked negative impact) were assigned to ELEs with a significant impact on daily life or functioning (e.g., the child’s father moved out of the house and plans to divorce the child’s mother). Finally, a score of 5 (severe negative impact) was assigned to ELEs with a significant, pervasive, and enduring impact on daily life or functioning (e.g., the family is evicted from their home; a primary caregiver or relative who lived with the youth died). Coders rated the objective severity or negative impact of the ELE based on anchors and multiple examples for each category.

Previous studies with high-risk populations have included different severity levels of ELEs, with one study including only ELEs rated as 4 (marked negative impact) and 5 (severe negative impact; Guan, Levy, Kim, Brown, et al., 2017a) and a more recent study including ELEs rated as 3 (moderate negative impact) through 5 (severe negative impact; Guan et al., 2018). This study employed a variation on this conceptualization of ELEs from previous literature, as the current sample tended to be lower risk for environmental stressors (e.g., higher parental income, education) compared with previous samples. In addition, a broader conceptualization of ELEs was utilized to ensure the capture of all of the unique stressors associated with ASD. Therefore, ELEs with severity ratings of 2 (mild negative impact) to 5 (severe negative impact) were included in the current analyses as the total number of mild to severe ELEs. An additional variable, number of moderate to severe ELEs, included only the ELEs with severity ratings of 3 (moderate negative impact), 4 (marked negative impact), and 5 (severe negative impact).

A team of nine psychology undergraduate, post-baccalaureate, graduate, and postdoctoral coders was trained to reliably code ELEs according to the coding manual. Three coders were fluent in Spanish and coded sessions conducted in Spanish. Coder training was conducted by two of the authors and included group didactics, individual practice sessions, and meetings with the trainers to discuss discrepancies with gold standard ratings. After coders achieved 80% reliability on at least six “gold standard” session recordings, they began coding sessions independently. Coding meetings were conducted every 2–3 weeks thereafter to clarify portions of the manual and prevent drift. The coding team was blind to study hypotheses. In previous research in diverse community mental health clinics, ELE occurrence in observationally coded sessions had acceptable agreement with clinician report of ELEs (Guan, Levy, Kim, Brown, et al., 2017a).

The following coding process was implemented to establish ELE occurrence. Two coders were assigned to each video-recorded session, and coded ELE occurrence independently. The two coders then met to reach a consensus regarding the occurrence of ELEs and rated their associated characteristics (e.g., objective negative impact). To ensure that these consensus ratings of ELE characteristics were reliable, if at least one ELE was identified, that session was assigned to a third coder who independently rated the ELE characteristics. In the current sample, using kappas for categorical variables and intraclass correlation coefficients (ICCs; model [2, k], consistency) for continuous variables, inter-rater reliability between consensus and third-party coders for ELE negative impact, reporter, and content category were within the fair to good ranges (kappas = 0.68–0.70; ICCs = 0.53–0.70; Cicchetti, 1994).

Clinician adherence to caregiver-directed strategies.

These data were extracted from the parent study. Clinicians’ use of in-session AIM HI strategies directed to caregivers was measured via naïve observer ratings of adherence to seven caregiver-directed strategies. Adherence ratings were comprised of an extensiveness rating reflecting both the presence/frequency and intensity of use of each strategy. Strategy extensiveness was rated on a scale of 0–6, with a score of 0 indicating that the strategy was not observed and a score of 6 indicating that the strategy was used with high extensiveness. The seven strategies included: (1) using an agenda/session schedule, (2) using materials with caregiver, (3) providing psychoeducation/ information to caregiver, (4) modeling/demonstrating skills to caregiver, (5) providing opportunity for in-session practice to caregiver, (6) providing feedback to caregiver, and (7) assigning/reviewing between-session practice to caregiver. To assess for inter-rater reliability, 25% of the recordings were randomly selected for double coding. Inter-rater reliability for each of the seven strategies was in the good to excellent ranges within the larger effectiveness trial (ICCs = 0.62–0.82) and internal consistency of a composite score representing the mean of these items for a given session (Cronbach’s α = 0.73) was fair. This composite for clinician adherence for caregiver-directed strategies score was used for all analyses.

Child and caregiver level variables.

Child and caregiver demographics.

Caregivers completed a demographic questionnaire concerning demographic information for themselves and their child at the time of enrollment in the study. Child characteristics included child gender, age, and race/ethnicity. In addition, the caregiver reported their own gender, age, race/ethnicity, education, and income.

Child comorbid diagnoses.

An adapted version of the MINI-KID-P (Sheehan et al., 1998) was used to determine the presence of co-occurring psychiatric disorders in addition to the child’s ASD diagnosis. The MINI-KID-P is a structured diagnostic interview designed to assess symptoms of Axis I disorders as listed in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) and ICD-10 (World Health Organization; WHO, 1993), and has been found to have robust psychometric properties (Sheehan et al., 2010). The MINI-KID-P was administered in person or via phone to the child’s primary caregiver prior to beginning AIM HI treatment. The following MINI-KID-P modules were used based on the most common psychiatric comorbid disorders for children with ASD in mental health settings (Brookman-Frazee et al., 2009; Joshi et al., 2010): Attention Deficit/Hyperactivity Disorder (ADHD), Oppositional Defiant Disorder (ODD), Panic Disorder, Agoraphobia, Separation Anxiety Disorder, Social Phobia, Specific Phobia, Obsessive Compulsive Disorder (OCD), Generalized Anxiety Disorder (GAD), Tic Disorders, Major Depressive Episode, Dysthymia, and Manic and Hypomanic Episodes. To facilitate interpretation of study findings, diagnoses derived from these MINI-KID-P modules were collapsed into four diagnostic categories: (1) ADHD, (2) ODD, (3) anxiety disorders, and (4) mood disorders. The anxiety disorders category included panic disorder, agoraphobia, separation anxiety disorder, social phobia, specific phobia, OCD, and GAD. The mood disorders category included major depressive episode, dysthymia, and manic and hypomanic episodes. Tic disorders remained as a separate diagnostic category but were not included in all analyses due to the limited sample size. In addition to diagnostic comorbidity in each of these four categories, a variable consisting of the number of comorbid diagnoses was calculated, ranging from 0–4.

All interviewers were trained to criterion prior to administering the MINI-KID-P by a licensed clinical psychologist with clinical expertise in ASD and diagnostic assessment. Six trained study personnel who had clinical research experience with children with ASD and mental health problems administered the MINI-KID-P to all caregivers in the study via in-person or phone interview. Inter-rater reliability was examined in a sub-sample using the kappa statistic. Kappa values ranged from 0.87–1.00 for the four diagnostic categories (ADHD, ODD, Anxiety, and Mood) examined in this study. Please refer to Stadnick et al. (2017) for a detailed description of the training procedures for MINI-KID-P administration and adaptations for ASD.

Child behavior problems.

The Eyberg Child Behavior Inventory (ECBI; Eyberg and Pincus, 1999) is a 36-item caregiver-report measure that assesses the frequency and intensity of child disruptive behaviors. The ECBI yields two scores that are converted into t-scores (M = 50; SD = 10): an Intensity score that represents the frequency of disruptive behaviors rated on a 7-point Likert scale from (1) never to (7) always, and a Problem score that represents the total number of behaviors that caregivers endorsed (yes/no) as being a problem for them. The ECBI has been shown to have strong test-rest reliability and good construct and concurrent validity (Boggs, Eyberg, & Reynolds, 1990; Eyberg & Ross, 1978; Robinson et al., 1980). In addition, it has also demonstrated strong psychometric properties in clinical samples of children with ASD (e.g., Whittingham, Sofronoff, Sheffield, & Sanders, 2009). In the current full study, the ECBI Intensity scale yielded strong internal consistency (α = .93), and the Intensity t-score was used to characterize the severity of child behavior problems at baseline. T-scores that are greater than 60 are considered clinically significant. In the current sample, 74% of the children met or exceeded the clinical cutoff score on the ECBI Intensity scale at baseline.

Child social-communication impairments.

The Social Responsiveness Scale-2 (SRS-2; Constantino & Gruber, 2012) is a 65-item caregiver-report measure of the severity of social-communication impairments associated with ASD. The SRS-2 has demonstrated strong internal consistency, interrater reliability, and diagnostic discrimination in school-age children (Constantino & Gruber, 2012). The SRS-2 Total Score, which is converted to a t-score (M = 50; SD = 10), was used to characterize the severity of social-communication difficulties. Scores of greater than 60 on the Total t-score are considered clinically significant. At baseline, 97% of the children in the current study scored in the clinically significant range.

Clinician level variables.

Clinician characteristics and training.

Clinicians completed the Therapist Background Questionnaire (Brookman-Frazee, Drahota, & Stadnick, 2012a) concerning personal and professional characteristics at the beginning of the study. Demographic variables included age, gender, and race/ethnicity. Professional background variables included licensure status, professional discipline, highest degree obtained, and years practicing as a clinician. In addition, clinicians indicated whether or not they had received previous training in EBIs or interventions for ASD.

Analytic Plan

Aim 1 sought to characterize the prevalence of caregiver-reported ELEs during early AIM HI sessions. Descriptive analyses were conducted to characterize the frequency, impact, and type of ELEs. For Aims 2 and 3, due to the nested nature of the data (sessions within clinician-child dyads within sites), we determined whether there was significant variance attributable to the clinician-child and site levels by running unconditional models with the outcome of number of ELEs. While a significant proportion of variance across outcomes was attributable to the clinician-child level (ICCs ranging from 0.19 to 0.32), the proportion of variance attributable to site was less than 0.006 for each outcome, demonstrating that modeling a third level was not necessary. Thus, all analyses employed a two-level model with session observations (Level 1; n = 100) nested within unique clinician-child dyads (Level 2; n = 38). All multilevel analyses were run using Stata/SE 15.1.

Aim 2 examined session, child, caregiver, and clinician predictors of ELEs. As indicated for counts of relatively rare events, multilevel Poisson models were run to predict the number of ELEs in a session. Child, caregiver, and clinician variables were included as Level 2 predictors and session variables were included as Level 1 predictors. Two sets of models were run: (1) all ELEs and (2) moderate to severe ELEs. Each predictor was first examined independently to predict number of ELEs, and then a model was analyzed that included all significant predictors. Aim 3 sought to examine the number of ELEs in a session predicting clinician adherence for caregiver-directed strategies. Multilevel regression models were employed to investigate the number of ELEs (Level 1 variable) as a predictor of adherence ratings (also Level 1).

Results

Aim 1: Describe ELEs in AIM HI Sessions

At least one ELE was identified in 36.0% of sessions for 57.9% of children. The majority of ELEs (56.9%) were rated as having mild negative impact, with 39.2% rated as having moderate negative impact, 3.9% rated as having marked negative impact, and 0% rated as having a severe impact. The most common category of ELE was related to medical issues (which included death, injury, or a healthcare problem of a loved one or the child), which occurred in 18 (35.3%) sessions. ELEs rated in the moderate to severe range occurred in 16.0% of sessions for 39.5% of children. See Table 1 for additional descriptive statistics regarding ELEs.

Table 1.

Descriptive Statistics for Emergent Life Events (n = 100 sessions)

Item Description	Mild to Severe ELEs	Moderate to Severe ELEs
Sessions with At Least One ELE, No. (%)	36 (36.0)	16 (16.0)
Clients with At Least One ELE, No. (%)	22 (57.9)	15 (39.5)
Number of ELEs per Session, Mean (SD)	0.5 (0.8)	0.2 (0.6)
Sessions with One ELE, No. (%)	26 (26.0)	11 (11.0)
Sessions with Two ELEs, No. (%)	5 (5.0)	4 (4.0)
Sessions with Three ELEs, No. (%)	5 (5.0)	1 (1.0)
Objective Negative Impact of ELE, Mean (SD)*	2.5 (0.6)	3.1 (0.3)
Mild, No. (%)	29 (56.9)	---
Moderate, No. (%)	20 (39.2)	20 (90.9)
Marked, No. (%)	2 (3.9)	2 (9.1)
Severe, No. (%)	0 (0)	0 (0)
ELE Content Category, No. (%)*
Death/Injury or Health-Care Problem of Loved One or Child	18 (35.3)	11 (50.0)
Family Conflict	16 (31.4)	5 (22.7)
Problems at School	9 (17.6)	4 (18.2)
Change in Family Routine or Structure	8 (15.7)	4 (18.2)
Child Engaging in Risky Behavior	8 (15.7)	3 (13.6)
Other	7 (13.7)	3 (13.6)

^(*)Note: No. = number, SD = standard deviation. Items marked with an asterisk are characteristics of identified ELEs (n = 51 for mild to severe, n = 22 for moderate to severe) rather than the full sample of sessions. For ELE Content Category, percentages do not add up to 100% given that a single ELE could be assigned multiple content categories.

Aim 2: Examine Session, Child, Caregiver, and Clinician Predictors of ELEs

Table 2 shows Pearson correlations for number of ELEs and session, child, caregiver, and clinician variables. Exploratory univariate, multilevel Poisson regression models were run to examine the effect of each predictor individually on the number of ELEs. Based on the results of the univariate, multilevel Poisson regression models, predictors that were associated with a significance of p < 0.05 were retained for the full model. In the univariate models with individual predictors, session language, setting, and child presence in session, child gender, age, baseline SRS score, caregiver gender, and income were not significantly associated with number of ELEs. Finally, clinician variables of gender, race/ethnicity, licensed, prior EBI training, or prior ASD training were not significantly associated with the number of ELEs.

Table 2.

Correlations between Study Variables

Variables	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
Session Level Variables:
1. Number of ELEs (Mild-Severe)	---
2. Number of ELEs (Mod-Severe)	.62**	---
3. Session language a	−.05	−.07	---
4. Session setting b	.14	.08	.01	---
5. Child in session c	−.10	−.13	−.06	.26**	---
Child Level Variables:
6. Child gender d	.09	.05	−.16	.08	−.12	---
7. Child age	−.15	−.02	.11	.14	.02	.12	---
8. Child race/ethnicity e	−.08	−.20*	.29**	.08	−.14	.20*	−.19	---
9. Child ECB1 (baseline)	.26*	.25*	.02	−.03	.04	−.19	−.19	−.26**	---
10. Child SRS (baseline)	.02	.04	.18	.02	−.13	.26**	.05	.08	.34**	---
11. Number of comorbid diagnoses	.30**	.05	−.05	−.01	.12	.29**	−.13	.08	.17	.22*	---
Caregiver Level Variables:
12. Caregiver gender d	−.05	−.13	.08	.09	.26**	−.54**	−.37**	−.22*	.09	−.14	.22*	---
13. Caregiver race/ethnicity e	−.25*	−.19	.44**	−.06	.03	.02	.04	.67**	−.30**	−.01	−.12	−.07	---
14. Caregiver income	.11	−.07	.50**	.10	.11	−.31**	−.02	.23*	−.05	−.14	−.06	.17	.16	---
15. Caregiver education f	.25*	.32**	−.26**	−.12	−.06	.44**	−.13	−.44**	.40**	.33**	.30**	−.06	−.43**	−.39**	---
Clinician Level Variables:
16. Clinician race/ethnicity e	−.12	−.10	.51**	−.05	−.11	−.08	.03	.08	.01	.23*	−.12	.17	27**	.18	−.15	---
17. Clinician year experience	−.21*	−.08	.09	−.28**	.08	−.06	.08	−.15	−.09	.03	.12	.16	.16	−.10	.20*	.09	---
18. Clinician licensed g	.07	.17	−.18	−.11	.13	−.26*	.01	−.44**	.38**	−.08	.03	.14	−.22*	−.21*	.31**	−.36**	.48**	---
19. Clinician prior EBI training h	−.01	.02	−.28**	.24*	−.15	.24*	.25*	−.27**	−.08	−.06	−.18	−.24*	−.43*	−.14	.22*	−.29**	−.07	.10	---
20. Clinician prior ASD training h	−.01	−.03	−.02	−.01	.07	.20*	.11	.01	.20*	.26**	.03	−.19	.07	−.07	.13	−.09	.20	.03	−.03

Note:

^aSession language was coded as 0 = English, 1 = Non-English language;

^bSession setting was coded as 0 = school or mixed setting, 1 = outpatient clinic setting;

^cChild in session was coded as 0 = child not present, 1 = child present;

^dGender was coded as 0 = male, 1 = female;

^eRace/ethnicity was coded as 0 = Non-Hispanic White, Other Minority, 1 = Hispanic;

^fCaregiver education was coded as 0 = high school or less, 1 = some college;

^gClinician licensed was coded as 0 = not licensed, 1 = licensed;

^hClinician prior training coded as 0 = no prior training, 1 = prior training.

^*p < 0.05,

^**p < 0.01.

Based on these results, a multilevel Poisson regression model predicting number of ELEs in session was run which included: total number of child comorbid diagnoses, child baseline score on the ECBI, caregiver race/ethnicity, caregiver education, and clinician years of experience. Table 3 includes the results of this model. When all of these variables were included, only the number of child comorbid diagnoses and clinician years of experience remained significant. Specifically, the expected number of all ELEs reported in session increased by 1.68 times for each additional comorbid diagnosis (95% CI [1.22, 2.32]), an effect that was significant (p < 0.01). In addition, the expected number of all ELEs reported significantly decreased for each year of clinician experience (event rate ratio = 0.91, 95% CI [0.84, 0.99], p < 0.05). A similar model was run to examine predictors of the number of moderate to severe ELEs (see Table 3). In this model, caregiver education significantly predicted the number of moderate to severe ELEs, with caregivers with some college education or more being expected to report 4.39 times more moderate to severe ELEs compared with caregivers with no college education (95% CI [1.22, 15.85], p < 0.05).

Table 3.

Child, Caregiver, and Clinician Predictors of Number of ELEs

Item Description	Mild to Severe ELEs		Moderate to Severe ELEs
	β	SE	β	SE
Child Variables
Number of co-occurring diagnoses	0.52**	0.16	0.01	0.24
Child ECBI (baseline)	0.02	0.02	0.03	0.03
Caregiver Variables
Caregiver race/ethnicity a
Hispanic	−0.35	0.42	−0.11	0.74
Other minority	0.86	0.45	0.68	0.59
Caregiver education b	0.33	0.35	1.48*	0.65
Clinician Variables
Clinician years experience	−0.09*	0.04	−0.07	0.06

Note:

^aRace/ethnicity reference group = Non-Hispanic White;

^bCaregiver education was coded as 0 = high school or less, 1 = some college.

^*p < 0.05,

^**p < 0.01.

The full models were also run as multilevel logit regression models for the presence of a mild to severe ELE in session and the presence of moderate to severe ELE in session. The results were the same as for number of ELEs. The number of child comorbid diagnoses (β = 0.80, SE = 0.28, p < 0.01) and clinician years of experience (β = −0.16, SE = 0.07, p < 0.05) significantly predicted the presence of a mild to severe ELE in session, and caregiver college education (β = 1.56, SE = 0.78, p < 0.05) significantly predicted the presence of a moderate to severe ELE in session.

Aim 3: Examine the Influence of ELEs on Clinician Adherence

Descriptives of in-session clinician adherence to caregiver-directed strategies were examined as an initial step. The clinician adherence composite had a range of 0.29 to 5.57, with a mean of 3.03, a standard deviation of 1.16, and a normal distribution (skewness = −0.001, kurtosis = −0.397), reflecting on average moderate extensiveness of caregiver directed strategies.

The influence of the number of ELEs on in-session clinician adherence to caregiver-directed strategies was investigated with a multilevel regression model, with clinician adherence as the outcome and number of ELEs as the predictor. This analysis showed that the number of all ELEs in a session was not significantly associated with clinician adherence in the same session (β = 0.14, SE = 0.14, p = 0.30). To examine whether higher intensity ELEs might have influenced clinician adherence differently, the number of moderate to severe ELEs was entered into a multilevel regression model, with clinician adherence as the outcome. Similar to the lower intensity ELEs, the number of moderate to severe ELEs in a session was not significantly associated with clinician adherence in the same session (β = −0.14, SE = 0.23, p = 0.55). These results held when controlling for other variables that were associated with ELEs, including clinician years experience.

These analyses were also run with adherence predicted by the dichotomous variable of ELE presence in session. There was no significant relationship between adherence and the presence of a mild to severe ELE in session (β = 0.04, SE = 0.24, p = 0.86) or between adherence and the presence of a moderate to severe ELE in session (β = −0.31, SE = 0.30, p = 0.29).

Discussion

The current study examined the role of caregiver-reported emergent life events (ELEs) in the delivery by community mental health clinicians of an intervention to reduce challenging behaviors in children with ASD. The first aim of this study sought to characterize ELEs in AIM HI sessions. Findings showed that in this context, ELEs were reported in approximately one third of sessions, with nearly all ELEs having mild or moderate impact. The second aim, which examined session, child, caregiver, and clinician predictors of ELEs showed that a greater number of child comorbid diagnoses and fewer years of clinician experience were significantly associated with a higher number of ELEs. Finally, the third aim investigated the influence of ELEs on clinician adherence during AIM HI sessions. These results showed no significant link between the number of ELEs or the presence of an ELE and clinician adherence within a session.

Further examination of the characterization of ELEs in AIM HI sessions revealed some variation in the prevalence of ELEs in children with ASD compared with previous studies involving children without ASD receiving publicly-funded mental health services, though likely not a statistically significant difference. Results from our descriptive analyses indicate that moderate to severe ELEs were reported in 16% of AIM HI sessions, compared with 13% of sessions in previous studies involving children without ASD receiving publicly-funded mental health services (Guan et al., 2018). The current study also expanded measurement of ELEs to include mild and moderate events, in addition to the ELEs with a moderate or severe negative impact examined in previous studies (Guan et al., 2018). When including all ELEs, 36% of the AIM HI sessions were found to include at least one ELE. Given the differences in populations in the current and previous studies, it is notable that the overall rates of ELEs were comparable. The current sample of children had ASD and co-occurring mental health conditions whereas the sample studied by Guan and colleagues (2018) excluded children with ASD. In addition, the current study focused on younger children and included only caregiver-reported ELEs. Previous studies have included both child and caregiver reported ELEs and a broader age range of children. For example, the sample studied by Guan and colleagues (2018) was very high-risk, with a third of families registered with child welfare (past or present), and high rates of homelessness, parental deportation and imprisonment, and child trauma.

In examining session, child, caregiver, and clinician predictors of ELEs, we found that a greater number of child comorbid diagnoses was related to more ELEs. This finding is consistent with a recent study showing similar patterns with child functional impairment and ELEs in a sample of children with non-ASD mental health problems (Chorpita et al., 2014). These results have significant implications for children with ASD, as data from the parent study indicates that children with ASD receiving publicly-funded mental health care present with more than two co-occurring psychiatric diagnoses over and above their ASD (Brookman-Frazee et al., 2017). It is therefore not surprising that caregivers of children with more complex presentations of ASD are reporting more life crises in therapy sessions. These findings suggest that it will be critical for future research to examine therapeutic responses to best address these crises in session within the context of complex diagnostic presentations.

With regard to caregiver factors, we found that sessions with a caregiver with a college education were more likely to have a moderate to severe ELE in session, compared with sessions with a caregiver without a college education. This is counter to what would be expected based on the risk literature, which would posit that families with lower caregiver education may be exposed to a higher number of stressors (Azad, Blacher, & Marcoulides, 2014; Serbin et al, 2011). However, it is possible that caregivers with higher levels of education feel more confident expressing their difficulties to clinicians, compared with caregivers with lower levels education.

We also found that more years as a practicing clinician were associated with fewer ELEs. The difference between this finding and the study by Guan and colleagues (2017a), which demonstrated no significant differences in clinician years of experience between ELE and non-ELE sessions, may be explained by differences in the clinician samples (i.e., clinicians in Guan and colleagues (2017a) study were primarily junior clinicians, with an average of 2.63 years of experience (SD = 2.03) whereas there was greater average and range of experience in present sample (M = 5.39, SD = 4.48). Previous findings have shown that in that sample of less experienced clinicians, ELE occurrence was associated with reduced treatment adherence in session (Guan, Levy, Kim, Brown, et al., 2017a). In combination with the current results, this suggests that more experienced clinicians may be comfortable keeping the sessions “on track” when caregivers raise ELEs.

The current study did not find a significant association between caregiver reported ELEs and clinician adherence to caregiver-directed AIM HI strategies in the same session. This is in contrast to previous work by Guan and colleagues (2018) which examined the impact of moderate to severe ELEs on adherence and found that MATCH sessions with an ELE had significantly lower clinician adherence (Guan et al., 2018). There are a number of possible explanations for this divergence in findings. First, there are several differences in the child population (diagnosis of ASD, younger age range, with and without Medicaid coverage [i.e., not all low income families]), service setting (outpatient and school based), and clinicians (broader range of experience and disciplines [e.g., school psychologists providing school-based mental health services]) between the two samples. It is possible that differences in the client population may help explain the different relationship between ELEs and clinician adherence in the current sample, compared with previous studies. It is also possible that the structure of the EBI may influence the relationship between ELEs and clinician adherence. AIM HI is a highly structured EBI, whereas MATCH has a more flexible content and delivery. Future work should explore the role of EBI prescribed content or order on ELE occurrence and clinician adherence to EBI strategies.

There were also differences between the MATCH and AIM HI samples in the nature of ELEs raised in sessions. Specifically, the MATCH sample had experienced ELEs of higher severity (mean of 3.5 compared with 3.1); and across a broader range of categories (e.g., child abuse, client risky behavior, housing insecurity) than the AIM HI sample (Guan et al., 2018). Perhaps the more minor events in the current study have less of a disruptive impact on treatment, and are more easily addressed within the context of an EBI protocol. In addition, the current study examined ELEs brought up in caregiver sessions, whereas the previous MATCH studies included ELEs that emerged from reports by children and caregivers.

Finally, the focus on a caregiver-mediated intervention for children with ASD may have influenced the relationship between ELEs and clinician adherence. In general, predictable routines and structure are important when working with children with ASD (Cooper, Loades, & Russell, 2018). AIM HI sessions in the first two months are highly structured with targeted activities requiring caregiver input. Establishing this pattern of treatment activities may help focus the content of the sessions and minimize the influence of ELEs for both clinicians and caregivers.

Limitations

Several limitations regarding this study should be noted. First, sample size was relatively small, with 100 sessions from 38 clinician-child dyads. It is possible that this did not provide enough power to detect differences in clinician adherence in sessions with and without ELEs. Although previous studies have found a link between ELEs and clinician adherence with smaller sample sizes (Guan, Levy, Kim, Brown, et al., 2017a), that study had a higher base rate of marked and severe ELEs. Second, ELE occurrence was coded using a consensus based procedure, rather than independently. However, it is important to note that this consensus based method has also been used in prior studies of ELEs. In addition, this study was conducted as part of an effectiveness trial, such that findings hold limited generalizability to the context of routine EBI implementation. For example, clinician behaviors such as adherence to caregiver-directed strategies were likely affected by the ongoing feedback that they received from AIM HI trainers as they delivered AIM HI. The difference in ELE severity assessed in this study also presents challenges in comparing these results with previous work on ELEs in other populations. Additionally, since these supplemental data regarding ELEs were not collected for a control group of children with ASD receiving usual care, it is not known whether the findings can be generalized to all children with ASD in treatment or if they are unique to children with ASD receiving AIM HI. Finally, the sessions included in the sample were selected at random and did not always occur in sequential order. Therefore, it was not possible to examine the effects of ELEs reported in one session on clinician adherence in the following sessions.

Implications

These findings have important implications for clinicians working with children with ASD in community settings, and for the implementation of EBIs for children with ASD. Clinicians are likely to confront ELEs when working with these families. For children with ASD, ELEs often relate to negative impacts of their symptom presentation (e.g., challenging behaviors results in exclusion from community activities [riding the bus]), or events related to family stress (e.g., caregiver separation and divorce). Therefore, it is particularly important to train clinicians working with families of children with ASD how to address and manage ELEs as they emerge during treatment. Several EBIs for other childhood disorders provide specific strategies for managing crises or ELEs, including: limiting the amount of time spent discussing the ELE when there is not a safety issue present, linking the ELE back to the assessment of the child’s functioning and treatment goals, incorporating the ELE as a “natural” opportunity to encourage the use of skills acquired in treatment, and providing additional services to address the ELE, such as wraparound or case management services (Brookman-Frazee, 2004; Cohen, Mannarino, & Deblinger, 2017; McNeil & Hembree-Kligin, 2010). Future research should focus on examining the impact of training clinicians in strategies to manage ELEs on treatment adherence and outcomes, and how training may be adapted to target stressors specific to certain populations, such as children with ASD.

In sum, this study expands the study of ELEs and their impact on EBI delivery to interventions for children with ASD. Results showed that mild to severe ELEs were reported in 36% of sessions, and that greater number of child comorbid diagnoses and less clinician experience were both significantly associated with a higher number of ELEs. The occurrence of mild to moderate ELEs appears to be influenced by diagnostic complexity, which is frequently a challenge in children with ASD. These findings suggest the importance of incorporating clinician training in addressing complex presentations and crises in the context of the larger treatment plan, while maintaining the integrity of the intervention.