Training Community Therapists to Deliver an Individualized Mental Health Intervention for Autism Spectrum Disorder: Changes in Caregiver Outcomes and Mediating Role on Child Outcomes

Abstract

OBJECTIVE:

This study examines the impact of training therapists to deliver “An Individualized Mental Health Intervention for autism spectrum disorder (ASD)” (AIM HI) for children with autism spectrum disorder on caregiver outcomes and the mediating role of changes in caregiver outcomes on child outcomes.

METHOD:

Data were drawn from a cluster randomized trial conducted in 29 publicly-funded mental health programs randomized to receive AIM HI training or usual care. Therapists were recruited from enrolled programs and child/caregiver participants enrolled from therapists’ caseloads. Participants included 202 caregivers of children aged 5 to 13 with autism spectrum disorder. Caregiver strain and sense of competence were assessed at baseline and 6 month post baseline. Child behaviors were assessed at baseline and 6, 12, and 18 months post baseline. Therapist delivery of evidence-based intervention strategies were assessed between baseline and 6 months.

RESULTS:

A significant training effect was observed for caregiver sense of competence, with AIM HI caregivers reporting significantly greater improvement relative to usual care. There was no significant training effect for caregiver strain. Observer-rated therapist delivery of evidence-based interventions strategies over 6 months mediated training effects for sense of competence at 6 months. Changes in sense of competence from baseline to 6 months was associated with reduced child challenging behaviors at 6 months and mediated child outcomes at 12 and 18 months.

CONCLUSION:

Combined with research demonstrating effectiveness of therapist AIM HI training on child outcomes, this study provides further evidence of the positive impact of training community therapists in the AIM HI intervention.

Introduction

Many evidence-based psychosocial interventions for children, including those for children with autism spectrum disorder (ASD) ^1–3, include substantial parent involvement by teaching parents strategies to facilitate their children’s skill development. For children with ASD, behavioral interventions often actively target parent participation as an essential element of treatment ⁴. Parent mediated interventions were developed for children with ASD to facilitate the generalization of child skills learned in therapy and to increase the intensity of intervention through increased teaching opportunities by allowing the parent to serve as a “co-therapist” in the home ^{5, 6}. In addition, as parents are the decision makers regarding their child’s treatment, they are able to provide important information that contributes to treatment planning and goal setting.

Challenging behaviors refer to a broad range of problem behaviors that may include physical or verbal aggression, noncompliance, tantrums, self-injury, or inappropriate social or withdrawal behaviors. They are common in children with ASD ⁷ and the primary presenting problems for children with ASD served in community mental health settings,⁸ with previous research suggesting that 80% of children in mental health services meet criteria for at least one disorder commonly associated with these behaviors (e.g., attention deficit hyperactivity disorder, oppositional defiant disorder) ⁹. Challenging behaviors are a significant sources of stress for caregivers ^{10, 11, 12} and can impact parent psychological adjustment with positive child behaviors, or lack thereof, serving as key drivers of maternal adjustment over time ¹³. They have been found to predict depression in caregivers of children with ASD, over and above the impact of ASD symptoms alone^{13, 14}. The potential impact of child behaviors is particularly relevant for parents of children with ASD as they experience considerably greater parenting stress than parents of typically developing children or children with other disabilities ¹⁵ with these increased levels of stress associated with an increased risk for mental health disorders in parents ^16–19.

Parent training interventions have increasingly been studied as an avenue to address challenging behaviors in children with ASD. A recent systematic review indicated support for the efficacy of parent training interventions in reducing challenging behaviors for children with ASD ²⁰, while a separate meta-analysis found positive effects of behavioral parent training interventions on both disruptive and hyperactive challenging behaviors¹¹.

In addition to these important findings related to child outcomes, given the significant role parents play in the treatment of children with ASD and the level of stress experienced by these caregivers, a focus on caregiver outcomes for interventions in this population is warranted. One study examining the impact of behavioral parent training in families of children with developmental disabilities and challenging behaviors found reduced parental stress and increased parent self-efficacy in managing child challenging behaviors in families who completed the intervention ²¹. A recent study examined outcomes for parents of children with ASD and challenging behavior from a randomized clinical trial of active parent training versus psychoeducation and found greater improvement in parent competence, and greater reduction on several indices of parental stress and strain for parents in the parent training condition ²². A recent systematic review examined the affective and parenting cognition outcomes of parent management training for mothers and fathers of children ages 2–13 years, across 48 controlled treatment studies. Substantial support was found for reductions in parenting stress, and increases in perceived parenting competence following parent management training²³.

The majority of studies including caregiver outcomes in parent training interventions have examined them as collateral effects of parent training and there is limited information available about the associations between caregiver outcomes and child outcomes. While there has been some work suggesting a mediating role of parenting behaviors in changes in child behaviors²⁴,²⁵, there is limited information available about the impact of changes in parental feelings of competence or stress and strain on child outcomes. One study examining mediators of change in parent training for 3- to 8-year-old children with ADHD found increased parental self-efficacy and reduced negative parenting statistically mediated reductions in child ADHD and conduct problems; however, time-lagged analyses were unable to detect a causal relation between prior change in caregiver mediator variables and subsequent child symptom reduction²⁶.

While caregiver outcomes have begun to be studied in efficacy trials, examination of these important secondary outcomes in the context of community effectiveness trials in which evidence-based interventions (EBIs) are tested under routine care conditions is needed. Examination of caregiver outcomes, and associated child outcomes, in children’s mental health services settings is especially warranted as publicly-funded mental health services are an important community service setting providing care for school-age children with ASD due to high rates of co-occurring psychiatric conditions (i.e. over 70%) in this population ^27–30. Challenging behaviors have been documented as the primary presenting problem for school age children seeking mental health treatment in this service setting ³¹. Research assessing gaps between research and routine care in children’s mental health services indicates that community mental health therapists have limited training in addressing mental health problems in the ASD population ^32,33 In response to the need for a scalable intervention protocol and therapist training model, AIM HI (“An Individualized Mental Health Intervention for ASD”) was developed for use in publicly funded mental health service settings for children with ASD and challenging behaviors.

The AIM HI intervention is a package of parent-mediated and child-focused intervention designed to reduce challenging behaviors in children with ASD aged 5 to 13 years. A pilot study of AIM HI provided preliminary evidence of the feasibility of training mental health therapists to deliver the AIM HI intervention and positive changes in child challenging behaviors after working with a therapist trained in AIM HI ^{31, 34}. Importantly, caregivers and therapists reported a significant shift in the focus and structure of psychotherapy such that caregivers attended more sessions, collaborated with the therapists to teach child skills, and learned skills to manage their child’s challenging behaviors. Following the pilot study, a large-scale cluster randomized controlled community effectiveness trial was conducted in two large counties to examine the impact of training mental health therapists in AIM HI. The primary analyses of this study indicated that children whose therapists participated in AIM HI training and consultation demonstrated significantly greater improvement in challenging behaviors compared to children whose therapists did not receive AIM HI training. The effects of therapist training on child outcomes were mediated by therapist use of EBI strategies included in the AIM HI intervention³⁵.

The current study extends the child outcome findings reported from the effectiveness trial by examining caregiver outcomes from caregiver self-report measures of parental stress and feelings of parental competence collected at baseline and after 6 months of services (i.e., the length of the therapy observation period and time period in which the largest percentage of families were active with a participant therapist ³⁵). Potential moderators (child and caregiver characteristics) and mediators (therapist EBI strategy delivery) of training effects on caregiver outcomes were examined. Examination of change in the caregiver outcomes as a potential mediator of child outcomes ³⁵ was also explored.

Method

Data on caregiver outcomes were extracted from a cluster randomized waitlist control design used to examine the effectiveness of AIM HI training/consultation. As described in the primary outcome report ³⁵, and using a protocol approved by the University of California, San Diego Institutional Review Board, publicly-funded outpatient clinic and school-based mental health programs in San Diego and Los Angeles counties were invited to enroll in the study. Programs were eligible if they were 1) publicly funded, and 2) provided ongoing psychotherapy services to children.

Randomization occurred at the program level to prevent contamination at the therapist level and was completed by an independent statistical investigator. Randomization was stratified by agency size using randomly permuted blocks according to a computer-generated assignment sequence prepared in advance by the study statistician. Based on an imbalance in the average number of participants enrolled in usual care (UC) vs AIM HI during the first year, a restricted randomization approach ^{36, 37} was used for subsequent years to achieve a better balance in the sample size by condition. The randomization ratio was recalibrated from 1:1 to a 2:1 ratio of programs randomized to UC..

Participants

Participants in the current study included therapists and caregivers enrolled in the community effectiveness trial. Following program enrollment and randomization, therapists were recruited from within enrolled programs and consented for participation. Children and caregivers were enrolled from participant therapists’ caseloads and caregivers consented for participation. Informed written consent was obtained from all therapist and caregiver participants.

Therapist participants.

Therapists were eligible for study participation if they (1) provided psychotherapy services in an enrolled program, (2) anticipated working in the program for at least the next seven months, and (3) had a least one eligible child on current caseload. See Table 1 for information about therapist participants.

Table 1.

Program, Participant and Treatment Characteristics

	Usual Care	AIM HI
Program characteristics	n=16	n=22
Setting
Outpatient clinic	50.0%	45.5%
School	50.0%	18.2%
Multiple settings (school, clinic, home)	0%	36.4%
Therapist characteristics	n=42	n=130
Therapist age (years) M(SD)	35.29 (8.20)	33.53 (8.07)
Therapist gender (female)	88.1%	84.6%
Therapist race/ethnicity (%)
Hispanic/Latinx	40.5%	32.3%
Non-Hispanic White	57.1%	46.2%
Other Minority/Multiracial	2.4%	21.5%
Primary mental health discipline (%)
Marriage and Family Therapy	47.6%	40.0%
Clinical Psychology	7.1%	19.2%
School Psychology	11.9%	10.0%
Social Work	26.2%	26.2%
Other Discipline a	7.2%	4.6%
Licensed in mental health discipline (%)	28.6%	27.7%
Caregiver/family characteristics	n=46	n=156
Child age (years) M(SD)	9.35 (2.38)	9.06 (2.46)
Child Gender (male)	82.6%	84.6%
Child Hispanic/Latinx	67.4%	57.7%
Child Cognitive Standard Score	86.00 (16.23)	89.27 (16.64)
Child ADOS-2 Comparison Score	6.84 (1.91)	7.11 (2.06)
Child SRS-2 Total T-Score	82.98 (9.795)	79.20 (11.45)
Caregiver age (years) M(SD)	38.76 (6.97)	40.55 (8.56)
Caregiver gender (female)	97.8%	92.3%
Caregiver race/ethnicity
Hispanic/Latinx	54.3%	50.6%
Non-Hispanic White	30.4%	35.9%
Other Minority/Multiracial	15.2%	13.5%
Caregiver preferred language (Spanish)	34.8%	28.2%
Caregiver marital status (married)	42.2%	55.1%
Maternal education level
Less than high school	15.6%	20.0%
Completed high school	48.9%	34.8%
Any trade school/college	35.6%	45.2%
Annual household income
≤$25,000	56.5%	41.7%
$25,001–75,000	30.4%	39.7%
>$75,000	13.0%	18.6%

Note: ADOS-2 = Autism Diagnostic Observation Schedule, Second Edition; SRS-2 = Social Responsiveness Scale, Second Edition

^aOther Discipline includes Psychiatry, Licensed Professional Clinical Counselor, Art Therapist

Caregiver and child participants.

After initial enrollment, therapists referred children for an eligibility assessment conducted by study staff. Children were eligible if they: 1) were aged 5–13 years old at the time of recruitment, 2) spoke English or Spanish as their primary language, 3) had an existing ASD diagnosis on record, and/or 4) exhibited clinically significant ASD symptoms on at least one of two standardized ASD diagnostic measures: a score in the ASD or autism range on the Autism Diagnostic Observation Schedule-2 (ADOS-2)³⁸ or a score in the moderate to severe range the Social Responsiveness Scale-2 (SRS-2) ³⁹. Caregiver participants enrolled with their participating child. See Table 1 for demographic information for children and caregivers. Caregiver participants whose children received treatment from an AIM HI trained therapist will be referred to as AIM HI caregiver while caregivers whose children received care as usual are referred to as usual care caregiver.

Intervention

The AIM HI clinical intervention is a package of evidence-based strategies aimed to reduce challenging behaviors in children with ASD aged 5 to 13 designed for delivery in publicly-funded mental health services. AIM HI is a parent mediated intervention which involves collaborating with the child’s caregiver to identify the primary functions of a child’s challenging behaviors and actively teaching (through modeling, behavioral rehearsal, reinforcement, and in-home practice) child and caregiver skills. AIM HI uses “within session elements,” or strategies designed to adapt psychotherapy structure for ASD clients and their families to facilitate engagement and skill building, such as using schedules and visuals supports and incorporating the child’s special interests into session activities (see previous publications ^{31, 35} for more information about the intervention and training protocol).

The AIM HI therapist training occurs over a 6-month training period that occurs while the therapist delivers the AIM HI intervention to a child on their current caseload. The training model includes active, direct, and explicit instructional methods and contains the following components: (a) an introductory training workshop involving didactic lecture, video examples, and planning for AIM HI delivery with a target client; (b) 11 consultation meetings with an expert AIM HI trainer (nine group, two individual), approximately 1 hour in length over a 6-month period that provide didactic instruction and case-specific feedback; (c) performance feedback from an AIM HI trainer based on review of the therapist’s session recordings with participating families; and (d) access to a therapist manual and printed and electronic forms.

Usual Care (UC) Waitlist Control

Therapist participants in the waitlist control condition delivered routine care to participating children and their caregivers. They submitted recordings of their therapy sessions to the research team over the 6-month observation period and completed all study measures; no AIM HI training or clinical feedback was provided to usual care therapists.

Procedure

Baseline eligibility assessments were conducted by the research team for all referred children. Data collection from caregivers occurred at baseline, 6 months, 12 months, and 18 months, regardless of whether their child remained in services with the participant therapist. Session recordings occurred only during the 6 month window of study participation (BL to 6 months). Families received a $40 giftcard for completion of the baseline assessment and a $25 giftcard for each subsequent interview.

Measures

Eligibility Assessments.

Baseline eligibility assessments consisted of Autism Diagnostic Observation Schedule (ADOS-2) ³⁸, Social Responsiveness Scale (SRS-2)³⁹, and a measure of cognitive functioning (Wechsler Abbreviated Scale of Intelligence-II (WASI-II) ⁴⁰ or the Differential Ability Scale-II (DAS-II) ⁴¹ based on the child’s age). Child and caregiver demographic information was collected from caregivers.

Outcome assessments.

Caregivers were asked to complete outcome assessments at Baseline, 6 months, 12, months, and 18 months.

Caregivers self-reported their experience of caregiver stress using the Caregiver Strain Questionnaire (CGSQ) ⁴³. The CGSQ is a 21-item scale that measures the impact of caring for a child with emotional and behavioral problems in six areas: economic burden, impact on family relations, disruption of family activities, psychological adjustment of family members, stigma/anger, and worry/guilt; higher scores indicate higher caregiver strain. The CGSQ total scores at baseline and 6 months were used in the current analyses.

Perceptions of parental competence were assessed using the Parenting Sense of Competence (PSOC) ⁴⁴. The PSOC scale measures parental competence and self-efficacy on two dimensions: Satisfaction and Efficacy. It is a 17 item Likert-scale questionnaire on a 6 point scale. Satisfaction section examines the parents’ anxiety, motivation and frustration, while the Efficacy section looks at the parents’ competence, capability levels, and problem-solving abilities in their parental role. Higher scores indicate greater reported competence. The PSOC total scores at baseline and 6 months were used in the current analyses.

Child’s challenging behaviors were assessed using the Eyberg Child Behavior Inventory (ECBI) ⁴². The ECBI is a parent-report measure assessing challenging behaviors in children aged 2–16 years. The ECBI includes 36 items, rated on a dichotomous Problem scale and a 7-point Intensity scale. Higher scores indicate more frequent (Intensity scale) or more problematic (Problem scale) behavior problems. The Intensity and Problem scales at Baseline, 6, 12, and 18 months were used for the child outcome analyses.

Observed therapist delivery of evidence-based intervention strategies.

Trained coders naive to study condition coded in-session therapist EBI strategy delivery (for both UC and AIM HI trained therapists) from video-recorded therapy sessions collected from baseline to 6 months. Recordings were randomly selected from two month windows representing the beginning (months 1–2), middle (months 3–4), and end (months 5–6) of the observation period and scores were averaged across coded sessions for the 6 month period. The average number of coded sessions per child was 7.5 sessions (SD= 3.27). To assess for inter-rater reliability, 25% of recordings were randomly selected for double coding; reliability was moderate to high with ICCs ranging from .62 to .89.

Ratings were made by observers using a Likert scale ranging from 0 to 6 with a score of 0 indicating that the clinical strategy was not observed; a score of 1 or 2 indicating the strategy observed to be used with low frequency or intensity; a score of 3 or 4 indicating that the strategy was observed to be used with moderate frequency/intensity; and a score of 5 or 6 indicating high frequency/intensity of strategy use. In the current study, a composite score reflecting the extent to which the therapist was observed to use in-session therapeutic strategies directed to caregivers (Therapist EBI Strategy Delivery of Caregiver Directed Strategies) and three Effectiveness ratings (Session Structure; Treatment Continuity, Therapist Pursuit of Caregiver Skill) reflecting the observed effectiveness of these strategies were used for analyses.

Data Analytic Plan and Power Calculation

Intraclass correlation coefficients (ICCs) were calculated to determine the amount of variability in each outcome that could be attributed to the clustered nature of the data (i.e., families/therapists nested within programs). Minimal variability was evident at the program level for all outcomes (<.001-.001). Nevertheless, robust standard errors were estimated for all models to account for the clustered data structure. Differences in caregiver outcomes at 6 months post training/intervention delivery between training groups were tested while controlling for baseline (pre-training and intervention delivery) levels. Moderators (e.g., caregiver/therapist characteristics) of treatment group differences were statistically evaluated by adding interaction terms to the models. To determine if therapist EBI strategy delivery ratings (e.g., therapist use of caregiver directed strategies, session structure, treatment continuity) mediated the training effect on caregiver 6 month outcomes, we tested the significance of indirect effects using asymmetric confidence intervals (CIs); CIs that do not contain the value 0 are considered statistically significant mediated effects ^{45, 46}. A similar approach was used to determine if change in caregiver outcomes at 6 months mediated the training effect on child behavior outcomes at 6, 12, and 18 months. These models also included unstructured autoregressive effects for the target outcomes controlling for baseline levels.

All analyses used an intent-to-treat approach and the Maximum Likelihood Robust estimation procedure implemented by the Mplus software ⁴⁷. The estimation procedure adjusts for missing data that can be treated as Missing at Random (MAR) and non-normality of the outcome variables. Though it is difficult to definitively determine that missing data meet these criteria, all study variables were tested and none were significantly associated with missing data patterns. Data were treated as though they were MAR ⁴⁸. The power analysis program RMASS2 ⁴⁹ was used to estimate the sample size necessary to find statistically significant (α=.05) AIM HI training effects, assuming a between-group effect size of d = .40 on the primary outcome of the trial (ECBI) and an overall attrition rate of 20%. Moreover, clustering at the clinic level was accounted for with the design effect, with an average number of therapists per clinic assumed to be 7 and an intraclass correlation coefficient of .05. Given these assumptions, 206 clients/therapists nested with 29 clinics were needed at the beginning of the study to detect the predicted training effect with 80% power.

Results

Descriptives for AIM HI caregivers and usual care caregivers are reported in Table 1 and treatment process and outcome data are reported in Table 2. In preliminary analyses, the association between proportion of treatment sessions attended by the caregiver was examined as potential covariate of caregiver outcomes. Caregivers in the AIM HI group (79.1%) attended a significantly greater proportion of sessions than caregivers in the Usual Care group (39.0%) , B = 39.73, p < .001; however, the proportion of treatment sessions attended by the caregiver was not significantly associated with caregiver strain, B = .00, p = .54, or parental sense of competence, B = .02, p = .14, and was not included in additional analyses.

Table 2.

Treatment Process and Outcomes

	Usual Care	AIM HI
Caregiver PSOC total score
Baseline M(SD)	72.98 (10.14)	71.09 (10.49)
6-month M(SD)	71.39 (9.25)	72.21 (10.67)
Caregiver CGSQ total score
Baseline M(SD)	2.76 (.81)	2.69 (.85)
6-month M(SD)	2.44 (.68)	2.47 (.79)
Child ECBI Problem Scale a
Baseline M(SD)	62.54 (10.04)	64.14 (10.53)
6-month M(SD)	62.89 (9.74)	62.49 (10.87)
12-month M(SD)	62.24 (10.84)	61.86 (12.20)
18-month M(SD)	61.76 (11.69)	60.55 (12.40)
Child ECBI Intensity Scale b
Baseline M(SD)	63.07 (9.50)	63.08 (10.70)
6-month M(SD)	59.78 (8.82)	60.29 (10.92)
12-month M(SD)	59.62 (9.47)	59.36 (10.85)
18-month M(SD)	59.11 (10.03)	59.30 (10.24)
Sample Size at each Timepoint
Baseline	46	156
6-month	36	140
12-month	34	132
18-month	28	122
Treatment Process
Total number of attended sessions M(SD)	13.59 (6.70)	15.05 (6.36)
Percent of total sessions attended by caregiver	39.0%	79.1%
Therapist Delivery of Caregiver Directed Strategiesc	1.84 (0.77)	3.22 (0.74)
Therapist Effectiveness Ratingse
Session Structure M(SD)	2.80 (1.0)	4.55 (0.71)
Treatment Continuity M(SD)	2.47 (1.11)	4.52 (0.80)
Therapist Pursuit of Caregiver Skill M(SD)	1.27 (1.30)	3.41 (1.11)

Note: CGSQ = Caregiver Strain Questionnaire; ECBI = Eyberg Child Behavior Inventory; PSOC = Parenting Sense of Competence

^aThe ECBI Intensity scale represents the frequency of disruptive behaviors.

^bThe ECBI Problem scale represents the total number of behaviors endorsed as being a problem for the caregiver.

^cTherapist EBI strategy delivery ratings were scored on a 7-point Likert scale reflecting the extent to which the strategy was used in a specific session. Scores of 0 indicate that the strategy was not observed. Scores of 1–2, 3–4, and 5–6 reflect low, moderate, and high extensiveness, respectively. Therapist EBI strategy delivery ratings were averaged across 6 months of services.

Caregiver 6 Month Outcomes by AIM HI Training Group

Caregiver Strain Questionnaire.

There were no significant differences between AIM HI caregivers and usual care caregivers on their reported strain at 6 months, B = .04, p = .59, d = .06, 95% CI for d [−.16, .27], after controlling for baseline levels in each group.

Parenting Sense of Competence.

AIM HI caregivers reported significantly higher parental sense of competence relative to usual care caregivers at 6 months, B = 2.35, p = .03, d = .23, 95% CI for d [.02, .44], after controlling for baseline levels in each group.

Caregiver and Child Characteristics as Moderators of Effects on Caregiver Outcomes

Caregiver characteristics.

Caregiver characteristics, such as race/ethnicity (Hispanic vs. White p = .26, other minority vs. White p = .76), preferred language (p = .77), maternal level of education (p = .60) did not significantly moderate differences between AIM HI caregivers and usual care caregivers in reported competence.

Child characteristics.

Children’s baseline level of challenging behaviors, as measured by baseline caregiver-rated ECBI score, did not significantly moderate differences between AIM HI caregivers and usual care caregivers in sense of competence (p = .19).

Therapy Process as Mediators of Effects on Parenting Sense of Competence

Although there was a significant difference in the percentage of sessions that caregivers attended (AIM HI caregivers attended 79% of sessions compared to 39% attendance for usual care caregivers), the percentage of sessions that caregivers attended (95% asymmetric CI [−1.7, .21]) did not significantly mediate group differences in parenting sense of competence.

Training group differences in parental sense of competence were not significantly mediated by observer rated caregiver-directed strategies composite (measuring the extent to which the therapist was observed to use EBI strategies directed to caregivers in session), 95% asymmetric CI [−.55, 4.15]; however, group differences were significantly mediated by observer treatment continuity scores, 95% asymmetric CI [.42, 5.31], and observer ratings of therapist pursuit of caregiver skills (measuring the effectiveness with which therapists pursued teaching a caregiver skill in session), 95% asymmetric CI [.01, 4.36].

Specifically, although AIM HI trained therapists displayed higher EBI strategy delivery of caregiver-directed stratgies, indicating they delivered significantly more extensive EBI strategies to caregivers in session relative to usual care therapists, B = 1.39, p < .001, 95% CI [.88, 1.90], R² = .39, caregiver-directed strategies was not associated with a greater parenting sense of competence, B = 1.30, p > .12 95% CI [−.33, 2.93], β = .12. AIM HI therapists also had significantly higher treatment continuity scores than usual care therapists, B = 2.05, p < .001, 95% CI [1.65, 2.39], R² = .5, which in turn was associated with greater sense of parental competence, B = 1.36, p = .02, 95% CI [.21, 2.51], β = .16. Moreover, therapists who received AIM HI training were rated to pursue caregiver skills more effectively than therapists who did not receive AIM HI training, B = 2.15, p < .001, 95% CI [1.52, 2.78], R² = .39, which was in turn associated with a greater parental sense of competence after 6 months of intervention delivery, B = .96, p = .049, 95% CI [.01, 1.76], β = .13.

Changes in the Parenting Sense of Competence as Mediators of Effects on Child Outcomes

Changes in parenting sense of competence and short term (6 month) child outcomes.

Indirect associations between group and ECBI Problem scale, 95% asymmetric CI [−.88, −.04] and the ECBI Intensity scale, 95% asymmetric CI [−.60, −.03] scores at 6 months via parent sense of competence were statistically significant. AIM HI caregivers reported higher parental sense of competence relative to usual care caregivers at 6 months, and parent sense of competence was associated with lower child ECBI Problem scores, B = −.17, p < .001, 95% CI [.21, 2.51], β = −.17 and lower ECBI Intensity scores, B = −.12, p < .001, 95% CI [−.25, −.08], β = −.12 at 6 months.

Changes in parenting sense of competence and long term (12 and 18 month) child outcomes.

Parent sense of competence significantly mediated the group effects for the ECBI Intensity scale at 12 months, 95% asymmetric CI [−.52, −.01]. AIM HI caregivers reported higher parental sense of competence relative to usual care caregivers at 6 months, which in turn was associated with lower child ECBI Intensity scores at 12 months, B = −.09, p = .006, 95% CI [−.16, −.03], β = −.10. Parent sense of competence significantly mediated the training group effects for the ECBI Problem scale at 18 months, 95% asymmetric CI [−1.31, −.003]. AIM HI caregivers reported higher parental sense of competence relative to usual care caregivers at 6 months, which in turn was associated with lower child ECBI Problem scores at 18 months, B = −.21, p = .03, 95% CI [−.40, −.02], β = −.19.

Discussion

The current study used a cluster randomized trial to examine the effectiveness of therapist AIM HI training and consultation on key caregiver outcomes – caregiver strain and parental sense of competence – within the context of publicly-funded mental health services. Both groups of caregivers whose children received mental health services reported significant decreases in caregiver strain after 6 months of intervention and there was not a significant difference between training groups. In contrast, AIM HI caregivers reported greater increases in parenting sense of competence after 6 months of services with a therapist who participated in AIM HI training compared to caregivers in the usual care group. The changes in parenting sense of competence were associated with improved child behavior outcomes at six months and mediated these effects at 12 (ECBI Intensity) or 18 months (ECBI Problem).

There results suggest that parental involvement in mental health services working with a therapist to address child challenging behaviors may reduce feeling of caregiver stress and strain. The differential findings between the two caregiver outcomes (strain versus competence) may relate to the focus of caregiver involvement in AIM HI. That is, there is not a specific component in AIM HI targeting reduction of caregiver strain or stress, there is, however, an explicit focus on building caregiver skills and increasing feelings of parental competence and confidence in using skills with their children. The development of caregiver skills is built into the AIM HI protocol across all intervention phases (Treatment Planning, Active Teaching, and Evaluating Progress) and guidance about caregiver skill development is built into the feedback provided to therapists at AIM HI training workshops and consultations. This focus on teaching caregivers skills to use with their child may be related to caregivers’ increases in reported feelings of competence in the parenting role.

Results from the mediation analyses highlight the importance of therapist delivery of evidence-based intervention strategies, with therapist observed strategy delivery over 6 months mediating caregiver 6 months outcomes, which is consistent with previous findings examining the impact of AIM HI on child behavior outcomes ³⁵. Results indicate that therapist use of treatment continuity across sessions mediated caregiver outcomes. This mediation finding was also observed in examination of child outcomes³⁵ indicating that helping therapists increase their continuity across treatment sessions is a strategy that has the potential to both reduce child challenging behaviors and increase parental competence. The additional therapist strategies that influenced outcomes differed for children and their caregivers, with previous findings indicating that effective use of session structure mediated child outcomes, while therapist pursuit of a caregiver skill mediated caregiver outcomes.

Lastly, changes in parenting sense of competence at 6 months was associated with effects on short and long term child behavior outcomes, highlighting the importance of directly targeting caregivers in child interventions to both teach caregiver skills and increase caregiver self-efficacy to improve child outcomes. This finding further supports the effectiveness in training therapists in a model targeting caregiver skills improves caregiver and child outcomes.

There are several important contributions from the current study. These results were from a community effectiveness trial of an ASD intervention conducted in publicly-funded mental health services and adds to the limited outcome data available from effectiveness and implementation trials in routine mental health care. The effectiveness of training community therapists to deliver AIM HI was tested under ecologically valid conditions which allows for examination of the factors needed to successfully implement AIM HI in community settings.Therapists trained in AIM HI were not required to demonstrate fidelity to the AIM HI intervention prior to intervention delivery with clients, rather the training and intervention delivery occurred concurrently in the first 6 months of the study. Even in the AIM HI group, observed therapist delivery of EBI strategies was modest and not all trained therapists ultimately reached criterion fidelity. This study context provides a very conservative test of AIM HI delivery in usual case settings using an intent to treat approach and as such, the small effect size for parental sense of competence is not surprising. This small effect is important as evidenced by the associations with the effects on child outcomes. Furthermore, the use of a cluster randomized trial mitigates concerns related to contamination given intervention implementation in community settings and was a strength of the study design.

The current findings should also be considered with the following limitations. The outcomes measured were limited to caregiver report of feelings of strain and competence and caregiver report of child behaviors; given their participation in the intervention, clients/caregivers were not blind to study condition. As a result, the trial relied on unblinded caregiver report for both caregiver and child outcomes, which is a limitation of this study. While no objective measures of strain or competence were collected, caregiver self-perceptions are meaningful and provide important information about their perceptions of success and strain in their parenting role. Although the AIM HI intervention is designed to be parent mediated and included explicit teaching of caregiver skills, given that the intervention was implemented by therapists working in community mental health settings, the participation of parents could not be guaranteed and parent involvement differed by training condition. Although AIM HI caregivers attended significantly more sessions than their usual care counterparts (see Table 2), they were not present for all therapy sessions.

The findings of this cluster randomized community trial provide support for the effectiveness of training therapists working in publicly-funded mental health services to deliver evidence-based strategies to reduce challenging behaviors to children with ASD with positive impacts of training on caregiver reported sense of competence and child outcomes. The mediating role of therapist use of session structure, home practice/treatment continuity, and explicit caregiver in session skill-building highlights the importance of developing and testing implementation strategies aimed to improve therapist delivery of evidence-based intervention strategies. Finally, the significant associations between changes in parental sense of competence on effects on child behavior outcomes and their mediating role for long term child outcomes highlights the importance of directly targeting the development of caregiver self-efficacy in managing behaviors in children with ASD. Increasing caregiver self-efficacy has the potential to positively impact child outcomes for children with ASD. Combined with research demonstrating the effectiveness of therapist AIM HI training on child outcomes, this study further demonstrates that training community therapists in AIM HI provides benefits to both caregivers and children and that caregiver and child benefits are related.

Figure 1: Study Flow Diagram.

Note: Figure 1 presents the CONSORT flow diagram from the parent trial (Brookman-Frazee et al., 2019³⁵) describing the cluster randomized waitlist control design. As described in Brookman-Frazee et al. (2019) after MH programs enrolled in the study, they were randomized to immediate (Wave 1 = W1) AIM HI training or to a usual care (UC) /waitlist control condition. After programs in the waitlist control completed the UC condition, they were invited to participate in the delayed (Wave 2 = W2) AIM HI training. Therapists were recruited from enrolled programs and children were recruited from the caseloads of enrolled therapists in each wave. AIM HI = An Individualized Mental Health Intervention for ASD; MH = ??.

^a There were 43 programs provisionally enrolled in the study; 13 had insufficient numbers of clients with autism spectrum disorder (ASD) and were thus ineligible.

^b Ineligible: 1 left the clinic; 32 did not refer the client.

^c Therapist may have referred more than 1 child if the first referred child was not enrolled in the study or if there was more than 1 eligible child on the caseload.

^d Ineligible: 1 left the clinic; 8 had no eligible client.

^e Three ineligible based on phone screen; 6 declined; 2 unable to contact.

^f Two unable to contact; ineligible: 1 left the clinic; 13 did not refer the client.

^g Two unable to contact; ineligible: 1 left the clinic; 1 did not provide psychotherapy; 32 did not refer the client.

^h Ineligible: 1 left the clinic; 8 had no eligible client.

ⁱ Ineligible: 11 had no eligible client.

^j Five ineligible based on phone screen; 2 declined; 2 unable to contact.

^k Fifteen ineligible after phone screen; 10 declined; 4 unable to contact.

^l Ineligible: 6 did not have ASD.

m Ineligible: 6 did not have ASD; 3 had an ineligible referring therapist.

Figure 2. Parenting Sense of Competence (PSOC) by Training Group.

Note: AIM HI = Individualized Mental Health Intervention for ASD.

Figure 3: Conceptual Mediation Model Examining Parenting Sense of Competence (PSOC) Outcomes at 6 Months via Therapist Delivery of Evidence-Based Intervention Strategies.

Note: Path coefficients labeled d, represent Cohen’s d effect sizes for between group effects. Path coefficients labeled β represented standardized linear regression coefficients.