Abstract
Objectives:
The purpose of this study was to examine patterns of service receipt and patient outcomes for children receiving Applied Behavior Analysis (ABA) for Autism Spectrum Disorder in an integrated healthcare system where commercially insured children were covered by a state autism mandate.
Method:
This retrospective, observational study used a random sample of children with ASD (3 to 17 years) who were members of a large integrated healthcare system in Southern California and referred for ABA between January 2016 and November 2018. From the 4145 children referred, a random stratified sample of 334 was selected to extract data from clinical reports over 24 months of services. The primary outcome measures were time in ABA and child adaptive behavior.
Results:
13% of the sample never received ABA after referral. Of those who were referred for ABA, 66% initiated ABA and remained in services for 12 months while less than half (46%) remained in services for 24 months. Having a history of special education was associated with longer time spent in ABA, while having a single parent was associated with discontinuation of ABA. A minority of children received a full ABA dose (28%), but the lowest functioning children still experienced clinically significant adaptive behavior gains after 24 months of ABA (P= 0.02).
Conclusion:
In a health system implementation of ABA for children with ASD, there were high rates of ABA discontinuation and low ABA dosing. These challenges may diminish the potential benefits of ABA, even in a context where there is mandated commercial insurance coverage.
Introduction
Autism Spectrum Disorder (ASD) is a developmental disorder characterized by impairments in social skills, behavior, and communication that affects 2.5% of children between the ages of 3 and 17 years in the United States (US).1 It is 4 times more common in boys than girls and often co-occurs with other developmental, psychiatric, chromosomal, or genetic disorders.1,2 Applied behavior analysis (ABA)—a therapeutic intervention for ASD that uses principles of behavioral theory, learning theory, and positive reinforcement to achieve behavioral goals—is widely considered to be a gold-standard, evidence-based intervention for ASD that can improve functional status, behavior, and communication.3-5 There is evidence that early and intensive ABA intervention can improve outcomes for children with ASD. Meta-analyses suggest that ABA results in small to moderate improvement in adaptive behavior, including socialization, communication, and expressive language.5 Such improvement follows a dose-response relationship with number of ABA intervention hours, and generally, 12 to 24 months of ABA or more are needed to produce clinically meaningful progress.3-5 However, the appropriate amount of ABA can vary considerably and more intense or lengthy services do not necessarily always translate to better outcomes for all children with ASD in real-world settings.6,7ABA is an expensive and, at times, intensive intervention, sometimes requiring as much as 40 hours per week of intervention to achieve such adaptive behavior gains.7,8
Historically, the cost of ABA has created access challenges for children and families and there has been limited or inconsistent coverage of ABA by commercial insurers. Some ABA services have been paid for by Medicaid Home & Community-Based Services 1915(c) waivers, which cover ASD services including ABA for children regardless of income.9 However, evidence suggests that despite the option of Medicaid waivers, there are still high levels of unmet service need in the US ASD population and approximately 30% of children with ASD do not receive any therapeutic services.10 The cost, complexity, and insurance challenges of care for youth with ASD has led to recent passage of state-level policies—commonly called autism mandates—that require that some commercial insurers cover ABA and other evidence-based ASD services.11 Approximately half of all commercial insurance plans are exempt from autism mandates, including self-insured firms contracting with health plans to administer employee health benefits only, not to manage their insurance risk pool.13 However, all 50 states and the District of Columbia now have autism mandates which apply to a substantial proportion of their ASD populations.12
Though autism mandates vary considerably by state, there is preliminary evidence that autism mandates may relieve some of the public-sector burden of ASD care and may increase ASD diagnosis rates and utilization of ASD services.12,13-15 However, studies of mandates have relied primarily on insurance claims data showing service utilization with limited measurement of patient clinical outcomes.14,15 In addition to a lack of evidence about the impact of mandate services on patient outcomes, there are many known implementation challenges to translating state autism mandates to actual service delivery, such as a lack of appropriate service providers, increased service demand, low reimbursement rates, high cost-sharing, difficulties with provider licensure, regulatory compliance challenges, and lack of clarity about mandate requirements.16 In order to evaluate how autism mandates affect patients and patient outcomes, there is a need for detailed data on both ABA intensity (number of hours, number of sessions, length of time in services) and patient functional status in an implementation context.17
There has been limited prior research on ASD service delivery patterns with attention to patient outcomes in autism mandate states.18,19 A study in Northern California assessed patterns of ABA referrals, initiation, and discontinuation in an integrated health system in a post-mandate context, but did not include measures of patient outcomes.18 A study of ABA in Southern California assessed patient outcomes after low-intensity ABA in a post-mandate context, but used a small and select sample that may not be representative of the general ASD population.19 It is not well understood how a more robust insurance coverage context for ABA via autism mandates affects patient clinical outcomes, especially in community or health system implementation of ABA where the controlled conditions of research cannot be guaranteed.20 Thus, the purpose of this study was to evaluate ABA service delivery patterns and patient outcomes in an autism mandate state using data from an integrated healthcare system in Southern California where all commercially insured patients were subject to the autism mandate.20 The study objectives were to (1) describe patterns of ABA referral, receipt and discontinuation; (2) identify patient and service history factors associated with length of time in ABA; and (3) identify whether ABA dose and length of time in ABA were associated with patient adaptive behavior outcomes. It was hypothesized that full ABA dosing, defined as receiving at least 80% of prescribed ABA, and longer length of time in ABA would be associated with patient outcome gains.
Methods
Design and Data.
This was a retrospective study that used data from a health system Autism Registry to identify the target population and obtain sociodemographic and service utilization data. Clinical reports obtained through chart review of electronic medical records were used to determine patient adaptive behavior at the time of referral for services and ABA service use characteristics at 6, 12, 18 and 24 months in treatment. The study was approved by the Institutional Review Boards of all study investigators.
Sample and Setting:
The setting for the study was an integrated healthcare system serving 4.7 million members with both commercial insurance and public Medi-Cal insurance from seven counties in Southern California. ABA was provided to commercially insured patients in the health system via a contract with a single external provider, which was initiated in response to the State of California autism mandate in 2012.21 The autism mandate applied to the insurance plans for all commercially insured children in this health system. There were 4145 children ages 3-17 years old in the Autism Registry who received at least one referral for a new episode of ABA with the healthcare system under study between January 2016 and November 2018, and from this patient population, we selected a stratified random sample of 334 children based on a power analysis (80% power to detect medium-size change at 0.05 level of significance). The sample was chosen using strata for age (3-6 years; 7-11 years; 12-17 years), gender (boys, girls), primary insurance type (Commercial, Medi-Cal), and race/ethnicity (White, Black, Hispanic, Asian, Other) to ensure adequate representation of small or under-represented sociodemographic groups in the study sample. Girls and racial/ethnic minorities were over-sampled due to under-representation of these groups in the ASD treatment literature.22
Procedures.
After selecting the sample, we conducted a detailed review of patient electronic medical records to extract data on services delivered and patient outcomes. We developed a chart abstraction form in consultation with operations partners and based upon prior research.22,23 Investigators conducted an initial test series of chart abstractions with 20 cases to develop consistent definitions for abstraction. Because the ABA services were provided by an external contractor, reports on treatment outcomes were provided as scanned reports in the patient electronic medical record. Progress towards treatment goals was reported every 6 months and adaptive behavior outcomes were assessed annually. The ABA referral was required to originate between 2016 and 2018, and then the child’s medical records were reviewed every 6 months for up to 24 months for as long as they were receiving ABA up until December 2019. When patients discontinued services, a termination report was provided with the reason for discontinuation noted. We abstracted discontinuation data for each timepoint.
Measures.
There were two main study outcomes: 1) length of time in ABA and 2) patient clinical outcomes as measured by the Adaptive Behavior Composite (ABC) on the Vineland Adaptive Behavior Scales-Second Edition (Vineland-II). 23,24 The predictor measures were ABA dose (percentage of prescribed ABA hours or sessions received), service history, and patient-level demographics (age, gender, race/ethnicity, and insurance status). Length of time in ABA, the outcome for Aim 1, was also a predictor in Aim 2 where patient outcomes were examined.
Outcome variables.
Length of time in ABA was categorized into three dichotomous variables: patient did not receive ABA, patient received at least 12 months of ABA, and patient received at least 24 months of ABA, for ABA provided by the health system contractor. There is not consensus about the ideal amount of time in ABA needed to achieve clinically meaningful outcomes and duration of ABA has been studied inconsistently in implementation approaches; thus, we examined both 12 and 24 months of ABA as time points.17,25
The Vineland-II measures adaptive behavior that allow function in everyday life for individuals with developmental disabilities.23 The ABC is an overall composite measure of adaptive behavior based on Vineland-II subscales for communication, daily living skills, and socialization. An age-normed ABC mean score is 100 with a standard deviation of 15.23,24 The minimal clinically important difference (MCID) in Vineland-II ABC is 2.0–3.75 points.25
The Vineland-II overall adaptive levels based on ABC are as follows: Low (20-70), Moderately Low (71-85), Adequate (86-114), Moderately High (115-129), and High (130-140).23,24 Because there were very few children in the sample who had a baseline adaptive level of Adequate or higher, these children were combined into a single category of ‘Adequate or above (>85)’ for analyses. The Vineland-II responses were reported by parents or caregivers to the contracted behavioral interventionist, who then returned the scores within scanned annual progress reports.
Predictor Variables and Covariates.
Our predictor variables were ABA dose and service history (past and current receipt of ABA, past and current receipt of special education, speech therapy, occupational therapy). ABA recommendations were sometimes recorded in clinical reports as number of hours and sometimes recorded as number of sessions; thus, we examined ABA dose as the overall percentage of hours or sessions prescribed that were actually received. We considered a full ABA dose to be receipt of at least 80% of prescribed ABA hours or sessions.18 Service history data were collected from parents and/or caregivers during the baseline assessment conducted by ABA providers. ABA dosing data was reported by providers at each 6-month interval. We also included demographic variables from the Autism Registry and progress reports where there may be an influence of family or community context, developmental, or structural inequities on group differences. These included patient age (years), gender, race/ethnicity, primary language spoken (recorded into English or Other), and parent marital/partnership status (married or living with a partner versus single, divorced, or widowed parents). Race/ethnicity categories were White, Hispanic, and Other; the ‘other’ category included Black, Asian American/Pacific Islander, and Native American Indian/Alaskan Native participants because the groups were too small for individual analysis.
Analysis.
Descriptive statistics and chi-square tests were used to compare group differences on all study variables based on length of time in ABA. Logistic regression models were used to identify predictors of time in ABA (no services, at least 12 months in ABA, at least 24 months in ABA) while also adjusting for socioedemographic covariates and baseline adaptive level. A mixed effects model with correlated random intercepts and slopes by subject was used to examine longitudinal change in ABC over length of time spent in ABA and whether time or ABA dose were associated with improved scores while also adjusting for socioedemographic covariates for the overall sample and for the sample stratified by baseline adaptive level. All analyses were conducted using R, version 3.8.0; data were missing at an overall rate of 11.2% for analytic variables and were multiply imputed with chained equations.26
Results
Sample Description
The sample was approximately 78% boys (N= 260) and 48% Hispanic (N= 161, Table 1). Thirty-seven percent of children (N= 124) were publicly insured. A majority (80%, N= 266) spoke English as a primary language and had parents who were married/partnered (63%, N=209). The sample was comprised of 19% children ages 12–17 years (N= 64), 40% children ages 7–11 years (N= 134), and 41% children ages 3–6 years (N= 136). A small percentage of children (15%) had received ABA in the past, while 7.5% reported current ABA at baseline.
Table 1.
Sample Description and Comparison by Length of Time in ABA (N=334)
| Overall Sample |
No ABA | At least 12 months in ABA |
At least 24 months in ABA |
|
|---|---|---|---|---|
| N=334 | N=44 | N=220 | N=154 | |
| N(%) | N(%) | N(%) | N(%) | |
| Gender | ||||
| Boys | 260 (77.8) | 39 (88.6) | 167 (75.9) | 116 (75.3) |
| Girls | 74 (22.1) | 5 (11.4) | 53 (24.1) | 38 (24.7) |
| Age group | ||||
| 3 to 6 years | 136 (40.7) | 14 (31.8) | 94 (42.7) | 65 (42.2) |
| 7 to 11 years | 134 (40.1) | 21 (47.7) | 87 (39.5) | 63 (40.9) |
| 12 to 17 years | 64 (19.2) | 9 (20.5) | 39 (17.7) | 26 (16.9) |
| Race/ethnicity | ||||
| White | 87 (26.0) | 14 (31.8) | 60 (27.3) | 43 (27.9) |
| Hispanic | 161 (48.2) | 21 (47.7) | 104 (47.3) | 78 (50.6) |
| Other | 86 (25.7) | 9 (20.5) | 56 (25.5) | 33 (21.4) |
| Insurance | ||||
| Commercial | 210 (62.8) | 27 (61.4) | 142 (64.5) | 101 (65.6) |
| Medi-Cal | 124 (37.1) | 17 (38.6) | 78 (35.5) | 53 (34.4) |
| Primary Language** | ||||
| English | 266 (79.6) | 24 (54.5) | 194 (88.2) | 134 (87.0) |
| Other language | 68 (20.4) | 20 (45.5) | 26 (11.8) | 20 (13.0) |
| Parent marital/partnership status** | ||||
| Married/partnered | 209 (62.6) | 13 (29.5) | 158 (71.8) | 113 (73.4) |
| Unmarried/unpartnered | 125 (37.4) | 31 (70.5) | 62 (28.2) | 41 (26.6) |
| History Special Education | ||||
| Past | 64 (19.2) | 6 (13.6) | 49 (22.3) | 36 (23.4) |
| Current** | 155 (46.4) | 12 (27.3) | 115 (52.3) | 80 (51.9) |
| History of ABA | ||||
| Past | 56 (15.0) | 5 (11.4) | 37 (16.8) | 24 (15.6) |
| Current | 25 (7.5) | 1 (2.3) | 20 (9.1) | 14 (9.1) |
| History of Occupational Therapy** | 142 (42.5) | 9 (20.5) | 106 (48.2) | 73 (47.4) |
| History of Speech Therapy** | 201 (60.2) | 13 (29.5) | 150 (68.2) | 106 (68.8) |
Percentages are by column. ABC= Adaptive Behavior Composite; ABA= Applied Behavior Analysis. This table shows chi-square tests comparing groups on study variables based on length of time in ABA with a sample (N=334) of children referred for ABA in 2016 or later (*Group difference was significant at the 0.05 level; **Group difference was significant at the 0.01 level).
There were 44 children (13%) who never received ABA. Sixty-six percent of the initial sample referred for ABA was receiving ABA at 12 months (N=220) and 46% of the initial referred sample was receiving ABA at 24 months (N=154) (Figure 1). For those who discontinued ABA at 12 months, 14% of discontinuations were because the patient met their treatment goals. For those who discontinued ABA at 24 months, 21% of discontinuations were because the patient met their treatment goals. All others discontinued of ABA for reasons unrelated to clinical progress (e.g., family declined, financial difficulties, family moved, insurance change; Supplemental Digital Content 1, Figure).
Figure 1. Time in Applied Behavior Analysis over 24 Months.
This figure shows the percentage of the sample (N= 334 children referred for Applied Behavior Analysis [ABA] in 2016 or later) retained in ABA at 6-month intervals over the course of the 24-month study. By 24 months, 46% of the sample (N=154) remained in services.
Of the 154 children who received ABA for 24 months, 28% received a full ABA dose. Fifty-eight percent of this sample achieved the ABC MCID between baseline and 12 months and 54% achieved the MCID between baseline and 24 months. Because there is no specified time frame for achieving the Vineland-II, MCID we examined both timepoints.25
Predictors of Time in ABA (None, 12 months, 24 months)
Children were similar on demographic characteristics (age, gender, race/ethnicity, insurance type, and parent marital/partnership status) across length of time spent in ABA (Table 1). Those who primarily spoke a language other than English were over-represented in the group that did not receive any services and under-represented in the group that received ABA for 12 or 24 months (P <.01). Children who were currently receiving special education services (P <.01) and who had a history of speech therapy (P <.01) were over-represented in the groups that were in ABA for 12 and 24 months.
In adjusted logistic regression models predicting time in ABA (Table 2), those with a married/partnered parent had lower odds of membership in the group that never received ABA. Likewise, in the model predicting membership in the 12- and 24-month time in ABA groups, parent married/partnered status was associated with higher odds of retention for both time points. Receiving special education services was associated with higher odds of 12 months of time in ABA.
Table 2.
Predictors of Time in ABA
| No ABA | At least 12 months of ABA |
24 months of ABA | ||||
|---|---|---|---|---|---|---|
| OR | 95%CI | OR | 95%CI | OR | 95%CI | |
| Female gender | 2.90 | 0.98, 8.57 | 0.66 | 0.35, 1.26 | 0.77 | 0.44, 1.34 |
| Age (years) | 1.07 | 0.96, 1.19 | 0.97 | 0.90, 1.05 | 0.96 | 0.89, 1.03 |
| Race/ethnicity (reference: Hispanic) | ||||||
| White | 1.15 | 0.48, 2.75 | 1.37 | 0.72, 2.61 | 1.06 | 0.60, 1.87 |
| Other | 0.99 | 0.38, 2.58 | 0.88 | 0.47, 1.63 | 0.55 | 0.31, 0.99 |
| Commercial insurance (reference: Medi-Cal) | 1.13 | 0.51, 2.53 | 0.89 | 0.52, 1.52 | 0.82 | 0.50, 1.36 |
| Parents partnered/married | 0.31 | 0.14, 0.67 | 2.16 | 1.28, 3.66 | 1.87 | 1.13, 3.11 |
| Primary language English | 0.49 | 0.14, 1.10 | 2.91 | 1.55, 5.47 | 1.84 | 0.97, 3.51 |
| Service history | ||||||
| History Special Education | 0.74 | 0.30, 1.82 | 1.83 | 1.02, 3.30 | 1.42 | 0.84, 2.39 |
| History of ABA | 0.57 | 0.19, 1.70 | 0.92 | 0.49, 1.76 | 1.01 | 0.56, 1.81 |
| History of Occupational Therapy | 0.73 | 0.28, 1.95 | 1.24 | 0.68, 2.24 | 1.03 | 0.61, 1.75 |
| History of Speech Therapy | 0.51 | 0.20, 1.32 | 1.37 | 0.74, 2.53 | 1.37 | 0.77, 2.41 |
| Baseline functional status (reference: Low [ABC <70]) | ||||||
| Moderately low (ABC 70-85) | 1.71 | 0.43, 6.83 | 1.18 | 0.58, 2.41 | 0.19 | 0.38, 1.21 |
| Adequate or above (ABC >85) | 0.04 | 0.0, >20.0 | 1.38 | 0.24, 8.10 | 0.51 | 0.19, 2.32 |
ABC= Adaptive Behavior Composite; ABA= Applied Behavior Analysis; OR= odds ratio; 95%CI= 95% confidence interval. This table shows logistic regression models predicting length of time in ABA (no ABA; at least 12 months in ABA; at least 24 months in ABA) for a sample of 334 children referred for ABA in 2016 or later.
Predictors of Patient Adaptive Behavior Outcomes
We examined ABC scores for the subsample of youth retained in ABA for 24 months (N=154; Table 3). ABC trajectories over the study time period showed trends towards improvement, particularly for the group with the lowest baseline adaptive level (Supplemental Digital Content 2, Figure). In bivariate tests, these children with the lowest baseline adaptive level (ABC <70) were more likely to achieve a MCID on the ABC at both 12 and 24 months. The average ABC change from baseline to 24 months was 4.9 points for the overall sample, with the following changes by subgroup: Low= 9 points; Moderately Low= 0.1 points; Adequate or above= −6.7 points. In the adjusted longitudinal model examining changes in ABC over time, neither time in ABA nor receiving a full ABA dose were significantly associated with ABC for the sample as a whole. The ABC gains associated with each 12-month increment of time in ABA reached the MCID and approached the level of statistical significance (β= 2.49, SE= 1.13, P= 0.06). For children with the lowest adaptive level at baseline, the adjusted 12-month ABC gain was 4.46 points (SE= 1.53, P= 0.02).
Table 3.
Comparison of 24-Month Sample by Baseline Adaptive Level (N=154)
| Total (N=154) |
Low (N=91, 59.1%) |
Moderately Low (N=55, 35.7%) |
Adequate or above (N=8, 5.2%) |
P | |
|---|---|---|---|---|---|
| N | N (%) | N (%) | N (%) | ||
| Gender | 0.63 | ||||
| Boy | 116 | 66 (56.9) | 44 (37.9) | 6 (5.2) | |
| Girl | 38 | 25 (16.2) | 11 (28.9) | 2 (5.3) | |
| Age group (baseline) | 0.01 | ||||
| 3 to 6 years | 65 | 31 (47.7) | 30 (46.2) | 4 (6.2) | |
| 7 to 11 years | 63 | 37 (58.7) | 22 (34.9) | 4 (6.3) | |
| 12 to 17 years | 26 | 23 (88.5) | 3 (11.5) | 0 (0) | |
| Race/ethnicity | 0.07 | ||||
| White | 43 | 23 (53.5) | 18 (41.9) | 2 (4.7) | |
| Hispanic | 78 | 47 (60.3) | 30 (38.5) | 1 (1.3) | |
| Other | 33 | 21 (63.6) | 7 (21.2) | 5 (15.2) | |
| Insurance | 0.16 | ||||
| Commercial | 101 | 54 (53.5) | 41 (40.6) | 6 (5.9) | |
| Medi-Cal | 53 | 37 (69.8) | 14 (26.4) | 2 (3.8) | |
| Primary Language | 0.25 | ||||
| English | 134 | 77 (57.5) | 49 (36.5) | 8 (6.0) | |
| Other language | 20 | 15 (75.0) | 5 (25.0) | 0 (0) | |
| Parent marital/partnership status | 0.09 | ||||
| Married/partnered | 113 | 61 (54.0) | 46 (40.7) | 6 (5.3) | |
| Unmarried/unpartnered | 41 | 30 (73.2) | 9 (22.0) | 2 (4.9) | |
| History of Special Education | 87 | 59 (67.8) | 23 (26.4) | 5 (5.7) | 0.04 |
| History of ABA (any) | 35 | 25 (71.4) | 9 (25.7) | 1 (2.9) | 0.27 |
| History of Occupational Therapy | 73 | 42 (57.5) | 28 (38.4) | 3 (4.1) | 0.72 |
| History of Speech Therapy | 106 | 62 (58.5) | 38 (35.8) | 6 (5.7) | 0.92 |
| Full ABA dose (≥80%) | 43 | 24 (55.8) | 17 (39.5) | 2 (4.7) | 0.81 |
| Vineland ABC Change ≥2.5 at 12 months (MCID) | 90 | 63 (70.0) | 25 (27.8) | 2 (2.2) | <.01 |
| Vineland ABC Change ≥2.5 at 24 months (MCID) | 83 | 57 (68.7) | 24 (28.9) | 2 (2.4) | 0.02 |
Percentages are by row. ABC= Adaptive Composite; ABA= Applied Behavior Analysis; MCID= Minimal clinically important difference. This table shows chi-square tests comparing baseline adaptive level groups on study variables with a sample (N=154) of children referred for ABA in 2016 or later.
Discussion
This study of service delivery and patient outcomes after implementation of California’s autism mandate found that approximately 66% of children referred for ABA stay in services for 12 months and 46% stay in services for 24 months, with relatively low rates of full ABA dosing. For those who initiated—but then discontinued—ABA, most documented reasons for ABA discontinuation were unrelated to progress on treatment goals. Children who had a prior history of receiving ASD services (speech therapy, special education) were more likely to stay in ABA than those who did not. This may be related to comfortability and experiences with services, family enthusiasm for services, or ASD severity when there is a higher level of service need. Having a parent who was married/partnered increased the odds of remaining in ABA for 12 and 24 months, suggesting that caregiver support plays a role in service receipt and continuation.
Of the 46% of children who remained in ABA for 24 months, the average adjusted ABC increase over 24 months was 2.49 points. Although this ABC increase was not statistically significant for the overall sample, children with the lowest baseline adaptive level were more likely than their higher-functioning peers to achieve the Vineland-II MCID of 2.0–3.75 points. They had a modest, but significant, ABC increase of 4.46 points associated with each 12 months of time spent in ABA. Children with the lowest baseline adaptive level maintained gains over time in ABA, despite the relatively low rate of full ABA dosing (28%). These findings align with prior studies showing the benefit of ABA for children with ASD,17 but it is important to consider that gains observed in our study may also reflect regression to the mean. This study included a small subgroup of children whose baseline adaptive level was adequate or above, and ABC trajectories suggested that these children benefitted little from ABA, contrary to what other studies have suggested.3-5 We were unable to explore this subgroup in more detail due to the small subgroup size, but future studies should explore the benefits of ABA and other interventions for children with ASD whose adaptive behavior is developmentally appropriate in community and health system settings.
These findings suggest that even in a state context where there is mandated commercial insurance coverage for ABA and there was a single contracted provider of ABA, there are health system implementation challenges that may limit the extent to which interventions translate to meaningful patient outcomes. A similar investigation of children with ASD in Northern California over 12 months of ABA found 31% ABA discontinuation, 15% ABA receipt of a full ABA dose, and 23% referral without subsequent services.18 Other studies have found low ABA utilization rates and high discontinuation rates for child behavioral health services generally.28 Given the real-world challenges of implementing interventions like ABA, it is perhaps unsurprising that our study identified modest changes in our patient outcome measure.29 In implementation science, health interventions shown to be effective in controlled research studies often fail to translate to meaningful patient outcomes in practice.29 Evidence for ABA’s efficacy comes from clinical trials with a high degree of intervention fidelity, but such conditions cannot always be guaranteed in a real-world health system. Recent evaluations of community implementation of ABA have shown lower dosing and more modest functional status gains that what would be expected based on research studies.20,30 Implementing optimal ABA dosing is especially challenging in practice because clinical need must be balanced with family acceptability and feasibility of services.28 States with autism mandates should consider whether an implementation strategy is in place for the mandate and how health systems and health insurance providers can be supported with implementation resources and account for real-world health system and community barriers to treatment access that may not have been considered in research supporting ABA’s use.29
This study has strengths and limitations that should be considered in interpreting the findings. Few studies to date have examined ASD services and their relationship to patient outcomes simultaneously. This research builds on prior studies of ABA after implementation of California’s autism mandate that were limited by sample size and generalizability issues and a lack of patient outcomes data.18,19 Our study used a diverse sample in terms of gender, age, and race/ethnicity and multiple measures of service history and patient characteristics that are not available in claims datasets. As a retrospective study using clinical data, there were some study limitations inherent to the design and dataset. The study sample was clinical in nature, and thus children in the sample may have had more severe ASD than the general ASD population. Most children in the sample had commercial insurance subject to a state autism mandate, limiting conclusions that can be drawn about the Medi-Cal population. We were unable to capture patient outcomes for those who discontinued ABA or to measure ABA fidelity, as well as variables not collected as part of routine healthcare including school outcomes and socioeconomic characteristics of families (e.g., family income, parent employment). The patient outcome measure available in health system records, the Vineland-II, includes only adaptive behavior, not maladaptive behavior or other behavioral measures. Finally, our primary outcome measure was a parent-report measure and as such may have been biased towards under-estimation of patient progress out of desire for service continuation.
Conclusion
Implementing evidence-based interventions for ASD in a real-world health system comes with challenges that may not lead to the same outcomes seen in clinical trials, even with mandated insurance coverage. Two-thirds of children referred for ABA stay in services for 12 months while less than half stay in services for 24 months. Despite low rates of full ABA dosing and high service discontinuation over time, children with the lowest adaptive level at baseline made clinically and statistically significant adaptive behavior gains. Future studies should explore unknown reasons for service discontinuation, additional patient outcome measures including those for maladaptive behaviors, optimal interventions for high-functioning children with ASD, and incorporate school records. ASD is a priority issue for health systems and communities, and it is important that health services and policies are optimized for improved patient-centered outcomes. To effectively translate research evidence on ASD interventions into wide-scale practice, an implementation framework is needed for states with autism mandates to address system, community, and family barriers and promote meaningful patient progress.
Supplementary Material
Supplemental Digital Content 1
Figure A: Reasons for Applied Behavior Analysis Discontinuation over 24 Months
This figure shows reasons for discontinuing Applied Behavior Analysis (ABA) at each study timepoint (N= 334 children referred for ABA in 2016 or later). For each timepoint, the number of discontinuations by reason is shown as a percentage of the total number of discontinuations at that timepoint.
Figure B: 24-Month Adaptive Behavior Trajectories by Baseline Adaptive Level
This figure shows trajectories in Adaptive Behavior Composite (ABC), a measure of adaptive behavior from the Vineland Adaptive Behavior Scales-Second Edition, over 24 months of Applied Behavior Analysis (N=154). The sample is stratified by baseline ABC adaptive level.
Acknowledgements
We are grateful to the children and families whose data made this research possible. The authors use person-centered language to describe Autism Spectrum Disorder in this paper because this is the standard in scientific and clinical literature. However, we acknowledge that such language is not preferred by all individuals in the disability community and that there is heterogeneity in how individuals with autistic traits and their families prefer to identify themselves. Clinicians should always refer to patients with the language the patient prefers.
Disclosures and Funding:
The authors have no conflicts of interest to disclose. This research was supported by a K12 (K12HS26407-01) career development award from the Agency for Healthcare Research and Quality (AHRQ) and the Patient-Centered Outcomes Research Institute (PCORI). The award was provided through the Stakeholder-Partnered Implementation Research and Innovation Translation (SPIRIT) Program at UCLA and Kaiser Permanente Southern California. Funding was also provided by the Southern California Permanente Medical Group.
References
- 1.Maenner MJ, Shaw KA, Baio J, et al. Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016 [published correction appears in MMWR Morb Mortal Wkly Rep. 2020. April 24;69(16):503]. MMWR Surveill Summ. 2020;69(4):1–12. Published 2020 Mar 27. doi: 10.15585/mmwr.ss6904a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Soke GN, Maenner MJ, Christensen D, et al. Prevalence of co-occurring medical and behavioral conditions/symptoms among 4-and 8-year-old children with autism spectrum disorder in selected areas of the United States in 2010. J Autism Dev Disord. 2018;48(8):2663–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ospina MB, Seida JK, Clark B, et al. Behavioural and developmental interventions for autism spectrum disorder: a clinical systematic review. PloS One, 2008;3(11):e3755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Reichow B, Barton EE, Boyd BA, et al. Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD). Cochrane Rev, 2012(10). [DOI] [PubMed] [Google Scholar]
- 5.Yu Q, Li E, Li L, et al. Efficacy of Interventions Based on Applied Behavior Analysis for Autism Spectrum Disorder: A Meta-Analysis. Psychiatry Investigation. 2020;17(5):432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Pellecchia M, Iadarola S, Stahmer AC. How meaningful is more? Considerations regarding intensity in early intensive behavioral intervention. Autism. 2019. 10.1177/1362361319854844 [DOI] [PubMed] [Google Scholar]
- 7.Virués-Ortega J Applied behavior analytic intervention for autism in early childhood: Meta-analysis, meta-regression and dose–response meta-analysis of multiple outcomes. Clin Psychol Rev. 2010;30(4):387–99. [DOI] [PubMed] [Google Scholar]
- 8.Buescher AV, Cidav Z, Knapp M, et al. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr. 2014;168(8):721–8. [DOI] [PubMed] [Google Scholar]
- 9.Velott DL, Agbese E, Mandell D, et al. Medicaid 1915 (c) Home-and Community-Based Services waivers for children with autism spectrum disorder. Autism. 2016;20(4):473–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Xu G, Strathearn L, Liu B, et al. Prevalence of autism spectrum disorder among US children and adolescents, 2014-2016. JAMA. 2018;319(1):81–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Choi KR, Knight EA, Stein BD, et al. Autism Insurance Mandates in the US: Comparison of Mandated Commercial Insurance Benefits Across States. Maternal and child health journal. 2020. Matern Child Health J. DOI: 10.1007/s10995-020-02950-2. [DOI] [PubMed] [Google Scholar]
- 12.Buchmueller TC, Cooper PF, Jacobson M, et al. Parity for whom? Exemptions and the extent of state mental health parity legislation. Health Aff, 2007;26(4):w483–7. [DOI] [PubMed] [Google Scholar]
- 13.Barry CL, Epstein AJ, Marcus SC, et al. Effects of state insurance mandates on health care use and spending for autism spectrum disorder. Health Aff, 2017;36(10):1754–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Stein BD, Sorbero MJ, Goswami U, et al. Impact of a private health insurance mandate on public sector autism service use in Pennsylvania. J Am Acad Child Adolesc Psychiatr. 2012;51(8):771–9. [DOI] [PubMed] [Google Scholar]
- 15.Mandell DS, Barry CL, Marcus SC, et al. Effects of autism spectrum disorder insurance mandates on the treated prevalence of autism spectrum disorder. JAMA Pediatr. 2016;170(9):887–93. [DOI] [PubMed] [Google Scholar]
- 16.Baller JB, Barry CL, Shea K, et al. Assessing early implementation of state autism insurance mandates. Autism, 2016;20(7):796–807. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Granpeesheh D, Tarbox J, Dixon DR. Applied behavior analytic interventions for children with autism: a description and review of treatment research. Annals of clinical psychiatry. 2009. August 1;21(3):162–73. [PubMed] [Google Scholar]
- 18.Croen LA, Shankute N, Davignon M, et al. Demographic and clinical characteristics associated with engagement in behavioral health treatment among children with autism spectrum disorders. J Autism Dev Disord. 2017;47(11):3347–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lotfizadeh AD, Kazemi E, Pompa-Craven P, et al. Moderate effects of low-intensity behavioral intervention. Behavior Modification. 2020;44(1):92–113. [DOI] [PubMed] [Google Scholar]
- 20.Nahmias AS, Pellecchia M, Stahmer AC, et al. Effectiveness of community-based early intervention for children with autism spectrum disorder: a meta-analysis. Journal of Child Psychology and Psychiatry. 2019. November;60(11):1200–9. [DOI] [PubMed] [Google Scholar]
- 21.State of California. Senate Bill 946 (SB-946) Health care coverage: mental illness: pervasive developmental disorder or autism: public health. 2012. Ch. 650. http://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201120120SB946
- 22.Kirkovski M, Enticott PG, Fitzgerald PB. A review of the role of female gender in autism spectrum disorders. J Autism Dev Disord. 2013;43(11):2584–603. [DOI] [PubMed] [Google Scholar]
- 23.Carter AS, Volkmar FR, Sparrow SS, et al. The Vineland Adaptive Behavior Scales: supplementary norms for individuals with autism. J Autism Dev Disord. 1998;28(4):287–302. [DOI] [PubMed] [Google Scholar]
- 24.Sparrow SS, Cicchetti DV, Balla DA. The vineland adaptive behavior scales. Major psychological assessment instruments. 1989;2:199–231. [Google Scholar]
- 25.Chatham CH, Taylor KI, Charman T, et al. Adaptive behavior in autism: Minimal clinically important differences on the Vineland-II. Autism Research. 2018;11(2):270–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. J Stat Software. 2011;45(3):1–67. [Google Scholar]
- 27.Yingling ME, Bell BA. Underutilization of early intensive behavioral intervention among 3-year-old children with autism spectrum disorder. J Autism Dev Disord. 2019;49(7):2956–64. [DOI] [PubMed] [Google Scholar]
- 28.Gopalan G, Goldstein L, Klingenstein K, et al. Engaging families into child mental health treatment: updates and special considerations. J Can Acad Child Adolesc Psychiatry. 2010;19(3):182–196. [PMC free article] [PubMed] [Google Scholar]
- 29.Odom SL, Hall LJ, Suhrheinrich J. Implementation science, behavior analysis, and supporting evidence-based practices for individuals with autism. European Journal of Behavior Analysis. 2019:1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Eldevik S, Titlestad KB, Aarlie H, et al. Community Implementation of Early Behavioral Intervention: Higher Intensity Gives Better Outcome. Eur J Behav Anal. 2019:1–8. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Digital Content 1
Figure A: Reasons for Applied Behavior Analysis Discontinuation over 24 Months
This figure shows reasons for discontinuing Applied Behavior Analysis (ABA) at each study timepoint (N= 334 children referred for ABA in 2016 or later). For each timepoint, the number of discontinuations by reason is shown as a percentage of the total number of discontinuations at that timepoint.
Figure B: 24-Month Adaptive Behavior Trajectories by Baseline Adaptive Level
This figure shows trajectories in Adaptive Behavior Composite (ABC), a measure of adaptive behavior from the Vineland Adaptive Behavior Scales-Second Edition, over 24 months of Applied Behavior Analysis (N=154). The sample is stratified by baseline ABC adaptive level.