Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Aug 1.
Published in final edited form as: Autism Res. 2022 Apr 19;15(8):1565–1575. doi: 10.1002/aur.2727

Super Responders: Predicting language gains from JASPER among limited language children with autism spectrum disorder

Jonathan Panganiban 1, Connie Kasari 2
PMCID: PMC9357035  NIHMSID: NIHMS1796769  PMID: 35437928

Abstract

Early intervention can provide a great benefit for children with autism spectrum disorder (ASD). However, no single intervention is effective for all children. Even when an intervention is effective overall, individual child response varies. Some children make incredible progress, and others make slow or no progress. Therefore, it is important that the field move towards developing methods to personalize intervention. Operationalizing meaningful change and predicting intervention response are critical steps in designing systematic and personalized early intervention. The present research used improvement in expressive language to group children that received a targeted social communication early intervention, Joint Attention, Symbolic Play, Engagement, and Regulation (JASPER), into super responders and slow responders. Using baseline data from traditional standardized assessments of cognition and behavioral data from validated experimental measures of play and social communication, we used conditional inference tree models to predict responder status. From a sample of 99 preschool age, limited language children with ASD, play diversity was the most significant predictor of responder status. Children that played functionally with a wider variety of toys had increased odds of being a super responder to JASPER. A combination of lower play diversity and impairments in fine motor abilities increased the odds of children being slow responders to JASPER. Results from the present study can inform future efforts to individualize intervention and systematic approaches to augmenting treatment in real time.

Keywords: Data-driven techniques, Intervention – Behavioral, Intervention Early, Language, Treatment research, JASPER, Machine Learning, Responders

Lay Summary

To help us answer the question of for whom an intervention works best, we examined 99 children, age three to five, who qualified as being limited spoken language communicators, and received a targeted intervention for social communication and language. We used child characteristics before intervention to predict which children would improve their language the most and found that the ability to play appropriately with a wider variety of toys predicted the best improvements in expressive language. These findings will help better inform future work to individualize intervention based on the unique needs of each child.

Introduction

Identifying measures of meaningful change and predicting intervention response are critically important as we begin to design early intervention programs that can systematize and personalize interventions for children with autism spectrum disorder (ASD). We know that it is unlikely that any single intervention is effective for all children (National Research Council, 2001); even when effective overall, individual child response varies. The present research examines characteristics of children that show accelerated growth (super responders - defined as children who make faster than expected gains in language based on standardized measurement) and slower growth (slow responders - continuing to show slower progress) in response to a targeted social communication early intervention known as Joint Attention, Symbolic Play, Engagement and Regulation (JASPER).

Predicting Intervention Response

To better understand ways to personalize treatment for children with ASD, researchers often attempt to identify predictors of treatment response. However, this is complicated by the many ways response is operationalized and measured. Some studies report predictors to a specific intervention while others may report predictors to intervention in general. For example, Itzchak et al. (2011) found no treatment differences between children receiving applied behavior analysis (ABA) with those receiving an eclectic intervention, and report general predictors of response. They found that lower ASD severity, lower age, and higher maternal age at entry predicted cognitive gains in their sample. Similarly, Rogers et al. (2012) found no treatment differences between parent Early Start Denver Model and community treatment as usual, but report that starting intervention at a younger age and more hours of intervention were related to better behavioral outcomes. These predictors may indicate characteristics that would predict better outcomes to any intervention.

Work in the field of applied behavioral analysis (ABA) also provides information on potential predictors of response to treatment. Starting ABA services at earlier ages is associated with better outcomes in IQ (Harris & Handelman, 2000; Smith et al., 2015). Increases in IQ are also associated with higher levels of IQ at the start of treatment (Harris & Handelman, 2000; Howlin et al., 2009; Smith et al., 2015). Interestingly, Smith et al. (2015) also found that social engagement (a combination of social approach, joint attention, and imitation) predicted more favorable outcomes in IQ and ASD severity.

Other intervention studies took a more nuanced approach and identified potential predictors of treatment response among other behavioral variables. In a randomized controlled trial of the parent mediated intervention “More than Words,” lower object interest at the start of intervention was related to more favorable outcomes in joint attention, requesting, intentional communication, and non-verbal communication despite no differences between the experimental intervention and the control group (Carter et al., 2011). In a study by Yoder and Stone (2006), while there were no overall group differences between treatments, object exploration, prior to treatment, differentiated response between the two different interventions. Children with relatively high object exploration exhibited better outcomes with a Picture Exchange Communication System, while children with relatively low object exploration exhibited better outcomes with Response Education and Prelinguistic Milieu Training. In both studies, object interest/exploration (the number of unique toys touched by the child) was an important predictor of treatment response.

Behavioral Profiles of Treatment Response

Developing profiles of children who respond well to an intervention can be informative for matching child to treatment approach. For example, Sherer and Schreibman (2005) examined data of participants that had already received Pivotal Response Treatment (PRT). Based on increases in raw scores on language assessments and observational measures of language and play, five responders and six non-responders to PRT were identified. Using several pre-treatment characteristics of the children (imitation, object interest, and repetitive behaviors), behavioral profiles were developed for responders and non-responders. Using these profiles, a new group of participants that fit the responder profile was matched (by language, age, and IQ) with participants that fit the non-responder profile. As predicted, children matching the responder profile (higher toy interest, tolerance of another person in close proximity, lower non-verbal self-stimulatory behavior, and higher verbal stimulatory behavior) showed positive outcomes while the non-responders (lower toy play, approach behavior, and verbal self-stimulatory behavior) did not (Sherer & Schreibman, 2005). In follow up work, another group of responders and non-responders were recruited into a single subject multiple baseline design study and received both PRT and Discrete Trial Training. The groups confirmed that the profiles specifically predicted response to PRT, but not to Discrete Trial Training (Schreibman et al., 2009). These results are encouraging, and can provide a model for utilizing pre-treatment characteristics in treatment planning using a specific intervention model.

Methodological Advances in Classifying Individuals with ASD

Machine learning paradigms may be useful tools in analyzing complex datasets to classify individuals with ASD. These techniques have been used to improve diagnosis of ASD by examining predictors of clinical assessment (Lord et al., 2012), to analyze specific item level scores of different diagnostic assessments (Wall, Dally, et al., 2012; Wall, Kosmicki, et al., 2012), and predict diagnosis among high-risk siblings (Bussu et al., 2018).

Classification and Regression Trees

While the term machine learning encompasses a range of statistical methods, one method particularly well suited for ASD intervention research is the use of classification and regression tree modeling (CART, Breiman et al., 1984). This is due to CART’s ability to take complex datasets, quickly and easily describe associations in data (Lemon et al., 2003), and identify subsets within a given population (Breiman et al., 1984). Researchers have leveraged the strengths of CART analyses to examine complex data like temporolimbic structures, which include 13 different regions in each hemisphere of the brain, and identify characteristics that distinguish individuals with ASD from typically developing controls (Neeley et al., 2006). Chawarska et al. (2014) examined the Autism Diagnostic Observation Schedule (ADOS) scores of 18-month-old high-risk siblings of children with ASD and found three specific behavior groupings related to an ASD diagnosis at three years of age.

Intervention studies have rarely made use of the CART method. One exception was a study of children with autism participating in a social skills intervention at school. The study utilized an assessment of mid -study changes in playground peer engagement to predict who would go on to exhibit higher or lower levels of playground peer engagement at the end of the intervention (Shih et al., 2016). CART analyses identified responders based on mid –study changes in engagement. Children exhibiting the most change (increases of at least 14% time peer engaged on the playground) mid –study were also the children to exhibit the most peer engagement at the end of intervention. This study illustrates two potential benefits of using CART analysis for intervention planning. First, it identified pertinent measures of change that predicted response to the intervention, and provided a metric that can guide future research and, potentially, clinical decisions for treatment.

While powerful, CART methods have important limitations that need to be considered. Traditional CART analyses show potential statistical biases to certain types of data when analyzing datasets with a mix of categorical and continuous variables. Continuous variables are often favored when determining decision points in the resulting trees (Hothorn et al., 2006). This is because CART models examine every possible split to determine the best possible split of the data. To address this, an extension of CART called conditional inference trees analysis (CIT) was developed. CIT analysis uses permutation based significance tests to identify significant variables before determining split points, and this process is recommended for datasets with a mix of continuous and categorical data (Hothorn et al., 2006).

Treatment Response in Joint Attention, Symbolic Play, Engagement, and Regulation

Joint Attention, Symbolic Play, Engagement, and Regulation (JASPER; Kasari et al., 2006) is one example of a specific intervention model with a well-established evidence base. JASPER is a naturalistic developmental behavioral intervention (Schreibman et al., 2015) that uses behavioral techniques within a developmental framework to target joint engagement, social communication, and language development of children with ASD. It is rooted in the idea that joint engagement provides the context for social communication development (Adamson et al., 2009), and early social communication skills of joint attention are critical for expressive language development (Tomasello & Farrar, 1986). Research in JASPER has shown immediate and persistent expressive language improvements in one to one settings (Kasari et al., 2008; Kasari et al., 2012; Gulsrud et al., 2014) with limited language children (Kasari et al., 2014) and in school based settings (Shih et al., 2021), establishing a connection between teaching joint attention and play with later expressive language development. Because multiple randomized trials of JASPER have been published with differences between JASPER and control groups, there are enough children receiving JASPER to explore potential predictors of intervention response. However, utilizing these data requires data analytics capable of handling their complexity.

Super and Slow Responders Among Preschoolers that Received JASPER

The current study attempts to predict treatment response for limited language preschoolers who received the JASPER intervention. This represents a critical time in development, as expressive language ability by age five is an important predictor for many outcomes for individuals with ASD (Lord & Bailey, 2002; Pickles et al., 2014). Furthermore it is unclear if children under five years of age with limited language are truly minimally verbal, or can still develop expressive language to avoid classification of minimally verbal at age 5 (DiStefano & Kasari, 2016). In a sample of preschool age children with ASD, Thurm et al. (2015) administered the ADOS at age three and again at age five. They found that 37% of children with ASD identified as limited language on ADOS at age three went on to develop phrase speech by age 5, further highlighting the importance of this time in development. Additionally, they found that expressive language age equivalents from the Mullen Scales of Early Learning (MSEL; Mullen, 1995) were strongly related to the verbal classification from the ADOS. For children classified as minimally verbal, there was little change in expressive language age equivalents from the age 3 and age 5 assessments. For children that achieved phrase speech by age 5, they found an average improvement of 15 months in expressive language age equivalents over 2 years, although these gains did not catch children up to their typical peers. The study does confirm the utility of using age equivalent scores as indicators of ability in children with ASD (Akshoomoff, 2006).

Based on these findings, we operationalized response to JASPER as change in expressive language age equivalents relative to the time between administrations of the MSEL. Accelerated changes in expressive language age equivalents may indicate the additive effect of JASPER over expected development. We define super response as an expressive language age equivalent change score that exceeds the duration (in months) between administrations of the MSEL.

To develop our prediction model a range of predictors were selected from pre-intervention standardized assessments of cognition and more nuanced behavioral data from validated experimental measures of play and social communication. Previous work identified nonverbal cognitive ability as an important predictor of language (Thurm et al., 2007). To explore this further, we included variables for visual reception and fine motor abilities from the MSEL. Early social communication, specifically joint attention, was also found to be predictive of language development in children with ASD (Charman, 2003), and experimental measures of social communication provided measures of joint attention and requesting gestures. Both joint attention and play have predicted language gains in previous JASPER studies (Kasari et al., 2006, Gulsrud et al., 2014; Shih et al., 2021), so measures of play development were also included as potential predictors of language in the current study. Due to the complexity of these data, a combination of traditional regression analyses and CIT modeling is used to identify significant predictors of treatment response and explore the clinical implications. Based on previous research in JASPER, we hypothesized that pre-intervention levels of joint attention gestures and play level would predict better language outcomes for limited language children with ASD that received JASPER.

Methods

Design

The current study is a secondary analysis of data gathered from participants across five separate intervention studies. Each of the studies implemented the same intervention (JASPER) and collected common measures of independently administered and blinded ASD assessment: standardized cognitive and language assessments, social communication skills, and play development.

Participants

Inclusion criteria was based on age and expressive language level at entry. Participants that entered their respective projects between the ages of 24 and 60 months and that were assessed using the ADOS 2, module 1, were included for analysis. Children receiving module 1 of the ADOS have no words to single words. Overall, the sample consists of 99 limited language, preschool age children with ASD that received between two to six months of the JASPER intervention. The parents of all participants provided written consent before participating in any of their respective study procedures.

The Intervention – JASPER

The JASPER intervention utilized for our sample is a targeted intervention based on naturalistic, developmental and behavioral principles. It is designed to address core challenges in social communication, play, and engagement. Five intervention studies demonstrated the efficacy of JASPER as a caregiver mediated intervention (Kasari et al. 2010; Kasari et al., 2015, therapist mediated in preschoolers with ASD (Goods et al., 2013; Kasari et al. under review), and as a teacher mediated intervention (Chang et al., 2016). Across the studies, children received two to six months of intervention. All studies report high fidelity of implementation.

Measures

Demographics: Prior to entering intervention, caregivers of the participants completed a demographic form, including information on child age, sex, and parent-reported race/ethnicity (via multiple choice). These non-malleable variables were used as predictors in the analysis.

ADOS (Lord et al., 2012): The ADOS is a standardized semi-structured assessment used to evaluate individuals suspected of having an ASD. The participant is engaged in a variety of activities designed to elicit social and communication behaviors associated with the diagnosis of ASD. Research reliable assessors administered the ADOS to all children across all studies to confirm diagnosis. As noted above, language skills indicated by the administration of ADOS module 1 were used to identify limited language children for inclusion in the current sample. ADOS comparison scores are also used as a proxy measure for ASD symptom severity.

Mullen Scales of Early Learning (MSEL; Mullen, 1995): The MSEL is an assessment of early intellectual development and school readiness. It is normed for the ages from birth to five years eight months. Child development is assessed across five domains; gross motor, visual reception, fine motor, expressive language, and receptive language. Age equivalents are calculated for each of the five domains. Due to the limited language sample of our study, a floor effect renders the standard scores for expressive language relatively insensitive to change over time. Therefore, age equivalents are used to assess expressive language progress from intervention, and to help account for the differing intervention intervals across studies. Additionally, age equivalents for fine motor and visual reception are used as indicators of cognitive pre-intervention ability and predictors in analyses. While previous research has also identified developmental quotient (DQ) as a potential predictor of treatment, we focus on fine motor and visual reception as they have the potential to provide more nuanced and clinically relevant information.

Early Social Communication Scales (ESCS; Mundy et al., 2003): A structured tabletop assessment, the ESCS is designed to elicit joint attention and requesting behavior. ESCS sessions last 15 – 25 minutes, are videotaped, and later coded for specific joint attention skills. The specific skills coded are coordinated looks, alternating gazes, points, gives, shows, and language. Skills can also be coded in combination with language and/or eye contact (eg. a point with language, or a give with eye contact and language). For the current study, total joint attention and requesting gestures are used as predictors.

Structured Play Assessment (SPA; Ungerer & Sigman, 1981): The SPA is a play-based assessment designed to measure a child’s level of play. SPA sessions are videotaped and later coded for play diversity and mastery. Total number of unique play types is scored for play diversity. Mastered play level is determined by the highest level of play in which a child exhibits two different types, with a frequency of at least five. There are 16 levels of play that fall under five broad categories; indiscriminate acts, discriminate acts, combination play, pre-symbolic play, and symbolic play. Play diversity (continuous) and mastered play level (ordinal) are included as predictors for analysis.

Analysis

The current study examines predictors of language gains among limited language preschool age children with ASD who received JASPER. To do this, we use an alternative form of CART analyses known as conditional inference trees (CIT). Preliminary analysis is conducted using conditional inference forests, an extension of CIT. These forests are used to generate variable importance values for large sets of predictors that will guide variable selection for the final model. Using predictors identified by the conditional inference forests, a combination of regression and CIT analyses are used to establish significant predictors of language gains. It is important to note that the current study does not have a comparison group of similar individuals that did not receive JASPER. Therefore, our findings should be interpreted with caution, and we address these limitations in our discussion. All statistical analyses were conducted using R version 4.0.3 (R Core Team, 2020).

Results

Participant Demographics

As noted in Table 1, the mean chronological age was 40 months, and the average expressive language age equivalent was 18 months; thus children showed significant language delays. The average non-verbal DQ for the sample was 66.67. The sample population was diverse with 65% identifying as belonging to racial minority groups (see Table 1), and majority male (86%).

Table 1.

Outcome Variable Overall Sample Super Responders Slow Responders
Expressive Languagea M (SD) 17.77 (8.92) 18.85 (7.64) 16.79 (9.91)
Predictor Variables
Agea M (SD) 39.79 (9.9) 37.74 (8.39) 41.63 (10.85)
Fine Motor Age Equivalenta M (SD) 25.83 (7.56) 27.85 (8.12) 24 (6.57)
Visual Reception Age Equivalenta M (SD) 25.6 (8.46) 27.34 (8.29) 24.02 (8.39)
Severityb 6.82 (1.93) 6.38 (1.89) 7.2 (1.90)
Joint Attention gesturesb M (SD) 4.79 (6.3) 6.11 (6.64) 3.6 (5.78)
Requesting gesturesb M (SD) 12.65 (9.61) 15.57 (10.92) 10 (7.4)
Play Diversityb M(SD) 20.81 (11.34) 24.62 (11.18) 17.37 (10.44)
Malec 86 (86%) 41 (87%) 45 (87%)
Play Masteryc
 Simple 7 (7%) 1 (2%) 6 (11%)
 Combination 27 (27%) 8 (19%) 18 (34%)
 Pre-symbolic 54 (54%) 30 (64%) 24 (46%)
 Symbolic 11 (11%) 7 (15%) 4 (9%)
Racec
 African American 4 (4%) 1 (2%) 3 (6%)
 White 35 (35%) 18 (38%) 17 (33%)
 Latino 14 (14%) 6 (13%) 8 (15%)
 Asian 22 (22%) 9 (19%) 13 (25%)
 Other 24 (24%) 13 (28%) 11 (21%)
Open in a new tab
a

months

b

frequency

c

n (%)

Calculating Response Status

Response status was determined by calculating change scores in expressive language age equivalents (ELAE) from the MSEL. Independent assessors administer the MSEL prior to the start of intervention and again after completion. To determine responder status, change scores in ELAE were calculated and divided by the duration (in months) between administration of pre and post MSEL tests. This produces a ratio where scores of one would indicate expressive language gains as would be expected in typical development. Scores above one indicated expressive language gains that would exceed those expected in typical development. In our sample 47% (n = 47) of individuals had change score ratios exceeding one and 53% (n = 52) had change scores equal to or less than one.

Conditional Inference Forest for Variable Selection

Outcome and predictor variables used in the following analyses are in Table 1. As this sample consists of participants from five different studies, preliminary analysis was conducted to examine any significant differences between participants in the different studies. Since studies differed on several elements (length of study, number of sessions, delivery agent) a variable was created to represent each study. The new study variable was included as a predictor in all subsequent analyses to explore potential effects of study differences. At each level of analysis, study differences were not found to be significant predictors.

Variables of interest were chosen based on prior research in predictors of treatment response, in particular age and ASD symptom severity. Additionally, previous work in JASPER has established a relationship between joint attention and play skills with later expressive language development. Requesting gestures was also included as another measure of non-verbal communication. While IQ has also been found to predict treatment response across several studies, we decided to explore this further by focusing on the fine motor and visual reception domains of the MSEL, which are each included in calculation of non-verbal DQ.

To identify variable importance and selection for the prediction model, conditional inference forests were generated with responder status as the outcome. Due to the random nature of forest building, three separate forests were generated, each containing 10,000 trees. The result of each forest provides variable importance scores of the predictors based on conditional permutation of the predictor variables as described by Strobl et al. (2008). Positive scores indicated predictors that increase model efficiency, whereas negative scores decreased model efficiency, and scores of zero indicated a neutral influence. Figure 1 shows that seven predictors were identified as increasing model efficiency, with three (requesting gestures, play diversity, and fine motor skills) showing particularly strong importance for the model. Conditional inference forest variable selection does not have an established significance test for variable importance scores, so predictors were chosen based on three criteria, strength of influence relative to the other predictors, parsimony, and clinical relevance. As such, the final predictors chosen for further analyses were requesting gestures, play diversity, and fine motor skills.

Figure 1.

Figure 1.

Open in a new tab

Conditional Inference Forest Variable Importance Plot. Conditional inference forests were run to determine variables predictive of responder status. JA = Joint Attention. VRAE = Visual Reception Age Equivalent. FMAE = Fine Motor Age Equivalent

Building the Prediction Model

Using the variables identified in the conditional inference forest, a CIT model was built at a .95 confidence level. The resulting tree included a single significant split with two terminal nodes. The single significant predictor of responder status was play diversity, with a splitting point at a play diversity score of 23 (p = .004), indicating 23 unique play acts during the SPA. The overall performance of the CIT model to accurately predict responder status was 67%, with a specificity of 55% and sensitivity of 78%.

Separate regression analyses were conducted using the same variables from the CIT model to confirm the significance of play diversity as a predictor of responder status and overall language gains. Our first model was a stepwise backward logistic regression with responder status as the outcome. The model started with requesting gestures, play diversity, and fine motor skills as predictors in the full model. Fine motor skills were removed in the next step, with requesting gestures to follow. The final model was significant, χ2 (1) = 10.69, p = .001, with play diversity as the only significant predictor of responder status (β = .062, p = .002). Another regression model with overall change scores in ELAE as the outcome was also significant (R2 = .26, F(7, 91) = 4.65, p < .001), play diversity predicted overall change scores in ELAE (β = .15, p = .05). To explore this further, the sample was divided into high and low diverse players based on the cutoff score of 23 from the CIT analysis. This also produced a significant model (R2 = .28, F(7, 91) = 5.01, p < .001) where high/low play diversity status predicted ELAE (β = 3.65, p = .01). Altogether, our models show play diversity accounts for 26–28% of the variance in ELAE of our sample.

Exploratory Analysis of the CIT Model

A second tree model was built reducing the confidence level to .90 so that predictors with p values lower than .10 would be included in the model. The CIT analysis was repeated with the same predictors, resulting in a new tree. Results from the second CIT analysis are depicted in Figure 2. The primary split remained play diversity, with a score of 23 identifying the two groups. However, now fine motor age equivalency of 26 months or less further distinguished the low play diversity group. The inclusion of fine motor ability in the model helps to better characterize the low play diversity group, as many slow responders also had delays in fine motor ability. Overall accuracy of the final model saw a slight increase to 70%, with a sensitivity of .72 and a specificity of .67.

Figure 2.

Figure 2.

Open in a new tab

Conditional Inference Tree of Play Diversity and Fine Motor Predicting Super vs. Slow Responders, confidence level of .90. The final model identifies two predictive variables; play diversity and fine motor ability.

In both CIT models, study was not a significant predictor of responder status. However, we did compare the final CIT model with and without including the study effect. The cutoffs identified for play diversity and fine motor did not change between the two CIT models.

Discussion

In order to better fit an intervention to a particular child, a first step is to determine how individual children respond to the same intervention. The current study attempts to accomplish this by retrospectively examining the responses of individuals who received an evidence-based early intervention, JASPER and to identify characteristics of children that may have had a super response or slower response on expressive language gains. It is understood that not all children respond the same to an intervention, but it is important to determine who does respond quickly and who might need more time or an augmentation to their intervention to achieve better outcomes.

In the current study, we used change scores of expressive language age equivalents from the MSEL as an indicator of response to treatment. Specifically, we were interested in individuals that not only improved during intervention, but whose change scores exceeded the duration (in months) between administrations of the MSEL. This change potentially represents accelerated growth in expressive language. In previous work (Thurm et al., 2015) a group of children with ASD were assessed at three years of age and again at five years age to track expressive language development on the MSEL. In terms of ELAE, children that were determined minimally verbal at three based on the ADOS module 1 and remained minimally verbal at five gained an average of two months in ELAE. This depressed growth is not unexpected for children that remain minimally verbal at five years of age. However, some of the children identified as minimally verbal at three did go on to develop phrase speech by age five, with ELAE improving by an average of 15 months over a time period of nearly 2 years. This growth of 15 months still lagged the duration between assessments (on average 22 months), which is in contrast to the super responders in our sample.

Predicting Response Status in JASPER

After identifying children in this sample as super responders and slow responders, CIT modeling was used to develop a prediction model to help better understand the characteristics of these responders. Initial variable selection through conditional random forests identified three predictors that stood out as influential, and clinically relevant to predicting response to JASPER. These three variables were play diversity, requesting gestures, and fine motor skills. In the context of JASPER, a play based naturalistic intervention, these three predictors have particular clinical relevance. Play diversity reflects the child’s flexibility and generalizability of being able to play with a variety of toys at the same developmental play level; this ability is critical to demonstrating mastery of a play level. Fine motor skills are important to a child’s physical ability to manipulate objects in play, and both play diversity and fine motor skills are likely beneficial in a play based intervention such as JASPER. While we had hypothesized that joint attention gestures would be important to expressive language change scores as noted in other studies (Gulsrud et al., 2014; Shih et al., 2021), requesting gestures were found to be a better predictor in this sample. Given the limited joint attention of the children to begin with, and the relative easier learning of requesting gestures, this may not be particularly surprising in this sample. Increased gesture use for communicative intent should theoretically have downstream effects on spoken language.

Our final model produced a tree with a primary split at play diversity, with a secondary split dividing the low play diversity group by fine motor age. This model indicates that children with more play diversity have increased odds of making expressive language gains in JASPER. Conversely, children with less play diversity and delayed fine motor skills have lower odds of making expressive language gains in JASPER. This final model had predictive accuracy of 70%. While play diversity seems to be an important predictor of super response to JASPER, the additional information of fine motor ability paints a slightly clearer picture of slow responders and adds support for the importance of fine motor ability in language development (Bal et al. 2020). Children that begin intervention at this age with low play diversity and low fine motor skills may not experience optimal expressive language gains through JASPER in a short period of time. These children may need more time, more intensity, the combining or sequencing of JASPER with other interventions, or simply another intervention. However, due to the exploratory nature of our analysis our findings will need to be replicated. Without a comparison group of children that did not received JASPER, it is unclear if our findings are intervention specific, or indicative of a more general response to play based intervention. Continued research in ways to inform and support treatment decisions is critical if we are to make progress individualizing treatments for individuals with ASD.

Play Diversity and Language Gains

Play diversity reflects flexibility in a child’s play with toys and can only be measured if children have interest in toys and playing. Play diversity (types) is also important in determining play level since children need to demonstrate multiple different play actions with toys at the same play level in order to show mastery of that play level (Kasari et al., 2006; Ungerer & Sigman, 1981). Play types have generally been considered more important than object or toy frequency for children with autism since many same actions on a toy can be viewed as repetitive, and non-functional (Ungerer & Sigman, 1981). Children who were categorized as super responders were demonstrating play diversity on assessments whereas slower responders had far less play diversity. Having more interest in and varied actions with toys lends itself to faster progress in an intervention such as JASPER.

Machine Learning in ASD Intervention Research

The current study explores the utility of machine learning paradigms in ASD intervention research. Methods like CART analyses are becoming increasingly popular for their flexibility, relative ease of use, and ease of interpretation. However, they can be deceptively easy. Inherent biases in the variable selection and model fitting process plague CART analysis (Strobl et al., 2007), and researchers must look closely at their data to determine the appropriateness of any statistical analysis. In the current study, traditional CART analysis would be ill advised due to the complexity of the data. Using traditional CART analysis with the current data set would raise many questions about the validity of the results, and the CIT model used in the study addresses these issues of bias (Hothorn et al., 2006; Strobl et al., 2008). However, combining the results of the CIT analysis with traditional regression analyses provides stronger support for the role of play diversity in predicting response to JASPER. Furthermore, the cutoff values for play diversity, and fine motor ability in our exploratory model, provide vital information that can inform future research in JASPER.

Limitations

While there are several strengths of the current sample, including a fairly large sample for examining intervention effects in autism research, and participants where the majority were from traditionally marginalized and minoritized populations, there are also limitations that should be noted. One is the lack of a test sample, as traditionally done using CIT analyses. Rather this study took advantage of the entire sample to maximize the power to identify significant predictors. To compensate for this, further analyses exploring the significance of play diversity as a predictor of language outcome were conducted. Play diversity was consistently significant, whether looking at responder status, or change scores in ELAE.

Other aspects of the study may also be noted as limitations, specifically the lack of a comparison sample. Thus, we cannot be sure that play diversity predicts language gains uniquely for children receiving JASPER, or if similar effects might be found for other intervention models. Further research is needed to better understand the role play diversity has on expressive language development. These novel measures of play do highlight the importance of exploring variables outside of those provided by standardized tests, such as observational measures of play (Yoder & Stone, 2006; Carter et al., 2011).

Finally, while we were particularly interested in super responders in this study, we note that even slower responders made gains. Of the slow responders, 85% (n = 44) made some gains in ELAE (M = 4.75, SD = 2.16). For those children who stayed the same on standardized measures of expressive language, it will be important to examine how they may be changing on other potentially important outcomes, and to consider strategies for enhancing their growth. Future work also may explore alternative ways of quantifying growth, such as gains relative to an individual’s delay (Georgiades & Kasari, 2018).

The current study focuses on cognitive and behavioral predictors of response to intervention. While we do include some basic demographic information, future studies may want to explore the influence of other ecological variables, such as socio –economic status and maternal education level. This is essential in creating an appropriate match of individuals with efficacious treatments (Stahmer et al., 2011).

Conclusions

This study highlights the importance and challenges of understanding for whom an intervention works best for a subgroup of young children who are identified with the least amount of spoken language at the outset of the study. For this sample of limited language children receiving JASPER, about half of the children were super-responders, even across several years of research studies. Predictors of response identified a number of variables from multiple independent assessments conducted by blinded assessors. Notably these predictors are malleable in that they can improve with targeted interventions. Understanding how to better improve trajectories of children who are not responding as quickly to an intervention should be a focus of future research studies.

Future studies should also continue to take advantage of novel approaches, and design studies to identify responder status much sooner, and systematically. Studies that employ prospective approaches like sequential multiple assignment randomized trials (SMART) can explore the idea of augmenting treatments in real time. Combined with retrospective work, like the current project, novel prospective designs can be very effective ways to inform truly individualized treatment.

Funding Sources:

Autism Speaks; Grant number: 7495; National Institute of Child Health and Human Development; Grant number: P50-HD-055784; National Institute of Mental Health; Grant number: MH064927; Grant number: R01MH084864; Organization for Autism Research; Grant number: 20072725

Footnotes

We have no conflicts of interest to disclose.

Contributor Information

Jonathan Panganiban, Semel Institute for Neuroscience and Human Behavior, University of California Los Angeles, 760 Westwood Plaza 6th Floor, Los Angeles CA 90024.

Connie Kasari, Semel Institute for Neuroscience and Human Behavior, University of California Los Angeles, 760 Westwood Plaza 6th Floor, Los Angeles CA 90024.

References

  1. Adamson LB, Bakeman R, Deckner DF, & Romski M (2009). Joint engagement and the emergence of language in children with autism and Down syndrome. Journal of autism and developmental disorders, 39(1), 84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Akshoomoff N (2006). Use of the Mullen Scales of Early Learning for the assessment of young children with autism spectrum disorders. Child Neuropsychology, 12(4–5), 269–277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bal VH, Fok M, Lord C, Smith IM, Mirenda P, Szatmari P, Vaillancourt T, Volden J, Waddell C, Zwaigenbaum L, Bennett T, Duku E, Elsabbagh M, Georgiades S, Ungar W, & Zaidman-Zait A (2020). Predictors of longer-term development of expressive language in two independent longitudinal cohorts of language-delayed preschoolers with autism spectrum disorder. Journal of Child Psychology and Psychiatry, 61(7), 826–835. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Breiman L, Friedman J, Stone CJ, & Olshen RA (1984). Classification and regression trees. CRC press. [Google Scholar]
  5. Bussu G, Jones EJ, Charman T, Johnson MH, & Buitelaar JK (2018). Prediction of autism at 3 years from behavioural and developmental measures in high-risk infants: a longitudinal cross-domain classifier analysis. Journal of Autism and Developmental Disorders, 48(7), 2418–2433. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Carter AS, Messinger DS, Stone WL, Celimli S, Nahmias AS, & Yoder P (2011). A randomized controlled trial of Hanen’s ‘More Than Words’ in toddlers with early autism symptoms. Journal of Child Psychology and Psychiatry, 52(7), 741–752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chang YC, Shire SY, Shih W, Gelfand C, & Kasari C (2016). Preschool Deployment of Evidence-Based Social Communication Intervention: JASPER in the Classroom. Journal of Autism and Developmental Disorders, 46(6), 2211–2223. [DOI] [PubMed] [Google Scholar]
  8. Charman T (2003). Why is joint attention a pivotal skill in autism?. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 358(1430), 315–324. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chawarska K, Shic F, Macari S, Campbell DJ, Brian J, Landa R, Hutman T, Nelson CA, Ozonoff S, Tager-Flusberg H, Young GS, Zwaigenbaum L, Cohen IL, Charman T, Messinger DS, Klin A, Johnson S, & Bryson S (2014). 18-month predictors of later outcomes in younger siblings of children with autism spectrum disorder: a baby siblings research consortium study. Journal of the American Academy of Child & Adolescent Psychiatry, 53(12), 1317–1327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. DiStefano C, & Kasari C (2016). The window to language is still open: Distinguishing between preverbal and minimally verbal children with ASD. Perspectives of the ASHA Special Interest Groups, 1(1), 4–11. [Google Scholar]
  11. Georgiades S, & Kasari C (2018). Reframing optimal outcomes in autism. JAMA pediatrics, 172(8), 716–717. [DOI] [PubMed] [Google Scholar]
  12. Goods KS, Ishijima E, Chang YC, & Kasari C (2013). Preschool based JASPER intervention in minimally verbal children with autism: Pilot RCT. Journal of Autism and Developmental Disorders, 43(5), 1050–1056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gulsrud AC, Hellemann GS, Freeman SF, & Kasari C (2014). Two to ten years: Developmental trajectories of joint attention in children with ASD who received targeted social communication interventions. Autism Research, 7(2), 207–215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Harris SL, & Handleman JS (2000). Age and IQ at intake as predictors of placement for young children with autism: A four-to six-year follow-up. Journal of Autism and Developmental Disorders, 30(2), 137–142. [DOI] [PubMed] [Google Scholar]
  15. Hothorn T, Hornik K, & Zeileis A (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics, 15(3), 651–674. [Google Scholar]
  16. Howlin P, Magiati I, & Charman T (2009). Systematic review of early intensive behavioral interventions for children with autism. American Journal on Intellectual and Developmental Disabilities, 114(1), 23–41. [DOI] [PubMed] [Google Scholar]
  17. Itzchak EB, & Zachor DA (2011). Who benefits from early intervention in autism spectrum disorders? Research in Autism Spectrum Disorders, 5(1), 345–350. [Google Scholar]
  18. Kasari C, Freeman S, & Paparella T (2006). Joint attention and symbolic play in young children with autism: A randomized controlled intervention study. Journal of Child Psychology and Psychiatry, 47(6), 611–620. [DOI] [PubMed] [Google Scholar]
  19. Kasari C, Paparella T, Freeman S, & Jahromi LB (2008). Language outcome in autism: randomized comparison of joint attention and play interventions. Journal of consulting and clinical psychology, 76(1), 125. [DOI] [PubMed] [Google Scholar]
  20. Kasari C, Gulsrud AC, Wong C, Kwon S, & Locke J (2010). Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism. Journal of Autism and Developmental Disorders, 40(9), 1045–1056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kasari C, Gulsrud A, Freeman S, Paparella T, & Hellemann G (2012). Longitudinal follow-up of children with autism receiving targeted interventions on joint attention and play. Journal of the American Academy of Child & Adolescent Psychiatry, 51(5), 487–495 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kasari C, Kaiser A, Goods K, Nietfeld J, Mathy P, Landa R, Murphy S, & Almirall D (2014). Communication interventions for minimally verbal children with autism: A sequential multiple assignment randomized trial. Journal of the American Academy of Child & Adolescent Psychiatry, 53(6), 635–646. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kasari C, Gulsrud A, Paparella T, Hellemann G, & Berry K (2015). Randomized comparative efficacy study of parent-mediated interventions for toddlers with autism. Journal of consulting and clinical psychology, 83(3), 554. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Kasari C, Shire S, Shih W, Landa R, Levato L, & Smith T. (under review). Language outcomes in limited language preschoolers with autism and intellectual disability: RCT of early intervention approaches. Journal of Child Psychology and Psychiatry. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lemon SC, Roy J, Clark MA, Friedmann PD, & Rakowski W (2003). Classification and regression tree analysis in public health: methodological review and comparison with logistic regression. Annals of Behavioral Medicine, 26(3), 172–181. [DOI] [PubMed] [Google Scholar]
  26. Lord C, & Bailey A (2002). Autism spectrum disorders In Rutter M & Taylor E Child and Adolescent Psychiatry, 4th Edn Blackwell Science Ltd, Oxford, 636–663. [Google Scholar]
  27. Lord C, DiLavore PC, & Gotham K (2012). Autism diagnostic observation schedule. Western Psychological Services. [Google Scholar]
  28. Lord C, Petkova E, Hus V, Gan W, Lu F, Martin DM,Ousley O, Guy L, Bernier R, Gerdts J, Algermissen M, Whitaker A, Sutcliffe JS, Warren Z, Klin A, Saulneir C, Hanson E, Hundley R, Piggot J, ... & Risi S (2012). A multisite study of the clinical diagnosis of different autism spectrum disorders. Archives of general psychiatry, 69(3), 306–313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mullen EM (1995). Mullen scales of early learning (pp. 58–64). Circle Pines, MN: AGS. Chicago [Google Scholar]
  30. Mundy P, Delgado C, Block J, Venezia M, Hogan A, & Seibert J (2003). Early social communication scales (ESCS). Coral Gables, FL: University of Miami. [Google Scholar]
  31. National Research Council. (2001). Educating children with autism. National Academies Press. [Google Scholar]
  32. Neeley ES, Bigler ED, Krasny L, Ozonoff S, McMahon W, & Lainhart JE (2007). Quantitative temporal lobe differences: autism distinguished from controls using classification and regression tree analysis. Brain and Development, 29(7), 389–399. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Pickles A, Anderson DK, & Lord C (2014). Heterogeneity and plasticity in the development of language: A 17-year follow-up of children referred early for possible autism. Journal of Child Psychology and Psychiatry, 55(12), 1354–1362. [DOI] [PubMed] [Google Scholar]
  34. R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/. [Google Scholar]
  35. Rogers SJ, Estes A, Lord C, Vismara L, Winter J, Fitzpatrick A, Guo M, & Dawson G (2012). Effects of a brief Early Start Denver Model (ESDM)–based parent intervention on toddlers at risk for autism spectrum disorders: A randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 51(10), 1052–1065. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Schreibman L, Stahmer AC, Barlett VC, & Dufek S (2009). Brief report: Toward refinement of a predictive behavioral profile for treatment outcome in children with autism. Research in Autism Spectrum Disorders, 3(1), 163–172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Schreibman L, Dawson G, Stahmer AC, Landa R, Rogers SJ, McGee GG, Kasari C, Ingersoll B, Kaiser AP, Bruinsma Y, McNerney E, Wetherby A, & Halladay A (2015). Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. Journal of autism and developmental disorders, 45(8), 2411–2428. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Sherer MR, & Schreibman L (2005). Individual behavioral profiles and predictors of treatment effectiveness for children with autism. Journal of Consulting and Clinical Psychology, 73(3), 525. [DOI] [PubMed] [Google Scholar]
  39. Shih W, Patterson SY, & Kasari C (2016). Developing an adaptive treatment strategy for peer-related social skills for children with autism spectrum disorders. Journal of Clinical Child & Adolescent Psychology, 45(4), 469–479. [DOI] [PubMed] [Google Scholar]
  40. Shih W, Shire S, Chang YC, & Kasari C (2021). Joint engagement is a potential mechanism leading to increased initiations of joint attention and downstream effects on language: JASPER early intervention for children with ASD. Journal of Child Psychology and Psychiatry. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Siller M, Hutman T, & Sigman M (2013). A parent-mediated intervention to increase responsive parental behaviors and child communication in children with ASD: A randomized clinical trial. Journal of autism and developmental disorders, 43(3), 540–555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Smith T, Klorman R, & Mruzek DW (2015). Predicting outcome of community-based early intensive behavioral intervention for children with autism. Journal of abnormal child psychology, 43(7), 1271–1282. [DOI] [PubMed] [Google Scholar]
  43. Stahmer AC, Schreibman L, & Cunningham AB (2011). Toward a technology of treatment individualization for young children with autism spectrum disorders. Brain research, 1380, 229–239. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Stahl D, Pickles A, Elsabbagh M, Johnson MH, & BASIS Team. (2012). Novel machine learning methods for ERP analysis: a validation from research on infants at risk for autism. Developmental Neuropsychology, 37(3), 274–298. [DOI] [PubMed] [Google Scholar]
  45. Strobl C, Boulesteix AL, Zeileis A, & Hothorn T (2007). Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8(1), 25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Strobl C, Boulesteix AL, Kneib T, Augustin T, & Zeileis A (2008). Conditional variable importance for random forests. BMC Bioinformatics, 9(1), 307. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Thurm A, Lord C, Lee LC, & Newschaffer C (2007). Predictors of language acquisition in preschool children with autism spectrum disorders. Journal of autism and developmental disorders, 37(9), 1721–1734. [DOI] [PubMed] [Google Scholar]
  48. Thurm A, Manwaring SS, Swineford L, & Farmer C (2015). Longitudinal study of symptom severity and language in minimally verbal children with autism. Journal of Child Psychology and Psychiatry, 56(1), 97–104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Tomasello M, & Farrar MJ (1986). Joint attention and early language. Child development, 1454–1463. [PubMed] [Google Scholar]
  50. Ungerer JA, & Sigman M (1981). Symbolic play and language comprehension in autistic children. Journal of the American Academy of Child Psychiatry, 20(2), 318–337. [DOI] [PubMed] [Google Scholar]
  51. Wall DP, Dally R, Luyster R, Jung JY, & DeLuca TF (2012). Use of artificial intelligence to shorten the behavioral diagnosis of autism. PloS One, 7(8), e43855. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Wall DP, Kosmicki J, Deluca TF, Harstad E, & Fusaro VA (2012). Use of machine learning to shorten observation-based screening and diagnosis of autism. Translational Psychiatry, 2(4), e100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Yoder P, Stone W. L. (2006). A randomized comparison of the effect of two prelinguistic communication interventions on the acquisition of spoken communication in preschoolers with ASD. Journal of Speech, Language, and Hearing Research. 49(4):698–711. [DOI] [PubMed] [Google Scholar]

RESOURCES