Abstract
The severity of autism spectrum disorder (ASD) is often measured by co-occurring conditions, such as intellectual disability or language delay, rather than deficits in social interaction, and restricted interests and repetitive behaviors. The Autism Diagnostic Observation Schedule calibrated severity score (ADOS CSS) was created to facilitate comparison of the diagnostic features of ASD independent of related conditions over time. We examined the relationship between the ADOS CSS, ADOS total score, and clinician rated degree of impairment (DOI) in the Study to Explore Early Development. Like others, we confirmed that, among the measures we evaluated, the ADOS CSS was least influenced by developmental functioning and demographic factors and is therefore the best measure of core features of ASD in pre-school children.
Keywords: Autism Diagnostic Observation Schedule, Autism spectrum disorder, Calibrated severity score, Symptom severity
Introduction
Autism spectrum disorder (ASD) is a developmental disorder characterized by deficits in social communication and interaction, and the presence of restricted interests and repetitive behaviors that are recognized in early childhood (American Psychiatric Association 2013). ASD is considered a spectrum disorder because of the varied nature of symptom presentation and range of behavioral, developmental, and medical conditions that co-occur with ASD (Levy et al. 2010). ASD severity is often defined and influenced by the presence of co-occurring conditions, such as behavior problems, intellectual disability, language delay, motor delay, and sleep disturbance (Gotham et al. 2012; Jang et al. 2011; Jang and Matson 2015; MacDonald et al. 2014; Schreck et al. 2004; Shumway et al. 2012). These co-occurring conditions are notable components of adaptive functioning and treatment selection but are not diagnostic features of ASD. It is important to measure the severity of diagnostic features of ASD independent of related conditions, so the influence on outcomes can be gauged separately. Moreover, a measure of the severity of diagnostic features of ASD could help describe phenotypes and assess response to treatment over time.
Gotham et al. (2009) addressed the need for a standardized measure of ASD symptom severity by calibrating total scores from the Autism Diagnostic Observation Schedule (ADOS), the gold-standard instrument to quantify symptoms of and diagnose ASD (Lord et al. 1999). ADOS total scores—a simple sum of raw scores on diagnostic items— have previously been used as a measure of ASD severity. However, ADOS total scores are influenced by chronological age and language aptitude, which prevents comparison of ASD severity in different groups of children over time. Gotham et al., thus, created a calibrated severity score (CSS) to facilitate comparison of the diagnostic features of ASD independent of child age and related conditions. Children with ASD (n = 1118) were placed into age and language cells and ADOS CSS were generated within each cell based on percentiles of raw total scores (Gotham et al. 2009). Less variance in the ADOS CSS was explained by factors such as expressive language ability and maternal education than the ADOS total score. Therefore, the ADOS CSS was less influenced by developmental functioning and demographic factors than the ADOS total score. These findings were later supported by Shumway and colleagues (2012) who also found that the ADOS CSS was useful in controlling for differences in verbal development. Both sets of authors concluded that the ADOS CSS was a useful measure of ASD symptom severity in clinical, genetic, and neurobiological research.
Although, the ADOS CSS seems to measure ASD diagnostic symptoms independently from intellectual ability and language aptitude, other variables associated with ASD symptom severity were not included in previous analyses. First, past studies did not evaluate the effect of behavior problems (i.e., internalizing and externalizing behaviors) and sleep disturbance on severity measures. Second, clinical judgment is a critical component of the diagnosis of ASD; however, it is not clear how well it captures core ASD symptoms independently of related symptoms. Including clinical judgment as a measure of ASD severity would help determine the influence of developmental and demographic variables on clinician global impressions and how clinical judgment compares with the ADOS CSS and ADOS total score. Another limitation of prior analyses is the broad age range of participants (i.e., 2–16 years in the Gotham et al. (2009) sample and 2–12 years in the Shumway et al. (2012) sample). Limiting the sample to pre-school aged children would help define whether the ADOS CSS is less influenced by child and demographic characteristics than the ADOS total score in early childhood.
The objectives for the current study were two-fold: (1) to examine the correlation between the ADOS CSS, ADOS total score, and clinical impression of degree of impairment (DOI) among children classified as ASD in a case-control study to demonstrate a linear and positive relationship between these three measures of ASD severity and (2) determine which of these three measures was least influenced by developmental functioning and demographic factors. The Study to Explore Early Development (SEED) is a multi-site case-control study of risk factors of ASD in children 2–5 years of age. As part of SEED, children with ASD were given the ADOS among a host of other developmental and parent-report measures; clinicians also noted their clinical judgment on the degree of impairment associated with ASD symptoms for each child evaluated in SEED. Based on previous research, we predicted high correlations between the ADOS CSS, ADOS total score, and SEED DOI, and that the ADOS CSS would be least influenced by developmental and demographic characteristics.
Method
Participant Ascertainment
SEED is a case-control study conducted in California, Colorado, Georgia, Maryland, North Carolina, and Pennsylvania, and approved by Institutional Review Boards at each site and at the Centers for Disease Control and Prevention. Eligible children were born between September 1, 2003 and August 31, 2006, enrolled into the study between 2 and 5 years of age, resided in one of the study areas, and lived with a knowledgeable caregiver who was competent to communicate in English (or in California and Colorado, in English or Spanish). Three groups of children were recruited from each site: (1) those with known ASD and (2) those with known developmental delays identified from multiple educational and health providers or family or physician referral, and (3) those from the general population identified from state vital records. A detailed description of the SEED eligibility criteria, ascertainment methods, response rates, enrollment methods, and data collection procedures can be found in Schendel and colleagues (2012).
Data Collection Procedures
The Social Communication Questionnaire (SCQ) (Rutter et al. 2003) was administered to all families to provide an initial assessment of ASD risk and guide further assessment procedures. An SCQ score of 11 points or higher was chosen as an indicator of ASD risk, based on research that indicating it maximizes sensitivity and specificity in young children (Lee et al. 2007; Wiggins et al. 2007). Families of children who obtained a score of 11 or higher on the SCQ, had a previous ASD diagnosis, or demonstrated ASD behaviors during the clinic visit were asked to complete the Autism Diagnostic Interview—Revised (ADI-R) (Lord et al. 1994), ADOS (Lord et al. 1999), Mullen Scales of Early Learning (MSEL) (Mullen 1995), and Vineland Adaptive Behavior Scales—Second Edition (VABS-II) (Sparrow et al. 2005), along with the Child Behavior Checklist (CBCL) (Achenbach 1992). Clinicians who administered the ADI-R and ADOS established research reliability before study start and quarterly thereafter (i.e., 90% agreement with consensus scores on the ADI-R and 80% agreement with consensus scores on the ADOS).
Clinicians who evaluated the child noted the degree of impairment associated with ASD (SEED DOI) on a 7-point Likert scale, with 1 point indicating little impairment and 7 points indicating high impairment (Schendel et al. 2012; Wiggins et al. 2015). Children were not assigned SEED DOI values of 0 because the child had to demonstrate some ASD risk to receive a comprehensive SEED evaluation. Clinicians also noted whether ASD symptoms were better accounted for by another disorder; children with symptoms accounted for by another disorder were excluded from this analysis because our sample was limited to children with ASD.
ASD Case Status
The SEED ASD case status algorithm was based on best practice guidelines (Johnson et al. 2007), review of the literature on ASD classification, clinical experience among members of a clinical workgroup, and a desire to create a uniform method of characterizing ASD symptoms in large cohorts of children. ASD case status was based on the results of gold-standard ASD diagnostic instruments rather than previous diagnosis. Children classified as ASD were those who met ASD criteria on both the ADI-R (a comprehensive parent interview) and the ADOS (a direct assessment of the child) or who met ASD criteria on the ADOS and one of three alternate criteria on the ADI-R (i.e., met criteria on the social domain and was within two points on the communication domain, met criteria on the communication domain and was within two points on the social domain, or met criteria on the social domain and had two points noted on the behavioral domain). Details on the SEED final classification algorithm can be found in Wiggins et al. (2015).
Statistical Methods
The relationship between the ADOS CSS, ADOS total score, and SEED DOI was assessed with a bivariate correlation matrix. Pearson correlation coefficients of 0.50 or greater were considered to represent a large effect size (Cohen 1988). Similar to Gotham et al. (2009), the influence of developmental and demographic factors on severity outcomes was evaluated with hierarchical linear regressions. The following developmental variables were entered into the first block: CBCL externalizing behavior t-scores (derived from attention problems and aggressive behavior subscales), CBCL internalizing behavior t-scores (derived from emotionally reactive, anxious/depressed, somatic complaints, and withdrawn subscales), CBCL sleep problems t-scores, MSEL expressive language t-scores, MSEL fine motor t-scores, MSEL receptive language t-scores, MSEL visual reception t-scores, and VABS-II Adaptive Behavior Composite (VABS-II ABC). The following demographic and other variables were entered into the second block: child ethnicity, child race, child sex, maternal education, and SEED site. These demographic and other variables are those that could affect ASD symptoms but have less influence when child factors are controlled (Gotham et al. 2009). We did not include child age in the regression models since the age range of our participants was restricted to children 2–5 years. All statistical analyses were conducted using SPSS version 20.0.
Results
A total of 707 children were classified as ASD after a comprehensive SEED evaluation. Of these, 92.5% had a SEED DOI and were included in these analyses; 3.25% children had a note that ASD symptoms were better accounted for by another disorder, and 4.25% children had missing DOI data. The mean age of children in the study was 59.4 months (range 34.5–70.6 months; SD 6.69 months), and 82.0% of the sample was male. The race of child participants was 57.4% White, 19.0% Black, 12.2% Multiracial, and 11.4% other race. A total of 15.8% of the sample identified as Hispanic ethnicity. Among mothers, 1.80% had missing education data, 20.1% completed high school or less, 28.3% completed some college, 31.0% completed a Bachelor’s degree, and 18.8% completed a Master’s degree or higher.
The relationship between the ADOS CSS, ADOS total score, and SEED DOI is outlined in Table 1. All three measures of severity were positively correlated in bivariate comparisons, and all effects were statistically significant.
Table 1.
Bivariate correlations between the Autism Diagnostic Observation Schedule (ADOS) calibrated severity score (CSS), ADOS total score, and degree of impairment associated with ASD in the Study to Explore Early Development (SEED DOI)
Correlation is significant at the 0.01 level
Results of hierarchical linear regression analyses are presented in Tables 2, 3 and 4. The total amount of variance in ASD symptom severity accounted for by developmental and demographic characteristics was 17% for the ADOS CSS, R2 = 0.17, F(19, 608) = 6.66, p < .001; 41% for the ADOS total score, R2 = 0.41, F(19, 608) = 22.61, p < . 001; and 53% for the SEED DOI, R2 = 0.53, F(19, 608) = 33.98, p < .001. In all three models, the Δ R2 was statistically significant when adding demographic factors (Model 2) to developmental level only factors (Model 1) (for all models p < .001). In all three models, developmental characteristics influenced ratings of ASD symptom severity more than did demographic characteristics (Tables 2, 3, 4). Higher ratings of all three ASD symptom severity measures were associated with more internalizing behavior problems and fewer adaptive behavior skills, expressive language abilities, and fine motor abilities (Tables 2, 3, 4). SEED site and maternal education were also statistically significant terms in the ADOS CSS, ADOS total score, and SEED DOI models.
Table 2.
Hierarchical linear regression model for the Autism Diagnostic Observation Schedule calibrated severity score
| R2 | Δ R2 | B | SE B | β | t-value | |
|---|---|---|---|---|---|---|
| Step 1 | 0.10 | – | ||||
| Intercept | 9.45 | 0.64 | 14.68 | |||
| CBCL externalizing behaviors | −0.03 | 0.01 | −0.19 | −3.17** | ||
| CBCL internalizing behavior | 0.03 | 0.01 | 0.16 | 2.90** | ||
| CBCL sleep problems | −0.00 | 0.01 | −0.02 | −0.50 | ||
| MSEL expressive language | −0.02 | 0.01 | −0.12 | −1.53 | ||
| MSEL fine motor | −0.01 | 0.01 | −0.10 | −1.71 | ||
| MSEL receptive language | 0.00 | 0.01 | 0.01 | 0.14 | ||
| MSEL visual reception | 0.01 | 0.01 | 0.05 | 0.73 | ||
| Vineland Adaptive Behavior | −0.02 | 0.01 | −0.19 | −3.28** | ||
| Step 2 | 0.17 | 0.08 | ||||
| Intercept | 7.82 | 0.69 | 11.26 | |||
| CBCL externalizing behaviors | −0.00 | 0.01 | −0.14 | −2.32* | ||
| CBCL internalizing behavior | 0.03 | 0.01 | 0.18 | 3.27** | ||
| CBCL sleep problems | −0.02 | 0.01 | −0.01 | −0.22 | ||
| MSEL expressive language | −0.02 | 0.01 | −0.17 | −2.27* | ||
| MSEL fine motor | −0.02 | 0.01 | −0.14 | −2.25* | ||
| MSEL receptive language | 0.00 | 0.01 | 0.01 | 0.07 | ||
| MSEL visual reception | 0.01 | 0.01 | 0.08 | 1.22 | ||
| Vineland Adaptive Behavior | −0.02 | 0.01 | −0.15 | −2.63** | ||
| Child ethnicity not Hispanic (REF) | ||||||
| Child ethnicity Hispanic | 0.17 | 0.17 | 0.04 | 0.98 | ||
| Child sex male (REF) | ||||||
| Child sex female | −0.21 | 0.15 | −0.05 | −1.34 | ||
| Child race White (REF) | ||||||
| Child race Multiracial | 0.14 | 0.14 | 0.06 | 0.98 | ||
| Child race Black | 0.23 | 0.13 | 0.12 | 1.77 | ||
| Child race other | −0.27 | 0.16 | −0.11 | −1.66 | ||
| Maternal education | 0.20 | 0.06 | 0.14 | 3.42** | ||
| SEED site NC (REF) | ||||||
| SEED site CA | 0.01 | 0.14 | 0.00 | 0.05 | ||
| SEED site CO | 0.12 | 0.13 | 0.05 | 0.94 | ||
| SEED site GA | −0.52 | 0.13 | −0.20 | −4.05*** | ||
| SEED site MD | −0.25 | 0.14 | −0.09 | −1.77 | ||
| SEED site PA | 0.32 | 0.11 | 0.11 | 2.24* |
t-value significant at the 0.05 level
t-value significant at the 0.01 level
t-value significant at the 0.001 level
Table 3.
Hierarchical linear regression model for the Autism Diagnostic Observation Schedule total score
| R2 | Δ R2 | B | SE B | β | t-value | |
|---|---|---|---|---|---|---|
| Step 1 | 0.35 | – | ||||
| Intercept | 31.50 | 1.81 | 17.41 | |||
| CBCL externalizing behaviors | −0.08 | 0.02 | −0.18 | −3.61*** | ||
| CBCL internalizing behavior | 0.08 | 0.03 | 0.14 | 2.95** | ||
| CBCL sleep problems | −0.02 | 0.02 | −0.04 | −1.06 | ||
| MSEL expressive language | −0.09 | 0.03 | −0.20 | −3.05** | ||
| MSEL fine motor | −0.03 | 0.02 | −0.07 | −1.40 | ||
| MSEL receptive language | 0.01 | 0.03 | −0.03 | −0.33 | ||
| MSEL visual reception | −0.01 | 0.02 | −0.01 | −0.23 | ||
| Vineland Adaptive Behavior | −0.13 | 0.02 | −0.36 | −7.40*** | ||
| Step 2 | 0.41 | 0.06 | ||||
| Intercept | 26.22 | 1.92 | 13.66 | |||
| CBCL externalizing behaviors | −0.06 | 0.02 | −0.14 | −2.78** | ||
| CBCL internalizing behavior | 0.09 | 0.03 | 0.17 | 3.54*** | ||
| CBCL sleep problems | −0.02 | 0.02 | −0.03 | −0.77 | ||
| MSEL expressive language | −0.11 | 0.03 | −0.25 | −3.94*** | ||
| MSEL fine motor | −0.05 | 0.02 | −0.11 | −2.15* | ||
| MSEL receptive language | −0.01 | 0.03 | −0.03 | −0.42 | ||
| MSEL visual reception | 0.01 | 0.02 | 0.02 | 0.39 | ||
| Vineland Adaptive Behavior | −0.12 | 0.02 | −0.32 | −6.71*** | ||
| Child ethnicity not Hispanic (REF) | ||||||
| Child ethnicity Hispanic | 0.36 | 0.47 | 0.03 | 0.77 | ||
| Child sex male (REF) | ||||||
| Child sex female | −0.57 | 0.43 | −0.04 | −1.34 | ||
| Child race White (REF) | ||||||
| Child race Multiracial | 0.61 | 0.40 | 0.08 | 1.54 | ||
| Child race Black | 0.47 | 0.36 | 0.07 | 1.29 | ||
| Child race other | −0.77 | 0.44 | −0.10 | −1.73 | ||
| Maternal education | 0.64 | 0.16 | 0.14 | 4.03*** | ||
| SEED site NC (REF) | ||||||
| SEED site CA | 0.27 | 039 | 0.03 | 0.71 | ||
| SEED site CO | −0.03 | 0.36 | −0.00 | −0.10 | ||
| SEED site GA | −1.91 | 0.35 | −0.22 | −5.41*** | ||
| SEED site MD | −0.61 | 0.39 | −0.07 | −1.58 | ||
| SEED site PA | 1.45 | 0.39 | 0.15 | 3.65*** |
t-value significant at the 0.05 level
t-value significant at the 0.01 level
t-value significant at the 0.001 level
Table 4.
Hierarchical linear regression model for the Study to Explore Early Development Degree of Impairment Associated with Autism
| R2 | Δ R2 | B | SE B | β | t-value | |
|---|---|---|---|---|---|---|
| Step 1 | 0.46 | – | ||||
| Intercept | 8.78 | 0.43 | 20.54 | |||
| CBCL externalizing behaviors | −0.08 | 0.01 | −0.07 | −1.52 | ||
| CBCL internalizing behavior | 0.01 | 0.01 | 0.07 | 1.62 | ||
| CBCL sleep problems | −0.01 | 0.01 | −0.05 | −1.27 | ||
| MSEL expressive language | −0.02 | 0.01 | −0.20 | −3.27** | ||
| MSEL fine motor | −0.01 | 0.01 | −0.07 | −1.36 | ||
| MSEL receptive language | −0.00 | 0.01 | −0.04 | −0.39 | ||
| MSEL visual reception | −0.00 | 0.01 | −0.04 | −0.67 | ||
| Vineland Adaptive Behavior | −0.00 | 0.01 | −0.44 | −9.54*** | ||
| Step 2 | 0.53 | 0.07 | ||||
| Intercept | 7.85 | 0.44 | 17.78 | |||
| CBCL externalizing behaviors | −0.01 | 0.01 | −0.05 | −1.06 | ||
| CBCL internalizing behavior | 0.01 | 0.01 | 0.09 | 2.03* | ||
| CBCL sleep problems | −0.00 | 0.01 | −0.03 | −0.90 | ||
| MSEL expressive language | −0.03 | 0.01 | −0.26 | −4.36*** | ||
| MSEL fine motor | −0.01 | 0.01 | −0.10 | −2.06* | ||
| MSEL receptive language | −0.00 | 0.01 | −0.01 | −0.15 | ||
| MSEL visual reception | −0.00 | 0.01 | −0.01 | −0.16 | ||
| Vineland Adaptive Behavior | −0.04 | 0.00 | −0.40 | −9.04*** | ||
| Child ethnicity not Hispanic (REF) | ||||||
| Child ethnicity Hispanic | 0.12 | 0.11 | 0.04 | 1.16 | ||
| Child sex male (REF) | ||||||
| Child sex female | −0.05 | 0.10 | −0.01 | −0.44 | ||
| Child race White (REF) | ||||||
| Child race Multiracial | 0.04 | 0.09 | 0.02 | 0.48 | ||
| Child race Black | 0.01 | 0.08 | 0.01 | 0.13 | ||
| Child race other | −0.09 | 0.10 | −0.03 | −0.52 | ||
| Maternal education | 0.09 | 0.04 | 0.08 | 2.30* | ||
| SEED site NC (REF) | ||||||
| SEED site CA | 0.05 | 0.09 | 0.02 | 0.56 | ||
| SEED site CO | −0.28 | 0.08 | −0.13 | −3.38** | ||
| SEED site GA | −0.54 | 0.08 | −0.26 | −6.74*** | ||
| SEED site MD | 0.02 | 0.09 | 0.01 | 0.28 | ||
| SEED site PA | 0.33 | 0.10 | 0.13 | 3.26*** |
t-value significant at the 0.05 level
t-value significant at the 0.01 level
t-value significant at the 0.001 level
Discussion
Measuring ASD symptom severity independent of other developmental and demographic factors provides quantification of the core symptoms of ASD for various research and treatment paradigms. The goals of this study were to examine the relationship between the ADOS CSS, ADOS total score, and SEED DOI among children classified as ASD in SEED and determine which of these three measures of ASD severity was least influenced by developmental and demographic characteristics. Results of the correlation analysis showed that the ADOS CSS was significantly and positively related to the ADOS total score and SEED DOI; results of the linear regression analyses showed that the ADOS CSS reduced the effects of co-occurring conditions and demographic features on the severity of ASD diagnostic symptoms more so than other measures of ASD severity. Our study findings are in agreement with those of Gotham et al. (2009) and others that assessed children into school-age and suggest that, among the measures we evaluated, the ADOS CSS is the best measure of core features of ASD in pre-school children.
Adaptive behavior skills explained a significant amount of variance of ASD symptom severity, with the SEED DOI having the strongest relationship with the VABS-II ABC, followed by the ADOS total score then ADOS CSS. Poor adaptive behavior is a common co-occurring condition for children with ASD and may partially result from core deficits in social and communication skills also measured on adaptive tests (Kanne et al. 2010). However, a global measure of adaptive behavior such as the VABS-II also measures daily living skills and motor skills, and is therefore not diagnostic of or specific to ASD. The present study indicates that the ADOS CSS mitigates the influence of adaptive behavior on ASD severity to a greater degree than the other measures of ASD severity. Thus, the ADOS CSS may be a metric that is relatively free of the impact of adaptive behavior skills compared to the ADOS total score and SEED DOI.
Expressive language abilities as measured by the MSEL also explained a significant amount of variance of ASD symptom severity for all three severity outcomes. This finding could be due to the challenges of appraising expressive language skills (e.g., describing events, labeling objects, and using appropriate grammar) separate from pragmatic language skills (e.g., engaging in conversation and using verbal and non-verbal language to initiate and maintain social interactions). Indeed, the most common first concern among parents of children with ASD is delayed language development (Kozlowski et al. 2011). However, the diagnostic features of ASD focus on the social use of language rather than the practical use of language, even though expressive language delay is a frequent co-occurring condition. The ADOS CSS mitigates the influence of expressive language abilities on ASD severity ratings more than do other severity outcomes (Tables 2, 3, 4). Consequently, the ADOS CSS may be a more accurate measure of difficulties with social communication versus problems with expressive language than the ADOS total score and SEED DOI.
Internalizing behavior problems and fine motor delays were significantly associated with the ADOS CSS, ADOS total score, and SEED DOI. These findings replicate recent research that highlights an association between ASD severity and avoidant behaviors (Jang and Matson 2015). The CBCL internalizing behavior problem scale—which was used as a measure of internalizing behaviors in this study— is comprised of emotional reactivity, anxiety/depression, somatic complaints, and withdrawn behavior. Items on these subscales overlap with ASD diagnostic features captured on the ADOS (e.g., avoids eye contact, disturbed by change, and little interest in others). Clinicians and parents may therefore recognize similar symptoms in the child that are noted by the parent on the CBCL and noted by the clinician on the ADOS and SEED DOI. Early motor delays have been implicated in the subsequent development of ASD (Flanagan et al. 2012) and ASD severity ratings (MacDonald et al. 2014), although the relationship between ASD severity and fine motor delays is sparse. The association between fine motor delays and ASD severity should be investigated in future research.
In terms of demographic and other variables included in the analyses, maternal education and SEED site were also significantly associated with the ADOS CSS, ADOS total score, and SEED DOI. Distribution of severity by site may vary due to many factors, including sources utilized for case identification and site specific recruiting practices. Importantly, the SEED study used an extremely robust quality control procedure to ensure consistency of ADOS scoring across sites. However, even with reliable coding on standardized diagnostic instruments, there are likely to be differences in the characteristics of state populations that are associated with and could impact severity. Our observation that maternal education impacts ASD severity regardless of outcome measure suggest that maternal education should be accounted for in future analyses.
Developmental characteristics, particularly adaptive abilities and expressive language skills, explained the most amount of variance in the SEED DOI outcome, indicating that clinicians respond to global child functioning more than diagnostic features of ASD when estimating ASD symptom severity. This is not surprising given that many clinicians are trained to focus on the whole child when determining global severity ratings, and adaptive and language delays could be especially impairing. Clinical ratings of the core symptoms of ASD independent from adaptive, language, or other child characteristics might require specific training. Further, reliability and validity of those ratings would need to be demonstrated.
CBCL externalizing behavior problems were significant factors in the ADOS CSS and ADOS total score, but not the SEED DOI. Attention problems and aggressive behavior could impact ADOS items such as quality of social overtures and quality of rapport, whereas attention problems and aggressive behavior could be viewed by clinicians as associated features of ASD. This again suggests that clinicians making global ratings could find it useful to consider the type of impairment they are capturing when assessing ASD severity.
This study had a number of strengths and limitations. The sample size was large and represented participants from multiple geographic locations. Children were classified as ASD after a comprehensive evaluation conducted by clinicians who established and maintained research reliability on diagnostic instruments. Results are comparable with previous research and suggest the ADOS CSS best measures ASD symptom severity while controlling for the influence of other developmental and demographic factors. A few limitations were noted. SEED site was a significant predictor of ASD symptom severity despite outcome measure and reliability standards. Higher maternal education also predicted higher ASD severity scores. These findings can help guide future research on ASD severity in young children.
In sum, we found that the ADOS CSS effectively measures ASD symptom severity in pre-school children from multiple geographic areas. The ADOS CSS is less impacted by developmental and demographic factors than the ADOS total score or clinician ratings. We thus conclude that the ADOS CSS is an appropriate measure for ASD symptom severity in clinical, epidemiological, and treatment research; although purpose should be considered when selecting a severity measure in pre-school children.
Acknowledgments
This publication was supported by six cooperative agreements from the Centers for Disease Control and Prevention: Cooperative Agreement Number U10DD000180, Colorado Department of Public Health; Cooperative Agreement Number U10DD000181, Kaiser Foundation Research Institute (CA); Cooperative Agreement Number U10DD000182, University of Pennsylvania; Cooperative Agreement Number U10DD000183, Johns Hopkins University; Cooperative Agreement Number U10DD000184, University of North Carolina at Chapel Hill; and Cooperative Agreement Number U10DD000498, Michigan State University. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC or HRSA.
Footnotes
Author Contributions Study concept (Lisa Wiggins), study design and methods (all authors), statistical analysis (Brian Barger), statistical review and interpretation (all authors), manuscript preparation and/or review (all authors).
Compliance with Ethical Standards
Conflict of interest The authors do not have any conflicts of interest to report.
Ethical Standard All procedures performed in this study were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent Informed consent was obtained from all individual participants included in the study.
References
- Achenbach T. Child Behavior Checklist. Burlington: Achenbach System of Empirically based Assessment; 1992. [Google Scholar]
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th. Arlington: American Psychiatric Publishing; 2013. [Google Scholar]
- Cohen J. Statistical power analysis for the behavioral sciences. 2nd. Hillsdale: Lawrence Earlbaum Associates; 1988. [Google Scholar]
- Flanagan JE, et al. Head lag in infants at risk for autism: A preliminary study. Journal of Occupational Therapy. 2012;66:577–585. doi: 10.5014/ajot.2012.004192. [DOI] [PubMed] [Google Scholar]
- Gotham K, Pickles A, Lord C. Standardizing ADOS scores for a measure of severity inautism spectrum disorders. Journal of Autism and Developmental Disorders. 2009;39(5):693–705. doi: 10.1007/s10803-008-0674-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gotham K, Pickles A, Lord C. Trajectories of autism severity in children using standardized ADOS scores. Pediatrics. 2012;130:e1278–e1284. doi: 10.1542/peds.2011-3668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jang J, Dixon DR, Tarbox J, Granpeesheh D. Symptom severity and challenging behavior in children with ASD. Research in Autism Spectrum Disorders. 2011;5(3):1028–1032. [Google Scholar]
- Jang J, Matson JL. Autism severity as a predictor of comorbid conditions. Journal of Developmental and Physical Disabilities. 2015;27:405–415. [Google Scholar]
- Johnson CP, Myers SM, The Council on Children with Disabilities Identification and evaluation of children with autism spectrum disorders. Pediatrics. 2007;120:1183–1215. doi: 10.1542/peds.2007-2361. [DOI] [PubMed] [Google Scholar]
- Kanne SM, Gerber AJ, Quirmbach LM, Sparrow SS, Cicchetti DV, Saulnier CA. The role of adaptive behavior in autism spectrum disorders: Implications for functional outcome. Journal of Autism and Developmental Disorders. 2011;41(8):1007–1018. doi: 10.1007/s10803-010-1126-4. [DOI] [PubMed] [Google Scholar]
- Kozlowski AM, Matson JL, Horovitz M, Worley JA, Neal D. Parents’ first concerns of their child’s development in toddlers with autism spectrum disorder. Developmental Neurorehabilitation. 2011;14(2):72–78. doi: 10.3109/17518423.2010.539193. [DOI] [PubMed] [Google Scholar]
- Lee L, David AB, Rusyniak J, Landa R, Newschaffer CJ. Performance of the Social Communication Questionnaire in children receiving preschool special education services. Research in Autism Spectrum Disorders. 2007;1:126–138. [Google Scholar]
- Levy SE, Giarelli E, Lee LC, Schieve L, Kirby R, Cunniff C, et al. Autism spectrum disorder and co-occurring developmental, psychiatric, and medical conditions among children in multiple populations of the United States. Journal of Developmental and Behavioral Pediatrics. 2010;31(4):267–275. doi: 10.1097/DBP.0b013e3181d5d03b. [DOI] [PubMed] [Google Scholar]
- Lord C, Rutter M, DiLavore PC, Risi S. Autism Diagnostic Observation Schedule. Los Angeles: Western Psychological Services; 1999. [Google Scholar]
- Lord C, Rutter M, Le Couteur AL. Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders. 1994;24(5):659–685. doi: 10.1007/BF02172145. [DOI] [PubMed] [Google Scholar]
- MacDonald M, Lord C, Ulrich DA. Motor skills and calibrated autism severity in young children with autism spectrum disorder. Adapted Physical Activity Quarterly. 2014;31(2):95–105. doi: 10.1123/apaq.2013-0068. [DOI] [PubMed] [Google Scholar]
- Mullen E. Mullen Scales of Early Learning. San Antonio: Pearson; 1995. [Google Scholar]
- Rutter MA, Bailey A, Lord C. The Social Communication Questionnaire. Los Angeles: Western Psycholgical Services; 2003. [Google Scholar]
- Schendel D, DiGuiseppi C, Croen L, Fallin D, Reed P, Schieve L, et al. The Study to Explore Early Development (SEED): A multi-site epidemiologic study of autism by the Centers for Autism and Developmental Disabilities Research and Epidemiology (CADDRE) Network. Journal of Autism and Developmental Disorders. 2012;42:2121–2140. doi: 10.1007/s10803-012-1461-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schreck KA, Mulick JA, Smith AF. Sleep problems as possible predictors of intensified symptoms of autism. Research in Developmental Disabilities. 2004;25:57–66. doi: 10.1016/j.ridd.2003.04.007. [DOI] [PubMed] [Google Scholar]
- Shumway S, Farmer C, Thurm A, Joseph L, Black D, Golden C. The ADOS calibrated severity score: Relationship to phenotypic variables and stability over time. Autism Research. 2012;5(4):267–276. doi: 10.1002/aur.1238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sparrow S, Balla D, Cicchetti D. Vineland Adaptive Behavior Scales. 2nd. San Antonio: Pearson; 2005. [Google Scholar]
- Wiggins LD, Bakeman R, Adamson LB, Robins DL. The utility of the Social Communication Questionnaire in screening for autism in children referred for early intervention. Focus on Autism and Other Developmental Disabilities. 2007;22:33–38. [Google Scholar]
- Wiggins LD, Reynolds A, Rice C, Moody EJ, Bernal P, Blaskey L, Rosenberg SA, Lee LC, Levy S. Using standardized diagnostic instruments to classify children with autism in the Study to Explore Early Development. Journal of Autism and Developmental Disorders. 2015;45:1271–1280. doi: 10.1007/s10803-014-2287-3. [DOI] [PMC free article] [PubMed] [Google Scholar]