Abstract
There is a critical need for validated screening tools for autism spectrum disorder in very young children so families can access tailored intervention services as early as possible. Few screeners exist for children between the recommended screening ages of 18–24 months. This study examined the utility of a new autism-specific parent-report screening tool, the Early Screening for Autism and Communication Disorders for children 12–36 months. Field-testing was conducted from five sites with 471 children screened for communication delays in primary care or referred for familial risk or concern for autism spectrum disorder. The Early Screening for Autism and Communication Disorders was evaluated in three age groups: 12–17, 18–23, and 24–36 months. A best-estimate diagnosis of autism spectrum disorder, developmental delay, or typical development was made. Receiver operating characteristic curves were examined for all 46 items and the 30 items that best discriminated autism spectrum disorder from the non-spectrum groups. Area under the curve estimates for the total were greater than 0.90 across age groups. Cutoffs were established for each age group with sensitivity between 0.86 and 0.92 and specificity between 0.74 and 0.85. Results provide preliminary support for the validity of the Early Screening for Autism and Communication Disorders as an autism-specific screener in children 12–36 months with elevated risk of communication delay or autism spectrum disorder.
Lay abstract
There is a critical need for accurate screening tools for autism spectrum disorder in very young children so families can access tailored intervention services as early as possible. However, there are few screeners designed for children 18–24 months. Developing screeners that pick up on the signs of autism spectrum disorder in very young children has proved even more challenging. In this study, we examined a new autism-specific parent-report screening tool, the Early Screening for Autism and Communication Disorders for children between 12 and 36 months of age. Field-testing was done in five sites with 471 children screened for communication delays in primary care or referred for familial risk or concern for autism spectrum disorder. The Early Screening for Autism and Communication Disorders was tested in three age groups: 12–17, 18–23, and 24–36 months. A best-estimate diagnosis of autism spectrum disorder, developmental delay, or typical development was made. Analyses examined all 46 items and identified 30 items that best discriminated autism spectrum disorder from the non-spectrum groups. Cutoffs were established for each age group with good sensitivity and specificity. Results provide preliminary support for the accuracy of the Early Screening for Autism and Communication Disorders as an autism-specific screener in children 12–36 months with elevated risk of communication delay or autism spectrum disorder.
Keywords: autism spectrum disorder, Early Screening for Autism and Communication Disorders, field-testing, screening, validation
Advances in research have documented reliable diagnoses of autism spectrum disorder (ASD) can be made between 14 and 24 months of age (Guthrie et al., 2013; Ozonoff et al., 2015; Pierce et al., 2019; Zwaigenbaum, Bauman, Stone, et al., 2015). Since 2007, the American Academy of Pediatrics (AAP) has recommended screening all children for ASD at 18 and 24 months (Johnson & Myers, 2007) and recently reaffirmed these recommendations (Hyman et al., 2020). Yet, the median age of diagnosis in the United States has hovered at 4–5 years for more than a decade (Maenner et al., 2020). Mounting research has shed light on limitations of available autism screeners (Carbone et al., 2020; Guthrie et al., 2019; Stenberg et al., 2014; Sturner, Howard, Bergmann, Stewart, & Afarian, 2017; Yuen et al., 2018), one factor contributing to late diagnosis. There is a critical need for screeners to improve early detection. This article reports on field-testing of a new autism screener for children 12–36 months with elevated risk of communication delay or ASD.
Limitations of existing screening approaches and tools
The Modified Checklist for Autism in Toddlers (M-CHAT) or M-CHAT-Revised (M-CHAT-R; Robins et al., 2001, 2014) is the most widely studied and utilized autism screener for toddlers. Reported sensitivity and specificity for this screener may be overestimated because, for many studies, only children receiving positive screens completed diagnostic evaluations (Zwaigenbaum, Bauman, Fein, et al., 2015). Studies by Robins and colleagues have reported estimated sensitivities over 0.80 (Chlebowski et al., 2013; Kleinman et al., 2008; Robins et al., 2001, 2014) based on screening children referred for suspected ASD, receiving early intervention (EI), or with familial risk as well as in primary care. However, more recent large studies conducted in primary care have shown limitations in the performance of the M-CHAT/M-CHAT-R (Carbone et al., 2020; Guthrie et al., 2019; Stenberg et al., 2014, 2021; Sturner, Howard, Bergmann, Morrel, et al., 2017; Sturner, Howard, Bergmann, Stewart, & Afarian, 2017).
Robins et al. (2014) concluded that the M-CHAT should be used with the Follow-Up interview (i.e., M-CHAT/F) to reduce false positives and avoid unnecessary referrals, as positive predictive value (PPV) increased from 0.36 to 0.74 with the Follow-Up interview. However, in a review of electronic medical records of universal screening using the M-CHAT/F between 16 and 24 months in a large pediatric practice (N = 25,999), the M-CHAT/F showed substantially lower sensitivity (0.39; Guthrie et al., 2019), suggesting that this tool may miss more children with ASD than it detects. Similar findings were reported in a larger primary care sample using the M-CHAT (Carbone et al., 2020). The age between screening and diagnosis was much greater than in Robins et al. (2014), which may partially explain poorer estimates of accuracy. Furthermore, in both studies, children with positive screens were diagnosed earlier than those with a negative screen or no screen, suggesting that screening continues to be important.
Research characterizing children accurately identified by the M-CHAT/M-CHAT-R has indicated that developmental level and age are important factors. Robins et al. (2014) reported that their ASD sample had below average developmental levels, with standard scores approximately two standard deviations below the mean. Results of a meta-analysis of 19 studies comparing outcomes of 1,658 children with ASD ascertained using the M-CHAT indicated that developmental level was significantly higher in infant sibling samples (M = 81.25), than both community referral (M = 62.71) and universal screening samples (M = 60.91), providing evidence of a significant bias toward detecting children with lower developmental level (Micheletti et al., 2019).
Research on the M-CHAT/M-CHAT-R in primary care suggests it performs better in children 24–36 months compared to children under 24 months (Guthrie et al., 2019; Pandey et al., 2008; Sturner, Howard, Bergmann, Stewart, & Afarian, 2017; Toh et al., 2018; Yuen et al., 2018). For example, Guthrie et al. (2019) reported a sensitivity of 0.35 for children 16–20 months, compared to 0.49 for children 21–26 months. With regard to developmental level, in another population study, Stenberg et al. (2021) reported that at 18 months of age, the M-CHAT identified only 28.8% of children with ASD and identified 81.3% of children with intellectual disability without ASD. Collectively, these findings indicate that, in general population samples, the M-CHAT may be missing far more children with ASD than it is detecting at 18–24 months and identifying more children with developmental delays (DDs) without ASD, highlighting the need for improved screeners at younger ages and for primary care settings.
The new AAP clinical report (Hyman et al., 2020) identified two promising autism screening tools for children 18–30 months of age, the Parents Observations of Social Interactions (POSI; Smith et al., 2013) and the Early Screening for Autism and Communication Disorders (ESAC). The POSI is a brief questionnaire been studied with preliminary community-referred samples, showing favorable comparison on sensitivity, but lower performance on specificity than the M-CHAT (Salisbury et al., 2018; Smith et al., 2013). Additional validation in larger samples is needed to understand psychometric properties and clinical utility of the POSI. The ESAC is a parent-report tool developed with an initial set of 46 items to capture the heterogeneity of delayed social communication skills and early signs of autism symptoms. Preliminary research on the development of ESAC items revealed a group difference with a large effect (g = 1.15) in ESAC Total scores between children with and without ASD, who were administered the ESAC at 18–36 months (Stronach & Wetherby, 2017). Research studying the psychometric performance of the ESAC in preliminary community-referred and population samples is needed.
Accurately screening for ASD younger than 18 months is even more challenging. Research on the Early Screening of Autistic Traits Questionnaire (Dietz et al., 2006), designed to screen at 14–15 months in primary care, indicated a detection rate of 0.6/1,000. Similarly, the First Year Inventory for 12-month-olds (Reznick et al., 2007) demonstrated a sensitivity of 0.44 (Turner-Brown et al., 2013). While these screeners show promise, the low accuracy does not yet support clinical use.
Most children with ASD exhibit observable symptoms by 12–24 months (Bacon et al., 2018; Pierce et al., 2019; Zwaigenbaum, Bauman, Stone, et al., 2015), suggesting the potential of screening under 24 months. Existing tools may have limited utility capturing the heterogeneity of early behavioral manifestations of ASD, including symptoms in restricted, repetitive behaviors (RRBs) observed at or before 12–24 months (Dow et al., 2017, 2020; Elison et al., 2014; Kim & Lord, 2010; Richler et al., 2007; Watt et al., 2008; Wolff et al., 2014). The accuracy of autism screeners may be improved by incorporating questions capturing the range of ASD symptoms in young children.
Purpose of this study
The ESAC addresses the need for a more accurate and comprehensive parent-report screening tool by incorporating questions about early social communication and the presence of RRBs, based on core diagnostic features of ASD (American Psychiatric Association [APA], 2013). This study reports the performance of the ESAC during field-testing with children 12–36 months of age recruited from five sites using varied ascertainment sources. We recruited children with parental or professional concerns for ASD or DD, familial-risk, or positive primary care screening for communication delay yielding a diverse sample of children with elevated risk for ASD. The first aim of this study was to examine item-level performance in order to identify the items that best differentiated these children with ASD and without ASD. The second aim was to examine the preliminary psychometric properties of the best set of ESAC items and performance across three age groups in this elevated risk sample: 12–17, 18–23, and 24–36 months.
Methods
Procedures for sample selection and recruitment
Participants were 471 toddlers 12–36 months of age, recruited from five sites: Florida State University (FSU; n = 299), University of California, San Diego (UCSD; n = 66), National Institute of Mental Health (NIMH; n = 49), University of Michigan Autism and Communication Disorders Center (UM; n = 45), and University of California, Davis (UCD; n = 12). Children at FSU and UCSD were referred from primary care screening for communication delay using the Infant–Toddler Checklist (ITC; Wetherby & Prizant, 2002). Children with positive ITC screens, negative ITC screens with parental concern noted, and randomly selected negative ITC screens without parent concern (about 20% of FSU sample) were recruited for this study. Children at NIMH and UM were referred for diagnostic evaluations because of parental or professional concern about possible ASD. Children at UCD were referred because they had familial risk due to having an older sibling with ASD. All children participated in studies approved by Institutional Review Boards at each site, and parents provided informed consent.
Parents were asked to complete the ESAC prior to or at the time of the developmental or diagnostic evaluation, before most families received the diagnosis. Only nine ESACs were completed more than a month after the autism diagnostic evaluation. Some children were seen longitudinally, resulting in multiple ESACs. The earliest ESAC within each age group was chosen, yielding 596 ESACs (12–17 months: n = 159; 18–23 months: n = 187; 24–36 months: n = 250). At each site, a clinical best-estimate diagnosis of ASD (n = 183) or non-spectrum (NS; n = 288) was made by experienced clinicians using all available information. The NS group included children with DD (n = 77) and typical development (TD; n = 211). All children showing signs of ASD completed the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2000; Lord, Luyster, et al., 2012; Lord, Rutter, et al., 2012) and Mullen Scales of Early Learning (MSEL; Mullen, 1995) for diagnosis or rule out. All children with non-spectrum DD completed the MSEL, and 53% completed the ADOS. Most children with DD had one or more scores on the MSEL below the 10th percentile (i.e., T score ⩽ 37). There were 30 children across all sites in the DD group who had significant developmental or psychological concerns in domains not captured by the MSEL, including highly unintelligible speech or high anxiety. In these cases, classification of DD was based on clinical judgment. For children with TD, 74% received the MSEL and 22% also received the ADOS. The remaining 26% of the TD group were from the FSU site and received the Communication and Symbolic Behavior Scales (CSBS; Wetherby & Prizant, 2002) Behavior Sample, a standardized norm-referenced communication evaluation. Children assigned to the TD group had standard scores above the 10th percentile on the MSEL or CSBS and/or had no signs of delay based on clinical judgment.
Sample demographic and developmental characteristics are summarized in Table 1. Sex was reported for all but three children. Race and ethnicity were available for 73% (n = 343) of the sample. Maternal education (72%; n = 338) served as a proxy for socioeconomic status. The sample was primarily male and White, with most mothers having a college education.
Table 1.
Sample characteristics by diagnostic group.
| Characteristic | ASD (n = 183) | DD (n = 77) | TD (n = 211) | NS (n = 288) | ||||
|---|---|---|---|---|---|---|---|---|
| M | (SD) | M | (SD) | M | (SD) | M | (SD) | |
| Male, n (%) | 156 | (85) | 55 | (71) | 121 | (57) | 176 | (61) |
| Race, n (%) | ||||||||
| White | 97 | (53) | 35 | (46) | 145 | (69) | 180 | (63) |
| Black | 12 | (7) | 8 | (10) | 16 | (8) | 24 | (8) |
| Asian | 2 | (1) | 2 | (3) | 6 | (3) | 8 | (3) |
| Pacific Islander | 0 | (0) | 0 | (0) | 1 | (< 1) | 1 | (< 1) |
| Multiracial | 7 | (4) | 6 | (8) | 6 | (3) | 12 | (4) |
| NR | 65 | (36) | 26 | (34) | 37 | (18) | 62 | (22) |
| Ethnicity, n (%) | ||||||||
| Hispanic | 11 | (6) | 5 | (7) | 14 | (7) | 19 | (7) |
| Maternal education (years) | 15.57 | (2.28) | 16.18 | (2.27) | 15.84 | (2.20) | 15.91 | (2.21) |
| ADOS, n (%) | 183 | (100) | 41 | (53) | 47 | (22) | 88 | (31) |
| Age (months) | 28.79 | (7.42) | 29.19 | (7.45) | 26.20 | (7.35) | 27.59 | (7.51) |
| SA CSS | 6.24 | (2.11) | 2.39 | (1.43) | 1.79 | (1.16) | 2.07 | (1.32) |
| RRB CSS | 6.89 | (2.32) | 4.00 | (2.32) | 2.62 | (2.20) | 3.26 | (2.35) |
| Total CSS | 6.37 | (2.07) | 2.15 | (1.22) | 1.60 | (0.93) | 1.85 | (1.10) |
| MSELa, n (%) | 183 | (100) | 77 | (100) | 157 | (74) | 234 | (81) |
| Age (months) | 27.97 | (7.34) | 28.66 | (6.74) | 27.89 | (6.77) | 28.15 | (6.74) |
| VR | 40.12 | (14.12) | 46.29 | (12.00) | 60.46 | (10.52) | 55.76 | (12.88) |
| FM | 38.02 | (13.11) | 43.64 | (11.83) | 54.55 | (10.73) | 50.93 | (12.22) |
| RL | 32.63 | (14.01) | 42.53 | (13.02) | 57.10 | (9.85) | 52.27 | (12.95) |
| EL | 33.70 | (13.17) | 37.64 | (11.11) | 54.88 | (11.46) | 49.16 | (13.94) |
| ELC | 75.10 | (20.77) | 86.19 | (16.51) | 113.36 | (15.80) | 104.34 | (20.50) |
| Nonverbal DQ | 87.11 | (19.99) | 97.04 | (18.25) | 116.81 | (16.18) | 110.31 | (19.25) |
| Verbal DQ | 69.40 | (28.59) | 85.34 | (22.07) | 116.63 | (20.93) | 106.33 | (25.87) |
ASD: autism spectrum disorder; DD: developmentally delayed; TD: typically developing; NS: non-spectrum; NR: not reported; ADOS: Autism Diagnostic Observation Schedule; SA: Social Affect; CSS: calibrated severity score; RRB: restrictive/repetitive behavior; MSEL: Mullen Scales of Early Learning; VR: Visual Recognition; FM: Fine Motor; RL: Receptive Language; EL: Expressive Language; ELC: Early Learning Composite; DQ: developmental quotient; NS groups include DD and TD children. MSEL T scores based on M = 50, SD = 10; ELC based on M = 100 and SD = 15.
MSEL subscale and composite scores were not reported for n = 7; DQs were provided.
Measures
ESAC
The ESAC was initially developed as an autism screener to follow up broadband screeners like the ITC, an AAP-recommended screener (Hyman et al., 2020; Johnson & Myers, 2007) for communication delays including ASD. The ESAC field-test version included 46 items generated by three authors (A.M.W., J.W., and C.L.) covering ASD symptom domains (APA, 2013) to measure lack of typical social communication development and presence of RRBs. The majority of items (35) were rated on a 3-point scale as occurring Often, Sometimes, or Not Yet/Rarely/Never. Higher scores indicated greater symptom severity (i.e., absence of typical skills or presence of unusual behaviors). Eight of the remaining 11 items comprised lists of specific behaviors that parents indicated as present/absent or rated on a 3-point scale. Responses for these eight items were summed to create an item-level score. Two items asked about number of words and word loss but were not included in the total score.
MSEL
The MSEL is a standardized assessment of developmental level. Standard T scores for Visual Reception, Fine Motor, Expressive and Receptive Language subscales were obtained. Because nearly 25% of children received one or more T scores of 20 (i.e., the floor), developmental quotients (DQs) were calculated. DQ was derived by averaging Visual Reception and Fine Motor age equivalents for Nonverbal DQ and Receptive and Expressive Language age equivalents for Verbal DQ, and then dividing each by chronological age and multiplying by 100.
ADOS
The ADOS is a semi-structured, standardized assessment that measures symptoms of ASD and provides domain and total scores. Children received different modules based on age and verbal ability. To allow comparison of scores across modules, calibrated severity scores (CSSs) were derived (Esler et al., 2015; Gotham et al., 2009; Hus et al., 2014).
Statistical analyses
Receiver operating characteristic (ROC) curves, based on logistic regressions, were estimated to generate area under the curve (AUC) values and examine performance of each item in discriminating between the ASD and NS groups.
For items with derived sums, ROC analysis results were used to determine item-level cutoffs that would convert scores into the same 3-point rating scale. Criteria for assigning a “2” included a specificity value as close to 0.90 as possible and sensitivity ⩾0.60. The cutoff for a “1” was based on optimal sensitivity and specificity (i.e., as close to 0.80 as possible). Cutoff values for these items were examined separately across age groups.
Using the best performing items, ROC curves were estimated with the sum of retained items creating an ESAC total to generate AUC and sensitivity/specificity values. Curves were estimated separately across age groups to compare optimal cutoffs and estimate PPVs and negative predictive values (NPVs). Internal consistency of the ESAC was estimated using coefficient alpha. Test–retest reliability was approximated for the subsample with two ESAC administrations (n = 111).
Community involvement
There was no community involvement in this study.
Results
Site comparisons of children with ASD
The 121 children recruited from primary care samples (FSU, UCSD) were significantly younger than the 62 recruited from referred samples (NIMH, UM, UCD; p < 0.001) at the time of the first ESAC and the ADOS (p < 0.01) and MSEL (p < 0.05). However, comparisons across sites showed nonsignificant differences for ADOS Total CSS and MSEL scores (all p > 0.05).
Item-level analyses
Item-level results are presented in Table 2. Twenty items had AUC values less than 0.65. We chose to retain four of these items because they captured the heterogeneity of clinically relevant RRBs (Items 32, 41, 44, 46; endorsing these items likely indicates clinical concern for ASD but frequency of endorsement is low). A total of 30 items were retained. Cutoffs, sensitivity, and specificity values for the seven continuous items are presented in Table 3. Because these items had different ranges (see Table 2), the values that met the criteria outlined above for a “1” and “2” varied across items and age. Different cutoff values were identified across age groups for three items (21, 31, and 37); cutoffs did not vary by age for the remaining items.
Table 2.
Area under the curve and item-level information across diagnostic groups.
| Item | Item content | AUC | ASD | Non-spectrum | Score range |
|---|---|---|---|---|---|
| M (SD) | M (SD) | ||||
| Q1 | Clearly requests | 0.66 | 0.58 (0.70) | 0.16 (0.44) | 0–2 |
| Q2 | Express happiness | 0.59 | 0.50 (0.92) | 0.17 (0.57) | 0–4 |
| Q3 | Enjoy social touches/affection | 0.59 | 0.25 (0.49) | 0.04 (0.22) | 0–2 |
| Q4 | Directed smiling/laughing | 0.62 | 0.33 (0.52) | 0.05 (0.23) | 0–2 |
| Q5 | Response to name | 0.78 | 0.76 (0.55) | 0.15 (0.36) | 0–2 |
| Q6 | Response to voice | 0.76 | 0.94 (0.62) | 0.31 (0.49) | 0–2 |
| Q7 | “Checking in” eye gaze | 0.73 | 1.09 (0.65) | 0.46 (0.53) | 0–2 |
| Q8 | Eye gaze to get attention | 0.73 | 0.87 (0.67) | 0.29 (0.48) | 0–2 |
| Q9 | Response to directions with gestures | 0.75 | 0.90 (0.69) | 0.25 (0.46) | 0–2 |
| Q10 | Response to joint attention | 0.76 | 0.98 (0.72) | 0.25 (0.48) | 0–2 |
| Q11 | Ease of interactions | 0.72 | 0.62 (0.66) | 0.09 (0.30) | 0–2 |
| Q12 | Emotional dysregulation | 0.64 | 1.65 (1.48) | 0.94 (1.22) | 0–6 |
| Q13 | Calming behavior | 0.64 | 0.54 (0.58) | 0.27 (0.51) | 0–2 |
| Q14 | Initiation of communication | 0.73 | 0.81 (0.75) | 0.16 (0.40) | 0–2 |
| Q15 | Response to ignore | 0.54 | 1.11 (0.76) | 1.01 (0.65) | 0–2 |
| Q16 | Attention-getting behavior | 0.59 | 0.69 (0.67) | 0.47 (0.54) | 0–2 |
| Q17 | Initiation of joint attention for social purposes | 0.75 | 1.30 (0.70) | 0.57 (0.63) | 0–2 |
| Q18 | Drawing attention | 0.79 | 1.40 (0.67) | 0.53 (0.67) | 0–2 |
| Q19 | “Uh oh” response | 0.74 | 1.23 (0.84) | 0.40 (0.68) | 0–2 |
| Q20 | Hand as tool | 0.48 | 0.89 (0.86) | 0.88 (0.80) | 0–2 |
| Q21 | Gestures | 0.84 | 7.53 (3.77) | 2.54 (2.71) | 0–14 |
| Q22 | Integration of gestures, sounds, and eye gaze | 0.75 | 1.11 (0.83) | 0.32 (0.59) | 0–2 |
| Q23 | Unusual sounds | 0.58 | 1.10 (0.80) | 0.84 (0.83) | 0–2 |
| Q24 | Imitation of gestures, sounds, and words | 0.66 | 0.86 (0.68) | 0.41 (0.54) | 0–2 |
| Q25 | Unusual intonation | 0.59 | 0.33 (0.58) | 0.11 (0.38) | 0–2 |
| Q26 | Number of Words | 0.38 | 21.37 (33.46) | 28.67 (35.80) | 0–200 |
| Q27 | Loss of words | 0.60 | 0.55 (0.90) | 0.16 (0.54) | 0–1 |
| Q28 | Stereotyped phrases | 0.49 | 0.30 (0.64) | 0.28 (0.53) | 0–2 |
| Q29 | Unusual phrases | 0.49 | 0.31 (0.65) | 0.27 (0.52) | 0–2 |
| Q30 | Back and forth conversation | 0.68 | 1.68 (0.61) | 1.09 (0.84) | 0–2 |
| Q31 | Play with objects | 0.75 | 3.47 (2.97) | 1.23 (1.98) | 0–12 |
| Q32 | Play with a variety of toys | 0.63 | 0.41 (0.62) | 0.08 (0.29) | 0–2 |
| Q33 | Focus on objects rather than people | 0.78 | 1.24 (0.72) | 0.42 (0.59) | 0–2 |
| Q34 | Intense focus on a certain toy | 0.72 | 0.85 (0.78) | 0.26 (0.53) | 0–2 |
| Q35 | Sticky attention to a certain toy | 0.75 | 0.91 (0.68) | 0.28 (0.49) | 0–2 |
| Q36 | Pretend play | 0.70 | 1.30 (0.68) | 0.76 (0.66) | 0–2 |
| Q37 | Pretend play with toys | 0.74 | 5.95 (2.06) | 3.95 (2.32) | 0–8 |
| Q38 | Pretend play with two sequenced actions | 0.66 | 1.80 (0.48) | 1.35 (0.74) | 0–2 |
| Q39 | Attachment to unusual object | 0.59 | 0.47 (0.78) | 0.17 (0.46) | 0–2 |
| Q40 | Interest in other children own age | 0.76 | 1.14 (0.73) | 0.39 (0.60) | 0–2 |
| Q41 | Sensory responses | 0.64 | 1.06 (1.29) | 0.42 (0.80) | 0–8 |
| Q42 | Repetitive behaviors/body mannerisms | 0.74 | 1.61 (1.41) | 0.52 (0.91) | 0–6 |
| Q43 | Insistence on sameness | 0.71 | 1.85 (1.50) | 0.79 (1.09) | 0–6 |
| Q44 | Rituals and routines | 0.59 | 0.36 (0.61) | 0.12 (0.35) | 0–2 |
| Q45 | Unusual vocabulary | 0.54 | 0.26 (0.67) | 0.07 (0.37) | 0–2 |
| Q46 | Intense interests | 0.63 | 0.84 (1.33) | 0.23 (0.66) | 0–5 |
AUC: area under curve; ASD: autism spectrum disorder.
Bolded items were retained for the final item set.
Table 3.
Cutoff values for retained continuous items based on receiver operating characteristic analysis.
| Item | Age group (months) | Score of “2” Cutoff | Score of “1” Cutoff | ||||
|---|---|---|---|---|---|---|---|
| Cutoff value | Sensitivity | Specificity | Cutoff value | Sensitivity | Specificity | ||
| Q21 | 12–17 | 9 | 0.63 | 0.93 | 6 | 0.79 | 0.72 |
| 18–23 | 7 | 0.61 | 0.93 | 4 | 0.81 | 0.76 | |
| 24–36 | 5 | 0.70 | 0.91 | 3 | 0.85 | 0.82 | |
| Q31 | 12–17 | 6 | 0.56 | 0.89 | 4 | 0.81 | 0.73 |
| 18–23 | 3 | 0.63 | 0.93 | 2 | 0.78 | 0.82 | |
| 24–36 | 2 | 0.49 | 0.90 | 1 | 0.70 | 0.84 | |
| Q37 | 12–17 | 8 | 0.54 | 0.80 | 7 | 0.67 | 0.72 |
| 18–23 | 7 | 0.52 | 0.89 | 5 | 0.79 | 0.62 | |
| 24–36 | 6 | 0.52 | 0.86 | 4 | 0.82 | 0.63 | |
| Q41 | 12–17 | 2 | 0.21 | 0.96 | 1 | 0.40 | 0.78 |
| 18–23 | 2 | 0.22 | 0.95 | 1 | 0.49 | 0.66 | |
| 24–36 | 2 | 0.35 | 0.88 | 1 | 0.65 | 0.66 | |
| Q42 | 12–17 | 2 | 0.49 | 0.90 | 1 | 0.63 | 0.72 |
| 18–23 | 2 | 0.52 | 0.87 | 1 | 0.66 | 0.64 | |
| 24–36 | 2 | 0.47 | 0.89 | 1 | 0.79 | 0.63 | |
| Q43 | 12–17 | 2 | 0.42 | 0.86 | 1 | 0.61 | 0.70 |
| 18–23 | 2 | 0.48 | 0.73 | 1 | 0.73 | 0.45 | |
| 24–36 | 2 | 0.65 | 0.70 | 1 | 0.88 | 0.54 | |
| Q46 | 12–17 | 1 | 0.26 | 0.92 | – | – | – |
| 18–23 | 1 | 0.24 | 0.85 | – | – | – | |
| 24–36 | 1 | 0.54 | 0.80 | – | – | – | |
Psychometric properties of final ESAC item set
ROC curves showed high AUC values, indicating the ESAC performed consistently well across age groups (Table 4); however, optimal cutoffs differed by age group. Within the 12- to 17-month subsample a cutoff of 20 yielded balanced sensitivity (0.86) and specificity (0.84). The cutoff of 18 yielded the strongest sensitivity (0.87–0.88) and specificity (0.84–0.85) estimates for both the 18- to 23-month and 24- to 36-month subsamples. Estimated PPVs were highest for the oldest age group and NPVs were comparable across ages (Table 4).
Table 4.
ESAC total score receiver operating characteristic analysis results.
| Optimal cutoff | AUC | [95% CI] | Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|---|---|---|
| 12–17 months (n = 159) | 20 | 0.91 | [0.86, 0.96] | 0.86 | 0.82 | 0.64 | 0.94 |
| 18–23 months (n = 187) | 18 | 0.93 | [0.89, 0.97] | 0.87 | 0.85 | 0.76 | 0.92 |
| 24–36 months (n = 249) | 18 | 0.93 | [0.89, 0.96] | 0.88 | 0.84 | 0.81 | 0.90 |
AUC: area under curve; CI: confidence interval; PPV: positive predictive value; NPV: negative predictive value.
Developmental characteristics by screening status
ESAC Totals and developmental characteristics by screening status across age groups are reported in Table 5. A majority of children with ASD screened positive on the ESAC across age groups. Children with false negatives demonstrated a pattern of higher MSEL scores on average, compared to the children with true positives. Across age groups, there were only 28 false negatives of the 596 screens, and most of these were from the primary care sample. Children with true negatives demonstrated a pattern of higher MSEL scores on average than children with false positives. Across age groups, 52%–77% of the children with false positives had a classification of DD. Furthermore, the average time between screening and diagnosis based on the ADOS decreased with age (12–17 months: n = 70, M = 8.85 months, SD = 8.73; 18–23 months: n = 97, M = 6.40 months, SD = 7.32; 24–36 months: n = 160, M = 1.68 months, SD = 3.76).
Table 5.
Developmental characteristics by screening status.
| True positive | False positive | False negative | True negative | |||||
|---|---|---|---|---|---|---|---|---|
| M | SD | M | SD | M | SD | M | SD | |
| 12–17 months | n = 37 | n = 21 | n = 6 | n = 95 | ||||
| ESAC score | 33.62 | (8.93) | 26.00 | (5.71) | 15.00 | (2.61) | 11.19 | (4.26) |
| ADOS, n (%) | 37 | (100) | 11 | (52) | 6 | (100) | 16 | (17) |
| SA CSS | 6.57 | (2.05) | 2.82 | (1.72) | 4.50 | (2.07) | 1.94 | (1.39) |
| RRB CSS | 6.03 | (2.54) | 3.45 | (2.42) | 5.83 | (2.99) | 2.38 | (2.13) |
| Total CSS | 6.46 | (2.28) | 2.36 | (1.12) | 4.33 | (2.07) | 1.75 | (1.18) |
| MSEL, n (%) | 35 | (95) | 20 | (95) | 6 | (100) | 60 | (63) |
| VR | 43.06 | (13.62) | 50.75 | (11.03) | 56.00 | (19.39) | 60.07 | (11.22) |
| FM | 41.40 | (12.30) | 48.50 | (10.92) | 46.00 | (16.08) | 54.12 | (10.74) |
| RL | 33.34 | (14.13) | 49.10 | (9.77) | 50.83 | (17.88) | 56.65 | (11.91) |
| EL | 31.40 | (10.44) | 45.40 | (10.55) | 49.83 | (20.70) | 52.93 | (13.43) |
| ELC | 77.00 | (18.29) | 97.53 | (15.45) | 102.67 | (30.45) | 111.87 | (18.23) |
| 18–23 months | n = 58 | n = 18 | n = 9 | n = 102 | ||||
| ESAC score | 33.83 | (9.27) | 24.00 | (7.64) | 13.67 | (3.16) | 9.71 | (4.01) |
| ADOS, n (%) | 58 | (100) | 9 | (50) | 9 | (100) | 21 | (21) |
| SA CSS | 6.67 | (1.98) | 2.89 | (1.17) | 5.67 | (1.94) | 2.33 | (1.43) |
| RRB CSS | 6.57 | (2.39) | 4.33 | (1.94) | 6.89 | (1.45) | 2.48 | (2.18) |
| Total CSS | 6.66 | (1.88) | 2.33 | (1.00) | 5.67 | (2.00) | 1.90 | (1.00) |
| MSEL, n (%) | 56 | (97) | 18 | (100) | 9 | (100) | 85 | (83) |
| VR | 39.55 | (13.45) | 48.44 | (9.84) | 48.56 | (17.05) | 58.93 | (11.57) |
| FM | 39.98 | (12.93) | 46.06 | (9.97) | 42.78 | (7.97) | 53.60 | (11.00) |
| RL | 31.25 | (12.93) | 38.22 | (15.06) | 42.33 | (13.26) | 55.74 | (10.29) |
| EL | 33.32 | (13.38) | 38.83 | (12.68) | 41.11 | (11.20) | 51.19 | (13.34) |
| ELC | 74.91 | (19.43) | 86.94 | (18.83) | 88.22 | (19.20) | 109.59 | (16.63) |
| 24–36 months | n = 96 | n = 22 | n = 13 | n = 118 | ||||
| ESAC score | 33.97 | (9.30) | 27.91 | (7.75) | 14.15 | (3.05) | 7.36 | (4.71) |
| ADOS, n (%) | 96 | (100) | 14 | (64) | 13 | (100) | 36 | (31) |
| SA CSS | 6.30 | (2.17) | 2.00 | (1.04) | 5.77 | (1.74) | 2.14 | (1.46) |
| RRB CSS | 7.54 | (2.15) | 4.43 | (2.20) | 8.15 | (1.41) | 3.81 | (2.20) |
| Total CSS | 6.65 | (2.13) | 2.07 | (1.33) | 6.38 | (1.45) | 1.89 | (1.14) |
| MSEL, n (%) | 93 | (97) | 22 | (100) | 13 | (100) | 94 | (80) |
| VR | 35.80 | (12.93) | 40.64 | (15.23) | 42.62 | (16.46) | 57.32 | (12.51) |
| FM | 34.06 | (12.18) | 38.55 | (12.39) | 38.69 | (17.99) | 51.72 | (12.64) |
| RL | 28.77 | (11.89) | 39.86 | (15.04) | 42.23 | (15.36) | 53.22 | (11.46) |
| EL | 30.30 | (11.71) | 37.64 | (14.27) | 43.69 | (13.51) | 50.20 | (13.22) |
| ELC | 68.41 | (18.16) | 81.05 | (21.99) | 84.62 | (26.78) | 106.15 | (19.69) |
SD: standard deviation; ESAC: Early Screening for Autism and Communication Disorders; ADOS: Autism Diagnostic Observation Schedule; SA: Social Affect; CSS: calibrated severity score; RRB: restrictive/repetitive behavior; MSEL: Mullen Scales of Early Learning; VR: Visual Recognition; FM: Fine Motor; RL: Receptive Language; EL: Expressive Language; ELC: Early Learning Composite.
MSEL T scores based on M = 50, SD = 10; ELC based on M = 100 and SD = 15.
Internal consistency and test–retest reliability
Coefficient alphas for the final ESAC item set ranged from 0.92 to 0.95, indicating good internal consistency. For the subsample with greater than or equal to two administrations (n = 111), we observed a correlation of 0.82 between the first and second ESAC total with an average of 8.37 months (SD = 3.49) between administrations, indicating a strong association between scores over time.
Discussion
This study examined ESAC results from preliminary field-testing of children with elevated risk for ASD recruited from primary care and referred samples. ROC curves were examined to determine the items that best discriminated children with and without ASD. AUCs exceeded 0.90 for total scores of the best 30 items, and preliminary cutoffs were established for the 30-item ESAC across age groups with promising sensitivity and specificity. A cutoff score was recommended at 20 for 12–17 months and 18 for 18–36 months. Results provide initial support for validity of the ESAC as an autism screener for children 12–36 months with elevated risk, accurately identifying over 80% of children with ASD in this sample. The ESAC had strong psychometric properties in the AAP-recommended screening age range of 18–24 months, as well as adequate sensitivity down to 12 months, which is particularly promising given challenges of accurately screening for ASD under 24 months.
The ESAC was more accurate in a sample with a higher developmental level by 18 points on the MSEL Early Learning Composite (ELC), than children with ASD identified through community referrals as well as universal screening with the M-CHAT (see Micheletti et al., 2019). Likewise, MSEL average T scores of children with ASD were higher by at least 5 points than samples detected with the M-CHAT (Kleinman et al., 2008) and M-CHAT/F (Robins et al., 2014). The average MSEL ELC for children detected with ASD in this field-test sample was 22 points higher than the average of the Bayley Mental Development Index (Bayley, 1993) for children with ASD in the initial study of the M-CHAT in primary care (Robins et al., 2001). Furthermore, Kleinman and colleagues found that their primary care and referred ASD groups had similar scores on the MSEL, suggesting that even in a primary care group, the M-CHAT is biased toward identifying children with lower developmental levels. The percentage of children with ASD with average or above-average intellectual abilities at 8 years of age has increased over time to more than half (Maenner et al., 2020), highlighting the importance of detecting children with higher developmental levels.
This study extends the findings of Wetherby et al. (2008) and Pierce et al. (2011) by documenting the accuracy of the ESAC as an autism-specific parent-report screener to follow up a broadband screen. Over 75% of our sample were recruited using the ITC in primary care at two sites (FSU and UCSD) and were younger, on average, at age of screening at these sites. Sensitivity and specificity of the ESAC were better than psychometric properties reported in early validation studies of the M-CHAT (Kleinman et al., 2008; Robins et al., 2001), which included children screened in primary care and those referred through EI. Importantly, our sample included a large number of children screened under 24 months. Most M-CHATs in Robins and colleagues’ (2001) early study were administered to children at 18- or 24-month well-child visits, though they did not report the mean age at screening. Furthermore, only children who were detected by their M-CHAT score or by the primary care provider concerns were invited for a diagnostic evaluation; therefore, the actual sensitivity is likely to be lower than what was reported, given that children who screened negative and did not have provider concerns were assumed to not have ASD in specificity calculations. Studies of wide-scale population use of the M-CHAT with more systematic follow-up of screen negatives have yielded poorer psychometric properties (Carbone et al., 2020; Guthrie et al., 2019). A large-scale study of the ESAC with an independent sample recruited from primary care is underway and is critical for further validation.
Our findings highlight the importance of examining performance across different age ranges from 12 to 36 months to retain adequate psychometric properties while still detecting children who present with “subtler” differences in social communication and RRBs (Sturner, Howard, Bergmann, Morrel, et al., 2017; Sturner, Howard, Bergmann, Stewart, & Afarian, 2017). Consistent with research on other screeners, the ESAC’s PPV increased with age. However, it was encouraging that the ESAC maintained strong psychometric properties across the age ranges considering the greater amount of time between screening and diagnosis in the youngest screening group. Interestingly, among the continuous items, some cutoff scores were observed to change with age and some did not. Different cutoffs were needed for early developing social communication skills, including gestures, play with objects, and pretend play with toys. Conversely, no adjustments to cutoff scores were needed for items that tapped into features specific to ASD like sensory responses, repetitive behaviors, insistence on sameness, and intense interests.
Finally, this study demonstrates the accuracy of parent report to screen young children. Parent concern, by itself, may be less accurate for children at younger ages (Wetherby et al., 2008). Retrospective and prospective studies show that about 75% of parents of children with ASD have concerns by 24 months of age, 50% at 18 months, and 30% at 12 months (Chawarska, Paul et al., 2007; Wetherby et al., 2008). Results of this study support that parents are accurate at reporting what their child does and does not do, whether they are concerned or not. Furthermore, for the subsample with two ESACs, a strong association was observed between total scores, supporting the stability of ESAC scores over time.
Strengths, limitations, and future directions
Strengths of this study include a large sample size of children recruited from five sites and prospectively followed and evaluated using gold-standard measures for ASD. Our sample was large enough to examine three separate age intervals, with ample sampling of positive and negative screens and a variety of ascertainment methods. Despite these strengths, our field-test trial has limitations. Because our sample had elevated risk for ASD, the reported accuracy of the ESAC may be higher than in a low-risk primary care sample. Group differences in developmental level may have contributed to the high discrimination among groups. The TD group had above-average developmental skills, and despite being higher than previous screening studies, the developmental level of the ASD group was lower than the DD and TD groups. Thus, it will be important for follow-up studies to examine contributions of developmental differences to the ESAC’s accuracy and specifically test the ESAC’s predictive utility in children with low and high developmental level.
Demographic information was not available for a substantial portion of children and this sample may be limited in racial, ethnic, and socioeconomic diversity. In addition, the full diagnostic battery was not completed for all children in the NS group. Given the small sample sizes at some sites, the performance of the ESAC could not be examined separately across sites and ascertainment method. Finally, in order to increase sample size and resulting power, multiple ESACs (if available) from the same child were included across (but not within) age groups. While assumptions of independence were not violated, in the older age groups, about a third of the ESACs were repeats, which may bias its accuracy. However, given AAP recommendation that children be screened for ASD at 18- and 24-month well-child visits, it is important that accuracy be established for initial and repeated administrations of screening tools.
Future directions include examining the final 30-item set of the ESAC across new independent samples of children at 12–36 months and the performance of the ESAC as an autism screener in a low-risk primary care sample. Further examination of psychometric features is needed, including replication with new samples, test–retest reliability, and a factor analysis to further establish validity. Research is needed to study parent-report screeners with follow-up observational measures (e.g., Dow et al., 2020) to accelerate referrals for diagnosis and eligibility for early intervention.
Conclusion
Results of this study provide preliminary support for the validity of the ESAC as an autism screener for children 12–36 months of age with elevated risk and add to research documenting the accuracy of parent report to screen young children. Use of a parent-report screening tool like the ESAC minimizes the time required of healthcare providers, maximizes the role of the family, and provides reasonably accurate information about whether to refer for further evaluation. These findings offer promise for this new screener to more accurately identify more children at younger ages and contribute to reducing the age of diagnosis and access to intervention.
Footnotes
Declaration of conflicting interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: A.M.W. is co-author of the Communication and Symbolic Behavior Scales and receives royalties but not from this study. Catherine Lord is author of the Autism Diagnostic Observation Schedule–Second Edition (ADOS-2). C.L. and W.G. are authors of the ADOS Toddler Module (ADOS-T). They receive royalties from use of the ADOS-2/ADOS-T, but not from this study. The remaining authors have no financial relationships relevant to this article to disclose.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by the Eunice Kennedy Shriver National Institute of Child Health & Human Development grants RO1HD078410 and RO1HD065272 (PI: A.M.W.), the National Institute on Deafness and Other Communication Disorders grant R01DC007462 (PI: A.M.W.), and the Centers for Disease Control and Prevention Cooperative Agreement U01DD000304 (PI: A.M.W.), in part by National Institute of Mental Health completed as part of protocol 06-M-0065, NCT00271622 under ZIAMH002868. The content is solely the responsibility of the authors and does not necessarily represent the official views of these federal agencies.
ORCID iDs: Amy M Wetherby 
https://orcid.org/0000-0001-9072-8537
Jessica L Hooker
https://orcid.org/0000-0002-6219-5005
Catherine Lord
https://orcid.org/0000-0001-5633-1253
References
- American Psychiatric Association. (2013). The diagnostic and statistical manual of mental disorders, fifth edition: DSM-5. American Psychiatric Publishing. [Google Scholar]
- Bacon E. C., Courchesne E., Barnes C. C., Cha D., Pence S., Schreibman L., Stahmer A. C., Pierce K. (2018). Rethinking the idea of late autism spectrum disorder onset. Development and Psychopathology, 30(2), 553–569. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bayley N. (1993). The Bayley Scales of infant development (2nd ed.). Harcourt Brace. [Google Scholar]
- Carbone P. S., Campbell K., Wilkes J., Stoddard G. J., Huynh K., Young P. C., Gabrielsen T. P. (2020). Primary care autism screening and later autism diagnosis. Pediatrics, 146(2), Article e20192314. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chawarska K., Paul R., Klin A., Hannigen S., Dichtel L. E., Volkmar F. (2007). Parental recognition of developmental problems in toddlers with autism spectrum disorders. Journal of Autism and Developmental Disorders, 37(1), 62–72. [DOI] [PubMed] [Google Scholar]
- Chlebowski C., Robins D. L., Barton M. L., Fein D. (2013). Large-scale use of the modified checklist for autism in low-risk toddlers. Pediatrics, 131, e1121–e1127. 10.1542/peds.2012-1525 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dietz C., Swinkels S., van Daalen E., van Engeland H., Buitelaar J. K. (2006). Screening for autistic spectrum disorder in children aged 14-15 months. II: Population screening with the Early Screening of Autistic Traits Questionnaire (ESAT). Design and general findings. Journal of Autism and Developmental Disorders, 36, 713–722. 10.1007/s10803-006-0114-1 [DOI] [PubMed] [Google Scholar]
- Dow D., Day T. N., Kutta T. J., Nottke C., Wetherby A. M. (2020). Screening for autism spectrum disorder in a naturalistic home setting using the Systematic Observation of Red Flags (SORF) at 18-24 months. Autism Research, 13(1), 122–133. doi.org/10.1002/aur.2226 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dow D., Guthrie W., Stronach S. T., Wetherby A. M. (2017). Psychometric analysis of the Systematic Observation of Red Flags for autism spectrum disorder in toddlers. Autism, 21(3), 301–309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elison J. T., Wolff J. J., Reznick J. S., Botteron K. N., Estes A. M., Gu H., Hazlett H. C., Meadows A., Paterson S. J., Zwaigenbaum L., Piven J. & Infant Brain Imaging Study Network. (2014). Repetitive behavior in 12-month-olds later classified with autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 53(11), 1216–1224. 10.1016/j.jaac.2014.08.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Esler A. N., Bal V. H., Guthrie W., Wetherby A., Weismer S. E., Lord C. (2015). The Autism Diagnostic Observation Schedule, Toddler module: Standardized severity scores. Journal of Autism and Developmental Disorders, 45(9), 2704–2720. 10.1007/s10803-015-2432-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gotham K., Pickles A., Lord C. (2009). Standardizing ADOS scores for a measure of severity in autism spectrum disorders. Journal of Autism and Developmental Disorders, 39(5), 693–705. 10.1007/s10803-008-0674-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guthrie W., Swineford L. B., Nottke C., Wetherby A. M. (2013). Early diagnosis of autism spectrum disorder: Stability and change in clinical diagnosis and symptom presentation. Journal of Child Psychology and Psychiatry, 54, 582–590. 10.1111/jcpp.12008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guthrie W., Wallis K., Bennett A., Brooks E., Dudley J., Gerdes M., Pandey J., Levy S., Schultz R. T., Miller J. S. (2019). Accuracy of autism screening in a large pediatric network. Pediatrics, 144(4), Article e20183963. [DOI] [PubMed] [Google Scholar]
- Hus V., Gotham K., Lord C. (2014). Standardizing ADOS domain scores: Separating severity of social affect and restricted and repetitive behaviors. Journal of Autism and Developmental Disorders, 44(10), 2400–2412. 10.1007/s10803-012-1719-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hyman S. L., Levy S. E., Myers S. M. & AAP Council on Children with Disabilities, Section on Developmental and Behavioral Pediatrics. (2020). Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics, 145(1), Article e20193447. 10.1542/peds.2019-3447 [DOI] [PubMed] [Google Scholar]
- Johnson C. P., Myers S. M. (2007). Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120, 1183–1215. 10.1542/peds.2007-2361 [DOI] [PubMed] [Google Scholar]
- Kim S. H., Lord C. (2010). Restricted and repetitive behaviors in toddlers and preschoolers with autism spectrum disorders based on the Autism Diagnostic Observation Schedule (ADOS). Autism Research, 3, 162–173. 10.1002/aur.142 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kleinman J. M., Robins D. L., Ventola P. E., Pandey J., Boorstein H. C., Esser E. L., Wilson L. B., Rosenthal M. A., Sutera S., Verbalis A. D., Bartons M., Hodgson S., Green J., Dumont-Mathieu T., Volkmar F., Chawarska K., Klin A., Fein D. (2008). The Modified Checklist for Autism in Toddlers: A follow-up study investigating the early detection of autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, 827–839. 10.1007/s10803-007-0450-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lord C., Luyster R., Gotham K., Guthrie W. (2012). Autism Diagnostic Observation Schedule—Toddler module. Western Psychological Services. [Google Scholar]
- Lord C., Risi S., Lambrecht L., Cook E. H., Leventhal B. L., Dilavore P. C., Pickles A., Rutter M. (2000). The Autism Diagnostic Observation Schedule—Generic : A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205–223. [PubMed] [Google Scholar]
- Lord C., Rutter M., DiLavore P., Risi S., Gotham K., Bishop S. (2012). Autism Diagnostic Observation Schedule, Second Edition: ADOS-2. Western Psychological Services. [Google Scholar]
- Maenner M. J., Shaw K. A., Baio J., Washington A., Patrick M., DiRienzo M., Christensen D. L., Wiggins L. D., Pettygrove S., Andrews J. G., Lopez M., Hudson A., Baroud T., Scwenk Y., White T., Rosenberg C. R., Lee L., Harrington R. A., Huston M., . . . Dietz P. (2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 69(4), 1–23. 10.15585/mmwr.ss6706a1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Micheletti M., McCracken C., Constantino J. N., Mandell D., Jones W., Klin A. (2019). Research review: Outcomes of 24- to 36-month-old children with autism spectrum disorder vary by ascertainment strategy: A systematic review and meta-analysis. Journal of Child Psychology and Psychiatry, 61(1), 4–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mullen E. (1995). Mullen Scales of Early Learning. American Guidance Service. [Google Scholar]
- Ozonoff S., Young G. S., Landa R. J., Brian J., Bryson S., Charman T., Chawarska K., Macari S. L., Messigner D., Stone W. L., Zwaigenbaum Z., Iosif A. M. (2015). Diagnostic stability in young children at risk for autism spectrum disorder: A baby siblings research consortium study. Journal of Child Psychology and Psychiatry, 56, 988–998. 10.1111/jcpp.12421 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pandey J., Verbalis A., Robins D. L., Boorstein H., Klin A. M. I., Babitz T., Chawarska K., Volkmar F., Green J., Barton M., Fein D. (2008). Screening for autism in older and younger toddlers with the Modified Checklist for Autism in Toddlers. Autism, 12, 513–535. 10.1177/1362361308094503 [DOI] [PubMed] [Google Scholar]
- Pierce K., Carter C., Weinfeld M., Desmond J., Hazin R., Bjork R., Gallagher N. (2011). Detecting, studying, and treating autism early: The one-year well-baby check-up approach. The Journal of Pediatrics, 159(3), 458–465. 10.1016/j.jpeds.2011.02.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pierce K., Gazestani V. H., Bacon E., Barnes C. C., Cha D., Nalabolu S., Lopez L., Moore A., Pence-Stophaeros S., Courchesne E. (2019). Evaluation of the diagnostic stability of the early autism spectrum disorder phenotype in the general population starting at 12 months. JAMA Pediatrics, 173(6), 578–587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reznick J. S., Baranek G. T., Reavis S., Watson L. R., Crais E. R. (2007). A parent-report instrument for identifying one-year-olds at risk for an eventual diagnosis of autism: The first year inventory. Journal of Autism and Developmental Disorders, 37, 1691–1710. 10.1007/s10803-006-0303-y [DOI] [PubMed] [Google Scholar]
- Richler J., Bishop S. L., Kleinke J. R., Lord C. (2007). Restricted and repetitive behaviors in young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 37, 73–85. 10.1007/s10803-006-0332-6 [DOI] [PubMed] [Google Scholar]
- Robins D. L., Casagrande K., Barton M., Chen C.-M. A., Dumont-Mathieu T., Fein D. (2014). Validation of the Modified Checklist for Autism in Toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics, 133(1), 37–45. 10.1542/peds.2013-1813 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robins D. L., Fein D., Barton M. L., Green J. A. (2001). The Modified Checklist for Autism in Toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disorders, 31, 131–144. [DOI] [PubMed] [Google Scholar]
- Salisbury L. A., Nyce J. D., Hannum C. D., Sheldrick R. C., Perrin E. C. (2018). Sensitivity and specificity of 2 autism screeners among referred children between 16 and 48 months of age. Journal of Developmental & Behavioral Pediatrics, 39(3), 254–258. [DOI] [PubMed] [Google Scholar]
- Smith N. J., Sheldrick R. C., Perrin E. C. (2013). An abbreviated screening instrument for autism spectrum disorders. Infant Mental Health Journal, 34(2), 149–155. [Google Scholar]
- Stenberg N., Bresnahan M., Gunnes N., Hirtz D., Hornig M., Lie K. K., Lipkin W. I., Lord C., Magnus P., Reichborn-Kjennerud T., Schjølberg S., Surén P., Susser E., Svendsen B. K., von Tetzchner S., Øyen A., Stoltenberg C. (2014). Identifying children with autism spectrum disorder at 18 months in a general population sample. Paediatric and Perinatal Epidemiology, 28(3), 255–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stenberg N., Schjolberg S., Shic F., Volkmar F., Oyen A. S., Bresnahan M., Svendsen B. K., von Tetzchner S., Thronæs N. T., Macari S., Cicchetti D. V., Chawarska K., Suren P., Øien R. A. (2021). Functional outcomes of children identified early in the developmental period as at risk for ASD utilizing the The Norwegian Mother, Father and Child Cohort Study (MoBa). Journal of Autism and Developmental Disorders, 51(3), 922–932. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stronach S. T., Wetherby A. M. (2017). Observed and parent-report measures of social communication in toddlers with and without autism spectrum disorder across race/ethnicity. American Journal of Speech-Language Pathology, 26(2), 355–368. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sturner R., Howard B., Bergmann P., Morrel T., Landa R., Walton K., Marks D. (2017). Accurate autism screening at the 18-month well-child visit requires different strategies than at 24 months. Journal of Autism and Developmental Disorders, 47(10), 3296–3310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sturner R., Howard B., Bergmann P., Stewart L., Afarian T. E. (2017). Comparison of autism screening in younger and older toddlers. Journal of Autism and Developmental Disorders, 47, 3180–3188. 10.1007/s10803-017-3230-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Toh T. H., Tan V. W., Lau P. S., Kiyu A. (2018). Accuracy of Modified Checklist for Autism in Toddlers (M-CHAT) in detecting autism and other developmental disorders in community clinics. Journal of Autism and Developmental Disorders, 48(1), 28–35. [DOI] [PubMed] [Google Scholar]
- Turner-Brown L. M., Baranek G. T., Reznick J. S., Watson L. R., Crais E. R. (2013). The first year inventory: A longitudinal follow-up of 12-month-old to 3-year-old children. Autism, 17, 527–540. 10.1177/1362361312439633 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watt N., Wetherby A. M., Barber A., Morgan L. (2008). Repetitive and stereotyped behaviors in children with autism spectrum disorders in the second year of life. Journal of Autism and Developmental Disorders, 38, 1518–1533. 10.1007/s10803-007-0532-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wetherby A. M., Brosnan-Maddox S., Peace V., Newton L. (2008). Validation of the Infant–Toddler Checklist as a broadband screener for autism spectrum disorders from 9 to 24 months of age. Autism, 12, 487–511. 10.1177/1362361308094501 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wetherby A. M., Prizant B. M. (2002). Communication and Symbolic Behavior Scales: Developmental profile. Paul H. Brookes. [Google Scholar]
- Wolff J. J., Botteron K. N., Dager S. R., Elison J. T., Estes A. M., Gu H., Hazlett H. C., Pandey J., Paterson S. J., Schultz R. T., Zwaigenbaum L., Piven J. (2014). Longitudinal patterns of repetitive behavior in toddlers with autism. Journal of Child Psychology and Psychiatry and Allied Disciplines, 55(8), 945–953. 10.1111/jcpp.12207 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yuen T., Penner M., Carter M. T., Szatmari P., Ungar W. J. (2018). Assessing the accuracy of the Modified Checklist for Autism in Toddlers: A systematic review and meta-analysis. Developmental Medicine & Child Neurology, 60(11), 1093–1100. [DOI] [PubMed] [Google Scholar]
- Zwaigenbaum L., Bauman M. L., Fein D., Pierce K., Buie T., Davis P. A., Newschaffer C., Robins D. L., Wetherby A., Choueiri R., Kasari C., Stone W. L., Yirmiya N., Estes A., Hansen R. L., McPartland J. C., Natowicz M. R., Carter A., Granpeesheh D., . . . Wagner S. (2015). Early screening of autism spectrum disorder: Recommendations for practice and research. Pediatrics, 136, S41–S59. 10.1542/peds.2014-3667C [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zwaigenbaum L., Bauman M. L., Stone W. L., Yirmiya N., Estes A., Hansen R., McPartland J. C., Natowicz M. R., Choueiri R., Fein D., Kasari C., Pierce K., Buie T., Carter A., Davis P. A., Granpeesheh D., Mailloux Z., Newschaffer C., Robins D., . . . Wetherby A. M. (2015). Early identification of autism spectrum disorder: Recommendations for practice and research. Pediatrics, 136, S10–S40. 10.1542/peds.2014-3667C [DOI] [PMC free article] [PubMed] [Google Scholar]