Abstract
Objective
Parents rely on pediatricians to monitor their child’s development. The American Academy of Pediatrics recommends routine developmental screening with both broadband and autism-specific instruments at specified ages. If broadband screeners can detect autism risk, this might minimize the burden of administering autism-specific screens to all children. The current study examines the ability of the Ages and Stages Questionnaire-Third Edition (ASQ-3) to identify children at risk for autism. We looked at ASQ-3 scores of children who screen positive on the Modified Checklist for Autism in Toddlers-Revised, children who continue to screen positive on the M-CHAT-R follow-up interview, and children diagnosed with ASD.
Methods
2848 toddlers, aged 16–30 months, were screened with the ASQ-3 and M-CHAT-R across 20 pediatric sites. Children who screened positive on the M-CHAT-R and its follow-up interview were offered a diagnostic evaluation.
Results
Using the “monitor and/or fail” cutoff on any domain, the ASQ-3 identified 87% of the children who screened positive on the M-CHAT-R with follow-up and 95% (20/21) of those diagnosed with an ASD. “Monitor and/or Fail” on the Communication domain alone also identified 95% of the diagnosed children.
Conclusions
Scores below the “monitor” cutoff on the Communication domain of the ASQ-3 can indicate initial concern requiring autism-specific follow-up. If these results are confirmed with a sample large enough to separately examine toddlers of different ages and different cultural backgrounds, it may be feasible to implement a two-stage screening strategy, with autism specific screening reserved for those who are positive on a broad band screen.
Key Terms: Developmental screening, Autism Spectrum Disorder (ASD), Pediatrics, Modified Checklist for Autism in Toddlers – Revised (M-CHAT-R/F), Ages and Stages Questionnaire – Third Edition (ASQ-3)
Introduction
Recent studies have highlighted the importance of early intervention for children with developmental disorders, such as Autism Spectrum Disorder (ASD; 1, 2). By providing early intervention services to children with ASD, there is a greater chance for improved social, communicative, adaptive and cognitive outcomes (3, 4). The prevalence of ASD has risen to one in 68 children (5), and the timely identification of these children has never been more important.. On average, children in the United States are diagnosed with ASD between three and six years of age (6, 7, 8). However, parents frequently report their first developmental concerns around 18 months (9). Systematic developmental screening helps to close this gap, leading to reliable diagnoses closer to age two (10, 11,12).
Parents rely mainly on their primary care provider to monitor their child’s development and to detect the need for further assessment or intervention; thus, it is critical that pediatricians have the most effective tools to screen for possible delays. Glascoe (13) suggests that early detection rates for developmental disorders have been low overall due to an overreliance on clinical surveillance alone. She suggests the need for more sensitive means of detection within primary care, including standardized screening, which has been shown to facilitate timely referral to specialists (14).
Routine developmental screening with a validated instrument leads to improved outcomes for those children who are identified and referred for evaluation (15). Child development is dynamic and any single source of information may be incomplete; therefore, regular and repeated screening combined with developmental surveillance will optimize the detection of delays (13, 16). Since 2006, the American Academy of Pediatrics (AAP) has recommended developmental screening, using a validated screening tool, at specific well- child visits (9-, 18-, and 24- or 30-months), as well as ASD-specific screening at 18- and 24-month well-child visits (16).
There are many reported barriers to effective screening by primary care professionals including time, practitioner knowledge, costs, and practice implementation constraints (17), with the predominant concern being time. Primary care practitioners most often use broadband developmental screening tools, as opposed to autism specific screening measures, because they are general, quick, easy to use, readily interpretable and detect a range of disorders (18). Therefore, if the broad screeners could be shown to be sensitive to autism, primary care providers might be able to save time by administering general, broadband developmental screening measures before narrowband screens, such as instruments for detecting autism (19,13), With this procedure, broadband screening could be used as a first level screen and autism-specific screening as a second level screen for those with autism risk indicated on the broadband screening.
One autism-specific measure commonly used in the United States is the Modified Checklist for Autism in Toddlers, or M-CHAT. The Checklist for Autism in Toddlers (CHAT), developed in the UK, was the first autism-specific screening measure (21). Robins and colleagues adapted it for use in the United States in 2001, producing the M-CHAT (22), and its revision, the M-CHAT-R (23). The M-CHAT-R was recently validated as a two-stage, level one ASD screening tool for use at 18- and 24-month well-child visits. In the first stage, parents complete the M-CHAT-R at a well child visit. A score of 0–2 items failed indicates low risk for autism. The M-CHAT-R Follow-Up Interview (M-CHAT-R/F) is used when the parent’s responses indicate medium risk, with a score of 3–7. If the parent’s responses indicate high risk, with a score of 8 or higher, it is acceptable to bypass the Follow-Up and refer immediately for diagnostic evaluation and eligibility evaluation for early intervention (23). The M-CHAT-R/F effectively identifies toddlers who should receive a more thorough assessment for possible ASD; physicians using the screener plus follow-up interview can be confident that most screen-positive cases warrant evaluation and referral for further screening and assessment; screen positive children not found to have ASD are usually found to have another developmental issue warranting attention (23). Widespread implementation of universal screening can lower the average age of ASD diagnosis by two years, increasing time available for early intervention (24).
The question remains, then, whether using an autism specific instrument such as the M-CHAT-R as a second level screen, following a positive screen on a broad instrument, would have sufficient sensitivity for detecting autism. There are very little data on this question. Two widely used, validated broad developmental screeners are the Parents Evaluation of Developmental Status (PEDS;25), and the Ages and Stages Questionnaire (13). One study (4) examined children identified by the PEDS compared to those identified by the M-CHAT. This revealed little overlap between the two screeners, with fewer than one-third of children identified by the M-CHAT also screening positive with predictive concerns on the PEDS. Glascoe and colleagues (26) also compared children screening positive on PEDS with those screening positive on MCHAT: 34% of children who screened positive on the PEDS screened negative on the M-CHAT, but these potential over-referrals could be reduced by 70% by requiring multiple specific concerns on the PEDS. However, neither of these studies on the PEDS conducted formal ASD evaluations. A recent study by Wiggins and colleagues (27) compared the agreement of the PEDS and M-CHAT screeners with the results of a comprehensive clinical autism evaluation. While the M-CHAT demonstrated higher agreement with the clinical diagnosis, all children diagnosed with an ASD demonstrated at least one domain of concern on the PEDS, indicating good sensitivity for ASD if a fairly low threshold is used. Using this threshold, the PEDS identified over twice as many children as the M-CHAT, and would need to be followed by an autism-specific screen to reduce autism over-referrals. The authors concluded that both broad developmental and autism-specific screeners should be regularly administered at well-child visits.
Another such general developmental screening instrument is the Ages and Stages Questionnaire (ASQ). The ASQ is currently in its 3rd version (28) and is nationally recognized as an effective broad developmental screening measure (13). The ASQ-3 has strong psychometric properties when it is used to detect developmental delays, with .85 specificity and .86 sensitivity across age groups (28). However, there is little or no data on the ASQ-3’s sensitivity as a screening instrument for specific disorders, such as ASD. The current study examines the sensitivity of the ASQ-3 domain scores in identifying children detected by the M-CHAT-R screening and follow-up and then receiving an ASD diagnosis. The study also examines each of the five domains of the ASQ-3 to see which domain(s) is most useful in detecting the autism cases. Based on the core symptoms of autism and the nature of early parent concerns, we hypothesized that the Personal/Social and Communication domains would be most sensitive.
Methods
Research Design
Children at risk for autism were detected with the use of the M-CHAT-R. This sample was then followed up with the M-CHAT-R interview, and children continuing to screen positive were evaluated. The ASQ-3 was filled out at the same time as the M-CHAT-R. ASQ-3 scores (pass, monitor, fail) on each domain were examined for: children screening positive on the M-CHAT-R, children continuing to screen positive on the M-CHAT-R/F, and children evaluated and receiving a diagnosis of ASD, in order to estimate sensitivity of the ASQ-3 for children at each stage of autism screening/evaluation.
Participants
Screening data were collected from twenty pediatricians across Connecticut over the course of four years (2009–2013) as part of the IRB approved study on early detection of ASD at the University of Connecticut. A sample of 2848 children was screened during their 18- or 24-month well-child checkups (1458 male, 1390 female). Children ranged in age from 16 months 1 day to 30 months 25 days (Mean =20.44 months, SD=2.9 months). A group of 207 Spanish speakers were included, using the Spanish language M-CHAT-R and ASQ-3.
Physicians were recruited to participate in the study via widespread mailings across Connecticut, which explained the benefits of using the M-CHAT-R as an autism specific screening tool and the current recommendation by the American Academy of Pediatrics for using such a tool for screening. The ASQ-3 was an optional addition to the M-CHAT-R screening; therefore, only pediatricians who used both the M-CHAT-R and the ASQ-3 were included in the current study (20 offices out of the total 47 offices in the larger study). Further, only children whose parents completed both screening instruments were included in this study (n=2848).
Children were excluded from the study if the M-CHAT-R was not completed within the appropriate age range (16 months to 30 months and 29 days), if the incorrect age-specific form of the ASQ-3 was used, or if the ASQ-3 was incomplete. Children were also excluded from the study if the child’s legal guardian did not complete the M-CHAT-R or ASQ-3, or if the family did not speak either English or Spanish. Children who had severe motor problems or sensory impairments that would prevent the child from participating in further diagnostic assessments were excluded, as well as children who had been given an ASD diagnosis prior to the screener’s completion.
Measures
Ages and Stages Questionnaire – Third Edition (ASQ-3)
The ASQ-3 has twenty-one age-specific developmental questionnaires starting at one month and ending at five years six months of age. For this study we used the 16, 20, 22, 24, 27, and 30-month ASQ-3 forms. There are five domains: Fine Motor, Gross Motor, Communication, Problem-Solving, and Personal/Social; each domain contains six questions that can be answered with a yes (10 points), sometimes (5 points), or not yet (0 points), as well as nine open-ended questions. The ASQ-3 is written at a fourth-to-sixth grade level and overall, it takes ten to fifteen minutes to complete. Based on the total score, each domain is coded as “Fail”, “Monitor”, or “Pass”. If the score is below a specified cutoff, then the child has failed that domain (i.e., screened positive), and further assessment is indicated. If the score is within a specified range of the cutoff, then there are concerns and it is advised that learning activities are provided and the child’s development is monitored. If the child’s score is well above the cutoff then the child passes; the child’s development appears to be on schedule (28).
In the current study, the five ASQ-3 domain scores (Gross Motor, Fine Motor, Communication, Personal-Social, and Problem Solving) were each coded as: 0= pass, 1= monitor, or 2= failed (24). This coding system is standard across all versions of the ASQ-3 (the 16-month questionnaire through 30 months); therefore, the levels of risk could be compared between the M-CHAT-R (see below) and the ASQ-3 results.
Modified Checklist for Autism in Toddlers-Revised (M-CHAT-R)
The M-CHAT-R (23) is an autism-specific screener composed of 20 yes or no items that have been reworded from the original M-CHAT; examples have also been added to clarify items. Further, three of the lowest performing M-CHAT items were dropped in the revision. The parent-completed questionnaire is designed for children aged 16 to 30 months, ideal for use at their 18- and 24-month well-child visit. Scores on the M-CHAT-R determine follow-up: 0–2 failed items are considered a negative screen and no further follow-up is necessary, 3–7 failed items require the M-CHAT-R/F and potentially a referral for an evaluation, 8 or more failed items can use the telephone follow-up to verify results or can bypass the follow-up and be referred directly for evaluation (23). The M-CHAT-R/F is a structured interview based on the items each child failed; it contains additional examples and clarifying questions.
The M-CHAT-R has an option for a pediatrician to “red flag” a child for evaluation if the pediatrician has ASD-specific concerns regardless of the parent’s answers. The pediatrician checks a box at the top of the questionnaire and is later contacted by the research team for follow-up; if the concerns relate to ASD, the child is offered an evaluation.
Procedures
Pediatric office staff gave parents an information sheet describing the study, including all potential follow-up procedures. If they chose to participate, parents were given the M-CHAT-R and the appropriate ASQ-3 form, which they completed at their pediatrician’s office. Participation was voluntary, and parents could withdraw their child from participation at any time. The information sheet explained that with parental consent, the data would be de-identified and included in the study.
The pediatrician’s staff sent the completed questionnaires back to the University of Connecticut, where all measures were evaluated for validity and scored. If a child screened positive on the M-CHAT-R by failing any three items (23), the parents were called to complete the M-CHAT-R/F. Graduate students trained on the structured interview asked parents to elaborate their answers, including specific examples of the behavior in question and how frequently it occurs. The child was considered to have failed the M-CHAT-R/F if he/she failed any two items.
If the child passed the M-CHAT-R or passed the M-CHAT-R/F and the ASQ-3, no further follow-up was conducted. If the child passed the M-CHAT-R and/or the M-CHAT-R/F but failed one or more domains on the ASQ-3, a letter was sent to the pediatrician with a copy of the ASQ-3 suggesting that the child be referred to an early intervention provider. Irrespective of the ASQ-3 result, if the child was “red flagged” on the M-CHAT-R by their pediatrician, failed eight or more M-CHAT-R items, or failed two or more on the M-CHAT-R/F items, the child was offered a free diagnostic and developmental evaluation. This evaluation was completed at the University of Connecticut with a licensed clinical psychologist or a developmental/behavioral pediatrician and a graduate student in clinical psychology.
The developmental and diagnostic evaluation consisted of parent measures (Vineland II (29), Toddler ASD Symptom Interview (unpublished), and a developmental history form), as well as measures completed with the child: the Autism Diagnostic Observation Scale (ADOS; 30) and the Mullen Scales of Early Learning (31). All clinicians and graduate students were research reliable on the diagnostic measures administered. Once the testing was complete, the family was given verbal feedback, which included the screening results, information about the child’s developmental functioning, any appropriate diagnoses, and recommendations. Due to the nature of the evaluation and feedback, researchers could not be blind to the screening data as parents often wanted to discuss the results of screening during the evaluation feedback session. Families were later provided with a full written report.
Data Analysis
Descriptive statistics, including simple counts and percentages, were conducted in order to initially analyze the number of children who screened positive on the ASQ-3 and M-CHAT-R/F. Next, we describe the ASQ-3 scores (and resulting sensitivity) of the children who screened positive on the MCHAT-R, children who continued to screen positive on the follow-up interview, and children who went on to receive an ASD diagnosis.
Results
A total of 2,848 children were screened with both the ASQ and the M-CHAT-R (see Table 1 for demographic information and Table 2 for the full ASQ-3 screening results by domain.) Of this sample, 1,038 children (36% overall) scored in the ‘monitor’ range on one or more domains of the ASQ; of those children, 424 (15% of the whole sample) failed one or more domains. Of the children who failed one or more domains, 218 (51% of those who failed any domain; 7.7% overall) scored in the monitor or failed range on the Communication domain specifically.
Table 1.
Demographic information for all children who were screened.
| Factor | n | % |
|---|---|---|
| Total | 2848 | |
| Age (months) | ||
| Mean (SD) | 20.44 (2.9) | |
| Range | 16-30.83 | |
| Gender | ||
| Male | 1458 | 51.2 |
| Female | 1390 | 48.8 |
| Race/Ethnicity | ||
| White, not Hispanic | 1489 | 52.3 |
| White, Hispanic | 196 | 6.9 |
| Black, not Hispanic | 235 | 8.3 |
| Black, Hispanic | 48 | 1.7 |
| Hispanic (race unknown) | 455 | 16.0 |
| Asian/Pacific Islander | 169 | 5.9 |
| Biracial | 186 | 6.5 |
| Other | 17 | 0.5 |
| Not reported | 53 | 1.9 |
| Maternal Education | ||
| Did not complete HS/GED | 220 | 7.7 |
| HS diploma or GED | 462 | 16.2 |
| Vocational/technical degree | 110 | 3.9 |
| Some college | 657 | 23.1 |
| Bachelor’s degree | 672 | 23.6 |
| Advanced degree | 608 | 21.3 |
| Not reported | 119 | 4.2 |
| Language | ||
| English | 2641 | 92.7 |
| Spanish | 207 | 7.3 |
Table 2.
ASQ-3 results by domain for all 2,848 children who were screened.
| Domain | Monitor | Fail | Monitor OR Fail | Pass |
|---|---|---|---|---|
| Communication | 257 (9%) | 155 (5.4%) | 412 (14.5%) | 2,436 (85.5%) |
| Gross Motor | 185 (6.5%) | 89 (3%) | 274 (9.6%) | 2,574 (90.4%) |
| Fine Motor | 241 (8.5%) | 143 (5%) | 384 (13.5%) | 2,464 (86.5%) |
| Problem Solving | 295 (10.4%) | 153 (5.4%) | 448 (15.7%) | 2,400 (84.3%) |
| Personal-Social | 252 (8.8%) | 111 (3.8%) | 363 (12.7%) | 2,485 (87.3%) |
| One or more domains | 869 (30.5%) | 424 (14.9%) | 1038 (36.4%) | 1,810 (63.6%) |
On the M-CHAT-R, 276 children screened positive (10%; Figure 2) and 2,572 children (90.3% of total sample) passed on the initial M-CHAT-R screen. Of the 276 screen positive children, 132 (48%) failed at least one domain on the ASQ, and 193 (70%) scored in the monitor and/or fail range on at least one domain, suggesting a sensitivity of 70% for the ASQ-3 to detect children screening positive on the M-CHAT. Study personnel were unable to conduct a phone interview for 32 children (1.1% of total sample) due to refusal, an inability to contact, or lack of proficiency in either English or Spanish; these children were excluded from the current study. Parents of 244 children completed the follow-up interview; 166 children (5.8% of total sample) demonstrated initial concerns, but went on to screen negative on the M-CHAT-R/F. Seventy-eight children (30%; 2.7% of total sample) continued to screen positive. Of these 78 children, 59 (76% of the screen positive children) failed at least one domain on the ASQ, and 68 (87%) scored in the monitor and/or failed range on at least one domain, suggesting a sensitivity of .87 for the ASQ-3 to detect children who screened positive on the M-CHAT-R/F. Additionally, 61 (78%) of the 78 children who screened positive on the M-CHAT-R/F scored in the monitor and/or fail range of the ASQ-3 Communication domain specifically. For a summary of the M-CHAT-R screening, see Figure 1.
Figure 2.
ASQ-3 overall screening results on combined domains for those who failed the M-CHAT-R
*ASQ screening results are classified by any one or multiple domains that are noted as “Fail” or “Monitor”. These results do not refer to a specific domain.
Figure 1.
M-CHAT-R Screening Results
Please note: the asterisked sections of Figure 1 are expanded in Figure 2 to reflect the ASQ-3 results at each corresponding point.
Fifty-six children received an evaluation (this includes four children who were “red-flagged” by their pediatrician) and 21 were determined to have an ASD. Of the children diagnosed with an ASD, seventeen (81%) failed one or more ASQ-3 domain and twenty (95%) scored as monitor or fail on at least one domain on the ASQ, suggesting a sensitivity of .95 for the ASQ-3 to detect the children with ASD in the current sample. Examination of Tables 3 and 4 shows that using fail or monitor on Personal-Social OR Communication (the two domains with the highest percent failing) was no more sensitive than using fail or monitor on Communication alone (95%).
Table 3.
ASQ-3 results by domain for the 21 children who received an ASD diagnosis.
| Domain | Monitor | Fail | Monitor OR Fail | Pass |
|---|---|---|---|---|
| Communication | 5 (23%) | 15 (71%) | 20 (95%) | 1 (4%) |
| Gross Motor | 7 (33%) | 3 (14%) | 10 (47%) | 11 (52%) |
| Fine Motor | 7 (33%) | 7 (33%) | 14 (66%) | 7 (33%) |
| Problem Solving | 6 (28%) | 8 (38%) | 14 (66%) | 7 (33%) |
| Personal-Social | 4 (19%) | 11 (52%) | 15 (71%) | 6 (28%) |
| One or more domains | 16 (76%) | 17 (81%) | 20 (95%) | 15 (71%) |
Table 4.
Examination of the ASQ-3 Communication and Personal Social domains only for the 21 children that received an ASD diagnosis.
| 21 Dx: ASD Communication and Personal Social Domain Only | ||
|---|---|---|
| Failed either domain | 17 | 81% |
| Failed BOTH domains | 9 | 43% |
| Failed OR Monitor on either domain | 20 | 95% |
| Failed OR Monitor on BOTH domains | 15 | 71% |
The ASQ’s sensitivity for identifying individuals with ASD detected by the M-CHAT-R is, therefore, calculated to be 81%, using failing any one domain as the cutoff point. However, if the cutoff criteria are expanded to include either failing or having a score in the monitor range, the sensitivity rises to 95% based solely on the Communication domain. Specificity is unable to be calculated due to a lack of information regarding individuals who failed the ASQ-3 but were not evaluated.
Discussion
The initial results of the ASQ-3 screening suggest that approximately one in every three children will initially require monitoring for their development based on scoring in the fail or monitor range on any domain, while 14% will require further screening for autism based on scoring fail or monitor on the Communication Domain. Although alone the ASQ-3 identifies a greater number of children than the M-CHAT-R, if it were used as a part of a two-stage screening procedure, the ASQ-3 could greatly reduce the number of children requiring further screening. Thus the current study suggests a two-stage screening process may be effective for identifying children with ASD. Specifically, the ASQ-3 may be useful as a general level-one screening instrument for the identification of ASD in toddlers. Using fail or monitor on any domain, sensitivity was 70% (unacceptably low) for children screening positive on the M-CHAT-R; this rose to 87% for children screening positive on the M-CHAT-R/F interview, (and 78% using monitor and/or fail cutoff on the Communication domain only). Sensitivity rose to 95% (20 of 21 children) of those going on to receive a diagnosis of ASD, using the monitor or fail cutoff in the Communication domain.
This study demonstrates the importance of using the monitor cutoff point on the ASQ-3. This broadened criterion (monitor cutoff point) correctly identified 95% of children with ASD using the Communication domain, as opposed to the fail cutoff, which identified only 81 percent. However, it is important to note that, using the monitor cutoff, the ASQ-3 identified 1,038 children (36% of the total sample) and therefore is highly nonspecific for ASD concerns. When using only the Communication domain to identify concerns, the ASQ-3 identified 412 children, or about 14% of the entire screened sample. These results highlight the possibility of reducing ASD screening in toddlers through these methods, and it is worth considering and exploring on a larger scale in the future.
These results are similar to those found by Wiggins et al. (27), who compared the agreement between the PEDS and the M-CHAT. Wiggins and colleagues demonstrated that using the PEDS (combined Path A and B) effectively identified children with ASD; however, it lacked specificity and would over-identify children to be evaluated for an ASD. Based on the results of the current study as well as the study by Wiggins et al., we expect that using a two-stage screening model, with PEDS or ASQ-3 as a level one screen, would reduce the number of children needing ASD specific screening.
There were several limitations to the current study. Primarily, the current study did not follow children who scored in the monitor or fail ranges on any ASQ domain but screened negative on the M-CHAT-R. Therefore, we are unable to assess the specificity of the ASQ-3 or its ability to pick up children with ASD who were missed by the M-CHAT-R/F.
The current study also used the general ASQ-3; however, there is a related measure that specifically targets the child’s social and emotional development: the ASQ: Social-Emotional (ASQ: SE) and a second edition due out in 2015. Whereas the ASQ-3 focuses on broad development, the ASQ: SE was designed to pick up children with social-emotional delays, including autism (32). The ASQ: SE examines self-regulation, compliance, communication, adaptive behavior, autonomy, affect, and interpersonal interactions to address symptoms of depression, anxiety, and other antisocial behaviors (33). While these behaviors may also be associated with characteristics of ASD and might be more specific, this screener adds another 10–15 minutes to the screening process; the current study intended to assess the utility of the more generally used ASQ-3 measure.
Consistent with other publications on the M-CHAT-R/F, the current study lost approximately one quarter of the initial screen positives to follow-up (34, 35). Anecdotal evidence suggests that some parents are confused about the systems serving them and indicate that their child has already received an assessment or is already receiving services. These parents may not realize the difference between types of assessments and services. Windham and colleagues (36) also noted that parents may not be ready to accept a potential diagnosis and therefore do not pursue an evaluation. Additionally, parents may not feel that their child’s developmental delays are significant enough to warrant follow-up. The attrition of 32 children in our study represents an unfortunate source of potential bias, as data from these cases were not available for inclusion in the analyses.
It is important to note that universal ASD screening across pediatric care sites continues to be a challenge. Similar to findings from Windham and colleagues (36), in the current study screening was not conducted during sick child visits, so the sample may not represent families who did not attend regular well-child exams, potentially reducing the sample’s heterogeneity. Additionally, research personnel were responsible for scoring the screeners, contacting families, and conducting follow-up interviews (37, 34, 35, 24). Therefore, the current study may lack generalizability and ecological validity within unsupported pediatric practices.
Due to our reliance on office staff, and our efforts to make the study design as simple for pediatric practices as possible, we were unable to counterbalance which measure was completed first. The two screeners were given to the parents together, and they could fill them out in any order they chose; therefore, it is unlikely that either screener was systematically done first. When comparing the Communication domain on the ASQ-3 18-month version to the M-CHAT-R, there is minimal overlap between items. Two items, one item about pointing to wanted or interesting objects, and one regarding understanding instructions were the only similarities. The 24-month ASQ-3 version has no overlap with items on the M-CHAT-R. Thus, with very little overlap in items, the potential impact of completing one measure before the other is minimal.
It is possible that the ASQ may be more sensitive to autism at older or younger ages, but since we only have 21 children with ASD and 7 possible forms of the ASQ, our sample size is too small to examine this effect. However, future research could examine whether the ASQ is more sensitive to ASD at different ages. While the majority of the children diagnosed with ASD completed the 18-month form, four were screened with the 24-month version and the remaining three used forms for different ages. The one child who did not have a monitor/fail result on the ASQ was screened using the 27-month version. Thus, it’s possible the ASQ best identifies concerns for ASD in children at younger ages. The only form of the ASQ having enough children in the sample to examine is the 18-month version and all of those children with an ASD diagnosis screened positive on the ASQ-3.
The current study did not follow children who screened negative on the M-CHAT-R; therefore we cannot say whether the M-CHAT-R missed any children who were later diagnosed with an ASD. It is highly likely that there are missed cases and we cannot know how they would have done on the ASQ. Therefore, it is important to note that our sensitivity on the ASQ is an estimate.
Additionally, as previously mentioned, researchers could not be blind to the screening data because all children who qualified for an evaluation were known to have screened positive on the screening measures. This limitation is inherent to the design of the study and is accounted for using a variety of assessment measures and current DSM diagnostic criteria.
Future Considerations
The results from the current study suggest the utility of using a general screener to identify risk for ASD. However, further research should be conducted with a larger sample size, and across varied settings and demographic characteristics before adopting a two-stage screening procedure for detection of ASD. While our study suggests that the ASQ-3 may be able to detect ASD using a low threshold “monitor” score in the Communication domain, these findings must be verified with a larger sample before they be incorporated into clinical recommendations. Findings by Windham and colleagues (36) suggest that the ASQ is less affected by demographic characteristics than the M-CHAT and support the continued implementation of both general and autism-specific screening measures.
In addition, studies could examine whether the ASQ more effectively identifies ASD at different ages and whether adding the ASQ: SE to the general ASQ-3 makes it more specific for autism. Finally, future studies should examine ASD and developmental screening in pediatric practices without research support, and across broader cultural backgrounds, including translation, validation and refinement of screening instruments so that children from all the diverse populations in this country can have access to the most timely diagnosis and intervention.
Acknowledgments
This study is funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development.
DF is part owner of the M-CHAT-R LLC. The M-CHAT-R is provided free of charge for physician’s use but provides royalties from companies that charge for its use.
Footnotes
Statement of Conflict of Interest
The other authors of the current study certify that no personal, material or financial conflict of interest exists regarding the material discussed in this manuscript.
Contributor Information
Sarah Hardy, Department of Psychology, University of Rhode Island, Kingston, RI
Lauren Haisley, Department of Psychology, University of Connecticut, Storrs, CT
Courtney Manning, Department of Psychology, University of Connecticut, Storrs, CT
Deborah Fein, Department of Psychology, University of Connecticut, Storrs, CT.
References
- 1.Dawson G, Sterling L, Faja S. Autism: Risk factors, risk processes, and outcome. In: de Haan M, Gunnar M, editors. Handbook of developmental social neuroscience. New York, NY, US: Guilford Press; 2009. pp. 435–458. [Google Scholar]
- 2.Reichow B. Translating research to practice. Journal of Autism and Developmental Disorders. 2012;42(6):1153–1155. doi: 10.1007/s10803-012-1537-5. [DOI] [PubMed] [Google Scholar]
- 3.Myers S, Johnson C. American Academy of Pediatrics Council on Children With Disabilities. Management of children with autism spectrum disorders. Pediatrics. 2007;120(5):1162–1182. doi: 10.1542/peds.2007-2362. [DOI] [PubMed] [Google Scholar]
- 4.Lord C, McGee J, editors. Educating Children With Autism. Washington, DC: National Academies Press; 2001. [Google Scholar]
- 5.Baio J. Prevalence of Autism Spectrum Disorders — Autism and Developmental Disabilities Monitoring Network, 14 Sites, United States, 2008. Surveillance Summaries. 2012;61(3):1–19. [PubMed] [Google Scholar]
- 6.Pinto-Martin JA, Young LM, Mandell DS, Poghosyan L, Giarelli E, Levy SE. Screening strategies for Autism Spectrum Disorders in pediatric primary care. Journal of Developmental & Behavioral Pediatrics. 2008;29(5):345–350. doi: 10.1097/DBP.0b013e31818914cf. [DOI] [PubMed] [Google Scholar]
- 7.Filipek PA, Accardo PJ, Ashwal S, et al. Practice parameter: screening and diagnosis of autism: report of the Quality Standards Subcommittee of the American Academy of Neurology and the Child Neurology Society. Neurology. 2000;55:468–479. doi: 10.1212/wnl.55.4.468. [DOI] [PubMed] [Google Scholar]
- 8.Howlin P, Asgharian A. The diagnosis of autism and Asperger syndrome: findings from a survey of 770 families. Developmental Medicine & Child Neurology. 1999;41:834–839. doi: 10.1017/s0012162299001656. [DOI] [PubMed] [Google Scholar]
- 9.Gray K, Tonge B, Brereton A. Screening for autism in infants, children, and adolescents. International Review of Research in Mental Retardation. 2006;32:197–227. [Google Scholar]
- 10.Kleinman JM, Robins DL, Ventola PE, Pandey J, Boorstein HC, Esser EL, Wilson LB, Rosenthal MA, Sutera S, Verbalis AD, Barton M, Hodgson S, Green J, Dumont-Mathieu T, Volkmar F, Chawarska K, Klin A, Fein D. The Modified Checklist for Autism In Toddlers: A follow-up study investigating the early detection of Autism Spectrum Disorders. Journal of Autism and Developmental Disorders. 2008;38:8270839. doi: 10.1007/s10803-007-0450-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Earls MF, Hay SS. Setting the stage for success: Implementation of developmental and behavioral screening and surveillance in primary care practice – the North Carolina Assuring Better Child Health and Development (ABCD) Project. Pediatrics. 2006;118(1):183–188. doi: 10.1542/peds.2006-0475. [DOI] [PubMed] [Google Scholar]
- 12.Pinto-Martin JA, Dunkle M, Earls M, Fliedner D, Landes C. Developmental stages of developmental screening: Steps to implementation of a successful program. American Journal of Public Health. 2005;95(11):1928–1932. doi: 10.2105/AJPH.2004.052167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Glascoe F. Screening for developmental and behavioral problems. Mental Retardation and Developmental Disabilities Research Reviews. 2005;11(3):173–179. doi: 10.1002/mrdd.20068. [DOI] [PubMed] [Google Scholar]
- 14.Hix-Small H, Marks K, Squires J, Nickel R. Impact of implementing developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120:381–389. doi: 10.1542/peds.2006-3583. [DOI] [PubMed] [Google Scholar]
- 15.Bailey D, Hebbeler K, Spiker D, et al. Thirty-six-month outcomes for families of children who have disabilities and participated in early intervention. Pediatrics. 2005;116:1346–1352. doi: 10.1542/peds.2004-1239. [DOI] [PubMed] [Google Scholar]
- 16.Johnson CP, Myers SM. Identification and evaluation of children with autism spectrum disorders. Pediatrics. 2007;120(5):1183–1215. doi: 10.1542/peds.2007-2361. [DOI] [PubMed] [Google Scholar]
- 17.Anderson L, Shinn C, Fullilove M, et al. The effectiveness of early childhood development programs: a systematic review. American Journal Preventative Medicine. 2003;24(3):32–46. doi: 10.1016/s0749-3797(02)00655-4. [DOI] [PubMed] [Google Scholar]
- 18.Barton M, Dumont-Mathieu T, Fein D. Screening Young Children for Autism Spectrum Disorders in Primary Practice. Journal of Autism and Developmental Disorders. 2012;42(6):1165–1174. doi: 10.1007/s10803-011-1343-5. [DOI] [PubMed] [Google Scholar]
- 19.Committee on Children and Disabilities, American Academy of Pediatrics. Developmental surveillance and screening for infants and young children. Pediatrics. 2001;108(1):192–195. [Google Scholar]
- 20.Rydz D, Srour M, Oskoui M, et al. Screening for developmental delay in the setting of a community pediatric clinic: a prospective assessment of parent-report questionnaires. Pediatrics. 2006;118(4):1178–1186. doi: 10.1542/peds.2006-0466. [DOI] [PubMed] [Google Scholar]
- 21.Baron-Cohen S, Wheelwright S, Cox A. The early identification of autism: The checklist for autism in toddlers (CHAT) Journal of the Royal Society of Medicine. 2000;93:521–525. doi: 10.1177/014107680009301007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Robins DL, Fein D, Barton ML, Green J. The modified checklist for autism in toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disorders. 2001;31(2):131–144. doi: 10.1023/a:1010738829569. [DOI] [PubMed] [Google Scholar]
- 23.Robins D, Casagrande K, Barton M, et al. Validation of the Modified Checklist for Autism in Toddlers, Revised with Follow-up (M-CHAT-R/F) Pediatrics. 2014;133(1):37–45. doi: 10.1542/peds.2013-1813. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Robins DL. Screening for autism spectrum disorders in primary care settings. Autism. 2008;12(5):537–556. doi: 10.1177/1362361308094502. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Glascoe FP, Macias MM, Wegner LM, Robertshaw NS. Can a broadband developmental-behavioral screening test identify children likely to have autism spectrum disorder? Clinical pediatrics. 2007;46:801–805. doi: 10.1177/0009922807303928. [DOI] [PubMed] [Google Scholar]
- 26.Glascoe FP, Macias MM, Wegner LM, Robertshaw NS. Can a broadband developmental-behavioral screening test identify children likely to have autism spectrum disorder? Clinical pediatrics. 2007;46:801–805. doi: 10.1177/0009922807303928. [DOI] [PubMed] [Google Scholar]
- 27.Wiggins L, Piazza V, Robins D. Comparison of a broad-based screen versus disorder-specific screen in detecting young children with an autism spectrum disorder. Autism: The International Journal of Research and Practice. 2014;18(2):76–84. doi: 10.1177/1362361312466962. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Squires J, Bricker D, Potter L. Ages & Stages Questionnaires, Third Edition (ASQ-3) User's Guide. Baltimore, MD: Paul H. Brookes Publishing; 2009. [Google Scholar]
- 29.Sparrow SS, Cicchetti DV, Balla DA. Vineland-II: Vineland Adaptive Behavior Scales, Second Edition, survey forms manual. Circle Pines, MN: AGS Publishing; 2005. [Google Scholar]
- 30.Lord C, Rutter M, DiLavore P, Risi S, Gotham K, Bishop S. ADOS-2: Autism diagnostic observation schedule. Los Angeles, CA: Western Psychological Services; 2012. [Google Scholar]
- 31.Mullen EM. Mullen Scales of Early Learning: AGS Edition. Circle Pines, MN: American Guidance Service; 1995. [Google Scholar]
- 32.Briggs RD, Stettler EM, Silver EJ, Schrag RD, Nayak M, Chinitz S, Racine A. Social-emotional screening for infants and toddlers in primary care. Pediatrics. 2012;129(2):1–8. doi: 10.1542/peds.2010-2211. [DOI] [PubMed] [Google Scholar]
- 33.Squires J, Bricker D, Twombly E. The ASQ:SE user's guide: For the Ages & Stages Questionnaires: Social-emotional. Baltimore, MD: Paul H Brookes Publishing; 2002. [Google Scholar]
- 34.Chlebowski C, Robins DL, Barton M, Fein D. Large-scale use of the modified checklist for autism in low-risk toddlers. Pediatrics. 2013;131(4):1121–1127. doi: 10.1542/peds.2012-1525. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Miller JS, Gabrielsen T, Villalobos M, Alleman R, Wahmhoff N, Carbone PS, et al. The each child study: Systematic screening for autism spectrum disorders in a pediatric setting. Pediatrics. 2011;127(5):866–871. doi: 10.1542/peds.2010-0136. [DOI] [PubMed] [Google Scholar]
- 36.Windham GC, Smith KS, Rosen N, Anderson MC, Grether JK, Coolman RB, Harris S. Autism and developmental screening in a public, primary care setting primarily serving hispanics: Challenges and results. Journal of Autism and Developmental Disorders. 2014 doi: 10.1007/s10803-014-2032-y. Advance online publication. [DOI] [PubMed] [Google Scholar]
- 37.Canal-Bedia R, Garcia-Primo P, Martin-Cilleros MV, Santos-Borbujo J, Guisuraga-Fernandez Z, Herraez-Garcia L, et al. Modified checklist for autism in toddlers: Cross-cultural adaptation and validation in Spain. Journal of Autism and Developmental Disorders. 2011;41(10):1342–1351. doi: 10.1007/s10803-010-1163-z. [DOI] [PubMed] [Google Scholar]