Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Sep 1.
Published in final edited form as: J Psychopathol Behav Assess. 2018 Jan 20;40(3):367–375. doi: 10.1007/s10862-018-9642-1

Accuracy of Reported Community Diagnosis of Autism Spectrum Disorder

Moran Hausman-Kedem 1,2,3, Barry E Kosofsky 1,2,4, Gail Ross 1, Kaleb Yohay 1,2,4, Emily Forrest 1, Margaret H Dennin 5, Reena Patel 1, Kristen Bennett 5, James P Holahan 5, Mary J Ward 1
PMCID: PMC6154796  NIHMSID: NIHMS954778  PMID: 30270970

Abstract

To compare community diagnoses of Autism Spectrum Disorder (ASD) reported by parents to consensus diagnoses made using standardized tools plus clinical observation. 87 participants (85% male, average age 7.4 years), with reported community diagnosis of ASD were evaluated using the Autism Diagnostic Observation Schedule) (ADOS-2), Differential Ability Scale (DAS-II), and Vineland Adaptive Behavior Scales (VABS-II). Detailed developmental and medical history was obtained from all participants. Diagnosis was based on clinical consensus of at least two expert clinicians, using test results, clinical observations, and parent report. 23% of participants with a reported community diagnosis of ASD were classified as non-spectrum based on our consensus diagnosis. ASD and non-spectrum participants did not differ on age at evaluation and age of first community diagnosis. Non-verbal IQ scores and Adaptive Behavior Composite scores were significantly higher in the non-spectrum group compared to the ASD group (104.5 ± 21.7 vs. 80.1 ± 21.6, p < .01; 71.1 ± 15 versus 79.5 ± 17.6, p < .05, respectively). Participants enrolled with community diagnosis of PDD-NOS were significantly more likely to be classified as non-spectrum on the study consensus diagnosis than Participants with Autism or Asperger (36% versus 9.5%, Odds Ratio = 5.4, p < .05). This study shows suboptimal agreement between community diagnoses of ASD and consensus diagnosis using standardized instruments. These findings are based on limited data, and should be further studied, taking into consideration the influence of DSM 5 diagnostic criteria on ASD prevalence.

Keywords: Autism, Autism spectrum disorder (ASD), Pervasive developmental delay not otherwise specified (PDD-NOS), Autism diagnostic observation schedule (ADOS)

Introduction

Relatively little is known about the way in which children in the US are diagnosed with autism. Current epidemiological estimates and studies are frequently based on administrative data as well as survey- based valuation to estimate the national prevalence of autism (Mandell and Palmer 2005; Newschaffer et al. 2005; Shattuck 2006). The latest figures from the Centers for Disease Control and Prevention (CDC) report a prevalence of 1:68 of autism in children, as compared to a rate of 1:1000 in 1989, with CDC figures based on telephone survey of parents, as well as chart reviews (Boyle et al. 2011; CDC 2009, 2012, 2014; Blumberg et al. 2013). There has been considerable debate regarding the basis for the significant increase in the diagnosis of autism over the past 20 years. Some possible explanations have included: changes in the environment that have led to genetic mutations (Herbert et al. 2006; Herbert 2010), increased awareness among pediatric professionals who refer children for diagnosis and the type of measures used to make a diagnosis of autism (Rutter 2005; Bishop et al. 2008, Taylor 2006; Fombonne 2005; Lundström et al. 2015). Inconsistencies in the application of Autism spectrum disorder diagnoses and nomenclature, e.g., Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS), Autism, and Asperger's Disorder, have led to eliminating these terms and subsuming these disorders under the term Autism Spectrum Disorder (ASD) in the latest Diagnostic and Statistical Manual of Mental Disorders, fifth Edition manual (DSM-5) (First 2013). Moreover, the current ASD diagnosis places emphasis on deficits in social communication and restricted interests and repetitive behaviors, independent of language delay.

The complex nature of ASD, coupled with the inadequacy of medical or biological markers for diagnosis, as well as changes in clinical definitions over time, has increased challenges in diagnosing and monitoring the prevalence of ASD. Therefore, the diagnosis has depended on the abilities of experienced clinicians to identify ASD through detailed history taking, direct behavioral observation, and use of assessment instruments designed to identify this disorder.

Current ‘gold standard’ diagnostic practice involves a best estimate clinical consensus (BEC) based a detailed developmental history from caregivers; multidisciplinary professionals' opinions; results of standardized assessments; observation of the individual in multiple settings; and diagnostic criteria as described in the DSM-5 (Filipek et al. 2000; Baird et al. 2000, 2011). At present, there are a number of standardized tools and questionnaires designed to assess for autism that are routinely used in the clinical setting, e.g.; Childhood Autism Rating Scale (CARS), Modified Checklist for Autism in Toddlers (M-CHAT), Gilliam Autism Rating Scale (GARS) (Ozonoff et al. 2005; Robins et al. 2001; Schopler et al. 1980; South et al. 2002; Gilliam 2008), Autism Diagnostic Interview-Revised (ADI-R) (Lord et al. 1994). Among them, the Autism Diagnostic Observation Schedule (ADOS) (Lord et al. 2000; Gotham et al. 2007) has been recognized as the “gold standard” for diagnosing autism (Ozonoff et al. 2005; Filipek et al. 1999). The ADOS, particularly when combined with the ADI-R, displays very high sensitivity and specificity, and shows levels of accuracy comparable to the correct classification rates for the current ‘gold standard’ diagnostic procedure, which is based on multidisciplinary team assessment and clinical judgment (So Hyun Kim 2012; Falkmer et al. 2013). The use of the ADOS/-2 and the ADI-R have been recommended to increase the reliability or objectivity in sampling the autistic symptomatology (Ozonoff et al. 2005; Filipek et al. 1999; Woolfenden et al. 2012; Le Couteur et al. 2008). However, administration and scoring of the ADOS requires more time and cost than other measures, as well as highly trained professionals to administer and score the measure, so that accuracy and reliability are ensured. As a consequence, the ADOS is widely used for research studies and is less often employed in community settings.

Studies evaluating ASD diagnosis in the community are predominantly focused on early detection of ASD and use of various ASD screening methods (Swanson et al. 2014; Earls and Hay 2006; Al-Qabandi et al. 2011; Barton et al. 2012), as well as regional demographic and socioeconomic factors influencing various aspects of ASD diagnosis (Fountain et al. 2011; Coo et al. 2012; Randall et al. 2016; Daniels and Mandell 2014). However, data regarding how children in the community are eventually diagnosed with ASD, as well as the validity of ASD diagnosis in the community, is scarce.

The objective of this research was to determine to what extent diagnoses of Autism made in the community by pediatric professionals are reproducible by performing diagnostic evaluation of autism by using BEC diagnosis, as well as administration of the ADOS by trained practitioners.

Methods

The study took place in an outpatient clinic of a tertiary medical center in NYC. It was approved by the university Institutional Review Board (IRB), and informed consent was obtained before initiating any research procedures.

Participants

Families were eligible for enrollment if there was at least one family member with a formal diagnosis of ASD (Autism, PDD-NOS and Asperger) from a community health care provider (psychologist or physician). All participants with ASD had no significant birth complications and no known genetic disorders. Eighty-seven participants with ASD, all from New York metropolitan area, were included. Mean age of participants was 7.6 ± 5 years (range = 2.2 to 25.9 years); 74 (85%) were male.

Measures

After parental consent and minor assent were obtained, a clinical diagnostic assessment performed in a fixed order was completed in the same day for all participants with ASD by at least 2 expert clinicians. These included the following:

Parent Questionnaire

Parents completed items concerning perinatal and early developmental history, comorbidities (seizure disorders, immune disorders); information on the community diagnostic process including age of the child's primary diagnosis, profession of healthcare provider who provided the diagnosis, which instruments were used in order to provide the diagnosis, type of formal diagnosis given, and additional diagnoses other than ASD. Detailed family history, including familial history of neuropsychiatric disorders, was reported. An ASD load score was then computed for each family, using the following equation: 0.5 points for each parent or sibling with a diagnosis of ASD plus 0.25 for other family members with ASD.

Medical History and Neurological Evaluation

Detailed medical as well as developmental history were obtained by a physician, including; perinatal history, developmental milestones, first behavioral-developmental concerns described by parents (language, motor, social interaction, repetitive behavior, restricted interests/insistence on sameness, and regression) in early childhood or later in life. A physical and neurologic examination was performed for all participants.

Autism Diagnostic Observation Schedule [ADOS]

The ADOS is a semi-structured, standardized assessment of communication and social interaction, which is a gold-standard assessment designed for diagnosing ASD (Lord et al. 2000). ADOS-second edition was used (Lord et al. 2012a). ADOS-2 administrators had obtained research reliability on the instrument. All participants had a minimum developmental age of 12 months and were assessed using the appropriate module of the ADOS-2 for given language level (Module 1 [nonverbal/few words], N = 32; Module 2 [fluent phrase speech], N = 25; Module 3 [verbally fluent school-aged child/adolescent], N = 25; Module 4 [verbally fluent adolescent/adult], N = 4; Toddler module, N = 1). All sessions were coded using the ADOS-2 algorithms (Gotham et al. 2007). An ADOS-2 Comparison Score (CS), which provides a measure of ASD symptoms across modules of administration, was recorded (Gotham et al. 2009; Shumway et al. 2012). Score of 6 or higher corresponds to the cutoff of classification of Autism on the ADOS. Score of 4–5 to the cutoff for classification of ASD on the ADOS, and score of 1–3 corresponds to classifi-cation of Non-spectrum classification on the ADOS.

Differential Ability Scales—Second Edition [DAS-II]

Cognitive ability was measured by evaluators, using the Differential Ability Scales, 2nd Edition (DAS-II) (Elliott 1990). The DAS-II was designed to assess a wide range of cognitive abilities in children aged 2 years 6 months through 17 years 11 months and has been used with children with ASD in numerous studies (Anderson et al. 2007; Thurm et al. 2007; Huerta and Lord 2012; Lord et al. 2012b). The DAS-II is an applicable assessment tool for evaluating the cognitive abilities of children with ASD, since it relies less on expressive language ability than other cognitive assessments, and is relatively easy to use. The DAS-II provides standard scores for overall ability (Global Conceptual Ability [GCA]) in addition to standard scores for the Verbal, Nonverbal, and Spatial domains. The Special Nonverbal Composite Score, a composite of non-verbal reasoning and spatial abilities, commonly used for ASD children whose language is not well developed (Joseph et al. 2002), was used for analysis in our study. This measure was used as many of the participants in this study had significant language delays; thus, the non-verbal tests of abstract reasoning in the DAS-II were considered a more valid measure of cognitive ability. For participants younger than the appropriate age range for the DAS-II (4 participants), we used standardized non-verbal cognitive testing appropriate for their age (Mullen Scales of Early Learning) (Akshoomoff 2006).

Participants older than 17.11 years old (3 participants) were tested with the Perceptual Reasoning subtests of the Wechsler Adult Intelligence Scale-IV (Wechsler 2008), as abstracted from their medical records. All tests had a mean of 100 +/− 15 and were measures of non-verbal cognitive ability.

Vineland Adaptive Behavior Scales II [VABS-II]

The VABS-II (Sparrow et al. 1984, 2005; Ray-Subramanian et al. 2011) is a parent-interview measure of adaptive behavior designed for use with individuals from birth through age 90. In addition to an overall Adaptive Behavior Composite (ABC), the VABS-II provides standard scores (mean = 100; standard deviation = 15) for four behavioral domains: Communication, Daily Living Skills, Socialization, and Motor Skills. The overall ABC standard score was utilized in our analyses of adaptive behavior.

Best Estimate Clinical (BEC) Diagnosis

For purposes of analysis, upon completion of all above mentioned assessments, participants were given a consensus diagnosis based on the ADOS-2 algorithm classification, clinical observations in the DAS-II, the neurological evaluation, parent report in the VABS-II, developmental history, and consistency of autism symptomatology with the DSM-5 diagnostic criteria.

Statistical Analyses

All analyses were performed using SPSS software ((version 20.0; SPSS Inc., Chicago, IL, USA). Independent sample t-tests were used for continuous variables when variances met assumptions. Categorical variables and difference in frequencies were analyzed using chi square analyses or fisher exact test of significance. Alpha of p < .05 was considered statistically significant. The agreement between the existing diagnosis made by community clinician, and the study diagnosis was examined using the Gwet's inter-rater reliability test (Gwet 2014). Gwet's inter-rater reliability test has been suggested to be less affected by the trait prevalence in the population under consideration. The interpretation of the Gwet's agreement is the same as Cohen's kappa as follows (Altman 1991): poor: less than 0.20, fair: 0.21–0.40, moderate: 0.41–0.60, good: 0.61–0.80 and very good: 0.81–1.00.

In addition, the sensitivity, and the positive predictive values (PPV) of the study diagnosis when using community diagnosis as the reference were calculated. Due to the study design, no children without an existing diagnosis of autism were included in this sample. Also, the examiner was not blind to the clinical status of the study participants. Thus, the specificity and the negative predictive values of the study diagnosis were not calculated.

Results

Forty-three (49%) participants had a community diagnosis of Autism, 38 (44%) a diagnosis of PDD-NOS, and 6 (7%) a diagnosis of Asperger Syndrome. Mean age of first community diagnosis of ASD was 3.3 ± 2.6 years (range- 1-19 years). Most participants (n = 48; 55%) were reported to receive ASD diagnoses from only one community health care provider. Twenty-four (28%) were evaluated by 2 health care providers, 10 (12%) participants were evaluated by 3 community health care providers, and 6 (6%) were reported to be evaluated by 4 to 5 community health care providers. Twenty-three participants (27%) were evaluated using the ADOS. Only 5 were reported to have performed the ADI-R. Twent-four (27%) participants performed the VABS. 7 (8%) participants were evaluated using the Wechsler Intelligence Scale for Children (WISC), 7 (8%) were evaluated using the Stanford Binet test.

Based on the study consensus diagnosis, 67 participants (77%) were classified with ASD, and 20 (23%) were classified as non-spectrum. Table 1 provides descriptive information on the sample by study diagnoses.

Table 1. Demographic and clinical characteristics of study participants.

Variable Diagnosis based on study clinical consensus
Autism spectrum disorder (n = 67) Non-spectrum (n = 20) P value
Male n (%) 60 (89%) 14 (70%) NS
Age of first parental concern (months) 18.7 ± 9.9 16.9 ± 8 NS
Age at diagnosis (years) 3 ± 1.9 4.2 ± 4 NS
Age at study evaluation (years) 7.9 ± 5 6.8 ± 5.1 NS
Time between first diagnosis and study evaluation (years) 4.9 ± 4.6 2.6 ± 2.5 0.005
Language impairment (%) 27 (40%) 10 (50%) NS
Non-Verbal IQ score 80.1 ± 21.6 104.5 ± 21.7 0.003
Vineland Adaptive Behavior Composite 65.6 ± 13.2 72.7 ± 12.2 0.04
ASD family load score 0.1 0.25 NS
Reported developmental regression (%) 27 (40%) 6 (30%) NS
History of seizures n (%) 4 (6%) 1 (5%) NS
Macrocephaly n (%) 13 (20%) 2 (10%) NS
Microcephaly n (%) 4 (6%) 1 (5%) NS
Subjects diagnosed by 1 health care provider 34 (50%) 13 (65%) NS
Subjects diagnosed by 2 or more health care providers 32 (48%) 7 (35%) NS
Subjects evaluated by ADOS in the community 19 (28%) 4 (20%) NS
Subjects evaluated by VABS in the community 19 (28%) 5 (25%) NS
Open in a new tab

Continuous data are presented as Mean + Standard deviation. Categorical data presented as n (%) NS non-significant, ADOS Autism Diagnostic Observation Schedule, VABS Vineland Adaptive Behavior Scale

Non-verbal IQ and VABS-II ABC scores were significantly higher for the non-spectrum group compared to the ASD group (80.1 ± 21.6 vs. 104.5 ± 21.7, p = 0.003 and 65.6 ± 13.2 vs. 72.7 ± 12.2, p = 0.04, respectively). There were no significant differences between the ASD group and the non-spectrum group on age at study evaluation (7.9 ± 5 vs. 6.8 ± 5.1 years, NS), age at first community diagnosis (3.0 ± 1.9 vs.4.2 ± 4, NS), and age of parental first concern (18.7 ± 9.9 vs. 16.9 ± 8 months, NS) (Table 1). First behavioral-developmental concerns described by parents (language, motor, social interaction, repetitive behavior, restricted interests/insistence on sameness) did not differ significantly between the non-spectrum and ASD group. Similarly, there was no significant difference in reported occurrence of developmental regression between the ASD and non-spectrum groups (30% vs. 40%, NS). The non-spectrum and ASD group did not differ in seizure prevalence (6% vs. 5%, NS), macrocephaly (15% vs. 10%, NS), or microcephaly (6% vs. 5%, NS).

While more participants later found to not have a diagnosis of ASD (to be non-spectrum) received a diagnosis of ASD from only one healthcare provider rather than from multiple health care providers (65% vs. 50%), the difference was not statistically significant (χ2 = 2.1071, p-value = 0.3487 (Fig. 1).

Fig. 1.

Fig. 1

Open in a new tab

Distribution of study consensus diagnoses by number of health care providers providing community ASD diagnosis

The proportion of participants evaluated by standardized tests in the community did not differ significantly between the ASD and non-spectrum group. (ASD group- 28%, non-spectrum group- 20%, χ2 = 0.20696, p-value = 0.6492). All participants with a non-spectrum diagnosis as per study assessment who were reportedly evaluated by ADOS in the community were reported to have a community diagnosis of PDD-NOS.

Participants enrolled with community diagnosis of PDD-NOS were significantly more likely to be classified as non-spectrum than participants with community diagnoses of Autism or Asperger Syndrome (36% versus 10%, Odds Ratio = 5.4, p < .05) (Fig. 2).

Fig. 2. Distribution of study consensus diagnosis among participants.

Fig. 2

Open in a new tab

a Distribution of consensus diagnosis in participants with a community diagnosis of PDD-NOS (based on DSM-IV criteria). b Distribution of consensus diagnosis in participants with a community diagnosis of Autism/Asperger syndrome (based on DSM-IV criteria)

The sensitivity of the study evaluation was 0.58, (95% Confidence Interval (CI):46%, 70%), whereas the PPV was 0.93 (95% CI: 81%, 99%) when using community diagnosis of Autism as the reference. Using the Gwet's alternative chance-corrected statistic, the agreement between the community diagnosis and the study diagnosis was very good (Gwet's alternative chance-corrected Index = 0.71, (95% CI: 0.57,0.84, p > 0.001).

The time between first diagnosis and study evaluation was significantly shorter in the non-spectrum group compared to the ASD group, the latter having been diagnosed for almost twice as long as the non-spectrum group (58.8 ± 55.9 vs.30.9 ± 30 months, p < .01).

Discussion

Our study examined the relationship between reported community diagnosis of ASD and diagnosis based on the ADOS, a standardized test with strong predictive validity against best estimate Autism diagnosis, considered the “gold standard” for diagnosing autism.

Importantly, due to the timing of the study, all subjects carried an active DSM-IV based diagnosis upon study evaluation. Although the agreement between the community diagnosis and the ADOS- based diagnosis was good, nearly one-quarter (23%) of children with a community-derived diagnosis of autism, PDD, or Asperger's Disorder were found to not be on the autistic spectrum based on our later ADOS based evaluation. In this sample, ADOS-2 algorithm classification matched consensus diagnosis in all cases. Although we did not perform additional diagnostic batteries for the non-spectrum group in order to provide them with a definite diagnosis, we recorded a considerable amount of data from the families in addition to conducting the ADOS, including demographic information, family background, age of first concern and type of concern, neurologic comorbidities, and also performed measures of adaptive functioning and non-verbal IQ, in order to better characterize the study group. In our analyses, IQ and Vineland Adaptive Behavior Composite score significantly distinguished children with ASD from those who were non-spectrum. This result is in agreement with that of other studies, showing that adaptive behavior correlates with IQ and with ADOS total algorithm scores and is negatively associated with autism symptomatology (Ray-Subramanian et al. 2011; Kenworthy et al. 2010; Kanne et al. 2011; Paul et al. 2014). Moreover, there is evidence to suggest that Vineland scores are lower in children with ASD compared to children with other developmental delays and correlate with ADOS scores (Ray-Subramanian et al. 2011; Paul et al. 2014). None of the other variables in our study, including perinatal and developmental parameters, age of first concern or age of original community- provided diagnosis, and comorbidities, was related to whether children were diagnosed as ASD or not on the ADOS.

There are several possible explanations for this discrepancy of diagnosis between community practitioners and the consensus diagnosis using standardized instruments used by our research team. Importantly, the change in grouping children with ASD and removal of PDD-NOS and Asperger's diagnoses from the DSM-5 definition of autism (First 2013), emphasizing the core features of social affect and fixated, repetitive behaviors, rather than language communication, results in some misclassification, which possibly explains some of the discrepancy in diagnosis demonstrated in this study. This is reflected by emerging evidence suggesting that ASD prevalence is lower under the DSM-5 diagnostic criteria compared to DSM-4 (Kulage et al. 2014; Maenner et al. 2014; Lundström et al. 2015), particularly with regards to children previously diagnosed with PDD-NOS (Christiansz et al. 2016). The low sensitivity demonstrated in our study was reported in other studies comparing the new DSM-5 criteria with those of the DSM-IV (McPartland et al. 2012; Taheri and Perry 2012). This may reflect the tendency of the DSM-5 criteria to particularly exclude those with higher cognitive ability. It is important to emphasize that the higher NVIQ and Vineland-II composite scores could also serve as confounders as described by Kim and Lord 2012, who found that ASD cases misclassified as non-spectrum had higher NVIQ and Vineland scores than the correctly classified ASD cases. Despite the above-mentioned explanation Our rate of discrepancy is higher than reported in the literature; studies comparing DSM-IV and DSM-5 concluded that more than 90% of children who received a DSM-IV PDD-NOS diagnosis met the DSM-5 ASD criteria, including more cognitively able individuals (Huerta et al. 2012; Sumi et al. 2014), which is much higher rates than the rates reported in our study.

Notably, all patients with a non-spectrum diagnosis did not qualify for a diagnosis of ASD according to the ADOS-2, suggesting that the change in classification could not be the sole explanation for the discrepancy of diagnoses demonstrated in our study. Moreover, as DSM-5 criteria have the tendency to particularly exclude younger patients, not qualifying for all criteria, as well as those with Asperger-like presentations, we did not show a statistically significant difference in patients age as well as the rates of language impairment between the ASD group and “non- spectrum” group (Table 1). As there was a time difference between the occurrence of the community diagnosis and the ADOS-based diagnosis, it is also possible that some children who were diagnosed as non-spectrum on the current ADOS- based evaluation had “recovered” from autism (although the timed elapsed between diagnoses for the non-spectrum children was shorter than for the children who continued to have an ASD diagnosis). There is some basis for this explanation in the literature, as longitudinal research has shown that approximately 10% of children who are initially diagnosed as autistic no longer have that diagnosis as they get older (Lord and Richler 2006). That result is supported by our finding that the children who were not diagnosed on the spectrum in our study were significantly more likely to have higher IQ scores and adaptive functioning scores, suggesting the possibility that they had higher learning abilities (Kraijer 2000), and were potentially more likely to show sufficient improvements in their language and social communication skills over time so as to lose their ASD diagnosis. Importantly, the majority of children found to not be on the spectrum in our study, had a DSM-IV community diagnosis of PDD-NOS. This is in concordance with findings from Lord et, al's longitudinal research showing that children diagnosed with PDD-NOS had the highest chance of losing their diagnosis (Lord et al. 2006). Unlike a clear diagnosis of Autism, a diagnosis of PDD-NOS appears to be a more ambiguous one that is inclusive of children with a wide variety of neurodevelopmental issues, including expressive and receptive language delays, sensory processing disorder, and behavioral difficulties. As the children develop, there is a clarification of diagnosis, and it may become more apparent that speech and language, for example, is the major problem, rather than ASD. While it is possible that some of these children resolved their diagnosis of ASD by the time they were assessed by us, all participants carried a current diagnosis. Thus, it is plausible to assume that at least some of the non-spectrum group were originally provided an inaccurate diagnosis. Lastly, the number of developmental intervention services hours and breadth of services received by children with a diagnosis of ASD are often much lower than recommended levels for non- ASD children (Stahmer and Mandell 2007). As a specific diagnosis of ASD is required to access higher intensity and specialized intervention services in many communities (Stahmer and Mandell 2007; Lord and Richler 2006), this may lead to a potentially greater motivation to making a diagnosis of ASD than of another diagnostic category. This situation also is true for school-aged children, who require a formal diagnosis to qualify for special education services and classroom supports under the Individuals with Disabilities Education Act. The problem of accurately diagnosing ASD is exacerbated by the fact that limitations in community resources make it difficult to provide well-timed, standardized and comprehensive evaluations for autism. The fact that only 28% of the families reported the use of standardized tests such as the ADOS or ADI-R, in the community diagnostic process may assuredly represent parental under report, but may also account for over-identification of autism as well. A larger proportion of the non-spectrum children were initially diagnosed by one examiner only, rather than a multi-disciplinary team. However, this observation was not statistically significant, potentially due to the relatively small cohort. As only some US states recommend specific ASD diagnostic tools (Kim 2012), future changes in evaluation practices, as well as refinements to standardized diagnostic instruments, will also potentially affect future trends in ASD prevalence estimation, since those tests can increase the reliability or objectivity in sampling the autistic symptomatology and inform clinical judgment.

Over- or mis-diagnosis of ASD can convey serious psychological, educational, and financial consequences for the proper and timely provision of appropriate services, both for children who truly have a diagnosis of ASD and those who do not (Sikora et al. 2008). It can also affect estimates of the prevalence of ASD and, thereby, affects public policy and use of financial resources. Having a standardized means of diagnosing autism also is of great importance in advancing the under-standing and treatment of this disorder. Although many neurobiological research studies include standardized tests in the process of identifying autistic participants for genetic repositories, some researchers have relied on community diagnosis of ASD or other non-standardized, rapid screening tools in efforts to collect sufficient numbers of participants in a timely and less costly manner (Mandell et al. 2010; Kogan et al. 2009). However, inclusion of children who would not be diagnosed as ASD through standardized testing clearly limits ability to obtain meaningful results and conclusions (Warren et al. 2012).

Limitations

A number of study limitations should be considered. Primarily, our study was limited by a relatively small sample size. Moreover, our information regarding the diagnostic process used in the community was based solely on parental report. In addition, we did not perform detailed psychological evaluations on the non-spectrum children to identify their current diagnosis.

Despite these limitations, by putting forward the suboptimal agreement between community-reported diagnosis of ASD and a subsequent consensus diagnosis using standardized instruments, our results underline the importance of a standardized and multi-disciplinary diagnostic process for correct ASD diagnosis and more appropriate, cost-efficient interventions.

Acknowledgments

Funding: This study was funded by NIH NCATS CTSA 2UL TR000457–06.

The funder had no involvement in the study design, collection, analysis and interpretation of the data, writing of the report, and in the decision to submit the paper for publication.

Grant Support No honorarium, grant, or other form of payment was given to anyone to produce the manuscript.

Abbreviations

ADOS

Autism diagnostic observation schedule

ASD

Autism spectrum disorder

CSS

Calibrated severity score

PDD-NOS

Pervasive developmental disorder not otherwise specified

Footnotes

Conflict of Interest: Moran Hausman-Kedeml, Barry E. Kosofsky, Gail Ross, Kaleb Yohay, Emily Forrest, Margaret H. Dennin, Reena Patel, Kristen Bennett, James P. Holahan and Mary J. Ward declare that there is no conflict of interest.

Compliance with Ethical Standards: Experiment Participants: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent: Informed consent was obtained from all individual participants included in the study or their parents. An assent was obtained from all minors participating in the study.

References

  1. Akshoomoff N. Use of the Mullen scales of early learning for the assessment of young children with autism spectrum disorders. Child Neuropsychology. 2006;12(4-5):269–277. doi: 10.1080/09297040500473714. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Al-Qabandi M, Gorter JW, Rosenbaum P. Early autism detection: are we ready for routine screening? Pediatrics. 2011;128:e211–e217. doi: 10.1542/peds.2010-1881. [DOI] [PubMed] [Google Scholar]
  3. Altman D. Practical statistics for medical research. 1st. London: Chapman & Hall; 1991. [Google Scholar]
  4. Anderson DK, Lord C, Risi S, et al. Patterns of growth in verbal abilities among children with autism spectrum disorder. Journal of Consulting and Clinical Psychology. 2007;75(4):594–604. doi: 10.1037/0022-006X.75.4.594. [DOI] [PubMed] [Google Scholar]
  5. Baird G, Charman T, Baron-Cohen S, Cox A, Swettenham J, Wheelwright S, Drew A. A screening instrument for autism at 18 months of age: a 6-year follow-up study. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39(6):694–702. doi: 10.1097/00004583-200006000-00007. [DOI] [PubMed] [Google Scholar]
  6. Baird G, Douglas HR, Murphy MS. Recognising and diagnosing autism in children and young people: summary of NICE guidance. BMJ. 2011;343:d6360. doi: 10.1136/bmj.d6360. [DOI] [PubMed] [Google Scholar]
  7. Barton ML, Dumont-Mathieu T, Fein D. Screening young children for autism spectrum disorders in primary practice. Journal of Autism and Developmental Disorders. 2012;42:1165–1174. doi: 10.1007/s10803-011-1343-5. [DOI] [PubMed] [Google Scholar]
  8. Bishop DV, Whitehouse AJ, Watt HJ, Line EA. Autism and diagnostic substitution: evidence from a study of adults with a history of developmental language disorder. Developmental Medicine and Child Neurology. 2008;50(5):341. doi: 10.1111/j.1469-8749.2008.02057.x. [DOI] [PubMed] [Google Scholar]
  9. Blumberg SJ, Bramlett MD, Kogan MD, Schieve LA, Jones JR, Lu MC. Changes in prevalence of parent-reported autism spectrum disorder in school-aged U.S. Children: 2007 to 2011–2012. National Health Statistics Reports Number 65. 2013 [PubMed] [Google Scholar]
  10. Boyle CA, Boulet S, Schieve LA, Cohen RA, Blumberg SJ, Yeargin-Allsopp M, et al. Trends in the prevalence of developmental disabilities in US children, 1997-2008. Pediatrics. 2011;127:1034–1042. doi: 10.1542/peds.2010-2989. [DOI] [PubMed] [Google Scholar]
  11. Centers for Disease Control and Prevention (CDC) Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, United States, 2006. MMWR Surveillance Summaries. 2009;58(SS10):1–20. [PubMed] [Google Scholar]
  12. Centers for Disease Control and Prevention (CDC) Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, 14 sites, United States, 2008. MMWR Surveillance Summaries. 2012;61(3):1–19. [PubMed] [Google Scholar]
  13. Centers for Disease Control and Prevention (CDC) Prevalence of autism spectrum disorder among children aged 8 years - autism and developmental disabilities monitoring network, 11 sites, United States. MMWR Surveillance Summaries. 2014;63:1–21. [PubMed] [Google Scholar]
  14. Christiansz J, Gray KM, Taffe J, Tonge B. Autism spectrum disorder in the DSM-5: diagnostic sensitivity and specificity in early childhood. Journal of Autism and Developmental Disorders. 2016;46(6):2054–2063. doi: 10.1007/s10803-016-2734-4. ISSN: 0162–3257. [DOI] [PubMed] [Google Scholar]
  15. Coo H, Ouellette-Kuntz H, Lam M, et al. Correlates of age at diagnosis of autism spectrum disorders in six Canadian regions. Chronic Diseases and Injuries in Canada. 2012;32(2):90–100. [PubMed] [Google Scholar]
  16. Daniels AM, Mandell DS. Explaining differences in age at autism spectrum disorder diagnosis: a critical review. Autism. 2014;18(5):583–597. doi: 10.1177/1362361313480277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Earls MF, Hay SS. Setting the stage for success: implementation of developmental and behavioral screening and surveillance in primary care practice—the North Carolina assuring better child health and development (ABCD project) Pediatrics. 2006;118(1):e183–e188. doi: 10.1542/peds.2006-0475. [DOI] [PubMed] [Google Scholar]
  18. Elliott CD. Differential ability scales. San Antonio: The Psychological Corporation; 1990. [Google Scholar]
  19. Falkmer T, Anderson K, Falkmer M, Horlin C. Diagnostic procedures in autism spectrum disorders: a systematic literature review. European Child & Adolescent Psychiatry. 2013;22(6):329–340. doi: 10.1007/s00787-013-0375-0. [DOI] [PubMed] [Google Scholar]
  20. Filipek PA, Accardo PJ, Baranek GT, Cook EH, Jr, Dawson G, Gordon B, et al. The screening and diagnosis of autistic spectrum disorders. Journal of Autism and Developmental Disorders. 1999;29(6):439–484. doi: 10.1023/a:1021943802493. [DOI] [PubMed] [Google Scholar]
  21. Filipek PA, Accardo PJ, Ashwal S, Baranek GT, Cook EH, Jr, Dawson G, et al. Practice parameter: screening and diagnosis of autism: report of the quality standards subcommittee of the American Academy of neurology and the child neurology society. Neurology. 2000;55(4):468–479. doi: 10.1212/wnl.55.4.468. [DOI] [PubMed] [Google Scholar]
  22. First MB. Diagnostic and statistical manual of mental disorders, 5th edition, and clinical utility. J NervMent Dis. 2013;201(9):727. doi: 10.1097/NMD.0b013e3182a2168a. [DOI] [PubMed] [Google Scholar]
  23. Fombonne E. Epidemiology of autistic disorder and other pervasive developmental disorders. Journal of Clinical Psychology. 2005;66(Suppl. 10):3–8. [PubMed] [Google Scholar]
  24. Fountain C, King M, Bearman PS. Age of diagnosis for autism: individual and community factors across 10 birth cohorts. Journal of Epidemiology and Community Health. 2011;65(6):503–510. doi: 10.1136/jech.2009.104588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Gilliam J. JPsychoeduc Assess. Vol. 26. Austin, TX: PRO-ED; 2008. Test review: GARS-2: Gilliam autism rating scale—second edition; pp. 395–401. [Google Scholar]
  26. Gotham K, Risi S, Pickles A, Lord C. The autism diagnostic observation schedule: revised algorithms for improved diagnostic validity. Journal of Autism and Developmental Disorders. 2007;37:613–627. doi: 10.1007/s10803-006-0280-1. [DOI] [PubMed] [Google Scholar]
  27. Gotham K, Pickels A, Lord C. Standardizing ADOS scores for a measure of severity in autism spectrum disorders. Journal of Autism and Developmental Disorders. 2009;39(5):693–705. doi: 10.1007/s10803-008-0674-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Gwet KL. Handbook of inter-rater reliability. 4th. Gaithersburg: Advanced Analytics Press; 2014. [Google Scholar]
  29. Herbert MR. Contributions of the environment and environmentally vulnerable physiology to autism spectrum disorders. CurrOpin Neurol. 2010;23:103–110. doi: 10.1097/WCO.0b013e328336a01f. [DOI] [PubMed] [Google Scholar]
  30. Herbert MR, Russo JP, Yang S, Roohi J, Blaxill M, Kahler SG, et al. Autism and environmental genomics. Neurotoxicology. 2006;27:671–684. doi: 10.1016/j.neuro.2006.03.017. [DOI] [PubMed] [Google Scholar]
  31. Huerta M, Lord C. Diagnostic evaluation of autism spectrum disorders. Pediatric Clinics of North America. 2012;59(1):103–111. doi: 10.1016/j.pcl.2011.10.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Huerta M, Bishop SL, Duncan A, Hus V, Lord C. Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. The American Journal of Psychiatry. 2012;169(10):1056–1064. doi: 10.1176/appi.ajp.2012.12020276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Joseph RM, Tager-Flusberg H, Lord C. Cognitive profiles and social-communicative functioning in children with autism spectrum disorder. J Child Psychol Psychiatry. 2002;43(6):807–821. doi: 10.1111/1469-7610.00092. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kanne SM, Gerber AJ, Quirmbach LM, Sparrow SS, Cicchetti DV, Saulnier CA. The role of adaptive behavior in autism spectrum disorders: implications for functional outcome. Journal of Autism and Developmental Disorders. 2011;41(8):1007–1018. doi: 10.1007/s10803-010-1126-4. [DOI] [PubMed] [Google Scholar]
  35. Kenworthy L, Case L, Harms MB, Martin A, Wallace GL. Adaptive behavior ratings correlate with symptomatology and IQ among individuals with high-functioning autism spectrum disorders. Journal of Autism and Developmental Disorders. 2010;40(4):416–423. doi: 10.1007/s10803-009-0911-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kim SH. Ph.D dissertation. The University of Michigan; 2012. Developing and refining instruments and methods for diagnostic and language assessment of young children with autism Spectrum disorders (ASD) Available: https://deepblue.lib.umich.edu/bitstream/handle/2027.42/91478/sohkim_1.pdf. [Google Scholar]
  37. Kim SH, Lord C. Combining information from multiple sources for the diagnosis of autism spectrum disorders for toddlers and young preschoolers from 12 to 47 months of age. Journal of Child Psychology and Psychiatry. 2012;53(2):143–151. doi: 10.1111/j.1469-7610.2011.02458.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Kogan MD, Blumberg SJ, Schieve LA, Boyle CA, Perrin JM, Ghandour RM, et al. Prevalence of parent-reported diagnosis of autism spectrum disorder among children in the US, 2007. Pediatrics. 2009;124:1395–1403. doi: 10.1542/peds.2009-1522. [DOI] [PubMed] [Google Scholar]
  39. Kraijer D. Review of adaptive behavior studies in mentally retarded persons with autism/pervasive developmental disorder. Journal of Autism and Developmental Disorders. 2000;30(1):39–47. doi: 10.1023/a:1005460027636. [DOI] [PubMed] [Google Scholar]
  40. Kulage KM, Smaldone AM, Cohn EG. How will DSM-5 affect autism diagnosis? A systematic literature review and meta-analysis. Journal of Autism and Developmental Disorders. 2014;44(8):1918–1932. doi: 10.1007/s10803-014-2065-2. [DOI] [PubMed] [Google Scholar]
  41. Le Couteur A, Haden G, Hammal D, McConachie H. Diagnosing autism spectrum disorders in pre-school children using two standardised assessment instruments: the ADI-R and the ADOS. Journal of Autism and Developmental Disorders. 2008;38:362–372. doi: 10.1007/s10803-007-0403-3. [DOI] [PubMed] [Google Scholar]
  42. Lord C, Richler J. Early diagnosis of autism spectrum disorders. In: Charman T, Stone W, editors. Social and communication development in autism spectrum disorders: Early identification, diagnosis, and intervention. NewYork: Guilford; 2006. pp. 35–60. [Google Scholar]
  43. Lord C, Rutter M, Le Couteur AJ. Autism diagnostic interview-revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Autism and Developmental Disorders. 1994;24(5):659–685. doi: 10.1007/BF02172145. [DOI] [PubMed] [Google Scholar]
  44. Lord C, Risi S, Lambrecht L, Cook EH, Jr, Leventhal BL, DiLavore PC, et al. The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders. 2000;30:205–223. [PubMed] [Google Scholar]
  45. Lord C, Risi S, DiLavore PS, Shulman C, Thurm A, Pickles A. Autism from 2 to 9 years of age. Archives of General Psychiatry. 2006;63:694–701. doi: 10.1001/archpsyc.63.6.694. [DOI] [PubMed] [Google Scholar]
  46. Lord C, Rutter M, DiLavore PC, et al. Autism diagnostic observation schedule, second edition: ADOS-2. Torrance: Western Psychological Services; 2012a. [Google Scholar]
  47. Lord C, Petkova E, Hus V, et al. A multisite study of the clinical diagnosis of different autism spectrum disorders. Archives of General Psychiatry. 2012b;69(3):306–313. doi: 10.1001/archgenpsychiatry.2011.148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Lundström S, Reichenberg A, Anckarsäter H, Lichtenstein P, Gillberg C. Autism phenotype versus registered diagnosis in Swedish children: prevalence trends over 10 years in general population samples. BMJ. 2015;350:h1961. doi: 10.1136/bmj.h1961. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Maenner MJ, Rice CE, Arneson CL, Cunniff C, Schieve LA, Carpenter LA, et al. Potential impact of DSM- criteria on autism spectrum disorder prevalence estimates. JAMA Psychiatry. 2014;71(3):292–300. doi: 10.1001/jamapsychiatry.2013.3893. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Mandell DS, Palmer R. Differences among states in the identification of autistic spectrum disorders. Archives of Pediatrics & Adolescent Medicine. 2005;159(3):266–269. doi: 10.1001/archpedi.159.3.266. [DOI] [PubMed] [Google Scholar]
  51. Mandell DS, Morales KH, Xie M, Polsky D, Stahmer A, Marcus SC. County-level variation in the prevalence of medicaid-enrolled children with autism spectrum disorders. Journal of Autism and Developmental Disorders. 2010;40:1241–1246. doi: 10.1007/s10803-010-0982-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. McPartland JC, Reichow B, Volkmar FR. Sensitivity and specificity of proposed DSM-5 diagnostic criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 2012;51(4):368–383. doi: 10.1016/j.jaac.2012.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Newschaffer CJ, Falb MD, Gurney JG. National autism prevalence trends from United States special education data. Pediatrics. 2005;115(3):277–282. doi: 10.1542/peds.2004-1958. [DOI] [PubMed] [Google Scholar]
  54. Ozonoff S, Goodlin-Jones BL, Solomon M. Evidence-based assessment of autism spectrum disorders in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2005;34:523–540. doi: 10.1207/s15374424jccp3403_8. [DOI] [PubMed] [Google Scholar]
  55. Paul R, Loomis R, Chawarska K. Adaptive behavior in toddlers under two with autism spectrum disorders. Journal of Autism and Developmental Disorders. 2014;44(2):264–270. doi: 10.1007/s10803-011-1279-9. [DOI] [PubMed] [Google Scholar]
  56. Randall M, Albein-Urios N, Brignell A, Gulenc A, Hennel S, Coates C, et al. Diagnosing autism: Australian paediatric research network surveys. Journal of Paediatrics and Child Health. 2016;52(1):11–17. doi: 10.1111/jpc.13029. [DOI] [PubMed] [Google Scholar]
  57. Ray-Subramanian CE, Huai N, Ellis Weismer S. Brief report: adaptive behavior and cognitive skills for toddlers on the autism spectrum. Journal of Autism and Developmental Disorders. 2011;41(5):679–684. doi: 10.1007/s10803-010-1083-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Robins DL, Fein D, Barton ML, Green JA. The modified checklist for autism in toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disorders. 2001;31:131–144. doi: 10.1023/a:1010738829569. [DOI] [PubMed] [Google Scholar]
  59. Rutter M. Incidence of autism spectrum disorders: changes over time and their meaning. ActaPaediatr. 2005;94:2–15. doi: 10.1111/j.1651-2227.2005.tb01779.x. [DOI] [PubMed] [Google Scholar]
  60. Schopler E, Reichler RJ, DeVellis RF, Daly K. Toward objective classification of childhood autism: Childhood autism rating scale (CARS) Journal of Autism and Developmental Disorders. 1980;10:91–103. doi: 10.1007/BF02408436. [DOI] [PubMed] [Google Scholar]
  61. Shattuck PT. The contribution of diagnostic substitution to the growing administrative prevalence of autism in US special education. Pediatrics. 2006;117(4):1028–1037. doi: 10.1542/peds.2005-1516. [DOI] [PubMed] [Google Scholar]
  62. Shumway S, Farmer C, Thurm A, Joseph L, Black D, Golden C. The ADOS calibrated severity score: Relationship to phenotypic variables and stability over time. Autism Research. 2012;5(4):267–276. doi: 10.1002/aur.1238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Sikora DM, Hall TA, Hartley SL, Gerrard-Morris AE, Cagle S. Does parent report of behavior differ across ADOS-G classifications: analysis of scores from the CBCL and GARS. Journal of Autism and Developmental Disorders. 2008;38:440–448. doi: 10.1007/s10803-007-0407-z. [DOI] [PubMed] [Google Scholar]
  64. South M, Williams BJ, McMahon WM, Owley T, Filipek PA, Shernoff E, et al. Utility of the Gilliam autism rating scale in research and clinical populations. Journal of Autism and Developmental Disorders. 2002;32:593–599. doi: 10.1023/a:1021211232023. [DOI] [PubMed] [Google Scholar]
  65. Sparrow SS, Balla DA, Cicchetti DV. Vineland adaptive behavior scales. American Guidance Service; Circle Pines: 1984. [Google Scholar]
  66. Sparrow SS, Cicchetti VD, Balla AD. Vineland adaptive behavior scales. 2nd. American guidance service; circle pines: 2005. [Google Scholar]
  67. Stahmer AC, Mandell DS. State infant/toddler program policies for eligibility and services provision for young children with autism. Administration and Policy in Mental Health. 2007;34:29–37. doi: 10.1007/s10488-006-0060-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Sumi S, Miyachi T, Ohya K, Ohashi K, Saitoh S. Application of the final DSM-5 criteria for young children with autism Spectrum disorder. Application of the final DSM-5 criteria for young children with autism spectrum disorder. Autism Open Access. 2014;4:135. doi: 10.4172/2165-7890.1000135. [DOI] [Google Scholar]
  69. Swanson AR, Warren ZE, Stone WL, Vehorn AC, Dohrmann E, Humberd The diagnosis of autism in community pediatric settings: does advanced training facilitate practice change? Autism. 2014;18(5):555–561. doi: 10.1177/1362361313481507. [DOI] [PubMed] [Google Scholar]
  70. Taheri A, Perry A. Exploring the proposed DSM-5 criteria in a clinical sample. Journal of Autism and Developmental Disorders. 2012;42(9):1810–1817. doi: 10.1007/s10803-012-1599-4. [DOI] [PubMed] [Google Scholar]
  71. Taylor B. Vaccines and the changing epidemiology of autism. Child: Care, Health and Development. 2006;32:511–519. doi: 10.1111/j.1365-2214.2006.00655.x. [DOI] [PubMed] [Google Scholar]
  72. Thurm A, Lord C, Lee L, et al. Predictors of language acquisition in preschool children with autism spectrum disorders. Journal of Autism and Developmental Disorders. 2007;37(9):1721–1734. doi: 10.1007/s10803-006-0300-1. [DOI] [PubMed] [Google Scholar]
  73. Warren Z, Vehorn A, Dohrmann E, Nicholson A, Sutcliffe JS, Veenstra-Vanderweele J. Accuracy of phenotyping children with autism based on parent report: what specifically do we gain phenotyping “rapidly”? Autism Research. 2012;5:31–38. doi: 10.1002/aur.230. [DOI] [PubMed] [Google Scholar]
  74. Wechsler D. Wechsler adult intelligence scale – fourth edition. San Antonio: Harcourt Assessment; 2008. [Google Scholar]
  75. Woolfenden S, Sarkozy V, Ridley G, Coory M, Williams K. A systematic review of two outcomes in autism spectrum disorder - epilepsy and mortality. Developmental Medicine and Child Neurology. 2012;54:306–312. doi: 10.1111/j.1469-8749.2012.04223.x. [DOI] [PubMed] [Google Scholar]

RESOURCES