Comparison of 2 Case Definitions for Ascertaining the Prevalence of Autism Spectrum Disorder Among 8-Year-Old Children

Abstract

The Autism and Developmental Disabilities Monitoring (ADDM) Network conducts population-based surveillance of autism spectrum disorder (ASD) among 8-year-old children in multiple US communities. From 2000 to 2016, investigators at ADDM Network sites classified ASD from collected text descriptions of behaviors from medical and educational evaluations which were reviewed and coded by ADDM Network clinicians. It took at least 4 years to publish data from a given surveillance year. In 2018, we developed an alternative case definition utilizing ASD diagnoses or classifications made by community professionals. Using data from surveillance years 2014 and 2016, we compared the new and previous ASD case definitions. Compared with the prevalence based on the previous case definition, the prevalence based on the new case definition was similar for 2014 and slightly lower for 2016. Sex and race/ethnicity prevalence ratios were nearly unchanged. Compared with the previous case definition, the new case definition’s sensitivity was 86% and its positive predictive value was 89%. The new case definition does not require clinical review and collects about half as much data, yielding more timely reporting. It also more directly measures community identification of ASD, thus allowing for more valid comparisons among communities, and reduces resource requirements while retaining measurement properties similar to those of the previous definition.

Autism spectrum disorder (ASD) is a developmental disability that is characterized by impairments in social communication and the presence of repetitive behaviors or restricted interests. The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) describes the symptoms comprising the criteria for ASD; however, there are different approaches to diagnosing ASD, including a variety of behavior-based diagnostic tests and the application of clinical judgment. Since 2000, the Centers for Disease Control and Prevention has funded the Autism and Developmental Disabilities Monitoring (ADDM) Network to conduct ASD surveillance among 8-year-old children in even-numbered years. When this multisite surveillance began in 2000, fewer than two-thirds of the children ascertained as having ASD were being diagnosed or were receiving educational services for ASD (1, 2).

Given the variation in recognition of ASD in communities, the ADDM Network developed a protocol in which a clinical review of evidence from developmental evaluations and health and educational records was key to establishing whether a child met the surveillance case definition for ASD. This process involved abstracting the verbatim (text) descriptions of behaviors from the evaluations, which were reviewed by 1 or more ADDM Network clinicians. The Network clinicians identified descriptions of behaviors and determined whether the child’s documented symptoms were consistent with the ASD diagnostic criteria of the DSM-5 (3). This process was labor-intensive, and it took at least 4 years to publish the data from a given surveillance year of 8-year-olds (e.g., for children who were aged 4 and 8 years in 2014, the ADDM Network published the data in 2018 and 2019; for children who were aged 4 and 8 years in 2016, the Network published the data in 2020) (4–7).

The need to produce more timely public health data, combined with improved recognition and diagnosis of ASD over the intervening time period, led to a reevaluation of the ADDM Network’s methods. We identified a potential new case definition that required fewer resources to implement and to report surveillance data. A comprehensive assessment of the new case definition was undertaken to evaluate the consistency of results from the new definition as compared with the old definition.

The purpose of this assessment was to evaluate and compare a new surveillance case definition of ASD with the previous ADDM Network (DSM-5) surveillance case definition. Along with estimating the prevalence of ASD, the ADDM Network also describes characteristics of children with ASD (such as demographic characteristics, special education classification, and intellectual ability), monitors community-level progress toward timely ASD identification (a US Department of Health and Human Services Healthy People 2020 measure), and reveals disparities in how different groups of children are evaluated and identified as having ASD. In addition to greater efficiency, a new surveillance case definition needs to generate consistent community-level data related to these indicators.

METHODS

ASD surveillance system and data

The ADDM Network conducts ASD surveillance in multiple geographically defined communities in the United States every 2 years. The surveillance protocol involves requesting health and special education records for children aged 8 years who are living in the study area during the surveillance year. Health records are requested from medical sources if they are assigned specific International Classification of Diseases (ICD) billing codes relevant to developmental disabilities, and special education records are requested from schools based on special education exceptionality codes. The ADDM Network sites then send trained record abstractors into the field to review the records for different types of ASD behavioral symptoms, ASD tests, or suspected or confirmed ASD diagnoses (see Web Table 1, available at https://doi.org/10.1093/aje/kwab106).

Previous method and case definition

For the previous case definition, if a child’s record contained 1 or more descriptions of behavior or indications of ASD, all of a child’s developmental evaluations were copied into the surveillance database. Verbatim descriptions of behavioral information were copied from the evaluations and compiled across sources into a single record for each child. ADDM Network clinicians then reviewed all abstracted evaluations, following a protocol to code each evaluation for specific descriptions of autism symptoms. During the surveillance years 2000–2012, the ASD case definition was based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR) criteria for autistic disorder or pervasive developmental disorder. Beginning in 2014, ADDM Network clinicians also coded for a DSM-5–based case definition (4), which was the primary case definition used in 2016. The DSM-IV-TR and DSM-5 criteria, as well additional details about the ADDM Network methods, have been described elsewhere (4, 8–12).

Under the previous case definition, a child’s case status was determined by ADDM Network clinicians who reviewed the collected evaluations from health and education records. The ADDM Network trained clinicians to systematically review the contents of the evaluations to code descriptions of autism symptoms, other behaviors, autism diagnoses, and other co-occurring conditions. A child could meet the case definition if they met at least 1 of 2 criteria: 1) behaviors consistent with the DSM-5 diagnostic features for ASD and/or 2) a written ASD diagnosis from a qualified professional (such as a physician, psychologist, or other health-care provider specializing in developmental conditions). The ADDM Network clinicians could disqualify children meeting 1 or both criteria if there was insufficient or conflicting information or if they decided that another diagnosed condition better accounted for the child’s symptoms. If an ADDM Network clinician reviewer was uncertain about the classification, a secondary reviewer would evaluate the child’s record and the 2 clinicians would discuss their reviews to reach a consensus. To maintain reliability, 10% of the records were reviewed by a second clinician.

New method and case definition

The new (updated/revised) surveillance case definition classifies children as having ASD if there is documentation in health or education records that they have ever received any of the following: 1) a written ASD diagnosis by a qualified professional; 2) a special education classification of autism (either primary exceptionality of ASD or an evaluation which concluded that the child meets criteria for autism eligibility); or 3) an autism ICD code (International Classification of Diseases, Ninth Revision, codes 299.00–299.99; International Classification of Diseases, Tenth Revision, codes in the F84 range) obtained from administrative or billing information. Autism diagnoses and special education classifications triggered record abstraction under the previous method, but ICD codes did not. (ASD-related ICD codes were systematically collected under the previous method.)

The new method requires ADDM Network data abstractors to collect less information on each child than the previous method. Abstractors will be able to determine whether the child meets the case definition and record summary information from evaluations, intellectual test information, and diagnoses or classifications of autism or other developmental conditions. Abstractors will not record verbatim narrative descriptions of behavior, because the new method does not include review by ADDM Network-trained clinicians.

Study population

The data for this study were obtained from 11 ADDM Network sites funded to conduct surveillance among 8-year-old children in 2014 and 2016, published previously (4–6). All 11 ADDM Network sites (Arizona, Arkansas, Colorado, Georgia, Maryland, Minnesota, Missouri, New Jersey, North Carolina, Tennessee, and Wisconsin) from 2016 were included. Eight of the 11 sites were included in the analysis of 2014 data, but 3 sites (Arizona, Colorado, and Missouri) could not be included because they performed the DSM-5 clinician reviews for only a subregion of their DSM-IV-TR study area; this prevented us from knowing whether some children who met the new case definition lived in the DSM-5 study area. For these 3 sites, we calculated the minimum and maximum possible values for ASD prevalence separately from the main analysis. Data from all sites participating in ADDM Network surveillance years 2000–2012 were also used to examine the sensitivity of community ASD identification over time.

Analysis

We compared the overall concordance, sensitivity, and positive predictive value of the new case definition with that of the previous (DSM-5) case definition. For both the previous and new case definitions, we generated estimates for prevalence (including prevalence ratios between subgroups) overall and by ADDM Network site. We also calculated several other key indicators that the ADDM Network uses to track improvements in ASD awareness and timely diagnosis: the median age of the earliest known ASD diagnosis (for children with a documented ASD diagnosis), the median age of children at their earliest evaluation (based on evaluations abstracted by the ADDM Network and among children born in the state of their age-8 residence), and proportion with intellectual disability (based on abstracted information from an intelligence quotient (IQ) test). Using the same approach as in the ADDM Network ASD surveillance reports, we calculated these indicators among children with data available for each measure. We also compared characteristics between groups of children that met the previous case definition only, the new case definition only, and both case definitions. Using all ADDM Network ASD cases from every ADDM Network surveillance year (2000–2016) (1, 2, 4, 6, 9–12), we described the proportion of children meeting the new criteria and the proportion with a documented ASD diagnosis. We also compared the amounts of information surveillance that staff had to collect and review in order to apply each case definition. We calculated 95% confidence intervals for prevalence estimates and used Pearson χ² tests to detect statistically significant between-group differences among children meeting 1 or both case definitions. For median values, we calculated 95% confidence intervals using the bootstrap method in R (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

In total, 9,153 children were classified as ASD cases under the previous case definition for the 2 surveillance years compared with 8,783 children who met the new case definition, with relatively high concordance between case definitions (Table 1). Using the previous case definition as the basis for comparison, the sensitivity of the new case definition was 85.7% and the positive predictive value was 89.3%. Out of all 59,179 children whose records were screened and who met residency and age requirements, the simple agreement between case definitions was 96.2% (56,933/59,179), and Cohen’s κ was 0.85.

Table 1.

Concordance (Number of Children) Between Previous and New Case Definitions^a of Autism Spectrum Disorder at 11 Sites in the ADDM Network^b, United States, 2014 and 2016^c

Case Definition Status	Case Definition Status
	Meets Previous Definition	Does Not Meet Previous Definition	Total
Meets new definition	7,845	938	8,783
Does not meet new definition	1,308	49,088	50,396
Total	9,153	50,026	59,179

Abbreviations: ADDM, Autism and Developmental Disabilities Monitoring; ASD, autism spectrum disorder.

^aPrevious case definition: child had sufficient descriptions of ASD symptoms to meet operationalized Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, ASD criteria and/or had received an ASD diagnosis, and case was confirmed by ADDM Network clinical review. New case definition: child 1) has received a written ASD diagnosis in an evaluation, 2) has received a special education classification of ASD, or 3) has been assigned an International Classification of Diseases, Ninth Revision, code for ASD (including administrative or billing data).

^bIncludes ADDM Network study areas in Arizona, Arkansas, Colorado, Georgia, Maryland, Minnesota, Missouri, New Jersey, North Carolina, Tennessee, and Wisconsin.

^cOverall percent agreement was 96.2% (56,933 of 59,179); Cohen’s κ = 0.85, sensitivity = 85.7%, positive predictive value = 89.3%.

The 8,783 children meeting the new case definition had 1 or more documented indicators of ASD: 6,954 children (79.2%) had a written ASD diagnosis, 5,125 (58.4%) had a special education classification of ASD, and 3,988 (45.4%) had an ICD code for ASD contained in administrative records. Most children had more than 1 indicator of ASD; 3,089 (35.2%) had 1 indicator, 4,104 (46.7%) had 2, and 1,590 (18.1%) had all 3 (data not shown). Most of the 9,153 children meeting the previous case definition also had 1 or more documented indicators of ASD: 2,251 (24.6%) had only 1 indicator, 4,007 (43.8%) had 2 of the 3 indicators, and 1,587 (17.3%) had all 3. Importantly, 1,308 children (14.3%) had no documented indicators for ASD, but the clinicians defined them as cases based solely on their review of abstracted developmental evaluations (data not shown).

Results were consistent between the 2 case definitions. Overall prevalence from 8 sites for surveillance year 2014 was 17.3 cases/1,000 children and 17.2 cases/1,000 children for the previous and new case definitions, respectively. For 2016, overall ASD prevalence was 18.5 cases/1,000 children and 17.3 cases/1,000 children for the previous and new case definitions, respectively. ASD prevalence ratios by race/ethnicity or sex were similar between the 2 case definitions (Table 2). The median age at first evaluation remained unchanged in both years, and the median age at first ASD diagnosis and the proportion of children with co-occurring intellectual disability were also similar under the previous and new case definitions. The amount of missing information varied by indicator and case definition. Age at first evaluation was complete under both case definitions, since it required linkage to an in-state birth certificate. Combining both years, age at first ASD diagnosis was missing 25.9% of the time under the previous case definition and 20.8% of the time under the new case definition. Information on intellectual disability was missing 22.3% of the time under the previous case definition and 25.2% of the time under the new case definition.

Table 2.

Comparison of ADDM Network Indicators of Autism Spectrum Disorder Using Previous and New Case Definitions at ADDM Network Sites^a, Surveillance Years 2014 and 2016

Finding or Indicator	ADDM Network Surveillance Year
	2014				2016
	Previous Case Definition	95% CI	New Case Definition	95% CI	Previous Case Definition	95% CI	New Case Definition	95% CI
ASD prevalence, no. of cases/1,000 children aged 8 years
Total	17.3	16.8, 178	17.2	16.7, 177	18.5	18.0, 19.1	17.3	16.8, 17.8
By sex
Male	27.7	26.8, 28.7	27.7	26.7, 28.6	29.7	28.8, 30.6	27.8	270, 28.7
Female	6.4	6.0, 6.9	6.3	5.9, 6.8	6.9	6.5, 74	6.3	5.9, 6.7
By race/ethnicity
Non-Hispanic White	17.3	16.6, 18.1	17.1	16.3, 179	18.5	178, 19.3	17.0	16.4, 177
Non-Hispanic Black	16.1	15.1, 171	16.2	15.2, 172	18.3	172, 19.4	16.8	15.8, 179
Hispanic	15.0	13.9, 16.2	14.2	13.1, 15.4	15.4	14.4, 16.4	14.3	13.4, 15.3
ASD prevalence ratio
Male:female	4.30	3.97, 4.66	4.37	4.03, 4.74	4.27	3.99, 4.59	4.42	4.10, 4.76
White:Black	1.08	1.00, 1.16	1.06	0.98, 1.14	1.01	0.94, 1.09	1.01	0.94, 1.09
White:Hispanic	1.15	1.05, 1.26	1.20	1.10, 1.32	1.20	1.12, 1.30	1.19	1.10, 1.28
Hispanic:Black	0.93	0.85, 1.03	0.88	0.79, 0.97	0.84	0.77, 0.92	0.85	0.78, 0.93
Other indicators
Age at first evaluation, monthsb,c	41	40, 42	41	39, 42	40	39, 41	40	39, 41
Age at ASD diagnosis, monthsb,d	52	51, 53	53	51,54	51	50, 53	52	51, 53
% with known IQ ≤70e	33	32, 35	34	32, 36	33	32, 35	35	33, 37

Abbreviations: ADDM, Autism and Developmental Disabilities Monitoring; ASD, autism spectrum disorder; CI, confidence interval; IQ, intelligence quotient.

^aEight ADDM Network study areas (Arkansas, Georgia, Maryland, Minnesota, New Jersey, North Carolina, Tennessee, and Wisconsin) were included in 2014, and 11 areas (Arizona, Arkansas, Colorado, Georgia, Maryland, Minnesota, Missouri, New Jersey, North Carolina, Tennessee, and Wisconsin) were included in 2016.

^bValues are expressed as median and bootstrap 95% CI.

^cCalculated among children with linked in-state birth certificates.

^dHistorically, the ADDM Network reports the median age of ASD diagnosis among children who have a diagnosis—children who do not have a diagnosis are excluded. In 2014, 25.4% of children meeting the previous case definition and 23.0% of children meeting the new case definition did not have a known age of ASD diagnosis and were excluded. In 2016, 26.3% and 19.0% were excluded, respectively.

^eChildren with unknown or missing cognitive function data were excluded. In 2014, 21.8% of children meeting the previous case definition and 25.5% of children meeting the new case definition were excluded. In 2016, 22.7% and 25.0% were excluded, respectively.

When the new case definition was applied to children classified as ASD cases in previous ADDM Network surveillance years, the proportion who would have met the new case definition increased over time (Table 3). Only 64% of surveillance year 2000 ASD cases (based on the DSM-IV-TR) would have met the new case definition, whereas 85% of 2016 ASD cases met the new case definition. When considering only clinical ASD diagnoses, there was a similar pattern of increases over the same time period (Table 3, last column).

Table 3.

Among Children Meeting the Previous Case Definition of Autism Spectrum Disorder, Proportion of Children Meeting the New Case Definition and Proportion With an Autism Spectrum Disorder Diagnosis, by Year, ADDM Network, 2000–2016^a

ADDM Network Surveillance Year	No. of 8-Year-Old Children	ASD Status Among Children Meeting the Previous Case Definition
		% Meeting New Case Definition	95% CI	% With Written ASD Diagnosis	95% CI
DSM-IV-TR
2000	187,761	64.0	61.3, 66.7	53.1	50.4, 55.9
2002	407,578	73.8	72.1,75.4	65.1	63.3, 66.9
2004	172,335	79.1	76.8, 81.2	66.0	63.5, 68.5
2006	307,790	77.3	75.7 78.8	67.7	65.9, 69.4
2008	307,757	78.7	77.4, 79.9	68.8	673, 70.2
2010	363,749	80.4	79.3, 81.5	71.6	70.4, 72.8
2012	346,978	82.4	81.3, 83.4	73.6	72.3, 74.8
2014	234,070	82.0	80.9, 83.1	70.0	68.6, 71.3
DSM-5
2014	234,070	87.1	86.1, 88.0	74.8	73.6, 76.0
2016	275,419	84.5	83.5, 85.4	73.7	72.5, 74.9

Abbreviations: ADDM, Autism and Developmental Disabilities Monitoring; ASD, autism spectrum disorder; CI, confidence interval; DSM-IV-TR, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision; DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition.

^aSites and study areas vary from year to year. For comparability and consistency, the 2014 DSM-IV-TR and DSM-5 data correspond to the same sites and study areas as those used elsewhere in this paper. “New case definition” includes children with written ASD diagnostic statements in evaluations, an autism classification in special education, or an International Classification of Diseases, Ninth Revision, code for ASD. Data for the DSM-5 case designation were available only for the year 2014, when clinicians assessed records for meeting both DSM-IV-TR and DSM-5 criteria.

We compared characteristics of children according to whether they met the previous case definition only, the new case definition only, or both case definitions (Table 4). By definition, none of the children who met the previous case definition but not the new case definition had a written diagnosis, ICD code, or special education classification of ASD, and all met the ADDM Network DSM-5 ASD behavioral criteria. In contrast, 14.8% of the children who met only the new definition met the behavioral criteria, and 18.4% had a written diagnosis despite not being considered cases under the previous definition. The proportions of children who had ever had an autism diagnosis ruled out were similar between the previous-only and new-only groups (22.0% vs. 21.0%; P = 0.68); in the “met both” group, only 8% of children had ever had ASD ruled out. The groups were comparable in terms of sex and race/ethnicity; however, a greater percentage of children had a confirmed IQ less than or equal to 70 in the “met both” group. Service provider records from children in the “new-only” group were more likely to be missing, empty, or inaccessible to the ADDM Network site. Of children in the “new-only” group, 34% did not have information in their records requiring abstraction under the previous method; they also had the highest proportion of children with unknown IQ scores (53%).

Table 4.

Characteristics of Children Meeting Either the Previous Case Definition of Autism Spectrum Disorder or the New Case Definition, ADDM Network, 2014 and 2016^a

Group or Characteristic	Case Definition Status						χ2 P Value (Previous Definition vs. New Definition)
	Met Previous Definition Only (n = 1,308)		Met New Definition Only (n = 938)		Met Both Definitions (n = 7845)
	No. of Children	% of Total	No. of Children	% of Total	No. of Children	% of Total
Written ASD diagnosis in evaluation	—b	0	173	18.4	6,781	86.4	—b
ICD code for ASD	—b	0	517	55.1	3,471	44.2	—b
ASD special education classification	—b	0	351	37.4	4,774	60.8	—b
Sufficient behavioral symptoms for ADDM Network DSM-5 ASD	1,308	100.0	139	14.8	6,620	84.4	—b
Abstracted evaluation data	1,308	100.0	622	66.3	7,845	100.0	<0.001
Evaluation in which examiner ruled out ASD	288	22.0	197	21.0	634	8.1	0.676
Assessed using an ASD test or screener	561	42.9	318	33.9	5,933	75.6	0.005
Sex							0.073
Male	1,013	77.4	741	79.0	6,462	82.4
Female	295	22.6	194	20.7	1,383	17.6
Missing data	0	0	3	0.3	0	0
Race/ethnicity (when knownc)							0.069
Non-Hispanic White	662	51.4	420	52.3	3,958	51.4
Non-Hispanic Black	291	22.6	209	26.0	1778	23.1
Hispanic	222	17.2	123	15.3	1,325	17.2
Other	112	8.7	51	6.4	633	8.2
IQ status							<0.001
Known IQ ≤70	215	16.4	129	13.8	2,139	27.3
Known IQ >70	777	59.4	314	33.5	3,985	50.8
Unknown IQ	316	24.2	495	52.8	1721	21.9
Lack of information in 1 or more sources	125	9.6	426	45.4	1,273	16.2	<0.001
No qualifying evaluations	44	3.4	324	34.5	626	8.0	<0.001
Missing record	52	4.0	123	13.1	518	6.6	<0.001
No access to record or location	32	2.4	35	3.7	208	2.7	0.117

Abbreviations: ADDM, Autism and Developmental Disabilities Monitoring; ASD, autism spectrum disorder; DSM-5, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; ICD, International Classification of Diseases; IQ, intelligence quotient.

^aChildren ascertained only under the new case definition were more likely to have missing data because the previous surveillance method did not require collection of all data elements for these children.

^bChildren could not be identified as having ASD without meeting the new case definition, so statistical testing was not performed for these variables.

^cData on race/ethnicity were missing for 9 children (1.7%) in the previous-only case definition group, 135 children (14.4%) in the new-only group, and 151 children (1.9%) in the met-both group.

Generally, there was consistency between the previous and new prevalence estimates of children with ASD (Figure 1; Pearson’s r = 0.92 in 2014 and Pearson’s r = 0.94 in 2016). The largest differences in prevalence occurred in New Jersey for both 2014 and 2016, which had the greatest number and probability of including children without an existing ASD diagnosis or educational classification of ASD as cases of the ADDM Network sites (Web Table 2). Comparing prevalence estimates based on the new case definition, the previous (DSM-5) case definition, and the prior DSM-IV-TR case definition when all 3 were ascertained in 2014, the new case definition prevalence was highest of the 3 at 3 sites, in the middle at 2 sites, and lowest at 3 sites (Web Table 3). We calculated the upper and lower bounds of the prevalence estimates using the new case definition at the 3 sites where full comparisons were not possible in 2014 (Web Table 4, Web Appendix, Web Figure 1), and all were in line with their 2016 results.

Figure 1.

Comparison of prevalence estimates (number of cases/1,000 children) based on previous and new case definitions of autism spectrum disorder, by Autism and Developmental Disabilities Monitoring (ADDM) Network site, United States, 2014 and 2016. A) Surveillance year 2014 (8 sites; unweighted Pearson’s r = 0.922); B) surveillance year 2016 (11 sites; unweighted Pearson’s r = 0.942).

The new case definition requires less data collection than the previous case definition (Table 5). The number of children with abstracted data is reduced by half, and there are concomitant decreases in the numbers of evaluations and tests that must be abstracted for these children. The number of diagnoses or behavioral findings that necessitate record abstraction decreased from 35 under the previous method to 3 under the new method.

Table 5.

Comparison of Data Collection Requirements for the Previous and New Case Definitions of Autism Spectrum Disorder, ADDM Network, 2014 and 2016

Data Collection Element	Surveillance Method		% Change
	Previous Method	New Method
No. of ASD cases identified	9,153	8,783	−4.0
No. of criteria for record abstraction that abstractors must identify	35	3	−91.4
No. of children whose records need to be abstracted	18,408	8,783	−52.3
No. of developmental evaluations abstracted	94,989	52,719	−44.5
No. of IQ tests abstracted	24,374	12,096	−50.4
No. of adaptive tests abstracted	37,417	23,897	−36.1
No. of autism tests abstracted	19,299	15,678	−18.8
No. of composite record reviews carried out by ADDM Network cliniciansa	23,027	0	−100.0

Abbreviations: ADDM, Autism and Developmental Disabilities Monitoring; ASD, autism spectrum disorder; IQ, intelligence quotient.

^aThe number of clinician reviews is greater than the number of children with abstracted records because the protocol required that some children’s records be reviewed multiple times. The reduction in abstracted tests and evaluations is due to the reduced number of children whose information is abstracted under the new case definition.

DISCUSSION

When the ADDM Network was created in 2000, its methods were designed to address the possibility that practitioners might describe a child with symptoms indicative of ASD but not have the skills or training to diagnose it. However, over the course of 9 surveillance cycles (and 16 years), the proportion of children identified under the previous case definition who had an existing ASD diagnosis or classification increased from 64% to 85%. Given the progressive increases in awareness, training, and autism-specific services, it was important to reevaluate these methods in consideration of present-day knowledge and community practices. We found little difference in the overall prevalence estimates based on the previous and new ADDM Network case definitions for 2014, and a slightly lower prevalence based on the new ADDM Network case definition for 2016. In addition, we found no clear differences by sex or race/ethnicity. There was essentially no difference by median age at first evaluation, median age at first diagnosis, and proportion of children with intellectual disability. Both case definitions failed to identify children who had been identified only by the other method. The new case definition identified some children who were missed by the previous case definition because the child’s evaluations or records could not be obtained. Additionally, a small proportion of cases identified using only the new definition had ASD diagnoses from a community provider who evaluated the child but the ADDM Network clinicians overruled the diagnosis. Likewise, the previous case definition classified some cases by reviewing evaluations and overruling the community clinician who concluded that the child did not have ASD.

Even though the 2 case definitions produced nearly equal results for 2014 and slight differences for 2016, the interpretation of these data may be more useful for public health action under the new case definition versus the previous case definition. For instance, the variability in prevalence across sites has sometimes been described as a failure of the surveillance system to completely ascertain all cases of ASD in communities. Under the new method, the differences in prevalence could be usefully interpreted as differences in community-level practices for identifying ASD or availability of ASD services; a community with low prevalence might highlight a need for better ASD identification in that community. For example, in New Jersey for 2014 and in New Jersey, Minnesota, and North Carolina for 2016, identification in the community was lower than identification by the previous ADDM Network method. Disparities in ASD prevalence between racial or ethnic groups are routinely interpreted as differences in access to care or identification; however, since the previous case definition included children who were not diagnosed (4, 6, 12), disparities in ASD prevalence may not be equivalent to disparities in diagnosing ASD within communities. Despite reductions in racial disparities in ASD prevalence (estimates which include undiagnosed children) (4), another study using the 2014 ADDM Network data found that among children with ASD, non-Hispanic White children were more likely than non-White children to have a formal documented ASD diagnosis (13). A recent study using ADDM Network data also suggested that racial disparities in prevalence are due to unequal access to services (and thus, unequal amounts of available information), which could not be overcome by an in-depth clinician review process (14, 15). The new case definition would seem to be better suited to transparently assessing disparities in actual ASD identification.

The volume of information reviewed by the ADDM Network has dramatically increased over time; for instance, the Georgia ADDM Network site reviewed 8 times as many evaluations in 2010 as in 2000 (9,811 vs. 1,152) (16). Prior attempts at improving efficiency in the ADDM Network focused primarily on the clinician review workflow. In 2002, the ADDM Network implemented a process called “streamlining,” which expedited clinician review for children with an existing autism diagnosis. For these children, less data collection was required, and the average clinician review time was reduced from 47 minutes per child to 20 minutes per child (2). However, this only applied to a fraction of all children, it was accompanied by specialized procedures for “streamlined” cases, and a different subset of variables was available for these cases versus the nonstreamlined cases. More recently, machine learning approaches were evaluated (16, 17); while initial algorithms showed reasonable performance as compared with clinicians, several logistical issues limited the potential to improve timeliness. The algorithms required much of the same manual record abstraction and would increase complexity through the necessity to train and validate machine learning models at each ADDM Network site, and it would still require some (human) clinician review.

In contrast to previous strategies for enhancing efficiency, the new method will considerably reduce data collection burdens and improve overall system simplicity. Under the previous method, training of data abstractors or clinicians in order for them to be deemed “reliable” could take months; an approach using any amount of clinician review would not reduce the training burden for record abstractors or clinicians. The new method greatly eases the process of determining whether a child’s information will be abstracted and eliminates the clinician review component (further reducing the complexity of the database structure and workflow). The previous method’s sequence of abstracting data, printing (to paper or portable document format (PDF)) compiled records, clinical review and scoring of records, and performing data entry for all clinician coding required building and maintaining a complex workflow and custom data system. The new method uses off-the-shelf tools (e.g., REDCap (18, 19) and statistical software) that are currently available at universities and health departments. We anticipate lower operating costs and less system-caused “downtime” compared with the previous system, as well as improved flexibility to modify the system to capture information on emerging issues.

This comparison of case definitions was based on data for the 2 surveillance years using the previous case definition, based on the DSM-5. Consistent with the DSM-5 ASD criteria (3), children automatically met the previous ADDM Network case definition if they had a documented ASD diagnosis; this could have contributed to the closer alignment with community-identified ASD than with the DSM-IV-TR case definition in 2014. Additionally, we were not able to make a direct comparison using data from every site in both years; for 3 sites, we calculated the range of possible outcomes, and the results were compatible with the actual results from those sites in 2016. We would not expect results to be comparable with those of earlier years with the new case definition; the increase in community ASD identification over time has made the new case definition feasible. Trend analyses using a consistent case definition could be performed for the DSM-IV-TR (2000–2014), the DSM-5 (2014–2016), and the new case definition (2014 onward). In addition to using data from all 2016 ADDM Network sites and most 2014 sites, strengths of this analysis include the use of data from previous years and the availability of detailed information with which to characterize the groups that only met 1 case definition. As the new definition is being used in the field, we will be able to assess the amount of time saved; this analysis supported the implementation of the new methods.

In the future, it will be important to conduct validation studies for the ADDM Network, with a focus on measuring undiagnosed children, as well as false-positive ASD diagnoses. This validation would reflect community ASD identification practices and inform communities on needed screening and evaluation services. It might also identify the need for additional community education or training if the rate of false-positive diagnosis (compared with a standardized clinical assessment) is high. There are limited data on the sensitivity of the previous methods for ascertaining undiagnosed children with ASD: a single-site study of 177 children who were born in 1997 and were 8 years of age in 2005 (20). Those authors reported a sensitivity of 0.60 with clinical evaluation as the “gold standard”; missed cases were attributed to surveillance coding rules or missing information from health or education sources (20). Additional studies for estimating the frequency of undiagnosed ASD could be conducted to make statistical adjustments to ASD prevalence data. Additional studies are also needed to continue to assess the performance of administrative ASD classifications in different contexts, such as large claims databases (21).

The new ASD case definition will broaden the types of data sources ADDM Network sites can use to identify all children with ASD, compared with the previous method, which required written evaluations. For instance, statefunded autism service data (e.g., Medicaid) could be used to ascertain additional cases even if the ADDM Network site does not capture any comprehensive written evaluations for these children. The inclusion of new data sources could provide new insights into different paths children and their families take to having ASD diagnosed and disparities in receiving services.

The new ASD case definition also more closely reflects community identification of ASD and should reduce the resource and time requirements for conducting autism surveillance while retaining measurement properties similar to those of the previous case definition. With the adoption of the new method, the ADDM Network will continue to function as an active, multisite surveillance system which utilizes information from multiple sources in a community. It will continue to utilize record review to identify the ages at which children are diagnosed and evaluated, collect information about intellectual ability and other co-occurring diagnoses, and provide a comprehensive description of the numbers and characteristics of children with ASD in multiple communities in the United States. The new ASD case definition reduced the overall prevalence of ASD slightly in 2016 but not in 2014. The new method may slightly reduce estimated prevalence at a few sites, but the benefits of improving the timeliness of public health surveillance data on ASD will allow more rapid identification of changes in prevalence that could be related to modifiable risk factors, improving the utility of surveillance data’s use in planning for needed services in the community.

Abbreviations:

ADDM

Autism and Developmental Disabilities Monitoring

ASD

autism spectrum disorder

DSM-IV-TR

Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision

DSM-5

Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition

ICD

International Classification of Diseases

IQ

intelligence quotient