Surveillance of ADHD Among Children in the United States: Validity and Reliability of Parent Report of Provider Diagnosis

Abstract

Objective:

To evaluate the appropriateness of parent-reported diagnosis of ADHD as a surveillance tool.

Method:

We assessed agreement over time and concordance of parent-reported diagnosis against Diagnostic and Statistical Manual (DSM)-based criteria. We compared concordance of diagnosis and DSM-based criteria by child characteristics, including treatment.

Results:

Among parents who reported their child had ADHD, 95.7% reported it again 2 years later. Comparing diagnosis with DSM-based criteria, specificity and negative predictive value were high, sensitivity was moderate, and positive predictive value was low. Most children with an ADHD diagnosis who did not meet DSM-based criteria met sub-threshold criteria or took medication for ADHD. Concordance differed by child characteristics and treatment.

Conclusion:

Parent-reported diagnosed ADHD is reliable over time. Although differences in parent-reported diagnosis and DSM-based criteria were noted, these may reflect children with milder symptoms or treated ADHD. Parent-report of child ADHD ever diagnosis may be a good single-item indicator for prevalence.

Introduction

ADHD is associated with an increased risk for many adverse outcomes, such as academic problems, poor social functioning, drug use/addictive behavior, antisocial behavior, lower self-esteem, and obesity, that often persist throughout adulthood (Loe & Feldman, 2007; Shaw et al., 2012). Surveys conducted to obtain nationally-representative estimates of ADHD prevalence, such as the National Survey of Children’s Health and the National Health Interview Survey, often rely on parent report of an ADHD diagnosis received from a healthcare provider to identify children with ADHD (Danielson et al., 2018; Zablotsky, Black, et al., 2019). ADHD prevalence estimates provide essential information for clinicians, public health professionals, educators, and families (Holbrook et al., 2017).

A survey asking a parent whether their child has ever been told by a healthcare provider that their child had ADHD (ADHD ever diagnosis) is a relatively quick and simple way to ascertain case status compared to administering symptom- and impairment-based interviews to determine whether the child meets Diagnostic and Statistical Manual (DSM; American Psychiatric Association [APA], 1994) diagnostic criteria; however, researchers have raised concerns regarding limitations of parent-reported ADHD diagnosis as an appropriate surveillance tool (Getahun et al., 2013; Song et al., 2018). Prevalence estimates based on parent-reported ADHD diagnosis rely on an appropriate diagnosis from a clinician and parents accurately reporting clinician diagnoses, and are subject to recall or social desirability bias (Holbrook et al., 2017). The Centers for Disease Control and Prevention (CDC)’s National Center for Health Statistics studied parent-reported ADHD and found that some parents misreported diagnoses because they disagreed with the provider, were not given a concrete diagnosis, or were given differing diagnoses by different providers (Noel et al., 2012). Using medical records from a Southern California population of children, Getahun et al. (2013) found a lower ADHD prevalence compared to previous U.S. estimates from the NSCH. The authors criticized estimates obtained from parent report as over-representing ADHD, primarily due to misclassification of the disorder. However, a follow-up study conducted by Visser et al. (2013) found that when considering insurance status, geography, and age of the child, ADHD prevalence estimates by parent-reported diagnosis are in line with the estimates Getahun et al. (2013) calculated using medical records, providing evidence for the validity of the parent-reported diagnosis indicator. Nevertheless, parent report of receipt of a diagnosis as a surveillance indicator will not account for children with undiagnosed ADHD which may be particularly prevalent among some demographic subgroups (Coker et al., 2016; Holbrook et al., 2017).

Although prior work supports the validity of parent-reported ADHD diagnosis, understanding whether parents consistently report their child’s ADHD diagnosis across time, as well as how parent-reported ADHD diagnosis compares to the application of DSM-based diagnostic criteria, would help inform future use of this method to assess ADHD prevalence. If appropriate, this report of ever ADHD diagnosis is simple to obtain from parents, and considerably more time efficient compared to a DSM-based diagnostic interview.

Objectives

Study 1.

Our first objective was to assess agreement over time of parent-report of child ADHD ever diagnosis by a doctor or other healthcare provider, and to test whether severity of symptoms or child age were associated with agreement over time. We expected overall agreement would be high; however, we hypothesized that parents who rated their child’s ADHD as less severe would be less likely to report their child as having ever been diagnosed with ADHD 2 years later, as compared to parents who rated their child’s ADHD symptoms as severe. Additionally, we hypothesized that parents of younger children would be more likely to disagree on their child’s ADHD diagnosis between time points, compared to parents of older children, as younger children are more likely to be at the beginning of their diagnostic experience when an ADHD diagnosis may be less stable.

Study 2.

Our second objective was to determine the validity of parent-report of child ADHD ever diagnosis compared to DSM-based criteria. We aimed to assess concordance of diagnosis receipt and presence of ADHD symptoms and impairment overall and by child and family characteristics to identify demographic subgroups for whom there may be gaps in appropriate diagnosis or treatment of symptoms. We hypothesized that a greater proportion of children with a parent-reported ADHD diagnosis who did not meet DSM-based criteria would be receiving behavioral or medication treatment (potentially appropriately treated) compared to those without a diagnosis who did meet DSM-based criteria (potentially undiagnosed). Additionally, we hypothesized that non-White children would more often meet DSM-based criteria without a parent-reported ADHD diagnosis compared to White children, in line with prior work demonstrating potential racial and ethnic disparities in the diagnosis of ADHD for non-Hispanic Black and Hispanic children (Coker et al., 2016).

We present methods and results from the two studies separately to address our objectives.

Study 1. Assess Agreement Over Time of Parent-Reported ADHD Diagnosis

Method

Sample and measures.

To assess agreement over time of a parent-reported diagnosis of ADHD by a doctor or other healthcare provider, we used data from the 2011 to 2012 National Survey of Children’s Health (NSCH; CDC National Center for Health Statistics, 2017) and the 2014 National Survey of the Diagnosis and Treatment of ADHD and Tourette Syndrome (NS-DATA; CDC National Center for Health Statistics, 2015). The 2011 to 2012 NSCH was a telephone-based cross-sectional survey of parents of 95,677 children, designed to be representative of all noninstitutionalized children aged 0 to 17 years in the United States. The survey was conducted by the U.S. CDC and funded and directed by the Health Resources and Services Administration’s Maternal and Child Health Bureau. The 2014 NS-DATA was a follow-up telephone survey of parents of children who were reported to have ever been diagnosed with ADHD or Tourette syndrome on the 2011 to 2012 NSCH survey. Among the 6,034 eligible NSCH respondents, 3,582 responded to the screener for inclusion in NS-DATA (percent of eligible respondents screened = 59.4%). In both the NSCH and NS-DATA surveys, parents were asked, “has a doctor or other health care provider ever told you that [your child] had attention deficit disorder or attention deficit/hyperactivity disorder, that is, ADD or ADHD?”. Parents also reported on child age and if the child had current ADHD. Parents who reported their child had current ADHD on the 2011 to 2012 NSCH were also asked about their child’s ADHD severity (mild, moderate, or severe; n = 2,921).

Statistical analyses.

We performed analyses using Stata 14 statistical software (StataCorp, 2015). To examine agreement over time of a parent-reported ever diagnosis of ADHD by a doctor or other healthcare provider, we assessed whether participants screened for inclusion to the 2014 NS-DATA based on the 2011 to 2012 NSCH reported their child was ever diagnosed with ADHD consistently across both time points. We also tested whether parent agreement on ADHD diagnosis varied by child age (in years; 2–5, 6–11, and 12–17) and severity of ADHD symptoms (mild vs. moderate/severe) reported in the 2011 to 2012 NSCH using a design-based F test statistic.

Results

Of the 3,582 respondents with a parent-reported ever ADHD diagnosis on NSCH who responded to the screener for inclusion in NS-DATA, 95.7% (n = 3,428) of parents reported that their child had ever been diagnosed with ADHD by a doctor or other healthcare provider at the NS-DATA interview 2 years later; there were no significant differences in agreement detected by child age. Of the 2,921 parents who reported their child had current ADHD and were asked about symptom severity on the 2011 to 2012 NSCH, parents who rated their child’s symptoms as moderate or severe in 2011 to 2012 (98.4%) were more likely to consistently report their child ever had ADHD when they were screened for inclusion to NS-DATA in 2014, compared to parents who had rated their child’s symptoms as mild (94.4%; p = .02). See Table 1 for a summary of results.

Table 1.

Weighted Percent Agreement of Parent Report of an Ever ADHD Diagnosis From 2011 to 2012 National Survey of Children’s Health (NSCH) to 2014 National Survey of the Diagnosis and Treatment of ADHD and Tourette Syndrome (NS-DATA) Among Parents Who Reported ADHD Diagnosis at NSCH, by Parent-Reported Child Age and Symptom Severity at NSCH.

	Reported ever ADHD at NS-DATA
	Yes (agree)	No (disagree)
	% [95% CI]	% [95% CI]	p-Value†
Child age			.10§
2–5 years	93.4 [87.7, 96.6]	6.6 [3.4, 12.3]
6–11 years	96.7 [94.9, 97.8]	3.3 [2.2, 5.1]
12–17 years	96.7 [90.7, 98.9]	3.3 [1.1, 9.3]
Symptom severity*			.02¶
Mild	94.4 [88.7, 97.4]	5.6 [2.6, 11.3]
Moderate or severe	98.4 [96.5, 99.2]	1.6 [0.7, 3.5]

^*Only respondents who reported their child had current ADHD in 2011 to 2012 were asked to rate their child’s ADHD severity on the 2011 to 2012 NSCH (n = 2,921).

^†p-Value calculated from design-based F test statistic comparing agreement between 2011 and 2012 NSCH and 2014 NS-DATA.

^§Designed-based F (1.485,093.86) = 2.54.

^¶Designed-based F (12,819) = 5.77.

Study 2. Determine Validity of Parent-Reported ADHD Diagnosis

Method

Sample.

For our second objective, to determine the validity of parent-report of child ADHD ever diagnosis compared to DSM-based diagnostic criteria, we used data from families participating in the Project to Learn about Youth—Mental Health (PLAY-MH; Danielson et al., 2021). Detailed methodology for PLAY-MH is described elsewhere (Danielson et al., 2021). Briefly, PLAY-MH is a multi-state, community-based, two-stage study using parent and teacher ratings of child mental health, including participants from two time points in South Carolina (SC); with an initial sample (N = 276) and a second larger sample referred to as the Project to Learn about Youth Replication (SC Re-PLAY; N = 571; Wanga et al., 2022), and one time point in Florida (FL; N = 293), Colorado (CO; N = 239), and Ohio (OH; N = 162). We used the largest PLAY-MH sample, SC Re-PLAY, for our primary analysis. We also conducted replication analyses for CO, FL, OH, and the original SC sample to determine if patterns of results were consistent across sites and samples.

Measures.

In stage 1 of the study, children were screened for inclusion in stage 2 based on up to three symptom-based questionnaires, the Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001) and/or the Behavior Assessment System for Children—Second Edition, Behavioral and Emotional Screening System (BASC-2 BESS; Kamphaus & Reynolds, 2007), and the Proxy Report Questionnaire for tics (PRQ; Cubo et al., 2011). The screening instruments were completed by teachers to assess student externalizing, internalizing, and tic behaviors in the classroom. The teacher SDQ and teacher BASC-2 BESS have both been shown to have high internal reliability and high test-retest reliability (Goodman, 2001; Kamphaus & Reynolds, 2007). Children classified as high and low risk for having an externalizing, internalizing, or tic disorder based on teacher responses to the SDQ and/or BASC-2 BESS and the PRQ were stratified into groups based on biological sex and school level (elementary vs. middle/high school) and sampled for participation in stage 2 of the study (Danielson et al., 2021). In stage 2, parents completed a questionnaire that included the question, “has your child ever been diagnosed with a mental, emotional, or behavioral disorder?”, followed by “if yes, what has he/she been diagnosed with? (select all that apply)” and “does he/she currently have this disorder?”. Parents had the option of selecting a variety of mental, emotional, and behavioral disorders including “ADHD or attention deficit disorder (ADD).”

During stage 2, parents also completed the “past-year” Diagnostic Interview Schedule for Children—Version IV (DISC-IV), including the ADHD module, that assessed symptom and impairment criteria from the DSM-IV (Shaffer et al., 2000). DISC-IV interviews were administered by trained interviewers supervised by a licensed psychologist or psychiatrist. Meeting impairment criteria was based on parent report of at least two domains with moderate impairment or at least one domain with severe impairment out of six domains, including: (1) caretaker annoyed or upset because of disorder symptoms, (2) symptoms kept child from doing things or going places with family, (3) symptoms kept child from doing things or going places with other children of the same age, (4) disorder symptoms caused problems with schoolwork or work, (5) teachers annoyed or upset with child because of disorder symptoms, and (6) disorder symptoms seemed to make child feel bad or very bad. Based on prior studies, the DISC-IV has moderate to high test-retest reliability, high agreement with clinician diagnoses, and demonstrates predictive validity for ADHD (Fisher et al., 1993; Rolon-Arroyo et al., 2016; Schwab-Stone et al., 1996; Shaffer et al., 2000).

Children were considered to meet our ADHD case definition if they met both symptom and impairment criteria on the parent-reported DISC-IV and had at least two or more teacher-reported ADHD symptoms on either the BASC-2-BESS or the SDQ. The items and responses on the teacher SDQ that we included to measure ADHD symptoms were: (1) restless, overactive, cannot stay still for long (certainly true); (2) constantly fidgeting or squirming (certainly true); (3) easily distracted, concentration wanders (certainly true); (4) thinks things out before acting (not true); and (5) good attention span, sees work through to the end (not true). The items and responses on the teacher BASC-2-BESS considered to measure ADHD symptoms were: (1) pays attention (never); (2) is well organized (never); (3) completes assignments incorrectly because of not following instructions (always or often); (4) is easily distracted from class work (always or often); (5) disrupts other children’s activities (always or often); (6) has trouble concentrating (always or often); and (7) has a short attention span (always or often). We included both parent and teacher report to follow DSM-IV criteria requiring impairment across multiple settings (i.e., school and home; APA, 1994), in line with prior work (Danielson et al., 2021; Wolraich et al., 2014). The case definitions for CO and the original SC sample differed slightly due to which screeners were collected with usable data in Stage 1, using only the BASC-2-BESS and the SDQ for teacher report of symptoms, respectively.

In stage 2, parents also reported demographic characteristics (child sex, age, race/ethnicity, insurance status, parental education, and income level) and whether the child received mental health treatment (psychosocial therapy or medication for ADHD). Children were considered to have received psychosocial therapy if their parent reported that their child received at least one specified type of psychosocial therapy in the past year, including individual counseling, family group therapy (specified as “sessions you and your child attend together”), group counseling, social skills training, cognitive behavioral therapy, and parent training. The question about parent training was directed to the parent, “In the past year, have you received parent training as related to your child’s mental, emotional, or behavioral problem? (parent training includes sessions that you attend without your child and you learn strategies to try at home to help change your child’s behavior).” Questions about receipt of psychosocial therapy were not specific to ADHD. Children were considered to have received medication for ADHD if the parent responded “yes” to the question “Is he/ she [the child] currently taking medication to treat this disorder? Attention-deficit/ hyperactivity disorder (ADHD) or attention deficit disorder (ADD)?”

Statistical analyses.

We performed data management using SAS software version 9.4 (SAS Institute Inc., 2013) and weighted analyses using SAS-callable SUDAAN software (RTI International, 2018) to account for the complex sample design. Sample weights were developed to adjust for differential probability of selection, non-response, and demographic characteristics of the sample populations in order to produce estimates that are representative of the participating school districts (Danielson et al., 2021). We calculated weighted sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of parent-reported ever ADHD diagnosis compared to whether the child met DSM-based criteria for ADHD (our case definition). We considered the child’s ADHD diagnosis and DSM-based criteria to be “concordant” if the parent report of child ever diagnosis (Dx) was consistent with meeting DSM-based symptom and impairment criteria (Sx; i.e., Dx+/Sx+ or Dx−/Sx−); we considered a child “discordant” if parent report of ever diagnosis was not consistent with DSM-based symptom and impairment criteria (i.e., Dx+/Sx− or Dx−/Sx+).

To explore the possibility that concordance better reflected parent reports of current ADHD diagnosis rather than validity of parent report of child ADHD ever diagnosis, we also calculated weighted sensitivity, specificity, PPV, and NPV estimates for parent-reported current ADHD diagnosis compared to the same case definition using DSM-based criteria for ADHD. If concordance better reflected current ADHD diagnosis, we would expect a higher PPV due to a fewer number of “false positives” (i.e., fewer number of children Dx+/Sx−) for those with a current ADHD diagnosis compared to those with an ever ADHD diagnosis, and higher specificity due to a greater number of “true negatives” (i.e., greater number of children Dx−/Sx−) for those with a current ADHD diagnosis compared to those with an ever ADHD diagnosis.

We assessed potential demographic differences by concordance status by child age (mean age and age groups 5–11 vs. 12–19 years), child sex, child race/ethnicity (non-Hispanic White vs. non-Hispanic Black/Hispanic/Other), whether the child had inadequate or no insurance (yes vs. no), whether the child had public insurance (i.e., Medicaid; yes vs. no), highest level of parental education (high school diploma or less vs. at least some college or technical school), and whether the household income was at or below 200% vs. above 200% of the federal poverty level (FPL).

We also examined which aspects of the case definition were contributing to discordant Dx+/Sx− classifications in a series of post-hoc analyses. We calculated individual weighted percentages and cumulative weighted percentages for four scenarios representing DISC-IV sub-threshold diagnostic- and symptom-based criteria (see Figure 1): (A) child meeting parent-reported DSM-based criteria only (i.e., meeting criteria on the DISC-IV module only, but not the teacher-reported symptoms criterion of the study case definition); (B) child meeting between three and five DISC-IV symptoms of inattention or hyperactivity/impulsivity (vs. six required to meet full diagnostic criteria) with one severe or two moderate ratings of impairment; (C) child meeting three or more DISC-IV symptoms of inattention or hyperactivity/impulsivity with at least one rating of mild impairment; and (D) child meeting three or more DISC-IV symptoms of inattention or hyperactivity/impulsivity without any reported impairment. For children with Dx+/Sx− we also assessed factors related to interview timing that might contribute to discordance. We calculated the proportion of children who: (1) were diagnosed with ADHD more than 3 years prior to the DISC-IV interview administration who may no longer have current ADHD; (2) were interviewed more than 1 year apart between stage 1 and stage 2, who may have had teacher-reported symptoms in stage 1 that were no longer present at the time of the DISC-IV interview in stage 2; and (3) received the DISC-IV interview in the summer (June–August) when they might show fewer ADHD symptoms (Kovalenko et al., 2000) compared to months where children are in school. Additionally, we explored whether children Dx+/Sx− were receiving medication for ADHD or psychosocial therapy to identify whether receipt of treatment might explain the discordant findings (if children were appropriately treated, they may not currently meet diagnostic criteria as a result).

Figure 1.

Prevalence of treatment, symptoms, and diagnostic characteristics among children with a parent-reported ADHD diagnosis but who do not meet the DSM-based diagnostic criteria case definition for ADHD.

Note. Dx+/Sx− = with ADHD diagnosis, without ADHD symptoms; CI = confidence interval.

*Total n’s shown are unweighted; all percentages are weighted to account for complex sampling.

^†We tested four sub-threshold definitions (Criteria A–D) for children who did not meet our case definition for ADHD based on parent- and teacher-reported ADHD symptoms. Criterion A includes children who had parent-reported symptoms that would meet our case definition but did not meet the teacher-reported symptom criteria. Criterion B includes children who had three to five DISC-IV symptoms of inattention or hyperactivity/impulsivity (vs. six required to meet full diagnostic criteria) with one severe or two moderate ratings of impairment, but did not meet our case definition or criterion A. Criterion C includes children who had three or more DISC-IV symptoms of inattention or hyperactivity/impulsivity with at least one rating of mild impairment, but did not meet our case definition or criteria A and B. Criterion D includes children who three or more DISC-IV symptoms of inattention or hyperactivity/impulsivity without any reported impairment, but not meet our case definition or criteria A and C.

^§Child was considered to have received treatment if their parent reported they were currently taking medication for ADHD or that they received psychosocial therapy in the past year (includes individual counseling, family group therapy, group counseling, social skills training, cognitive behavioral therapy, and parent training related to child’s mental, emotional, or behavioral problem). Treatment status was missing on one child unexplained by sub-threshold diagnostic criteria A to D.

^¶Estimate is unstable; relative standard error = 36.1%.

We calculated prevalence estimates and 95% CIs for ADHD for parent-reported diagnosis and symptom-based diagnostic criteria. To determine if patterns of results were consistent across sites and samples, we calculated weighted sensitivity, specificity, PPV, NPV, and prevalence across the remaining four PLAY-MH samples (CO, FL, OH, and the original SC sample).

Results

Comparing parent-report of child ever ADHD diagnosis with a parent- and teacher-reported DSM-based ADHD case definition, specificity and NPV were high (79.7% and 96.5%), sensitivity was moderate (67.1%), and PPV was low (22.7%), shown in Table 2. Estimates were consistent for parent-reported current ADHD diagnosis, with comparably high specificity and NPV (80.6% and 96.8%), and moderate sensitivity and low PPV (70.0% and 24.2%). Examining unweighted numbers of children, differences between ever ADHD diagnoses and current ADHD compared to DSM-based criteria were driven by two children moving from Dx+/Sx− to Dx−/Sx−.

Table 2.

Validity of Parent Report of Child Ever Receiving a Diagnosis of ADHD and Having Current ADHD Compared to Whether the Child Met DSM-Based Criteria for ADHD^* in the South Carolina Replication Sample of the Project to Learn About Youth Mental Health (SC Re-PLAY).

	Parent report of child ever ADHD diagnosis	Parent report of child current ADHD diagnosis
Validity measures	Est [95% CI]	Est [95% CI]
Sensitivity	67.1 [48.9, 82.2]	70.0 [53.6, 83.4]
Specificity	79.7 [73.6, 85.0]	80.6 [74.6, 85.8]
PPV	22.7 [12.9, 35.3]	24.2 [13.9, 37.3]
NPV	96.5 [94.2, 98.1]	96.8 [94.6, 98.3]

Note. PPV = positive predictive value; NPV = negative predictive value.

^*Child met DSM-based criteria for ADHD in the past year based on parent report on the Diagnostic Interview Schedule for Children—Version IV (DISC-IV), ADHD module and teacher report on the Behavior Assessment System for Children—Second Edition, Behavioral and Emotional Screening System (BASC-2 BESS), or the Strengths and Difficulties Questionnaire (SDQ).

Concordance status by demographic factors.

Most children had concordant ever diagnosis and symptoms (78.7%); however, 18.6% of children had Dx+/Sx− and 2.7% of children had Dx−/Sx+ (Table 3). There were demographic differences in concordance status by child sex, whether the child had public insurance, and federal poverty level, shown in Table 3. Most estimates for children Dx−/Sx+ were unstable with a relative standard error between 30% and 50%; therefore, results for this group should be interpreted with caution. Compared to males, females were more often concordant on having an ever ADHD diagnosis and current symptoms (85.8% vs. 72.2%), and less often Dx+/Sx− (12.9% vs. 23.9%) and Dx−/Sx+ (1.3% vs. 3.9%; p = .009). Children without public insurance were more often concordant (85.7% vs. 71.0%) and Dx−/Sx+ (3.7% vs. 1.6%) and less often Dx+/Sx− (10.6% vs. 27.4%) compared to children with public insurance (p = .003). Children living in higher income households (>200% FPL) were more often concordant (87.3% vs. 75.4%) and less often Dx+/Sx− (10.1% vs. 21.5%) compared to those living in lower-income households (≤200% FPL; p = .003). There were no significant differences in concordance by child mean age, age group, race/ethnicity, inadequate or no insurance, or parental educational attainment. Most children who were Dx+/Sx+ (83.0%, 95% CI [66.9, 93.4]) and Dx+/Sx− (78.9%, 95% CI [67.1, 87.9]) were receiving behavioral or ADHD medication treatment compared to relatively few children who were Dx−/Sx+ (9.3%, 95% CI: [0.8, 32.4]; data not shown).

Table 3.

Comparison of Concordant and Discordant Parent-Reported Ever Child ADHD Diagnosis and Parent- and TeacherReported ADHD Symptoms by Selected Demographic Characteristics.

	Ever ADHD diagnosis and symptoms match	Ever ADHD diagnosis, without ADHD symptoms	No ADHD diagnosis, with ADHD symptoms
	n = 427*	n = 112*	n = 22*
Demographic characteristic	% [95% CI]†	% [95% CI]†	% [95% CI]†	Total	p-Value§
Overall	78.7 [73.0, 83.7]	18.6 [13.7, 24.3]	2.7 [1.5, 4.4]	100%
Child sex					.009§
Male	72.2 [62.4, 80.6]	23.9 [15.7, 33.8]	3.9 [1.9, 7,2]¶	52.1 [47.9, 56.3]
Female	85.8 [80.9, 89.7]	12.9 [8.9, 17.8]	1.3 [0.3, 3.5]¶	47.9 [43.7, 52.1]
Child mean age	12.4 ± 0.3	13.3 ± 0.8	10.5 ± 1.2	12.5 ± 0.3	.191
Child age group (years)					.348
5–11	79.4 [74.6, 83.6]	16.8 [13.0, 21.3]	3.8 [2.0, 6.5]	42.1 [38.0, 46.3]
12–19	78.2 [68.6, 86.0]	19.9 [12.2, 29.6]	1.9 [0.6, 4.6]¶	57.9 [53.7, 62.0]
Child race/ethnicity**					.76
Non-Hispanic White	77.8 [69.5, 84.7]	19.3 [12.5, 27.6]	3.0 [1.3, 5.7]¶	62.2 [56.5, 67.6]
Other	80.3 [72.7, 86.5]	17.5 [11.4, 25.2]	2.2 [0.8, 4.8]¶	37.8 [32.4, 43.5]
Inadequate or no insurance††					.448
Yes	81.5 [74.2, 87.4]	15.0 [9.9, 21.4]	3.5 [0.9, 8.9]¶	29.7 [24.7, 35.0]
No	77.6 [69.9, 84.1]	20.1 [13.7, 28.0]	2.3 [1.1, 4.3]	70.3 [65.0, 75.3]
Public insurance§§					.003§
Yes	71.0 [61.4, 79.4]	27.4 [19.0, 37.3]	1.6 [0.5, 3.7]	47.7 [41.7, 53.8]
No	85.7 [80.2, 90.2]	10.6 [6.8, 15.5]	3.7 [1.6, 7.0]¶	52.3 [46.2, 58.3]
Parent college or technical school diploma					.124
Yes	80.7 [71.5, 87.9]	15.7 [8.8, 25.1]	3.6 [1.7, 6.6]¶	56.3 [50.4, 62.1]
No	76.1 [69.1, 82.2]	22.4 [16.4, 29.5]	1.5 [0.4, 3.7]¶	43.7 [37.9, 49.6]
Federal poverty level¶¶					.003§
≤200% (lower income)	75.4 [69.8, 80.4]	21.5 [16.9, 26.8]	3.0 [1.2, 6.1]¶	49.0 [43.1, 54.8]
>200% (higher income)	87.3 [82.0, 91.5]	10.1 [6.4, 15.1]	2.6 [1.0, 5.5]¶	51.0 [45.2, 56.9]

Note. CI = confidence interval.

^*Total n’s shown for each category are unweighted; all analyses are weighted to account for complex sampling.

^†Percentages are weighted. Row percentages might not sum to 100% because of rounding.

^§p-Value for weighted analysis of variance F-test (continuous variable) or weighted Wald chi-square test (categorical variable). All p-values of <.05 indicate significant difference between concordant and discordant groups.

^¶Estimate is unstable; relative standard error is between 30% and 50%.

^**Other race/ethnicity category includes non-Hispanic Black, Hispanic, non-Hispanic Asian, and other.

^††Inadequate or no insurance based on a negative value for any of four variables based on these questions: (1) “Does your child have any kind of health care coverage, including health insurance, prepaid plans such as HMOs, or government plans such as Medicaid?” (2) “Does your child’s health insurance offer benefits (or cover services) that meet his/her needs?” (3) “Does your child’s health insurance allow your child to see the health care providers he/she needs?” (4) “Not including health insurance premiums or costs that are covered by insurance, do you pay any money for your child’s health care?” and if yes, “How often are these costs reasonable?”.

^§§Parents reported whether the child had public insurance (i.e., Medicaid; yes/no).

^¶¶The relationship of household income to the federal poverty level was calculated using 2016 HHS poverty guidelines: https://aspe.hhs.gov/2016-poverty-guidelines.

Characteristics of children who were Dx+/Sx−.

Among the 18.6% of children who were Dx+/Sx− (unweighted n = 112), 14.7% (unweighted n = 17) would meet parent-reported DSM-based criteria only (i.e., they did not have ADHD symptoms reported by their teacher in the screening stage; Figure 1, criterion A). Of those who did not meet criterion A, 26.9% (unweighted n = 35) had three to five DISC-IV symptoms with at least one severe or two moderate ratings of impairment (criterion B). Of those who did not meet criteria A or B, 24.2% (unweighted n = 20) had three or more DISC-IV symptoms with at least one rating of mild impairment (criterion C). Of those who did not meet criteria A, B, or C, 20.8% (unweighted n = 18) had three or more DISC-IV symptoms without any reported impairment (criterion D). Cumulatively, these four sub-threshold diagnostic and symptom criteria could explain the discordance in 62.6%, or an unweighted 90 out of 112 children with Dx+/ Sx−. Among the 22 children with Dx+/Sx− who were not explained by DISC-IV sub-threshold diagnostic and symptom criteria, 16 received either behavioral or ADHD medication treatment (80.1%). Only six children with Dx+/ Sx− were not potentially explained by either DISC-IV sub-threshold diagnostic and symptom criteria or ADHD treatment. Exploring additional interview-related potential explanations for children Dx+/Sx−, nearly two thirds (61.5%, 95% CI [47.2, 74.5]), or an unweighted 54 of 111 children (date of ADHD diagnosis was unknown for one child), were diagnosed with ADHD more than 3 years prior to the DISC-IV interview administration, and more than half (54.8%, 95% CI [37.6, 71.1]), or an unweighted 64 of 112 children, were interviewed more than 1 year apart between stage 1 and stage 2 (data not shown). More than a third (39.3%, 95% CI [22.6, 58.1]), or an unweighted 38 of 112 children, received the DISC-IV interview in the summer months (data not shown). All 112 children with Dx+/ Sx− met at least one of the potential explanatory or interview-related criteria that we explored (data not shown).

Prevalence estimates by differing case definitions.

Comparing prevalence estimates calculated using different case definitions (Table 4), estimates calculated from parent-reported ever diagnosis were higher than DSM-based parent and teacher diagnostic criteria (24.1% vs. 8.1%). Looking at DSM-based diagnostic criteria based on parent responses only (criterion A), prevalence was 13.5%. Prevalence of sub-threshold diagnostic symptoms (criterion B) was 24.7%. Parent-reported ever ADHD diagnosis vs. current ADHD diagnosis were similar (24.1% vs. 23.5%).

Table 4.

Prevalence of ADHD Based on Different Case Definitions, South Carolina Replication Sample of the Project to Learn About Youth Mental Health (SC Re-PLAY).

ADHD case definition	Unweighted n	Weighted prevalence* [95% CI]
Parent-reported diagnosis
Parent-reported ever diagnosis	160	24.1 [18.8, 30.0]
Parent-reported current diagnosis	158	23.5 [18.3, 29.4]
Parent- and teacher-DSM-based criteria (DISC-IV and SDQ and/or BASC-2 BESS)†
Diagnosis based on parent- and teacher-reported DSM-based criteria§	70	8.1 [5.4, 11.7]
Criterion A. Parent-reported DISC-IV criteria met	104	13.5 [10.1, 17.4]
Criterion B. Parent reported three to five symptoms¶ with at least one symptom causing severe impairment or two symptoms causing moderate impairment	169	24.7 [20.2, 29.5]
Criterion C. Parent reported three or more symptoms¶ with at least one symptom causing mild impairment	217	31.5 [26.6, 36.7]
Criterion D. Parent reported three or more symptoms¶ without impairment	312	44.5 [38.8, 50.3]

Note. CI = confidence interval; DISC-IV = diagnostic interview schedule for children—version IV; SDQ = Strengths and Difficulties Questionnaire; BASC-2 BESS = Behavior Assessment System for Children—Second Edition, Behavioral and Emotional Screening System.

^*Unweighted denominator = 561 children

^†Criteria are not mutually exclusive.

^§ADHD case definition for the study.

^¶Symptoms include DISC-IV symptoms of inattention or hyperactivity/impulsivity (six required to meet full diagnostic criteria).

Replication analyses.

Sensitivity of the parent report of ever diagnosis of ADHD was 46.3%, 53.4%, 78.9%, and 64.1% for CO, FL, OH, and SC, respectively (Supplemental Table 1). For parent report of current diagnosis, sensitivity was 39.9%, 46.5%, 71.0%, and 64.1% for CO, FL, OH, and SC, respectively. Across the four sites, the PPV for parent report of ever or current diagnosis ranged between 12.9% and 44.9%. The weighted prevalence of parent-reported ADHD diagnosis was highest in Ohio (29.9% for ever diagnosis and 29.2% for current diagnosis) and lowest in Colorado (9.1% for ever diagnosis and 5.1% for current diagnosis; Supplemental Table 2). For diagnosis based on DSM-based criteria (parent and teacher), Ohio had the highest ADHD prevalence (8.5%) followed by South Carolina (6.8%), Colorado (5.7%), and Florida (5.1%).

Discussion

Researchers and public health professionals often estimate the childhood prevalence of ADHD by asking parents whether their child has ever received a diagnosis from a healthcare provider; therefore, it is important to understand reliability over time and validity of this method. Using data from the 2011 to 2012 National Survey of Children’s Health (NSCH; CDC National Center for Health Statistics, 2017) and the 2014 National Survey of the Diagnosis and Treatment of ADHD and Tourette Syndrome (NS-DATA; CDC National Center for Health Statistics, 2015), we found that parents reported an ever diagnosis of child ADHD from a doctor or other health care provider consistently across time. Parents who reported their child’s ADHD as moderate or severe in NSCH more often reported ADHD 2 years later in NS-DATA compared to parents who reported their child’s ADHD as mild, though both estimates were above 94%. Comparing parent-reported ADHD diagnosis to parent- and teacher-reported ADHD symptoms with impairment to align with DSM-IV diagnostic criteria, we found a high NPV and high specificity for parent-reported provider diagnosis of ADHD, indicating that parent-reported lack of diagnosis was good at correctly identifying children without ADHD. However, children with an ever ADHD diagnosis by parent report did not always meet symptom criteria, and not all children meeting diagnostic criteria based on current symptoms had been reported to have received a diagnosis.

Although NPV and specificity were relatively high, PPV and sensitivity were low. However, when we examined sub-threshold criteria and ADHD treatment, most (106 of 112) children who were Dx+/Sx− had some level of ADHD symptomatology or were receiving treatment for ADHD, including medication for ADHD or psychosocial therapy. It is possible that many children with an ADHD diagnosis who did not meet DSM-based criteria were being adequately treated for their ADHD, as most (78.9%) children who were Dx+/Sx− were receiving treatment. More than 3 of 5 (62.6%) of children Dx+/Sx− met at least one of four sub-threshold DSM-based diagnostic criteria based on parent report, and 14.7% met DISC-IV criteria based on parent-reported DSM-based diagnostic criteria (not including teacher reports). It is possible teachers did not recognize symptoms of ADHD in the classroom among children receiving treatment. PPV for parent-reported child diagnosis compared to DSM-based criteria may have been higher had we been able to compare to a clinician diagnosis in the child’s health records. In addition, prevalence estimates calculated using a parent-reported diagnosis of ADHD were higher than those calculated using DSM-based criteria; for example, the SC Re-PLAY estimate that used parent-reported ever diagnosis was nearly three times higher (24.1% vs. 8.1%). Elevated prevalence using parent-reported ever diagnosis of ADHD compared to DSM-based criteria is generally consistent with findings recently published from a nationwide study of school-aged children in Brazil (7.1% vs. 3.9%; Arruda et al., 2022). This difference has implications for informing policy or programs, as using parent-reported ADHD diagnosis to estimate prevalence may overstate population-level service needs, while using DSM-based criteria may underestimate population-level service needs if children with ADHD are being appropriately treated and do not currently meet diagnostic criteria as a result. Nonetheless, the single question to parents to identify children who have ever received an ADHD diagnosis correctly categorizes most children by ADHD status relative to DSM-based criteria and is relatively simple to ascertain compared to a full diagnostic interview. However, the parent-reported ever ADHD diagnosis indicator misses some children showing symptoms, which may reflect under-diagnosis of ADHD. Only 9.3% of children Dx−/Sx+ were receiving treatment, which suggests that most children in this group may have unidentified ADHD.

There is evidence that there may be racial and ethnic disparities in the diagnosis of ADHD for Black and Hispanic children due to the under-diagnosis of ADHD in these groups rather than an over-diagnosis among White children (Coker et al., 2016). However, more recent published estimates of ADHD prevalence have shown that non-Hispanic Black children do not have a lower prevalence of diagnosed ADHD than non-Hispanic White children (Danielson et al., 2018; Zablotsky & Alford, 2020). We did not find significant differences in concordance status by race/ethnicity in the SC Re-PLAY sample; however, our sample size may not have had the power to detect small differences in concordance by race/ethnicity.

Differences in concordance of diagnosis and symptoms by demographic characteristics reveal an area for possible future research. Children with public insurance and living in households with lower income were more often Dx+/Sx−. One potential explanation for these findings is that children with Medicaid may be receiving appropriate treatment for ADHD that reduces symptoms to below the diagnostic threshold. An alternate explanation is that the children in these groups may be over diagnosed; however, this study did not collect information about symptoms and impairment at the time of diagnosis, so the possibility of over-diagnosis cannot be evaluated with these data. Estimates were unstable in Dx−/Sx+ group by insurance and FPL and should be interpreted with caution but appear fairly similar in size.

Replication analyses across the four other PLAY-MH sites showed child ADHD prevalence estimates differed by site and from state-level estimates published elsewhere (Bitsko et al., 2022). PLAY-MH prevalence estimates of parent-reported ever ADHD were higher than 2016 to 2019 NSCH state estimates for Florida (PLAY-MH: 21.2%, NSCH: 9.5%, 95% CI [7.9, 11.4]), Ohio (PLAY-MH: 29.9%, NSCH: 11.6%, 95% CI [9.9, 13.7]), and South Carolina (PLAY-MH: 28.1% and SC Re-PLAY: 24.1%, NSCH: 12.7%, 95% CI [11.0, 14.6]), but were similar for Colorado (PLAY-MH: 9.1%, NSCH: 8.9%, 95% CI [7.3, 10.8]; Bitsko et al., 2022). The age range included in NSCH estimates was 3 to 17 years, which is slightly younger than the PLAY-MH age range of 5 to 19 years and likely partially contributes to the difference. In addition, differing estimates are likely at least partially explained by varying demographic breakdowns across sites; for example, sites varied by socioeconomic disadvantage, a risk factor for ADHD (Miller et al., 2018; Russell et al., 2016). In the Colorado PLAY-MH site, only 23.0% (95% CI [16.2, 31.0]) of children were living in higher income households (≥200% FPL) compared to 58.7% (95% CI [50.3, 66.8]) in Florida and 45.2% (95% CI [44.9, 64.4]) in South Carolina (Ohio did not collect the information needed to calculate the relationship of household income to FPL; Danielson et al., 2021). Similarly, community-level estimates are not representative of state-level estimates and have differing demographic breakdowns compared to the state overall; both Florida (PLAY-MH: 58.7% vs. NSCH: 47.5%) and South Carolina (PLAY-MH: 54.8% vs. NSCH: 46.7%) PLAY-MH sites had a greater proportion of children living in lower income households compared to state-level NSCH estimates of poverty (Child and Adolescent Health Measurement Initiative), which could contribute to higher ADHD prevalence estimates in PLAY-MH. Finally, the parent-reported ever ADHD question administered in PLAY-MH differed somewhat from the question included on NSCH which may limit comparability. The question included on PLAY-MH did not specify who the child received the diagnosis from, asking whether the child has “ever been diagnosed with a mental, emotional, or behavioral disorder” followed by “if yes, what has he/she been diagnosed with,” of which ADHD was a choice. In contrast, the question included on NSCH specified whether a “doctor or other health care provider” has ever told the parent that their child had ADHD. Estimates of parent-reported ADHD diagnoses in PLAY-MH may be inflated by parents reporting teacher reports of ADHD in addition to clinician diagnoses.

Limitations

This study is subject to limitations. First, most indicators rely on parent-report and could be subject to recall or social desirability bias. Specifically, parent-report of ADHD may be influenced by current symptoms exhibited by the child. For example, if a child is exhibiting few symptoms of ADHD, parents might be less likely to report a previous ADHD diagnosis, either because they forgot an earlier diagnosis, or because they disagree or feel the diagnosis no longer applies. This would make it appear the parent correctly reported a true non-case of ADHD according to symptom criteria, resulting in an artificially high specificity. Second, NS-DATA is subject to non-response bias as nonrespondents differed on multiple demographic and health-related characteristics from respondents. For example, children of nonrespondents were less likely to need or use more medical care, mental health, or educational services than most children of same age, compared to respondents (Zablotsky, Bramlett, et al., 2019), which could reflect less severe ADHD. If parents of children with more severe ADHD were both more likely to complete the NS-DATA screener and report their child had ADHD again at NS-DATA, our results may overestimate agreement of this question over time. In addition, high levels of agreement may be influenced by the short time between measurements (i.e., 2 years); calculating longer-term stability coefficients are a possible area of future research. Third, the NS-DATA respondent was not necessarily the same parent who responded to NSCH; reporting by two parents who may disagree about the child’s ADHD status could underestimate agreement of this question over time. A fourth limitation to our study was that to receive a provider diagnosis, children must see a healthcare provider; therefore, children with symptoms but without a previous diagnosis may include those without access to healthcare. Considering children from racial and ethnic minority groups more often experience inequities in health care access compared to non-Hispanic White children (e.g., Rodgers et al., 2022; Shi & Stevens, 2005), this is an important area of future research. Fifth, past-year DSM-based criteria reflects active symptoms only and may not account for children with appropriately treated ADHD. Thus, children receiving appropriate treatment for their ADHD and not meeting current criteria may be inappropriately classified as not having ADHD based on the DISC-IV interview. We attempted to address this limitation by evaluating whether the child received medication for ADHD or behavioral treatment and sub-threshold criteria based on the DISC-IV, but these approaches do not address children with fewer than three symptoms without impairment. Sixth, the time frame of measures could have influenced our results. Specifically, we compared current symptoms based on the DISC-IV with an ever diagnosis, because the PLAY-MH study protocol did not include lifetime assessment of ADHD. However, the results comparing parent report of current ADHD and the DSM-based ADHD diagnostic criteria results were consistent to the results considering parent report of an ever diagnosis. Seventh, we were unable to determine whether parent-reported psychosocial therapy was for treatment of ADHD or for a different mental health condition. Finally, we did not have adequate sample size to replicate differences by demographic characteristics across the additional four sites; however, we observed similar estimates for weighted sensitivity, specificity, PPV, NPV, and prevalence across sites.

Conclusions

Mental health surveillance among children remains a challenge. There is no single best way to estimate the prevalence of ADHD or other mental disorders among children and adolescents. Asking parents whether their child has ever received a diagnosis from a healthcare provider may miss a subset of children who have received a diagnosis of ADHD for mild ADHD symptoms in the past if their parent does not report the diagnosis, as well as children who meet DSM-based criteria who have never been diagnosed. Despite limitations of a parent-reported ever ADHD diagnosis indicator, particularly the potential for elevated prevalence estimates compared to DSM-based criteria as well as low PPV, our study provides support for the use of this single-item measure to obtain estimates of pediatric ADHD at the state and national level. In addition to the high specificity and NPV of parent-reported ever ADHD diagnosis, we found most children Dx+/Sx− were receiving treatment for ADHD, leaving the possibility that these children may have ADHD that is appropriately treated and therefore could be considered when estimating population-level service use. Public health professionals and researchers may wish to explore how identified demographic differences in concordance status between diagnosis and symptoms, such by insurance type (public or private) and poverty level, may impact prevalence estimates when targeting outreach and intervention efforts.

Author Biographies

Robyn A. Cree is an epidemiologist with the Disability and Health Promotion Branch in the National Center on Birth Defects and Developmental Disabilities (NCBDDD), Centers for Disease Control and Prevention (CDC). Her work focuses on studying risk and protective factors related to mental, emotional, and behavioral disorders, particularly among individuals with disabilities.

Rebecca H. Bitsko is a health scientist with the Child Development and Disabilities Branch in the National Center on Birth Defects and Developmental Disabilities (NCBDDD), Centers for Disease Control and Prevention (CDC). Her work focuses on children’s mental health, including epidemiology and surveillance of Tourette syndrome, ADHD, and other mental, behavioral, and developmental disorders among children.

Melissa L. Danielson is a statistician with the Child Development and Disabilities Branch in the National Center on Birth Defects and Developmental Disabilities (NCBDDD), Centers for Disease Control and Prevention (CDC). Her work focuses on the epidemiology of ADHD and other mental, emotional, and behavioral disorders among children, including work on disorder prevalence, clinical presentation, service utilization, and outcomes for children diagnosed with these disorders.

Valentine Wanga is an epidemiologist with the Child Development and Disabilities Branch in the National Center on Birth Defects and Developmental Disabilities (NCBDDD), Centers for Disease Control and Prevention (CDC). She conducts research in mental health among children and adolescents.

Joseph Holbrook is the Team Lead for the Disability Science and Program Team with the Disability and Health Promotion Branch in the National Center on Birth Defects and Developmental Disabilities (NCBDDD), Centers for Disease Control and Prevention (CDC). He conducts scientific and programmatic work related to improving health and wellness for individuals with intellectual and developmental disabilities and mobility limitations, including research in child development, mental health, and disability.

Kate Flory is Professor and Associate Chair in the University of South Carolina Department of Psychology, where she studies factors related to ADHD in children through young adults as well as prevalence of mental health disorders in youth.

Lorraine F. Kubicek is a Developmental Psychologist and a member of the faculty at JFK Partners, Department of Pediatrics, at the University of Colorado School of Medicine. Her research interests include the development of early language and communication, parent-child relationships, and infant and early childhood mental health and is active in state and national efforts to increase public awareness of, and foster professional development in, infant and early childhood mental health.

Steven W. Evans is a Distinguished Professor of Psychology and Co-director of the Center for Intervention Research in Schools at Ohio University. His research is focused on treatment development and evaluation for adolescents with ADHD.

Julie Sarno Owens is a Professor of Psychology and Co-director of the Center for Intervention Research in Schools. Her research focuses on developing and evaluating school-based services for youth with ADHD and related problems.

Steven P. Cuffe is Professor and Chair of the Department of Psychiatry at the University of Florida College of Medicine, Jacksonville. He has over 25 years experience in epidemiological research on child psychiatric disorders.