Validation of the Diagnostic Interview Schedule for Children (DISC-5) Tic Disorder and Attention-Deficit/Hyperactivity Disorder Modules

Abstract

Effective methods to assess mental disorders in children are necessary for accurate prevalence estimates and to monitor prevalence over time. This study assessed updates of the tic disorder and attention-deficit/hyperactivity disorder (ADHD) modules of the Diagnostic Interview Schedule for Children, Version 5 (DISC-5) that reflect changes in diagnostic criteria in the Diagnostic and Statistical Manual of Mental Disorders (Fifth edition, DSM-5). The DISC-5 tic disorder and ADHD parent- and child-report modules were compared to expert clinical assessment for 100 children aged 6–17 years (40 with tic disorder alone, 17 with tic disorder and ADHD, 9 with ADHD alone, and 34 with neither) for validation. For the tic disorder module, parent-report had high (>90%) sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy, while the youth-report had high specificity and PPV, moderate accuracy (81.4%), and lower sensitivity (69.8%) and NPV (67.3%). The ADHD module performed less well: parent-report had high NPV (91.4%), moderate sensitivity (80.8%), and lower specificity (71.6%), PPV (50.0%), and accuracy (74.0%); youth-report had moderate specificity (82.8%) and NPV (88.3%), and lower sensitivity (65.0%), PPV (54.2%), and accuracy (78.6%). Adding teacher-report of ADHD symptoms to DISC-5 parent-report of ADHD increased sensitivity (94.7%) and NPV (97.1%), but decreased specificity (64.2%), PPV (48.7%), and accuracy (72.2%). These findings support using the parent-report tic disorder module alone or in combination with the child report module in future research and epidemiologic studies; additional validation studies are warranted for the ADHD module.

Introduction

Validated and reliable diagnostic instruments are important for identification of mental disorders, including tic disorders and attention-deficit/hyperactivity disorder (ADHD), in both research and clinical settings. Tics, which are “sudden, rapid, recurrent, nonrhythmic motor movements or vocalizations” (American Psychiatric Association, 2013), occur with high prevalence during childhood and adolescence. Although estimates vary across studies, approximately 25% of children experience tics at some point during development (Snider et al., 2002), 1.6% of children will meet criteria for a persistent (or chronic) tic disorder, (Knight et al., 2012), and 0.5-0.8% for Tourette syndrome (also referred to as Tourette’s disorder) (Knight et al., 2012; Scharf et al., 2015). Tic disorders are associated with negative health and social consequences, such as diminished child and parent quality of life and family functioning (Eapen et al., 2016; McGuire et al., 2013; Murphy et al., 2013; Vermilion et al., 2020), and a high risk of co-occurring mental, behavioral, and developmental disorders, including attention-deficit/hyperactivity disorder (ADHD) (Fernandez de la Cruz & Mataix-Cols, 2020; Murphy et al., 2013).

Despite being fairly common during childhood, tics and persistent tic disorders are often not identified (Lewin et al., 2014). Observation, even by an expert, may not identify tics because tics “wax and wane” and can be suppressed for periods of time (Cohen et al., 2013). Even when a child or parent knows the child has a tic disorder, they may not report having a current tic (Lewin et al., 2014). When a child presents for evaluation, tics may be attributed to allergic symptoms (sniffing, coughing), vision problems (eye blinking), or nervous habits. Identification of tics is also challenging due to the lack of validated measures for screening and diagnosis of tics and persistent tic disorders. Effective assessment tools are needed that can be administered at low cost to large samples and that are reliable and valid. This study was designed to evaluate the tic disorder module of the fifth version of the Diagnostic Interview Schedule for Children (DISC-5). As up to half of children with a persistent tic disorder also meet diagnostic criteria for ADHD (Hirschtritt et al., 2015; Murphy et al., 2013), a secondary objective was to examine the ADHD module of DISC-5.

The DISC is a highly structured interview that was designed specifically for administration by non-clinically trained interviewers in large scale epidemiologic studies to identify the most common mental disorders of youth (Shaffer et al., 2000). The DISC-IV, corresponding to the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) (American Psychiatric Association, 2000) and the International Classification of Diseases, Tenth Revision (ICD-10) (World Health Organization, 2004), became available in 1997 and has been used widely in epidemiologic and clinical research studies (Cuffe et al., 2020; Danielson et al., 2021; Fazel et al., 2008; Merikangas et al., 2010; Wolraich et al., 2014). The interview has two parallel forms, one for parents (or caregivers) to report about their child (aged 6–17 years; DISC-P [Parent]) and the other administered directly to youth (aged 9–17 years) about themselves (DISC-Y [Youth]). DISC-IV questions are read to the parent or child exactly as they are written, with responses limited to “yes” or “no” or a close-ended frequency or severity choice; very few questions call for an open-ended response.

Limited psychometric studies of the DISC have shown that it generally performs well in identifying ADHD (McGrath et al., 2004; Rolon-Arroyo et al., 2016), but less well for tic disorders (Lewin et al., 2014). Two studies have looked at tic disorders identified by the DISC. In the first, Fisher et al., (1993) evaluated the sensitivity of the DISC-2 in identifying tic disorders in a small group of children (n=12) who were diagnosed with Tourette syndrome by a board certified child psychiatrist at a center with extensive research experience on tic disorders. They found that sensitivity of the parent version of the DISC-2 (DISC-P) was excellent; all 12 youths were identified as having a tic disorder, with 67% correctly identified as having Tourette syndrome. The youth completed version (DISC-Y) was less sensitive; 8 youths (67%) were identified as having a tic disorder, of which four were correctly identified as having Tourette syndrome. The second study examined a larger sample of children with Tourette syndrome (n=181), identified through a tic specialty clinic, and a sample of community controls without a tic disorder (n=101; Augustine et al., 2017). In this study, the DISC-IV (both parent and youth report) showed poor sensitivity in identifying Tourette syndrome as compared with expert clinical assessment (44% for parent-report, 27% for youth report); only 54% of children with Tourette syndrome were identified by the DISC as having that disorder, using either parent or youth report (Lewin et al., 2014). By youth and parent report, respectively, 31.8% and 15.1% of children with Tourette syndrome were not idenified by the DISC-IV as having any tic disorder. Thus, children with a tic disorder may not be identified by the DISC-IV, suggesting a need to modify the tic module in subsequent versions.

With the release of the DSM-5 (American Psychiatric Association, 2013), the need to update the DISC resulted in a cooperative agreement between the CDC and the University of South Carolina’s Disability Research and Dissemination Center (DRDC); a subcontract was awarded to the Research Foundation for Mental Hygiene, Inc. (Columbia University, Department of Psychiatry; Prudence Fisher, PI) to prepare a paper-and-pencil version of the DISC-5 (i.e., updating the DISC-IV to adhere to DSM-5 diagnostic criteria). Development of the DISC-5 followed the same general approach as previous DISC updates. In short, there was an iterative process which included careful review and collaboration with experts (see Acknowledgments) and limited “cognitive testing” with youth and their parents to ensure that the questions were understood. Changes were also made to incorporate improvements to the interview based on analyses and user feedback; these included streamlining some modules, optional skips, simplifying symptom readbacks, and revising specific questions identified as problematic.

The diagnostic criteria for tic disorders of the DSM-IV (American Psychiatric Association, 2000) and DSM-5 (American Psychiatric Association, 2013) had few substantial differences (Walkup et al., 2010). Specifically, the wording from DSM-IV was changed from requiring tics to “occur many times a day (usually in bouts) nearly every day or intermittently” for Tourette syndrome to stating that tics “may wax and wane in frequency” in DSM-5. Also, the DSM-IV required that there not be a tic-free period that lasted more than 3 months; this criterion was eliminated in the DSM-5 revision. Finally, the category of transient tics was eliminated, and replaced by provisional tic disorder, which only requires the presence of tics for less than 12 months. In preparing the DISC-5, the tic disorder module was substantially revised because of poor agreement between the DISC-IV and expert clinical diagnosis of tic disorders described above (Lewin et al., 2014). The DISC-IV (Shaffer et al., 2000) asked a single, lengthy question to assess for the presence of any potential motor tics in the past year, and a second lengthy question for vocal tics. The DISC-5 was revised to ask a series of specific questions about a variety of common motor (e.g., eye blinking, nose movements) and vocal (e.g., coughing or throat clearing, sniffing) tics. Altogether, the DISC-5 contains seven stem questions about specific motor tics, six stem questions about specific vocal tics, and one question each about unspecified motor and vocal tics. Each stem question asks whether the child has experienced a type of tic (e.g., eye movements) since they were 5 years old. If they respond no, they skip to the next stem question. If they respond yes to the stem question, they are asked whether it was “difficult to stop” the tic. Again, if they respond “no” they skip to the next stem question. For those who respond the tic was “difficult to stop”, they are asked in most cases (seven motor tics, four vocal tics) to describe the tic and whether (in all cases) the child has had this tic in the last year. A sample question set can be found in the figure.

Figure.

Example of a motor tic question from the Diagnostic Interview Schedule for Children Version 5 (DISC-5)

Changes to the DSM-5 criteria for ADHD were limited in number: a significant change to the required age of onset from “before age 7 years” (DSM-IV) to “prior to age 12 years” and primarily minor wording changes and addition of examples in the diagnostic criteria to apply to a larger age range (American Psychiatric Association, 2000, 2013; Epstein & Loren, 2013),DSM-5 also allows individuals diagnosed with autism spectrum disorder to receive an ADHD diagnosis. Changes to the ADHD module from the DISC-IV to DISC-5 were similarly few – two questions were added to address additional examples of criteria (to keep questions short) and minor wording changes were made to other questions, mostly to include new DSM-5 examples for older youth.

Under the same cooperative agreement established between the CDC and the DRDC, the University of Rochester Medical Center (URMC) received a contract to validate tic measures, including the DISC-5 tic module, against expert clinical assessment. Although the primary focus of the study was on tic disorders, given the high co-occurrence of ADHD with tic disorders, the DISC-5 ADHD module was also compared with expert clinical assessment. In this paper we present methods and findings of that study.

Materials and Methods

Participant Recruitment and Study protocol (University of Rochester)

URMC conducted an observational, cross-sectional, case-control study to compare expert-clinician diagnoses of tic disorders and ADHD to diagnoses made based on responses to the tic disorder and ADHD modules of the DISC-5. Children qualified for the study if they were between 6–17 years of age with a confirmed tic disorder diagnosis based on previous clinical assessment (n = 55: Tourette syndrome, Persistent Motor, or Persistent Vocal Tic Disorder) or without a known tic disorder (n = 45). Following expert assessment (see below), two children recruited without a known tic disorder were identified as having Tourette syndrome; therefore, the tic disorder group included 57 children and there were 43 children in the “no tic disorder” group. Other inclusion criteria were: English speaking, enrollment in a traditional school program (i.e., not home-schooled; in order to obtain teacher ratings), parent and child able and willing to provide permission and assent as appropriate for study participation, and child deemed able (in the opinion of parent and/or study personnel) to participate in the study procedures. Approximately one-third (n=34) participants had a sibling in the study. Children with a confirmed tic disorder diagnosis were recruited from within the clinical practice of the investigators at URMC, through the two local chapters of the Tourette Association of America that serve Rochester and the surrounding region (see Acknowledgments), and by targeted outreach to primary providers in the region with a known caseload of patients with tic disorders. Those without tic disorder diagnoses (comparison group) were recruited through flyers posted in a variety of community locations (e.g., local libraries, after-school programs, local hospital) and through the Greater Rochester Practice Based Research Network (GR-PBRN), a consortium of community based medical practices that maintain a research alliance with the University. Approximately 80% of children living in Monroe County, NY (of which Rochester is the county seat) are served by GR-PBRN practices. Remuneration was provided to children and parents for their participation and to classroom teachers who completed and returned study materials. All parents completed a written, informed process to provide parental permission for their and their child’s participation in the study. Children between 8–12 years of age provided verbal assent; children between 13–17 years of age completed a written assent process. All study activities were approved by the University of Rochester Research Subjects Review Board (RSRB00064456).

Procedures

Each parent/child pair completed a diagnostic assessment by an expert clinician and the DISC-5 conducted by a trained non-expert interviewer. Because many of the participants with tic disorder diagnoses were recruited through URMC clinics, the diagnostic status of those children was typically known to the expert clinician during their reference standard assessment. However, we maintained blinding of the DISC interviewers to the child’s status (tic disorder vs. comparison group) to reduce potential for bias in administration and ratings of the primary study measures. The sequence of assessments by the expert clinicians and trained DISC interviewers was counterbalanced to minimize the potential for order effects. Also, the DISC interviewers were randomly assigned at the start of each study visit to conduct either the Parent or the Youth assessment, so that no particular interviewer inadvertently developed biases or an imbalance in experience related to evaluating predominantly parents or children and all interviewers were blind to the reports of the other informant.

Expert Clinical Assessment.

The presence of a DSM-5 tic disorder and/or ADHD was determined by an expert clinician, who conducted diagnostic evaluations with participating youth (including obtaining information from their parents). Expert clinicians were all clinical faculty in the URMC Tic Disorders clinic and included: pediatric neurologists (EFA, JWM, PM), a pediatric neurology nurse practitioner (AV), and a clinical child psychologist (HRA). For each participant, they reviewed DSM-5 diagnostic criteria for persistent tic disorders (Tourette syndrome, persistent motor tic disorder, persistent vocal tic disorder) and for ADHD. A tic disorder diagnosis was determined based upon their clinical interviews, review of history, and relevant observations (e.g., tics) during the study visit. The presence of an ADHD diagnosis was based upon these same activities in combination with review of information from Vanderbilt ADHD Diagnostic Parent- and Teacher-Rating Scales, respectively (VADPRS; VADTRS). The VADTRS was included because the diagnosis of ADHD requires the presence of impairing symptoms in at least two settings (American Psychiatric Association, 2013; Bard et al., 2013; Wolraich et al., 2003). When the VADTRS was absent, the clinical expert relied upon information gathered from the clinical interview with the parent about the presence of symptoms in multiple settings.

Measures

DISC-5.

The introductory, tic disorder, and ADHD modules from the DISC-5 (parent and youth) were programed into REDCap (Harris et al., 2009) to improve efficiency of administration (compared to paper and pencil) for the study. Children and their parents met separately with a trained DISC interviewer who administered the Parent or Child version of the three modules. The youngest children in the study had difficulty benchmarking time frames for events that had happened at certain time points in the past (e.g., 6 months ago, a year ago, etc.), so for children aged 9–10 years the introductory module was conducted as a joint interview with both parent and child. Eight children under 9 years of age participated in the DISC-5 interview early in the study; three of these were excluded from the tic disorder analysis and two were excluded from the ADHD analysis because of missing data. Following completion of the DISC-5 interview, a standardized algorithm was used to determine whether children met criteria for a tic disorder (i.e., Tourette syndrome, persistent motor or vocal tic disorder, provisional tic disorder) and/or ADHD. Although the ADHD module can be used to distinguish the different current presentations of ADHD (predominantly hyperactive/impulsive, predominantly inattentive, combined), this study did not examine differences in performance of the DISC-5 by presentation.

Vanderbilt ADHD Diagnostic Parent- and Teacher-Rating Scales.

Both the parent and teacher-based Vanderbilt scales (VADPRS and VADTRS, respectively) are screening instruments that ask about specific symptoms of ADHD experienced by a child. The VADPRS has high agreement with other measures of parent-report measures of ADHD, including DISC-IV (Wolraich et al., 2003). The VADTRS has lower agreement with parent measures of ADHD (Wolraich et al., 2013) but has been used to gather additional information about ADHD symptoms from an additional informant in a second setting (i.e., school) as part of the ADHD diagnostic process (Yuki et al., 2016). The VADPRS was completed by all parents as part of the clinical assessment. With written consent from the child’s parent, we mailed the VADTRS to one classroom teacher for each child who participated in the study. For children with multiple teachers across academic subjects, either the Math or English teacher was selected with input from the participating child and their parent to identify the teacher. Family input was solicited to identify the teacher likely to have the greatest amount of instructional time with (and hence observation of) the child, and who the family trusted to provide a fair assessment of the child’s classroom function. Each teacher received an initial outreach phone call to inform them that the questionnaire was being sent, and a follow-up phone call if they did not return the form. A total of 72 teachers returned the completed VADTRS. Parents and children also completed additional measures related to the identification of tics and persistent tic disorders that are not included in this report (see Adams et al., in press).

Analyses

The DISC-5 data were processed using an algorithm programmed in SAS to determine whether children met criteria for a tic disorder or ADHD, based on DSM-5 criteria. For tic disorders, this included determining the number of motor and vocal tics (present and difficult to stop, see questions 2 and 2A in the figure), and the duration of tics (less than or greater than one year). Questions asking the respondent to describe tics were not included in the algorithm. For ADHD, the algorithm included determining the number of inattentive and hyperactive/impulsive symptoms, the age of onset of symptoms, and whether the ADHD symptoms were associated with impairment. The results of the expert-clinician diagnoses of tic disorders and ADHD were compared to results of the parent- and youth-reported tic and ADHD modules of the DISC-5. Parent- and youth-reported DISC-5 modules were evaluated separately and together (i.e., whether the parent or child endorsed symptoms meeting full criteria for the disorder). For ADHD, expert clinician diagnosis was also compared to the parent- and youth-reported ADHD DISC-5 supplemented by information obtained on the VADPRS, and VADTRS (if available). Including teacher ratings as part of the ADHD case definition has been used previously in an epidemiologic study of ADHD to allow direct reporting of symptoms in a second setting (Wolraich et al., 2014). Frequencies were calculated using SAS v.9.4 (SAS Institute, Inc.; Cary, NC).

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy, as well as 95% confidence intervals (CI) were calculated to determine the utility of the DISC-5 for identifying children with any tic disorder, Tourette syndrome specifically, and ADHD. Sensitivity analyses were conducted for any tic disorder and ADHD by including only a single sibling per family to determine if the inclusion of siblings biased the overall results; this resulted in excluding 18 participants from the full sample. The addition of the specific tic questions in DISC-5 also enabled us to examine the frequency of patterns of reporting specific tics, and to determine if any questions were redundant or if the interview could be shortened.

Results

Sample Description

The overall sample (n=100) included mostly male (63%) and non-Hispanic White children (59%) with a mean age of 12 years. There were 34 participants who participated with a sibling, which comprised 14 sibling dyads and two sibling triads. Most of these children (n=26/34; 76%) were in the comparison (no tic disorder) group. Two dyads and one triad included one child with a tic disorder and remaining siblings in the comparison group (one of the participants recruited for the comparison group was determined to have a tic disorder); otherwise, each sibling dyad or triad was exclusively comprised of children who were only members of the tic disorder group or the comparison group.

Overall, there were 43 children with no tic disorder, of which 9 had ADHD, and 57 children with a tic disorder, of which 17 also had ADHD (total ADHD n = 26). In the comparison (no tic disorder) group, there were fewer males (44.2%) compared to the tic disorder (77.2%) group. The 26 children with an expert diagnosis of ADHD were also more likely to be male (73.1%); they were also slightly younger (mean age 11.4 years) than the children in the no tic disorder comparison (12.2 years) and tic disorder (12.0 years) groups (note the ADHD group was not mutually exclusive from the tic disorder and comparison groups). The comparison (“no tic disorder”) group had the largest percent of non-Hispanic White children (69.8%), and the tic disorder group had the lowest percent of Hispanic children (1.8%; see Table 1).

Table 1.

Demographic characteristics of the overall sample of participants in the validation study of the Diagnostic Interview Schedule for Children Version 5, and by study group determined by expert diagnosis of tic disorders and attention-deficit/hyperactivity disorder

		Total Sample (n=100)	No Tic Disorder Group* (n=43)	Tic Disorder Group* (n=57)	ADHD Diagnosis (n=26)**
Child Age (years)	Mean (SD)	12.0 (3.3)	12.2 (3.3)	12.0 (3.2)	11.4 (3.6)
	Range	6.2-18.0	6.2-17.5	6.6-18.0	6.7-17.3
Child Sex	Female	37.0%	55.8%	22.8%	26.9%
	Male	63.0%	44.2%	77.2%	73.1%
Child Race/Ethnicity	Non-Hispanic White	59.0%	69.8%	50.9%	57.7%
	Hispanic	6.0%	11.6%	1.8%	15.4%
	Other or Multiple	35.0%	18.6%	47.4%	26.9%

ADHD = attention-deficit/hyperactivity disorder; SD = standard deviation

^*Note that two children recruited as part of the comparison group were identified during clinical assessment as meeting criteria for Tourette syndrome and are included in the Tic Disorder group in this table and throughout the paper.

^**In the ADHD group (n=26), 9 did not have a tic disorder and 17 had ADHD and a tic disorder.

Within the tic disorder group, 50 children had Tourette syndrome, 4 had a chronic motor tic disorder, and 1 had a chronic vocal tic disorder, based on expert assessment (data not shown in tables). In the initially recruited comparison group, two children were identified as having Tourette syndrome during the expert clinical assessment and were therefore included in the tic disorder group when comparing the DISC-5 tic module to expert diagnosis.

DISC-5 Tic Module compared to expert clinical assessment

As shown in Table 2, the DISC-5 parent-report tic module had good (>97%) sensitivity, specificity, PPV, NPV, and accuracy for identifying children aged 6–17 years with any tic disorder. The youth report had lower sensitivity (69.8%) and NPV (67.3%), but high specificity and PPV (both 100%); accuracy was 81.4%. Combined parent and youth report (i.e., either parent or child reporting sufficient symptoms to meet diagnostic criteria) on the DISC-5 tic disorder module had perfect agreement with expert assessment determination of the child having any tic disorder. Sensitivity analyses that included a single sibling per family found similar results to the full sample, and therefor are not presented in addition to Table 2. All estimates of sensitivity, specificity, PPV, and NPV for the restricted sample were within 2.5% of the corresponding estimates for the full sample; one exception was that the NPV for youth report decreased from 67.3% to 58.3%.

Table 2.

Comparison of the Diagnostic Interview Schedule for Children Version 5 (DISC-5) Tic Disorder module and expert clinical assessment.

Assessment of Tic Disorders	n*	Sensitivity	Specificity	Positive Predictive Value (PPV)	Negative Predictive Value (NPV)	Accuracy
Any Tic Disorder
Parent-Reported DISC-5	100	56/57 = 98.2% CI: 95.7-100.0%	42/43 = 97.7% CI: 94.7-100.0%	56/57 = 98.2% CI: 95.7-100.0%	42/43 = 97.7% CI: 94.7-100.0%	98/100 = 98.0% CI: 95.3-100.0%
Youth-Reported DISC-5	86	37/53 = 69.8% CI: 60.1-80.0%	33/33 = 100.0% CI: 100.0-100.0%	53/53 = 100.0% CI: 100.0-100.0%	33/49 = 67.3% CI: 57.4-77.3%	70/86 = 81.4% CI: 73.2-89.6%
Parent-Reported DISC-5 & Youth-Reported DISC-5	86	53/53 = 100.0% CI: 100.0-100.0%	33/33 = 100.0% CI: 100.0-100.0%	37/37 = 100.0% CI: 100.0-100.0%	33/33 = 100.0% CI: 100.0-100.0%	86/86 = 100% CI: 100.0-100.0%
Tourette syndrome
Parent-Reported DISC-5	95	52/52 = 100.0% CI: 100.0-100.0%	43/43 = 100.0% CI: 100.0-100.0%	52/52 = 100.0% CI: 100.0-100.0%	43/43 = 100.0% CI: 100.0-100.0%	95/95 = 100.0% CI: 100.0-100.0%
Youth-Reported DISC-5**	79	31/46 = 67.4% CI: 57.1-77.7%	33/33 = 100.0% CI: 100.0-100.0%	31/31 = 100.0% CI: 100.0-100.0%	33/48 = 68.8% CI: 58.5-79.0%	64/79 = 81.0% CI: 72.4-89.7%
Parent-Reported DISC-5 & Youth-Reported DISC-5**	79	46/46 = 100.0% CI: 100.0-100.0%	33/33 = 100.0% CI: 100.0-100.0%	46/46 = 100.0% CI: 100.0-100.0%	33/33 = 100.0% CI: 100.0-100.0%	79/79 = 100.0% CI: 100.0-100.0%

CI: 95% confidence interval.

^*A total of 100 parents completed the DISC-5 assessment, and 86 children completed the youth report DISC-5.

^**Four children with a clinical diagnosis of Tourette syndrome did not respond to the DISC-5 question about tics being present for over a year, and were excluded from the analysis (each met symptom criteria for Tourette syndrome). The children were aged 7 (2 children), 8, and 12 years.

The DISC-5 parent-report tic module also had excellent sensitivity, specificity, PPV, NPV, and accuracy (all 100%) for identifying children aged 6–17 years with Tourette syndrome. Again, the sensitivity (67.4%) and accuracy (100%) of the youth report was lower, but combined parent and youth report had perfect agreement with expert assessment for Tourette syndrome.

Report of specific tics on DISC-5 tic module

Motor tics:

The most commonly parent-reported motor tics for children with an expert diagnosis of tic disorder were: “other motor” tics (75.4%), blinking (71.9%), and eye movements (64.9%; see Table 3). Over half of the parents also reported head (61.4%), shoulder (61.4%), and mouth, lip, or tongue (59.7%) movements, while making unusual faces (45.6%) and nose movements (42.1%) were each reported for fewer than half of children with a tic disorder. Among children who participated in the DISC-5 interview (n=53) in the tic disorder group based on expert clinical diagnosis (n=57), “other motor” tics was reported by over half (54.7%), followed by head movements (49.1%), blinking (45.3%), and shoulder movements (47.2%). All the other queried tics were each reported by less than 40.0% of children with a tic disorder. For the group of children without a tic disorder, one parent reported that their child had motor tics, specifically mouth, lip, or tongue movements and “other motor” tics, on the DISC-5. No children without a tic disorder reported motor tics.

Table 3.

Report of specific tics on the DISC-5 tic disorder module by tic disorder status and parent vs. child report

	Tic Disorder		No Tic Disorder
	Parent-report (n=57) n (%)	Youth report (n=53) n (%)	Parent-report (n=43) n (%)	Youth report (n=33) n (%)
Motor Tics
Blinking	41 (71.9)	24 (45.3)	0	0
Eye movements	37 (64.9)	21 (39.6)	0	0
Head movements	35 (61.4)	26 (49.1)	0	0
Shoulder movements	35 (61.4)	25 (47.2)	0	0
Mouth, lip, or tongue movements	34 (59.7)	15 (28.3)	1 (2.3)	0
Making unusual faces	26 (45.6)	16 (30.2)	0	0
Nose movements	24 (42.1)	15 (28.3)	0	0
Other motor*	43 (75.4)	29 (54.7)	1 (2.3)	0
Vocal Tics
Coughing or clearing throat (“coughing”	41 (71.9)	27 (50.9)	1 (2.3)	0
High-pitched sounds	20 (35.1)	15 (28.3)	0	0
Sniffing	30 (52.6)	19 (35.9)	0	0
Animal sounds	17 (29.8)	13 (24.5)	0	0
Repeating others’ words or phrases	11 (19.3)	8 (15.1)	0	0
Blurting out words or phrases	9 (15.8)	9 (17.0)	0	1 (3.03)
Other vocal**	24 (42.1)	14 (26.4)	0	0

^*“Thinking back to since he/she/they turned five years old, was there ever a time when he/she/they kept making other sudden jerks or twitches or quick movements that I haven't asked you about … things that he/she/they kept doing again and again with another part of his/her/their body, like his/her/their arms, legs, feet, or stomach?”

^**“Since he/she/they turned five years old, was there ever a time when he/she/they kept making other sounds or noises that I haven't asked you… sounds or noises that he/she/they made again and again?”

Vocal tics:

Only two vocal tics were reported for more than half the children in the tic group by their parents on the DISC-5: coughing (71.9%) and sniffing (52.6%; see Table 3). The two lowest reported vocal tics by parents of children in the tic disorder group were repeating others’ words or phrases (19.3%) and blurting out words or phrases (15.8%). Among children in the tic disorder group, by DISC-5 self-report, the most commonly reported vocal tics were coughing (50.9%), sniffing (35.9%), high-pitched sounds (28.3%), “other vocal” tics (26.4%), and animal sounds (24.5%).

In the comparison group (children without a tic disorder), one parent reported a single vocal tic (coughing) on the DISC-5. One child in the comparison group reported a single tic (blurting out words or phrases) on the DISC-5.

Patterns of tics:

Patterns of tics reported were generally unique (data not shown). For parent-report of motor tics on the DISC-5, for the tic disorder group, most combinations of tics occurred only once (n=40 unique combinations of tics). Two parents endorsed both mouth and head movements, two endorsed blinking, eye, nose and shoulder movements as well as “other motor”, two endorsed blinking, eye, nose, and head movements as well as “other motor”, two endorsed all motor tics except head movements, four endorsed all except nose movements, and five endorsed all motor tics including “other motor.” No parent endorsed only eye, nose, mouth, head, or shoulder movements, or making unusual faces without endorsing a second motor tic (i.e., at least one other motor tic was endorsed when any of these tics were endorsed by a parent). Seven children in the tic disorder group reported no motor tics, five reported shoulder movements and “other motor,” and two each reported: head and shoulder movements plus “other motor;” blinking and “other motor;” blinking, eye, face, and head movements, plus “other motor;” all 8 motor tics. The other 33 children with a tic disorder endorsed unique combinations of tics. No child with a tic disorder only endorsed making unusual faces, head, or shoulder movements.

For vocal tics, there was more overlap in reporting of specific tics. By parent-DISC-5 report, three children with a tic disorder had no vocal tics reported, six only reported coughing, and five reported coughing, sniffing, and “other vocal” tics. Three parents each reported the following patterns: sniffing only; coughing and animal sounds. Two parents from the tic disorder group endorsed all seven vocal tics. No parents only endorsed high-pitched sounds, animal sounds, blurting out words or phrases, or “other vocal” tics. By child self-report on the DISC-5, among children with a tic disorder, seven children reported no vocal tics, seven only reported “other vocal” tics, six only reported coughing, four only reported sniffing, and two reported coughing and repeating others’ words. None of the children with a tic disorder endorsed all six vocal tics, and all other combinations of tics endorsed (n=25) were unique. No child with a tic disorder endorsed the following tics in isolation: high-pitched sounds, animal sounds, and repeating others’ words or phrases; however, one child only reported high-pitched and animal sounds, and only one endorsed blurting out and repeating words and phrases.

DISC-5 ADHD Module compared to expert clinical assessment

As shown in Table 4, the DISC-5 ADHD parent module had good (80.8%) sensitivity for identifying children aged 6–17 years with ADHD diagnosed through expert clinical assessment. Although sensitivity for the youth-report module was lower (65%), the sensitivity of combined parent or youth report together (85.0%) was higher than for either alone. In addition, adding teacher report of ADHD symptoms based on the VADTRS increased sensitivity over parent-report alone, and including information from all three respondents (parent, youth, teacher) resulted in 100% sensitivity, although positive predictive value was low (41.0%).

Table 4.

Comparison of the Diagnostic Interview Schedule for Children Version 5 (DISC-5) Attention-deficit/hyperactivity disorder (ADHD) module and expert clinical assessment.

Assessment of ADHD	n	Sensitivity	Specificity	Positive Predictive Value (PPV)	Negative Predictive Value (NPV)	Accuracy
Parent-Reported DISC-5	100	21/26 = 80.8% CI: 73.0-88.5%	53/74 = 71.6% CI: 62.8-80.5%	21/42 = 50.0% CI: 40.2-59.8%	53/58 = 91.4% CI: 85.9-96.9%	74/100 = 74.0% 65.4-82.6%
Youth-Reported DISC-5	84	13/20 = 65.0% CI: 54.8-75.2%	53/64 = 82.8% CI: 73.9-90.3%	13/24 = 54.2% CI: 43.5-64.8%	53/60 = 88.3% CI: 81.5-95.2%	66/84 = 78.6% CI: 69.8-87.3%
Parent-Reported DISC-5 & Youth-Reported DISC-5	84	17/20 = 85.0% CI: 77.4-92.6%	39/64 = 60.9% CI: 50.5-71.4%	17/42 = 40.5% CI: 30.0-51.0%	39/42 = 92.9% CI: 87.3-98.4%	56/84 = 66.7% CI: 56.6-76.7%
Parent-Reported DISC-5 & VADTRS	72	18/19 = 94.7% CI: 89.6-99.9%	34/53 = 64.2% CI: 53.1-75.2%	18/37 = 48.6% CI: 37.1-60.2%	34/35 = 97.1% CI: 93.3-100.0%	52/72 = 72.2% CI: 61.9-82.6%
Youth-Reported DISC-5 & VADTRS	66	14/16 = 87.5% CI: 79.5-95.5%	40/50 = 80.0% CI: 70.3-89.7%	14/24 = 58.3% CI: 46.4-70.2%	40/42 = 95.2% CI: 90.1-100.0%	54/66 = 81.8% CI: 72.5-91.1%
Parent-Reported DISC-5, Youth-Reported DISC-5, & VADTRS	66	16/16 = 100.0% CI: 100.0-100.0%	27/50 = 54.0% CI: 42.0-66.0%	16/39 = 41.0% CI: 29.2-52.9%	27/27 = 100.0% CI: 100.0-100.0%	43/66 = 65.2% CI: 53.7-76.6%

CI: 95% confidence interval; VADTRS - Vanderbilt ADHD Diagnostic Teacher- rating scale

^*A total of 100 parents completed the DISC-5 assessment, and 86 children completed the youth report DISC-5. Two children (both aged 7 years) were excluded from the youth self-report analysis for ADHD because they were missing data (did not report) for age of onset before the age of 12 years, impairment, and whether the symptoms occurred in more than one setting. A teacher report was available for 72 of the children in the study.

Specificity was highest for youth report alone (82.8%) and for youth and teacher report combined (80.0%). Positive predictive value was generally low, with the highest PPV of 58.3% for youth and teacher report (VADTRS) combined. Negative predictive value was high for all respondents and combinations (88.3% or greater). Accuracy was lowest for combined parent, youth, and teacher report (65.2%) and highest for youth and teacher report (81.8%).

Sensitivity analyses that included a single sibling per family generally found similar results to the full sample. Most estimates of sensitivity, specificity, PPV, and NPV for the restricted sample were within 5% of the estimates of the full sample, and therefore are not presented in addition to Table 4. The exceptions were that the sensitivity of youth report increased 5.6%, and specificity decreased over 5% for parent report (5.5%), parent/youth report (7.1%), parent/teacher report (6.0%), and combined parent/youth/teacher report (6.4%).

Discussion

Accurate identification of mental disorders including tic disorders and ADHD is important in both clinical and research settings to provide appropriate care and to precisely characterize study groups. The findings reported here suggest the DISC-5 tic disorder module may be a good measure for assessing DSM-5 diagnostic criteria for these disorders. This is an improvement over the poor agreement between the DISC-IV tic module and expert clinical assessment (Lewin et al., 2014). Importantly, in the earlier study examining the performance of the DISC-IV tic module, half of the children with tic disorders and all the comparison children were recruited from the same clinic and community as the current study (although the specific participants were unique), suggesting that the improvement in the DISC-5 tic disorder module over that of DISC-IV module was not specific to a difference in the underlying population sample. The ADHD module did not perform as well as in previous studies (McGrath et al., 2004; Rolon-Arroyo et al., 2016); since the sample size for ADHD was particularly small in our study, especially for children who did not also have tic disorders, additional evaluation of the ADHD module may be needed.

In our analysis of individual tics, we found all tics included in the DISC-5 were more commonly reported for children with tic disorders than for children without tic disorders (who had very few tics reported), parents were more likely to report tics than children (for the tic disorder group), and motor tics were endorsed somewhat more frequently than vocal tics. In agreement with previous studies (Ganos et al., 2015), eye tics were reported most often. Among children with tic disorders, there was great variation in the pattern of specific tics reported. Based on the DISC-5 algorithm used to evaluate tic disorder criteria in this study, and the strong agreement between the DISC-5 tic module and expert clinical assessment, future versions of the DISC-5 might be able to eliminate or combine items to shorten the length of the module. Specifically, we did not include information obtained from the open-ended questions asking about tics (see question 2B in the Figure) and this question (which is repeated for 6 other motor tics and 4 vocal tics) may not be necessary. For motor tics, nose movements and making unusual faces were least commonly reported by parents of children with tic disorders. Nose movements and making unusual faces, as well as mouth, lip, or tongue movements were reported by fewer than one-third of children with tic disorders. In addition, making unusual faces was never the only tic reported, and therefore could be eliminated without changing the results of this analysis. For vocal tics, the two least commonly reported tics by both parents and children were blurting out words or phrases and repeating other people’s words or phrases. For children with tic disorders, no parent or child only reported high-pitched sounds or animal sounds. Based on these results, the DISC-5 will be revised for future studies. Specifically, the item inquiring about making unusual faces will be dropped, and the questions inquiring about high-pitched and animal sounds will be combined. In addition, the items asking for a description of endorsed tics will be optional, except for the final questions about “other motor” or “other vocal” tics.

Previous studies have shown parents may be better reporters of ADHD symptoms than children (Jensen et al., 1999), which aligns with the better sensitivity of the parent-report DISC-5 (vs. child-report) with clinical diagnosis. Another earlier version of the DISC (corresponding to DSM-III; American Psychiatric Association, 1980) showed poor agreement between youth report and clinician diagnosis of ADHD in a psychiatric hospital setting (Aronen et al., 1993). While each of these studies support use of the parent-report DISC when assessing ADHD, additional research is needed to understand how youth report can contribute to establishing whether a child meets criteria for a specified disorder. The addition of teacher report of ADHD symptoms on the VADTRS to parent- and child-report on the DISC-5 also improved sensitivity for ADHD.

Similarly, in the DISC-2 (Fisher et al., 1993), DISC-IV (Lewin et al., 2014), and current study using DISC-5, parent-report of tic disorders performed better than youth report. A study of a different diagnostic measure for tics found a substantial number of children who met diagnostic criteria for a tic disorder where only the child (and not the parent) endorsed tic symptoms (Bitsko et al., in preparation for this issue). Validated measures that include child or teacher report may be most useful in large epidemiologic studies conducted in school settings where parents may not participate.

Our study has several limitations. First, given the relatively low prevalence of tic disorders, participants were recruited from a specialty clinic to maximize the number of children with tic disorders for assessment; these children may be more likely to have more easily recognizable tics, given that they are receiving treatment, than children from the community. Additional research is needed to understand how the DISC-5 performs in identifying children from among a general population sample, who haven’t yet sought treatment or been given a tic disorder diagnosis but would meet criteria if expertly assessed, which is important for clinical assessment, research, and epidemiologic studies. Since PPV and NPV are influenced by prevalence of the disorder, the DISC-5 may not perform as well for these measures in a general population sample with a lower prevalence of tic disorders (Yi et al., 2004). This point is illustrated by the findings in two studies using another diagnostic measure for tic disorders, the Description of Tic Symptoms (DoTS; Adams et al., in press; Bitsko et al., in preparation). The DoTS performed well in identifying tic disorders as part of the same study reported here; sensitivity for both parent and child DoTS was 100%, specificity of the parent DoTS was 92.7% and child DoTS specificity was 75.9% for any tic disorder (Adams et al., in press). However, the DoTS did not perform as well in identifying tic disorders in a community-based epidemiologic study that was composed primarily of individuals without tic disorders (Bitsko et al., in preparation). Also, despite efforts to recruit from a diverse population, over half of participants in each study group were non-Hispanic White children, and there were no Black participants in the study. Females were also under-represented in the tic disorder and ADHD groups, although this aligns with the higher prevalence of these disorders in males (American Psychiatric Association, 2013). Future studies need to determine whether the DISC-5 performs similarly in a more diverse and generally representative sample. Third, our study included only 26 participants with ADHD and most had a tic disorder. Furthermore, most of the participants in the tic disorder group had Tourette syndrome; thus, we could not evaluate the performance of the DISC-5 for other tic disorders (i.e., persistent motor or vocal tic disorders, or provisional tic disorder) and it is possible that ADHD symptoms are reported differently for children with tic disorders. Future studies could recruit a sample of children with tic disorders as well as an enriched sample for ADHD and other disorders of interest to address these limitations, as well as to evaluate other modules of the DISC-5. It is important to note that there were few changes to the DISC-5 ADHD module and previous validation studies of the DISC (versions II and III) ADHD module showed it performed well, particularly with parent report (McGrath et al., 2004; Piacentini et al., 1993). Finally, approximately one-third of the sample included siblings, which could have biased parent-reporting or represented a smaller overall sample of parents reporting on the DISC-5. However, most of the estimates of the sensitivity analysis that included only a single sibling per family were similar to those obtained from the full sample.

Despite these limitations, the results are encouraging for the use of DISC-5 for identifying children with tic disorders in clinical and epidemiologic settings and demonstrate an improvement in sensitivity and specificity over previous versions of the DISC. Parent-report or a combination of parent and child report performed best for the assessment of tic disorders. Additional research with a larger more representative sample could help validate the DISC-5 ADHD module. Because the DISC-5 does not rely on expert assessment, it may serve as a useful instrument in future epidemiologic studies to improve our understanding of the prevalence of childhood tic disorders, ADHD, and other mental disorders.