Utility of the Diagnostic Interview Schedule for Children for Assessing Tourette Syndrome in Children

Abstract

Objective: The Diagnostic Interview Schedule for Children IV (DISC) has been used extensively in research and screening. Despite wide use, little information exists on the validity of the DISC for diagnosing tic disorders.

Methods: Participants were 181 youth with expert clinician-diagnosed Tourette syndrome (TS).

Results: Using expert clinician-diagnosed TS as the gold standard, the sensitivity of the DISC-Y (youth, 0.27) and DISC-P (parent, 0.44) was poor. The DISC-Y identified 29.7% of youth with diagnosed TS whereas the DISC-P identified 47.4% of cases. Only 54% of cases of TS were detected by either the DISC-Y or -P. Diagnostic agreement between the DISC and expert clinician diagnosis was poor. The DISC-Y/P results did not differ as a function of tic severity.

Conclusions: Despite utility for assessing child psychiatric disorders, the sensitivity of the DISC for detecting TS appears poor. This study suggests that DISC has low agreement with expert clinician diagnosis of TS. Findings highlight the need for modification of the DISC and/or the identification and development of more sensitive measures for TS screening.

Introduction

Tourette Syndrome (TS) is a movement disorder characterized by motor and vocal tics that wax and wane in severity (American Psychiatric Association 2000). Peak onset occurs between ages 5 and 7 years, and has a male preponderance (Leckman 2002). Maximal tic severity is typically in early adolescence, generally followed by a gradual decrease in severity (Leckman et al. 1998) with many cases remitting by young adulthood (Bloch et al. 2006). Prevalence estimates of TS and other tic disorders vary widely across studies, with estimates of TS ranging from 1 to 30 per 1000 children (Kraft et al. 2012). Although recent prevalence estimates for TS have fallen into a narrower range of 3–8 per 1000, there is still much uncertainty about the prevalence of TS and other chronic tic disorders (CTD), especially because many individuals may not seek treatment (Scharf et al. 2012).

Diagnostic criteria for TS are relatively unambiguous. The American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, 4th ed., Text Revision (DSM-IV-TR) requires that: 1) Both multiple motor and at least one phonic tic be present during the illness, 2) tics occur many times a day, nearly every day (or off and on) persisting for at least 1 year without a single tic-free gap of >3 months, 3) onset before 18 years of age, and 4) tics not accounted for by substance or other medical condition. However, diagnosis of TS is not always straightforward. Tics may be mistaken for stereotypies or movement disorders (e.g., dystonia, chorea [Zinner and Mink 2010; Murphy et al. 2013]). In addition, waxing and waning symptoms and high rates of co-occurring conditions may contribute to difficulties identifying TS (Coffey et al. 2000). For example, rituals associated with obsessive-compulsive disorder (OCD), a common comorbid condition with TS, may mimic complex tics (Mansueto and Keuler 2005; Murphy et al. 2013). Further hindering diagnosis, tics may go unrecognized by parents and teachers and may be mislabeled even when they are noticed, with many patients initially presenting to otolaryngology, ophthalmology, or asthma/allergy specialists (Kovacich 2008). To date, there are no formal laboratory or imaging assays for TS and the practice recommendation for establishing psychiatric diagnosis in youth including TS is via clinician evaluation (which includes a developmental history and examination) (Scahill et al. 2006; Cath et al. 2011; Murphy et al. 2013).

There is a need to enhance identification and diagnosis of TS, especially outside of specialty programs where continued study of prevalence and impact of TS is needed. Although both research-supported pharmacotherapy and behavioral treatment options are available for youth with chronic and impairing tic disorders, referral to expert providers hinges on reliable and efficient identification and diagnosis (Selles et al. 2013). The new practice parameter for the assessment and treatment of TS and tic disorders from the American Association of Child and Adolescent Psychiatry (Murphy et al. 2013) highlights the importance of a comprehensive assessment in order to prescribe optimal treatment. The new practice guidelines highlight the importance of assessing impairment (Lewin et al. 2011) in determining treatment course, strengthening the need for evidence-based assessment (McGuire et al. 2012). Structured diagnostic interviews may provide a reliable and systematic approach for establishing diagnoses for both clinical and research purposes. The National Institute of Mental Health Diagnostic Interview Schedule for Children (DISC) (Shaffer et al. 2000) is a structured diagnostic interview (4th edition) originally developed to identify symptoms associated with the most common psychopathologies affecting youth (Costello et al. 1985). The DISC was designed to be administered by interviewers without any formal clinical training (Fisher et al. 1993). Originally intended for large-scale epidemiologic surveys of children, the DISC has been used in many clinical studies, screening projects, and service settings (Shaffer, et al. 1993; Roberts, et al. 2007; Ezpeleta et al. 2011). The interview covers >30 diagnoses, including tic disorders, and assigns probable diagnoses following an algorithm based on DSM-IV (American Psychiatric Association 2000) criteria. The DISC has a number of strengths not seen in other structured diagnostic interviews, because of the systematic structure and reduced subjectivity inherent in the algorithm-based assessment (Hodges 1993). Strong sensitivity (Fisher et al. 1993) and test–retest reliability (Jensen et al. 1995; Roberts et al. 1996; Shaffer et al. 2000) have been demonstrated for eating disorders, OCD, psychosis, major depressive episode, and substance use disorders.

Nevertheless, prior studies have shown low agreement between a gold standard clinician diagnosis and diagnosis by the DISC for other conditions (Costello et al. 1984). In a study of 163 child inpatients, uniformly low agreement was obtained with DISC-generated diagnoses when compared with psychiatrist diagnosis (Weinstein et al. 1989). There was a strong tendency toward overdiagnosis by the DISC in that study (which featured a previous version of the DISC). Although marginally improved, agreement remained poor when a secondary DISC algorithm designed to assign diagnoses (based on a more conservative diagnostic threshold) was implemented. Notably, this older edition of the DISC did not include a parent report, and the algorithm did not sufficiently correspond to the current diagnostic criteria from the American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, 3rd ed. (DSM-III) (American Psychiatric Association 1980).

A more recent study examining clinician–DISC agreement using the most updated DISC (i.e., the DISC-IV) edition found deviations between DISC and clinician diagnosis in 240 youth recruited from a community mental health center. Specifically, the prevalence of attention-deficit/hyperactivity disorder (ADHD), disruptive behavior disorders, and anxiety disorders was significantly higher based on the DISC diagnosis, whereas the prevalence of mood disorders was higher based on the clinician's diagnosis (Lewczyk et al. 2003). As the DISC does not assess all DSM criteria (e.g., exclusion based on a medical condition), this could contribute to some of the differences between prevalence estimates.

Despite its wide use, there is little information on the validity of the DISC as a diagnostic tool for tic disorders. In a study of 12 children with TS, the sensitivity of the DISC (2nd ed.) for any tic disorder was high; using the parent report, the DISC identified all 12 children who had TS as having a tic disorder (Fisher et al. 1993). Using the child report, 8 of 12 cases were correctly identified. However, the criteria for accuracy only stated that the DISC should identify the child with any tic disorder, not a specific tic disorder (e.g., TS). Therefore, no conclusion can be drawn from that study on the sensitivity of the DISC for diagnosing TS specifically.

The principal aim of our study was to evaluate the validity of the tic disorder portion of the DISC-IV (hereafter referred to as DISC) for the assessment of well-characterized sample youth with TS. Secondary aims included examining: 1) Parent–youth agreement on the tic disorder module of the DISC, 2) age variation in agreement, and 3) associations between DISC-generated TS diagnoses and tic severity assessed on the Yale Global Tic Severity Scale (YGTSS) (Leckman et al. 1989). Based on results from the validity evaluation, we also examined the DISC classification algorithm for TS to identify areas where the classification system went awry.

Method

Participants

Participants were 181 children and adolescents with a clinician-diagnosis of TS, recruited from the normal patient flow of the University of South Florida's (USF) Child and Adolescent OCD and Tic Disorder Clinic and the University of Rochester's (UR) Tourette Syndrome Clinic. All participants were part of a larger study examining psychosocial functioning among youth with TS (in comparison with controls without TS or another tic disorder). Inclusion criteria for participants with TS were that youth had a current diagnosis of TS made by an expert clinician and were between 6 and 18 years of age at the time of evaluation. Participants were excluded if there was a positive diagnosis of intellectual disability, psychosis, mania, suicidal intent, or any other psychiatric condition that would limit their ability to understand or complete study assessments. Inclusion criteria for controls were that youth did not have any tic disorder; youth with first degree relatives with TS were excluded. Control subjects were recruited at the UR site from community pediatric practices, as well as through study advertisements posted in public settings, in the community, and via online parenting forums, and used for comparative analyses.

Measures

The DISC is a highly structured psychiatric diagnostic interview with parallel versions for parents of children and adolescents 6–18 years of age (DISC-P) and youth ages 9–18 (DISC-Y). The majority of DISC questions are designed so respondents can answer “yes,” “no,” or “sometimes/somewhat.” The DISC is scored using a computer algorithm, programmed in SAS (Statistical Analysis System) (SAS 2008). Algorithms have been prepared to score both the parent and the youth versions of the DISC-IV according to the symptom criteria listed in the DSM-IV diagnostic system.

In the present study, the DISC-Tic Disorders Module was administered. The module produces the following tic diagnoses: TS, CTD (chronic motor or phonic tic disorder), transient tic disorder (TTD), and no tic diagnosis. Parents and youth were administered the DISC independent of each other, but in the presence of a clinician or study staff with tic disorder expertise. From this point forward, reference to the DISC refers to the Tic Disorder Module.

Establishment of TS diagnosis on the DISC requires fulfillment of two criteria. Criterion A, the presence of multiple motor tics and at least one phonic tic; and criterion B, tics occurring many times a day, nearly every day, for at least 1 year, without a 3 month absence of tics. Respondents are first asked about the presence of tics symptoms but not about frequency or timeline of tics, starting with a single motor tic. If they have a motor tic, they are asked about the presence of additional motor tics. Next, respondents are asked about the presence of phonic tics. Chronicity (i.e., frequency, timeline) of motor and subsequently phonic tics is ascertained for any respondent with at least one motor (and subsequently phonic) tic.

The YGTSS is a clinician-rated, semistructured interview that assesses tic symptoms and severity in the preceding week (Leckman et al. 1989). Motor and phonic tics are rated separately providing a severity score of 0–50. Excellent reliability data have been reported, including high internal consistency (α=0.92–0.93), interrater reliability (intraclass correlation coefficients [ICCs] for index scores=0.62–0.85), and 7 week stability (ICC=0.77–0.90) (Leckman et al. 1989). Validity support is derived from a stable factor structure, positive correlations with parent- and clinician-rated tics, and weak nonsignificant correlations with obsessive-compulsive symptoms, depression, and anxiety.

Procedures

Participants were screened via phone prior to their clinic visit for study appropriateness, and the full assessment was conducted when they presented to clinic. As part of the larger study, all parent and youth participants at both sites completed a 3–4 hour long study visit that consisted of separate parent and child DISC interviews, a detailed clinician assessment (including the YGTSS), and several detailed self-rating and parent-rating forms to assess child behavior, psychological comorbidities, family functioning, quality of life, and sociodemographic and healthcare indicators. Licensed/board certified child and adolescent psychologists and board-certified physicians administered the YGTSS with parents and children jointly. These clinicians were independent of the clinician assigning the expert diagnosis. All clinicians were highly experienced with youth with tic disorders and underwent YGTSS rater reliability training, which included several practice interviews using training videos developed by Lawrence Scahill (Yale University). Specifically, each rater scored above the 85% reliability on three consecutive training videos prior to conducting ratings for this study. Following this, trained research associates (with bachelor's degrees in psychology and at least 6 months experience working with TS in a clinical setting) administered the DISC independently to parents as well as youth aged ≥9 years of age. Ratings were completed by parents and/or youth; research assistants (supervised by licensed and board-certified clinicians) facilitated the administration. Families were given compensation to offset travel and expenses for completing assessments associated with the overarching study. The institutional review boards of both USF and UR approved all study procedures, and written informed consent and child assent was completed by parents and youth respectively.

Establishment of expert diagnosis

TS diagnosis was established by a comprehensive diagnostic evaluation conducted by the respective site principal or co-investigators (board certified child and adolescent psychiatrist and pediatric neurologists)¹ using all available clinical information, including examination, review of history/medical records for chronicity of symptoms, consensus review with other evaluating (MD/PhD) clinicians, and unstructured clinical interview (but not the DISC/YGTSS). Utilizing expert clinician evaluation is consistent with strategy for evaluating sensitivity-of-measurement as proposed previously (Fisher et al. 1993), examining performance in classifying uncommon neuropsychiatric syndromes in specialty centers with excepted expertise in diagnosis, which can serve as valid criterion references. Expert diagnosis is considered the gold standard of assessment of TS (Murphy et al. 2013). Prior to the study, the expert clinicians reviewed a series of cases under direction of an expert consultant to demonstrate full agreement of TS diagnoses. Of the 181 TS patient–parent dyads, 173 parents and 146 youth completed DISC assessments (DISC-Y was not administered to youth under age 9). Data on the DISC algorithm were available for 158 and 144 DISC-P and -Y respectively. Parent and child DISC data were then compared to the clinician diagnosed TS criterion (clinicians were not informed of DISC-Y/P ratings).

Data analysis

DISC Tic Disorders Module scoring and algorithm

SAS youth and parent scoring algorithms were used to generate diagnoses, criteria, or symptoms present for TS, CTD, or TTD for periods encompassing the past year and the past 4 weeks. Algorithms were supplied by the DISC Group, Columbia University.

Statistical analyses

Chi-square analyses were used to test for differences in the frequency of DISC-generated tic disorder diagnoses (e.g., TS, CTD, TTD, no tic diagnosis) across the two study sites. Analysis of variance (ANOVA) was employed to evaluate 1) age differences in DISC-generated diagnoses and 2) associations between DISC-generated diagnoses and tic severity (as rated by the YGTSS) with Tukey's post-hoc tests when indicated. Cohen's κs were reported for youth–parent agreement. We examined the frequency of DISC-generated tic diagnosis in recruited controls.

Results

Demographics

Youth ranged in age from 6 to 17 years old (mean=11.3±3.0). Control subjects (n=101) had a mean age=11.0±2.8. Participant demographics are described in Table 1.

Table 1.

Sample Demographics

	Total (%)	Male (%)	Female (%)
Youth with Tourette syndrome
Subjects enrolled	181	138 (76.2)	43 (23.8)
University of South Florida	97 (53.6)	77 (55.8)	20 (44.2)
University of Rochester	84 (46.4)	61 (44.2)	23 (55.8)
Race*
Caucasian	163 (90.1)	124 (91.2)	39 (90.7)
Hispanic	22 (11.7)	19 (13.2)	3 (7.0)
Asian	3 (1.7)	2 (1.5)	1 (2.3)
African American	11 (6.1)	6 (4.4)	5 (11.6)
Age (mean, SD)	11.3±3.0	11.3±3.1	11.2±2.5
Controls
Subjects enrolled	101	60 (59.4)	41 (40.6)
Race*
Caucasian	85 (84.2)	54 (90.0)	31 (75.6)
Hispanic	9 (8.9)	7 (11.7)	2 (4.9)
Asian	3 (3.0)	1 (1.7)	2 (4.9)
African American	18 (17.8)	9 (15.0)	9 (22.0)
Age (mean, SD)	11.0±2.8	11.0±2.8	11.0±2.9

^*Multiple race categories could be selected.

Diagnostic agreement between the DISC-Y/P and expert diagnosis

For the 146 youth who were all determined (via expert clinician diagnosis) to have TS, the DISC-Y generated the following tic-spectrum diagnoses (based on youth report): 29.7% TS, 31.1% CTD, 7.4% TTD, and 31.8% no tic disorder diagnosis. The DISC-P, administered to 173 parents of youth determined to have TS, identified the following tic disorders: 47.4% TS, 35.8% CTD, 1.7% TTD, and 15.1% no tic disorder diagnosis. Findings are presented in Figure 1. There were no site differences in DISC-Y/P tic diagnoses (χ²[3]=5.8 p=0.12 and χ²[3]=3.2, p=0.36, respectively) on the proportion of DISC-generated tic diagnoses (i.e., TS, CTD, TTD, and no tic diagnosis). Although ANOVA suggested possible age differences on the DISC-Y (F[3,144]=2.8, p=0.04), a Tukey's post-hoc test suggested that youth identified on the DISC-Y as TS were slightly younger (mean age=11.3) than youth identified on the DISC-Y as CTD (mean age=12.8; p=0.03). Age did not differ as a function of DISC-P tic diagnosis (F[3,167]=0.11, p=0.95) (Table 2). The sensitivity of the DISC-P (0.44) and DISC-Y (0.27) were poor, suggesting poor agreement between the DISC and expert clinical diagnosis (agreement did not differ by site). There were no false positives (no recruited controls were identified on the DISC as having TS or any other tic disorder).

FIG. 1.

Breakdown of Diagnostic Interview Schedule for Children (DISC)-generated tic disorder diagnosis for youth and parent respondents.

Table 2.

Percent of Subjects, by Age, with Tourette Syndrome by Expert Clinician Diagnosis, who Meet Criteria for Tourette Syndrome on the Diagnostic Interview Schedule for Children (DISC)

	6	7	8	9	10	11	12	13	14	15	16	17	Total
n (DISC-Y)	-	-	-	19	28	20	24	11	17	5	10	12	146
% DISC-Y	-	-	-	44%	41%	30%	38%	18%	25%	0%	11%	18%	30%
n (DISC-P)	4	16	15	18	28	20	23	11	14	5	9	10	173
% DISC-P	0%	44%	40%	63%	39%	60%	52%	46%	50%	60%	33%	40%	47%
n (DISC-P or Y)	4	16	15	19	28	20	24	11	17	5	10	12	181
% DISC-P or Y	0%	44%	40%	79%	54%	65%	63%	46%	59%	60%	40%	33%	54%

n, number of DISC interviews available for a given age; % DISC-Y, percentage of youth found to have Tourette syndrome (TS) based on youth (Y) report on the DISC; DISC-P, percentage of youth found to have TS based on parent (P) report; DISC-P or Y, intersection of both interviews. Only subjects ≥9 years of age completed the DISC-Y.

Tic severity

We examined whether DISC-generated diagnoses differed as a function of current tic severity. Tukey's post-hoc tests suggested that YGTSS tic severity was higher for youth with DISC-P-generated TS diagnoses (versus no tic disorder); otherwise, YGTSS did not differ as a function of DISC-Y/P-generated tic diagnoses (eg., TS, CTD, TTD, and no tic disorder).

Child–parent DISC diagnostic agreement

Rater agreement between the DISC-Y and DISC-P was poor (see Table 3). Among the 142 cases with both DISC-Y and DISC-P available, in only 27 cases did both DISC-Y and DISC-P generate a TS diagnosis (in 60 cases, neither DISC-Y nor DISC-P yielded a TS diagnosis and in 10 cases, both DISC-Y and DISC-P generated “no tic diagnosis” [i.e., no TS, CTD or TTD]).

Table 3.

Agreement of Youth Report with Parent Report on the Diagnostic Interview Schedule for Children (DISC) Among Youth Diagnosed with Tourette Syndrome

		Parent report on DISC (DISC-P)		κ
		TS+	TS-	0.21
Youth report on DISC (DISCY)	TS+	27	14
	TS-	41	60

Analysis of algorithm procedure

In order to better understand why youth with clinician-diagnosed TS were not being classified as having TS via the DISC, we examined responses to individual criterion on the DISC-Y/P diagnostic algorithms for the DISC Tic Disorders Module

DISC-Y

Algorithm data were available for 144 youth. Criterion A required having multiple motor/one or more phonic tics, and 55 youth failed to meet this criterion. Notably, 13 youth did not have any motor tics. Forty-nine youth failed DISC criterion B for TS: [Tics] many times a day/nearly every day These data are presented in Figure 2.

FIG. 2.

Youth respondents failing criteria for Tourette syndrome based on Diagnostic Interview Schedule for Children (DISC) algorithm.

DISC-P

Algorithm data, available for 158 DISC-P administrations, are presented in Figure 3. Twenty-three parents did not have enough tic symptoms to meet criterion A, and an additional 66 failed to meet the chronicity for criterion B. Twenty-eight youth did not meet chronicity requirements for motor tics, 21 failed for phonic tics, and 18 failed for both motor and phonic.

FIG. 3.

Parent respondents failing criteria for Tourette syndrome based on Diagnostic Interview Schedule for Children (DISC) algorithm.

Comparisons with YGTSS

Although the YGTSS assesses the presence and severity of tics over the past 7–10 days, the information solicited in YGTSS Severity Scale Item 1 (number of tics) closely resembles computerized DISC questions that assess the presence of motor and phonic tics (over the past year). The DISC queries (for motor tics), “Now I would like to ask you about muscle jerks or twitches, called tics, which people sometimes make. I'm talking about movements that a person can't keep from doing, like…blinking their eyes like this (instructions to the examiner to demonstrate)…or making other movements of the face like this…or shrugging their shoulders…or jerking their heads…or suddenly moving their arms or twisting their bodies. In the last year – that is, since [date] of last year – have you had any tics or movements that you felt you had to make?” Notably, of the 55 youth who failed DISC-Y criterion A, 34 were found to have multiple motor tics and at least one phonic tic on the YGTSS. In other words, they denied having any tics at any point over the past year, but indicated that they had had them in the past week. Similarly, of the 23 who failed DISC-P criterion A, all 23 were found to have had multiple motor tics and at least one phonic tic on the YGTSS during the past week. Similarly, of the 49 youth failing criterion B on the DISC-Y, 45 reported motor tics on at least a daily basis over the past week and 38 reported phonic tics on at least a daily basis over the past week (37 reported both motor and phonic tics on at least a daily basis over the past week). For the 66 failing Criterion B on the DISC-P, 58, 48, and 43 reported motor, phonic, or both tics, respectively, over the past week.

Discussion

These data show low agreement between the DISC-Y/P and expert clinical diagnosis of TS in a well-characterized sample of youth with TS. Although it has been suggested that the DISC may be the structured diagnostic interview of choice to avoid false negatives (Angold et al. 2012), the sensitivity of the DISC was poor across all ages, detecting only 54% of clinician-diagnosed cases (lower when considering sensitivity of either the parent or child interview when used singularly). Strikingly, a sizable percentage of youth determined by clinicians to have TS did not meet criteria for any tic disorder diagnosis when assessed via the DISC-Y/-P. Agreement between youth and parent DISC-generated tic diagnosis was low across all ages; this has been reported previously for externalizing disorders (Jensen, et al. 1999; Grills and Ollendick 2002). Although the DISC may offer a convenient and standardized alternative to clinician interview for establishing a TS diagnosis, the two diagnostic methods generally do not produce equivalent determinations.

Why the algorithm breaks down

Given that the DISC follows a systematic algorithm to derive diagnosis (based on the DSM), it is surprising that sensitivity for TS was so poor. It has been posited that structured interviews such as the DISC may be most appropriate for diagnoses with predictable patterns of symptoms and courses that are relatively consistent across settings and time (McClellan and Werry 2000). Perhaps the inherent fluctuation in tic symptoms may have contributed to poor detection of true cases of TS. A related explanation of the poor concordance between DISC and expert diagnosis is that respondents fail to adequately comprehend the questions related to required time parameters for experiencing tic symptoms (i.e., criterion B). However, weakening both potential explanations is the fact that 53% of youth and 26% of parents completing the DISC-Y/P failed DISC criterion A. In other words, they denied the presence of the requisite tics independent of time specifiers. Even more surprising, the overwhelming preponderance of youth failing to meet DISC-Y/-P criterion B stated that they had had frequent tics over the past week on the YGTSS. Notably, at both sites, the YGTSS was conducted prior to the DISC. It is striking that tic symptom endorsement was so low on the DISC, despite an explicit, joint parent–child–clinician discussion of tic phenomenology in the context of the YGTSS, preceding administration of the DISC. A discrepancy between the DISC TS algorithm and the DSM-IV-TR TS criteria may explain some cases missed cases. Specifically, the DSM-IV-TR requires that “both multiple motor and one or more vocal tics have been present at some time during the illness but not necessarily concurrently.” However, the DISC algorithm requires the presence of both multiple motor and at least one phonic tic, each many times a day/most days, over a period of >1 year. Notably only two (DISC-Y) and one (DISC-P) cases failed to be classified as TS because of the aforementioned algorithmic discrepancy. Consequently, this deviation from DSM criteria does not explain the majority of cases that were not correctly identified. It is interesting that both parents and children commonly failed endorsement of criterion B. Even if youth struggled with comprehension of the items, the high rates of parents failing to endorse symptoms suggests that youth comprehension is not the only barrier. Although the aim of this study was to examine DISC classification of TS, the USF site also examined DISC-generated diagnoses of youth with clinician expert-identified CTD and TDD. Rates of correct classification mirrored findings for TS, suggesting that the DISC would perform poorly in correct classification of other specific tic disorders.

As discussed, responses on the YGTSS were robustly consistent with DSM criteria for TS (with the obvious exception of the different timing windows; the YGTSS only capturing symptoms over the past 10 days). Nevertheless, even if only considering the presence/topography of tic symptoms, the YGTSS (conducted by an independent clinician) was consistent with the expert diagnosis, whereas the DISC tended to deviate from both (note that the YGTSS rater was independent of expert diagnosis). Perhaps the more open-ended format of the YGTSS allowed for flexibility of follow-up queries, provided an opportunity for increased dialogue between the clinician and the respondent, and allowed the clinician to directly ask about observed symptoms, resulting in more reliable solicitation of pertinent information. Furthermore, not only does the YGTSS enable the clinician evaluator to ask follow-up questions about symptoms, but it also includes observations in the completion of the form. That is, even if a child/parent does not endorse a tic, if the evaluator observes a tic, it can be noted on the YGTSS (or discussed in the context of the evaluation). Therefore, in essence, the YGTSS evaluator becomes a third informant in the assessment of TS.

Limitations

There are several noteworthy limitations to discuss. First, our study design prevents us from being able to examine specificity and comment on negative predictive value. Whereas sensitivity and specificity are characteristics of the DISC as a test, positive predictive value (PPV) and negative predictive value (NPV) depend upon the prevalence of TS in our sample. Because our sample had so many TS cases, it is almost a given that we would have high PPV and low NPV. Additionally, whereas it is encouraging that no recruited controls were identified as having TS using the DISC, a more substantial test of specificity would be best conducted in a sample that was not screened to exclude tic disorders. Generalization presents another limitation; both sites are specialty centers for childhood tic disorders. It is unclear how the DISC would perform in contrast to clinician diagnosis among providers who are not experts in the assessment and treatment of TS. It is noteworthy that at two specialty sites, with DISC administration in large part by technicians who were trained and closely supervised by TS experts, the DISC performed poorly in identifying cases diagnosed by a clinician. Additionally, the sample was primarily Caucasian; the need for improved minority recruitment is clear for understanding the detection and classification of tic disorders in other ethnic groups. Although the majority of youth with TS were male, females were the majority among controls. Finally, even though 1) the study principal investigators are senior experts who demonstrated diagnostic agreement prior to the study and 2) cases were reviewed for consensus, it is possible that expert diagnosis is not always correct. However, the extent of disagreement would unlikely be explained even if cases were misidentified by the expert.

Conclusions

Although the DISC has utility for the diagnosis of many child psychiatric disorders, this study revealed weaknesses in detecting TS. Notably, there are a number of advantages offered by structured interviews such as the DISC relative to unstructured approaches to diagnosis. For example, in following an algorithmic approach to disease classification tied to DSM criteria, the DISC eliminates variability in information queried, probes symptoms that may be missed in an unstructured review, avoids clinician subjectivity, and allows nonclinicians to administer the interview (Weinstein et al. 1989; McClellan and Werry 2000). The findings in this study suggest improved reliability between more subjective approaches (semi-structured interview [YGTSS] and clinician diagnostic interview) in gathering information about tics. It appears there are roles for structured and unstructured assessment of childhood tic disorders. Perhaps a clinician-assisted computer interface combined with highly structured queries is not sufficiently flexible in its present state for ascertaining the requisite information necessary to quantify tic presence and chronicity, let alone establish a TS diagnosis. Modification to the algorithm, including more careful construction of the structured interview and greater similarity to expert clinician approach may enhance correct TS identification. Perhaps aspects of the YGTSS can be incorporated into the DISC. With the YGTSS, many more prompts about different types of tics, across different categories of motor and phonic tics, are embedded. Perhaps adding the requisite chronicity questions within this format could improve accuracy.

Clinical Significance

Changes needed for American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, 5th ed. (DSM-V)

Modifications in TS criteria for the DSM-V pertain mostly to relaxing chronicity restrictions (American Psychiatric Association 2013). Instead of stating “tics occur many times a day (usually in bouts) nearly every day or intermittently throughout a period of more than 1 year,” as in DSM-IV-TR, the DSM-V states “tics may wax and wane in frequency but have persisted for more than 1 year since first tic onset.” Prohibition from diagnosis for a tic-free 3 month period is removed. Consequently, many of the questions in Section B are no longer necessary. The only chronicity restriction that is required is determining whether tics have been present for >1 year since first tic onset (in order to separate TS from provisional tic disorder in DSM-V). However, even if we omit the prohibition of a 3 month tic-free interval to more closely approximate DSM-V criteria, only two additional youth would be identified as TS (on the DISC-P). Five youth (DISC-Y) and six (DISC-P) would meet TS criteria if the >1 year requirement were waived. However, whereas the DISC-IV requires motor and vocal tics over the past year, the DSM-V allows for motor and vocal tic presence over any single year (not necessarily concurrent). Consequently, even if a revision to the DISC is produced based on DSM-V changes for TS diagnostic criteria, our data suggest continued preponderance of false negatives. Consequently, broader changes to future DISC Tic Module iterations are needed to enhance sensitivity of diagnosing TS (and likely other CTDs).

Although there are several studies supporting the reliability of the DISC, our data suggest poor parent–youth agreement, and, moreover, unacceptable criterion validity when assessing TS. Not only does the DISC show low agreement with expert clinical diagnosis of TS in a well- characterized sample of youth with TS, but also a sizable percentage of youth were determined to have no tic disorder. Endorsement of tic symptoms is in striking contrast to those reported on the YGTSS. Perhaps the psychoeducation inherent in the YGTSS could be incorporated into the DISC for improved reporting. For example, prior to the YGTSS checklist, definitions and examples of tics were provided (e.g., motor vs. phonic, simple and complex). This education by experienced child and adolescent psychologists may have facilitated responding on the YGTSS. Although the reason for poor performance may not be fully understood, it is apparent that the DISC is not sufficiently sensitive for identifying TS as diagnosed by expert clinicians. Relying on the DISC alone will likely produce underestimates (especially given that youth in the sample were recruited and comprehensively screened for having TS with symptoms currently present). Findings highlight the need for the identification and/or development of more sensitive measures for identifying TS in epidemiologic studies. Modification of questions to correspond to the DSM-V may reduce the complexity in establishing criterion B, but broader changes to the administration format may be required for any overall improvement in the detection of TS.

Acknowledgments

We acknowledge the assistance of Leah Jung with this research.

Disclosures

Adam B. Lewin serves as a consultant for Otsuka America Pharmaceutical and ProPhase, Inc. He receives grant support from International Obsessive Compulsive Disorder Foundation (IOCDF), National Alliance for Research on Schizophrenia and Depression, University of South Florida Research Foundation, Inc., and the Springer Textbook Honorarium. He has received travel support from University of South Florida Research Foundation, Inc., has a publishing agreement with Springer and Taylor & Francis, and receives a speaker's honorarium from the Tourette Syndrome Association (TSA). Jonathan W. Mink serves as a consultant for Medtronic, Inc. He has received grants from the Centers for Disease Control and Prevention (CDC), the Food and Drug Administration (FDA) of the United States Public Health Service, and the National Institute of Neurological Disorders and Stroke (NINDS). He is on the Data and Safety Monitoring Board for Edison Pharmaceuticals and receives an honorarium from the American Academy of Neurology and the Tourette Syndrome Association. Rebecca H. Bitsko has no financial relationships to disclose, as Dr. Bitsko works for the Centers for Disease Control and Prevention. Joseph R. Holbrook has no financial relationships to disclose, as Dr. Bitsko works for the Centers for Disease Control and Prevention. E. Carla Parker-Athill has no financial relationships to disclose. Camille Hanks receives research support from CDC and Shire Pharmaceuticals Inc. Eric A. Storch serves on the advisory board for the International Obsessive Compulsive Disorder Foundation. He serves as a consultant for Otsuka America Pharmaceutical, Inc. and ProPhase Inc. He receives grant support from Centers for Disease Control; National Institutes of Health; Ortho-McNeil Neurologics; and the Tourette Syndrome Association. He has intellectual property with Springer and Taylor & Francis. He serves on the speakers bureau for the International Obsessive Compulsive Disorder Foundation. Erika F. Augustine has received grant support from the TSA, the FDA, the International Essential Tremor Foundation, the New York State Department of Health, and the National Institute of Neurological Disorders and Storke. She is on a Data Safety Monitoring Board for Edison Pharmaceuticals and receives an honorarium from the American Academy of Neurology. Heather R. Adams receives grant support from the Tourette Syndrome Association (TSA). Amy E. Vierhile has no financial relationships to disclose. Alyssa R. Thatcher has no financial relationships to disclose. Tanya K. Murphy receives research funding from AstraZeneca Research & Development, Brain and Behavior Research Foundation, the CDC, F. Hoffmann-La Roche Ltd., Indevus Pharmaceuticals, IOCDF, National Institutes of Health/National Institute of Mental Health (NIH/NIMH), Ortho-McNeil Janssen Pharmaceuticals, Otsuka Pharmaceuticals, Pfizer, Inc., and Shire Pharmaceuticals. She has received travel support from the Tourette Syndrome Association and honoraria from grand rounds lectures.