Screening Current and Future Diagnosis of Psychiatric Disorders Using the Revised Drug Use Screening Inventory

Abstract

The revised Drug Use Screening Inventory (DUSI-R) is a valid and reliable self-report questionnaire used for quantifying problems that frequently precede and co-occur with substance abuse. The present investigation determined whether the DUSI-R's items can be aggregated into scales that implicate current and future psychiatric disorders. Scales were derived to screen for attention deficit, conduct, antisocial, anxiety, depression, and substance use disorders in a longitudinally tracked cohort of 328 boys. Evaluations were conducted when the boys were 12–14, 16, 19, and 22 years of age. All of the scales identified youths qualifying for current DSM-IV diagnosis with excellent accuracy. Predictive validity of the scales ranged from good to excellent. Accordingly cut-off scores were determined for each scale for use in practical settings to identify youths who require comprehensive diagnostic evaluation. Thus in addition to its utility for detecting problems that precede and correlate with substance abuse, the DUSI-R is cost-efficient for screening youths for mental disorders.

The revised Drug Use Screening Inventory (DUSI-R) is a 159-item self-report questionnaire designed to quantify problems which commonly presage and co-occur with substance abuse (1). The measurement domains are: 1) substance use, 2) behavior patterns, 3) health status, 4) psychiatric disturbance, 5) social competence, 6) family system, 7) school performance, 8) work adjustment, 9) peer relationships, and 10) leisure/recreation. A validity scale evaluates the respondent's truthfulness. Psychometric studies have documented the DUSI-R's validity in adolescents (2-8). Because the DUSI-R was specifically designed for use in practical settings, the scores indexing severity across the ten domains are ranked on a common metric (0–100%) so that intervention resources can be tailored to the magnitude each problem contributes to the person's maladjustment as quantified by the overall problem density score. Moreover, charting the scores on the DUSI-R scales during the course of repeated evaluations provides the opportunity to objectively quantify changes and trends occurring during prevention, treatment, and aftercare (9).

Research directed at elucidating etiology has also recently shown that the DUSI-R's items can be aggregated into six traits having strong heritability (10, 11). Moreover, the traits cohere to comprise a developmental trajectory leading to the diagnosis of substance use disorder (SUD) (12). Specifically, hyperactivity leads to externalizing (conduct problems) and internalizing (neurotocism) disturbances that predict low self-esteem which, in turn, predicts school problems and social withdrawal leading to SUD by young adulthood.

Notably, the DUSI-R's items have also been aggregated into a construct that can assist prediction of violence between early and late adolescence (13). Extending this line of research, the present investigation determined whether the DUSI-R's items can be aggregated into scales which correspond to the DSM-IV psychiatric disorders that frequently precede and co-occur with SUD. In view of the substantial skill, expense, and effort required to conduct a comprehensive diagnostic psychiatric evaluation, this study thus evaluated whether the DUSI-R can accurately identify youths who currently qualify for psychiatric disorder as well as youths who subsequently develop a disorder in the future. Accordingly, concurrent and predictive validity of newly derived DUSI-R scales measuring attention deficit hyperactivity disorder (ADHD), conduct disorder (CD), antisocial personality disorder (ASPD), anxiety disorder, depression disorder, and substance use disorder (SUD) were examined.

METHODS

Subjects

The sample consisted of 328 boys who have been enrolled in a longitudinal investigation since ages 10–12 directed at elucidating the etiology of SUD consequent to consumption of illegal drugs. This report is confined to boys because recruitment of girls began several years later; accordingly, the sample of females is too small at this time to conduct the predictive validity analyses. The boys were recruited through their fathers (probands) who qualified for either lifetime SUD consequent to use of an illegal drug, or had a non-SUD psychiatric disorder, or had no adult onset psychiatric disorder. The DUSI-R was administered for the first time when the boys were 12–14 years of age with repeat administrations conducted when they were 16, 19, and 22 years of age. To be admitted into the study, the boys were required to be in good physical health, have no history of psychosis, have a full scale WISC-III-R IQ greater than 80, and have no history of neurological injury requiring hospitalization.

Table 1 summarizes the characteristics of the sample. As can be seen, attrited subjects had lower family socioeconomic status. In addition, a higher proportion of minority youths did not participate in the age 22 follow-up evaluation. Full scale IQ was also lower in subjects who attrited during the course of the study, although their intelligence level was in the normal range. Grade level at the time of the baseline evaluation was not different between attrited and retained subjects. Most importantly, the overall problem severity score on the DUSI-R, the instrument from which the scales were derived, was not different between retained and attrited subjects.

Table 1.

Comparison of retained and attrited subjects at baseline

	Retained (n = 215)		Attrited (age 22) (n = 113)
	M	(s)	M	(s)	F	p
Full Scale IQ	111.56	15.98	104.82	16.58	12.38	<.001
Family Socioeconomic Status	41.44	14.01	38.21	13.41	3.90	.05
Grade in School	4.61	1.14	4.52	1.00	.49	.48
DUSI-R Overall Problem Density Score	17.15	10.99	18.72	13.96	1.22	.27
Ethnicity	n	%	n	%	Chi-square	p
European-American	170	79.1	78	69.0	4.05	.04
African-American	45	20.9	35	31

Instrumentation

The adolescent version of the Drug Use Screening Inventory—revised (DUSI-R) (1)—was administered using the paper and pencil version when the boys were 12–14 and 16 years of age. The adult version, consisting of the same ten measurement domains, was administered when the subjects were 19 and 22 years of age.

Development of the scales to screen for psychiatric disorder began by selecting items corresponding to the characteristics associated with attention deficit hyperactivity disorder (ADHD), conduct disorder (CD) to age 16, antisocial personality disorder (ASPD) from ages 17–22, depression, anxiety, and substance use disorder (SUD; abuse or dependence). This task was conducted by one of the authors (RT) and reviewed by the senior author prior to initiating the analyses. Next, the items provisionally assigned to each disorder were submitted to exploratory factor analysis (EFA) at each age. Items having a loading of .3 or higher were retained and submitted to confirmatory factor analysis (CFA) to document unidimensionality. Table 2 presents the five scales, the factor loading of each item, coefficient alpha, and the EFA and CFA results. As can be seen, each scale consists of a major factor accounting for at least 30% of variance and has good to superior internal consistency.

Table 2.

DUSI-R scales derived to screen psychiatric disorders

	Exploratory factor analyses		Confirmatory factor analyses
DUSI-R scale	Item number	Loading
Attention Deficit Hyperactivity Disorder Variance explained by 1st factor = 44% alpha = .73	25	.496	χ2 = 28.29, df = 25, p = .29 RMSEA = .019 CFI = .99, TLI = .99
	28	.387
	52	.836
	53	.820
	55	.806
	101	.775
	102	.451
	107	.521
	111	.656
	125	.413
Conduct Disorder/ASPD Variance explained by1st factor = 37% alpha = .73	9	.530	χ2 = 34.96, df = 29, p = .21 RMSEA = .024 CFI = .98, TLI = .98
	25	.593
	27	.734
	30	.595
	31	.676
	33	.664
	49	.657
	50	.773
	51	.409
	103	.695
	106	.565
	119	.606
	121	.398
	122	.478
	124	.580
	126	.466
	129	.471
	142	.708
	148	.494
Depression Variance explained by 1st factor = 39% alpha = .79	26	.487	χ2 = 66.73, df = 55, p = .13 RMSEA = .024 CFI = .99, TLI = .99
	34	.818
	35	.788
	36	.604
	40	.445
	42	.489
	54	.671
	55	.615
	56	.751
	67	.593
	71	.445
	74	.719
	82	.440
	143	.406
	144	.695
	145	.727
	154	.772
	156	.640
Anxiety Spectrum Variance explained by 1st factor = 41% alpha = .73	29	.559	χ2 = 58.20, df = 49, p = .17 RMSEA = .023 CFI = .98, TLI = .99
	46	.316
	55	.656
	57	.311
	58	.618
	59	.603
	60	.725
	61	.714
	62	.769
	66	.845
	72	.541
	78	.335
	98	.739
	110	.662
	113	.481
Substance Use Disorder Variance explained by 1st factor = 62% alpha = .83	1	.790	χ2 = 7.73, df = 7, p = .36 RMSEA = .017 CFI = .99, TLI=.99
	2	.905
	3	.476
	4	.926
	5	.912
	6	.883
	7	.827
	8	.926
	9	.907
	10	.734
	11	.900
	12	.879
	13	.740
	14	.950
	15	.771
	116	.862
	117	.924
	118	.917
	127	.673
	128	.868
	130	.905
	155	.845

The Schedule for Affective Disorders and Schizophrenia for children (K-SADS) (14) was administered when the participants were 12–14 and 16 years of age and the Structured Clinical Interview for Diagnosis (SCID) (15) was administered when the subjects were 19 and 22 years of age. Diagnoses were formulated employing DSM-IV criteria in a clinical conference using the best estimate procedure (16). Specifically, the subject's medical, legal, and psychiatric history, in conjunction with the K-SADS or SCID results, were reviewed and discussed by the committee members to formulate current (past 6 months) diagnoses. This conference was chaired by a psychiatrist certified in addiction psychiatry, another psychiatrist or psychologist, and the clinical associates who conducted the diagnostic interviews. Table 3 presents the frequency of first time psychiatric diagnoses at ages 12–14, 16, 19, and 22. As can be seen, the rates of psychiatric disorder approximate the rates observed in epidemiological studies with the exception of SUD owing to the fact that the sample was accrued on the basis of elevated risk for this disorder.

Table 3.

Percent of sample qualifying for psychiatric disorder and odds ratios of DUSI-R scales for detecting the disorder

DUSI-R assessment age	N	ADHD	CD	ASPD	Depression	Anxiety	SUD
12–14	328	6.7%	6.4%	N/A	1.5%	4.0%	1.2%
		3.31 p <.001	3.92 p < .001		2.51	2.14	2.36
					p = .067	p = .002	p = .005
16	298	3.4%	11.4%	N/A	3.0%	2.0%	7.4%
		2.70 p = .006	4.27 p < .001		3.74	2.12	3.84
					p < .001	p = .033	p < .001
19	257	4.7%	N/A	4.7%	8.2%	5.1%	23.7%
		2.48 p = .002		3.14 p < .001	3.27	2.21	5.55
					p < .001	p < .001	p < .001
22	215	1.9%	N/A	4.7%	9.3%	9.3%	39.5%
		3.20 p = .013		3.38 p < .001	2.03	2.48	5.75
					p = .08	p < .001	p < .001

Procedure

Written assent was obtained from the participants when they were 12–14 and 16 years of age, and written informed consent was obtained from the parents using forms and procedures approved by the University of Pittsburgh Institutional Review Board. At ages 19 and 22, the subjects provided written informed consent. Prior to initiating the evaluation, the subjects underwent a breath alcohol test and urine drug screen. Subjects testing positive were rescheduled to ensure that the results were not confounded by biased reporting caused by the acute effects of drugs. At the conclusion of the session, the subjects were compensated for their time and to offset travel and parking expenses.

Statistical Analysis

Logistic regression was employed to determine whether the DUSI-R scales predicted the corresponding psychiatric disorder. Next, receiver operating curve (ROC) analysis was performed to assess accuracy of each DUSI-R psychiatric scale to identify the individuals who currently qualify for the corresponding psychiatric disorder followed by determination of cut-off scores calibrated to 80% sensitivity at ages 12–14, 16, 19, and 22. This same procedure was used to determine accuracy of the DUSI-R psychiatric scales at age 12–14 for predicting subsequent psychiatric disorder.

RESULTS

Concurrent Validity

Table 3 depicts the results of the logistic regression analyses conducted on each scale. As can be seen, the DUSI-R scales measuring ADHD, CD/ASPD, anxiety, and SUD accurately predicted these diagnoses at each age. The DUSI-R depression scale was less consistent, approaching significance at ages 12–14 (p = .67) and 22 (p = .08) while attaining significance at ages 16 and 19.

The results of the ROC analyses, shown in Table 4, demonstrate good to outstanding accuracy of the DUSI-R psychiatric scales for detecting individuals who qualify for psychiatric disorder. Classification accuracy ranged from 72%–97%. Thus, used for screening purposes, the DUSI-R scales have practical utility by determining whether a formal psychiatric diagnostic evaluation is warranted. Notably, as shown in Table 4, true positives (sensitivity) are identified at a high level of accuracy; thus the scales “flag” individuals who require comprehensive evaluation.

Table 4.

Accuracy of DUSI-R scales for diagnosing psychiatric disorders

		Classification accuracy
Disorder	Age at time of assessment	Overall	Sensitivity	Specificity	Cut-off score1
ADHD	12–14	78%	80%	66%	3
	16	77	67	67	4
	19	73	75	50	3
	22	86	75	51	4
CD	12–14	87	83	73	3
	16	89	85	77	4
ASPD	19	84	83	71	3
	22	91	89	78	3
Anxiety	12–14	80	83	68	5
	16	73	83	70	6
	19	72	69	63	4
	22	76	79	60	3
Depression	12–14	81	67	80	4
	16	88	89	80	4
	19	84	76	80	2
	22	72	68	63	2
SUD	12–14	97	100	96	3
	16	87	82	90	3
	19	85	88	71	4
	22	83	85	61	3

¹Cut-off score is based on 80% sensitivity (true positive)

Predictive Validity

As can be seen in Table 5, the DUSI-R CD/ASPD scale at age 12–14 predicted a new diagnosis of CD two years later with 83% accuracy and ASPD at age 19 and 22 with, respectively, 89% and 73% accuracy. Analyses were not conducted for ADHD because this neurodevelopmental disorder does not manifest for the first time in adulthood. The DUSI-R anxiety scale at age 16 predicted this disorder at age 19 and 22 with 70% and 62% accuracy. The depression scale of these ages predicted this disorder with 79% and 75% accuracy. The SUD scale at age 16 predicted this outcome with 71% and 73% accurate at ages 19 and 22. Sensitively (true positives) ranged from good to excellent.

Table 5.

Accuracy of DUSI-R scales for predicting psychiatric diagnosis

DUSI-R cale prediction	Outcome age	OR	p	Sensitivity	Specificity	Overall area under the curve	Cut-off score on DUSI-R scale
				Conduct Disorder
CD/ASPD (age 12–14)	16	2.79	< .001	.74	.73	.83	3
				Antisocial Personality Disorder
CD/ASPD (age 12–14)	19	2.55	< .001	.90	.80	.89	3
	22	1.74	.029	.80	.66	.73	3
CD/ASPD (age 16)	19	2.41	< .001	.91	.65	.84	3
	22	2.50	< .001	.75	.84	.81	5
				Anxiety Disorder
Anxiety (age 16)	19	2.03	< .001	.72	.56	.70	3
	22	1.73	.006	.67	.59	.62	3
				Depression Disorder
Depression (age 16)	19	2.89	< .001	.79	.59	.79	4
	22	2.82	< .001	.78	.48	.75	3
				Substance Use Disorder
SUD (age 16)	19	1.96	< .001	.72	.60	.71	4
	22	2.60	< .001	.75	.53	.73	5

DISCUSSION

The results of this study demonstrated that the DUSI-R's items can be aggregated to form psychiatric screening scales that have concurrent and predictive validity. The scales gauge the presence of attention deficit hyperactivity disorder, conduct disorder, depression, anxiety, antisocial personality disorder, and substance use disorder. The items comprising these scales are dispersed throughout the DUSI-R's item set; however, scoring time is eliminated by using the Web-administered format (http://www.ecenterresearch.com).

The DUSI-R psychiatric scales are not a substitute for formal diagnostic evaluation. Rather, as a method to manage costs, and efficiently utilize professional resources, the scales are informative for estimating current and future psychiatric disorder which, in conjunction with results obtained in the 10-standard measurement domains, yields information pertaining to the need for comprehensive diagnostic evaluation prior to implementing intervention.

Several limitations of this study are noted. Importantly, the sample was confined to males and thus the cut-off scores shown in Tables 4 and 5 cannot be assumed to apply to females. Gender differences have been reported on the DUSI-R (3-7) as well as in many studies directed at elucidating the etiology and correlates of substance abuse. In addition, the sample size was relatively small (N = 328). Also, the higher attrition rate in African American, and among low SES subjects, may have compromised the predictive validity analyses. Lastly, it should be noted that a random sampling strategy was not employed. Hence, the potential for bias cannot be entirely ruled out. These caveats notwithstanding, the findings reported herein indicate that the DUSI-R has utility for implicating current psychiatric diagnosis as well as for estimation of risk for diagnosis up to 8–10 years later. However, prediction accuracy on several scales is modest, underscoring the difficulty of estimating long-term outcomes and the need to view these scales as indicators of risk and not as measures of inevitable progression to psychiatric disorder.

Whereas the adolescent version of the DUSI-R was shown in this study to yield scales that are useful for screening current psychiatric disorder and future disorder, research remains to be conducted pertaining to the utility of this instrument for this purpose in adults. Notably, the scales of the adult version of the DUSI-R map to the adolescent version, and findings to date indicate that it has sound psychometric properties (17). Hence, it is potentially useful for screening psychiatric disorder as a cost-efficient first step before conducting a comprehensive diagnostic evaluation.

In summary, five scales were derived from the DUSI-R which have superior concurrent validity for detecting youths who qualify for psychiatric diagnoses that commonly precede and co-occur with SUD. The CD/SPD and SUD scale scores at age 12–14 predicted these outcomes, respectively at age 19 and 22 with sensitivity ranging from 80%–94%. Thus, in addition to detecting health, behavior, and social problems, the DUSI-R has practical utility for determining whether a comprehensive diagnostic psychiatric examination is warranted prior to implementing prevention or treatment.