Accuracy of Self-Report, Biological Tests, Collateral Reports and Clinician Ratings in Identifying Substance Use Disorders among Adults with Schizophrenia

Abstract

Identifying substance use disorders among adults with schizophrenia presents unique challenges, but is critical to research and practice. This study examined: a) the accuracy of assessments completed using various approaches in identifying substance use disorders; b) their ability to discriminate between disorders of abuse and dependence; and c) the benefits of using multiple indicators to identify substance use disorders. Data are from the Clinical Antipsychotic Trials of Intervention Effectiveness study. The sample comprised 1,460 community-based adults with schizophrenia, 15.8% (n = 230) of whom were positive for a current (past month) drug or alcohol use disorder using the Structured Clinical Interview for DSM-IV Disorders (SCID). Clinician ratings, self-report, collateral reports, and results of hair and urine tests were compared to SCID diagnoses. Congruence with SCID diagnoses was good across approaches and evidence for superiority of one approach over another was limited. No approach discriminated between abuse and dependence. There was limited benefit of using multiple indicators. Findings suggest that the decision regarding the ‘best’ approach for identifying substance use disorders among adults with schizophrenia may be made through consideration of practical issues and assessment purpose, rather than selection of the approach that yields the most accurate diagnostic assessment.

Substance use disorders are common among adults with severe mental illness (SMI), such as schizophrenia, bipolar disorder, and major depressive disorder: almost half evince a lifetime substance use disorder--a rate three times higher than found in the general population (Regier et al., 1990). Rates for current substance use disorders also are high (Mueser, Drake, & Wallace, 1998). Though research converges on the finding that adults with SMI are at increased risk, estimates of disordered use range widely, from 10–70% depending on assessment method and diagnosis (i.e., substance abuse and/or substance dependence) (Blanchard, Brown, Horan, & Sherwood, 2000; Cantor-Graae, Nordstrom, & McNeil, 2001; Dixon, 1999; Goswami, Mattoo, Basu, & Singh, 2004; Kavanagh, McGrath, Saunders, Dore, & Clark, 2002; Kessler, et al., 1997; McCreadie, 2002; Mueser, et al., 1998; Regier, et al., 1990; Salyers & Mueser, 2001). Substance use disorders are associated with serious adverse outcomes among adults with SMI (Caton et al., 1994; Chouljian et al., 1995; Cournos et al., 1991; Dixon, 1999; Drake, Osher, & Wallach, 1991; Gerding, Labbate, Measom, Santos, & Arana, 1999; Lamb & Lamb, 1990; Linszen, Dingemans, & Lenior, 1994; Swanson et al., 2006; Swartz et al., 1998a; Swartz et al., 1998b; Swofford, Kasckow, Scheller-Gilkey, & Inderbitzin, 1996; Van Dorn, Volavka, & Johnson, 2012). Thus, reliable and valid approaches are needed for identifying disordered use, over and above use without impairment (Carey & Correia, 1998; Drake et al., 1990), and for distinguishing between the disorders of substance abuse and substance dependence, distinctions that may be critical to research and practice (Drake, Rosenberg, & Mueser, 1996).

Approaches for Identifying Substance Use Disorders among Adults with Schizophrenia

Formal diagnosis requires use of a structured or semi-structured interview (Bennett, 2009; Samet, Waxman, Hatzenbuehler, & Hasin, 2007), such as the Structured Clinical Interview for DSM-IV Disorders (SCID) (First, Spitzer, Gibbon, & Williams, 1996). Widely recognized as a gold standard approach for diagnosing mental and substance use disorders, SCID assessments have shown good reliability for identifying substance use disorders among adults with SMI (Albanese, Bartel, Bruno, Morgenbesser, & Schatzberg, 1994; Bryant, Rounsaville, Spitzer, & Williams, 1992); however, they are not always feasible. Specifically, SCID assessments can be time consuming, expensive and require considerable training (Bennett, 2009; Blaine, Forman, & Svikis, 2007). For these reasons, several alternative assessment methods for detecting substance use disorders among adults with SMI have been developed.

The Alcohol and Drug Use Scales (AUS/DUS) (Drake et al., 1990), for example, are widely used in clinical settings (Ries et al., 2002; Swartz, Perkins et al., 2003) and may be a more practical diagnostic approach than completing SCID assessments (Carey & Correia, 1998; Drake, Alterman & Rosenberg, 1993; Drake et al., 1998; Ries, 1994). However, few studies have examined the reliability and validity of AUS/DUS ratings. Those that have are limited by small samples (N < 200) and reporting validity only for AUS ratings (though reliability has been reported for both) (Drake et al., 1990; Drake, Osher, & Wallach, 1989; Drake & Wallach, 1989). Other research provides evidence of concurrent validity, but only with respect to alcohol abuse and not alcohol dependence (Carey, Cocco, & Simons, 1996). In another study, AUS/DUS and SCID results were combined to create criterion measures against which other instruments were compared (Wolford et al., 1999). Doing so revealed discrepancies between AUS ratings and SCID diagnoses: only 39% of participants identified as having an alcohol use disorder were identified by both AUS ratings and SCID assessments. No data were provided on agreement between DUS ratings and SCID assessments. Thus, despite the widespread application of this approach in clinical practice (Ries et al., 2002), empirical support for reliability and validity of AUS/DUS ratings for identifying substance use disorders among adults with SMI, though promising, is limited.

Other common assessment approaches include self-report, collateral reports, and biological tests. Though failure to disclose use is a concern (Carey, 2002), adults with SMI can provide accurate and reliable self-reports (Stasiewicz, et al., 2008; Van Dorn, Desmarais, Young, Sellers, & Swartz, in press). In one study, self-report was more accurate for identifying substance use disorders among adults with SMI than assessments completed using other approaches (Wolford, et al., 1999). Reports from collateral informants can augment self-report accuracy (Stasiewicz et al., 2008), but may not be available due to family estrangement and social isolation common among this population (Carey & Correia, 1998). Finally, biological tests can be effective, but also are expensive and invasive. Additionally, recent research questions the reliability and validity of their results (Cherwinski, Petti & Jekelis, 2007; Hendrickson & Morocco, 2003; Lancelin, Kraoul, Flatsichler, Brovedani-Rousset & Piketty, 2005; Moeller, Lee, & Kissack, 2008; Santos et al., 2007; Sena, Kazimi & Wu, 2002; Van Dorn et al., in press; Widschwendter, Zernig & Hofer, 2007; Wolford et al., 1999).

Challenges of Identifying Substance Use Disorders among Adults with Schizophrenia

Identifying substance use disorders among adults with schizophrenia presents many challenges, and, unfortunately, they frequently go undetected (Ananth, Vanderwater, Kamal, Brodsky, Gamal & Miller, 1989; Carey & Correia, 1998; Drake & Mueser, 2000; Shaner et al., 1993; Shaner et al., 1998; Stone, Greenstein, Gamble, & McLellan, 1993). Research demonstrates diagnostic uncertainty among clinicians assessing patients with chronic psychoses and co-occurring substance abuse or dependence (see Lehman, Myers, Dixon, & Johnson, 1996; Rosenthal, Hellerstein, & Miner, 1992; Shaner et al., 1993; Shaner et al., 1998). Sources of difficulty in identifying substance use disorders among adults with SMI include: insufficient periods of abstinence for clinicians to establish baseline functioning and behavior; poor memory and/or inconsistent self-reporting; potential interference of antipsychotic medications with the results of biological tests; and lack of corroborating information (Carey & Correia, 1998; Shaner et al., 1998; Moeller et al., 2008). Clinicians also report difficulties in differentiating between schizophrenia and chronic substance-induced psychoses due to similarity in symptoms (Carey & Correia, 1998; Gregg, Baarrowclough, & Haddock, 2007; Horsfall, Cleary, Hunt, & Walter, 2009; Lehman et al., 1996; Rosenthal et al., 1992; Shaner et al., 1993; Shaner et al., 1998).

In addition to these clinical challenges, measures of substance use may over-identify disordered use among adults with schizophrenia. This is not problematic if individuals with schizophrenia are indeed “supersensitive” to the negative consequences associated with substance use (Mueser et al., 1998) and at increased risk of disordered use (Drake & Wallach, 1993). Though these arguments frequently show up in the literature, we are aware of only one evaluation of whether adults with SMI are supersensitive to the negative consequences of substance use (Gonzales, Bradizza, Vincent, Stasiewicz, & Paas, 2007). In that study, comparisons between non-SMI substance abusers, dually diagnosed substances abusers and SMI-only individuals (i.e., non-substance abusers) failed to show higher rates of negative consequences among substance abusers with SMI, but did identify higher rates of psychological symptoms.

In an attempt to overcome the challenges of identifying substance use disorders among adults with SMI, many researchers have adopted multimethod assessment approaches. For example, by employing three methods—namely, clinical records, research interviews using standard alcohol assessment instruments, and case managers’ ratings—Drake and colleagues (1990) reported they were better able to determine who had alcohol use disorders among a sample of 75 schizophrenic patients. Similarly, Swartz, Swanson and Hannon (2003) concluded that combining results of self-report, urine test, and hair assay improved detection of illicit drug use among 203 adults with schizophrenia over results of any single assessment approach. Importantly, neither study explicitly compared accuracy of combined results to those of individual approaches, though multimethod approaches have been shown to improve assessment accuracy for many other psychological constructs (e.g., Diener & Eid, 2006). Moreover, empirical support for including information from multiple sources to identify substance abuse or substance dependence, rather than substance use without impairment, is lacking (Wolford et al., 1999). Nonetheless, experts recommend combining self-report with both structured and semi-structured interviews to enhance diagnostic processes, as well as utilizing longitudinal behavioral observations, collateral information from other informants, and biochemical tests from blood, breath or urine samples, if available (Carey & Correia, 1998; Drake et al., 1990; Swartz et al., 2006).

The Present Study

In sum, the identification of substance use disorders among adults with schizophrenia can be challenging, yet critical to research and practice. Various assessment approaches are available to assist in this process, each with their own strengths and weaknesses, and there is no consensus regarding the ‘best’ diagnostic assessment approach. Furthermore, there is scant information regarding the ability of various assessment approaches to discriminate between the disorders of substance abuse and substance dependence. Evidence regarding gains in accuracy attributable to multimethod approaches is similarly limited. Thus, in a large sample of community-based adults with schizophrenia, we examined:

1. The relative accuracy of AUS/DUS ratings, self-report, collateral ratings, and biological tests in identifying substance use disorders compared to SCID diagnoses;

2. The ability of each approach to discriminate between abuse and dependence; and

3. The benefits of using multiple indicators to identify substance use disorders.

Method

Study Design

Data are from the National Institute of Mental Health Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study, a double-blind, randomized clinical trial conducted between January 2001 and December 2004 at 57 sites (16 university clinics, 10 state mental health agencies, seven Veteran’s Affairs Medical Centers, six private nonprofit agencies, four private practice sites, 14 mixed system sites). Data from one site (n = 33) were excluded due to concerns regarding data integrity. The CATIE protocol was approved by local IRBs, and participants gave written informed consent prior to enrollment. The CATIE design and enrollment have been described previously (Stroup et al., 2003). Briefly, men and women aged 18 to 65 years of age, who met DSM-IV diagnostic criteria for schizophrenia were recruited from the participating sites. Potential participants comprised new or existing patients with chronic or recurrent schizophrenia. First episode and treatment-refractory patients were excluded. There were few other exclusion criteria, and only 7% of screened patients were excluded from the study. (For further details regarding recruitment, as well as inclusion and exclusion criteria, see Stroup et al., 2003).

Participants

We analyzed data from baseline assessments (i.e., before randomization and initiation of experimental treatments) of 1,460 adults with schizophrenia enrolled in the CATIE study. Most were male (73.9%; n = 1,079), white (60%; n = 874), had completed high school (74.3%; n = 1,085), and were not married nor cohabitating with a partner (81.0%, n = 1,181). Average age was 40.56 years (SD = 11.10). Detailed descriptions of the sample characteristics are available elsewhere (Keefe et al., 2006; Swartz et al., 2006). Prior research demonstrates that the sample resembled a usual-care, quasi-random, observational, noninterventional sample in its demographic and clinical characteristics (Swartz et al., 2006).

Measures and Procedures

Criterion

SCID (First et al., 1996) diagnoses served as criterion measures for current (past month) substance use disorders. Assessments were completed by experienced, SCID-certified clinicians based upon all clinical information available at the beginning of the baseline assessments. Although it was not possible to calculate inter-rater reliability of SCID diagnoses completed by clinicians across the 57 study sites, the CATIE study utilized a clinical rater training and certification process that has been shown to minimize assessment error and increase reliability and validity in large multisite trials (Tracy et al., 1997; Müller & Wetzel, 1998; Salyers et al., 2001; Warshaw, Dyck, Allsworth, Stout, & Keller, 2001). In particular, clinicians were required to complete initial certification training on the SCID, as well as yearly recertification (Swartz, Perkins et al., 2003). To be certified on the SCID, clinicians were required to achieve a correct completion rate of 80% on 10 vignettes. Due to the multisite nature of the study, an initial in-person training event was conducted at the coordinating center prior to study implementation, which was digitally captured and placed on a secure website. The online training website comprised self-guided units of video and audio from the initial training event, edited transcripts and support materials. Those sites that were not able to access the online training website received a compact disc replicating the website content. Prior studies using similar training approaches have found good to excellent inter-rater agreement (e.g., κ = .65 for alcohol use/dependence and .77 for drug abuse/dependence, Lobbestael, Leurgans, & Arntz, 2011; κ = .94 for alcohol use disorders and .82–1.00 for other substance use disorders, Martin, Pollock, Bukstein, & Lynch, 2000; κs = 1.00 for alcohol use/dependence and drug abuse/dependence, Zanarini et al., 2000).

AUS/DUS

AUS/DUS (Drake et al., 1990) ratings completed by an MD or other experienced clinician were the primary method of assessing alcohol and drug use in the CATIE study (Swartz, Perkins et al., 2003). Clinicians were instructed to provide separate ratings (1 = abstinent, 2 = use without impairment, 3 = abuse, 4 = dependence, and 5 = dependence with institutionalization) regarding their client’s use of alcohol and drugs over the last three months by weighting evidence from self-report, interviews, behavioral observations, and collateral reports (e.g., family, day center, community, etc.). Results of SCID assessments and biological tests were not available to these clinicians. Training on the AUS/DUS followed the procedures described for the SCID assessments, but certification was not required. Inter-rater reliability data are not available; however, prior studies using similar training protocols have found good inter-rater agreement (e.g., κ = .80 for alcohol use and .95 for drug use, Drake et al., 1989; κ = .85 for current alcohol use disorder, Drake et al., 1990). For analyses, ratings of 2 indicated use, 3 indicated abuse, 4 or 5 indicated dependence, and 3 or greater indicated abuse or dependence.

Self-report

Participants self-reported alcohol and drug use over the previous three months during research interviews. Specifically, participants were asked to respond “yes” (1) or “no” (0) regarding whether they had used alcohol, marijuana, cocaine, opiates, PCP, amphetamines or other drugs in the past three months. For analyses, a positive response to the alcohol question indicated alcohol use; positive response to at least one of the drug questions indicated drug use; and a positive response to at least one question indicated substance use.

Collateral informant ratings

Family member/caregiver interviews were conducted for 645 participants. Ratings (0 = never, 1 = rarely, 2 = occasionally, 3 = often) of participants’ problems with excessive use of drugs and alcohol in the prior month were provided. To calculate prevalence, 1 indicated use, 2 indicated abuse, 3 indicated dependence, and 2 or 3 indicated abuse or dependence (Swartz et al., 2006).

Biological tests

Hair specimens were collected and analyzed by radioimmunoassay (RIA), which assays drugs and their metabolites transferred from capillary circulation through the hair follicle to the internal hair structure (Baumgartner, Hill, & Bland, 1989). A tuft of hair about the diameter of pencil lead was cut from the back of the scalp. A larger volume was removed from participants with short hair. If none was present on the head, hair was taken from the chest, arm, or leg. Samples 1.5 inches long were taken, affording assessment of drug use in the preceding three months. All hair samples were collected by study sites and shipped to PsycheMedics Corporation for analysis. Values more than three standard deviations from the mean of a comparison sample of drug-free individuals were considered positive once confirmed using gas chromatography/mass spectrometry.

Drug urinalysis was performed with a commercially available rapid multiple immunoassay urine drug test (Bayer Multistix Microscopic Manual) and analyzed by Quintiles Laboratories. Participants who tested positive for a prescribed medication through either RIA of hair or drug urinalysis were not considered to be using.

Statistical Analyses

Prevalence

We report prevalence of substance use disorders by assessment method and measures of central tendency for AUS/DUS and collateral ratings. We calculated detection ratios for results of the various assessment approaches compared to the SCID diagnoses (i.e., the ratio of assessment prevalence over SCID prevalence). Values less than 1.00 indicate under-identification of substance use disorders compared to SCID diagnoses. Conversely, values greater than 1.00 indicate over-identification. McNemar tests, chi-square tests for correlated proportions, were used to determine whether the prevalence ratio (i.e., disorder present/absent) for various assessments approaches and the SCID results differed significantly from 1.00 (Conover, 1999).

Accuracy and discrimination

We calculated sensitivity, specificity, positive predictive values, negative predictive values, percent classified correctly, and the Areas Under the Curve (AUCs) of Receiver Operating Characteristics curves to examine accuracy with SCID diagnoses as the criteria. These statistical methods provide different measures of diagnostic accuracy calculated as a function of the number of hits (true positives) and misses (false negatives), as well as correct rejections (true negatives) and false alarms (false positives). AUCs between .70–.90 indicate good accuracy (Swets, 1988). We calculated z-scores to identify statistically significant differences in accuracy (Hanley & McNeil, 1983; McNeil & Hanley, 1984). Bonferonni corrections were made based on 10 comparisons for drug use disorders (α = .005) and three comparisons each for alcohol use disorders and substance use disorders (α = .017). Cohen’s kappas, an index of agreement for categorical data that takes into account agreement occurring by chance, were computed to evaluate agreement with SCID diagnoses. Values between .00–.20 indicate slight, .21–.40 fair, and .41–.60 moderate agreement (Landis & Koch, 1977).

Incremental validity

We conducted direct entry hierarchical logistic regression analyses, controlling for study site, to evaluate incremental validity. Significant chi-square change values reflected model improvements and significant odds ratios indicated contributions of individual predictors.

Results

Prevalence

Overall, 15.8% (n = 230) of participants received one or more SCID diagnoses of current substance use disorder; 10.5% (n = 153) were identified as abusing and 8.2% (n = 120) as dependent. Of those, 7.8% (n = 113) received SCID diagnoses of drug abuse, 5.5% (n = 80) drug dependence, and 11.6% (n = 169) a drug use disorder (abuse or dependence). Disordered alcohol use was less prevalent: 4.6% (n = 67) received SCID diagnoses of alcohol abuse, 4.2% (n = 61) alcohol dependence, and 7.6% (n = 111) alcohol use disorder (abuse or dependence).

AUS/DUS ratings ranged from 1 to 5 (M = 1.36, SD = 0.72, and M = 1.45, SD = 0.71, respectively), but 5s were rare (n = 1 for drugs, 2 for alcohol). These ratings indicated prevalence of 7.3% for drug abuse, 2.4% for drug dependence, and 9.6% for drug use disorder, and 5.1% for alcohol abuse, 2.5% for alcohol dependence, and 7.5% for alcohol use disorder. Collateral ratings also reflected the full range, M = 0.30 (SD = 0.77) for drugs and M = 0.40 (SD = 0.08) for alcohol. Prevalence according to collateral ratings was similar to rates associated with AUS/DUS ratings for drugs (abuse = 7.9%; dependence = 3.8%; abuse or dependence = 11.7%), but not alcohol (abuse = 10.1%; dependence = 4.4%; abuse or dependence = 14.5%). Rates of self-reported use were: 22.2% (n = 322) drugs, 34.6% (n = 501) alcohol, and 40.3% (n = 584) substance use (alcohol or drugs). Urine and hair tests for drug use were positive for 15.6% and 27.7% of participants, respectively.

McNemar tests showed variation in prevalence by assessment method. Compared to SCID diagnoses, AUS/DUS ratings under-identified drug dependence (detection ratio = 0.44), alcohol dependence (detection ratio = 0.60), and substance dependence (detection ratio = 0.48). DUS ratings also under-identified drug use disorder (detection ratio = 0.83, p = .01) and substance use disorder (detection ratio = 0.84, p = .03). Collateral ratings over-identified alcohol (detection ratio = 2.20) and substance (detection ratio = 1.35) abuse, and alcohol (detection ratio = 1.91) and substance (detection ratio = 1.18) use disorders. Measures of use (as opposed to disordered use), over-identified all SCID diagnoses: self-report detection ratios ≥ 2.55, and biological tests’ detection ratios ≥ 1.34. (All ps < .001 unless specified.)

Accuracy

Table 1 presents accuracy by assessment approach and cutoff compared to SCID diagnoses of drug use disorders. Assessment accuracy was good and comparable across methods. Considering all performance measures, DUS ratings demonstrated the greatest accuracy, followed by self-report, collateral ratings, and biological tests. All assessments were better at identifying SCID diagnoses of drug abuse and drug use disorder than drug dependence. AUC values for DUS ratings ≥ 2 and self-report were higher than those observed for biological tests for all diagnoses (zs ≥ 3.07, ps < .005) with one exception: self-report and hair tests were comparable in identifying drug dependence. Percent classified correctly ranged between 74.3% (hair identifying drug dependence) and 94.8% (DUS ratings ≥ 4 identifying drug dependence). Kappas were fair to moderate, though biological tests demonstrated only slight agreement with SCID diagnoses of drug dependence.

Table 1.

Accuracy compared to SCID diagnoses of drug use disorders

METHOD	SCID DIAGNOSIS
	Drug Abuse
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	False POS	False NEG	Kappa
DUS Ratings
≥ 2	.87 (.02)	.84–.90***	92.0%	81.6%	82.4%	69.7%	0.8%	0.38
≥ 3	.73 (.03)	.67–.79***	52.2%	94.0%	90.7%	56.9%	4.2%	0.42
≥ 4	.52 (.03)	.46–.58	6.2%	98.0%	90.6%	79.0%	7.7%	0.06
Collateral Ratings
≥ 1	.76 (.05)	.67–.85***	62.8%	89.3%	87.2%	66.1%	3.5%	0.36
≥ 2	.76 (.05)	.67–.85***	60.5%	92.1%	89.6%	59.9%	3.6%	0.42
3	.62 (.05)	.52–.72**	25.6%	97.9%	92.2%	48.0%	6.2%	0.30
Self-Report	.85 (.02)	.81–.89***	86.7%	83.3%	83.6%	68.9%	1.4%	0.38
Biological Tests
Urine	.75 (.03)	.69–.80***	60.7%	88.2%	86.0%	69.0%	3.7%	0.33
Hair	.75 (.03)	.69–.80***	73.0%	76.1%	75.8%	79.0%	3.0%	0.22
	Drug Dependence
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	False POS	False NEG	Kappa
DUS Ratings
≥ 2	.83 (.02)	.79–.88***	87.3%	79.4%	79.6%	78.7%	1.0%	0.25
≥ 3	.74 (.04)	.67–.81***	55.7%	93.0%	90.8%	66.4%	3.0%	0.36
≥ 4	.64 (.04)	.56–.71***	27.8%	99.1%	94.8%	33.0%	4.4%	0.37
Collateral Ratings
≥ 1	.76 (.06)	.65–.87***	64.0%	87.7%	86.3%	75.0%	2.6%	0.24
≥ 2	.73 (.06)	.61–.85***	56.0%	90.3%	88.2%	73.2%	3.0%	0.25
3	.61 (.07)	.48–.73	24.0%	97.1%	92.7%	65.3%	4.8%	0.22
Self-Report	.83 (.02)	.78–.88***	84.8%	81.4%	81.6%	77.5%	1.2%	0.27
Biological Tests
Urine	.68 (.04)	.61–.75***	49.4%	86.3%	84.1%	81.2%	3.6%	0.19
Hair	.71 (.03)	.65–.78***	68.2%	74.7%	74.3%	85.3%	2.6%	0.15
	Drug Use Disorder1
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	False POS	False NEG	Kappa
DUS Ratings
≥ 2	.87 (.02)	.84–.90***	89.3%	84.4%	84.9%	56.3%	1.7%	0.50
≥ 3	.75 (.03)	.70–.79***	53.0%	96.0%	90.9%	35.5%	6.3%	0.53
≥ 4	.57 (.03)	.52–.62**	14.3%	99.2%	89.0%	28.6%	10.5%	0.21
Collateral Ratings
≥ 1	.78 (.04)	.71–.86***	65.0%	91.3%	88.2%	49.4%	5.0%	0.48
≥ 2	.77 (.04)	.69–.85***	60.0%	93.8%	89.8%	43.0%	5.5%	0.51
3	.63 (.04)	.53–.70**	25.0%	98.7%	89.8%	28.3%	9.4%	0.33
Self-Report	.86 (.02)	.82–.89***	85.1%	86.0%	85.9%	54.6%	2.3%	0.51
Biological Tests
Urine	.73 (.02)	.68–.77***	55.4%	89.6%	85.5%	58.0%	6.4%	0.39
Hair	.75 (.02)	.70–.79***	71.5%	78.1%	77.3%	69.2%	4.7%	0.31

Notes. DUS: N = 1,448; Collateral: N = 579; Self-report: N = 1,448; Urine: N = 1,457; Hair: N = 1,182.

^*p < .05.

^**p < .01.

^***p < .001.

¹SCID diagnosis for at least one of drug abuse or dependence. %CC = percent classified correctly.

Table 2 presents accuracy by assessment method and cutoff compared to SCID diagnoses of alcohol use disorders. Again, accuracy was good and comparable across methods. Pairwise comparisons failed to show that assessments completed using one method outperformed those completed using any other method, all ps >.391. Approaches were mixed in their ability to identify specific SCID diagnoses. Considering results across performance measures, AUS ratings demonstrated the highest accuracy for dependence and the lowest accuracy (though still good) for abuse; collateral ratings showed better accuracy for alcohol abuse and alcohol use disorder compared to dependence; self-report showed slightly better detection of alcohol use disorder than abuse or dependence. Percent classified correctly ranged between 68.4% (AUS ratings ≥ 2 identifying dependence) and 96.6% (AUS ratings ≥ 4 identifying dependence). Kappas were poor to moderate.

Table 2.

Accuracy compared to SCID diagnoses of alcohol use disorders

METHOD	SCID DIAGNOSIS
	Alcohol Abuse
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	False POS	False NEG	Kappa
AUS Ratings
≥ 2	.79 (.02)	.74–.83***	89.6%	67.5%	68.6%	87.3%	0.8%	0.14
≥ 3	.74 (.04)	.67–.82***	53.7%	94.6%	92.6%	65.5%	2.5%	0.37
≥ 4	.53 (.04)	.46–.61	9.0%	97.8%	93.4%	82.2%	4.7%	0.09
Collateral Ratings
≥ 1	.82 (.05)	.72–.91***	81.8%	81.8%	81.8%	80.9%	1.2%	0.20
≥ 2	.76 (.06)	.64–.88***	63.6%	87.4%	86.2%	79.0%	2.1%	0.21
3	.62 (.07)	.48–.76	27.3%	96.5%	93.0%	71.1%	3.8%	0.22
Self-Report	.77 (.03)	.72–.82***	86.6%	67.9%	68.9%	87.6%	1.0%	0.13
	Alcohol Dependence
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	False POS	False NEG	Kappa
AUS Ratings
≥ 2	.80 (.02)	.76–.85***	93.3%	67.4%	68.4%	89.4%	0.4%	0.13
≥ 3	.81 (.04)	.74–.88***	66.7%	95.0%	93.8%	64.5%	1.4%	0.44
≥ 4	.69 (.04)	.60–.77***	38.3%	99.1%	96.6%	37.0%	2.5%	0.46
Collateral Ratings
≥ 1	.80 (.05)	.70–.90***	78.3%	81.7%	81.6%	84.8%	1.1%	0.20
≥ 2	.74 (.06)	.62–.86***	60.9%	87.4%	86.3%	83.2%	1.8%	0.21
3	.64 (.07)	.50–.77*	30.4%	96.6%	94.0%	72.7%	2.9%	0.25
Self-Report	.79 (.03)	.74–.84***	90.0%	67.8%	68.7%	89.6%	0.6%	0.13
	Alcohol Use Disorder1
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	False POS	False NEG	Kappa
AUS Ratings
≥ 2	.81 (.02)	.77–.84***	91.8%	69.5%	71.3%	79.2%	1.0%	0.23
≥ 3	.77 (.03)	.72–.83***	58.2%	96.6%	93.5%	40.5%	3.6%	0.55
≥ 4	.61 (.03)	.55–.67***	22.7%	99.2%	93.1%	29.4%	6.3%	0.32
Collateral Ratings
≥ 1	.82 (.04)	.75–.90***	80.5%	83.9%	83.6%	69.7%	2.0%	0.34
≥ 2	.75 (.05)	.66–.84***	61.0%	89.0%	86.8%	67.5%	3.7%	0.33
3	.62 (.05)	.52–.72**	26.8%	97.3%	91.6%	54.1%	6.1%	0.29
Self-Report	.80 (.02)	.76–.83***	89.1%	69.9%	71.4%	79.5%	1.3%	0.22

Notes. AUS: N = 1,448; Collateral: N = 587; Self-report: N = 1,448.

^*p < .05.

^**p < .01.

^***p < .001.

¹SCID diagnosis for at least one of alcohol abuse or dependence. %CC = percent classified correctly.

Table 3 presents accuracy by assessment method and cutoff compared to SCID diagnoses of substance use disorders. AUCs indicated good predictive accuracy. AUS/DUS ratings slightly outperformed self-report, which was slightly more accurate than collateral ratings. All methods were better at identifying SCID diagnoses of substance use disorder compared to abuse or dependence. Percent classified correctly ranged between 65.5% (AUS/DUS ratings ≥ 2 identifying dependence) and 93.6% (AUS/DUS ratings ≥ 4 identifying dependence). Kappas were small to moderate in size, and generally demonstrated better agreement for substance use disorder than for substance abuse or dependence.

Table 3.

Accuracy compared to SCID diagnoses of substance use disorders

METHOD	SCID DIAGNOSIS
	Substance Abuse2
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	PPV	NPV	Kappa
AUS/DUS Ratings1
≥ 2	.80 (.02)	.77–.83***	94.8%	65.1%	68.1%	76.8%	0.9%	0.26
≥ 3	.75 (.03)	.70–.80***	58.2%	91.9%	88.5%	55.6%	4.8%	0.45
≥ 4	.53 (.03)	.48–.58	9.8%	96.8%	88.1%	74.9%	9.4%	0.09
Collateral Ratings
≥ 1	.77 (.03)	.70–.83***	76.7%	80.7%	80.3%	69.4%	3.1%	0.35
≥ 2	.75 (.04)	.68–.83***	68.3%	86.3%	84.5%	64.3%	3.9%	0.39
3	.63 (.04)	.55–.71***	31.7%	95.9%	89.5%	53.9%	7.3%	0.32
Self-Report	.79 (.02)	.76–.82***	92.2%	65.8%	68.4%	77.0%	1.3%	0.26
	Substance Dependence3
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	PPV	NPV	Kappa
AUS/DUS Ratings1
≥ 2	.78 (.02)	.74–.81***	91.6%	63.3%	65.5%	82.2%	1.1%	0.19
≥ 3	.79 (.03)	.74–.84***	66.4%	91.3%	89.4%	60.0%	3.1%	0.45
≥ 4	.67 (.03)	.61–.73***	34.5%	98.8%	93.6%	28.7%	5.5%	0.44
Collateral Ratings
≥ 1	.74 (.04)	.65–.82***	70.0%	78.2%	77.5%	78.2%	3.2%	0.21
≥ 2	.70 (.05)	.61–.79***	57.5%	83.6%	81.5%	76.7%	4.2%	0.22
3	.63 (.05)	.53–.74**	32.5%	94.9%	90.0%	64.1%	5.8%	0.27
Self-Report	.77 (.02)	.73–.81***	89.9%	64.1%	66.2%	82.1%	1.3%	0.19
	Substance Use Disorder4
	AUC (SE)	95%CI	Sensitivity	Specificity	%CC	PPV	NPV	Kappa
AUS/DUS Ratings1
≥ 2	.81 (.01)	.79–.84***	93.4%	68.6%	72.6%	63.8%	1.8%	0.38
≥ 3	.77 (.02)	.73–.81***	59.4%	95.2%	89.5%	29.6%	7.5%	0.58
≥ 4	.59 (.02)	.55–.64***	19.2%	98.9%	86.2%	22.6%	13.5%	0.26
Collateral Ratings
≥ 1	.78 (.03)	.73–.83***	75.0%	83.6%	82.2%	53.4%	5.4%	0.46
≥ 2	.75 (.03)	.68–.81***	63.6%	88.5%	84.6%	48.6%	7.3%	0.47
3	.63 (.04)	.56–.70***	29.5%	97.0%	86.2%	34.5%	12.1%	0.34
Self-Report	.81 (.01)	.78–.83***	91.7%	69.3%	72.9%	63.7%	2.2%	0.37

Notes. AUS/DUS: N = 1,448; Collateral: N = 594; Self-report: N = 1,448.

^*p < .05.

^**p < .01.

^***p < .001.

¹Positive result on either the AUS or DUS.

²SCID diagnosis for at least one of alcohol or drug abuse.

³SCID diagnosis for at least one of alcohol or drug dependence.

⁴SCID diagnosis for at least one of alcohol or drug abuse or dependence.

Abuse and Dependence

There was limited evidence of discrimination between abuse and dependence (full results not presented but available upon request). Assessments of use--self-report and biological tests--performed comparably in detecting abuse and dependence, as did collateral ratings (see Tables 1–3). Analyses revealed a mismatch between AUS/DUS labels and SCID diagnoses. For instance, 3 = drug abuse on the DUS; yet, this rating demonstrated poorer accuracy for identifying drug abuse compared to combining ratings of 2 or 3, AUCs = .71 and .85, respectively, z = 3.11, p = .002. Similarly, the AUC for ratings ≥ 4 was lower for drug dependence than for ratings ≥ 3, z = 2.11, p = .035. That said, ratings of 3 were more accurate in identifying drug abuse than were ratings ≥ 4, z = 3.76, p < .001. Similar results were found for the ability of AUS ratings to discriminate between SCID diagnoses of alcohol abuse and dependence. Ratings ≥ 4 demonstrated poorer accuracy in identifying alcohol dependence than did ratings ≥ 3, z = 2.13, p = .033; however, AUS = 3 was more accurate than ratings ≥ 4 for identifying abuse, z = 3.89, p < .001. No differences in AUCs were found for AUS = 3 compared to combining ratings of 2 or 3 for alcohol abuse.

Multimethod Assessment Strategies

We conducted two sets of regression analyses to explore the benefits of using multiple indicators for identifying substance use disorders. We first examined whether biological tests and AUS/DUS ratings added to the capacity of self-report to predict drug, alcohol, and substance use disorders. For drug use disorders, self-report ratings were added in Step 1 of each of three models with SCID drug abuse, drug dependence, and drug use (abuse or dependence) disorder as criteria. All models were significant (see Table 4). Adding biological test results in Step 2 improved accuracy, but increased R² only minimally (from .01 for drug dependence to .05 for drug abuse). In Step 3, DUS ratings were added in one block. Model fit improved, but, again, R² increases were small (from .01 for drug abuse to .07 for drug dependence).

Table 4.

Logistic regression analyses testing incremental validity for SCID diagnoses of drug use disorders

	SCID DIAGNOSIS
	Drug Abuse (N = 1,172)				Drug Dependence (N = 1,173)				Drug Use Disorder1 (N = 1,173)
STEP 1	Model fit Χ2(2) = 188.29***, R2 = .36				Model fit Χ2(2) = 126.61***, R2 = .30				Model fit Χ2(2) = 290.97***, R2 = .43
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	3.55***	.34	34.93	18.04–67.64	3.22***	.37	25.12	12.17–51.85	3.59***	.27	36.35	21.60–61.18
STEP 2	Model fit Χ2(4) = 214.44***, R2 = .41				Model fit Χ2(4) = 130.70***, R2 = .31				Model fit Χ2(4) = 316.83***, R2 = .46
	Χ2change(2) = 26.15***				Χ2change (2) = 4.09***				Χ2change (2) = 25.86***
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	2.83***	.37	16.86	8.19–34.70	2.92***	.41	18.47	8.31–41.05	2.99***	.29	19.89	11.30–35.03
Biological Tests
Urine	1.01***	.29	2.73	1.56–4.79	0.07	.31	1.07	0.58–1.96	0.63*	.25	1.88	1.15–3.07
Hair	0.70*	.31	2.01	1.11–3.64	0.60	.33	1.82	0.95–3.48	0.63***	.26	2.37	1.44–3.90
STEP 3	Model fit Χ2(5) = 225.28***, R2 = .42				Model fit Χ2(5) = 163.82***, R2 = .38				Model fit Χ2(5) = 354.65***, R2 = .51
	Χ2change (1) = 10.83***				Χ2change (1) = 33.12***				Χ2change (1) = 37.82***
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	1.98***	.45	7.21	3.01–17.30	1.28**	.50	3.59	1.36–9.48	1.62***	.36	5.06	2.50–10.21
Biological Tests
Urine	1.13***	.29	3.10	1.74–5.50	0.28	.33	1.33	0.70–2.52	0.78**	.27	2.18	1.30–3.66
Hair	0.54	.31	1.72	0.94–3.17	0.31	.35	1.36	0.69–2.69	0.63*	.27	1.88	1.10–3.20
DUS Ratings	0.61***	.18	1.84	1.29–2.65	1.13***	.20	3.10	2.11–4.57	1.05***	.17	2.86	2.03–4.03

Notes.

^*p <.05.

^**p < .01.

^***p <.001. R² values are Nagelkerke.

¹SCID diagnosis for at least one of drug abuse or dependence.

We repeated Steps 1 and 3 for alcohol use disorders (see Table 5) and substance use disorders (see Table 6). All Step 1 models were significant. Adding clinicians’ ratings improved model fit for all alcohol use disorders and substance use disorders and generally contributed to considerable increases in R² (from .07 for alcohol abuse to .22 for alcohol dependence).

Table 5.

Logistic regression analyses testing incremental validity for identifying SCID diagnoses of alcohol use disorders

	SCID DIAGNOSIS
	Alcohol Abuse				Alcohol Dependence				Alcohol Use Disorder1
STEP 1	Model fit Χ2(2) = 85.33***, R2 = .18				Model fit Χ2(2) = 87.57***, R2 = .20				Model fit Χ2(2) = 161.39***, R2 = .25
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	2.58***	.36	13.22	6.49–26.95	2.91***	.44	18.35	7.83–42.99	2.91***	.31	18.41	9.99–33.93
STEP 2	Model fit Χ2(3) = 116.54***, R2 = .25				Model fit Χ2(3) = 186.85***, R2 = .42				Model fit Χ2(3) = 279.66***, R2 = .42
	Χ2change (1) = 31.21***				Χ2change (1) = 99.28***				Χ2change (1) = 118.28***
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	1.24**	.44	3.46	1.45–8.25	0.34	.54	1.41	0.49–4.05	0.65	.40	1.91	0.88–4.15
AUS Ratings	1.00***	.17	2.72	1.94–3.82	1.85***	.20	6.33	4.29–9.34	1.78***	.18	5.91	4.14–8.44

Notes. N = 1,447.

^*p < .05.

^**p < .01.

^***p < .001. R² values are Nagelkerke.

¹SCID diagnosis for at least one of alcohol abuse or dependence.

Table 6.

Logistic regression analyses testing incremental validity for identifying SCID diagnoses of substance use disorders

	SCID DIAGNOSIS
	Substance Abuse1				Substance Dependence2				Substance Use Disorder3
STEP 1	Model fit χ2(2) = 207.53***, R2 = .27				Model fit χ2(2) = 150.66***, R2 = .23				Model fit χ2(2) = 327.28***, R2 = .35
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	3.09***	.31	22.00	12.06–40.14	2.72***	.31	15.10	8.22–27.75	3.19***	.25	24.16	14.86–39.27
STEP 2	Model fit χ2(3) = 269.95***, R2 = .35				Model fit χ2(3) = 260.36***, R2 = .38				Model fit χ2(3) = 504.12***, R2 = .51
	χ2change (1) = 62.42***				χ2change (1) = 109.70***				χ2change (1) = 176.84***
	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI	β	S.E.	Odds Ratio	95% CI
Self-Report	1.87***	.35	6.49	3.26–12.94	0.78*	.38	2.19	1.03–4.64	1.12***	.31	3.07	1.68–5.61
AUS/DUS Ratings	0.61***	.08	1.84	1.57–2.15	0.90***	.10	2.46	2.04–2.97	1.13***	.10	3.09	2.53–3.77

Notes. N = 1,447.

^*p < .05.

^**p < .01.

^***p < .001. R² values are Nagelkerke.

¹SCID diagnosis for at least one of alcohol or drug abuse.

²SCID diagnosis for at least one of alcohol or drug dependence.

³SCID diagnosis for at least one of alcohol or drug abuse or dependence.

A second set of models included collateral ratings prior to the AUS/DUS ratings (full results not presented but available upon request). Collateral ratings improved accuracy and demonstrated incremental predictive utility for all substance use disorders but drug dependence. Across models, all other assessments remained unique predictors, with the exception of DUS ratings for identifying drug abuse.

In addition to testing incremental validity, we created multimethod indices by combining test results. Participants were coded positive for drug abuse if DUS = 3, collateral rating = 2, they self-reported drug use or had a positive biological test; participants were coded positive for drug dependence if DUS ≥ 4, collateral rating = 3, they self-reported drug use or had a positive biological test; participants were coded positive for drug use disorder if they were positive for drug abuse or dependence, as just described. The same approach was used to create indices of alcohol abuse, alcohol dependence, and alcohol use disorder (to the exclusion of biological test results). Not surprisingly, these indices over-identified disorders compared to SCID diagnoses; for example, 27.4% (n = 400) were identified as having drug use disorders and 35.7% (n = 517) as having alcohol use disorders. These indices were no more accurate than AUS/DUS ratings, collateral ratings or self-report, but demonstrated superiority over biological tests in identifying all SCID diagnoses of disordered drug use, zs > 2.77, ps < .006.

Discussion

Though reliable and valid assessments are critical to research and practice (Carey & Correia, 1999; Drake et al., 1990), there have been few evaluations of the individual and combined utility of various approaches for identifying substance use disorders among adults with SMI. We report findings from a large sample of community-based adults with schizophrenia regarding: 1) the diagnostic validity of the AUS/DUS, collateral ratings, self-report, and biological tests in identifying substance use disorders compared to SCID diagnoses; 2) the ability of each approach to discriminate between disorders of abuse and dependence; and 3) the benefits of including information from multiple approaches to assessment accuracy. In the sections that follow, we discuss our findings with respect to each of these study aims.

Accuracy of Various Assessment Approaches

The only measure designed to detect disordered substance use, AUS/DUS ratings performed best in that function. Though there is inherent logic to this finding, it nonetheless deserves comment with regard to the performance of other assessment approaches. Self-report often performed as well as AUS/DUS ratings and contributed uniquely to the prediction of SCID diagnoses. This is consistent with prior research demonstrating the predictive accuracy of self-reported substance use (Wolford et al., 1999). However, the tendency for self-report to over-identify disordered use compared to the SCID is evident, particularly for alcohol. Also consistent with prior work (Van Dorn et al., in press; Wolford et al., 1999), biological tests demonstrated the lowest accuracy for identifying disordered drug use and added little information over other measures. We return to this issue later. That said, congruence with SCID diagnoses was good across assessment approaches. Differences in diagnostic accuracy, when they were found, were relatively small in nature, providing limited support for superiority of one assessment approach.

Discrimination between Disorders of Abuse and Dependence

Discrimination between abuse and dependence by any of the assessment approaches was not supported in these data. Instead, findings suggest a mismatch between the labels and cutoffs that optimize accuracy compared with SCID diagnoses. For instance, using DUS ratings of 2 (use without impairment) and 3 (abuse) to identify drug abuse was more accurate than using ratings of 3 alone. Similarly, combining ratings of 3 (abuse) and 4 (dependence) improved accuracy in identifying drug dependence over ratings ≥ 4. Matches between AUS labels and SCID diagnoses also were not found: ratings ≥ 3 were most accurate across diagnoses. We are not aware of other examinations of concordance between AUS/DUS labels and SCID diagnoses. Clearly, there is a need for continued work in this area. In particular, future research should examine factors that may promote or reduce the ability of clinicians’ AUS/DUS ratings to discriminate between abuse and dependence among adults with schizophrenia, including those associated with training. Moreover, given the lack of discrimination found in the current study, as well as past research, whether abuse and dependence represent distinct diagnostic categories should be examined (e.g., Martin, Chung & Langenbucher, 2008).

Findings of these analyses also speak to the issue of “supersensitivity.” In contrast with the assertion that adults with SMI are unable to use without disorder, many participants in this study were using without meeting diagnostic criteria for disordered use. However, as discussed above, we found that combining DUS ratings of 2 (use without impairment) and 3 (abuse) increased congruence with SCID diagnoses compared to ratings of 3 alone; this was not true for AUS ratings. There are many possible explanations for these results that should be explored in future research. For instance, clinicians’ ability to distinguish between use with and without impairment for alcohol and drugs may differ. Alternatively, adults with schizophrenia may be more “supersensitive” to the effects of drugs than alcohol. Moreover, there are many consequences associated with substance use beyond the psychobiological vulnerabilities described by the supersensitivity model (Mueser et al., 1998) that are relevant to research and practice with adults with schizophrenia. For example, both disordered and non-disordered drug and alcohol use increase the likelihood of a host of negative outcomes among adults with schizophrenia, including treatment nonadherence, exposure to criminogenic factors, violence, and victimization (Hiday, Swartz, Swanson, Borum & Wagner, 2002; Swanson et al., 2006; Swartz et al., 1998a; Van Dorn et al., 2012; Volavka & Swanson, 2010).

Benefits of Multimethod Assessment Strategies

A central question of this study was whether assessment strategies that include multiple indicators improve detection of disordered use. In conjunction with results of prior work (Swartz, Swanson et al., 2003; Wolford et al., 1999), findings suggest that multimethod approaches may be more appropriate for identifying use rather disordered use. Including information from multiple sources improved accuracy incrementally, particularly for the identification of alcohol use disorders, but the multimethod indices also over-identified disordered use compared to SCID diagnoses. When incremental validity was found in the identification of drug use disorders, increases in accuracy generally were small, questioning whether multiple measures, and the biological tests in particular, are worth the time, effort, invasiveness and risks associated with diagnostic false positives. We are unable, however, to determine how knowledge of biological testing may have affected self-report accuracy. Indeed, the value of biological tests in identifying drug use disorders may not be in their contribution of unique information, but rather in their promotion of disclosure.

Limitations

Results should be understood in the context of limitations of the CATIE study design. Though use of SCID diagnoses as criterion measures represents an advance over prior research, it is not infallible and information was not available regarding inter-rater reliability. We also were unable to examine reliability of AUS/DUS ratings. That said, the current research improves upon prior research in several ways and with respect to the studies of the AUS/DUS specifically. Previous evaluations of AUS/DUS assessments have been limited by small samples and an inability to examine accuracy vis-à-vis dependence (e.g., Carey et al., 1996; Drake et al., 1990; Drake et al., 1989; Drake & Wallach, 1989). Additionally, despite the widespread application of this approach in psychiatric settings (Ries et al., 2002), this is only the second study to our knowledge that reported data regarding the validity of DUS ratings, as other research has focused on AUS ratings. Past studies of AUS/DUS ratings and other assessment approaches have been hampered by a lack of detail in the reporting of results and failure to compare assessments against an accepted gold standard.

Additionally, due to the nature of the study, data missingness was a problem. Specifically, collateral interviews were conducted for a subset of participants only; the CATIE study included biological tests of drug but not alcohol use; and results of biological tests for drug use were not available for all participants. Assessment timeframes also differed slightly across approaches, which may account for some variability in performance. Finally, the CATIE study was limited to adults with schizophrenia who were willing to enroll in a longitudinal clinical trial of antipsychotic medication. The generalizability of our findings to adults with other mental illnesses and untreated adults with schizophrenia will need to be tested in future research. Despite these limitations, this study is the largest and most comprehensive evaluation of multiple approaches for identifying substance use disorders among adults with schizophrenia to date.

Conclusions

In conclusion, findings showed congruence with SCID diagnoses for assessment approaches frequently used in research and practice to identify substance use disorders among adults with schizophrenia. Performance was good across methods and there was limited evidence for superiority of assessments completed using one approach over another. Consequently, the decision regarding the ‘best’ diagnostic assessment approach may be made through consideration of practical issues (e.g., cost, administration time) and purpose (e.g., treatment planning, program eligibility, outcome evaluation), rather than selection of the approach that yields the ‘most accurate’ diagnostic assessment.