Latent class analysis of discordance between results of drug use assessments in the CATIE data

Abstract

Objective

The primary aim is to examine concordant/discordant results of drug use assessments in adults with schizophrenia.

Methods

Latent class analysis and multinomial logistic regression were used to examine concordance/discordance between drug use measures and identify characteristics differentiating participants across classes.

Results

Four classes – non-users, users, probable users, and RIA discordant –fit best. Age, sex, race/ethnicity, and psychiatric symptoms differed significantly across classes.

Conclusions

Findings showed that discordance between results occurs at non-trivial rates and is, in part, attributable to individual characteristics. Results suggest the need for strategies to limit discordance and improve detection of drug use in adults with schizophrenia.

1. Introduction

The co-occurrence of schizophrenia and illicit drug use is associated with adverse outcomes such as violence, homelessness, and treatment noncompliance (Swofford et al., 2000; Swanson et al., 2006; Reimherr et al., 2010). Accordingly, accurate identification of illicit drug use is critical for research and clinical practice (Drake et al., 1989; Carey and Correia, 1998; Bennett, 2009). Frequently used measures include self-report, collateral report, clinician interviews, and biological tests.

Such measures are increasingly used in combination to improve identification of drug use. Though this approach may increase detection rates (Drake et al., 1990; Swartz et al., 2003), it introduces the potential for discordance, when measures disagree in their classification of drug use or non-use. Convention has been to classify an individual as drug using if at least one measure produces a positive result (Bahorik et al., 2013; Drake et al., 1990; Swartz et al., 2003). However, doing so may result in the misallocation of limited treatment resources to non-users in cases of false positives. Alternatively, it may preclude treatment or reduce housing options (Drake et al., 2001; Brunette et al., 2004). Moreover, false positives may overestimate the prevalence of drug use in epidemiological research and misinform related policies.

2. Methods

2.1. Study design and sample

We used baseline data from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study, a randomized clinical trial examining antipsychotic medication effectiveness in adults with schizophrenia (N = 1460). Study design and protocol are provided elsewhere (Stroup et al., 2003).

2.2. Measures

2.2.1. Drug use measures

The use of marijuana, cocaine, opiates, PCP, amphetamines, and other illicit drugs was assessed at baseline using: (1) self-report (participants’ self-reported drug use in the prior three months); (2) collateral report¹ (family members/caregivers’ ratings of participants’ drug use in the prior month); (3) clinician ratings (Drug Use Scale ratings of drug use in the prior three months); (4) hair RIA (drug use in the prior three months); and (5) drug urinalysis (drug use in the past one to four days, but up to three weeks). For all measures, responses were dichotomized to indicate use or non-use (Van Dorn et al., 2012; Desmarais et al., 2013).

2.2.2. Participant characteristics

Participant sex was measured dichotomously (1 = female, 0 = male). Age was measured continuously (in years), reflecting age at baseline. Race/ethnicity was measured categorically (3 = White, 2 = Black, 1 = Hispanic, 0 = other race/ethnicity). Psychiatric symptoms were assessed at baseline with the Positive and Negative Syndrome Scale (Kay et al., 1987); we used a 4-factor model to calculate continuous scores for affect, negative, positive, and disorganized cognitive processing (DCP) symptoms (Van Dorn et al., under review).

2.3. Statistical analyses

Latent class analysis (LCA) was conducted in Mplus to identify concordant and discordant classes of drug use measures. The bootstrap likelihood ratio test (BLRT) and adjusted Bayesian Information Criterion (BIC) were used to determine number of latent classes (Nylund et al., 2007). Data in the CATIE is missing at random (Shortreed and Moodie, 2012; Van Dorn et al., 2013); thus, maximum likelihood estimation was used in our analyses to account for missing data. We then conducted two multinomial logistic regressions in SPSS. Non-users served as reference group in the first model, and users, the second. Male and White participants served as reference groups. Odds ratios show the probability of membership in each class as compared to the reference class.

3. Results

3.1. Latent class analyses

Both adjusted BICs and BLRT identified a 4-class model as best fitting the data (adjusted BICs: 3-class = 4371.41, 4-class = 4353.48, 5-class = 4372.36).² Conditional probabilities, which illustrate the probability of each measure indicating drug use, are plotted for latent classes in Fig. 1.

Fig. 1.

Conditional probabilities of drug use assessment measures.

Classes 1 and 2 were both concordant in nature, and Classes 3 and 4 were discordant. Class 1, termed non-users, included participants with near-zero probabilities of being classified as a drug user by each of the measures. Class 2, named users, included participants for whom all measures indicated drug use over half of the time. In Class 3, termed probable users, participants had near-zero probabilities of being classified as drug users by urinalysis, hair RIA, and collateral report, but were almost always identified as drug users by self-report and clinician DUS. In Class 4, named RIA discordant, participants were unlikely to be classified as drug users by all measures except hair RIA, which always classified them as users.

Non-users comprised a majority of the sample (66.2%), followed by users (18.6%), RIA discordant (10.1%), and probable users (5.2%). All four classes exhibited high posterior probabilities, indicating that there were few cases of ambiguity regarding classification.³

3.2. Descriptive statistics

Table 1 presents participant characteristics overall and within classes.

Table 1.

Sample characteristics overall and by latent classes.

Sample characteristics	Total		Latent classes
			Non-users		Users		Probable users		RIA discordant
	N	%	N	%	N	%	N	%	N	%
Overall	1460	100.0	966	66.2	271	18.6	76	5.2	147	10.1
Categorical
Sex
Female	381	26.1	283	74.3	37	9.7	19	5.0	42	11.0
Male	1079	73.9	683	63.3	234	21.7	57	5.3	105	9.7
Race/ethnicity
White	722	49.5	529	73.3	102	14.1	44	6.1	47	6.5
Black	506	34.7	274	54.2	133	26.3	20	4.0	79	15.6
Hispanic	170	11.6	123	72.4	25	14.7	7	4.1	15	8.8
Other/mixed	61	4.2	39	63.9	11	18.0	5	8.2	6	9.8
	M	SD	M	SD	M	SD	M	SD	M	SD
Continuous
Age	40.56	11.10	41.73	10.99	37.35	10.60	34.20	11.27	42.07	10.45
Psychiatric symptoms
Affect	12.14	4.13	11.94	4.13	12.99	4.17	11.82	3.51	12.05	4.18
Positive	17.37	5.64	17.01	5.63	18.38	5.24	19.58	5.88	16.73	5.83
Negative	19.33	6.69	19.60	6.68	18.68	6.61	18.39	6.58	19.30	6.88
DCP	26.82	7.72	26.69	7.85	27.62	7.19	27.13	8.32	26.04	7.41

Notes. Non-users: participants with near-zero probabilities of being classified as drug users by all measures; users: participants for whom all measures indicated drug use over half of the time; probable users: participants who had near-zero probabilities of being classified as drug users by urinalysis, hair RIA, and collateral report, but were almost always identified as drug users by self-report and clinician DUS; RIA discordant: participants who were unlikely to be classified as drug users by all measures except hair RIA, which always classified them as drug users. DCP = disorganized cognitive processing. % = valid percent. Inconsistencies across cells reflect missing data.

3.3. Multinomial logistic regression

Compared to non-users, younger, male, and Black participants were more likely to be classified as users, as were participants with higher and lower levels of affect and negative symptom scores, respectively (see Table 2). Younger participants, and those with higher positive and lower negative symptom scores, were more likely to be probable users. Black participants were more likely to be RIA discordant.

Table 2.

Multinomial logistic regression analysis with non-users as reference.

Variables	Users vs. non-users			Probable users vs. non-users			RIA discordant vs. non-users
	Estimate	SE	Odds (95% CI)	Estimate	SE	Odds (95% CI)	Estimate	SE	Odds (95% CI)
Intercept	−1.960a	0.483	–	−0.268b	0.760	–	−2.669c	0.601	–
Age	−0.038***	0.007	0.963 (0.950–0.976)	−0.066***	0.011	0.945 (0.924–0.967)	0.005	0.009	1.005 (0.988–1.022)
Sex
Female (reference)	–	–	–	–	–	–	–	–	–
Male	1.118***	0.203	3.058 (2.056–4.550)	0.045	0.285	1.165 (0.667–2.036)	0.201	0.204	1.222 (0.819–1.825)
Race/ethnicity
White (reference)	–	–	–	–	–	–	–	–	–
Black	1.087***	0.161	2.965 (2.162–4.065)	−0.139	0.287	0.826 (0.471–1.449)	1.244***	0.202	3.470 (2.336–5.154)
Hispanic	−0.066	0.256	0.936 (0.567–1.545)	−0.529	0.428	0.541 (0.234–1.254)	0.359	0.316	1.4327 (0.771–2.658)
Other race/ethnicity	0.287	0.389	1.333 (0.622–2.856)	0.275	0.516	1.344 (0.489–3.693)	0.619	0.469	1.858 (0.740–4.663)
Psychiatric symptoms
Affect	0.090***	.020	1.094 (1.053–1.138)	−0.029	0.033	0.971 (0.911–1.036)	0.036	0.024	1.005 (0.988–1.022)
Positive	0.025	0.016	1.025 (0.995–1.057)	0.101***	0.024	1.106 (1.055–1.159)	−0.014	0.020	1.037 (0.988–1.087)
Negative	−0.053***	0.013	0.948 (0.925–0.972)	−0.048*	0.021	0.953 (0.915–0.993)	−0.005	0.015	0.986 (0.948–1.026)
DCP	0.011	0.012	1.011 (0.988–1.035)	−0.008	0.020	0.992 (0.955–1.031)	−0.008	0.015	0.992 (0.962–1.022)

Notes. Non-users (n = 966): participants with near-zero probabilities of being classified as drug users by all measures; users (n = 271): participants for whom all measures indicated drug use over half of the time; probable users (n = 76): participants who had near-zero probabilities of being classified as drug users by drug urinalysis, hair RIA, and collateral report, but were almost always identified as drug users by self-report and clinician DUS; RIA discordant (n = 147): participants who were unlikely to be classified as drug users by all measures except hair RIA, which always classified them as drug users. DCP = disorganized cognitive processing. Non-users served as reference group in the analyses.

^*p < .05.

^***p < .001.

^ap < .001.

^bp = .724.

^cp < .001.

Compared to members of the users class, participants classified as probable users were significantly more likely to be younger and female, and more likely to be White than Black (see Table 3). Additionally, probable users had lower affect and higher positive symptom scores. Participant age and sex also distinguished members of RIA discordant from users, with older and female participants more likely to be classified in the former. Participants exhibiting more negative symptoms also were more likely to be in RIA discordant than users.

Table 3.

Multinomial logistic regression analysis with users as reference.

Variables	Probable users vs. users			RIA discordant vs. users
	Estimate	SE	Odds (95% CI)	Estimate	SE	Odds (95% CI)
Intercept	1.692a	0.840	–	−0.709b	0.699	–
Age	−0.029*	0.013	0.971 (0.948–0.996)	0.042***	0.010	1.043 (1.023–1.064)
Sex
Female (reference)	–	–	–	–	–	–
Male	−1.163**	0.336	0.313 (0.162–0.604)	−0.917**	0.264	0.400 (0.238–0.670)
Race/ethnicity
White (reference)	–	–	–	–	–	–
Black	−1.225***	0.309	0.294 (0.160–0.538)	0.157	0.234	1.170 (0.740–1.850)
Hispanic	−0.462	0.473	0.630 (0.249–1.593)	0.425	0.380	1.530 (0.727–3.220)
Other race/ethnicity	−0.013	0.593	0.988 (0.309–3.155)	0.332	0.560	1.394 (0.466–4.175)
Psychiatric symptoms
Affect	−0.119**	0.036	0.887 (0.827–0.952)	−0.054	0.028	0.947 (0.896–1.001)
Positive	0.075**	0.026	1.078 (1.024–1.136)	−0.039	0.023	0.962 (0.919–1.006)
Negative	0.005	0.023	1.005 (0.962–1.051)	0.049**	0.018	1.050 (1.014–1.087)
DCP	−0.019	0.021	0.981 (0.941–1.023)	−0.020	0.018	0.980 (0.947–1.015)

Notes. Non-users (n = 966): participants with near-zero probabilities of being classified as drug users by all measures; users (n = 271): participants for whom all measures indicated drug use over half of the time; probable users (n = 76): participants who had near-zero probabilities of being classified as drug users by drug urinalysis, hair RIA, and collateral report, but were almost always identified as drug users by self-report and clinician DUS; RIA discordant (n = 147): participants who were unlikely to be classified as drug users by all measures except hair RIA, which always classified them as drug users. DCP = disorganized cognitive processing. Users served as reference group in the analyses.

^*p < .05

^**p < .01.

^***p < .001.

^ap = .044.

^bp = .311.

4. Discussion

LCA identified four classes of concordant and discordant test results when multiple measures were used to detect drug use in a sample of 1460 adults with schizophrenia: non-users, users, probable users, and RIA discordant. Together, findings raise general concerns regarding multi-method drug use assessments as well as specific considerations for schizophrenia researchers and clinicians.

Results showed that, compared to non-users, probable users were significantly more likely to be younger in age, with higher positive and lower negative symptom scores. Compared to users, probable users were younger, female, and White, with higher positive and lower affect symptom severity. Prior research has found better reliability in self-reporting drug use in this population associated with younger age, female sex (Drake et al., 1995), and White race/ethnicity (Fendrich et al., 2004; Ledgerwood et al., 2008). Accordingly, the discordance between measures present in probable users is likely due, in part, to participant age, sex, and race/ethnicity and their respective associations with self-reported drug use. Additionally, heightened psychiatric symptoms may affect reliability of self-report, and the associated changes in behavior may influence clinician ratings (Carey and Correia, 1998).

Compared to non-users, those in RIA discordant were significantly more likely to be Black than White. In contrast to users, participants in RIA discordant were more likely to be older and female, and have higher negative symptom severity. It is possible that structural differences across specific hair types lead to selective binding/accumulation of a drug in certain types, resulting in a greater likelihood of testing positive for use (Cone and Joseph, 1996). Moreover, membership in RIA discordant over users differed across levels of negative symptoms; this variation may reflect false positives attributable to similarities in chemical structures of illicit drugs and antipsychotic medication metabolites (Nielsen et al., 2010).

Rates of illicit drug use vary widely across studies, in part due to the different drug use assessment measures employed (Mueser et al., 1990). The decision rule to classify an individual as using if at least one measure tests positive increases detection rates but also heightens risk of false positives. Indeed, requiring at least two positive results would categorize those in RIA discordant as non-users. The prevalence of substance users in our study, conservatively limited to those in the users and probable users classes, is 23.8%; this value falls on the low end of prevalence rates reported in the extant literature, suggesting that prior research may overestimate use (see Van Dorn et al., 2012).

4.1. Limitations

The present study presents several directions for future research. First, we examined age, sex, race/ethnicity, and psychiatric symptoms in our regression analyses; however the low variance explained as a function of demographic variables herein and in other research (Drake et al., 1995) suggests the need to examine additional clinically-relevant characteristics that may distinguish participants between classes. Moreover, examination of concordance and discordance in other populations and by drug class is needed to further investigate methodological concerns related to multi-method assessments. Second, collateral reports were available for only a subset (44.2%) of the full sample and hair RIA data were missing for 323 participants (22.1%). Subsequently, there may have been classes that went undetected. Despite missingness, the remaining sample is still sufficiently large for LCA (Nylund et al., 2007). Finally, the CATIE study sample was comprised of adults in a clinical trial; generalizability of these findings to other samples of adults with schizophrenia is unknown. However, prior research illustrates that the CATIE sample resembles a usual-care, non-interventional population in its characteristics (Swanson et al., 2006).

4.2. Conclusions

This study marks the first application of LCA to evaluate discordance between results of different drug use assessment methods. Results showed that discordance between assessment results occurs at nontrivial rates and is, in part, attributable to participant age, sex, race/ethnicity, and psychiatric symptom severity. Overall, these results suggest the need for strategies to limit discordance, reduce false positives, and improve detection of drug use in adults with schizophrenia; for instance, requiring at least two positive test results—rather than just one—for an individual to be identified as using, or implementing a data weighting approach to reduce misclassifications and optimize data collection efficiency. However, the validity of such strategies would need to be established empirically.