Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jun 1.
Published in final edited form as: Psychopharmacology (Berl). 2021 Feb 8;238(6):1513–1529. doi: 10.1007/s00213-021-05782-2

Beyond abstinence and relapse II: momentary relationships between stress, craving, and lapse within clusters of patients with similar patterns of drug use

Leigh V Panlilio 1, Samuel W Stull 1,2, Jeremiah W Bertz 1, Albert J Burgess-Hull 1, Stephanie T Lanza 2, Brenda L Curtis 1, Karran A Phillips 1, David H Epstein 1, Kenzie L Preston 1
PMCID: PMC8141007  NIHMSID: NIHMS1672037  PMID: 33558983

Abstract

Rationale

Given that many patients being treated for opioid-use disorder continue to use drugs, identifying clusters of patients who share similar patterns of use might provide insight into the disorder, the processes that affect it, and ways that treatment can be personalized.

Objectives and methods

We applied hierarchical clustering to identify patterns of opioid and cocaine use in 309 participants being treated with methadone or buprenorphine (in a buprenorphine-naloxone formulation) for up to 16 weeks. A smartphone app was used to assess stress and craving at three random times per day over the course of the study.

Results

Five basic patterns of use were identified: frequent opioid use, frequent cocaine use, frequent dual use (opioids and cocaine), sporadic use, and infrequent use. These patterns were differentially associated with medication (methadone vs. buprenorphine), race, age, drug-use history, drug-related problems prior to the study, stress-coping strategies, specific triggers of use events, and levels of cue exposure, craving, and negative mood. Craving tended to increase before use in all except those who used sporadically. Craving was sharply higher during the 90 minutes following moderate-to-severe stress in those with frequent use, but only moderately higher in those with infrequent or sporadic use.

Conclusions.

People who share similar patterns of drug-use during treatment also tend to share similarities with respect to psychological processes that surround instances of use, such as stress-induced craving. Cluster analysis combined with smartphone-based experience sampling provides an effective strategy for studying how drug use is related to personal and environmental factors.

Keywords: Ecological momentary assessment, opioid use disorder, time-varying effects modeling (TVEM), stress, craving, cocaine, buprenorphine, methadone

Introduction

The terms abstinence and relapse represent extremes in the possible outcomes that occur during treatment for substance use disorder. However, most patients fall somewhere between these extremes, and many benefit from treatment without becoming abstinent (Kiluk et al. 2017; Roos et al. 2019a). In our experience treating patients with opioid and cocaine use disorders, a range of drug-use patterns emerge during treatment, with some people using frequently, some becoming abstinent, and others using infrequently or sporadically. We recently suggested that patterns such as these can be formally identified and used as an outcome measure for clinical trials of treatments for substance-use disorder (Panlilio et al. 2020). Specifically, we showed that unsupervised machine learning techniques (Hastie et al. 2017; James et al. 2013) can be used to identify clusters of people who share similar patterns of use, and that some of these non-abstinence patterns are associated with desirable outcomes, such as reductions in drug craving and other symptoms of substance use disorder.

In our previous report (Panlilio et al. 2020), we focused on methodology and practical applications for this clustering approach, using cluster membership as an outcome measure for assessing the effects of randomized experimental interventions. As we briefly mentioned in that paper but have not described in detail prior to the present paper, we originally developed this approach using a different set of data that were obtained from people being treated with methadone or buprenorphine in a series of natural-history studies that were conducted to gain insight into how stress and craving influence drug use. The present paper describes those original analyses, which focus on characteristic differences between clusters of people with specific patterns of drug use.

In addition to sequences of drug test results, the natural-history data analyzed here included baseline assessments (self-reported drug use, drug-related problems, exposure to emotional abuse, and strategies for coping with stress, all prior to joining the study), as well as smartphone-based ecological momentary assessment (EMA), which can provide a richly detailed sample of the person’s experiences in daily life (Bolger and Laurenceau 2013; Lukasiewicz et al. 2007; Shiffman et al. 2008). For each participant, we had extensive EMA data over the course of the study, encompassing up to 16 weeks during treatment for opioid-use disorder. These data allowed us to assess momentary levels of stress, drug-cue exposure, and drug craving, as well as self-reports of what precipitated specific drug-use events and how the person felt afterward. Momentary levels of stress and craving tend to be positively correlated (Moran et al. 2018; Preston and Epstein 2011; Preston et al. 2017; 2018b), and craving often precedes drug use (Preston and Epstein 2011; Preston et al. 2018a; Preston et al. 2009). However, we suspect that these relationships vary considerably between individuals and contribute to differences in patterns of drug use. Therefore, we expect that the clusters we identify based strictly on the temporal pattern of drug use will also differ with respect to relationships between stress, craving, and use. If so, this would provide external validation for the clusters and demonstrate the utility of the clustering approach.

Our overarching goal in the present paper is to complement and externally validate our clustering results by examining the demographic and psychosocial characteristics associated with cluster membership, with a focus on stress and craving (which the studies were designed to track). Specific goals include determining how drug-use clusters differ with respect to: 1) age, sex, race and treatment medication; 2) mental-health and drug-related problems prior to starting treatment; 3) what kinds of strategies they generally describe themselves as using to deal with stress; 4) average levels of craving, stress, cue exposure, positive mood, and negative mood; 5) how they feel after a drug-use event; 6) what they report as triggers of use; 7) whether they experience craving prior to use events; and 8) how much and how long they experience craving as a reaction to stress.

Methods

Participants

All participants (N=309) were treated with opioid agonist medication throughout the study, with 51% receiving methadone and 49% receiving buprenorphine-naloxone (which for brevity we will refer to as buprenorphine). A majority of participants were male (78%). Most participants self-identified as African American (64%) or white (34%), with the remainder identifying as Asian or more than one race. The mean age was 48 years (standard error of the mean [SEM] = 0.6). Participants were recruited through advertisements in a variety of newspapers that are read by both sexes and all ethnicities. All participants made regular visits to our Archway research clinic in Baltimore, Maryland, to provide urine samples for drug testing and to maintain the smartphones that we provided for EMA data collection. Eligibility criteria were: age 18–75 years; physical dependence on opioids; and residence in or near Baltimore city. Exclusion criteria were: any DSM-IV psychotic disorder; history of bipolar disorder; current major depressive disorder; current DSM-IV dependence on alcohol or sedative-hypnotics; cognitive impairment severe enough to preclude informed consent or valid self-report; or medical illness that would compromise participation.

Study Design and Procedures

All data were obtained as part of an intensive longitudinal study (Bolger and Laurenceau 2013; Ginexi et al. 2014) that used a natural-history design to assess relationships between stress, drug craving, and drug use during outpatient treatment for opioid use disorder (National Clinical Trial Identifier NTC-00787423). Enrollment ran from July 2009 through April 2018. Other findings from this study have been reported previously (Furnari et al. 2015; Moran et al. 2018; Panlilio et al. 2019; Preston et al. 2017; 2018a; b; Preston et al. 2018c). The present paper describes a secondary analysis that includes all participants from the previous reports, plus additional participants who completed the study more recently or who were enrolled in an arm of the study that received buprenorphine or methadone treatment in the community (Treatment Elsewhere arm), whose data were not analyzed previously. Ethical approval for the study was granted by The NIH Addictions Institutional Review Board, and all participants provided informed consent.

The study as a whole was conducted in three closely related, temporally overlapping arms that shared most procedures but differed with respect to the number of urine samples per week, study duration, medication provider, and the size of incentives given for compliance with study requirements. Participants in the Methadone-Buprenorphine arm (MTD-BUP; n = 194; conducted July 2009 through October 2015) received medication from Archway and provided urine samples 3 times per week (Moran et al. 2018; Preston et al. 2017; 2018b). Participants in the Office-Based Opiate Treatment arm (OBOT; n = 47; conducted February 2016 through September 2017) received medication from Archway and provided urine samples 2 times per week. Participants in the Treatment Elsewhere arm (TE; n = 68; conducted April 2016 through September 2017) received their medication at various community clinics in or near Baltimore and provided urine samples at Archway 3 times per week. All participants were offered counseling for opioid use disorder at Archway. Data were analyzed from the maintenance phase of the study, which was scheduled to last 16 weeks for MTD-BUP and OBOT and 8 weeks for TE. Maintenance started after 2 weeks of induction in the MTD-BUP and OBOT arms. Participants in the TE arm were already in the maintenance phase of treatment before joining our study.

In the TE arm, each participant’s medication (buprenorphine or methadone) had been initiated at an offsite treatment provider prior to joining our study. In the other arms, the medication was determined by the participant’s preference, the clinical judgment of the study physician, and also by what was being offered by Archway at the time of the individual’s enrollment (for details, see Panlilio et al, 2019). Medication doses were optimized for each participant to minimize withdrawal symptoms and reduce illicit opioid use without causing intolerable side effects. Contingency management was used to encourage abstinence from opioids and cocaine and to enhance compliance with EMA reporting. Incentive values for contingency management varied somewhat across study arms and enrollment dates, ranging from $5-$10 per visit if urine samples were negative and consistent with self-reports (EMA), but with a smaller incentive ($3–6 per visit) if samples were positive and consistent with self-reports, and no incentive if samples were positive but not reported. An additional $10-$30 was given for completing at least 82% of randomly prompted reports within the week. Maximum compensation was $25 per week for MTD-BUP and $50 per week for OBOT and TE.

Data Collection

At the beginning of the study, each participant completed the COPE inventory (Carver et al. 1989; Hassanbeigi et al. 2013), which assesses the use of 15 strategies for coping with stress (positive reinterpretation and growth, mental disengagement, focus on and venting of emotions, use of instrumental social support, active coping, denial, religious coping, humor, behavioral disengagement, restraint, use of emotional social support, acceptance, planning, suppression of competing activities, substance use). They also completed version 5 of the Addiction Severity Index (ASI) (McLellan et al. 1985) in a semi-structured interview that gathered information about the individual’s history of drug use and psychological/emotional problems (e.g., depression, anxiety, thoughts of suicide) that were “not a direct result of drug/alcohol use.”

Urine samples were tested for recent use of opioids (opiates, oxycodone, nonprescribed use of methadone or buprenorphine), cocaine, cannabis, amphetamines, barbiturates, benzodiazepines, methadone, and buprenorphine, but opioids and cocaine use were by far the most prevalent type of positive result and are the focus of this analysis. Maximum post-use detection times for these tests are approximately 2–4 days for cocaine, 2–3 days for opiates, and 1–2 days for oxycodone. We did not test for fentanyl, which was increasingly used in Maryland as an additive to heroin during the last few years of the study (Maryland Department of Health 2018).

Each participant was issued a smartphone and trained to use our custom-made app for four kinds of EMA reports: 1) randomly prompted reports; 2) participant-initiated reports of drug use (Furnari et al. 2015); 3) participant-initiated reports of more-than-usual stress (Preston et al. 2017); and 4) scheduled end-of-day reports (Preston et al. 2018c).

Random prompts were scheduled three times per day (during each participant’s normal waking hours) to assess mood, drug-cue exposure, stress, and craving at the time of the prompt. Mood was assessed with ratings for each of 24 adjectives, which were converted into two composite scores for analysis: positive mood and negative mood (Epstein et al. 2014). Cue exposure was defined as seeing drugs, seeing someone selling drugs, or being offered drugs within the last 5 minutes or since arriving at the current location. Heroin craving, cocaine craving, and stress were each rated on a Likert scale (1 = not at all; 2 = a little; 3 = moderate; 4 = a lot; 5 = extreme). Randomly prompted EMA reports also included some other information that was not analyzed for this paper, including who the participant was with and what they were doing at the time of the report. The mean (± standard error of the mean, SEM) number of randomly prompted EMA reports completed per participant in each arm was 269.2 ± 3.7 (MTD-BUP), 297.3 ± 5.8 (OBOT) and 150.9 ± 3.9 (TE). Out of the three random prompts that were programmed each day, the mean ± SEM number of completed reports per participant per day was 2.37 ± 0.04 (MTD-BUP), 2.58 ± 0.05 (OBOT) and 2.73 ± 0.07 (TE). The mean (± SEM) number of end-of day EMA reports completed per participant in each arm was 81.5 ± 1.6 (MTD-BUP), 94.0 ± 2.3 (OBOT) and 47.7 ± 1.3 (TE); per day, these were 0.72 ± 0.01 (MTD-BUP), 0.82 ± 0.02 (OBOT) and 0.86 ± 0.02 (TE).

Participant-initiated reports of drug use included information about how the event was triggered. In response to the question “What reasons do you think were important for using this time?,” a participant could select all that applied from the following list: boredom, being asked to use, being asked to “cop” (buy drugs) for someone else, routine, feeling sick, stress, anger/frustration, anxiety/fear, seeing something/someone that reminded them of using (cues), wanting to feel good, and wanting to see what would happen. Participant-initiated reports of drug use also included self-ratings of how much the person enjoyed the use and how guilty they felt afterwards. Drug use that had not been recorded in participant-initiated reports could be recorded in a randomly prompted report (if it occurred “Within 5 minutes of the beep / Since you got to your present location”) or in the end-of-day report (in response to the question, “Did you use any drugs at all today without reporting it?”), but only participant-initiated drug reports provided specific times of use and information about triggers and feelings of enjoyment and guilt. End-of-day reports also included some other information that was not analyzed for this paper, including hassles experienced during the day, sexual behavior, and the amount and quality of sleep in the previous night. The mean (± SEM) number of participant-initiated reports of drug use in the three arms were 20.1 ± 1.7 (MTD-BUP), 18.4 ± 2.1 (OBOT) and 23.6 ± 4.0 (TE) per participant; per day, these were 0.18 ± 0.01, 0.16 ± 0.02 and 0.45 ± 0.07, respectively.

In each randomly prompted report and in each end-of-day report, participants were asked to report any drug use that had not yet been reported. The mean (± SEM) number of randomly prompted reports per participant that included otherwise-unreported drug use in each arm of the study was 10.9 ± 1.7(MTD-BUP), 11.1 ± 2.6 (OBOT) and 20.2 ± 3.7 (TE) per participant; per day, these were 0.09 ± 0.01, 0.1 ± .0.02 and 0.37 ± 0.01. The mean (± SEM) number of end-of-day reports that included otherwise-unreported drug use were 4.9 ± 0.6 (MTD-BUP), 4.6 ± 1.2 (OBOT) and 4.2 ± 1.1 (TE) per participant; per day these were 0.04 ± 0.01, 0.04 ± 0.01 and 0.08 ± 0.02. Mean percentage agreement (± SEM) between use reported in EMA (whether through participant-initiated reports of drug use, randomly prompted reports, or end-of-day reports) and the results from the next urine sample were 83.9 ± 0.01 (MTD-BUP), 84.2 ± 0.03 (OBOT) and 84.8 ± 0.02 (TE).

Statistical Analysis

Clustering.

A sequence of urinalysis results was obtained from each participant. For each test, five kinds of results were possible: positive only for opioids, positive only for cocaine, positive for both opioids and cocaine (dual use), negative for both opioids and cocaine, or missing. Participants who were missing 20% or more of their scheduled urinalysis results were excluded from hierarchical clustering, but were designated a priori as a cluster for other analyses. This Dropout cluster included all participants who left the study voluntarily or were expelled due to noncompliance, as described and analyzed earlier (Panlilio et al. 2019), plus two participants who were not expelled but had intermittently missing data that exceeded 20%. For all other participants, urinalysis sequences were clustered using agglomerative hierarchical clustering with the Ward method (Maechler et al. 2018) using distance scores determined by an optimal-matching algorithm that takes sequential patterns into account (Gabadinho et al. 2011; Studer and Ritschard 2016), as opposed to matching based only on ordinal position (i.e., test number). For test results prior to the last observation in a sequence, missing points are treated as belonging to the “missing” category; any missing points after the last observation are ignored. We previously showed that this clustering procedure is sensitive to changes in drug use during treatment with methadone combined with contingency management (Panlilio et al. 2020). Since the three arms of the present study differed in their frequencies of drug testing and in their study durations, clustering was applied to each arm separately. Clustering techniques require judgment by the investigator regarding how fine the distinctions should be between clusters (Hastie et al. 2017; Hennig 2015; Milligan and Cooper 1987). In other words, the investigator must choose the number of divisions that makes the most sense for the task at hand, based on interpretability of the clusters (in the sense that a pattern can be recognized visually and described verbally), consistency of patterns within clusters, and distinctness of patterns between clusters. We chose the number of clusters for each arm of the study based on visual inspection of the dendrograms and heatmaps (shown in the Supplementary Material), prior to conducting any of the analyses of cluster characteristics described below. Because the results from each of the three arms were well-described by five prototypical cluster types, these clusters were combined across arms for the following analyses.

Characteristics of the clusters (including average EMA data, triggers and consequences of drug use, and coping strategies).

After the clusters were identified, demographic data, questionnaire data from the ASI and COPE, and EMA data were analyzed to further characterize the clusters. Numeric variables were compared between clusters using analysis of variance or multilevel modeling, followed by paired comparisons with familywise error controlled using the Holm procedure. For numeric variables, effect sizes are reported as reffect (Rosnow and Rosenthal 1996; Rosnow et al. 2000). Non-numeric variables were analyzed with χ2 tests of independence, assessing effect size with Cramer’s V (with 0.3 considered a medium effect and 0.5 considered a large effect). Opioid and cocaine craving were combined for analysis, and opioid cues and cocaine cues were also combined; these combined measures and the positive and negative mood scores were each expressed as the mean of their components to maintain comparability with the original Likert scale (1–5), with a score of 5 for the combined craving measure representing maximal craving for opioids and cocaine at the same time. To simplify analysis of EMA and coping-strategy data, clusters with high levels of drug use were combined across drug classes; that is, high-use opioid, high-use cocaine, and high-use dual clusters were combined into a single “High-Use” cluster. The variability of EMA responses from randomly prompted reports was quantified as root-mean-square of successive deviations (rMSSD; the average amount of change between successive measures, with higher values indicating more instability) and analyzed with analysis of variance. The 15 categories of coping styles in the COPE inventory (each with a score from 0–16, with higher scores indicating endorsement of more items in the category) were analyzed using a multilevel linear regression (with random intercepts and common slopes for participants, since there was only one observation per strategy per person).

Craving prior to drug use.

Multilevel logistic regression (generalized linear mixed modeling) was conducted to analyze differences between the clusters in whether craving was experienced prior to use of opioids and/or cocaine. The general approach used in this analysis was applied earlier to analyze craving and other EMA variables prior to drug use in unclustered samples (Epstein et al. 2009; Preston and Epstein 2011; Preston et al. 2018a; Preston et al. 2009). The period prior to use is of particular interest because post-use ratings could be influenced by direct effects of cocaine and heroin. This logistic regression modeled the presence vs. absence of craving in randomly prompted EMA reports obtained during the 24-hour period prior to drug events that occurred at least 24 hours since previous use. To conduct this analysis, it was necessary to define periods of nonuse for each participant, and to exclude from analysis any periods when use vs. nonuse was ambiguous. Time of use was specified in each participant-initiated report, but time of use was ambiguous in randomly prompted reports and end-of-day reports (which asked whether any unreported use had occurred). Therefore, to restrict the analysis to periods in which at least 24 hours of nonuse preceded a use event, craving data were included in the analysis if they were obtained at least 24 hours since any report of use and no more than 24 hours prior to the specific time indicated in a participant-initiated report of use. For periods where use was reported in randomly prompted or end-of-day EMA (i.e., without a specific time), the period of nonuse was calculated as if use had occurred at the time the report was completed. Thus, time since use was conservatively defined to avoid mislabeling periods of actual use as nonuse, but possibly excluding some actual periods of nonuse. There were 102 High-Use, 25 Sporadic, and 26 Negative participants who reported drug-use events that met this criterion. Craving had a prominent mode at the lowest level (i.e., in most EMA reports, participants did not report any craving), so for this analysis craving was dichotomized as 1 (“not at all”) vs. anything greater than 1, then modeled as a function of cluster, time until the use event, and their interaction. Information-criterion fit statistics for this multilevel logistic model supported using random intercepts with common slopes for participants.

Stress-craving reactivity.

Time-varying effects modeling (TVEM) (Lanza et al. 2014; Tan et al. 2012) was used to explore within-day, time-lagged association between stress and craving. TVEM allows the association between variables to be modeled continuously across time without imposing parametric assumptions about the nature of the association (e.g. linear or quadratic). This approach is well suited for studying the timecourse of effects when the values are measured at random points in time. Our analyses were adapted from the procedure used by Shiyko et al. (2014) to test the association between negative affect and subsequent urge to smoke tobacco. In our model, time is defined as the lag between consecutive randomly prompted EMA reports, so the time-varying intercept and slope provide a model of the craving level in the current report as a function of the stress level in the previous report. TVEM (using the B-spline method) was applied separately to data from the High-Use cluster, Sporadic cluster, and Negative (i.e., low-use) cluster. Based on the procedure of Shiyko et al. (2014), the maximum lag was set to 7 hours. Only lags occurring within the participant’s normal waking hours were included. Time of drug use was not included as part of the TVEM model, which only assessed the relationship between stress and craving. Participants who never reported stress or craving (i.e., whose ratings were never higher than “not at all”) were excluded from this analysis. There were 98 High-Use, 25 Sporadic, and 71 Negative participants who had lagged data that met criteria for this analysis. The relative model fit of each TVEM was tested by fitting different combinations of knots ranging from 1 to 6 for the intercept and slope, and then comparing them based on Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). The knots represent splitting points in the coefficient functions, where more knots reflect greater complexity in time-varying change. In all models, the best fit was obtained with one knot for intercept and one knot for slope.

All analyses except TVEM were conducted with R software (R Core Team 2018), including the packages “TraMineR” (Gabadinho et al. 2011) for optimal matching, “cluster” (Maechler et al. 2018) for hierarchical clustering, “ggplot2” (Wickham 2009) for heatmaps, and “lme4” (Bates et al. 2015) with “car” (Fox and Weisberg 2019) for multilevel linear and logistic regression. TVEM was conducted using the “SAS %TVEM” macro (Li et al. 2017).

The F and χ2 values reported in the Results describe omnibus tests, and reffect refers to paired comparisons. P values are described as significant if p<.05 and marginal if .05 < p <.1. Since cluster analysis of these data was not planned when the studies were originally designed, we are using p values in an exploratory context (Gaus 2015; Rubin 2017). Accordingly, we do not interpret the p values in the increasingly challenged sense that they mark the presence or absence of a finding (Wasserstein and Lazar 2016); instead, we use them as a graded, imperfect guide to determining how our findings might be prioritized for further study. Although we conducted many analyses, they all focus on characteristics of the clusters related to drug use (especially the role of stress and craving); we conducted the analyses as described, and we are reporting the results in full.

Results

Identification of the clusters

The selected clustering solution divided the urinalysis sequences (Fig. 1) into five naturally interpretable categories that have face validity for classifying treatment outcomes: 1) mostly negative; 2) mostly positive for opioids; 3) mostly positive for cocaine; 4) mostly positive for dual use; and 5) sporadically positive for one or both drugs. With four types of outcome (opioid, cocaine, both, and negative), it is natural to first consider a four-cluster solution, with one cluster corresponding to each type of outcome (such as we reported earlier with a completely different set of data; Panlilio et al. 2020). However, in the present analysis it was necessary to include a Sporadic cluster because certain clusters were not internally consistent otherwise (see Supplemental Material, Figs S2S4). Specifically, with a four-cluster solution instead of the five-cluster solution that we selected, the participants with sporadic patterns were similar to each other but different from the rest of the other participants they were grouped with; in the four-cluster solution the participants with sporadic patterns were grouped together with the Cocaine cluster for the MTD-BUP arm, the Negative cluster for the OBOT arm, and the Dual-Use cluster for the TE arm.

Fig. 1.

Fig. 1.

Heatmap depicting the sequence of urinalysis results. Each row represents the results from one participant. Each cell represents the result of a single sample, which could be negative for opioids and cocaine (‘Neg’), positive for opioids (‘Opi’), positive for cocaine (‘Coc’), or positive for both (‘Both’, i.e. dual use). White cells represent missing data. The first five clusters were identified by hierarchical clustering analysis and named according to the pattern of urinalysis results. “Dropout” refers to the participants who provided less than 80% of scheduled samples. The three study arms are shown in separate columns: standard methadone or buprenorphine treatment (MTD_BUP), office-based buprenorphine (OBOT), and receiving treatment elsewhere in the community (TE). Within each arm of the study, the order of participants within each cluster (except Dropout) was determined by the hierarchical clustering algorithm.

With five clusters, the results were similar across all three arms of the study, with high consistency within clusters and high distinctness between clusters. However, as can happen in unsupervised machine learning procedures (in which the clusters are not predefined and boundaries are fuzzy), some cases were ambiguous. Two participants from the shorter-duration arm of the study (TE) were classified as being in the Sporadic cluster despite having only drug-positive results, probably because the sequences in this arm contain less information (fewer observations) per participant, and both of these participants were missing three consecutive samples. Considering the otherwise consistent results across all arms, to maintain interpretability of the Sporadic cluster these two participants were included with the Dual-Use cluster for subsequent analyses of cluster characteristics. Combined across all 309 participants from all three arms of the study (including the 27.7% of participants who were Dropouts), 30.0% of participants were in the Negative cluster, 8.1% were in the Sporadic cluster, and the rest were split between the three high-use clusters (12.7% Opioid, 10.1% Cocaine, 11.4% Dual). Visual inspection of the urinalysis results (Fig. 1, lower panels) suggests that the Dropout category had relatively few participants with mostly negative results. In the OBOT arm, which unlike the other arms did not include participants treated with methadone, the Dropout category also had a relatively large number of participants whose tests were mostly positive for opioids only.

General characteristics of the clusters

The number of participants receiving methadone treatment was disproportionately high in the Cocaine cluster and disproportionately low in the Opioid cluster (Fig. 2A; χ25 = 29.8, p < .0001; Cramer V1 = .31). The number of African-American participants was disproportionately high in the Cocaine and Dual-Use clusters and disproportionately low in the Dropout cluster (Fig. 2B; χ25 = 27.3, p < .0001; Cramer V1 = .3). None of the clusters differed significantly from the expected proportions of male and female participants (Fig. 2C; χ25 = 6.0, p= .3). The Opioid and Dual-Use clusters were slightly older than the Sporadic and Dropout clusters, and the Dual-Use cluster was also slightly older than the Negative cluster (Fig. 2D; F5, 324 = 6.1, p < .0001; reffect range = .15 to .22). During the 30 days prior to enrolling in the study (as self-reported retrospectively in their ASI interviews), the Dropout participants had more emotional problems than the Opioid and Negative clusters (Fig. 2E; F5, 323 = 3.05, p = .01; reffect range = .15 to .18) and more days with recent drug-related problems compared to the Negative cluster (Fig. 2F; F5, 323 = 2.28, p = .046; reffect = .16).

Fig. 2.

Fig. 2.

Demographics, self-reported psychosocial measures, and self-reported history of drug use prior to the study. A: Proportion of participants receiving methadone, as opposed to buprenorphine. B: Proportion of African-American participants. C: Proportion of female participants. D: Mean age. E: Mean number of recent days in which psychological/emotional problems were experienced. F: Mean number of recent days in which drug-related problems were experienced. G: Mean years of heroin use. H: Mean years of cocaine use. I: Mean years of polydrug use. J: Recent heroin use. K: Recent cocaine use. L: Recent polydrug use. “Recent “refers to the 30-day period before the study. Error bars indicate standard error of the mean (SEM). For categorical variables (panels A-C), dotted lines represent the proportion in the overall unclustered sample, and an asterisk (*) or numeric p-value indicates a cluster proportion that differed significantly (p < .05) or marginally from the overall proportion. For numeric variables (panels D-L), * or a numeric p-value indicates a mean that differed significantly (p < .05) or marginally from another mean or means in the same panel, as follows. In panel D, the Dual cluster was older than the Dropout, Negative and Sporadic clusters, and the Opioid cluster was older than the Dropout and Sporadic clusters. In panel E, the Negative cluster differed from the Dropout cluster and the Opioid Cluster. In panel F, the Negative cluster differed marginally from the Dropout cluster. In panel G, Dual vs. each other cluster except Opioid, and Opioid vs. Negative. In panel H, Dual vs. each other cluster. In panel I, Dual vs. Negative and Dropout. In panel J, Negative vs. each other cluster except Cocaine. In panel K, the Dual, Cocaine and Sporadic clusters vs. each of the other three clusters (i.e., Negative, Dropout, Opioid). In panel L, Negative vs. each other cluster.

With regard to lifetime history of drug use, the Dual-Use cluster had more years of heroin use than all other clusters except the Opioid cluster, and the Opioid cluster had more years of heroin use than the Dropout cluster (Fig. 2G; F5, 324 = 8.7, p < .0001; reffect range = .16 to .26). The Dual-Use cluster had more years of cocaine use than the Negative, Opioid and Dropout clusters (Fig. 2H; F5, 323 = 5.4, p < .0001; reffect range = .19 to .22), and the Dual-Use cluster also had more years of polydrug use (i.e., using more than one drug at a time) than the Negative and Dropout clusters (Fig. 2I; F5, 316 = 3.6, p = .003; reffect range = .2 to .22). During the 30 days prior to enrolling in the study (i.e., based on retrospective self-report in the ASI), the Negative cluster already had lower rates of heroin use compared to all other clusters except the Cocaine cluster (Fig. 2J; F5, 324 = 5.8, p < .0001; reffect range = .18 to .24) and lower rates of polydrug use than all other clusters (Fig. 2L; F5, 323 = 6.3, p < .0001; reffect range = .17 to .22); also, the Dual-Use, Cocaine and Sporadic clusters already had more cocaine use than the Negative, Dropout and Opioid clusters (Fig. 2K; F5, 324 = 16.6, p < .0001; reffect range = .16 to .26).

Mood, stress, cue exposure and craving in randomly prompted EMA

In all clusters, stress and craving were most often reported to be at the lowest level (“1, not at all”). In fact, “not at all” was always reported as the current level of craving by 26 participants (8.4%) and the current level of stress by 11 participants (3.6%); among these 37 participants, 4 always reported “not at all” for both stress and craving. Among those who never reported craving, most (69.3%) were in the Negative cluster, which had a higher within-cluster percentage of noncravers (19.3%) than did any other cluster (χ23 = 21.0, p = .0001, with p values < .004 for all paired comparisons between the Negative cluster and the Sporadic cluster, the Dropout cluster, or the combined High-Use clusters). The number of participants who never reported stress was too small to formally compare across clusters, but all were in the High-Use or Negative clusters.

For stress and for craving, there were progressively fewer endorsements at each level of the Likert scale from 1 through 5. The highest levels, 4 (“a lot”) and 5 (“extreme”), were endorsed a substantial number of times for stress, but were rarely endorsed for craving. Averaging over time within the study, analysis of variance indicated that the High-Use clusters reported significantly more craving (Fig. 3A; F2, 221 = 6.0, p = .003; reffect = .19), marginally more exposure to drug-related cues (Fig. 3A; F2, 221 = 2.67, p = .07; reffect = .11), and significantly less experience of positive mood (Fig. 3A; F2, 221 = 4.56, p = .01; reffect = .16), but they did not differ with respect to stress or negative mood (Fig. 3A). The clusters also differed significantly or marginally in the stability of all of these measures except negative mood (Fig. 3B; craving: F2, 219 = 12.9, p < .0001, reffect = .56; stress: F2, 219 = 2.7, p = .06, reffect = .24; cue exposure: F2, 219 = 11.0, p = < .0001, reffect range = .14 to .51; positive mood: F2, 219 = 3.0, p = .05, reffect = .11); in short, the Negative cluster had the lowest, most stable levels of craving, stress and cue exposure, and the High-Use clusters had the lowest, most stable levels of positive mood.

Fig. 3.

Fig. 3.

Ecological momentary assessment (EMA) and coping style results by cluster, with the opioid, cocaine, and dual clusters combined (“High Use”). A: Levels of craving, stress, cue exposure, positive mood and negative mood averaged over the course of the study, as reported in randomly prompted EMA entries. B: Within-person instability (i.e., root-mean-square of successive deviations, rMSSD) for the same variables shown in first panel, with higher values indicating more instability. C: EMA ratings from participant-initiated reports of drug events, including feelings of guilt and enjoyment associated with the use event, and the proportion of reports in which the use event was precipitated by stress, cues, or being asked to use drugs (not mutually exclusive). D: Coping strategies endorsed prior to the study. Except for the combination of all strategies (far right), only individual strategies that differed significantly between clusters are shown in the figure. All variables are shown as mean (± SEM). * indicates a significant difference vs. the Negative cluster (p < .05). # indicates a significant difference vs. the Sporadic cluster. ‘B’ indicates a significant difference vs. both other clusters.

Triggers and immediate emotional effects of drug use

In participant-initiated reports of drug use, the High-Use clusters reported marginally more enjoyment from the use than the Negative cluster did (Fig. 3C; F2, 175 = 2.8, p = .06; reffect = .17), but the clusters did not differ in their ratings of how guilty they felt afterwards. In all clusters, participants attributed about 20% of drug-use events to stress, but the Sporadic cluster attributed few drug-use events to cue exposure (Fig. 3C). There were clear differences between the clusters in the proportion of drug-use events that were attributed to being asked to use by someone else (Fig. 3C; F14, 175 = 6.4, p = .002; reffect range= .24 to .37), with the Sporadic cluster endorsing this trigger more than the other clusters, the Negative cluster endorsing it rarely, and the High-Use clusters endorsing it at a level intermediate to the Sporadic and Negative clusters. The other potential triggers of use did not differ between clusters; collapsing across clusters, the proportions ± SEM in descending order were: wanting to feel good (.27 ± .33), anger/frustration (.21 ± .28), boredom (.18 ± .06), routine (.13 ± .26), feeling sick (.11 ± .24), anxiety/fear (.1 ± .21), being asked to “cop” for someone else (.09 ± .18), and “wanting to see what would happen if I used just a little” (i.e., testing self-control) (.05 ± .14) .

Coping strategies

Strategies for coping with stress differed as a function of cluster (Fig. 3D; strategy: F14, 209 = 126.8, p < .001; cluster × strategy interaction: F70, 209 = 1.6, p = .005), and paired comparisons revealed between-cluster differences in 4 of the 15 types of strategies (reffect range= .25 to .41). Most interestingly, compared to the other clusters, the Negative cluster showed less endorsement of substance use as a strategy for coping with stress. The High-Use clusters had slightly lower endorsement of disengagement and humor as coping strategies, and the Sporadic cluster had low endorsement of religion-based coping. Although it might be expected that implementing multiple strategies would be more effective for coping with stress and would thereby decrease the frequency of drug use, the average for all strategies combined did not differ across clusters (Fig. 3D).

Craving prior to drug use

The High-Use and Negative clusters showed increased incidence of craving as a drug-use event approached, but the Sporadic cluster did not (Fig. 4A). For the multilevel logistic regression, time was expressed as hours until use, with use occurring at 0; thus, increased craving prior to use would be indicated by an odds ratio less than 1, with values farther away from 1 indicating a stronger effect. This analysis indicated that craving during the 24-hour period prior to use was associated with time and cluster (time until use: χ21 = 33.6, p < .001; cluster: χ22 = 7.4, p = .025; time × cluster interaction: χ21 = 7.0, p = .03). Odds ratios from this analysis, with High-Use as the reference, indicate that: 1) craving was not present in most reports, as indicated by a low odds ratio (OR) for the intercept (with the 95% confidence interval in brackets, OR = 0.22 [0.11, 0.4]); 2) craving was increasingly likely as the use event approached, as indicated by the odds ratio for time until use being less than one (OR = 0.96 [0.94, 0.97]) ; and 3) craving increased more steeply over time in the Negative cluster, as indicated by this cluster’s odds ratio for the interaction of cluster with time until use being less than one (OR = 0.88 [0.79, 0.97]).

Fig. 4.

Fig. 4.

Craving levels during the hours prior to drug-use events in the High-Use, Sporadic and Negative clusters during time periods that preceded a use event that occurred after at least 24 hours of nonuse. Each point indicates the mean (± SEM) proportion of EMA reports in which craving occurred, with craving treated as a binary outcome and the proportion of positive reports calculated within each participant over the 12 consecutive two-hour periods prior to use. Lines depict linear regression of the points in each panel, to visualize the data used in the logistic regression.

Stress-craving reactivity

The time-varying effect models (TVEM, Fig. 5) describe how the stress-craving relationship changed depending on the amount of time elapsed since the stress level was measured. In these models, statistically significant differences (p<.05) between clusters can be conservatively assessed by whether confidence bands are nonoverlapping at a particular time lag. The intercept curves (Fig. 5A) indicate the mean craving level when the stress level in the previous report had been minimal (“Not at all”); these show that the Negative cluster generally had low basal levels of craving, with most of the curve lower than the High-Use and Sporadic clusters. The increase in basal craving level at longer lags in the Sporadic cluster suggests increased craving later in the day, since long lags could not occur early in the participant’s waking hours. Most interestingly, the slope functions (Fig. 5B) depict the strength of the stress-craving relationship as a function of lag time between assessments, with the high but descending part of the curve for the High-Use clusters showing significantly stronger reactivity to stress than the other clusters for about 90 minutes after stress was assessed. The Sporadic and Negative clusters were also reactive to stress, as indicated by nonzero slopes across the whole curve, but there was no substantial change in the strength of association across lag time that would have indicated more recent stress had a stronger effect on craving.

Fig. 5.

Fig. 5.

Results from time-varying effects models (TVEM) of the time-lagged effects of stress on craving in the High-Use, Sporadic and Negative clusters. The regression coefficients model the current craving level as a function of the stress level in the previous report, with the coefficients allowed to vary depending on the amount of time since the previous report. A: The intercept coefficient represents the mean craving level following a report in which the stress level had been reported to be minimal (i.e., “Not at all”). B: The slope coefficient represents the association between the previous stress level and the current craving level, with larger slope values indicating higher stress-craving reactivity. C: Model-derived estimates of craving levels as a function of the stress level in the previous report and the amount of time passed since the previous report. In all panels, zero is indicated on the time axis for reference purposes (indicating the time that stress was assessed), but none of the data represent a lag of zero. Shaded regions indicate 95% confidence bands.

Model-fitted levels of craving (Fig. 5C) show increased craving in all clusters as a function of the prior stress level, but with clear differences in the High-Use clusters compared to the Sporadic and Negative clusters. Following a stress level of “Not at all” in the previous report, the Negative cluster generally had lower craving than the Sporadic and High Use clusters. At successively higher levels of stress, from “A little” through “Extreme”, the High Use cluster exhibited increasingly higher spikes in craving, which gradually decreased over about a 90-minute period; after this period, craving in the High-Use clusters was statistically similar to craving in the other clusters. These results help to pinpoint a key time window for risk (90 minutes after more than minimal stress is reported) for a particular group of individuals (High Use).

Discussion

Identification of natural groups

The hierarchical clustering procedure identified five distinct patterns of opioid and cocaine use during treatment with methadone or buprenorphine. Earlier(Panlilio et al. 2020), when we applied cluster analysis to sequences of drug test results from three randomized clinical trials of contingency-management, we identified Opioid, Cocaine, Both, and Negative clusters, but not a sporadic-use cluster. The fact that a sporadic pattern was more prominent in the present natural-history study than in the randomized trials could be due to differences in the sample populations, since by design the randomized trials only included participants who used both opioids and cocaine frequently prior to intervention; it could also be due to differences in how the clustering was conducted. Specifically, the present clustering procedure was applied to sequences of individual test results, but the earlier procedure was applied to test results aggregated by week. Thus, more subtle differences in pattern (e.g., sporadic use) were detected at a finer-grained level of analysis.

In the earlier report (Panlilio et al. 2020), we showed differences between use clusters with respect to other kinds of outcome measures, indicating that lower-use patterns were associated with reduced symptoms of substance-use disorder, including craving. In the present study, participants with a sporadic pattern did not experience increased craving prior to a use event, and they did not exhibit the strong stress-craving reactivity seen in participants who used more frequently. These findings illustrate how (compared to a high-use pattern) a sporadic use pattern, detected at a relatively coarse temporal level by urine testing, may reflect differences in momentary responses (e.g., less reactivity to stress, but perhaps more reactivity to some other kind of event). Identifying such differences could potentially be used to personalize clinical approaches to managing drug use.

There are various procedures that can be used to identify patterns of drug use, and it is reassuring when different methods produce results that are in agreement; for example, when we compared hierarchical clustering with K-means clustering, we found that there was strong agreement between these methods (Panlilio et al. 2020). When Roos et al. (2019b) performed latent class analysis of cocaine use over an eight-week period of treatment, they identified a high-use cluster and a low-use cluster (using about 1.5 times per week) in addition to an a priori abstinent cluster; these roughly correspond to our Cocaine, Sporadic, and Negative clusters, respectively. In most previous studies of substance-use where hierarchical clustering was applied to drug-use data, the cluster procedure was applied to combined measures of drug use, symptom severity, and other psychosocial variables (Beitchman et al. 2009; Ding et al. 2018; Edwards et al. 2010; Magallon-Neri et al. 2012; Vida et al. 2009). To produce a more general measure that could facilitate comparisons between studies, we took a different approach by restricting our cluster analysis to only the sequence of drug-test results. Two previous studies took a hierarchical clustering approach similar to ours, but on a different time scale, with less frequent but more prolonged drug testing: about three times per month for up to 36 months (Magura et al. 1998) or every 3 months for 24 months (Dobler-Mikola et al. 2005). Dobler-Mikola et al. (2005) analyzed heroin, cocaine and alcohol separately; for each drug they selected a three-cluster solution they described as non-use, occasional use, and daily use, which correspond to our Negative, Sporadic, and High-Use clusters. Magura et al. (1998) identified clusters based on patterns of heroin and cocaine use in methadone-treated participants, but did so based on the proportion of drug-positive tests, with data collapsed into 3-month bins. Looking only at the first 3-month bin, they identified four clusters, which correspond to our Negative, Opioid, Cocaine and Dual-Use clusters. When drug tests from all 36 months were analyzed, three other clusters appeared, including one with “fluctuating patterns of heroin and cocaine use” (i.e., with intermediate levels of use during the first 3 months) that loosely corresponds to our Sporadic cluster, plus two that involved shifting over time from cocaine use to heroin use or from heroin use to cocaine use. The latter clusters confirm that this technique can detect patterns of increasing or decreasing use, even though such patterns were not common in the present study and therefore were not a distinguishing feature of any of our clusters.

Characteristics of the clusters

Analysis of cluster characteristics revealed informative differences between the clusters with regard to: 1) baseline demographics, including age and race; 2) average levels of positive mood, stress, and craving in EMA reports; and 3) dynamics of stress and craving (i.e., changes in craving prior to drug use and the momentary relationship between stress and craving). Some of these characteristics could potentially be useful for early identification of patients who would benefit from treatment of stress-related symptoms (Kowalczyk et al. 2017), or for ongoing detection of increases in the likelihood of lapse based on real-time monitoring of EMA data (Marsch 2020; Scott et al. 2018).

Treatment with the full opioid agonist methadone was associated with being in the Cocaine cluster, and treatment with the partial opioid agonist buprenorphine was associated with being in the Opioid cluster. The Dual-Use cluster was evenly split between methadone and buprenorphine recipients. Methadone and buprenorphine can decrease cocaine use in some patients, for example in those who use cocaine less frequently at baseline (Roux et al. 2016), but methadone has been associated with relatively high levels of cocaine use in several previous studies (Gastberger et al. 2019; Giacomuzzi et al. 2003; Montoya et al. 2004; Schottenfeld et al. 1993; Schottenfeld et al. 1997; Silverman et al. 1998). Stine and Kosten (1994) suggested that cocaine might increase opioid withdrawal symptoms in people receiving buprenorphine. Methadone can enhance the subjective effects of cocaine (Preston et al. 1996), and buprenorphine might also have this effect, to a lesser extent (Strain et al. 2010). In rhesus monkeys, intravenous cocaine self-administration was decreased by buprenorphine (Mello et al. 1993) but was either enhanced (Wang et al. 2001)or not affected (Negus and Mello 2004) by methadone. Thus, there is evidence that methadone is more compatible with cocaine use (i.e., making cocaine more rewarding or less aversive), and this might contribute to the association between methadone treatment and the Cocaine cluster.

In the present study, participants were recruited if they were seeking treatment for opioid use disorder, regardless of whether they engaged in cocaine use. Since the pharmacotherapy (methadone versus buprenorphine) was not randomly assigned, a causal direction cannot be ascribed to the associations between medication type and drug-use clusters in the present study. Thus, it is possible that methadone is more compatible with cocaine use, and it is also possible that participants who used cocaine are more likely to initiate and/or maintain treatment with methadone. Our findings are also consistent with evidence that buprenorphine can lower cocaine use in those with current or previous opioid use disorder (Ling et al. 2016).

The finding that African Americans were overrepresented in the Dual-Use and Cocaine clusters is consistent with earlier findings (Hartel et al. 1995), but it does not support a simple conclusion that African Americans being treated for opioid use disorder are generally more likely to use cocaine during treatment or to have worse overall outcomes (in fact they were underrepresented in the Dropout cluster, i.e., disproportionately likely to stay in treatment). As with most findings of a difference by race (or sex), these findings may be informative at the population level, but probably should not be taken as starting points toward precision medicine. This is not simply because they are imperfect predictors at the level of individual patients, but because they carry cultural weight, with accompanying risk of ecological-fallacy-related harm. The prognostic use of racial categories is already being reconsidered in other biomedical contexts where it was, until very recently, considered useful (Norris et al. 2021).

On average, participants in the Dual-Use cluster were older and had longer histories of heroin, cocaine, and polydrug use. Recent psychological/emotional problems were identified as contributing to dropout in the present analysis and also in an earlier analysis (Panlilio et al. 2019) of data from the present study, but without the Treatment Elsewhere arm and with dropping out being defined based on information beyond the amount of missing data. People in most of the clusters in the present analysis reported drug-related problems on about 2 out of 3 days during the month prior to starting the study, but there were slightly fewer drug-related problems in the Negative cluster, which also reported substantially less drug use during the same time period. Thus, to some extent, participants’ patterns of use during the study may have been related to patterns existing prior to the study, consistent with previous findings that baseline severity of symptoms predicts outcomes for mental health problems in general (Friedman et al. 2012; Kennard et al. 2018), and that baseline frequency of use in particular predicts treatment outcome for substance use disorders (Adamson et al. 2009; Ahmadi et al. 2006; Laffaye et al. 2008; Preston et al. 1998; Roos et al. 2019b).

Stress, craving, and other influences

Craving is important as an unpleasant symptom (FDA 2018; Tiffany and Wray 2012) and also as a potential trigger of drug use. Laboratory studies show that stress can induce craving (Fox et al. 2008; Jobes et al. 2011; Sinha 2009; Sinha et al. 2003), and EMA studies show that stress and craving are correlated when assessed within the same EMA report (Moran et al. 2018; Neupert et al. 2017; Preston and Epstein 2011; Preston et al. 2017; 2018b). However, the evidence for a temporal order of effects (e.g., stress followed by craving followed by use) has been less clear (Fronk et al. 2020; Preston and Epstein 2011; Preston et al. 2018a; Preston et al. 2009). The present results suggest that this lack of clarity could be at least partly due to the stress/craving/use relationship being obscured by individual differences in whole populations, and that it can be revealed within clusters of people who share similar patterns of drug use.

When they entered the study, participants in the Negative cluster reported less self-identification with statements describing the use of drugs to cope with stress, and during the study they reported lower levels of craving. However, despite their differences in the frequency of use, the High-Use, Sporadic and Negative clusters all attributed about 20% of their drug-use events to stress when they were asked which reasons triggered specific drug-use events during the study. Averaged over the course of the study, all clusters reported about the same amount of stress, but the High-Use clusters reported more craving, more cue exposure, more enjoyment from drug taking, and less experience of positive mood compared to the Negative cluster. Most importantly with respect to the goal of understanding the effects of stress on craving, the High-Use clusters showed strong reactivity to stress, in the form of sharply higher levels of craving during the 90 minutes after they experienced moderate to extreme stress. The Sporadic and Negative clusters also experienced increases in craving after they were stressed, but not the sharp, time-sensitive increases seen in the High-Use clusters.

Perhaps the most striking finding regarding the Sporadic cluster is that they were more likely than the others to attribute drug-use events to being asked to use by someone else. This trigger was also reported occasionally in the High-Use clusters, but rarely in the Negative cluster. It is not clear if participants in the Negative cluster were asked to use less frequently or if they were less responsive to being asked, but this finding suggests that understanding more about how social interactions (Pelloux et al. 2019) and social networks (Latkin et al. 1999; Linton et al. 2016) can deter or facilitate drug use might be especially important in those who use sporadically.

There are several potential limitations or drawbacks of the study and analysis. The present analyses are consistent with the aims the study was designed to achieve, but they were not formally planned and should therefore be considered exploratory. Sampling of participants and assignment of medication (buprenorphine versus methadone) were not random. However, the sample was broadly representative of people with opioid-use disorder in the Baltimore area, and the selection of medication was naturalistic in that people who have a preference for a specific medication generally seek a treatment program that offers the medication they want. The EMA and questionnaire data consist of self-report, which can potentially be unreliable but still offers the best window into people’s personal experiences. The EMA procedure and drug testing schedule were demanding, so the results might not apply to people who would not join or complete this kind of study. Although we focused on craving in this study, about 8% of participants reported no craving during the study, and our clinical experience indicates that certain people with opioid or cocaine use disorders explicitly state that they never experience craving. It is unclear whether the small to medium-sized effects reported here would be useful for prognostic or treatment-tailoring purposes.

Despite these limitations, we believe that our findings provide valid and useful insights into behavioral processes that are not easily accessible by any other method. The cluster-based findings described here extend our previous reports that showed relationships between stress, craving, and drug use in the sample as a whole (Furnari et al. 2015; Preston et al. 2017; 2018a; b); the present results clarify that these relationships differ between certain subgroups of patients who share specific patterns of drug use. We believe the results presented here and in our earlier paper (Panlilio et al. 2020) demonstrate the utility of cluster analysis for achieving two central aims of addiction research: obtaining a qualitative measure of treatment success, and identifying prototypical patterns of behavior. This approach is especially useful when combined with intensive longitudinal measures, but we suggest that it would be a valuable tool in any analysis that involves sequences of drug use.

Supplementary Material

1672037_Supp_FigS1-S4

Acknowledgments

Funding and Disclosure

This study was supported by the Intramural Research Program of NIH, NIDA. SWS was supported by National Science Foundation grant DGE1255832. STL was supported by National Institute on Drug Abuse grant NIDA P50-DA039838. The authors report no conflicts of interest.

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of a an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

References

  1. Adamson SJ, Sellman JD, Frampton CM (2009) Patient predictors of alcohol treatment outcome: a systematic review. J Subst Abuse Treat 36: 75–86. [DOI] [PubMed] [Google Scholar]
  2. Ahmadi J, Kampman K, Dackis C (2006) Outcome predictors in cocaine dependence treatment trials. The American journal on addictions 15: 434–9. [DOI] [PubMed] [Google Scholar]
  3. Bates D, Mächler M, Bolker B, Walker S (2015) Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67. [Google Scholar]
  4. Beitchman JH, Adlaf EM, Douglas L, Atkinson L, Young A, Johnson CJ, Escobar M, Wilson B (2009) Comorbidity of Psychiatric and Substance Use Disorders in Late Adolescence: A Cluster Analytic Approach. The American Journal of Drug and Alcohol Abuse 27: 421–440. [DOI] [PubMed] [Google Scholar]
  5. Bolger N, Laurenceau JP (2013) Intensive Longitudinal Methods: An Introduction to Diary and Experience Sampling Research. Guilford Press, New York [Google Scholar]
  6. Carver CS, Scheier MF, Weintraub JK (1989) Assessing coping strategies: a theoretically based approach. J Pers Soc Psychol 56: 267–83. [DOI] [PubMed] [Google Scholar]
  7. Ding X, Salmeron BJ, Wang J, Yang Y, Stein EA, Ross TJ (2018) Evidence of subgroups in smokers as revealed in clinical measures and evaluated by neuroimaging data: a preliminary study. Addiction Biology 24: 777–786. [DOI] [PubMed] [Google Scholar]
  8. Dobler-Mikola A, Hattenschwiler J, Meili D, Beck T, Boni E, Modestin J (2005) Patterns of heroin, cocaine, and alcohol abuse during long-term methadone maintenance treatment. J Subst Abuse Treat 29: 259–65. [DOI] [PubMed] [Google Scholar]
  9. Edwards SA, Bondy SJ, Kowgier M, McDonald PW, Cohen JE (2010) Are occasional smokers a heterogeneous group? An exploratory study. Nicotine Tob Res 12: 1195–202. [DOI] [PubMed] [Google Scholar]
  10. Epstein DH, Tyburski M, Craig IM, Phillips KA, Jobes ML, Vahabzadeh M, Mezghanni M, Lin JL, Furr-Holden CDM, Preston KL (2014) Real-time tracking of neighborhood surroundings and mood in urban drug misusers: application of a new method to study behavior in its geographical context. Drug Alcohol Depend 134: 22–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Epstein DH, Willner-Reid J, Vahabzadeh M, Mezghanni M, Lin JL, Preston KL (2009) Real-time electronic diary reports of cue exposure and mood in the hours before cocaine and heroin craving and use. Archives of general psychiatry 66: 88–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. FDA (2018) Public meeting on patent-focused drug development for opioid use disorder. https://wwwfdagov/industry/prescription-drug-user-fee-amendments/fda-led-patient-focused-drug-development-pfdd-public-meetings#opioid
  13. Fox HC, Hong KI, Siedlarz K, Sinha R (2008) Enhanced sensitivity to stress and drug/alcohol craving in abstinent cocaine-dependent individuals compared to social drinkers. Neuropsychopharmacology 33: 796–805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fox J, Weisberg S (2019) An R Companion to Applied Regression. Sage, Thousand Oaks CA [Google Scholar]
  15. Friedman ES, Davis LL, Zisook S, Wisniewski SR, Trivedi MH, Fava M, Rush AJ (2012) Baseline depression severity as a predictor of single and combination antidepressant treatment outcome: results from the CO-MED trial. European neuropsychopharmacology : the journal of the European College of Neuropsychopharmacology 22: 183–99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Fronk GE, Sant’Ana SJ, Kaye JT, Curtin JJ (2020) Stress Allostasis in Substance Use Disorders: Promise, Progress, and Emerging Priorities in Clinical Research. Annu Rev Clin Psychol 16: 401–430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Furnari M, Epstein DH, Phillips KA, Jobes ML, Kowalczyk WJ, Vahabzadeh M, Lin JL, Preston KL (2015) Some of the people, some of the time: field evidence for associations and dissociations between stress and drug use. Psychopharmacology 232: 3529–37. [DOI] [PubMed] [Google Scholar]
  18. Gabadinho A G, Ritschard NS, lier M, Studer M (2011) Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software 40: 1–37. [Google Scholar]
  19. Gastberger S, Baumgartner MR, Soyka M, Quednow BB, Hulka LM, Herdener M, Seifritz E, Mutschler J (2019) Concomitant Heroin and Cocaine Use among Opioid-Dependent Patients during Methadone, Buprenorphine or Morphine Opioid Agonist Therapy. European addiction research 25: 207–212. [DOI] [PubMed] [Google Scholar]
  20. Gaus W (2015) Interpretation of Statistical Significance - Exploratory Versus Confirmative Testing in Clinical Trials, Epidemiological Studies, Meta-Analyses and Toxicological Screening (Using Ginkgo biloba as an Example). Clinical & Experimental Pharmacology 05. [Google Scholar]
  21. Giacomuzzi SM, Riemer Y, Ertl M, Kemmler G, Rössler H, Hinterhuber H, Kurz M (2003) Buprenorphine versus methadone maintenance treatment in an ambulant setting: a health-related quality of life assessment. Addiction 98: 693–702. [DOI] [PubMed] [Google Scholar]
  22. Ginexi EM, Riley W, Atienza AA, Mabry PL (2014) The promise of intensive longitudinal data capture for behavioral health research. Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco 16 Suppl 2: S73–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hartel DM, Schoenbaum EE, Selwyn PA, Kline J, Davenny K, Klein RS, Friedland GH (1995) Heroin use during methadone maintenance treatment: the importance of methadone dose and cocaine use. American journal of public health 85: 83–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hassanbeigi A, Askari J, Hassanbeigi D, Pourmovahed Z (2013) The Relationship between Stress and Addiction. Procedia - Social and Behavioral Sciences 84: 1333–1340. [Google Scholar]
  25. Hastie T, Tibshirani R, Friedman J (2017) Unsupervised Learning. In: Diggle P, Gather U, Zeger S (eds) The elements of statistical learning: data mining, inference, and prediction. Springer, New York, pp 485–585 [Google Scholar]
  26. Hennig C (2015) What are the true clusters? Pattern Recognition Letters 64: 53–62. [Google Scholar]
  27. James G, Witten D, Hastie T, Tibshirani R (2013) Unsupervised learning. In: Casella G, Fienberg S, Olkin I (eds) An introduction to statistical learning - with applications in R. Springer, New York, pp 373–418 [Google Scholar]
  28. Jobes ML, Ghitza UE, Epstein DH, Phillips KA, Heishman SJ, Preston KL (2011) Clonidine blocks stress-induced craving in cocaine users. Psychopharmacology (Berl) 218: 83–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kennard BD, Mayes TL, Chahal Z, Nakonezny PA, Moorehead A, Emslie GJ (2018) Predictors and Moderators of Relapse in Children and Adolescents With Major Depressive Disorder. The Journal of clinical psychiatry 79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kiluk BD, Babuscio TA, Nich C, Carroll KM (2017) Initial validation of a proxy indicator of functioning as a potential tool for establishing a clinically meaningful cocaine use outcome. Drug and alcohol dependence 179: 400–407. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kowalczyk WJ, Bertz JW, Moran LM, Phillips KA, Ghitza UE, Epstein DH, Preston KL (2017) Clonidine Increases the Likelihood That Abstinence Can Withstand Unstructured Time in Buprenorphine-maintained Outpatients. J Addict Med 11: 454–460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Laffaye C, McKellar JD, Ilgen MA, Moos RH (2008) Predictors of 4-year outcome of community residential treatment for patients with substance use disorders. Addiction 103: 671–80. [DOI] [PubMed] [Google Scholar]
  33. Lanza ST, Vasilenko SA, Liu X, Li R, Piper ME (2014) Advancing the understanding of craving during smoking cessation attempts: a demonstration of the time-varying effect model. Nicotine Tob Res 16 Suppl 2: S127–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Latkin CA, Knowlton AR, Hoover D, Mandell W (1999) Drug network characteristics as a predictor of cessation of drug use among adult injection drug users: a prospective study. Am J Drug Alcohol Abuse 25: 463–73. [DOI] [PubMed] [Google Scholar]
  35. Li R, Dziak JD, Tan X, Huang L, Wagner AT, Yang J (2017) TVEM (time-varying effect modeling) SAS macro users’ guide (Version 3.1.1) http://methodology.psu.edu. The Methodology Center, Pennsylvania State University, University Park, PA [Google Scholar]
  36. Ling W, Hillhouse MP, Saxon AJ, Mooney LJ, Thomas CM, Ang A, Matthews AG, Hasson A, Annon J, Sparenborg S, Liu DS, McCormack J, Church S, Swafford W, Drexler K, Schuman C, Ross S, Wiest K, Korthuis PT, Lawson W, Brigham GS, Knox PC, Dawes M, Rotrosen J (2016) Buprenorphine + naloxone plus naltrexone for the treatment of cocaine dependence: the Cocaine Use Reduction with Buprenorphine (CURB) study. Addiction 111: 1416–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Linton SL, Cooper HL, Luo R, Karnes C, Renneker K, Haley DF, Hunter-Jones J, Ross Z, Bonney L, Rothenberg R (2016) People and places: Relocating to neighborhoods with better economic and social conditions is associated with less risky drug/alcohol network characteristics among African American adults in Atlanta, GA. Drug and alcohol dependence 160: 30–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lukasiewicz M, Fareng M, Benyamina A, Blecha L, Reynaud M, Falissard B (2007) Ecological momentary assessment in addiction. Expert review of neurotherapeutics 7: 939–50. [DOI] [PubMed] [Google Scholar]
  39. Maechler M, Rousseeuw P, Struyf A, Hubert M, Hornik K (2018) cluster: Cluster Analysis Basics and Extensions, R package version 2.0.7–1 [Google Scholar]
  40. Magallon-Neri E, Diaz R, Forns M, Goti J, Canalda G, Castro-Fornieles J (2012) Subtypes of adolescents with substance use disorders and psychiatric comorbidity using cluster and discriminant analysis of MMPI-A profiles. Adicciones 24: 219–27. [PubMed] [Google Scholar]
  41. Magura S, Kang SY, Nwakeze PC, Demsky S (1998) Temporal patterns of heroin and cocaine use among methadone patients. Subst Use Misuse 33: 2441–67. [DOI] [PubMed] [Google Scholar]
  42. Marsch LA (2020) Digital health data-driven approaches to understand human behavior. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology: 1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Maryland Department of Health (2018) Unintentional Drug- and Alcohol-Related Intoxication Deaths in Maryland Annual Report 2017. https://bhahealthmarylandgov/OVERDOSE_PREVENTION/Documents/Drug_Intox_Report_2017pdf.
  44. McLellan AT, Luborsky L, Cacciola J, Griffith J, Evans F, Barr HL, O’Brien CP (1985) New data from the Addiction Severity Index. Reliability and validity in three centers. The Journal of nervous and mental disease 173: 412–23. [DOI] [PubMed] [Google Scholar]
  45. Mello NK, Lukas SE, Mendelson JH, Drieze J (1993) Naltrexone-buprenorphine interactions: effects on cocaine self-administration. Neuropsychopharmacology : official publication of the American College of Neuropsychopharmacology 9: 211–24. [DOI] [PubMed] [Google Scholar]
  46. Milligan GW, Cooper MC (1987) Methodology Review: Clustering Methods. Applied Psychological Measurement 11: 329–354. [Google Scholar]
  47. Montoya ID, Gorelick DA, Preston KL, Schroeder JR, Umbricht A, Cheskin LJ, Lange WR, Contoreggi C, Johnson RE, Fudala PJ (2004) Randomized trial of buprenorphine for treatment of concurrent opiate and cocaine dependence. Clinical pharmacology and therapeutics 75: 34–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Moran LM, Kowalczyk WJ, Phillips KA, Vahabzadeh M, Lin JL, Mezghanni M, Epstein DH, Preston KL (2018) Sex differences in daily life stress and craving in opioid-dependent patients. Am J Drug Alcohol Abuse 44: 512–523. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Negus SS, Mello NK (2004) Effects of chronic methadone treatment on cocaine- and food-maintained responding under second-order, progressive-ratio and concurrent-choice schedules in rhesus monkeys. Drug Alcohol Depend 74: 297–309. [DOI] [PubMed] [Google Scholar]
  50. Neupert SD, Desmarais SL, Gray JS, Cohn AM, Doherty S, Knight K (2017) Daily stressors as antecedents, correlates, and consequences of alcohol and drug use and cravings in community-based offenders. Psychol Addict Behav 31: 315–325. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Norris KC, Eneanya ND, Boulware LE (2021) Removal of Race From Estimates of Kidney Function: First, Do No Harm. JAMA 325: 135–137. [DOI] [PubMed] [Google Scholar]
  52. Panlilio LV, Stull SW, Bertz JW, Burgess-Hull AJ, Kowalczyk WJ, Phillips KA, Epstein DH, Preston KL (2020) Beyond abstinence and relapse: cluster analysis of drug-use patterns during treatment as an outcome measure for clinical trials. Psychopharmacology 237: 3369–3381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Panlilio LV, Stull SW, Kowalczyk WJ, Phillips KA, Schroeder JR, Bertz JW, Vahabzadeh M, Lin J-L, Mezghanni M, Nunes EV, Epstein DH, Preston KL (2019) Stress, craving and mood as predictors of early dropout from opioid agonist therapy. Drug and Alcohol Dependence 202: 200–208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Pelloux Y, Giorla E, Montanari C, Baunez C (2019) Social modulation of drug use and drug addiction. Neuropharmacology 159: 107545. [DOI] [PubMed] [Google Scholar]
  55. Preston KL, Epstein DH (2011) Stress in the daily lives of cocaine and heroin users: relationship to mood, craving, relapse triggers, and cocaine use. Psychopharmacology (Berl) 218: 29–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Preston KL, Kowalczyk WJ, Phillips KA, Jobes ML, Vahabzadeh M, Lin JL, Mezghanni M, Epstein DH (2017) Context and craving during stressful events in the daily lives of drug-dependent patients. Psychopharmacology (Berl) 234: 2631–2642. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Preston KL, Kowalczyk WJ, Phillips KA, Jobes ML, Vahabzadeh M, Lin JL, Mezghanni M, Epstein DH (2018a) Before and after: craving, mood, and background stress in the hours surrounding drug use and stressful events in patients with opioid-use disorder. Psychopharmacology (Berl) 235: 2713–2723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Preston KL, Kowalczyk WJ, Phillips KA, Jobes ML, Vahabzadeh M, Lin JL, Mezghanni M, Epstein DH (2018b) Exacerbated Craving in the Presence of Stress and Drug Cues in Drug-Dependent Patients. Neuropsychopharmacology 43: 859–867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Preston KL, Schroeder JR, Kowalczyk WJ, Phillips KA, Jobes ML, Vahabzadeh M, Lin JL, Mezghanni M, Epstein DH (2018c) End-of-day reports of daily hassles and stress in men and women with opioid-use disorder: Relationship to momentary reports of opioid and cocaine use and stress. Drug and Alcohol Dependence doi: 10.1016/j.drugalcdep.2018.08.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Preston KL, Silverman K, Higgins ST, Brooner RK, Montoya I, Schuster CR, Cone EJ (1998) Cocaine use early in treatment predicts outcome in a behavioral treatment program. J Consult Clin Psychol 66: 691–6. [DOI] [PubMed] [Google Scholar]
  61. Preston KL, Sullivan JT, Strain EC, Bigelow GE (1996) Enhancement of cocaine’s abuse liability in methadone maintenance patients. Psychopharmacology 123: 15–25. [DOI] [PubMed] [Google Scholar]
  62. Preston KL, Vahabzadeh M, Schmittner J, Lin JL, Gorelick DA, Epstein DH (2009) Cocaine craving and use during daily life. Psychopharmacology (Berl) 207: 291–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. R Core Team (2018) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria [Google Scholar]
  64. Roos CR, Nich C, Mun CJ, Babuscio TA, Mendonca J, Miguel AQC, DeVito EE, Yip SW, Witkiewitz K, Carroll KM, Kiluk BD (2019a) Clinical validation of reduction in cocaine frequency level as an endpoint in clinical trials for cocaine use disorder. Drug and alcohol dependence 205: 107648. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Roos CR, Nich C, Mun CJ, Mendonca J, Babuscio TA, Witkiewitz K, Carroll KM, Kiluk BD (2019b) Patterns of Cocaine Use During Treatment: Associations With Baseline Characteristics and Follow-Up Functioning. Journal of studies on alcohol and drugs 80: 431–440. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Rosnow RL, Rosenthal R (1996) Computing contrasts, effect sizes, and counternulls on other people’s published data: General procedures for research consumers Psychological Methods 1: 331–340. [Google Scholar]
  67. Rosnow RL, Rosenthal R, Rubin DB (2000) Contrasts and correlations in effect-size estimation. Psychol Sci 11: 446–53. [DOI] [PubMed] [Google Scholar]
  68. Roux P, Lions C, Vilotitch A, Michel L, Mora M, Maradan G, Marcellin F, Spire B, Morel A, Carrieri PM, group AMs (2016) Correlates of cocaine use during methadone treatment: implications for screening and clinical management (ANRS Methaville study). Harm Reduct J 13: 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Rubin M (2017) Do p Values Lose Their Meaning in Exploratory Analyses? It Depends How You Define the Familywise Error Rate. Review of General Psychology 21: 269–275. [Google Scholar]
  70. Schottenfeld RS, Pakes J, Ziedonis D, Kosten TR (1993) Buprenorphine: dose-related effects on cocaine and opioid use in cocaine-abusing opioid-dependent humans. Biological psychiatry 34: 66–74. [DOI] [PubMed] [Google Scholar]
  71. Schottenfeld RS, Pakes JR, Oliveto A, Ziedonis D, Kosten TR (1997) Buprenorphine vs methadone maintenance treatment for concurrent opioid dependence and cocaine abuse. Arch Gen Psychiatry 54: 713–20. [DOI] [PubMed] [Google Scholar]
  72. Scott CK, Dennis ML, Gustafson DH (2018) Using ecological momentary assessments to predict relapse after adult substance use treatment. Addictive behaviors 83: 116–122. [DOI] [PubMed] [Google Scholar]
  73. Shiffman S, Stone AA, Hufford MR (2008) Ecological momentary assessment. Annual review of clinical psychology 4: 1–32. [DOI] [PubMed] [Google Scholar]
  74. Shiyko M, Naab P, Shiffman S, Li R (2014) Modeling complexity of EMA data: time-varying lagged effects of negative affect on smoking urges for subgroups of nicotine addiction. Nicotine Tob Res 16 Suppl 2: S144–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Silverman K, Bigelow GE, Stitzer ML (1998) 15 - Treatment of Cocaine Abuse in Methadone Maintenance Patients. In: Higgins ST, Katz JL (eds) Cocaine Abuse. Academic Press, San Diego, pp 363–388 [Google Scholar]
  76. Sinha R (2009) Modeling stress and drug craving in the laboratory: implications for addiction treatment development. Addict Biol 14: 84–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Sinha R, Talih M, Malison R, Cooney N, Anderson GM, Kreek MJ (2003) Hypothalamic-pituitary-adrenal axis and sympatho-adreno-medullary responses during stress-induced and drug cue-induced cocaine craving states. Psychopharmacology (Berl) 170: 62–72. [DOI] [PubMed] [Google Scholar]
  78. Stine SM, Kosten TR (1994) Reduction of opiate withdrawal-like symptoms by cocaine abuse during methadone and buprenorphine maintenance. The American journal of drug and alcohol abuse 20: 445–58. [DOI] [PubMed] [Google Scholar]
  79. Strain EC, Preston KL, Stitzer ML, Liebson IA, Bigelow GE (2010) The Effects of Cocaine in Buprenorphine - Maintained Outpatient Volunteers. The American Journal on Addictions 3: 129–143. [Google Scholar]
  80. Studer M, Ritschard G (2016) What matters in differences between life trajectories: a comparative review of sequence dissimilarity measures. Journal of the Royal Statistical Society: Series A (Statistics in Society) 179: 481–511. [Google Scholar]
  81. Tan X, Shiyko MP, Li R, Li Y, Dierker L (2012) A time-varying effect model for intensive longitudinal data. Psychological methods 17: 61–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Tiffany ST, Wray JM (2012) The clinical significance of drug craving. Annals of the New York Academy of Sciences 1248: 1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  83. Vida R, Brownlie EB, Beitchman JH, Adlaf EM, Atkinson L, Escobar M, Johnson CJ, Jiang H, Koyama E, Bender D (2009) Emerging adult outcomes of adolescent psychiatric and substance use disorders. Addictive behaviors 34: 800–5. [DOI] [PubMed] [Google Scholar]
  84. Wang NS, Brown VL, Grabowski J, Meisch RA (2001) Reinforcement by orally delivered methadone, cocaine, and methadone-cocaine combinations in rhesus monkeys: are the combinations better reinforcers? Psychopharmacology 156: 63–72. [DOI] [PubMed] [Google Scholar]
  85. Wasserstein RL, Lazar NA (2016) The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician 70: 129–133. [Google Scholar]
  86. Wickham H (2009) ggplot2: elegant graphics for data analysis. Springer, New York [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1672037_Supp_FigS1-S4

RESOURCES