Validation of the Opioid Overdose Risk Behavior Scale, Version 2 (ORBS-2)

Abstract

Objective:

To examine the factor structure of a revised and expanded opioid overdose risk behavior scale and assess its associations with known overdose indicators and other clinical constructs.

Background:

Opioid-related overdose remains high in the U.S. We lack strong instrumentation for assessing behavioral risk factors. We revised and expanded the opioid overdose risk behavior scale (ORBS-1) for use among a broader range of people who use opioids.

Setting & Sampling Frame:

Using respondent-driven sampling we recruited adults (18+) reporting current unprescribed opioid use and New York City residence.

Method:

Participants (N=575) completed the ORBS-1, ORBS-2, and a variety of clinical measures and then completed the ORBS-2 and overdose risk outcomes across monthly follow-up assessments over a 13-month period.

Results:

Principal components analysis was used to identify six ORBS-2 subscales, Prescription Opioid Misuse, Risky Non-Injection Use, Injection Drug Use, Concurrent Opioid and Benzodiazepine Use, Concurrent Opioid and Alcohol Use, and Multiple-Drug Polysubstance Use. All subscales showed moderate non-parametric correlations with the ORBS-1 and with corresponding clinical constructs. Five of the subscales were significantly (p < .01) positively associated with self-reported non-fatal overdose. Of note, the Risky Non-Injection Use subscale was the most strongly associated with past-month overdose indicators.

Conclusions:

Psychometrics for the opioid overdose risk behavior subscales identified suggest the ongoing utility of risk behavioral instrumentation for epidemiological research and clinical practice focused on risk communication and minimization. Use of the entire ORBS-2 measure can provide insight into the proximal/behavioral factors of greatest concern to reduce overdose mortality.

1. INTRODUCTION

Opioid-related morbidity mortality remains at epidemic levels in the United States (U.S.), and has been exacerbated by the COVID-19 pandemic (Centers for Disease Control and Prevention (CDC), 2020). Understanding the unique behavioral factors involved in these overdoses (ODs) and how those behaviors may vary geographically or by other factors is consequently an important form of epidemiological surveillance. For example, the route of administration and specific opioids involved in overdoses often differs by age group, race and geographic region (Daniulaityte et al., 2019; National Institute on Drug Abuse, 2020; Unick and Ciccarone, 2017). Despite the public health advantages represented by comprehensive tools to evaluate proximal, behavioral determinants of OD, instrumentation designed to measure opioid-related risks has remained largely in the domain of clinical tools for opioid-prescribing physicians.

A number of screener instruments emerged to assist physicians in assessing the risk of opioid misuse among patients (Butler et al., 2007; Butler et al., 2008; Chinman et al., 2019; Finkelman et al., 2017; Lindley et al., 2019; Rogers et al., 2020). These measures often included a range of proximal (e.g., current use of benzodiazepines and/or alcohol) and distal (e.g., family histories of substance abuse disorder) risk factors for misuse. Evidence that these measures can reliably predict misuse or overdose hazard is weak, and their sole focus on prescription opioids severely limits their utility for researchers working with populations of people who use illicit opioids.

Between 2014 and 2017, a number of the authors of this article developed instrumentation to provide a bridge between clinical and informal community settings for opioid distribution and use. The resulting instrument, the Opioid Overdose Risk Behavior Scale (ORBS; Pouget et al., 2017b) includes an extensive array of questions related to prescription opioid misuse, as well as subscales addressing heroin and contraindicated polysubstance use. Designed in the context of a study focused on OD risk and prevention among U.S. military veterans who use opioids, the instrument served as an important outcome measure for several analyses of cross-sectional and longitudinal covariates of OD risk severity (Bennett et al., 2019; Cleland et al., 2020).

The current analysis represents an extension of and modification to the original ORBS (henceforth, ORBS-1). Since the early 2000s, prescription opioid use, misuse and related OD mortality have begun to decline {Compton, 2016 #1;Schuler, 2021 #56;Pezalla, 2017 #57} while deaths related to heroin and fentanyls have shown steady increases (Daniulaityte et al., 2017; O’Donnell et al., 2017; Scholl et al., 2019; Wilson, 2020). Reflecting both the growing significance of illicitly manufactured opioids (much of which is apparently directed to counterfeit opioid medication and benzodiazepine pills (Centers for Disease Control and Prevention (CDC), 2015; Palmer, 2019; Pergolizzi Jr et al., 2018; Suzuki and El-Haddad, 2017; Vo et al., 2016)) and the extremely high prevalence of polysubstance ODs involving opioids (Barocas et al., 2019; Cicero et al., 2020), we sought to refine and broaden the ORBS to be more comprehensive in assessing illicit opioid use, including purchases of opioid pills from dealers or other unverified sources, and concurrent (specified as “within the same 6-hour period”) polysubstance use of opioids and substances contraindicated with opioids (including benzodiazepines, sleep medications, and alcohol). We rewrote items from ORBS-1 based on participant and staff feedback to create greater clarity or specificity, and we refined the specification of the OD risk construct, in part by removing a subscale about solitary opioid use, which is a risk factor for OD mortality but not—all other things being equal—for OD itself.

To assess the new measure, we used a data-driven approach to scale development that began with principal components analysis (PCA) to identify empirically cohesive subscales which we anticipated would have some theoretical or “face” validity, in that the behaviors grouped within a scale would be practically related. We then validated the ORBS-2 subscales by examining the strength of their associations with the original ORBS (Pouget et al., 2017b) and related behavioral risk factors for OD. With larger and more representative samples, these validation analyses present important opportunities to foster greater understanding of typologies of risk and where emergent clusters of risk behaviors (and their relative strength in predicting OD) may signal otherwise unrecognized need for targeted intervention.

2. METHOD

2.1. Participant Recruitment

Participants enrolled in the study were persons aged 18 or older who currently (defined as, within the 3 days prior to enrollment) reported risky opioid use (including heroin, fentanyls, and prescription opioids used without prescription). Self-report of opioid use was verified using a rapid urinalysis tool from BTNX that included 9 fentanyl-class drugs in addition to heroin/morphine, oxycodone, benzodiazepines, alcohol, amphetamines, oxycodone, marijuana, and methadone metabolites. Those reporting no unprescribed use of any opioid or testing negative for any opioid in urinalysis were not enrolled in the study. Recruitment followed a traditional respondent-driven sampling approach (Heckathorn, 1997; Heckathorn and Cameron, 2017) that used “coupons” to allow participants to refer as many as three of their opioid-using network members to the study. Ten “seeds” representing ethnic, gender, and geographical diversity were started, ultimately resulting in a sample of 575 participants with complete baseline data. Due to the COVID-19 pandemic, recruitment stopped in early March of 2020, roughly one week in advance of completing the target sample (N=600).

2.2. Data Collection

Data used in the current analyses include both the baseline assessment, a roughly 2-hour survey instrument, administered at time of enrollment, as well as monthly online follow-up surveys during the first 13 months of the study’s 24-month data collection. During those 13 months (April 2019 through the last week of April 2020) we used SMS/text-messaging and email reminders to contact participants every 30 days about their eligibility for a new follow-up survey. This allowed the earliest recruited participants to provide up to 12 waves of follow-up for up to 12.83 months after their initial recruitment.

The baseline assessment compensated eligible participants $60 for the approximate 2.5-hour visit, which included urinalysis and overdose prevention and naloxone training and provision. Monthly follow-up surveys compensated participants $20, transferred immediately upon completion of the surveys to their Clinical Trial Payer (CTPayer) Visa cards.

We employed several protocols to maximize retention. During enrollment, each participant completed a detailed locator form that included contact information for the participant and a number of family and peer relations whom the study could contact should the participant cease to respond to phone, SMS, and email notifications about monthly surveys. Staff directly contacted (via phone, SMS, and email) participants who were more than 60 days overdue for a follow-up. If participants’ phones were disconnected and no replies were received via email, participants’ family, peers, (including their referrers and referrals from RDS records) were also contacted.

All human subjects research protocols were approved by the NYU Grossman School of Medicine Institutional Review Board.

2.3. Measures

ORBS2 topics are listed in Table 2. All questions begin, “During how many days in the last 30 did you…” The ORBS2 is currently administered online as a 37-item instrument tailored to each individual’s use of substances selected from a screening question containing 11 opioid and opioid-contraindicated categories. Concurrent use of opioids and other contraindicated substances are generated based on the specific drugs selected from this list and progress from 2-drug to 5-drug combinations as indicated. Pictures of the substances are included to aid participants in identifying the polysubstance combinations being asked about and hypertext allows respondents to click or press on drug names in polysubstance questions to receive detailed definitions. For analysis, dichotomous “gateway” questions that led to more detailed, past-30-day questions for those responding in the affirmative were removed. Two items from the ORBS2 related to use of kratom and codeine/promethazine syrup were removed from analysis due to their almost complete lack of endorsement by participants. All questions are included in the full measure in Supplementary Materials, some of which may have greater utility in areas of the U.S. where use patterns differ (Grundmann, 2017; Miuli et al., 2020). Two further items, assessing return to use after a period of abstinence, were also removed due to a lack of consistency with the past 30-day response structure and to a lack of clear theoretical direction about scoring.

Table 2 -.

Overdose Risk Behavior Scale (ORBS-2) Subscales, Items, and Item-Properties

	Subscale Name (Bolded)				PCA Loadings
	Item Text (“In the last 30 days, how many days did you…”)	M (SD)	% endorsed	α / CITC	Continuous	Dichotomous
A	Prescription Opioid Misuse	0.96 (4.24)	7%	.70
1	use a prescribed opioid painkiller?	1.43 (5.99)	7%	.64	.90	.90
2	use more of your prescribed opioid painkillers than advised?	0.48 (3.27)	3%	.64	.90	.90
B	Risky Non-Injection Use	0.82 (2.64)	21%	.70
3	use opioid painkillers that you got from someone else who has a prescription or that you know came from a pharmacy?	1.35 (4.84)	15%	.58	.78	.79
4	use opioid painkillers that you got from a dealer or someone else who might not have a doctor’s prescription?	1.24 (4.52)	14%	.66	.77	.77
5	crush and snort opioid painkillers?	0.6 (3.41)	7%	.60	.81	.81
6	crush and smoke or vaporize opioid painkillers?	0.18 (1.9)	2%	.50	.64	.63
7	smoke or vaporize (aka chase the dragon) heroin?	0.73 (4.18)	5%	.15	.49	.48
C	Injection Drug Use & Speedballing	4.54 (5.86)	65%	.56
8	inject heroin?	7.59 (11.93)	35%	.36	.73	.73
9	inject any opioid other than heroin (crushed pills, methadone, or buprenorphine for example)?	0.93 (4.58)	6%	.20	.34	.40
10	use any opioids AND use a cocaine or amphetamine type stimulant within the same 6-hour period?	6.43 (9.7)	49%	.37	.63	.55
11	use any opioids AND a cocaine or amphetamine type stimulant at the same time (also known as a speedball)?	3.22 (7.79)	26%	.54	.82	.80
D	Opioid & Alcohol Combinations	3.08 (6.51)	35%	.83
12	use any opioids AND drink alcohol within the same 6-hour period?	4.44 (9)	34%	.81	.89	.88
13	use any opioids and drink 4 or more alcoholic beverages quickly within the same 6-hour period?	2.83 (7.55)	21%	.77	.83	.81
14	use any opioids and use a cocaine or amphetamine type stimulant AND drink alcohol within the same 6-hour period ?	1.99 (5.66)	20%	.57	.78	.80
E	Opioid & Benzodiazepine Combinations	2.11 (4.14)	39%	.64
15	use any opioids AND take benzos within the same 6-hour period?	5.25 (9.61)	38%	.48	.84	.84
16	use any opioids and take benzos AND use a cocaine or amphetamine type stimulant within the same 6-hour period?	1.77 (5.41)	19%	.64	.81	.82
17	use any opioids and take benzos AND drink alcohol within the same 6-hour period?	0.89 (3.65)	13%	.44	.54	.55
18	use any opioids and use benzos and use a cocaine or amphetamine type stimulant AND drink alcohol within the same 6-hour period?	0.54 (2.77)	8%	.50	.49	.49
F	Polysubstance Combinations	0.74 (2.62)	13%	.84
19	use any opioids AND take prescription sleeping pills (within the same 6-hour period)?	2.02 (6.8)	13%	.64	.72	.70
20	use any opioids and take a prescription sleeping pill AND use a cocaine or amphetamine type stimulant within the same 6-hour period?	0.98 (4.37)	8%	.77	.84	.82
21	use any opioids and take a sleeping pill and drink alcohol within the same 6-hour period?	0.52 (3.01)	5%	.55	.83	.82
22	use any opioids and take benzos AND take a prescription sleeping pill within the same 6-hour period?	1.07 (4.61)	9%	.68	.76	.76
23	use any opioids and use benzos and take a prescription sleeping pill and use a cocaine or amphetamine type stimulant within the same 6-hour period?	0.57 (3.25)	6%	.71	.79	.79
24	use any opioids and take a prescription sleeping pill and use a cocaine or amphetamine type stimulant AND drink alcohol within the same 6-hour period?	0.33 (2.23)	3%	.61	.87	.87
25	use any opioids and use benzos and take a prescription sleeping pill AND drink alcohol within the same 6-hour period?	0.23 (1.62)	3%	.60	.84	.85
26	use any opioids and use benzos and take a prescription sleeping pill and use a cocaine or amphetamine type stimulant AND drink alcohol within the same 6-hour period?	0.21 (1.57)	3%	.59	.84	.86

Note.

α = Cronbach’s alpha for all items in relevant subscale; I-T-C = Corrected Item-Total-Correlation of item with all other items in the subscale.

PCA = Loading of item on its respective factor during a principal components analysis.

Past-month indicators of non-fatal opioid-related OD were drawn from the Recent Overdose Experience Scale (RODES; Pouget et al., 2017a). The OD rates presented in Table 1 use the broadest question in the RODES metric, “In the past 30 days, on how many days do you think you overdosed on opioid (with or without alcohol or other drugs involved)?” Short mental health measures used to assess concurrent validity of ORBS2 were drawn from the DSM-5 Dimensional Cross-Cutting Symptom Assessment for Adult Patients (Narrow et al., 2013); alcohol and substance use disorder measures were drawn from DSM-5 short screening tools (National Institute on Alcohol Abuse and Alcoholism (NIAAA), 2016; Substance Abuse and Mental Health Services Administration (SAMHSA), 2016, respectively); and pain severity and interreference were assessed with the Brief Pain Inventory (Cleeland and Ryan, 1994; Tan et al., 2004).

Table 1.

Demographic Characteristics

		Overdose (OD) Events over Study
Characteristic	Full Sample (n=575)	No OD Events (n=351)	1+ OD Events (n=224)	Comparison
Category	M(SD) / n(%)	M(SD) / n(%)	M(SD) / n(%)	t-test OR χ2	p
Age	48.3 (11.3)	49.91 (10.76)	45.73 (11.59)	4.41	<.001
Gender				5.06	.080
Male	379 (66%)	232 (66%)	147 (66%)
Female	190 (33%)	118 (34%)	72 (32%)
Non-binary/Trans	6 (1%)	1 (0%)	5 (2%)
Ethnicity				0.49	.484
Non-Hispanic	344 (60%)	214 (61%)	130 (58%)
Hispanic/Latinx	231 (40%)	137 (39%)	94 (42%)
Race				3.18	.204
Black	247 (43%)	161 (46%)	86 (38%)
White	169 (29%)	99 (28%)	70 (31%)
Other	159 (28%)	91 (26%)	68 (30%)
Education				1.10	.294
High School or less	387 (67%)	242 (69%)	145 (65%)
At least some college	188 (32.7%)	109 (31%)	79 (35%)
Homeless Status				0.79	.373
Not Homeless	401 (70%)	240 (68%)	161 (72%)
Currently Homeless	174 (30%)	111 (32%)	63 (28%)
Opioid Overdose History				14.58	<.001
Never Overdosed	356 (62%)	239 (68%)	117 (52%)
1+ Overdose in Lifetime	219 (38%)	112 (32%)	107 (48%)
		23.68	22.42
Age of First Use	22.9 (8.8)	(8.99)	(8.62)	1.25	.210

2.4. Analytic Approach

The present analyses consisted of two separate phases, (1) scale development and (2) scale validation. During the development phase, we used item descriptives to remove items with such a low rate of endorsement that they would be unhelpful in most clinical settings. Then we conducted two rounds of PCA on two separate correlation matrices with oblimin rotation (to allow for correlated subscales), as PCA does not assume normality. The first matrix was created using Spearman’s rank correlations – a non-parametric correlation coefficient that is robust to mismatched distributions between variables – to examine the properties of these items when used continuously (e.g., for high-risk populations among which these scales will still be skewed). The second matrix was created by first dichotomizing the variables and then calculating separate Pearson correlation matrices to examine the items’ properties when dichotomized (e.g., if a researcher is working in a lower risk population or is working with smaller samples).

During the validation phase, we examined the performance of the full scale using non-parametric associations. Convergent validity was evaluated using Spearman’s rank correlations between the ORBS-2 subscales and the original ORBS-1subscales. Concurrent validity was evaluated using Spearman’s correlations between the ORBS-2 subscales and related clinical constructs. Predictive validity was evaluated using a simple two-level hierarchical linear model (HLM) predicting count of self-reported non-fatal overdose events each month on a Poisson distribution. Multilevel models such as HLM are advantageous as they differentiate between-person associations from within-person time-varying associations. At the between-person level of analysis, we entered baseline ORBS-2 scores as a predictor – thereby asking “Do individuals who have higher scores on a given ORBS-2 subscale at baseline experience a greater number of overdose events over the study?” At the within-person level of analysis, we centered individuals’ ORBS-2 scores about their own mean as a predictor of overdose events at the same wave (i.e., “time-varying covariation”)– thereby asking “Do individuals experience greater number of overdose events on months where they have a higher than usual ORBS-2 score?” Each subscale was standardized within-level in order to facilitate comparisons between each subscale’s predictive power. We then re-ran the models controlling for age, race, education, and history of overdose (all demographic factors that increased likelihood of an overdose event or were associated with likelihood of dropout).

3. RESULTS

3.1. Participant Characteristics

The baseline study sample was on average 48 years old and primarily non-white, with 43% reporting Black race and 28% other non-White race (Table 1). Over two-thirds had a high school education or less (67%) and 30% were currently homeless. On average, participants initiated opioid use at 23 years old and one-third had ever experienced an opioid overdose (38%). Table 1 also reports basic rates of OD experience at time of enrollment and over the approximately 13 months of longitudinal data collection from the date of first enrollment.

A total of 443 participants (77%) completed at least one follow-up assessment. Participants who provided follow-up typically completed M=4.59 (SD=3.08) additional assessments with their last follow-up typically M=5.85 months (SD=3.21) after their baseline assessment. Attrition analyses found that, compared to those providing at least one follow-up assessment, participants lost to follow-up were more likely to be older (M=51.84 yrs old vs. M=47.22 yrs old; t(573)=4.20; p<.01), black (59% vs. 38%; χ²(2)=18.43; p<.01), and less educated (75% high school or less vs. 65%; χ²(1)=4.61; p=.03). Attrition was not significantly related to age at first use, ethnicity, gender, income, homelessness status, or history of overdose (All p<.05).

Across the full sample of 575 participants, a total of 224 respondents (39%) reported an overdose event during at least one wave. Among the 443 participants providing at least one wave of follow-up, 217 participants (49%) reported an overdose at least one wave. This reflects the fact that the more follow-up one provides, the greater the likelihood that we will assess them at a period where they report this relatively rare event. However, it is notable that in the full sample, 62 participants (11%) reported one or more past 30-day overdose events at baseline. Those who experienced an overdose were more likely to be younger and more likely to have a history of overdose prior to the study (see Table 1).

3.2. Measure Development

3.2.1. Reducing item pool.

After removing items with almost no endorsement (see above), we then reduced the item pool further by conducting preliminary PCA analyses within both the Spearman’s Rho correlation matrix and the dichotomous correlation matrix. This process identified two items – days using methadone and days using buprenorphine – that did not consistently load onto any factor, likely given their use in treatment for opioid use disorder. Additionally, one item (snorting heroin) only demonstrated negative associations with each of the identified factors and was therefore removed. This process reduced the original 39-item measure (provided in Supplementary Materials) to 26 items.

3.2.2. Creating final subscales.

Both versions of the correlation matrix were submitted to the optimal implementation for parallel analysis for PCA (Timmerman & Lorenza-Seva, 2011). Using a 95% cut-off suggested a 6-component solution within the remaining 26-item pool. Inspection of the strongest loading items within each PCA solution suggested the same items tended to load onto the same components, though the magnitude of those loadings might differ slightly between the PCA in the raw item scores, and the dichotomized item scores (see loadings in Table 2).

As can be seen in Table 2, the first three subscales addressed different patterns of use including “Prescription Opioid Misuse,” two closely linked items regarding frequent use of opioid medications and using them more than prescribed; “Risky Non-Injection Use,” a 5-item subscale composed of methods of use that were high-potency but did not include injection; and “Injection Drug Use & Speedballing,” a 4-item subscale highlighting injection drug use and “speedballs” (generally, an injected combination of heroin and cocaine). The second three subscales referred to the respondent taking different combinations of drugs with high overdose potential over a short (6-hour) period, specifically “Opioid and Alcohol Combinations” (3 items); “Opioid and Benzodiazepine Combinations (4 items); and finally, “Higher Order Polysubstance Combinations” (8 items).

Cronbach’s alphas and corrected item-total correlations are also presented in Table 2. All subscales demonstrated adequate internal consistency (α>.70) with the exception of Injection Drug Use & Speedballing. Additionally, almost all items showed adequate item-to-total correlations with the remaining items in their intended factor (rs > .30;Nunnally and Bernstein, 1994), suggesting nearly every item “hangs together” within its subscale with the exception of the items, “smoke or vaporize heroin” and “injecting opioids other than heroin.” After discussion, these items were retained in their respective subscales as the item content was consistent with the remaining items and each item addressed important behaviors which we believed would be helpful to clinicians and researchers. To create the final subscale scores, we retained the strategy used in the ORBS 1 and averaged all items in each subscale, thereby weighting all items equally and providing subscale scores that ranged from 0–30.

3.3. Validation Phase

3.3.1. Convergent Validity of the ORBS-2.

Convergent validity of the ORBS-2 was evaluated through its associations with the ORBS-1 at baseline. As seen on Table 3, related subscales from ORBS-1 and ORBS-2 tended to have moderate-magnitude associations (approximately .3 - .4). However, as the ORBS-2 used empirically derived subscales, the “Concurrent Use of Other Psychoactive Drugs” scale from ORBS-1 has essentially been “broken up” into three relatively independently varying subscales (e.g., the “Opioid and Alcohol Combinations” subscale correlates less than .20 with all other subscales), and the solitary use subscale from ORBS-1 was excluded from the ORBS-2 revision and therefore has no clear correlates in ORBS-2. Importantly, the subscales of the ORBS-2 demonstrated relative independence from one another, highlighting an advantage over the ORBS-1, as each ORBS-2 subscale represents empirically and conceptually distinct constructs.

Table 3 -.

Descriptives and Spearman Rank Correlations between ORBS2 Subscales and Related Measures (Convergent and Concurrent Validity)

Scale Number		Intercorrelations between ORBS2 Subscales
	Subscale Name	A	B	C	D	E	F
A	Prescription Opioid Misuse
B	Risky Non-Inj ection Use	.16
C	Injection Drug Use & Speedballing	−.13	.03
D	Opioid & Alcohol Combinations	−.06	.07	.19
E	Opioid & Benzodiazepine Combinations	.01	.19	.06	.06
F	Polysubstance Combinations	−.01	.28	.12	.10	.31
Category of Concept		ORBS2 Correlations with Related Constructs
	Scale	A	B	C	D	E	F
ORBS 1.0 Scales (Convergent Validity)
	Non-Adherence to Opioid Dosage	.32	.44	−.05	.01	.07	.10
	Alternative Methods of Opioid Administration	.04	.30	.07	.03	.05	.05
	Solitary Opioid Use	.08	.13	−.05	.01	−.05	.02
	Use of Non-prescribed OD-Associated Drugs	.00	.15	.45	−.01	.39	.34
	Concurrent Use of Other Psychoactive Drugs	.13	.18	.33	.39	.40	.31
Relevant Clinical Constructs (Concurrent Validity)
	Pain Severity	.21	.16	−.03	−.05	.12	.12
	Pain Interference	.24	.14	−.06	−.06	.10	.11
	Opioid Use Disorder Symptoms	−.08	.17	.23	.08	.15	.17
	Alcohol Use Disorder Symptoms	−.06	.06	.10	.66	.08	.10
	PTSD Symptom Severity	.08	.07	.11	.02	.24	.20
	Anxiety Symptom Severity	.01	.06	.07	.01	.25	.20
	Depressive Symptom Severity	−.01	.02	.10	.00	.20	.18
	Presence of Suicidal Ideation	.00	.15	.03	.02	.10	.17

Note. All correlations with magnitude >.11 significant at p <.01. Correlations with magnitude > .20 bolded for ease of interpretation

3.3.2. Concurrent Validity with Clinical Constructs.

The bottom half of Table 3 examines the associations between the ORBS-2 scale and relevant clinical constructs. Notably, each subscale appears to represent a clinically distinct pattern of use. For example, Prescription Misuse appears uniquely correlated to both severity of pain and impairment due to pain, suggesting that variability in this subscale more closely related to a need for off-label pain management than any particular clinical concern. The severity of opioid use disorder symptoms is most strongly correlated with the Injection Drug Use & Speedballing subscale, and the severity of Alcohol Use Disorder symptoms is most strongly correlated with individuals using Opioid and Alcohol Combinations. Both the Opioid and Benzodiazepine Combinations subscale and Higher Order Polysubstance Combinations subscale were associated with PTSD symptom severity and depressive symptom severity. However only the Higher Order Polysubstance Combinations and Risky Non-injection Use subscales were associated with the presence of suicidal ideation.

3.4. Utility of ORBS-2 Subscales in Predicting Overdose.

We examined each ORBS-2 subscale’s capacity to predict non-fatal overdose in our sample over the period of follow-up assessments. As can be seen in Table 4, two of the six subscales (Risky Non-Injection Use; Opioid and Benzodiazepine Combinations) demonstrated significant longitudinal prediction, with higher levels of those risk behaviors at baseline being associated with a greater number of overdose events across the next year of follow-up. Additionally, four of the six ORBS-2 subscales showed time-varying covariation with the incidence rate of overdose in our sample. That is, on months where participants reported performing more of the risk-behaviors in those subscales (Prescription Misuse; Unsafe Methods of Administration; Injection Drug Use; Opioid and Alcohol Combinations), they demonstrated a greater likelihood of experiencing one or more overdose events that month. Notably, the Risky Non-Injection Use subscale demonstrated the strongest prediction both between-person (participants reporting these behaviors show higher risk across the year) and within-person (participants are at the highest risk on months where they are using these behaviors). Surprisingly, the higher order Polysubstance Combinations of drug use (using 2–5 drugs over a 6-hour window) did not significantly predict a greater amount of self-reported non-fatal overdoses.

Table 4.

Poisson Hierarchical Linear Modeling Results of Overdose Risk Behavior Scale 2 (ORBS-2) Subscales Predicting Days Overdose Each Month

ORBS-2 Subscale	Unadjusted		Adjusted for Demographicsa
Level of Analysis	Incidence Ratio	95% CI	Incidence Ratio	95% CI
Prescription Misuse
Between-Person Differences at Baseline	1.05	(0.89,1.25)	1.08	(0.91,1.29)
Within-Person Variation each Month	1.18	(1.15,1.22)	1.18	(1.15,1.22)
Risky Non-Injection Use
Between-Person Differences at Baseline	1.25	(1.06,1.47)	1.22	(1.04,1.44)
Within-Person Variation each Month	1.29	(1.26,1.33)	1.29	(1.26,1.33)
Injection Drug Use & Speedballing
Between-Person Differences at Baseline	1.20	(1,1.43)	1.10	(0.91,1.32)
Within-Person Variation each Month	1.29	(1.23,1.35)	1.28	(1.22,1.34)
Opioid and Alcohol Combinations
Between-Person Differences at Baseline	1.06	(0.88,1.27)	1.07	(0.89,1.28)
Within-Person Variation each Month	1.11	(1.05,1.16)	1.11	(1.06,1.17)
Opioid and Benzodiazepine Combinations
Between-Person Differences at Baseline	1.34	(1.13,1.58)	1.30	(1.1,1.54)
Within-Person Variation each Month	1.02	(0.98,1.06)	1.01	(0.97,1.05)
Polysubstance Combinations
Between-Person Differences at Baseline	1.08	(0.91,1.29)	1.08	(0.91,1.29)
Within-Person Variation each Month	0.98	(0.94,1.03)	0.98	(0.94,1.03)

Note. Values significant at p<.05 bolded for ease of interpretation. Each subscale standardized to facilitate comparisons.

^aCoefficients adjusted for: Age, race, education, and history of overdose across all waves (i.e., slope-intercept model).

4. DISCUSSION

Relative to the original ORBS1, the scale presented here used an expanded item pool that assessed a wider variety of overdose risk behaviors, while increasing specificity in item content (i.e., removing items that complicate the construct of risk) and item wording (i.e., precise definitions of concurrent use). Finally, ORBS 2 reduces participant burden through “gateway” questions/skip-rules and providing embedded definitions. Through these various modifications, we sought to encourage participants to reflect more carefully on their responses, allowing more distinct factors to emerge in analysis. In this analysis, a data-driven approach to subscale creation served to help further elucidate the structure of a construct—overdose riskiness—that has seen little attention in empirical research. The PCA indicated that different forms of overdose risk cluster together, in ways that, to a great extent, hold up to theoretical scrutiny. Opioid-related polysubstance use involving alcohol appeared substantially different than opioid-related polysubstance use involving benzodiazepines. Speedballing, although not specified in terms of route of administration in our measurement, was strongly correlated with opioid injection behaviors. Using unprescribed opioid medications was more strongly associated with a range of clinically non-adherent other behaviors, like crushing and snorting pills, than was using opioid medications prescribed by a doctor, even among a sample of participants defined by use of unprescribed or illicit opioids. ORBS2 subscales showed substantial correlation with ORBS-1 subscales, where scales had strong thematic overlap, but new subscales in ORBS2 diverged considerably from ORBS-1 through use of a data-driven innovation to identifying conceptually distinct constructs.

In the contexts of this sample, the PCA has demonstrated that the clusters of opioid use behaviors grouped into subscales have varying degrees of association with experiences of non-fatal OD, and that some of those most strongly associated are not necessarily those most likely to be perceived as high-risk. In particular, Subscale B, Risky Non-Injection Use, appears to represent a category of opioid use behaviors that might be perceived to be less risky. Prescription opioid use, in contradistinction with heroin, might be imagined by many people who use opioids to be an OD-reducing substance choice, just as snorting (or vaporizing) is widely held to be less OD-risky than injection. Our finding of increased risk associated with the use of opioids in pill form may have resulted from incorrect assumptions about their safety. It is likely that the rapid emergence of a national marketplace for illicitly manufactured counterfeits containing fentanyl-class opioids (Centers for Disease Control and Prevention (CDC), 2015; Green and Gilbert, 2016; Palmer, 2019; Vo et al., 2016) contributed to the reported overdoses. It is also possible that the dangerousness of legitimate prescription opioids, available in very high dosages and potency, may be frequently underestimated by those using them without prescription or physician supervision. Other subscales, related to polysubstance use with and without benzodiazepines, were more strongly correlated with mental health issues endemic within populations of people who use opioids and other drugs. Greater severity of depression, PTSD, and anxiety symptomatology was correlated with polysubstance subscales in particular, reiterating the disproportionate risks of OD experienced by those whose illicit substance use overlaps with self-medicating (and physician-directed) use of substances to reduce anxiety and help with sleep. The significant association between the polysubstance subscale and the non-injection risk behaviors further suggests that much of this risk may be driven by use of pharmaceutical opioids and other non-injection behaviors held to be more lawful and/or salubrious than heroin injection.

This study is not without its limitations. Self-reported opioid-involved overdose events are, by their nature, difficult to measure and subject to recall bias. While this analysis employed the broadest measure of overdose, events that transpire without other persons present are likely unrecoverable by their survivors, barring evident morbidity. This, combined with a potential survivor bias in longitudinal data collection among a high-risk sample of people who use unprescribed opioids, and significant loss to follow-up, makes the task of predicting OD events in our sample somewhat imprecise. The study was also hindered by a lack of precision in identifying conscious IMF use and in specifying whether opioid pill use came from a known source with a prescription or from a dealer (and thus more likely to involve counterfeiting/contamination) and whether speedball use involved injection or insufflation. Further, while we believe that the ORBS-2 represents a theoretically comprehensive inventory of opioid-related OD risk behaviors, grounded in ample clinical and field-based research, the notion of “OD riskiness” as a unified psychometric construct remains relatively novel. What this analysis suggests is that people who use opioids without prescription participate in clusters of related risk behaviors but that not all forms of risk are tightly intercorrelated with all others in a scale of this nature. The subscales empirically derived in this analysis are clearly the product of market and consumer dynamics local to NYC. While the sampling methodology and the inherent diversity of NYC’s resident population makes the factors identified in our analysis potentially more generalizable than those from a single-site setting elsewhere in the U.S., future work should test the stability of the ORBS-2 factor structure presented here by conducting confirmatory factor analyses within a large independently collected sample that might also allow for item-response theory analyses that can provide further insight into optimal weighting of items. Additional validation of the ORBS 2 in samples with richer outcome data including medical records review of nonresponders for unreported nonfatal and fatal overdose events will further enrich predictive analyses that can then be used to identify optimal risk thresholds for individual clinical prediction.

Despite these concerns, establishing reliable measurement of opioid-related risk behaviors presents opportunities in both clinical and research settings. For researchers seeking to better understand the distal precipitants of changes in opioid risk behavior (such as the psychosocial impacts of challenging or supportive life events), tools like the ORBS2 present an important barometer of hazard and a reasonable proxy outcome for OD events themselves. Use of the ORBS-2 may serve research that looks at social determinants of particular risk behaviors or the impact of interventions on risk behavior—for example, after a significant life event, including surviving an OD event (Banta-Green et al., 2019; Winhusen et al., 2020) which have traditionally looked at dichotomous outcomes that likely miss some of the more subtle efforts undertaken by survivors to reduce risk (Elliott et al., 2019). Further, when administered to regional subsamples of PWUO, a comprehensive index of OD risk behaviors can also provide an important epidemiological tracking tool to better pinpoint some of the specific local conditions that drive OD mortality. For clinicians, and particularly those working within substance treatment and harm reduction/syringe service program contexts, having a set of modular indices of a client’s current opioid use patterns can readily provide an important behavioral baseline for “risk communication” (Cho et al., 2014; Lundgren and McMakin, 2018) that addresses individually-tailored steps that can be taken to reduce OD hazard, such as changing polysubstance use intervals or using lower doses of heroin—or lower-risk methods of administration), particularly when drinking or taking benzodiazepines.

HIGHLIGHTS.

• The Overdose Risk Behavior Scale provides a comprehensive index of proximal opioid-related overdose risks

• Factor analysis of the full scale identifies a 6-subscale division of the item into relatively independent groups of overdose risk behaviors

• Five out of six subscales were positively associated with self-report of non-fatal opioid overdose experiences in longitudinal analysis

• Suicidality was only modestly associated with one subscale, indicating that most non-fatal overdoses are unintentional

• Experiences of Anxiety and Depression appear linked to polysubstance use patterns, particularly those involving opioids and benzodiazepines

• Non-injection behaviors involving diverted and potentially counterfeit opioid pills represented the strongest predictor of non-fatal overdose experiences in follow-up