Development of an Opioid-Related Overdose Risk Behavior Scale (ORBS)

Abstract

Background

Drug overdose has emerged as the leading cause of injury-related death in the U.S., driven by prescription opioid (PO) misuse, polysubstance use and use of heroin. To better understand opioid-related overdose risks that may change over time and across populations there is a need for a more comprehensive assessment of related risk behaviors. We developed the opioid-related Overdose Risk Behavior Scale (ORBS), drawing on existing research, formative interviews, and discussions with community and scientific advisors.

Methods

We enrolled military veterans reporting any use of heroin or POs in the past month using venue-based and chain referral recruitment. The final scale consisted of 25 items grouped into 5 subscales eliciting the number of days in the past 30 during which the participant engaged in each behavior. We assessed internal reliability using Cronbach’s alpha; test-retest reliability using intraclass correlation coefficients (ICCs); and criterion validity using Pearson’s correlations with indicators of having overdosed during the past 30 days.

Results

Data for 220 veterans were analyzed. The 5 subscales---A. Adherence to Opioid Dosage and Therapeutic Purposes, B. Alternative Methods of Opioid Administration, C. Solitary Opioid Use; D. Use of Non-prescribed Overdose-associated Drugs and E. Concurrent Use of POs, Other Psychoactive Drugs and Alcohol---generally showed good internal reliability (alpha range = 0.61 to 0.88), test-retest reliability (ICC range = 0.81 to 0.90), and criterion validity (r range = 0.22 to 0.66). The subscales were internally consistent with each other (alpha = 0.84). The scale mean had an ICC value of 0.99, and correlations with validators ranged from 0.44 to 0.56.

Conclusions

These results constitute preliminary evidence for the reliability and validity of the new scale. If further validated, it could help improve overdose prevention and response research, and could help improve the precision of overdose education and prevention efforts.

INTRODUCTION

Drug overdose has emerged as the leading cause of injury-related death in the U.S., driven by prescription opioid (PO) misuse, polysubstance use and use of heroin.^1–6 For overdose prevention and response research a broad assessment capable of capturing behavioral risks in populations with varying substance choices and use patterns is critically important, particularly as we seek to understand the precipitants of changes in overdose risk behaviors among at-risk populations.

Though no comprehensive overdose risk behavior scale exists, a number of clinical screening tools assess some aspects of the potential for PO misuse. For example, the Current Opioid Misuse Measure (COMM), the Screener and Opioid Assessment for Patients with Pain (SOAPP), and the Opioid-Related Behaviors in Treatment (ORBIT) scale were designed to help clinicians assess PO misuse risks among pain patients.^7–11 Reflecting the purpose of clinical screening, these scales include items on drug-seeking behaviors, past history of substance use, and mood states presumed to be related to PO misuse,^7,10–17 sometimes incorporating criteria to suggest when a clinician should consider modifying (or denying) PO treatment.^9,12–15,18 However, since these scales were not designed to assess overdose risk they lack items regarding the contraindicated use of opioids (including heroin, methadone and buprenorphine), other drugs and alcohol, and items regarding drug use behaviors that can increase risk, such as injecting opioids, or using opioids alone.^19–21

To better understand the factors that cause opioid-related overdose a first step is to comprehensively assess overdose risk behaviors, and test their associations with overdose events. One group with elevated overdose risks related to PO and heroin use is military veterans.^22,23 Below, we describe the development of an opioid-related Overdose Risk Behavior Scale (ORBS) and present results of preliminary reliability and validity analyses in a community-based sample of US military veterans.

METHODS

We enrolled 220 U.S. military veterans who reported any heroin or PO use (within past 30-days) at the time of enrollment between August 2014 and June 2016 as part of an ongoing study of opioid misuse and overdose risks.^23,24 We recruited participants in veterans’ service agencies, homeless shelters and opioid treatment programs throughout New York City using venue-based and chain referral methods. Enrolled participants completed survey-based assessments of their military service, alcohol, PO and other drug use, biological, psychological, social contextual factors and major life events.

Our aim was to construct a scale to assess known opioid-related overdose risk behaviors. We adapted 4 items from the COMM [“How often have you…taken your medications differently from how they are prescribed? …needed to take pain medications belonging to someone else? …had to take more of your medication than prescribed? …used your pain medicine for symptoms other than for pain?”] regarding POs that were prescribed by the patients’ own doctor.²⁵ Additional items were developed using a literature review,^19–21 discussions with members of the study’s community advisory board and scientific advisory board, and results of qualitative interviews with 50 veterans who reported experiencing at least one opioid-related overdose at some point in the past.²⁶ Although, theoretically, there was no strong rationale for assuming that these items would be correlated, items were naturally categorized into subscale domains that we hypothesized would be internally consistent. We used the number of days during the past 30 days as the response units for all items. Thus, responses are ratio scaled.²⁷

We pilot tested a preliminary version of the scale with 15 participants who were then interviewed about clarity, comprehensiveness and tone. After this process, the research team reconvened and eliminated 4 items (using borrowed, purchased or stolen POs, and using POs at different intervals than prescribed) because they were not understood consistently and overlapped with other items, and revised item wording for clarity and brevity.

After eligibility screening and informed consent procedures, participants completed a survey assessment using face-to-face methods with a trained and experienced interviewer. Participants provided informed consent, and received $20 for completing the assessment, which also included investigational domains hypothesized to precipitate change in overdose risk behaviors, including mood, physical pain and life events. All procedures and protocols were approved by the Institutional Review Board of the first author. Administration duration of the new scale was typically 5–10 minutes. Only the initial assessment from each participant was analyzed (with the exception of test-retest data). We used IBM SPSS (ver. 22) for analyses. We assessed reliability and validity of the total scale and of subscales. We report scale scores as an unweighted average of constituent items. Thus, the subscale and total scale means reflect the same 0–30 units as the constituent items.

We asked a subset of 38 sequential participants to repeat the assessment the next day, preferably at the same time of day; 34 of these provided retest data within the time frame. Participants were compensated an additional $20 for retests. We compared their responses using intraclass correlation coefficients (ICCs) with a one-way random effects formulation, ICC(1,1).²⁸ We also compared mean differences in test and retest assessments using t-tests for paired samples. To determine whether the participants who provided test-retest reliability data differed significantly from the rest of the sample we compared their characteristics using Student’s t-tests.

We assessed criterion validity retrospectively using Pearson’s correlations with two indicators of having experienced an overdose during the same past-30-day time period. We asked: “On how many days in the past 30 did you lose consciousness or pass out so that you could not really wake up or others could not wake you up?” and “On how many days in the past 30 did you or someone else call for medical assistance because of how sedated, drugged, or high you were after using opioids?”

We examined Pearson’s correlations among the subscales to assess how they were related.

RESULTS

Participant characteristics are shown in Table 1. Most participants were male, never married and unemployed, despite relatively high educational achievement. The majority were black/African American, and almost a quarter were Hispanic/Latino. Ages ranged from 21 to 60, averaging 36.9 years (standard deviation [SD] = 9.5). Almost half the sample reported being homeless or unstably housed at the time of enrollment. Heroin use (during the last 30 days) was reported by 27.3%, and PO use (including those who had been prescribed POs) was reported by 89.1%.

TABLE 1.

Participant characteristics. N = 220 veterans currently using opioids (past 30-days)

Characteristic	Percentage %1
Age
21–30	32.4
31–40	36.5
41–50	20.5
51–60	10.5
Gender (% male)	84.1
Racial category
White	26.6
Black/African American	69.2
Other2	4.2
Hispanic/Latino ethnicity	21.8
Marital status
Never married	51.1
Married	14.6
Divorced or separated	34.2
Educational achievement
Less than high school graduation	2.7
High school graduate or GED	37
More than high school graduate	60.3
Currently attending school or training	18.3
Employment status
Employed full-time or part-time	25.7
Unable to work due to disability or retired	24.3
Unemployed	50
Income category (per year)
No income from employment	62.8
Less than $20,000	23.7
$20,000 or more	13.5
Currently homeless or unstably housed	46
Discharge Type
Honorable	68
General	18.3
Other4	13.7
Years since discharge
Less than 3 years	23.5
3–10 years	53
More than 10 years	23.5
Receiving Veterans’ Disability	36.8
Current opioid analgesic prescription	46.8
Used analgesic prescription opioids past 30 days	89.1
Participating in methadone maintenance program	8.2
Used heroin past 30 days	27.3
Injected opioids past 30 days	12.3
Current anti-anxiety medication prescription	22.7
Used anti-anxiety drugs past 30 days	35.5
Current sleep medication prescription	25
Used sleep medication past 30 days	32.7
Current buprenorphine prescription	1.8
Used buprenorphine past 30 days	6.8
Current prescription for other psychiatric drugs5	24.1
Lost consciousness due to opioid use one or more times past 30 days	15.4
Called for medical assistance due to opioid use one or more times in the past 30 days	8.2

¹Sample sizes for category percentages vary due to non-response for some items, and percentages may not add up to 100 due to rounding

²Other = American Indian/Alaskan Native, Asian, Native Hawaiian/Other Pacific islander, and multiple racial groups

³Other = Coast Guard, Army National Guard

⁴Other = Other than honorable, bad conduct, dishonorable

⁵other psychiatric drugs were those for major depression, epilepsy, Parkinson’s disease, psychosis or schizophrenia.

Table 2 lists ORBS items, grouped by subscales. All or some items in 3 of the 5 scales (A, B, and C) are only applicable to participants who reported a current PO prescription. Therefore, the sample size for those scales is restricted to the 104 (47%) participants who met this criterion. Mean values represent the average number of days during the past 30 in which participants engaged in each behavior. As shown in the interquartile range column, many items were infrequently endorsed, with 25^th percentile, and sometimes 75^th percentile values of 0. The most frequent risk behavior was A.4. Use of POs to aid sleeping, averaging 6.0 days (SD = 9.2). Mean values of items regarding prescribed opioids were generally higher than those for non-prescribed drugs.

TABLE 2.

Scale items; Cronbach’s alpha values; means, standard deviations and interquartile ranges; item-total correlations; alpha values if item deleted; intraclass correlations; and correlations with overdose criterion validators (N = 220)

Subscale	Cronbach’ s Alpha	Mean (standar d deviation )	Interquartil e range			ICC 1	Correlatio n with Criterion validators2
Item3		Mean (standard deviation )	Interquartil e range	Item-total correlatio n	Alpha if item delete d
A. Adherence to Opioid Dosage and Therapeutic Purposes4	0.88	5.1 (7.3)	0.0, 6.5			0.85	0.43, 0.38
1. Did you take more of your prescribed opioid pain medicine than you were directed to take at one time?2		4.8 (7.9)	0.0, 5.0	0.65	0.87
3. Did you take opioid pain medicine not just to reduce pain, but for enjoyment or to get high?		4.1 (7.7)	0.0, 5.0	0.76	0.83
4. Did you take opioid pain medicine not just to reduce your pain, but to help you sleep?		6.0 (9.2)	0.0, 8.0	0.76	0.83
5. Did you take opioid pain medicine not just to reduce your pain, but to help you deal with anxiety, nervousness, sadness or a bad mood?		5.5 (9.3)	0.0, 6.0	0.78	0.82
B. Alternative Methods of Opioid Administration 4	0.73	1.0 (4.4)	0.0, 0.0			0.47	0.54, 0.66
1. Did you sniff (snort or nasally inhale) opioid pain medicine?		1.4 (4.9)	0.0, 0.0	0.54	0.67
2. Did you crush and smoke opioid pain medicine?		0.6 (3.5)	0.0, 0.0	0.69	0.53
3. Did you use a syringe to inject your prescribed opioid pain medicine?		0.9 (4.5)	0.0, 0.0	0.47	0.73
C. Solitary Opioid Use4	0.68	4.5 (7.6)	0.0, 5.0			0.85	0.53, 0.39
1. Were you alone, with no other people present, while you used more of your prescribed pain medicine than advised?		5.7 (9.1)	0.0, 10.0	0.53	-.-
2. Were you alone, with no other people present, while you used heroin?		2.7 (7.1)	0.0, 0.0	0.53	-.-
D. Use of Non- prescribed OD5- Associated Drugs	0.61	2.2 (3.5)	0.0, 2.5			0.84	0.47, 0.22
1. Did you take opioid pain medicine that you got from some source other than your own doctor’s prescription?		5.8 (8.6)	0.0, 9.8	0.33	0.59
2. Did you use heroin?		3.4 (7.8)	0.0, 1.0	0.56	0.49
3. Did you use methadone, either in pill or liquid form, from a clinic or any other source? (Only if not in methadone treatment)		0.4 (2.3)	0.0, 0.0	0.33	0.59
4. Did you inject any opioids at all (heroin, crushed pills, etc.)?		2.2 (7.2)	0.0, 0.0	0.55	0.46
5. Did you use anti-anxiety drugs (these are often referred to as benzodiazepine s or benzos)?		0.8 (4.0)	0.0, 0.0	0.27	0.59
6. Did you use sleep medication?		0.4 (3.0)	0.0, 0.0	0.25	0.60
E. Concurrent Use of Prescription Opioids, Other Psychoactive Drugs and Alcohol	0.77	2.4 (3.7)	0.0, 3.3			0.90	0.49, 0.38
1. Did you use heroin and any opioid pain medicine on the same day?		1.5 (5.2)	0.0, 0.0	0.21	0.78
2. Did you use methadone and any opioid pain medicine on the same day?		1.1 (4.7)	0.0, 0.0	0.18	0.78
3. Did you use buprenorphine and any opioid pain medicine on the same day?		0.3 (2.5)	0.0, 0.0	0.32	0.77
4. Did you use anti-anxiety drugs and any opioid pain medicine on the same day?		4.1 (8.7)	0.0, 2.0	0.51	0.73
5. Did you use sleep medication and any opioid pain medicine on the same day?		3.2 (7.7)	0.0, 0.0	0.50	0.74
6. Did you use alcohol and any opioid pain medicine on the same day?		5.0 (8.1)	0.0, 7.8	0.52	0.74
7. Did you use alcohol, use any opioids at all, and use anti-anxiety drugs on the same day?		2.2 (5.7)	0.0, 0.0	0.66	0.72
8. Did you drink alcohol, use any opioids at all, and use sleep medication on the same day?		2.1 (5.8)	0.0, 0.0	0.63	0.72
9. Did you use cocaine, crack, amphetamine, crystal meth or any other stimulant to try to reverse the effects of opioids?		1.9 (5.6)	0.0, 0.0	0.55	0.74
ORBS Mean (of all subscales)	0.84	2.6 (3.7)	0.4, 3.4			0.99	0.56, 0.44
4. Did you use anti-anxiety drugs and any opioid pain medicine on the same day?		4.1 (8.7)	0.0, 2.0	0.51	0.73
5. Did you use sleep medication and any opioid pain medicine on the same day?		3.2 (7.7)	0.0, 0.0	0.50	0.74
6. Did you use alcohol and any opioid pain medicine on the same day?		5.0 (8.1)	0.0, 7.8	0.52	0.74
7. Did you use alcohol, use any opioids at all, and use anti-anxiety drugs on the same day?		2.2 (5.7)	0.0, 0.0	0.66	0.72
8. Did you drink alcohol, use any opioids at all, and use sleep medication on the same day?		2.1 (5.8)	0.0, 0.0	0.63	0.72
9. Did you use cocaine, crack, amphetamine, crystal meth or any other stimulant to try to reverse the effects of opioids?		1.9 (5.6)	0.0, 0.0	0.55	0.74
ORBS Mean (of all subscales)	0.84	2.6 (3.7)	0.4, 3.4			0.99	0.56, 0.44

¹ICC = Intraclass correlations, which were performed on the test-retest subset (N = 34)

²Criterion validators were: Loss of consciousness, Calling for medical assistance, respectively (all correlations were significant at the p < 0.01 level)

³All items were asked with the following preface: “In the past 30 days…” and added parenthetically “enter the number of days, 0 to 30;”

⁴These subscales were assessed for participants who reported having a current opioid analgesic prescription

⁵OD = overdose.

There were no significant differences in participant characteristics between participants who provided test-retest data and those who did not, and there were no significant differences in mean values of ORBS items or scales between test and retest assessments.

Subscales A, B and C good internal consistency (alpha range = 0.68 to 0.88), test-retest reliability (ICC range = 0.47 to 0.85), and validity (r with criterion variables range = 0.38 to 0.66). (As shown in Table 2, all correlations of the total ORBS and of ORBS subscales with overdose event criterion variables were significant at the p < 0.05 level.)

Subscale D showed marginally acceptable internal consistency (alpha = 0.61), and good retest reliability (ICC = 0.84). Correlations with validators were moderate (r = 0.22, r = 0.47). We considered dropping the non-therapeutic use of methadone from the subscale because few participants (6.4%) endorsed this item. However, the item had a moderate item-total correlation (0.33), which suggested it was worth retaining. We also asked whether participants used non-prescribed buprenorphine. However, this item was the least frequently endorsed, and had a very low item-total correlation (0.08), so it was not retained.

Subscale E showed good internal consistency (alpha = 0.77) and retest reliability (ICC = 0.90). Some items were infrequently endorsed, but are retained to be consistent with our purpose of developing a more comprehensive assessment of opioid-related risk behaviors than is currently available.²⁹ For example, as shown in Table 2, the mean for E.3. Using buprenorphine and any other POs on the same day was 0.3 (4.1% of participants reported using buprenorphine and any other opioids on the same day once or more during the past 30 days). Correlations with validators were moderate (r = 0.38, r = 0.49).

Averaging the subscales yields a mean of 2.6 (SD = 3.7). This mean was correlated moderately to strongly with criterion validators (r = 0.44, r = 0.56). The alpha value for the Cronbach’s analysis of the 5 ORBS subscale means was 0.84.

Correlations among ORBS subscale means are in Table 3. Correlations ranged from 0.45 to 0.72.

TABLE 3.

Correlations among subscales.

	B. Alternative Methods of Opioid Administration	C. Solitary Opioid Use	D. Use of Non- prescribed Overdose- Associated Drugs	E. Concurrent Use of Prescription Opioids, Other Psychoactive Drugs and Alcohol
A. Adherence to Opioid Dosage and Therapeutic Purposes	0.51	0.58	0.65	0.45
B. Alternative Methods of Opioid Administration	--	0.61	0.67	0.56
C. Solitary Opioid Use	--	--	0.72	0.59
D. Use of Non- prescribed Overdose- Associated Drugs	--	--	--	0.51

Note. Correlations in the table represent those who reported having a current opioid analgesic prescription, except for the correlation between D and E subscales which represent the whole sample. All correlations are significant at the p < 0.05 level.

DISCUSSION

Our findings provide preliminary evidence that engagement in opioid-related overdose risk behaviors can be assessed comprehensively using a brief scale. Five subscales representing engagement in opioid using behaviors that can increase overdose risk were each, and as an aggregate, correlated with indicators of an opioid overdose event. These subscales included behaviors specific to PO misuse among patients, and behaviors that can apply to heroin use or PO misuse among patients and non-patients, such as solitary use and concurrent use of other drugs or alcohol. As the opioid overdose epidemic continues to worsen, it is all the more important to comprehensively assess risks related to heroin and polysubstance use.³⁰ Though it was designed for research purposes the ORBS also has the potential to be useful in therapeutic contexts where counselors, psychiatrists and social workers stand to aid individuals in developing strategies to minimize engagement in such behaviors. This value may also be realized in community-based settings wherein overdose education and prevention programs might be bolstered by more specific data about the nature and frequency of engagement in overdose risk behaviors.

There are several important limitations to the interpretation of these results, the first of which applies to all research using self-report. We did not collect or analyze biological data, leading to a potential underreporting of opioid use severity due to social desirability bias. Two subscales (C and D) with relatively weak internal reliability results retain moderate correlations with criterion validity indicators, which may suggest that the lower reliability resulted from infrequent responses for some items. For example, items regarding smoking or injecting POs were rarely endorsed, despite being associated with increased risk.^31,32 We retain these items as they may represent important risks in other populations, especially considering the evolving nature of the opioid overdose epidemic. Our limitations with regard to sample size and a cross-sectional ascertainment provide guidelines for future research; for example, broader investigations of at-risk cohorts in addition to opioid-using veterans could evaluate generalizability, and a prospective-longitudinal study could evaluate predictive validity.

In conclusion, results from this study show preliminary evidence that a comprehensive set of opioid-related overdose risk behaviors can be assessed reliably using a brief scale. The ORBS’ focus on overdose-associated polysubstance use, modes of administration, and social contexts for use may prove to be useful for better understanding overdose risks that can change over time or may differ across populations.^33–37 It has the potential to help to improve the precision of overdose prevention and response research, and overdose education and prevention efforts.