Opioid Risk Tool scores are not associated with opioid misuse after elective general surgery: A prospective study

Abstract

Introduction.

Assessment for opioid misuse is not widely utilized pre-operatively. Yet, it can offer valuable clinical insight for patient counseling. The opioid risk tool (ORT) and its opioid use disorder-specific variant (ORT-OUD) predict opioid misuse in chronic pain patients. While recommended for surgical patients, neither has been validated. Herein, we sought to validate the ORT and ORT-OUD in patients undergoing elective general surgery procedures.

Methods.

This was a prospective observational study. Consented patients filled out the ORT. Demographics, prescribed opioids, and adjuncts were collected. Pain control ratings after surgery and whether patients were still using opioids were collected at 30 days post-surgery. Patients were grouped as low, medium, and high-risk for opioid misuse based on ORT or ORT-OUD scores and compared. P < 0.05 was considered significant.

Results.

We included 178 patients; 139, 28, and 11 scored low, medium, and high-risk on the ORT, respectively. Compared to low-risk, medium- and high-risk patients were more likely to call for pain (8% vs. 32.1% vs. 36.4%; p<0.001) and rate their pain control as poor or fair (8.5% vs. 25% vs. 18.2%; p=0.047). Only three opioid-naïve patients were still using opioids 30 days post-surgery; they all scored low-risk on ORT. Similar results were obtained with the ORT-OUD.

Conclusions:

While higher ORT scores are associated with worse pain satisfaction and an increased likelihood of a phone call for pain, the ORT and ORT-OUD were not validated as predictive tools for prolonged opioid use or opioid misuse following elective general surgery.

INTRODUCTION

Post-operative pain management remains a critical challenge in surgical care, particularly among an increasingly older patient population. Despite the widespread use of opioids for acute post-surgical pain, there is considerable variability in prescribing practices, and a growing concern about the risk of prolonged opioid use following surgery.(1, 2) Studies have shown that a significant proportion of surgical patients continue to use opioids well beyond the expected recovery period, raising concerns about dependency and misuse.(3, 4) Literature regarding the outcomes of surgical patients who use opioids remains scarce.

The Opioid Risk Tool (ORT), originally developed to assess the risk of opioid misuse in patients with chronic pain, has gained attention for its potential utility in surgical settings.(5, 6) However, its predictive value for identifying patients at risk for long-term opioid use or opioid use disorder (OUD) after surgery remains underexplored.(7–10) The ORT-OUD, a modified version of the original tool, uses a different scoring mechanism that may offer advantages, and does not require asking a sensitive question about sexual abuse.(11) Understanding whether the ORT or ORT-OUD can reliably stratify surgical patients based on their risk for prolonged opioid use or opioid misuse is therefore essential for informing proactive, individualized pain management strategies. If validated in perioperative populations, these tools could help clinicians identify high-risk patients early, guide more cautious prescribing, and support targeted interventions that reduce the likelihood of developing persistent opioid use or OUD after surgery.

Because the ORT predicted aberrant opioid use behaviors in patients with chronic pain, we speculated that it may also have utility in predicting opioid misuse among surgical patients.(5) This study investigated whether the ORT and ORT-OUD can effectively identify surgical patients at risk for extended opioid use or misuse following their procedures. In addition to evaluating the predictive utility of these tools, we examined their associations with pain control and frequency of provider contact for pain-related concerns. The overarching goal was to determine whether presurgical administration of the ORT or ORT-OUD should be recommended to support identification of at-risk patients and inform pain management decisions.

METHODS

Ethical statement

This prospective cohort study approved by our institutional review board (IRB # 201810859). All participants were consented prior to the beginning of the study.

Patient population

All adult patients who underwent elective outpatient procedures provided by the Acute Care Surgery Division surgeons from November 2018 to April 2021 and who agreed to participate were included in this study. The Acute Care Surgery Division provides the following laparoscopic or open procedures: unilateral or bilateral inguinal hernia repair, umbilical hernia repair, ventral hernia repair, epigastric hernia repair, cholecystectomy, colon resection, exploratory laparotomy, cyst /mass/skin excision, wound exploration /closure, colostomy, diagnostic laparoscopy, excision of foreign body, takedown loop ileostomy. Pregnant women, subjects who were incarcerated, and those unable to provide consent were excluded.

Study design

This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines for cohort studies.(12, 13) Subjects were approached during their pre-surgery appointment. We presented them with the study and those who agreed to participate signed the informed consent form. Once written consent was obtained, participants completed the ORT/ORT OUD survey questions. (Figure 1) At the time of consent, for the last 80 included participants, the research team member provided a DisposeRx Single-Use Drug Disposal Packet that the subject could use to destroy any unused opioid prescription and an explanation on how to use this product.

Figure 1. Opioid risk tool (A) and Opioid risk tool OUD (B) scoring tools.

The subjects were given a handout on the risks associated with opioid use as part of their pre-surgery packet.

Data collection

The subjects’ medical records were reviewed to collect the following information: age, sex, comorbidities, type of surgery, duration of hospital stay, family history, and medications used before, during, and after surgery. Medications of interest included benzodiazepines, insomnia medications (zaleplon, zolpidem, zopliclone, eszopiclone), muscle relaxants, antidepressants, non-opioid pain medication, and use of opioids before and after surgery. The electronic medical records were also reviewed to record calls to the clinic with pain concerns and whether the patient were prescribed opioid refills.

Thirty to 45 days following surgery, a research team member called the subjects and asked them to rate their pain control after surgery, what pain medications they took after surgery, how many opioid pills they were given, if they received opioid prescription from someone other than their surgeon, and if they were still using opioid pain medication. The subjects were also asked about what other medications they were taking. (supplementary data 1: 30-day survey)

If the subjects stated that they were not currently taking opioids, they were asked how many days they used pain medication post-operatively, the approximate number of opioid pills they used, and if and how they disposed of the leftovers.

The subjects were also asked if they received a pamphlet describing opioids and their risks prior to surgery and what recommendations to reducing risk of overdose and what side effects of opioids they remembered reading about from the pamphlet.

If the subjects stated that they were still taking opioids when contacted for the first survey, they were called 90 to 105 days after surgery and asked approximately how many pills they had taken since surgery if they received opioids from someone other than their surgeon and they were administered the Current Opioid Misuse Measure (COMM). A score of 9 or higher indicates a positive screen for potential opioid misuse.(14, 15) The subjects were also asked about what other medications they were taking. (supplementary data 2:90 day survey)

Subjects who could not be reached within 30 and 45 days for the initial survey or within 90 and 105 days for the follow up survey were considered lost to follow up.

Statistical analysis

Our primary outcome was to assess whether the ORT and ORT-OUD allowed us to identify patients at risk of opioid misuse. Our secondary outcomes included pain control satisfaction and frequency of provider contact for pain-related concerns. Patients were classified based on their scores on the ORT. Patients who scored ≤3 were considered low risk, scoring 4–7 was considered moderate risk, and scoring ≥8 was considered high risk for opioid misuse.(5) We also calculated ORT-OUD scores using the relevant subset of ORT questions. Patients who scored ≤2 were considered low risk and those who scored ≥3 were considered high risk.(11) At the 90-day follow-up, those who scored 9 or greater on the COMM were considered to have signs suggestive of opioid misuse.

Categorical patient characteristics and outcomes were compared across ORT and ORT-OUD risk categories using chi-square tests, or Fisher’s exact tests if any cells had expected cell counts less than 5. Continuous and ordinal characteristics and outcomes were compared using Kruskal-Wallis tests. All analyses were completed using SAS version 9.4 (Cary, NC).

RESULTS

Patient characteristics

As shown in Figure 2, 192 subjects were enrolled in the study. Surgery was canceled for 13 subjects and one subject died before the 30 day follow up. A total of 178 subjects were included; 139 scored low-risk (<3), 28 moderate risk (4–7), and 11 high-risk (>8) on the ORT. Patient characteristics are presented in Table 1. Patients who scored high-risk on the ORT were significantly younger. They were more likely to be on Medicaid insurance and to present with psychiatric disorders, alcohol use disorder and/or a history of illegal drug use or prescription drug abuse. They were also more likely to be taking opioids prior to surgery. As shown in supplementary Table 1, most patients underwent laparoscopic cholecystectomy (30.3%), followed by unilateral open inguinal hernia repair (24.2%), cyst /mass/skin excision (13.5%), and unilateral laparoscopic inguinal hernia repair (9%). There were no significant differences between the group regarding the type of elective surgery performed. Additionally, there were no significant differences in drug disposal packet receipt by ORT risk category (p = 0.484).

Figure 2. Subject selection flowchart.

Table 1. Patient characteristics.

Patients were grouped based on their scores on the original ORT questionnaire.

Variables	Overall n = 178	Low n = 139	Medium n = 28	High n = 11	p-value
Age (years), median (IQR)	53 (35, 65)	55.0 (37.0, 66.0)	44.5 (27.5, 56.5)	39.0 (29.0, 57.0)	0.023
Female, n (%)	82 (46.1%)	62 (44.6%)	13 (46.4%)	7 (63.6%)	0.475
White, n (%)	152 (88.4%)	118 (88.7%)	23 (82.1%)	11 (100.0%)	0.425
Body mass index	27.7 (24.4, 32.1)	27.9 (24.6, 31.7)	26.6 (24, 31.9)	27.6 (20.6, 38.6)	0.908
Insurance, n (%)
Medicaid Insurance	18 (10.1%)	9 (6.5%)	5 (17.9%)	4 (36.4%)	0.004
Medicare Insurance	48 (27.0%)	41 (29.5%)	6 (21.4%)	1 (9.1%)	0.263
Private insurance	138 (77.5%)	109 (78.4%)	24 (85.7%)	5 (45.5%)	0.022
Military insurance	1 (0.6%)	1 (0.7%)	0 (0.00%)	0 (0.00%)	>0.999
Other insurance	12 (6.7%)	9 (6.7%)	0 (0.00%)	3 (27.3%)	0.018
Comorbidities, n (%)
Chronic pain	39 (21.9%)	31 (22.3%)	5 (17.9%)	3 (27.3%)	0.792
Depression	51 (28.7%)	35 (25.2%)	10 (35.7%)	6 (54.5%)	0.078
Anxiety	32 (18.0%)	20 (14.4%)	9 (32.1%)	3 (27.3%)	0.059
Obsessive compulsive disorder	3 (1.7%)	0 (0.00%)	1 (3.6%)	2 (18.2%)	0.002
Post-traumatic stress disorder	11 (6.2%)	4 (2.9%)	4 (14.3%)	3 (27.3%)	0.002
Attention Deficit Hyperactivity Disorder	9 (5.1%)	3 (2.2%)	4 (14.3%)	2 (18.2%)	0.005
Bipolar Disorder	6 (3.4%)	1 (0.7%)	4 (14.3%)	1 (9.1%)	0.002
Schizophrenia	0	0	0	0	ND
Alcohol use disorder	8 (4.5%)	0 (0.00%)	2 (7.1%)	6 (54.5%)	<0.001
History of illegal drug use	4 (2.2%)	0 (0.00%)	1 (3.6%)	3 (27.3%)	<0.001
History of prescription drug abuse	2 (1.1%)	0 (0.00%)	0 (0.00%)	2 (18.2%)	0.003
Medication history, n (%)
Pain medication	69 (38.8%)	52 (37.4%)	11 (39.3%)	6 (54.5%)	0.531
Opioid	19 (10.7%)	10 (7.2%)	6 (21.4%)	3 (27.3%)	0.010
Muscle relaxant	9 (5.1%)	6 (4.3%)	1 (3.6%)	2 (18.2%)	0.133
Antidepressant	45 (25.3%)	33 (23.7%)	8 (28.6%)	4 (36.4%)	0.591
Benzodiazepine	20 (11.2%)	10 (7.2%)	8 (28.6%)	2 (18.2%)	0.004
Insomnia medication	6 (3.4%)	5 (3.6%)	1 (3.6%)	0 (0.00%)	>0.999

IQR: interquartile

Outcomes based on scoring on the ORT.

As shown in Table 2, there was no significant difference between the groups in terms of hospital length of stay and number of opioid pills prescribed at discharge. Based on the subjects’ responses to the 30-day follow –up survey, the number of days patients used opioid pills post-surgery was not significantly different between the groups. However, patients who scored high-risk on the ORT were more likely to call the clinic with pain concerns. The number of patients who received prescription refills was not significantly different between the groups. Satisfaction with pain management was high; 88.4% of the subjects rated their pain control as excellent or good. However, patients who scored high on the ORT were more likely to rate their pain management as fair.

Table 2.

Discharge and follow up information based on the ORT scoring.

Variables	Overall n = 178	Low n = 139	Medium n = 28	High n = 11	p-value
Hospital length of stay, days, median (IQR)	1 (1, 1)	1 (1, 1)	1 (1, 1)	1 (1, 1)	0.589
Number of opioid pills prescribed at discharge, median (IQR)	10 (5, 15)	10.0 (5.0, 15.0)	10.0 (0.0, 12.5)	10.0 (10.0, 20.0)	0.244
Opioid morphine equivalent prescribed at discharge, median (IQR)	75 [75–150]	75 [75–150]	75 [75–150]	75 [75–150]	0.764
30-day Follow up
Days of opioid use, median (IQR)	2 (0, 4)	2 (0, 4)	3 (0, 4.5)	2 (0, 6)	0.559
Phone calls for pain, n (%)	24 (13.6%)	11 (8.0%)	9 (32.1%)	4 (36.4%)	<0.001
Received opioid refill, (%)	8 (4.5%)	4 (2.9%)	3 (10.7%)	1 (9.1%)	0.099
Pain management satisfaction, n (%)
Excellent	82 (50.0%)	67 (51.9%)	10 (41.7%)	5 (45.5%)	0.029
Good	63 (38.4%)	51 (39.5%)	8 (33.3%)	4 (36.4%)
Fair	16 (9.8%)	11 (8.5%)	3 (12.5%)	2 (18.2%)
Poor	3 (1.8%)	0 (0.00%)	3 (12.5%)	0 (0.00%)
Disposal of opioids
Disposed of opioids	49 (31.8%)	38 (31.1%)	6 (28.6%)	5 (45.5%)	0.911
Did not dispose of leftover opioids	47 (30.5%)	37 (30.3%)	8 (38.1%)	2 (18.2%)
Used all opioids prescribed after surgery	23 (14.9%)	19 (15.6%)	2 (9.5%)	2 (18.2%)
Did not fill an opioid prescription	35 (22.7%)	28 (23.0%)	5 (23.8%)	2 (18.2%)

IQR: Interquartile

Overall, 22.7% of our subjects reported not filling their opioid prescription, 14.9% used all their opioids, and 31.8% disposed of their opioids following the information provided at discharge. There was no significant difference between the groups.

During the 30-day post discharge phone call, only three opioid-naïve subjects indicated that they were still using opioids. All three patients scored low-risk on the ORT; scores were 1, 3, and 3. Their COMM scores were 1, 3, and 5, note suggestive of opioid misuse.

ORT-OUD

We next used the ORT-OUD and compared patients who scored ≤2 (n = 153) to those who scored ≥3 (n =25). Patient characteristics based on ORT-OUD scoring are presented in Table 3. Patients who scored higher on the ORT-OUD were younger, more likely to be female and on Medicaid insurance. They were more likely to present with comorbidities and to be on antidepressants and/or benzodiazepines.

Table 3.

Patient characteristics based on ORT-OUD scoring.

Variables	ORT-OUD ≤2 n = 153	ORT-OUD ≥3 n = 25	p-value
Age (years), median (IQR)	54.0 (38.0, 66.0)	35.0 (24.0, 51.0)	<0.001
Female, n (%)	65 (42.5%)	17 (68.0%)	0.018
White, n (%)	129 (87.8%)	23 (92.0%)	0.782
Body mass index	27.6 (24.4, 31.6)	28.2 (25, 38.6)	0.212
Insurance, n (%)
Medicaid Insurance	10 (6.5%)	8 (32.0%)	<0.001
Medicare Insurance	45 (29.4%)	3 (12.0%)	0.069
Private insurance	121 (79.1%)	17 (68.0%)	0.218
Military insurance	1 (0.7%)	0 (0.00%)	>0.999
Other insurance	9 (5.9%)	3 (12.0%)	0.380
Comorbidities, n (%)
Chronic pain	33 (21.6%)	6 (24.0%)	0.785
Depression	36 (23.5%)	36 (23.5%)	<0.001
Anxiety	15 (60.0%)	15 (60.0%)	0.004
Obsessive compulsive disorder	22 (14.4%)	22 (14.4%)	0.002
Post-traumatic stress disorder	10 (40.0%)	10 (40.0%)	0.001
Attention Deficit	0 (0.00%)	0 (0.00%)	0.005
Hyperactivity Disorder
Bipolar Disorder	3 (12.0%)	3 (12.0%)	<0.001
Schizophrenia	0	0	ND
Alcohol use disorder	1 (0.7%)	1 (0.7%)	<0.001
History of illegal drug use	7 (28.0%)	7 (28.0%)	0.009
History of prescription drug abuse	1 (0.7%)	1 (0.7%)	0.019
Medication history, n (%)
Pain medication	58 (37.9%)	11 (44.0%)	0.562
Opioid	14 (9.2%)	5 (20.0%)	0.152
Muscle relaxant	6 (3.9%)	3 (12.0%)	0.116
Antidepressant	33 (21.6%)	12 (48.0%)	0.005
Benzodiazepine	12 (7.8%)	8 (32.0%)	0.002
Insomnia medication	5 (3.3%)	1 (4.0%)	>0.999

IQR: Interquartile

Outcomes based on scoring on the ORT-OUD

As shown in Table 4, there was no significant difference between the group in terms of hospital length of stay and number of opioid pills prescribed at discharge. Based on the subjects’ responses to the 30-day follow-up survey, the number of days patients used opioid pills post-surgery was not significantly different between the groups. However, patients who scored ≥3 on the ORT-OUD were more likely to call the clinic with pain concerns. The number of patients who received prescription refills was not significantly different between the groups. Patients who scored ≥3 on the ORT-OUD were less likely to rate their pain management as excellent.

Table 4.

Discharge and follow up information based on the ORT-OUD scoring.

Variables	ORT-OUD ≤2 n = 153	ORT-OUD ≥3 n = 25	p-value
Hospital length of stay, days, median (IQR)	1 (1, 1)	1 (1, 1)	>0.999
Number of opioid pills prescribed at discharge, median (IQR)	10.0 (5.0, 15.0)	10.0 (5.0, 20.0)	0.819
Opioid morphine equivalent prescribed at discharge, median (IQR)	75 [75–150]	75 [66.8–150]	0.995
30-day Follow up
Days of opioid use, median (IQR)	2 (0, 4)	2 (0, 4.5)	0.665
Phone calls for pain, n (%)	15 (9.9%)	9 (36.0%)	0.002
Received opioid refill, (%)	6 (3.9%)	2 (8.0%)	0.315
Pain management satisfaction, n (%)
Excellent	73 (52.1%)	9 (37.5%)	0.019
Good	55 (39.3%)	8 (33.3%)
Fair	16 (9.8%)	11 (7.9%)
Poor	3 (1.8%)	1 (0.7%)
Disposal of opioids
Disposed of opioids	39 (29.8%)	10 (43.5%)	0.590
Did not dispose of leftover opioids	42 (32.1%)	5 (21.7%)
Used all opioids prescribed after surgery	20 (15.3%)	3 (13.0%)
Did not fill an opioid prescription	30 (22.9%)	5 (21.7%)

IQR: Interquartile

DISCUSSION

In this prospective cohort study of opioid-naive patients undergoing elective general surgery, we evaluated the predictive utility of the ORT and ORT-OUD. While higher scores on both tools were associated with increased postoperative pain dissatisfaction such as more frequent pain-related phone calls and lower satisfaction with pain control. Neither tool effectively predicted opioid use at 30 days postoperatively in those not taking opioids before surgery. These findings contribute to a growing body of literature questioning the utility of these tools in surgical populations.

Our results align with prior studies showing limited predictive value of the ORT in perioperative settings. For example, Clark et al. reported that the ORT did not reliably predict aberrant opioid-related behaviors in a tertiary pain management population, raising concerns about its generalizability beyond chronic pain contexts.(7) Similarly, Gong et al. conducted a systematic review and found that persistent opioid use among opioid-naïve surgical patients ranged from 3.9% to 14.0%, with risk factors including mental health comorbidities and higher postoperative opioid exposure (factors not fully captured by the ORT).(16)

More specifically, studies in orthopedic surgery have also highlighted the limitations of the ORT.(16) In a cohort of patients undergoing rotator cuff repair, Westermann et al. found that, while higher ORT scores were associated with increased opioid consumption in the immediate postoperative period, they did not predict persistent use at 90 days.(17) Similarly, a study by Khoury et al. on shoulder arthroplasty patients found that the ORT had poor discrimination for identifying patients at risk for prolonged opioid use, whereas the Screener and Opioid Assessment for Patients with Pain-Revised (SOAPP-R) demonstrated better predictive performance.(8) These findings suggest that tools incorporating behavioral and attitudinal dimensions such as SOAPP-R may offer advantages over the ORT, which focuses primarily on historical risk factors. Tools like the SOAPP-R or machine learning models incorporating broader data inputs, may offer more nuanced risk stratification particularly, in higher-risk surgical populations where prediction is more clinically consequential.

The low rate of prolonged opioid use in our cohort is consistent with approximately three patients recorded at 30 days. Recent large-scale studies indicate that certain elective surgeries carry inherently low-risk for persistent opioid use.(8, 18) Bologheanu et al. found that rates of new persistent opioid use varied significantly by surgery type, with general surgeries such as hernia repair and cholecystectomy associated with lower risk compared to orthopedic or spine procedures.(18) This context is important when interpreting the predictive performance of risk tools: in low-risk surgical populations, even well-calibrated tools may yield low positive predictive value due to the rarity of the outcome. Evaluating the ORT in this context remains important because it continues to be widely used in surgical settings despite limited evidence supporting its utility in populations other than chronic pain populations. Understanding its performance in low-risk procedures such as ours helps clarify whether reliance on the ORT meaningfully contributes to perioperative risk stratification or whether alternative tools may offer greater clinical value. Moreover, assessing the ORT within contemporary surgical cohorts provides critical insight into whether its historical risk-factor focus aligns with the evolving determinants of postoperative opioid use identified in recent literature.

Our analysis also showed that higher ORT and ORT-OUD scores were associated with younger age, Medicaid insurance, a greater prevalence of psychiatric comorbidities, and prior substance use. While these associations are consistent with prior research, it is important to note that psychiatric history and substance use are components of the ORT itself. These correlations may reflect the scoring algorithm rather than independent predictors. Future studies should explore whether individual components of these tools, rather than composite scores, offer better predictive granularity.

Several limitations should be acknowledged. This was a single-center study with a relatively small number of high-risk patients and a low incidence of prolonged opioid use which limited our ability to detect significant associations. Education about opioid risk and some patients receiving opioid disposal packets may have affected the incidence of long-term opioid use. We used the ORT and its OUD-specific variant which are validated for patients with chronic pain to assess their validity in general surgery patients. Both ORT and its OUD variant are weighted tools and the questions on personal history of substance abuse do weigh more than the other questions in the tools, which may have affected our results. Additionally, reliance on self-reported data introduces potential for recall and reporting bias, and participation bias cannot be excluded. Finally, we could not definitely assess whether patients were prescribed opioids by another provider post-discharge.

CONCLUSION

The predictive utility of the ORT and ORT-OUD was evaluated in this prospective cohort study of opioid-naive patients undergoing elective general surgery. While the ORT and ORT-OUD were associated with markers of pain control satisfaction, they did not predict prolonged opioid use or misuse following elective general surgery. Longer-term use among those not taking opioids before surgery was rare in this study, suggesting that research on pre-surgical screening would better focus on populations with chronic pain or undergoing surgeries that more often result in long-term pain and opioid use. Future research should focus on developing and validating surgical-specific risk assessment tools that integrate clinical, behavioral, and social determinants of health.

Supplementary Material

1

Supplementary data 1. 30-day follow up survey

2

Supplementary data 2. 90-day follow up survey

3

Supplementary data 3. Elective surgery information based on the ORT scoring.

DATA SHARING STATEMENT

Drs. Galet and Carnahan have full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.