Peer Comparison or Guideline-Based Feedback and Postsurgery Opioid Prescriptions: A Randomized Clinical Trial

Zachary Wagner; Allison Kirkegaard; Louis T Mariano; Jason N Doctor; Xiaowei Yan; Stephen D Persell; Noah J Goldstein; Craig R Fox; Chad M Brummett; Robert J Romanelli; Kathryn Bouskill; Meghan Martinez; Kyle Zanocco; Daniella Meeker; Satish Mudiganti; Jennifer Waljee; Katherine E Watkins

doi:10.1001/jamahealthforum.2024.0077

. 2024 Mar 15;5(3):e240077. doi: 10.1001/jamahealthforum.2024.0077

Peer Comparison or Guideline-Based Feedback and Postsurgery Opioid Prescriptions

A Randomized Clinical Trial

Zachary Wagner ¹, Allison Kirkegaard ¹, Louis T Mariano ², Jason N Doctor ³, Xiaowei Yan ⁴, Stephen D Persell ⁵, Noah J Goldstein ⁶, Craig R Fox ⁶, Chad M Brummett ⁷, Robert J Romanelli ^4,⁸, Kathryn Bouskill ¹, Meghan Martinez ⁴, Kyle Zanocco ⁹, Daniella Meeker ^10,¹¹, Satish Mudiganti ⁴, Jennifer Waljee ⁷, Katherine E Watkins ^1,^✉

¹RAND Corporation, Santa Monica, California

²RAND Corporation, Arlington, Virginia

³Sol Price School of Public Policy, University of Southern California, Los Angeles

⁴Palo Alto Medical Foundation, Palo Alto, California

⁵Division of General Internal Medicine, Department of Medicine, Center for Primary Care Innovation, Institute for Public Health and Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois

⁶Anderson School of Management, Department of Psychology, and Geffen School of Medicine, University of California at Los Angeles, Los Angeles

⁷University of Michigan Medical School, Ann Arbor

⁸RAND Europe, Westbrook Centre, Cambridge, United Kingdom

⁹Section of Endocrine Surgery, Division of General Surgery, Department of Surgery, David Geffen School of Medicine at UCLA, Los Angeles, California

¹⁰Keck School of Medicine, USC Leonard D. Schaeffer Center for Health Policy & Economics, Los Angeles, California

¹¹Yale School of Medicine, New Haven, Connecticut

Accepted for Publication: December 29, 2023.

Published: March 15, 2024. doi:10.1001/jamahealthforum.2024.0077

^✉

Corresponding Author: Katherine E. Watkins, MD, MSHS, RAND Corporation, 1776 Main St, Santa Monica, CA 90407 (kwatkins@rand.org).

Author Contributions: Dr Wagner had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Wagner, Kirkegaard, Mariano, Doctor, Yan, Goldstein, Fox, Brummett, Bouskill, Martinez, Zanocco, Meeker, Waljee, Watkins.

Acquisition, analysis, or interpretation of data: Wagner, Kirkegaard, Mariano, Doctor, Yan, Persell, Brummett, Romanelli, Martinez, Zanocco, Meeker, Mudiganti, Waljee, Watkins.

Drafting of the manuscript: Wagner, Kirkegaard, Mariano, Yan, Brummett, Romanelli, Watkins.

Critical review of the manuscript for important intellectual content: Wagner, Kirkegaard, Mariano, Doctor, Yan, Persell, Goldstein, Fox, Brummett, Bouskill, Martinez, Zanocco, Meeker, Mudiganti, Waljee, Watkins.

Statistical analysis: Wagner, Mariano, Yan.

Obtained funding: Fox, Bouskill, Meeker, Watkins.

Administrative, technical, or material support: Kirkegaard, Mariano, Goldstein, Brummett, Romanelli, Bouskill.

Supervision: Wagner, Doctor, Yan, Watkins.

Conflict of Interest Disclosures: Dr Kirkegaard reported grants from the National Institutes of Health/National Institute on Drug Abuse (NIH/NIDA) 1R01DA046226 (PI: Katherine Watkins) during the conduct of the study. Dr Mariano reported grants from NIH/NIDA Grant # 1R01DA046226 during the conduct of the study. Dr Doctor reported grants from NIH during the conduct of the study. Dr Persell reported personal fees from RAND during the conduct of the study; personal fees from Omron Healthcare Co., Ltd.; and personal fees from the National Committee for Quality Assurance outside the submitted work. Dr Brummett reported personal fees from Heron Therapeutics, Merck, Vertex Pharmaceuticals, and personal fees from expert testimony outside the submitted work. Dr Martinez reported grants from Sutter Health during the conduct of the study. Dr Zanocco reported grants from NIH, Health and Human Services/NIH/NIDA (Katherine E. Watkins 1R01DA046226) during the conduct of the study. Dr Meeker reported grants from NIH during the conduct of the study. Dr Mudiganti reported grants from NIDA during the conduct of the study. Dr Watkins reported grants from NIDA (Grant # 1R01DA046226) during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was funded by NIDA (Grant Number: 1R01DA046226; PI: Katherine E Watkins).

Role of the Funder/Sponsor: NIDA had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We thank the study Data and Safety Monitoring Board members: Ali Arab, PhD, Georgetown University; Liisa Ecola (Executive Secretary), MA, RAND Corporation; Stephen Rosenfeld (Chair), MD, Indian University; N. Parisa Sadoughi, MD, University of California, Los Angeles; and James Wu, MD, University of California, Los Angeles and the helpful contributions of Tiffany Hruby. None were compensated.

^✉

Corresponding author.

PMCID: PMC10943416 PMID: 38488780

Key Points

Question

Can email feedback on opioid prescribing informed by social norms reduce guideline-discordant postoperative opioid prescribing?

Findings

In this 3-arm cluster-randomized clinical trial that included 640 surgeons, 2 forms of social norm-based email feedback sent to surgeons with postoperative opioid prescriptions above guideline-recommended amounts significantly reduced guideline-discordant prescribing.

Meaning

Email feedback based on social norms is an effective and light-touch intervention for increasing guideline-concordant opioid prescribing after surgery.

This cluster randomized clinical trial assesses the effectiveness of 2 social norm-based interventions on reducing guideline-discordant opioid prescribing after surgery.

Abstract

Importance

Excess opioid prescribing after surgery can result in prolonged use and diversion. Email feedback based on social norms may reduce the number of pills prescribed.

Objective

To assess the effectiveness of 2 social norm-based interventions on reducing guideline-discordant opioid prescribing after surgery.

Design, Setting, and Participants

This cluster randomized clinical trial conducted at a large health care delivery system in northern California between October 2021 and October 2022 included general, obstetric/gynecologic, and orthopedic surgeons with patients aged 18 years or older discharged to home with an oral opioid prescription.

Interventions

In 19 hospitals, 3 surgical specialties (general, orthopedic, and obstetric/gynecologic) were randomly assigned to a control group or 1 of 2 interventions. The guidelines intervention provided email feedback to surgeons on opioid prescribing relative to institutionally endorsed guidelines; the peer comparison intervention provided email feedback on opioid prescribing relative to that of peer surgeons. Emails were sent to surgeons with at least 2 guideline-discordant prescriptions in the previous month. The control group had no intervention.

Main Outcome and Measures

The probability that a discharged patient was prescribed a quantity of opioids above the guideline for the respective procedure during the 12 intervention months.

Results

There were 38 235 patients discharged from 640 surgeons during the 12-month intervention period. Control-group surgeons prescribed above guidelines 36.8% of the time during the intervention period compared with 27.5% and 25.4% among surgeons in the peer comparison and guidelines arms, respectively. In adjusted models, the peer comparison intervention reduced guideline-discordant prescribing by 5.8 percentage points (95% CI, −10.5 to −1.1; P = .03) and the guidelines intervention reduced it by 4.7 percentage points (95% CI, −9.4 to −0.1; P = .05). Effects were driven by surgeons who performed more surgeries and had more guideline-discordant prescribing at baseline. There was no significant difference between interventions.

Conclusions and Relevance

In this cluster randomized clinical trial, email feedback based on either guidelines or peer comparison reduced opioid prescribing after surgery. Guideline-based feedback was as effective as peer comparison-based feedback. These interventions are simple, low-cost, and scalable, and may reduce downstream opioid misuse.

Trial Registration

ClinicalTrials.gov NCT05070338

Introduction

Opioids prescribed after surgery are a significant driver of chronic opioid use and opioid use disorder.^1,2,3,4 Postoperative opioid prescriptions also contribute to unused pills available for diversion to the community^5,6; more than half of the opioids prescribed are never used by the patient and rarely disposed of safely.^7,8,9 High-risk opioid prescribing is concentrated among surgeons who write the most prescriptions, with the top 5% of surgeons in terms of prescribing accounting for 40% of prescriptions written for large quantities.¹⁰ Despite widespread efforts in recent years to match opioid prescribing to patient need, excessive postoperative opioid prescribing has persisted.^8,9

Providing surgeons with feedback on the quantity of opioids they prescribe relative to a social norm may help reduce opioid prescription quantities by challenging what is considered acceptable behavior.^11,12 This could take the form of feedback that describes peer prescribing practices relative to an individual’s own (descriptive social norm) or feedback that provides information about opioid prescription quantities recommended by an authoritative source such as a guideline (injunctive social norm). Although previous randomized clinical trials have shown that social norms can be used to reduce inappropriate antibiotic prescribing among primary care clinicians¹³ and opioid prescribing in emergency departments,¹⁴ there is limited high-quality evidence on whether feedback based on social norms can influence opioid prescribing in the surgical setting, which has a different structure of care and has the second highest opioid prescribing rate of any specialty after pain medicine.^15,16 Moreover, although recent research has examined the influence of either peer-based or guideline-based feedback (or both) on clinician behaviors,^{17,18,19,20,21} little is known regarding how these directly compare to one another.

To address these gaps, this study tests the effectiveness of 2 email feedback interventions relative to usual care across 3 surgical specialties in a multihospital cluster-randomized trial: (1) a descriptive social norm intervention that compares opioid prescription quantities of individual surgeons with that of their peers and (2) an injunctive social norm intervention that compares opioid prescription quantities to guideline recommendations.

Methods

Study Design and Oversight

This was a 3-arm cluster-randomized clinical trial conducted October 19, 2021, to October 18, 2022, in 3 surgical specialties (general, orthopedic, and obstetrics/gynecology) at 19 hospitals within Sutter Health, a large health care system in northern California. Details of the intervention and analysis plan were prespecified in a published protocol and are also available in Supplement 1.²² Activities were approved by the institutional review boards at Sutter Health and the RAND Corporation and monitored by an independent data safety and monitoring board. All data were obtained from the Sutter Health’s electronic health record (EHR) database. Gender and ethnic and racial categories were self-reported by patients; other racial category includes American Indian/Alaska Native, Native Hawaiian/Pacific Islander, other race, and multiple races. This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.

Inclusion and Enrollment

All hospitals within the Sutter Health system as of May 2021 were enrolled in the study. To avoid spillover effects, facilities identified as regularly sharing surgical staff were grouped together, resulting in 19 hospitals for the analysis. Surgeons working exclusively in 1 of the 3 surgical specialties were eligible to participate in the study if they performed any surgical procedures for which opioid prescribing guidelines were available (guidelines described herein) at any of the 19 study hospitals between June 1, 2020, and May 31, 2021. Surgeons were included in the study if they also performed at least 1 surgical procedure for which opioid prescribing guidelines were available during the intervention period.

Opioid Prescribing Guidelines

Both interventions as well as our primary outcome were based on prescribing guidelines developed by teams at the Mayo Clinic.^23,24,25,26 These guidelines recommend procedure-specific ceilings for the total quantity of 5-mg oxycodone tablets needed over the postoperative recovery period (eTable 1 in Supplement 2). We converted these tablet quantities to morphine milligram equivalent (MME) quantities using the Centers for Disease Control and Prevention’s conversion table²⁷ so that we could evaluate a prescription’s guideline compliance regardless of the type or strength of opioid prescribed.

Intervention and Control Conditions

Both interventions provided feedback to surgeons if at least 2 of their patients were discharged with opioid prescriptions larger than the guideline-recommended amount in the prior month. Feedback was sent via emails from and signed by the surgeon’s department chair, chief medical executive, or chief of staff. Surgeons were not explicitly informed they were part of a study. We chose not to obtain informed consent from participants to avoid a Hawthorn effect and a waiver of informed consent was approved by the institutional review boards. However, hospital administrators could opt out of the intervention and surgeons could opt out of emails.

Although the content of the emails was distinct, the trigger for sending an email was identical: a surgeon having at least 2 patients with eligible discharge opioid prescriptions over guideline recommendations in the previous month. Eligible discharge prescriptions were those for adults discharged to home after a single surgical procedure with an applicable postoperative prescribing guideline, and where the prescription was for an oral medication. The email was sent to the operating surgeon even if the surgeon did not write the discharge prescriptions; 44% of discharge opioid prescriptions were written by someone other than the operating surgeon (usually a hospitalist) although the operating surgeon discharged the patient 84% of the time.

Peer Comparison (Descriptive Norm) Intervention

The subject line for the peer comparison email read “Your peers vs your opioid prescribing safety record.” The email content informed surgeons that their patients were prescribed opioid quantities exceeding the amounts prescribed by more than a particular percentage of their peers (eFigure 1 in Supplement 2). The peer percentage was calculated as the share of surgeons at Sutter Health in their specialty who had fewer than 2 patients discharged with opioid quantities exceeding guidelines in the past month, and ranged from 28% to 87% (mean, 61%) during the study period. The email also listed the procedures for which they prescribed higher quantities than their peers and the range prescribed by their peers for each procedure.

Guideline Feedback (Injunctive Norm) Intervention

The subject line of the guideline feedback email read “Best practice guidelines vs your opioid prescribing safety record.” The email content informed the surgeon that their patients were prescribed opioid quantities “exceeding the amounts recommended by safety guidelines for these procedures” (eFigure 2 in Supplement 2). The email also listed the procedures for which this occurred in the prior month and the guideline range for each procedure.

Control Condition

Surgeons in the control arm did not receive any emails and were not informed that they were being studied. We made this decision to avoid Hawthorn effects.

Implementation

Each month, intervention emails were automatically created and sent to eligible surgeons via an email management platform. Sutter Health staff were responsible ensuring emails were sent out correctly.

Randomization

Randomization occurred at the level of the surgical specialty. At each hospital, 1 specialty was randomized to each arm, stratified by hospital size and specialty. The allocation of specialties within hospitals to study arms was drawn using the sample function in R statistical software²⁸ (version 4.3.2, R Foundation) applied to each block. We chose to randomize by specialty within hospital rather than at the hospital level to maximize the number of randomization clusters, which increases statistical power, and to avoid chance imbalance which is more likely with fewer randomization units. After randomization but before the start of the intervention, 1 hospital-specialty combination refused to participate in the peer comparison intervention owing to a hospital administrator’s concerns. To preserve the integrity of the randomized design, we treat the 43 discharges (<1% of sample) from this group as peer comparison although they did not receive the intervention.

Outcome Measures

The primary prespecified outcome was the probability that a discharged patient was prescribed a quantity of opioids above the guideline for the respective procedure during the 12 intervention months. We extracted all initial discharge prescriptions from the EHR and defined a prescription as being above guidelines if the total MME quantity of opioids prescribed was above the ceiling of the guideline-recommended range. Surgeries where no opioids were prescribed were considered within guidelines.

We analyzed 2 prespecified secondary opioid prescribing outcomes: (1) the MME quantity prescribed at discharge, coded as zero when no opioid was prescribed and truncated at 500 MMEs (the 99th percentile), and (2) the probability that a discharged patient received any opioid prescription.

We also analyzed 3 prespecified outcomes that assess potential harmful effects of the interventions: (1) whether the patient got an additional opioid prescription filled in the 30 days after the surgery, (2) whether the patient had an emergency department visit in the 30 days after the surgery, and (3) whether the patient was hospitalized in the 30 days after the surgery.

Statistical Analysis

We conducted 3 prespecified analyses.²² In each, we estimated intervention effects using the surgeon’s treatment assignment, regardless of whether the surgeon received an email during the study period. The first set of analyses modeled effects on primary and secondary outcomes at the level of the surgical specialty using a hierarchical linear model (HLM)²⁹ with random effects for the surgeon and specialty within hospital and controlling for the baseline levels of the outcome (averaged by surgeon in the 24 months prior to the intervention). We included a version of the results controlling for a more comprehensive set of covariates in the eAppendix in Supplement 2. We used a logit-link for binary outcomes and reported absolute risk differences using recycled predictions.³⁰ The second analysis estimated intervention effects over time by using separate HLMs for each of the 12 study months. The third set of analyses estimated effects for prespecified subgroups of surgeons based on 3 surgeon characteristics: (1) effects by surgical procedure volume during the intervention period, (2) effects by baseline guideline-discordant opioid prescribing in the 24 months before the intervention, and (3) effects by surgical specialty.

We also conducted 2 exploratory analyses. First, we estimated average effects at the level of the discharge using ordinary least squares (OLS) adjusted for the baseline level of the outcome and with standard errors clustered by randomization cluster.³¹ This discharge-level model weights each discharge equally and captures the average effect per discharged patient. Conversely, discharges from higher-volume surgeons tend to be downweighted in the HLM owing to their tendency toward higher variation in the outcome.³² Second, we used quantile regression to estimate the effect of the interventions on each decile of MMEs prescribed to determine whether findings were robust with a nonnormal distribution of MMEs prescribed.

The study was powered to detect a difference of 12 percentage points between each study arm and the control group. Baseline outcome measures were more strongly correlated with the outcome than the original conservative estimate, yielding additional power.²² All statistical analyses are described in more detail in the eAppendix in Supplement 2.

Multiple Hypothesis Testing

We used false discovery rate (FDR) corrections to account for simultaneously tested hypotheses of the effect of both treatments.³³ All reported P values have been adjusted for an FDR of .05.

Results

Of the 778 surgeons eligible for inclusion in the study, 640 surgeons had 38 235 eligible discharges during the intervention period (Figure 1). Surgeon and patient characteristics were mostly balanced across arms (Table 1). The 3 arms were also balanced on the share of prescriptions above guideline quantities (primary outcome) and the share of discharges with any opioid prescribing in the 24 months leading up to the intervention. Although the guidelines arm had higher mean MMEs at baseline (100) relative to the other 2 arms (70 in control and 75 in peer comparison), median (IQR) MMEs were more balanced: 61 in guidelines (31-134), 57 in control (33-92), and 72 in peer comparison (35-99). More than 50% of total joint replacement procedures at baseline occurred in clusters assigned to guidelines, driving the guideline mean MME higher.

Table 1. Sample Characteristics^a.

Characteristic	No. (%)
Characteristic	Control	Peer comparison	Guidelines
Total hospitals, No.	19	19	19
Total surgeons, No.	217	207	216
Total patient discharges, No.	12 024	10 752	15 459
Surgeon characteristics
General specialty	51 (23.5)	55 (27.0)	50 (23.1)
OB/GYN specialty	108 (49.7)	98 (47.3)	91 (42.1)
Orthopedic specialty	58 (26.7)	53 1 (25.6)	75 (34.7)
Years since receiving degree, mean (SD)	25.1 (10.9)	26.0 (10.6)	24.6 (11.9)
Gender
Female	87 (40.0)	87 (42.0)	83 (38.4)
Male	122 (56.2)	109 (52.7)	128 (59.3)
Unknown	8 (3.7)	11 (5.3)	5 (2.3)
Patient characteristics
Age, mean (SD), y	49.4 (17.6)	50.1 (17.3)	53.7 (17.7)
Medicaid beneficiary	1414 (11.7)	1475 (13.7)	2254 (14.5)
Length of stay, mean (SD), h	37.9 (58.1)	38.7 (53.5)	31.4 (56.4)
Body mass index, mean (SD)^b	30.8 (7.93)	30.0 (7.32)	29.7 (6.95)
Pain condition	5948 (49.4)	5530 (51.4)	9159 (59.2)
Chronic opioid use	868 (7.21)	1219 (11.3)	1682 (10.8)
Opioid use past year	1873 (15.5)	1763 (16.3)	2853 (18.4)
Opioid use past month	1809 (15.0)	2439 (22.6)	3257 (21.0)
Opioid use past 24 h	3130 (26.0)	2808 (26.1)	4238 (27.4)
Gender^c
Female	9180 (76.3)	8383 (77.9)	10 577 (68.4)
Male	2839 (23.6)	2367 (22.0)	4878 (31.5)
Nonbinary	2 (0.016)	1 (0.009)	4 (0.025)
Race and ethnicity^c
Asian	1451 (12.0)	1527 (14.2)	1631 (10.5)
Black	949 (7.89)	450 (4.19)	764 (4.94)
White	6642 (55.2)	6554 (60.9)	9703 (62.7)
Other	2982 (24.8)	2220 (20.6)	3361 (21.7)
Outcomes at baseline (over 24 mo prior to intervention)
Prescriptions above guidelines^c	4491 (37.3)	4066 (37.8)	5516 (35.6)
MMEs, mean (SD)^d	70.2 (59.2)	75.4 (51.9)	100.6 (97.1)
MMEs, median (range)	57.1 (0-338)	72.1 (0-500)	60.8 (0-447)
Any opioid prescription	6117 (50.8)	5512 (51.2)	8082 (52.2)

Open in a new tab

Abbreviations: MME, morphine milligram equivalent; OB/GYN, obstetrics and gynecology.

^{^a}

Data are from Sutter Health’s electronic health records.

^{^b}

Calculated as weight in kilograms divided by height in meters squared.

^{^c}

Gender and ethnic and racial categories were self-reported by patients; other racial category includes American Indian/Alaska Native, Native Hawaiian/Pacific Islander, other race, and multiple races.

^{^d}

The quantity of MMEs was coded as zero when no opioids were prescribed. Continuous variables show mean (SD).

Intervention Effects on Above-Guideline Opioid Prescriptions

In the 12 months prior to the intervention, the share of discharges with an opioid prescription above guideline quantities was around 35% to 40% in all 3 arms (Figure 2A). During the intervention period, above guideline prescribing was 27.5% on average in the peer comparison arm and 25.4% in the guidelines arm compared with 36.8% in the control arm (Table 2). In adjusted hierarchical models (Table 2), the peer comparison intervention reduced the share of discharges with prescriptions above guideline quantities by 5.8 percentage points (95% CI, −10.5 to −1.1; P = .03) and the guidelines intervention reduced it by 4.7 percentage points (95% CI, −9.4 to −0.1; P = .05) compared with the control group. These constitute 17% and 14% relative reductions. Effect sizes were not significantly different between the 2 intervention arms. Figure 2B shows that effects became more pronounced over time in both arms, with 9 to 13 percentage point reductions in each of the past 4 months. Results were similar when regressions controlled for a more comprehensive set of covariates (eTable 3 in Supplement 2).

Figure 2. — A, The share of encounters with an opioid prescription at discharge exceeding guideline-recommended amounts. B, The effect of the intervention in each study month estimated using 12 separate hierarchical logistic models with random effects for surgeon and specialty within hospital. Intervention effects are absolute percentage point differences. Regression results for panel B are reported in eTable 2 in Supplement 2.

Table 2. Effect of Social Norms-Based Nudges on Primary and Secondary Outcomes^a.

Variable	Mean of outcome during 12 intervention months			Effect of peer comparison intervention, percentage points				Effect of guidelines intervention, percentage points
Variable	Control	Peer comparison	Guidelines	Unadjusted (95% CI)	P value	Adjusted (95% CI)	P value	Unadjusted (95% CI)	P value	Adjusted (95% CI)	P value
Primary outcome
Opioid prescriptions above guideline quantities, %	36.8	27.5	25.4	−9.3 (−20.5 to 1.9)	.11	−5.8 (−10.5 to −1.1)	.03	−11.4 (−22.2 to −0.7)	0.08	−4.7 (−9.4 to −0.1)	.047
Secondary outcomes
Morphine milligram equivalent, mean	67.7	58.4	78.7	−9.3 (−32.4 to 13.8)	.53	−4.6 (−18.8 to 9.6)	.94	10.9 (−23.0 to 44.8)	.53	−0.5 (−14.4 to 13.4)	.94
Any opioid prescribed, %	52.0	47.4	48.4	−4.6 (−16.7 to 7.5)	.55	0.4 (−4.1 to 4.8)	.90	−3.6 (−15.6 to 8.3)	.55	1.5 (−2.7 to 5.7)	.90

Open in a new tab

^{^a}

Unadjusted models compare means between each intervention arm and the control arm with standard errors clustered by specialty within hospital. Adjusted models include random effects for surgeon and specialty within hospital and control for baseline level of the outcome at the surgeon level. P values are adjusted for multiple testing using a false discovery rate of .05.

The effects among surgeons who performed above the median number of procedures during the intervention period were 8.8 and 9.5 percentage point reductions for peer comparison and guideline interventions, respectively (P < .001 for both; eTables 4 and 5 in Supplement 2; Figure 3), compared with 3.4 and 1.0 among surgeons with below-median numbers of procedures (P = .73 and P = .80, respectively; P values on difference in effect were .21 and .10, respectively). The effects among surgeons with above-median guideline-discordant prescribing at baseline were 10.6 and 10.9 percentage point reductions for peer comparison and guideline interventions, respectively (P = .005 for both), compared with 1.8 and 0.8 among surgeons with below-median guideline-discordant prescribing (P = .64 for both; P value on difference in effect was .03 for both). The effect was not significantly different across specialties (eTable 4 in Supplement 2).

Figure 3. — Intervention effects are shown in absolute percentage point differences. Each point estimate is the intervention effect from a separate hierarchal model including a logit link and random effects for surgeon and specialty within hospital and controlling for each surgeon’s share of discharges with guideline-discordant opioid prescriptions in the 24 months prior to the intervention. Subgroup categories are based on quartiles of each subgroup variable (described in more detail in the eAppendix in Supplement 2). Full regression results are reported in eTable 3 in Supplement 2. Effects sizes are significantly different between the top 2 and the bottom 2 categories for baseline prescribing but not for procedure volume (eTable 4 in Supplement 2). The whiskers indicate 95% CIs.

In models analyzed at the discharge level, there was an absolute 9.7–percentage point reduction in guideline-discordant prescribing in the peer-comparison arm (95% CI, −14.5 to −4.9; P < .001) and a 10.1 percentage point reduction in the guidelines arm (95% CI, −15.6 to −4.7; P = .001) relative to the control group (eTable 6 in Supplement 2).

Intervention Effects on MMEs Prescribed

During the intervention period, patients of surgeons in the control arm were prescribed 67.7 MMEs per discharge on average compared with 58.4 in the peer comparison arm and 78.7 in the guidelines arm (Table 2). Adjusted hierarchical models estimated nonsignificant reductions of 4.6 MMEs in the peer comparison arm (95% CI, −18.8 to 9.6; P = .94) and 0.5 MMEs in the guidelines arm (95% CI, −14.4 to 13.4; P = .94) compared with the control group. The discharge-level model estimated a 12.4 MME reduction in the peer comparison arm (95% CI, −24.2 to −0.6; P = .07) and a 12.2 MME reduction in the guidelines arm (95% CI, −25.2 to 0.80; P = .07; eTable 6 in Supplement 2). Quantile regressions estimated reductions at the median of 14.1 MMEs (95% CI, −21.0 to −7.3; P < .001) in the peer comparison arm and 13.8 in the guidelines arm (95% CI, −20.8 to −6.7; P < .001) compared with the control group, and reductions were larger at the higher end of the distribution (eFigure 3 in Supplement 2).

Intervention Effects on the Probability of an Opioid Being Prescribed

Overall, 52% of discharges during the intervention period in the control group had any opioid prescription compared with 48% in both intervention arms. Adjusted hierarchical models estimated no significant differences between either intervention arm relative to the control group on the probability of prescribing any opioid (Table 2). The discharge-level model estimated a 4.9–percentage point reduction in any opioid prescribed at discharge in each intervention arm (peer comparison: 95% CI, −8.8 to −1.1; P = .02; guidelines: 95% CI, −9.0 to −0.8; P = .02; eTable 6 in Supplement 2).

Intervention Effects on Patient Outcomes

We found no significant differences in the probability of getting an additional opioid prescription filled, having an emergency department visit, or being hospitalized in the 30 days after the discharge (eTable 7 in Supplement 2).

Sensitivity Analyses

We conducted 2 sensitivity analyses. First, because only 56% of surgeons received an email during the study period, accounting for 81% of discharges (see eFigure 4 in Supplement 2), we conducted a sensitivity analysis where we restricted to surgeons who received an email at least once during the intervention period. Results were more pronounced for this subgroup (eTable 8 in Supplement 2). Second, we estimated the intervention effects restricting to only surgeons who wrote the opioid prescription, if an opioid was prescribed, or discharged the patient themselves, if an opioid was not prescribed. Effect sizes for this subgroup were similar to the main results (eTable 8 in Supplement 2).

Discussion

Principal Findings

Despite federal and state efforts, opioid overprescribing remains common in our setting, with 37% of postsurgical discharge prescriptions being above guideline quantities in our cohort before the intervention. Email feedback informing surgeons their patients were prescribed opioid amounts that exceeded institutionally endorsed guidelines or their peers’ prescriptions reduced guideline-discordant prescribing by about 5 percentage points relative to usual care. Effect sizes grew stronger over time (approximately a 10–percentage point reduction in the final 4 months of the intervention). These interventions were light-touch and incured minimal ongoing costs after implementation, yet were associated with clinically meaningful improvements in opioid prescribing. Both interventions could be scaled to any hospital system with an EHR that recorded opioid prescriptions at patient discharge.

Implications for Public Health

A 5–percentage point reduction in guideline-discordant opioid prescribing could be important for public health outcomes. However, our prespecified analyses did not find significant effects on MMEs prescribed at discharge, which suggest the public health effects could be limited. In follow-up analysis, we found larger intervention effects (10–percentage point reductions) when we modeled effects at the discharge level (eTable 6 in Supplement 2) compared with the prespecified analyses, which included effects at the surgical specialty level. We also found significant reductions in MMEs prescribed and any opioids prescribed using the discharge-level model. This is because intervention effects were larger for surgeons with more discharges and discharges from higher-volume surgeons tend to be downweighted in the prespecified analyses.³⁴ Moreover, high-volume surgeons contribute disproportionately to overprescribing.¹⁰ Thus, discharge-level analyses may be more relevant for assessing public health effects.

With 26 193 discharges in the intervention arms over the study year, estimates from the discharge-level model of a 12 MME reduction in opioids prescribed per discharged patient corresponded to 319 554 fewer MMEs or the equivalent of about 42 000 fewer pills of 5-mg oxycodone in Sutter Health patients’ communities over the study period.

Implications for Scale-Up

Importantly, we found the guideline intervention was as successful as the peer comparison intervention. Going forward, guideline-based interventions to reduce excessive postoperative opioid prescribing may be preferable to peer-based interventions for several reasons. First, peer-comparison interventions are more challenging to implement because they require regularly updating feedback with current peer behavior. Second, although research suggests clinicians find peer comparison feedback helpful,¹⁸ some administrators in this study expressed concerns about peer comparison feedback. Third, guideline-based feedback can be implemented in smaller hospital systems where there are too few peers for a rich comparison.

Comparison With Other Studies

To our knowledge, this is the first study to examine the effect of feedback interventions on postoperative opioid prescribing using a randomized design, and one of the first studies to compare the effectiveness of 2 types of social norms on clinician behavior—peer-comparison descriptive norms and guideline-based injunctive norms. Prior efforts to change postoperative prescribing with behavioral interventions have used pretest-posttest designs and have shown reductions in the number of tablets prescribed.^{15,35,36,37,38,39} The present work is consistent with trials testing clinician feedback interventions on opioid prescribing in emergency department, primary care, and urgent care sites,^{14,17,40,41,42,43} although 2 other trials examining clinician feedback on concurrent prescribing of opioids and benzodiazepines did not show reductions^40,44 in concurrent prescribing. The finding that both the peer-based descriptive norm and guideline-based injunctive norm treatment conditions were equally effective contributes to an ongoing debate from the broader norms literature about which type of norm—descriptive or injunctive—is more likely to exert an effect in a fast-paced environment in which there is a great deal of cognitive depletion.^45,46

Limitations

This study has several limitations. First, we could not disentangle the effects of the social norms mechanisms from the effects of simply having one’s prescribing monitored. Of course, monitoring effects would also be present in any clinical version of the intervention (part of the intervention was informing surgeons that their opioid prescribing was being monitored), so this did not bias our estimates of overall intervention effects. Second, we did not assess patient outcomes such as self-reported pain levels, daily functioning, or quality of life. However, other work has not shown a relationship between reduced postoperative prescriptions and adverse patient outcomes,^{39,47,48,49,50,51} and patients in the intervention arms were not more likely to have an emergency department visit, an inpatient visit, or additional opioid prescription fill following surgery. Third, although we used a clustered design to avoid spillover, surgeons could talk across specialties within hospitals, which could have resulted in contamination and thus understated intervention effects. However, the lack of reduction in the control group suggests that the design prevented substantial spillover effects. Finally, the email was sent to the operating surgeon even when they did not make the discharge prescription. In some cases, surgeons might not have control over prescribing at discharge, and negative feedback for these surgeons could decrease receptivity to other institutional goals. This design choice was intended to avoid spillover (nonsurgeon prescribers are more likely to prescribe across specialties) and any scaled-up version of this intervention would likely send emails directly to the clinicians who make the discharge prescription.

Conclusions

This 3-arm cluster-randomized clinical trial found that, among surgeons specializing in orthopedic, general, and obstetric/gynecological surgery, both email feedback informing surgeons that they ordered opioid prescriptions above guideline-recommended amounts and email feedback comparing their prescribing to that of their peers reduced guideline-discordant opioid prescribing compared with receiving no feedback.

Supplement 1.

Trial Protocol

jamahealthforum-e240077-s001.pdf^{(490.4KB, pdf)}

Supplement 2.

eAppendix

eTable 1. Prescribing guidelines for procedures included in the present study

eTable 2. Intervention impacts over time (full output from Figure 2B in main text)

eTable 3. Intervention effects with full set of covariates included

eTable 4. Subgroup analyses of intervention impacts on guideline-discordant prescribing

eTable 5. Tests for differences in intervention impacts by subgroup

eTable 6. Effect of social norms-based nudges on primary and secondary outcomes (discharge level)

eTable 7. Intervention effects on patient outcomes

eTable 8. Sensitivity analyses using surgeons who were more exposed to the intervention

eFigure 1. Example of Peer Comparison Intervention Email

eFigure 2. Example of Guidelines Intervention Email

eFigure 3. Effect of social norms-based nudges on each quantile of MMEs prescribed at discharge

eFigure 4. Exposure to interventions

jamahealthforum-e240077-s002.pdf^{(635.5KB, pdf)}

Supplement 3.

Data Sharing Statement

jamahealthforum-e240077-s003.pdf^{(16.5KB, pdf)}

References

1.Brummett CM, Waljee JF, Goesling J, et al. New persistent opioid use after minor and major surgical procedures in U.S. adults. JAMA Surg. 2017;152(6):e170504. doi: 10.1001/jamasurg.2017.0504 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Compton WM, Jones CM, Baldwin GT. Relationship between nonmedical prescription-opioid use and heroin use. N Engl J Med. 2016;374(2):154-163. doi: 10.1056/NEJMra1508490 [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Lipari RN, Hughes A. How people obtain the prescription pain relievers they misuse. The CBHSQ Report. Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; January 12, 2017. https://www.samhsa.gov/data/sites/default/files/report_2686/ShortReport-2686.html [PubMed]
4.Makary MA, Overton HN, Wang P. Overprescribing is major contributor to opioid crisis. BMJ. 2017;359:j4792. doi: 10.1136/bmj.j4792 [DOI] [PubMed] [Google Scholar]
5.Agniel D, Brat GA, Marwaha JS, et al. Association of postsurgical opioid refills for patients with risk of opioid misuse and chronic opioid use among family members. JAMA Netw Open. 2022;5(7):e2221316. doi: 10.1001/jamanetworkopen.2022.21316 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Griesler PC, Hu MC, Wall MM, Kandel DB. Assessment of prescription opioid medical use and misuse among parents and their adolescent offspring in the US. JAMA Netw Open. 2021;4(1):e2031073. doi: 10.1001/jamanetworkopen.2020.31073 [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Hill MV, McMahon ML, Stucke RS, Barth RJ Jr. Wide variation and excessive dosage of opioid prescriptions for common general surgical procedures. Ann Surg. 2017;265(4):709-714. [DOI] [PubMed] [Google Scholar]
8.Neuman MD, Bateman BT, Wunsch H. Inappropriate opioid prescription after surgery. Lancet. 2019;393(10180):1547-1557. doi: 10.1016/S0140-6736(19)30428-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Schirle L, Stone AL, Morris MC, et al. Leftover opioids following adult surgical procedures: a systematic review and meta-analysis. Syst Rev. 2020;9(1):139. doi: 10.1186/s13643-020-01393-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Waljee JF, Gunaseelan V, Bicket MC, Brummett CM, Chua KP. Safety and distribution of opioid prescribing by US surgeons. Ann Surg. 2023;10:1097. doi: 10.1097/SLA.0000000000005802 [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Cialdini RB, Kallgren CA, Reno RR. A focus theory of normative conduct: A theoretical refinement and reevaluation of the role of norms in human behavior. In: Zanna MP, ed. Advances in Experimental Social Psychology Cambridge. Academic Press; 1991. doi: 10.1016/S0065-2601(08)60330-5 [DOI] [Google Scholar]
12.Goldstein NJ, Cialdini RB. Using social norms as a lever of social influence. In: Pratkanis AR, ed. The science of social influence: advances and future progress. Psychology Press; 2011:167-191. doi: 10.4324/9780203818565-7 [DOI] [Google Scholar]
13.Zeng Y, Shi L, Liu C, et al. Effects of social norm feedback on antibiotic prescribing and its characteristics in behaviour change techniques: a mixed-methods systematic review. Lancet Infect Dis. 2022;23(5):e175-e184. [DOI] [PubMed] [Google Scholar]
14.Navathe AS, Liao JM, Yan XS, et al. The effect of clinician feedback interventions on opioid prescribing. Health Aff (Millwood). 2022;41(3):424-433. doi: 10.1377/hlthaff.2021.01407 [DOI] [PubMed] [Google Scholar]
15.Zhang DDQ, Sussman J, Dossa F, et al. A systematic review of behavioral interventions to decrease opioid prescribing after surgery. Ann Surg. 2020;271(2):266-278. doi: 10.1097/SLA.0000000000003483 [DOI] [PubMed] [Google Scholar]
16.Levy B, Paulozzi L, Mack KA, Jones CM. Trends in opioid analgesic–prescribing rates by specialty, US, 2007–2012. Am J Prev Med. 2015;49(3):409-413. doi: 10.1016/j.amepre.2015.02.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Candon M, Xue L, Shen S, Cole ES, Donohue J, Rothbard A. The impact of opioid prescribing report cards in Medicaid. J Manag Care Spec Pharm. 2022;28(8):862-870. doi: 10.18553/jmcp.2022.28.8.862 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Klaiman T, Nelson MN, Yan XS, et al. Clinician perceptions of receiving different forms of feedback on their opioid prescribing. Am J Med Qual. 2023;38(1):1-8. doi: 10.1097/JMQ.0000000000000092 [DOI] [PubMed] [Google Scholar]
19.Navathe AS, Volpp KG, Bond AM, et al. Assessing the effectiveness of peer comparisons as a way to improve health care quality. Health Aff (Millwood). 2020;39(5):852-861. doi: 10.1377/hlthaff.2019.01061 [DOI] [PubMed] [Google Scholar]
20.Sacarny A, Barnett ML, Le J, Tetkoski F, Yokum D, Agrawal S. Effect of peer comparison letters for high-volume primary care prescribers of quetiapine in older and disabled adults: a randomized clinical trial. JAMA Psychiatry. 2018;75(10):1003-1011. doi: 10.1001/jamapsychiatry.2018.1867 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Sedhom R, Blackford AL, Gupta A, Smith TJ, Shulman LN, Carducci MA. Oncologist peer comparisons as a behavioral science strategy to improve hospice utilization. JCO Oncol Pract. 2022;18(7):e1122-e1131. doi: 10.1200/OP.21.00738 [DOI] [PubMed] [Google Scholar]
22.Kirkegaard A, Wagner Z, Mariano LT, et al. Evaluating the effectiveness of email-based nudges to reduce postoperative opioid prescribing: study protocol of a randomised controlled trial. BMJ Open. 2022;12(9):e061980. doi: 10.1136/bmjopen-2022-061980 [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Glaser GE, Kalogera E, Kumar A, et al. Outcomes and patient perspectives following implementation of tiered opioid prescription guidelines in gynecologic surgery. Gynecol Oncol. 2020;157(2):476-481. doi: 10.1016/j.ygyno.2020.02.025 [DOI] [PubMed] [Google Scholar]
24.Thiels CA, Ubl DS, Yost KJ, et al. Results of a prospective, multicenter initiative aimed at developing opioid-prescribing guidelines after surgery. Ann Surg. 2018;268(3):457-468. doi: 10.1097/SLA.0000000000002919 [DOI] [PubMed] [Google Scholar]
25.Wyles CC, Hevesi M, Ubl DS, et al. Implementation of procedure-specific opioid guidelines: a readily employable strategy to improve consistency and decrease excessive prescribing following orthopaedic surgery. JB JS Open Access. 2020;5(1):e0050. doi: 10.2106/JBJS.OA.19.00050 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Wyles CC, Hevesi M, Trousdale ER, et al. The 2018 Chitranjan S. Ranawat, MD Award: developing and implementing a novel institutional guideline strategy reduced postoperative opioid prescribing after TKA and THA. Clin Orthop Relat Res. 2019;477(1):104-113. doi: 10.1007/s11999.0000000000000292 [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Centers for Disease Control and Prevention . Table: CDC oral morphine milligram equivalents, September. Accessed April 14, 2023. https://www.cdc.gov/opioids/data-resources/index.html
28.R Core Team . R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Accessed April 17, 2023. https://www.R-project.org/
29.Bryk AS, Raudenbush SW. Hierarchical linear models: applications and data analysis methods. Sage Publications, Inc; 1992. [Google Scholar]
30.Graubard BI, Korn EL, Korn E. Predictive margins with survey data. Biometrics. 1999;55(2):652-659. doi: 10.1111/j.0006-341X.1999.00652.x [DOI] [PubMed] [Google Scholar]
31.Abadie A, Athey S, Imbens GW, Wooldridge J. When should you adjust standard errors for clustering? National Bureau of Economic Research; 2017. doi: 10.3386/w24003 [DOI] [Google Scholar]
32.Association of American Medical Colleges . Physician Specialty Data Report: Number of People per Active Physician by Specialty, 2021. Accessed April 14, 2023. https://www.aamc.org/data-reports/workforce/data/number-people-active-physician-specialty-2021
33.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57(1):289-300. doi: 10.1111/j.2517-6161.1995.tb02031.x [DOI] [Google Scholar]
34.Kahan BC, Li F, Copas AJ, Harhay MO. Estimands in cluster-randomized trials: choosing analyses that answer the right question. Int J Epidemiol. 2023;52(1):107-118. doi: 10.1093/ije/dyac131 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Billings JD, Huynh V, Leonard LD, et al. Addressing an epidemic: improving guideline-concordant opioid prescribing in surgical patients. Surgery. 2022;172(5):1407-1414. doi: 10.1016/j.surg.2022.06.033 [DOI] [PubMed] [Google Scholar]
36.Einarsson A, Chiu AS, Mori M, et al. Changing the default option in electronic medical records reduced postoperative opioid prescriptions after cardiac surgery. JTCVS Open. 2021;8:467-474. doi: 10.1016/j.xjon.2021.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Chiu AS, Jean RA, Hoag JR, Freedman-Weiss M, Healy JM, Pei KY. Association of lowering default pill counts in electronic medical record systems with postoperative opioid prescribing. JAMA Surg. 2018;153(11):1012-1019. doi: 10.1001/jamasurg.2018.2083 [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Chua KP, Thorne MC, Ng S, Donahue M, Brummett CM. Association between default number of opioid doses in electronic health record systems and opioid prescribing to adolescents and young adults undergoing tonsillectomy. JAMA Netw Open. 2022;5(6):e2219701. doi: 10.1001/jamanetworkopen.2022.19701 [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Vu JV, Howard RA, Gunaseelan V, Brummett CM, Waljee JF, Englesbe MJ. Statewide implementation of postoperative opioid prescribing guidelines. N Engl J Med. 2019;381(7):680-682. doi: 10.1056/NEJMc1905045 [DOI] [PMC free article] [PubMed] [Google Scholar]
40.Sacarny A, Avilova T, Powell D, Williamson I, Merrick W, Jacobson M. A randomized trial of letters to encourage prescription monitoring program use and safe opioid prescribing: study examines effectiveness of letters to clinicians that encourage prescription monitoring program use and safe opioid prescribing. Health Aff (Millwood). 2023;42(1):140-149. doi: 10.1377/hlthaff.2022.00859 [DOI] [PubMed] [Google Scholar]
41.Doctor JN, Nguyen A, Lev R, et al. Opioid prescribing decreases after learning of a patient’s fatal overdose. Science. 2018;361(6402):588-590. doi: 10.1126/science.aat4595 [DOI] [PubMed] [Google Scholar]
42.Kraemer KL, Althouse AD, Salay M, et al. Effect of different interventions to help primary care clinicians avoid unsafe opioid prescribing in opioid-naive patients with acute noncancer pain: a cluster randomized clinical trial. JAMA Health Forum. 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Michael SS, Babu KM, Androski C Jr, Reznek MA. Effect of a data-driven intervention on opioid prescribing intensity among emergency department providers: a randomized controlled trial. Acad Emerg Med. 2018;25(5):482-493. doi: 10.1111/acem.13400 [DOI] [PubMed] [Google Scholar]
44.Sacarny A, Safran E, Steffel M, et al. Effect of pharmacist email alerts on concurrent prescribing of opioids and benzodiazepines by prescribers and primary care managers: a randomized clinical trial. JAMA Health Forum. 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Jacobson RP, Mortensen CR, Cialdini RB. Bodies obliged and unbound: differentiated response tendencies for injunctive and descriptive social norms. J Pers Soc Psychol. 2011;100(3):433-448. doi: 10.1037/a0021470 [DOI] [PubMed] [Google Scholar]
46.Melnyk V. Herpen Ev, Fischer AR, van Trijp HC. To think or not to think: the effect of cognitive deliberation on the influence of injunctive versus descriptive social norms. Psychol Mark. 2011;28(7):709-729. doi: 10.1002/mar.20408 [DOI] [Google Scholar]
47.Bateman BT, Cole NM, Maeda A, et al. Patterns of opioid prescription and use after cesarean delivery. Obstet Gynecol. 2017;130(1):29-35. doi: 10.1097/AOG.0000000000002093 [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Hill MV, Stucke RS, McMahon ML, Beeman JL, Barth RJ Jr. An educational intervention decreases opioid prescribing after general surgical operations. Ann Surg. 2018;267(3):468-472. doi: 10.1097/SLA.0000000000002198 [DOI] [PubMed] [Google Scholar]
49.Lee JS, Howard RA, Klueh MP, et al. The impact of education and prescribing guidelines on opioid prescribing for breast and melanoma procedures. Ann Surg Oncol. 2019;26(1):17-24. doi: 10.1245/s10434-018-6772-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
50.Lee JS, Hu HM, Brummett CM, et al. Postoperative opioid prescribing and the pain scores on hospital consumer assessment of healthcare providers and systems survey. JAMA. 2017;317(19):2013-2015. doi: 10.1001/jama.2017.2827 [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Sekhri S, Arora NS, Cottrell H, et al. Probability of opioid prescription refilling after surgery: Does initial prescription dose matter? Ann Surg. 2018;268(2):271-276. doi: 10.1097/SLA.0000000000002308 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1.

Trial Protocol

jamahealthforum-e240077-s001.pdf^{(490.4KB, pdf)}

Supplement 2.

eAppendix

eTable 1. Prescribing guidelines for procedures included in the present study

eTable 2. Intervention impacts over time (full output from Figure 2B in main text)

eTable 3. Intervention effects with full set of covariates included

eTable 4. Subgroup analyses of intervention impacts on guideline-discordant prescribing

eTable 5. Tests for differences in intervention impacts by subgroup

eTable 6. Effect of social norms-based nudges on primary and secondary outcomes (discharge level)

eTable 7. Intervention effects on patient outcomes

eTable 8. Sensitivity analyses using surgeons who were more exposed to the intervention

eFigure 1. Example of Peer Comparison Intervention Email

eFigure 2. Example of Guidelines Intervention Email

eFigure 3. Effect of social norms-based nudges on each quantile of MMEs prescribed at discharge

eFigure 4. Exposure to interventions

jamahealthforum-e240077-s002.pdf^{(635.5KB, pdf)}

Supplement 3.

Data Sharing Statement

jamahealthforum-e240077-s003.pdf^{(16.5KB, pdf)}

[aoi240004r1] 1.Brummett CM, Waljee JF, Goesling J, et al. New persistent opioid use after minor and major surgical procedures in U.S. adults. JAMA Surg. 2017;152(6):e170504. doi: 10.1001/jamasurg.2017.0504 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r2] 2.Compton WM, Jones CM, Baldwin GT. Relationship between nonmedical prescription-opioid use and heroin use. N Engl J Med. 2016;374(2):154-163. doi: 10.1056/NEJMra1508490 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r3] 3.Lipari RN, Hughes A. How people obtain the prescription pain relievers they misuse. The CBHSQ Report. Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; January 12, 2017. https://www.samhsa.gov/data/sites/default/files/report_2686/ShortReport-2686.html [PubMed]

[aoi240004r4] 4.Makary MA, Overton HN, Wang P. Overprescribing is major contributor to opioid crisis. BMJ. 2017;359:j4792. doi: 10.1136/bmj.j4792 [DOI] [PubMed] [Google Scholar]

[aoi240004r5] 5.Agniel D, Brat GA, Marwaha JS, et al. Association of postsurgical opioid refills for patients with risk of opioid misuse and chronic opioid use among family members. JAMA Netw Open. 2022;5(7):e2221316. doi: 10.1001/jamanetworkopen.2022.21316 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r6] 6.Griesler PC, Hu MC, Wall MM, Kandel DB. Assessment of prescription opioid medical use and misuse among parents and their adolescent offspring in the US. JAMA Netw Open. 2021;4(1):e2031073. doi: 10.1001/jamanetworkopen.2020.31073 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r7] 7.Hill MV, McMahon ML, Stucke RS, Barth RJ Jr. Wide variation and excessive dosage of opioid prescriptions for common general surgical procedures. Ann Surg. 2017;265(4):709-714. [DOI] [PubMed] [Google Scholar]

[aoi240004r8] 8.Neuman MD, Bateman BT, Wunsch H. Inappropriate opioid prescription after surgery. Lancet. 2019;393(10180):1547-1557. doi: 10.1016/S0140-6736(19)30428-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r9] 9.Schirle L, Stone AL, Morris MC, et al. Leftover opioids following adult surgical procedures: a systematic review and meta-analysis. Syst Rev. 2020;9(1):139. doi: 10.1186/s13643-020-01393-8 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r10] 10.Waljee JF, Gunaseelan V, Bicket MC, Brummett CM, Chua KP. Safety and distribution of opioid prescribing by US surgeons. Ann Surg. 2023;10:1097. doi: 10.1097/SLA.0000000000005802 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r11] 11.Cialdini RB, Kallgren CA, Reno RR. A focus theory of normative conduct: A theoretical refinement and reevaluation of the role of norms in human behavior. In: Zanna MP, ed. Advances in Experimental Social Psychology Cambridge. Academic Press; 1991. doi: 10.1016/S0065-2601(08)60330-5 [DOI] [Google Scholar]

[aoi240004r12] 12.Goldstein NJ, Cialdini RB. Using social norms as a lever of social influence. In: Pratkanis AR, ed. The science of social influence: advances and future progress. Psychology Press; 2011:167-191. doi: 10.4324/9780203818565-7 [DOI] [Google Scholar]

[aoi240004r13] 13.Zeng Y, Shi L, Liu C, et al. Effects of social norm feedback on antibiotic prescribing and its characteristics in behaviour change techniques: a mixed-methods systematic review. Lancet Infect Dis. 2022;23(5):e175-e184. [DOI] [PubMed] [Google Scholar]

[aoi240004r14] 14.Navathe AS, Liao JM, Yan XS, et al. The effect of clinician feedback interventions on opioid prescribing. Health Aff (Millwood). 2022;41(3):424-433. doi: 10.1377/hlthaff.2021.01407 [DOI] [PubMed] [Google Scholar]

[aoi240004r15] 15.Zhang DDQ, Sussman J, Dossa F, et al. A systematic review of behavioral interventions to decrease opioid prescribing after surgery. Ann Surg. 2020;271(2):266-278. doi: 10.1097/SLA.0000000000003483 [DOI] [PubMed] [Google Scholar]

[aoi240004r16] 16.Levy B, Paulozzi L, Mack KA, Jones CM. Trends in opioid analgesic–prescribing rates by specialty, US, 2007–2012. Am J Prev Med. 2015;49(3):409-413. doi: 10.1016/j.amepre.2015.02.020 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r17] 17.Candon M, Xue L, Shen S, Cole ES, Donohue J, Rothbard A. The impact of opioid prescribing report cards in Medicaid. J Manag Care Spec Pharm. 2022;28(8):862-870. doi: 10.18553/jmcp.2022.28.8.862 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r18] 18.Klaiman T, Nelson MN, Yan XS, et al. Clinician perceptions of receiving different forms of feedback on their opioid prescribing. Am J Med Qual. 2023;38(1):1-8. doi: 10.1097/JMQ.0000000000000092 [DOI] [PubMed] [Google Scholar]

[aoi240004r19] 19.Navathe AS, Volpp KG, Bond AM, et al. Assessing the effectiveness of peer comparisons as a way to improve health care quality. Health Aff (Millwood). 2020;39(5):852-861. doi: 10.1377/hlthaff.2019.01061 [DOI] [PubMed] [Google Scholar]

[aoi240004r20] 20.Sacarny A, Barnett ML, Le J, Tetkoski F, Yokum D, Agrawal S. Effect of peer comparison letters for high-volume primary care prescribers of quetiapine in older and disabled adults: a randomized clinical trial. JAMA Psychiatry. 2018;75(10):1003-1011. doi: 10.1001/jamapsychiatry.2018.1867 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r21] 21.Sedhom R, Blackford AL, Gupta A, Smith TJ, Shulman LN, Carducci MA. Oncologist peer comparisons as a behavioral science strategy to improve hospice utilization. JCO Oncol Pract. 2022;18(7):e1122-e1131. doi: 10.1200/OP.21.00738 [DOI] [PubMed] [Google Scholar]

[aoi240004r22] 22.Kirkegaard A, Wagner Z, Mariano LT, et al. Evaluating the effectiveness of email-based nudges to reduce postoperative opioid prescribing: study protocol of a randomised controlled trial. BMJ Open. 2022;12(9):e061980. doi: 10.1136/bmjopen-2022-061980 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r23] 23.Glaser GE, Kalogera E, Kumar A, et al. Outcomes and patient perspectives following implementation of tiered opioid prescription guidelines in gynecologic surgery. Gynecol Oncol. 2020;157(2):476-481. doi: 10.1016/j.ygyno.2020.02.025 [DOI] [PubMed] [Google Scholar]

[aoi240004r24] 24.Thiels CA, Ubl DS, Yost KJ, et al. Results of a prospective, multicenter initiative aimed at developing opioid-prescribing guidelines after surgery. Ann Surg. 2018;268(3):457-468. doi: 10.1097/SLA.0000000000002919 [DOI] [PubMed] [Google Scholar]

[aoi240004r25] 25.Wyles CC, Hevesi M, Ubl DS, et al. Implementation of procedure-specific opioid guidelines: a readily employable strategy to improve consistency and decrease excessive prescribing following orthopaedic surgery. JB JS Open Access. 2020;5(1):e0050. doi: 10.2106/JBJS.OA.19.00050 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r26] 26.Wyles CC, Hevesi M, Trousdale ER, et al. The 2018 Chitranjan S. Ranawat, MD Award: developing and implementing a novel institutional guideline strategy reduced postoperative opioid prescribing after TKA and THA. Clin Orthop Relat Res. 2019;477(1):104-113. doi: 10.1007/s11999.0000000000000292 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r27] 27.Centers for Disease Control and Prevention . Table: CDC oral morphine milligram equivalents, September. Accessed April 14, 2023. https://www.cdc.gov/opioids/data-resources/index.html

[aoi240004r28] 28.R Core Team . R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Accessed April 17, 2023. https://www.R-project.org/

[aoi240004r29] 29.Bryk AS, Raudenbush SW. Hierarchical linear models: applications and data analysis methods. Sage Publications, Inc; 1992. [Google Scholar]

[aoi240004r30] 30.Graubard BI, Korn EL, Korn E. Predictive margins with survey data. Biometrics. 1999;55(2):652-659. doi: 10.1111/j.0006-341X.1999.00652.x [DOI] [PubMed] [Google Scholar]

[aoi240004r31] 31.Abadie A, Athey S, Imbens GW, Wooldridge J. When should you adjust standard errors for clustering? National Bureau of Economic Research; 2017. doi: 10.3386/w24003 [DOI] [Google Scholar]

[aoi240004r32] 32.Association of American Medical Colleges . Physician Specialty Data Report: Number of People per Active Physician by Specialty, 2021. Accessed April 14, 2023. https://www.aamc.org/data-reports/workforce/data/number-people-active-physician-specialty-2021

[aoi240004r33] 33.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57(1):289-300. doi: 10.1111/j.2517-6161.1995.tb02031.x [DOI] [Google Scholar]

[aoi240004r34] 34.Kahan BC, Li F, Copas AJ, Harhay MO. Estimands in cluster-randomized trials: choosing analyses that answer the right question. Int J Epidemiol. 2023;52(1):107-118. doi: 10.1093/ije/dyac131 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r35] 35.Billings JD, Huynh V, Leonard LD, et al. Addressing an epidemic: improving guideline-concordant opioid prescribing in surgical patients. Surgery. 2022;172(5):1407-1414. doi: 10.1016/j.surg.2022.06.033 [DOI] [PubMed] [Google Scholar]

[aoi240004r36] 36.Einarsson A, Chiu AS, Mori M, et al. Changing the default option in electronic medical records reduced postoperative opioid prescriptions after cardiac surgery. JTCVS Open. 2021;8:467-474. doi: 10.1016/j.xjon.2021.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r37] 37.Chiu AS, Jean RA, Hoag JR, Freedman-Weiss M, Healy JM, Pei KY. Association of lowering default pill counts in electronic medical record systems with postoperative opioid prescribing. JAMA Surg. 2018;153(11):1012-1019. doi: 10.1001/jamasurg.2018.2083 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r38] 38.Chua KP, Thorne MC, Ng S, Donahue M, Brummett CM. Association between default number of opioid doses in electronic health record systems and opioid prescribing to adolescents and young adults undergoing tonsillectomy. JAMA Netw Open. 2022;5(6):e2219701. doi: 10.1001/jamanetworkopen.2022.19701 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r39] 39.Vu JV, Howard RA, Gunaseelan V, Brummett CM, Waljee JF, Englesbe MJ. Statewide implementation of postoperative opioid prescribing guidelines. N Engl J Med. 2019;381(7):680-682. doi: 10.1056/NEJMc1905045 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r40] 40.Sacarny A, Avilova T, Powell D, Williamson I, Merrick W, Jacobson M. A randomized trial of letters to encourage prescription monitoring program use and safe opioid prescribing: study examines effectiveness of letters to clinicians that encourage prescription monitoring program use and safe opioid prescribing. Health Aff (Millwood). 2023;42(1):140-149. doi: 10.1377/hlthaff.2022.00859 [DOI] [PubMed] [Google Scholar]

[aoi240004r41] 41.Doctor JN, Nguyen A, Lev R, et al. Opioid prescribing decreases after learning of a patient’s fatal overdose. Science. 2018;361(6402):588-590. doi: 10.1126/science.aat4595 [DOI] [PubMed] [Google Scholar]

[aoi240004r42] 42.Kraemer KL, Althouse AD, Salay M, et al. Effect of different interventions to help primary care clinicians avoid unsafe opioid prescribing in opioid-naive patients with acute noncancer pain: a cluster randomized clinical trial. JAMA Health Forum. 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r43] 43.Michael SS, Babu KM, Androski C Jr, Reznek MA. Effect of a data-driven intervention on opioid prescribing intensity among emergency department providers: a randomized controlled trial. Acad Emerg Med. 2018;25(5):482-493. doi: 10.1111/acem.13400 [DOI] [PubMed] [Google Scholar]

[aoi240004r44] 44.Sacarny A, Safran E, Steffel M, et al. Effect of pharmacist email alerts on concurrent prescribing of opioids and benzodiazepines by prescribers and primary care managers: a randomized clinical trial. JAMA Health Forum. 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r45] 45.Jacobson RP, Mortensen CR, Cialdini RB. Bodies obliged and unbound: differentiated response tendencies for injunctive and descriptive social norms. J Pers Soc Psychol. 2011;100(3):433-448. doi: 10.1037/a0021470 [DOI] [PubMed] [Google Scholar]

[aoi240004r46] 46.Melnyk V. Herpen Ev, Fischer AR, van Trijp HC. To think or not to think: the effect of cognitive deliberation on the influence of injunctive versus descriptive social norms. Psychol Mark. 2011;28(7):709-729. doi: 10.1002/mar.20408 [DOI] [Google Scholar]

[aoi240004r47] 47.Bateman BT, Cole NM, Maeda A, et al. Patterns of opioid prescription and use after cesarean delivery. Obstet Gynecol. 2017;130(1):29-35. doi: 10.1097/AOG.0000000000002093 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r48] 48.Hill MV, Stucke RS, McMahon ML, Beeman JL, Barth RJ Jr. An educational intervention decreases opioid prescribing after general surgical operations. Ann Surg. 2018;267(3):468-472. doi: 10.1097/SLA.0000000000002198 [DOI] [PubMed] [Google Scholar]

[aoi240004r49] 49.Lee JS, Howard RA, Klueh MP, et al. The impact of education and prescribing guidelines on opioid prescribing for breast and melanoma procedures. Ann Surg Oncol. 2019;26(1):17-24. doi: 10.1245/s10434-018-6772-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r50] 50.Lee JS, Hu HM, Brummett CM, et al. Postoperative opioid prescribing and the pain scores on hospital consumer assessment of healthcare providers and systems survey. JAMA. 2017;317(19):2013-2015. doi: 10.1001/jama.2017.2827 [DOI] [PMC free article] [PubMed] [Google Scholar]

[aoi240004r51] 51.Sekhri S, Arora NS, Cottrell H, et al. Probability of opioid prescription refilling after surgery: Does initial prescription dose matter? Ann Surg. 2018;268(2):271-276. doi: 10.1097/SLA.0000000000002308 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Peer Comparison or Guideline-Based Feedback and Postsurgery Opioid Prescriptions

Zachary Wagner, PhD

Allison Kirkegaard, MTS

Louis T Mariano, PhD

Jason N Doctor, PhD

Xiaowei Yan, PhD

Stephen D Persell, MD, MPH

Noah J Goldstein, PhD

Craig R Fox, PhD

Chad M Brummett, MD

Robert J Romanelli, PhD, MPH

Kathryn Bouskill, PhD, MPH

Meghan Martinez, MPH

Kyle Zanocco, MD

Daniella Meeker, PhD

Satish Mudiganti, MS

Jennifer Waljee, MD

Katherine E Watkins, MD, MSHS

Key Points

Question

Findings

Meaning

Abstract

Importance

Objective

Design, Setting, and Participants

Interventions

Main Outcome and Measures

Results

Conclusions and Relevance

Trial Registration

Introduction

Methods

Study Design and Oversight

Inclusion and Enrollment

Opioid Prescribing Guidelines

Intervention and Control Conditions

Peer Comparison (Descriptive Norm) Intervention

Guideline Feedback (Injunctive Norm) Intervention

Control Condition

Implementation

Randomization

Outcome Measures

Statistical Analysis

Multiple Hypothesis Testing

Results

Figure 1. Consort Flow Diagram.

Table 1. Sample Characteristicsa.

Intervention Effects on Above-Guideline Opioid Prescriptions

Figure 2. Prescribing Above Guideline Quantities Over Time.

Table 2. Effect of Social Norms-Based Nudges on Primary and Secondary Outcomesa.

Figure 3. Heterogeneity in Intervention Effects by Procedure Volume and Baseline Guideline-Discordant Opioid Prescribing.

Intervention Effects on MMEs Prescribed

Intervention Effects on the Probability of an Opioid Being Prescribed

Intervention Effects on Patient Outcomes

Sensitivity Analyses

Discussion

Principal Findings

Implications for Public Health

Implications for Scale-Up

Comparison With Other Studies

Limitations

Conclusions

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases

Table 1. Sample Characteristics^a.

Table 2. Effect of Social Norms-Based Nudges on Primary and Secondary Outcomes^a.