Evaluating the effectiveness of email-based nudges to reduce postoperative opioid prescribing: study protocol of a randomised controlled trial

Abstract

Introduction

Surgical patients are commonly prescribed more opioids at discharge than needed to manage their postoperative pain. These excess opioids increase the risks of new persistent opioid use, opioid-induced ventilatory impairment and opioid diversion. This study tests the effectiveness of two behavioural nudges, one based on peer behaviour and one based on best practice guidelines, in reducing excessive postoperative opioid prescriptions.

Methods and analysis

The study will be conducted at 19 hospitals within a large healthcare delivery system in northern California, USA. Three surgical specialties (general surgery, orthopaedic surgery and obstetric/gynaecological surgery) at each hospital will be randomised either to a control group or to one of two active intervention arms. One intervention is grounded in the theory of injunctive norms, and provides feedback to surgeons on their postoperative opioid prescribing relative to prescribing guidelines endorsed by their institution. The other intervention draws from the theory of descriptive norms, and provides feedback similar to the first intervention but using peers’ behaviour rather than guidelines as the benchmark for the surgeon’s prescribing behaviour. The interventions will be delivered by a monthly email. Both interventions will be active for twelve months. The effects of each intervention relative to the control group and to each other will be tested using a four-level hierarchical model adjusted for multiple hypothesis testing.

Ethics and dissemination

Using behavioural nudges rather than rigid policy changes allows us to target excessive prescribing without preventing clinicians from using their clinical judgement to address patient pain. All study activities have been approved by the RAND Human Subjects Protection Committee (ID 2018-0988). Findings will be disseminated through conference presentations, peer-reviewed publications and social media accounts.

Trial registration number

NCT05070338.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The study includes multiple surgical specialties (general, orthopaedic, obstetric/gynaecologic) and a large sample size (19 hospitals) across diverse settings, allowing for broad generalisability.

• Randomised controlled trial design allows us to account for secular decline in opioid prescribing.

• Intervention is informed by behavioural theory, with careful attention to details that affect behavioural response.

• Incomplete prescribing data at the study site compromises some analyses.

Introduction

Background

Despite high awareness of the opioid epidemic, clinicians still overprescribe opioids after surgery.^1–7 This postoperative overprescribing puts both patients and communities at risk, increasing the patient’s likelihood of developing chronic opioid use^8–14 or opioid-induced ventilatory impairment¹¹ and adding to the reservoir of unused opioids available for misuse and diversion.^{11 15}

The discrepancy between clinicians’ awareness of the opioid epidemic and the degree of overprescribing—over half of opioid pills prescribed after surgery go unused⁷—suggests that prescribing practices are not based on purely rational decisions. Indeed, behavioural research has shown that judgement and decision making of both laypeople and experts in a variety of disciplines falls short of rational standards in systematic and predictable ways.^16–19 Even well-informed clinicians make cognitive errors when estimating the benefits and harms of treatment, and these errors are especially likely where there is uncertainty about risks and benefits (as with opioid prescribing decisions for individual patients).²⁰

In recent years, behavioural economists and experimental psychologists have successfully leveraged behavioural insights to design ‘choice architecture’ that ‘nudges’ individuals to make better decisions without infringing on their freedom of choice.^{21 22} Such behavioural nudges are promising strategies for changing clinician prescribing behaviour because they are often more cost-effective than traditional interventions,²³ can be integrated into existing clinical workflows, and are rapidly scalable once built.

One powerful type of behavioural nudge relies on the strong motivation that most people have to conform with their peers’ behaviour.^{24 25} Abundant research has found that people (including clinicians) are strongly motivated to adhere to prevailing social norms,^{24 25} and that nudges based on describing social norms can be used to influence prescribing decisions.²⁶

Another type of behavioural nudge relies on motivation to follow injunctive norms—to do what is considered the ‘right thing to do’. For example, clinicians may be motivated to follow best practice guidelines published by a well-respected organisation. Previous studies suggest that such guidelines are in reality often ignored and thus ineffective in changing behaviour,^{27 28} but there is insufficient evidence to determine whether they are more or less effective than nudges that describe peer behaviour.

Both of these types of nudges—nudges based on descriptive norms and nudges based on injunctive norms—have been applied to the issue of excessive postoperative opioid prescribing.^29–38 The results have been promising, but because most of these studies have used a pre–post design, it is possible that the observed decreases in prescribing can be explained by a secular trend. Furthermore, all of these studies have bundled and tested different interventions together (eg, grand rounds presentations or patient education in addition to nudges), making the effectiveness of the nudges alone unclear. Accordingly, the evidence base for the effectiveness of behavioural nudges in influencing postoperative opioid prescribing is limited. In this paper, we describe the protocol for a study that addresses these knowledge gaps, using a randomised controlled trial (RCT) design and testing nudges in the absence of other interventions. This study will also make a novel contribution to the literature by directly testing which type of nudge—descriptive or injunctive—is more effective.

Specifically, in this RCT, we will investigate the extent to which descriptive and injunctive norms, conveyed through nudges delivered monthly by email, can each change postoperative opioid prescribing behaviour. Across 19 hospitals in a large health system in northern California, USA, surgeons within three surgical specialties (general, orthopaedic and obstetric/gynaecological surgery) will be randomised to receive either nudges based on peer prescribing behaviour (descriptive norm), nudges based on prescribing guidelines (injunctive norm) or no nudges (status quo).

Research questions

1. How does an email-based nudge that alerts surgeons when they prescribe opioid quantities above guidelines (injunctive norm nudge) affect postoperative opioid prescribing at discharge compared with the status quo?

2. How does an email-based nudge that alerts surgeons that they are prescribing opioid quantities that are higher than what peers prescribe (descriptive norm nudge) affect postoperative opioid prescribing at discharge compared with the status quo?

3. What is the comparative effectiveness of an injunctive norm nudge versus a descriptive norm nudge in reducing postoperative opioid prescribing?

4. If surgeons do change their postoperative opioid prescribing behaviour in response to nudges, does this change persist 1 year after the nudges have stopped?

The null hypothesis is that surgeons who receive nudges will prescribe the same quantities of postoperative opioids as surgeons who do not; our alternative hypotheses are that surgeons who receive either type of nudge will prescribe fewer postoperative opioids than those who receive no nudges, surgeons who receive the descriptive norm nudge will prescribe fewer postoperative opioids than those who receive the injunctive norm nudge,²⁶ and these differences will persist 1 year after the nudges have stopped.

Significance

Our study will provide evidence regarding the comparative effectiveness of two low-cost behavioural nudges based on peer norms and guidelines, the interactions between clinician characteristics and the type of nudge, and the persistence of behaviour change after nudges are turned off. Results from this study may inform a scalable, low-cost intervention that can reduce patient harm by changing clinician behaviour in real-world practice.

Methods and analysis

Overview of design

We will conduct a three-arm cluster randomised controlled trial of two behavioural nudges compared with usual postsurgical care. One nudge will provide feedback on the surgeon’s prescribing behaviour relative to institutional prescribing guidelines (an injunctive norm); the other will provide feedback on their prescribing behaviour relative to their peers (a descriptive norm). Three surgical specialties (general surgery, orthopaedic surgery and obstetric/gynaecological surgery) within 19 hospitals will be randomised such that all surgeons within a given specialty at a given hospital will receive one of three conditions: control, guideline-based nudge or peer-based nudge.

Setting

This study will take place across 19 hospitals within Sutter Health, a large not-for-profit healthcare system in California, USA. Importantly for the generalisability of this study, these hospitals are geographically diverse and vary widely in size and the populations served (table 1).

Table 1.

Characteristics of study site hospitals

	No of hospitals
No of beds
0–99	10
100–499	5
500+	4
Urbanicity
Large central metro	5
Large fringe metro	5
Medium metro	6
Small metro	–
Micropolitan	2
Non-core	1
Proportion of patients on Medicaid*
Less than 25%	13
25%–50%	5
50%–75%	1
75% or more	–
Proportion of patients who identify as non-Hispanic white*
Less than 25%	1
25–50%	5
50–75%	8
75% or more	5

*Proportions calculated from electronic health record data among patients eligible for our study between June 2020 and May 2021.

Like many other healthcare organisations in the USA, this health system accepts multiple commercial preferred provider organisation and health management organisation plans, Medicare and Medicaid. Because of this payer mix, there is no single, fixed drug formulary and clinicians can prescribe as they choose, per patients’ individual plans or preferences.

Sample size and characteristics

Our study intervention targets 778 surgeons (table 2). Though discharge medication orders are sometimes written by a clinician other than the surgeon, such as a hospitalist or nurse practitioner, we posit that the surgeon is still ultimately responsible for all medication orders written for their patients. If surgeons cannot influence medication orders written by other clinicians for their patients, the effect of the intervention will be attenuated.

Table 2.

Characteristics of eligible study site surgeons

	Per cent of surgeons
	Total (n=778)	General surgery (n=187)	Orthopaedic surgery (n=244)	Obstetric/gynaecological surgery (n=347)
Year of medical degree
1960–1969	0.7	–	0.4	1.3
1970–1979	7.3	7.9	7.4	7.0
1980–1989	21.3	20.8	23.4	20.0
1990–1999	28.5	30.3	23.4	31.1
2000–2009	26.2	28.7	31.6	21.0
2010–2019	16.0	12.4	13.9	19.7
Sex
Female	39.9	28.2	5.0	71.7
Male	60.1	71.8	95.0	28.3

A total of 778 surgeons targeted by our study intervention operate at a total of 23 physical hospitals. One set of three physical hospitals and another set of two physical hospitals are located together, each set functioning as a hospital campus. A third set of two hospitals essentially share the same surgical staff. We treat each of these three sets as a single hospital for the purposes of this study, both to capture the organisation structure and to minimise the potential for spillover effects, resulting in 19 hospital units. For brevity and clarity, we refer to these 19 hospital units simply as ‘hospitals’ throughout.

Power considerations

Statistical power to identify effects of the nudges was examined using recent past data from the participating hospitals. We estimated design parameters required by the PowerUpR package in R software,³⁹ which provides the capability to estimate statistical power for randomised block clustered designs. Examining medication dose, input parameters for the calculation included unconditional intracluster correlations (ICC) for the hospital (ICC=0.005), service line (ICC=0.039) and provider (ICC=0.337) levels; the number of service line groups (up to three per hospital); the number of providers by service line expected to participate in the study and number of patients per service line. The ICCs were empirically determined from our preliminary data. We assumed that covariates informative of the dosage would explain between 25% and 50% of the dosage variation at each of the patient, provider and service line levels (ie, R² between 0.25 and 0.50). We derived statistical power, assuming one-third of the service line groups within hospital will be randomly assigned to each study arm (two treatment and one control). We computed power for pairwise comparison of each of the two nudge arms versus the no nudge arm and adjusted our alpha level to account for multiple comparisons (alpha=0.05/2). We will have 80% power to detect significant differences between the intervention conditions of at least a minimum detectable effect size (MDES)=0.347 SDs when R²=0.25, while R²=0.5 would yield an MDES of 0.305.

Randomisation scheme

The study design has four levels: patients, surgeons, surgical specialties and hospitals. Randomisation will take place at the level of the surgical specialty, using a blocked scheme to ensure that each arm has a balance of large and small hospitals and a sample size of surgeons similar to the other two arms.

Intervention

Surgeons randomised to our study intervention will receive one of two types of behavioural nudges delivered as monthly emails. The two nudges will be active for twelve months (October 2021–October 2022).

To ensure that the nudges target only inappropriate opioid prescribing, surgeons will receive nudges only when they write opioid prescriptions that exceed postoperative opioid prescribing guidelines developed by multidisciplinary teams at the Mayo Clinic^{32 40 41} (and personal communication with Professor Elizabeth Habermann, Ph.D., MPH, on opioid prescribing guidelines for caesarean section, 12 March, 2021; unreferenced). These guidelines recommend ranges of 5 mg oxycodone tablet quantities specific to the procedure performed and are partly based on patient surveys of actual postoperative opioid use. While some patients may require higher quantities (eg, patients with particularly high opioid tolerance, body mass index or pain levels), these guidelines are appropriate for the vast majority of patients.

In both nudge conditions, eligibility for receiving a monthly nudge is contingent on at least two of the surgeon’s patients being discharged with a postoperative opioid prescription exceeding the quantities specified by the Mayo Clinic guidelines. Though it may seem counterintuitive for the descriptive norm nudge to be based implicitly on prescribing guidelines, this choice ensures patient safety and avoids confounding the content of the nudge with the threshold for receiving a nudge.

Intervention arm 1: nudge based on descriptive norms

Surgeons randomised to this condition will receive an email with the following content at the end of each month in which at least two of their patients are discharged with a postoperative opioid prescription that exceeds the prescribing guideline for the procedure performed.

[Subject line: Your peers vs your opioid prescribing safety record]

Dear Dr. [Name],

In an effort to reduce opioid use among our surgical patients, Sutter Health is reviewing opioid prescriptions and prescribing patterns for surgeons and will be communicating the findings.

In [month], at least XX of your patients were discharged with opioid prescriptions exceeding the amount prescribed by YY% of your peers for these procedures.

YY% of [specialty] surgeons at Sutter Health prescribe within the ranges below.

We will continue to send you opioid prescribing safety reports.

Sincerely,

[Signature(s) of chief medical executive, chief of staff, and/or surgical department chair at the surgeon’s hospital]

[Table including each procedure type performed by this surgeon in the reference month and the corresponding ‘Amount prescribed by your peers (5 mg oxycodone tablets)’, with a footnote stating the conversion factors for hydrocodone and tramadol]

The ranges of 5 mg oxycodone tablets displayed in the email will be the same as the ranges stipulated by the prescribing guidelines, but this nudge will not include any language about guidelines.

Intervention arm 2: nudge based on injunctive norms

This condition will be identical to the first condition, except the content of the monthly emails will refer to safety guidelines rather than the surgeon’s peers.

[Subject line: Best practice guidelines vs your opioid prescribing safety record]

Dear Dr. [Name],

In an effort to reduce opioid use among our surgical patients, Sutter Health is reviewing opioid prescriptions and prescribing patterns for surgeons and will be communicating the findings.

In [month], at least XX of your patients were discharged with opioid prescriptions exceeding the amounts recommended by safety guidelines for these procedures.

For patient safety, Sutter Health recommends prescribing within the ranges below for these procedures. Doing so will also meet best practice safety guidelines for postoperative opioid prescribing.

We will continue to send you opioid prescribing safety reports.

Sincerely,

[Signature(s) of chief medical executive, chief of staff, and/or surgical department chair at the surgeon’s hospital]

[Table including each procedure type performed by this surgeon in the reference month and the corresponding ‘Amount recommended by Sutter Health (5 mg oxycodone tablets)’, with a footnote stating the conversion factors for hydrocodone and tramadol]

Control arm

Surgeons randomised to the control arm will not receive any nudges and will not be informed of the study. By not informing them of the study, we will prevent a Hawthorne effect and obtain an accurate representation of status quo prescribing behaviour against which to test the effects of the nudges.

Eligibility criteria

The nudges that a surgeon in either intervention arm will receive are based on that surgeon’s eligible discharge opioid prescriptions in the previous month. Eligible prescriptions meet all of the following criteria:

• The patient is at least 18 years old at the date of surgery.

• The patient is discharged to their home.

• The surgical procedure has an applicable post-operative opioid prescribing guideline.

• The surgical procedure is the only surgical procedure performed during the patient’s hospital stay.

• The prescription is for an opioid taken orally (tablets, capsules or liquid solution).

To avoid contamination between the intervention arms, surgeons who operate across multiple surgical specialties (defined as surgeons who performed less than 90% of their total procedures in one specialty between June 2020 and May 2021) will not be eligible.

Patient and public involvement

Since the study intervention only targets clinicians, we have not chosen to involve patients directly in the development of this study. However, the prescribing guidelines on which our intervention is based were created with input from patients via stakeholder groups and postdischarge surveys.^{32 40}

Data collection

Prescribing data, clinician characteristics (eg, sex, type of medical degree, year of medical degree), patient characteristics (eg, age, sex, body mass index, comorbidities) and case characteristics (eg, procedure type, length of operating time) will be obtained by querying the electronic health record database.

Hospital characteristics (eg, number of beds, urbanicity) will be obtained from California, USA’s Office of Statewide Health Planning and Development datasets.

Data analyses

Primary outcomes

Our primary outcome is the share of discharge prescriptions that were above the guideline for the respective procedure (see above for how guidelines were identified). Prescribing above guidelines is the outcome to which both nudges are linked (even though the descriptive norm nudge does not explicitly refer to guidelines) and thus a key measure of whether clinician behaviour responds to the nudges. We define a prescription as being above guidelines if the morphine milligram equivalent (MME) quantity of opioids prescribed is above the ceiling for the procedure-specific guideline (guidelines range from zero to a ceiling). If no opioid is prescribed at discharge, we will code this as within guideline.

Secondary outcomes

We will also analyse the following secondary outcomes to further understand the effects of the intervention.

• MMEs prescribed at discharge.

• Days’ supply of opioids prescribed at discharge.

• Share of discharges where any opioid was prescribed.

• Share of patients on opioids for greater than 3 months postdischarge.

• Number of 30-day all-cause emergency department visits.

• Number of 30-day all-cause hospitalisations.

• Share of discharge opioid prescriptions above prescribing guidelines in the 12 months after the nudges end.

Primary analysis

We will analyse outcomes at the level of the discharge using a four-level hierarchical linear model (HLM),⁴² thus capturing the clustering inherent in the study design and data generating process. We will analyse outcomes at the patient level, and patients are nested within surgeons, who are nested within specialties, which are nested within hospitals. Both primary and secondary outcomes will follow this modelling structure. To improve the precision of our estimates, we will also include a set of observable patient covariates (X), surgeon covariates (Z), specialty covariates (U) and hospital covariates (W). For patient $i$, treated by surgeon $p$, in specialty $s$, at hospital $h$, we consider the following HLM formulation for continuous outcomes ${\displaystyle Y_{ipsh}}$ :

$${\displaystyle Y_{ipsh}=\begin{array}{l}{\beta }_0+{\beta }_{1}ARM{1}_{sh}+{\beta }_{2}ARM{2}_{sh}+{\gamma }_{1h}ARM{1}_{sh}+{\gamma }_{2h}ARM{2}_{sh}+\\ {\omega }_1{X}_{ipsh}+{\omega }_2{Z}_{psh}+{\omega }_3{U}_{sh}+{\omega }_4{W}_h+{\gamma }_h+{\eta }_{sh}+{\phi }_{psh}+{\epsilon }_{ipsh}\mathrm{\#}\end{array}}$$ (1)

${ARM1}_{sh}$ and ${ARM2}_{sh}$ are indicator variables for whether specialty $s$, at hospital $h$ were assigned to treatment arm 1 or 2, respectively.

The key terms in the equation are ${\beta }_1$ and ${\beta }_2$, the covariate-adjusted treatment effects of arms 1 and 2 relative to the control arm; ${\beta }_1$ answers research question 1 and ${\beta }_2$ answers research question 2. We will use an F-test to compare coefficients ${\beta }_1$ and ${\beta }_2$ to answer research question 3. Thus, the effect of each nudge is estimated relative to receiving no nudges and to the other nudge.

The model allows for the possibility that the treatment effect varies across hospitals, as captured by the random effects (${\gamma }_{1h},{\gamma }_{2h})$. Unexplained variation in each of the levels is captured by the random effects ${\epsilon }_{ipsh}$, ${\phi }_{psh}$, ${\eta }_{sh}$, and ${\gamma }_h$. We will initially model these six random effects as independent but will also investigate whether including a covariance structure across these components is appropriate. The coefficients ${\omega }_1$, ${\omega }_2$, and ${\omega }_3$, capture the influence of the covariates at the patient, surgeon and specialty respectively, and covariates will be mean centred as appropriate to aid in model interpretation. Covariates may include but are not limited to the following: level 1: patient age, patient sex, patient comorbidities, procedure type, length of operating time; level 2: surgeon sex, year of surgeon’s medical degree; level 3: total volume of procedures within the specialty: level 4: number of beds, urbanicity, proportion of patients on Medicaid. Given that the covariates will not change the estimate of the treatment effect (in expectation), only reduce unexplained variance, we will choose a final pool of covariates that we find to be predictive the primary outcome. Model estimates of the treatment effects will adjust standard errors for clustering due to the due to clustered assignment of the interventions.

For binary outcomes, we implement a hierarchical generalised linear model by including a logit link for equation (1). Note that the level 1 error term ${\epsilon }_{ipsh}$ is also eliminated. The concatenated model for all four levels with a binary outcome then reduces to:

$${\displaystyle logit\left(Y_{ipsh}\right)=\begin{array}{l}{\beta }_0+{\beta }_{1}ARM{1}_{sh}+{\beta }_{2}ARM{2}_{sh}+{\gamma }_{1h}ARM{1}_{sh}+{\gamma }_{2h}ARM{2}_{sh}+\\ {\omega }_1{X}_{ipsh}+{\omega }_2{Z}_{psh}+{\omega }_3{U}_{sh}+{\omega }_4{W}_h+{\gamma }_h+{\eta }_{sh}+{\phi }_{psh}\mathrm{\#}\end{array}}$$ (2)

In the binary outcome version, the parameters ${\beta }_1$ and ${\beta }_2$ again identify the treatment effects of arms 1 and 2 relative to the control arm, with interpretation of these parameters adjusted relative to the link function implemented.

These analyses will be conducted after the intervention ends. Any interim analyses conducted during the intervention period will be solely for the purposes of safety monitoring or planning related studies; the intervention will not be altered unless recommended by the study’s data safety and monitoring board (DSMB).

Heterogeneity analysis

We will test for heterogeneity in the treatment effect along several domains. Specifically, we will add terms interacting characteristics of the surgeon with each treatment arm and conduct an F-test of the interaction terms for each nudge.

1. Specialty: We will also conduct analyses to test whether the response to each nudge varies by surgeons’ specialty.

2. Volume of surgeries: We will test for heterogeneity by number of surgeries performed over the 12-month study period. We will only include surgeries for which we have guidelines in this count.

3. Baseline opioid prescribing: We will categorise surgeons based on the portion of their surgeries in the 12 months prior to the start of the intervention that were above guidelines. We expect that the intervention will have a larger effect for surgeon with a higher share of prescription above guidelines.

Longitudinal analysis

In addition to assessing the treatment effect averaged over the entire 12-month period, we will also analyse treatment effects by month to assess how the treatment effect evolves over time. For this analysis, we will interact study month indicators with the treatment assignment indicators.

Persistence analysis

We will conduct a secondary analysis to examine whether nudge effects persist once the nudges are discontinued. The data will include the RCT data analysed in the model above, but also data collected for 1-year postintervention (the ‘persistence period’). The analysis model above will be modified by adding an indicator for the RCT period versus persistence period plus interaction terms for period and each nudge to the model. The statistical significance of these interaction terms will be used to assess whether the treatment effect significantly differs post-RCT.

Adjustment for multiple hypothesis testing

Two varieties of multiple testing concerns are present. For any instance of equations (1) or (2), we simultaneously test for a treatment effect in either study arm and difference in treatment effect between arms. Across secondary outcomes within the same domain, we also consider a series of tests for each arm. As appropriate, we will employ family-wise error rate and false discovery rate corrections^{43 44} to account for simultaneously tested hypotheses.

Ethics and dissemination

All study activities have been approved by the RAND Human Subjects Protection Committee (ID 2018-0988).

Throughout the development of this study, we paid careful attention to the possibility that reducing postoperative opioid prescriptions might result in greater postsurgical pain. We believe that the risk presented by our nudge interventions is negligible, both because the nudges do not prevent the clinicians from using their own clinical judgement and because previous studies have found that reducing the amount of opioids prescribed after surgical operations did not affect patient satisfaction,^{45 46} pain scores^45–47 or refill rates.^48–50 Given this negligible level of risk, the RAND Human Subjects Protection Committee approved a waiver of informed consent for participating clinicians and their patients.

Data indicative of adverse events (opioid refills and emergency department visits within 30 days of hospital discharge) will be monitored throughout the intervention period by an independent DSMB comprising four experts in surgery, interventional pain management, statistical methodology and risk assessment and research ethics. The DSMB may recommend modifying or terminating the trial based on its interim analyses.

Once results are obtained for primary and secondary outcomes, we will submit these results to ClinicalTrials.gov. Findings will also be disseminated through conference presentations, peer-reviewed publications and social media accounts. Deidentified data will be made available on reasonable request.

Ethics statements

Patient consent for publication

Not applicable.