Skip to main content
Lippincott Open Access logoLink to Lippincott Open Access
. 2020 Aug 24;16(3 1 Suppl):S36–S41. doi: 10.1097/PTS.0000000000000710

The Effect of Opioid Stewardship Interventions on Key Outcomes: A Systematic Review

Sarah J Shoemaker-Hunt 1, Brandy E Wyant 1
PMCID: PMC7447172  PMID: 32809999

Supplemental digital content is available in the text.

Key Words: opioids, opioid stewardship, overdose, implementation strategies

Abstract

Objectives

We sought to identify potential patient safety practices to reduce high-risk opioid prescribing.

Methods

We conducted a systematic review of the literature to identify opioid stewardship (OS) strategies implemented in primary care and other settings. Included studies evaluated an OS strategy or a multicomponent OS initiative to address potential harms of opioids and used experimental or quasi-experimental designs.

Results

We identified 14 studies and 1 systematic review that met inclusion criteria. Most studies examined multicomponent OS interventions, which often consisted of guideline-recommended clinical interventions or care processes (e.g., use urine drug screening, check Prescription Drug Monitoring Program), as well as implementation strategies (e.g., dashboards, audit and feedback). Most studies examined the effect of OS interventions on reducing the potential risks of opioids with judicious prescribing and guideline-concordant care (e.g., reduce inappropriate high opioid dosages, avoid co-prescribing opioids and benzodiazepines, use urine drug screening, treatment agreements).

Conclusions

The strength of the evidence is low to moderate that OS efforts decrease numbers of opioid prescriptions, proportion of patients on long-term opioids, or days’ supply. The strength of the evidence for OS initiatives producing significant reductions in opioid dosages was moderate. Future research is needed on the effectiveness of OS interventions, particularly studies with experimental designs and in diverse settings within the health care system.


Prescription opioids are commonly used in the treatment of pain in the United States. In 2016, an estimated 20.4% of U.S. adults (50 million) had chronic pain.1 Although opioids are a key treatment option in the management of acute, postoperative, procedural, and cancer pain, limited evidence exists of their efficacy for chronic pain.2,3

In the past 20 years, opioid prescribing increased dramatically, peaking in 2012 with 255 million prescriptions or a rate of 81.3 opioid prescriptions per 100 persons.4 Between 1999 and 2017, nearly 400,000 drug overdose (OD) deaths involved opioids (including prescription and illegal),5 signaling 3 waves of an opioid epidemic. The first wave of the opioid OD deaths began in 1999 with increased prescribing of opioids in the 1990s.6 The second wave began in 2010 with the increase in heroin-related OD deaths, and the third wave in 2013 with the increase in ODs involving synthetic opioids (e.g., illicitly manufactured fentanyl). Accordingly, in the National Action Plan for Adverse Drug Event Prevention, opioids are 1 of 3 drug classes targeted.7 In 2017, the Department of Health and Human Services declared the opioid epidemic a public health emergency.8

Methods for Selecting Patient Safety Practices

Given the current urgency of harms due to opioids, we conducted a systematic review to identify potential patient safety practices for both primary care practice and other settings. Safety practices that were not fully addressed in existing guidelines, systematic reviews, or standards were prioritized.

Opioid stewardship (OS)—similar to antibiotic stewardship—consists of a range of risk reduction interventions or strategies, often used in combination to prevent adverse consequences from prescription opioids, including misuse, abuse, and OD.9,10 The range of OS interventions or strategies includes the following, several of which are recommended in the CDC Guideline for Prescribing Opioids for Chronic Pain:

  • Conduct an individualized assessment of risks and benefits of opioids, and appropriateness of a tapering; tapering slowly to minimize withdrawal symptoms11

  • Avoid co-prescribing opioids and benzodiazepines or other sedative hypnotics (as appropriate)

  • Use treatment agreements (TAs, also known as controlled-substance agreements or pain contracts)

  • Urine drug screening (UDS)

  • Check Prescription Drug Monitoring Programs (PDMPs)

  • Pain and functional assessment

  • Registry of patients with chronic pain or patients on chronic opioid therapy (COT)

  • Limiting days’ supply for acute pain opioid prescriptions

  • Pill counts to detect aberrant drug-related behavior

  • Referrals to nonpharmacologic treatment providers (e.g., physical therapy), pain management, behavioral health, or addictionspecialists

  • Risk assessment

Besides recommending these specific interventions, most OS initiatives also include implementation strategies to actually change practice; these implementation strategies are not necessarily unique to OS efforts (Table 1).12,13 The studies included in this review used a range of implementation strategies to change practice, including electronic health record (EHR) tools (e.g., clinical decision support, templates, alerts, integrated PDMPs, auto-populated fields), dashboards for monitoring and/or audit and feedback, provider and staff education and training, academic detailing, committee or task force on opioids, telehealth, and nurse care management.

TABLE 1.

Overview of Articles’ Opioid Stewardship and Implementation Strategies (by Setting)

graphic file with name pts-16-s36-g001.jpg

METHODS

The question of interest for this review was: What is the effect of OS interventions on key process outcomes (e.g., PDMPs, TA, UDS, referrals); intermediate and clinical outcomes (e.g., opioid dosage, opioid prescriptions, OD); and unintended consequences (e.g., change in pain)?

Literature Search Strategy

Two databases (CINAHL and MEDLINE) were searched for articles published from the past 10 years using terms for opioids, the outcomes of interest (opioid abuse, OD, death), and several terms for OS and OS strategies.

Study Selection

The initial search yielded 392 abstracts and an additional 16 studies identified from authors’ knowledge of the field, expert recommendation, and reference lists. After removing duplicates, records of 408 studies were screened from which 24 studies were reviewed for full text. Fourteen individual studies and 1 systematic review met the inclusion criteria (Fig. 1).

FIGURE 1.

FIGURE 1

Diagram for the selection of studies included in the review of OS interventions.

Studies were included if they evaluated an OS strategy or a multicomponent OS initiative to address potential harms of opioids. Studies that examined only effective pain management approaches were excluded, if they did not concurrently address potential opioid harms. Studies of naloxone (opioid OD reversal drug) prescribing alone were excluded from this review because of their focus on tertiary prevention (OD reversal) versus risk reduction with primary and secondary prevention strategies; no studies included in this review had naloxone prescribing as part of their initiatives.

Studies were included if they used experimental or quasi-experimental designs with pre/post measurement, with or without a control group. Observational studies and qualitative studies without tests of significance or with fewer than 50 patients they were excluded.

Studies were excluded if the outcomes were not relevant to this review (e.g., focused only on clinician outcomes such as knowledge or perceptions), if the article was out of scope, or if the report did not describe an intervention.

RESULTS

The 14 single studies that met the inclusion criteria were characterized in terms of their setting, OS strategies examined, study design, and outcomes (see Evidence Table in Supplementary Material, Supplemental Digital Content 1, http://links.lww.com/JPS/A315).

Settings

Ten studies examined1423 OS interventions in primary care settings: 3 of which were in federally qualified health centers (FQHCs)14,15 or safety net settings19 and 2 were in Veteran’s Administration (VA) clinics.16,18 One of the 10 studies in primary care settings examined a health systemwide OS initiative,23 which included primary care practices as well as emergency departments (EDs) and hospitals. Two studies examined OS in EDs,24,25 1 in a hospital outpatient surgery department,26 and another in an urgent care setting.27

Opioid Stewardship Strategies

Most studies examined multicomponent OS interventions, which often consisted of guideline-recommended clinical interventions or care processes (e.g., use UDS, check PDMPs), as well as implementation strategies (e.g., dashboards, audit, and feedback), which are described previously. Descriptions of the various OS initiatives varied in their level of detail. See Table 1 for an indication of the specific components of the OS interventions reflected in the literature included in this review.

Study Designs

Fourteen single studies and 1 systematic review were included in this review. Six of the 14 studies had a control group: 2 were randomized trials,19,25 3 were interrupted time series with control groups,17,20,21 and 1 was a 1-way crossover intervention study with patients serving as their own control. The remaining studies included 6 preintervention/postintervention studies without a control or comparison group and 2 observational studies with tests of significance. The postintervention periods ranged in these studies from months to years.

The overall strength of the evidence on OS was ranked low to moderate, with some variation by outcome examined.

Outcomes Examined

The most clinically significant harms of opioids are opioid addiction or opioid use disorder (OUD), OD, and death. Most studies did not examine the effect of OS initiatives on OUD or OD with a few exceptions.21 Most studies examined the effect of OS interventions on reducing the potential risks of opioids with judicious prescribing and guideline-concordant care (e.g., reduce inappropriate high opioid dosages, avoid co-prescribing opioids and benzodiazepines, use UDS, TAs).

The outcomes are presented by intermediate outcomes, process outcomes and utilization, and OD.

Intermediate Outcomes

Most studies examined intermediate outcomes including opioid prescribing and high opioid dosages and potential misuse.

Opioid Prescribing

Seven studies examined effects of OS on prescribing any amounts of opioids. The evidence is low to moderate that OS efforts decrease numbers of opioid prescriptions, proportion of patients on long-term opioids, or days’ supply.

Six of 7 studies observed significant reductions in opioid prescribing either in prestudies/poststudies or compared with control groups,15,2327 with the exception of Anderson et al14 (2016), who observed no significant decline in opioid prescribing.

Anderson et al15 (2015) observed reductions in the proportion of patients on COT after their OS intervention (3.4%–3.1%, P = 0.057). In addition, Weiner et al23 (2019) found a reduction in the number of unique patients with an opioid prescription each month (−52.6 patients, P < 0.001).23

Von Korff et al20 (2016) found a significant decline in the proportion of patients receiving excess opioid days’ supplied (from 24.0% to 10.4% among interventions patients on COT and from 20.1% to 14.7% among COT patients for the control practices).

Finally, Hartford et al26 examined a hospital-based outpatient surgery OS initiative and found that only 78 (45%) of 172 patients filled their opioid prescription in the postintervention group (P < 0.001), with no significant difference in prescription renewals.

High Opioid Dosages

Six studies examined the effect of their OS interventions on opioid dosages, measured as morphine milligram equivalents or MMEs.16,19,20,22,23,26 Four were in primary care settings,16,19,20,22 1 health systemwide,23 and 1 in hospital outpatient surgery.26 The strength of the evidence for OS initiatives producing significant reductions in opioid dosages was moderate.

Although the OS strategies varied and the postintervention periods ranged from months to years, included studies observed reductions in MMEs with varying magnitudes and measured in various ways. The following is a summary of the findings by the different measures of dosage used in the studies, some of which reported dosage more than 1 way. Von Korff et al20 (2016) found that mean daily MMEs decreased by 47% compared with control at 30%. Weimer et al22 reported an average daily dose decreased by 64 mg (95% confidence interval [CI] = 32–96, P < 0.001). In terms of dosage reduction, Liebschutz et al19 (2017) found that intervention patients had a mean MME 6.6 mg lower than controls (P < 0.001), and intervention patients were more likely than controls to have either a 10% MME dose reduction or opioid treatment discontinuation (adjusted odds ratio = 1.6).

Studies examined high dosage by the proportion of patients on high dosages and observed a range of reductions in patients on high dosages. Von Korff et al20 (2016) reported greater reductions in intervention versus control group (16.8%–6.3% [63% reduction] versus 20.6%–13.6% [34% lower]). Dorflinger et al16 found the proportion of patients receiving high-dose opioids decreased from 27.7% to 24.7%. In the health systemwide study, Weiner et al23 (2019) found a significant decrease in mean MMEs per prescription (−0.4 MME per month, P < 0.001), and prescriptions containing 90 or more MMEs also decreased (−48.1 prescriptions/month, P < 0.001), which may or may not be statistically significant.23 In the study of the OS initiative in general outpatient surgery, MMEs for prescriptions filled for the intervention group were significantly fewer than for controls.26

Potential Misuse

Few studies included in this review examined misuse outcomes. One ED study found that the total number of unique controlled-substance prescribers decreased from 11 to 7 (31% decrease, 95% CI = 23–38), which may or may not equate to fewer prescribers.24 Another study in primary care found no difference in early refills in their intervention group compared with control.19

Process Measures and Utilization

The primary outcome targeted by most OS initiatives was to improve utilization of recommended clinical interventions or care processes or “guideline-concordant care.” Five studies examined these various process outcomes.

Guideline-Concordant Care

In a randomized trial Liebschutz et al19 (2017) found that intervention patients were more likely than controls to receive guideline-concordant care (65.9% versus 37.8%, P < 0.001, adjusted odds ratio = 6.0, 95% CI = 3.6–10.2). Similarly, Jacobs et al18 found significant improvements in guideline-concordant care after the pharmacist-led intervention in a VA setting.

Urine Drug Testing

Four single studies examined the effect of OS initiatives on the use of annual UDS and observed significant increases in screening rates.14,16,18,19 In their systematic review, Starrels et al9 (2010) found low to moderate evidence of the effectiveness of urine drug testing (UDT) for reducing opioid misuse.

Prescription Drug Monitoring Program

One study that examined prescribers’ use of the PDMP found significant increases in utilization with opioid prescribing.18

Treatment Agreements or Informed Consent

Four studies examined the effect of OS initiatives on the proportion of patients on COT with a TA and found significant improvements.14,16,18,19 In a systematic review, Starrels et al9 (2010) found that opioid misuse was modestly reduced after TAs (with or without UDT).

Referrals

Several OS initiatives aimed to increase referral to behavioral health and other specialists. Anderson et al14 (2016) found significant increases in the percentage of patients with pain who had a visit with behavioral health provider in their FQHC, whereas Dorflinger et al16 (2014) did not observe an increase. Anderson et al14 (2016) observed a significant increase in referral to a chiropractor, and Dorflinger et al16 (2014) to physical therapy and pain management. Anderson et al14 (2016) also observed a significant decline in referrals to neurosurgery or orthopedic surgery and to pain specialists.

Documentation of Pain, Pain Treatment, and Pain Follow-up

After implementing an OS initiative aimed to improve documentation, Anderson et al14 (2016) observed significant increases in the documentation of the presence of pain (64%–82%, P = 0.001), the source and/or cause of pain (62%–74%, P = 0.025), functional status (5%–19%, P = 0.001), treatment plan (92%–98%, P = 0.002), and pain reassessment (17%–39%, P = 0.001).

Emergency Department Visits

Two studies examined OS initiatives in EDs and observed significant decreases in ED visits of 34% (decrease from 14 to 4, a 58% decrease, 95% CI = 50–66)24 and 58% (incident rate ratios = 0.663, P < 0.001, 95% CI = 0.569–0.775).25

Overdose

Two studies21,23 examined whether their OS initiatives reduced ODs; neither study observed significant reductions. Von Korff et al21 (2019) found that changes in OD rates among patients did not differ significantly between intervention and control groups with the implementation of 2 different OS initiatives (dose reduction and risk stratification/monitoring). Secondary analyses revealed that OD rates decreased significantly (17% per year) from the dose reduction OS initiative for patients on COT in intervention settings (relative annual change = 0.83, 95% CI = 0.70–0.99), but not in control settings (0.98, 95% CI = 0.70–1.39). Von Korff et al21 (2019) argued that the results are inconsistent given the differences observed in primary versus secondary analyses. Although Weiner et al23 (2019) observed a downward trend in ODs, it was not statistically significant.

DISCUSSION

Most OS interventions are multicomponent, involving both the clinical interventions or care processes and often implementation strategies. The implementation strategies found in this review included education, policies, dashboards, audit and feedback, monitoring and metrics, health information exchange, and EHR tools. The EHR tools included an embedded PDMP, registry, alerts, auto-population features, and templates.

When multicomponent interventions were included in this review, we did not examine the differential effectiveness of different components. None of the included studies used implementation or implementation effectiveness designs13,28 to afford a systematic evaluation of different implementation strategies’ effectiveness. Instead, the researchers of select studies offered reflections and informal observations on facilitators and barriers to implementation of their respective OS initiatives.

Anderson et al15 (2015) fielded a survey of the participating primary care providers about their opioid dashboard. Respondents found the dashboard helpful for identifying patients on long-term opioids and gaps in services (85%), clinically useful (77%), and easy to use (69%). Electronic health record tools were identified as key facilitators to OS.16,23 On the other hand, Dorflinger et al16 (2014) also found EHRs limiting because of the challenges with capturing complementary health approaches (e.g., chiropractic). Weiner et al23 (2019) reflect on several lessons learned. They found that determining metrics and gaining access to data were critical at the start of an initiative to guide the OS effort. They also experienced conflicting expectations when primary care providers referred patients to pain specialists with the expectation that the pain physicians would prescribe opioids, whereas the specialists would only recommend opioid regimens and provide injections. In addition, although their health system had increased substance use disorder treatment, their outpatient practices perceived that there was inadequate access. Finally, they learned that many of these implementation challenges could be addressed by convening the various stakeholders to resolve the issues. Buy-in and administrative support were identified as key for 2 OS initiatives, as well.24,25

This systematic review expands the evidence on OS interventions beyond what was known from previous reviews of specific interventions but still points to several gaps and future directions for reducing the potential harms due to opioids. There is a need for more detailed descriptions of the OS initiatives to replicate the interventions in other practices and settings, as well as rigorously synthesize the evidence across studies. In addition, improving the quality of future studies with control groups to account for secular trends, given the attention on the opioid epidemic and changing external environment, policies, regulations, and evidence, is critical. Examining the benefits of different implementation strategies for changing practice in OS efforts and in different settings is also needed. The studies included in this review were not only in primary care settings but also in health systems, in EDs, and an urgent care center; there is still a need to further understand the uniqueness and effectiveness of OS efforts in different settings.

With the evolving epidemic and response to it, it may be important to also examine the effect of co-prescribing naloxone for patients on long-term opioid therapy on outcomes of interest, as well as examiner how best to identify and treat or refer patients using illicit opioids. It should be noted that most OS efforts are aimed at preventing or reducing harms due to opioids with appropriate prescribing and do not examine the unintended consequences.

CONCLUSIONS

This review offers additional insights for the field in responding to the opioid epidemic in the health care system and points to several potential gaps to expand our knowledge of the effectiveapproaches.

Supplementary Material

SUPPLEMENTARY MATERIAL
pts-16-s36-s001.docx (40.4KB, docx)

Footnotes

The authors disclose no conflict of interest.

This work was funded by the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services (Contract HHSP233201500013I; HHSP23337002T).

Supplemental digital contents are available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.journalpatientsafety.com).

REFERENCES

  • 1.Dahlhamer J Lucas J Zelaya C, et al. . Prevalence of chronic pain and high-impact chronic pain among adults - United States, 2016. MMWR Morb Mortal Wkly Rep. 2018;67:1001–1006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Chou R Turner JA Devine EB, et al. . The effectiveness and risks of long-term opioid therapy for chronic pain: a systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med. 2015;162:276–286. [DOI] [PubMed] [Google Scholar]
  • 3.Von Korff M Kolodny A Deyo RA, et al. . Long-term opioid therapy reconsidered. Ann Intern Med. 2011;155:325–328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Centers for Disease Control and Prevention [CDC]. U.S. Opioid Prescribing Rate Maps Available at: https://www.cdc.gov/drugoverdose/maps/rxrate-maps.html. Accessed May 16, 2019.
  • 5.Scholl L Seth P Kariisa M, et al. . Drug and opioid-involved overdose deaths – United States, 2013-2017. MMWR Morb Mortal Wkly Rep. 2018;67:1419–1427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kolodny A Courtwright DT Hwang CS, et al. . The prescription opioid and heroin crisis: a public health approach to an epidemic of addiction. Annu Rev Public Health. 2015;36:559–574. [DOI] [PubMed] [Google Scholar]
  • 7.U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion National Action Plan for Adverse Drug Event Prevention. Washington, DC: U.S. Department of Health and Human Services; 2014. Available at: https://health.gov/sites/default/files/2019-09/ADE-Action-Plan-508c.pdf. Accessed May 1, 2020. [Google Scholar]
  • 8.US Department of Health and Human Services [HHS] HHS acting secretary declares public health emergency to address national opioid crisis. Available at: https://www.hhs.gov/about/news/2017/10/26/hhs-acting-secretary-declares-public-health-emergency-address-national-opioid-crisis.html. Accessed May 16, 2019.
  • 9.Starrels JL Becker WC Alford DP, et al. . Systematic review: treatment agreements and urine drug testing to reduce opioid misuse in patients with chronic pain. Ann Intern Med. 2010;152:712–720. [DOI] [PubMed] [Google Scholar]
  • 10.Dowell D, Haegerich T, Chou R. CDC guideline for prescribing opioids for chronic pain - United States, 2016. MMWR Recomm Rep. 2016;65:1–49. [DOI] [PubMed] [Google Scholar]
  • 11.Dowell D, Haegerich T, Chou R. No shortcuts to safer opioid prescribing. N Engl J Med. 2019;380:2285–2287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Powell BJ Waltz TJ Chinman MJ, et al. . A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Shoemaker-Hunt SJ. Implementation Research to Understand Effective Approaches to Opioid Management in Health Systems. Issue Brief. AcademyHealth. December 2019. Available at: https://www.academyhealth.org/publications/2019-12/issue-brief-highlights-importance-implementation-science-address-opioid-epidemic. Accessed May 1, 2020. [Google Scholar]
  • 14.Anderson DR Zlateva I Coman EN, et al. . Improving pain care through implementation of the Stepped Care Model at a multisite community health center. J Pain Res. 2016;9:1021–1029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Anderson D Zlateva I Khatri K, et al. . Using health information technology to improve adherence to opioid prescribing guidelines in primary care. Clin J Pain. 2015;31:573–579. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Dorflinger L Moore B Goulet J, et al. . A partnered approach to opioid management, guideline concordant care and the stepped care model of pain management. J Gen Intern Med. 2014;29(suppl 4):870–876. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Dublin S Walker RL Shortreed SM, et al. . Impact of initiatives to reduce prescription opioid risks on medically attended injuries in people using chronic opioid therapy. Pharmacoepidemiol Drug Saf. 2019;28:90–96. [DOI] [PubMed] [Google Scholar]
  • 18.Jacobs SC Son EK Tat C, et al. . Implementing an opioid risk assessment telephone clinic: outcomes from a pharmacist-led initiative in a large Veterans Health Administration primary care clinic, December 15, 2014-March 31, 2015. Subst Abus. 2016;37:15–19. [DOI] [PubMed] [Google Scholar]
  • 19.Liebschutz JM Xuan Z Shanahan CW, et al. . Improving adherence to long-term opioid therapy guidelines to reduce opioid misuse in primary care: a cluster-randomized clinical trial. JAMA Intern Med. 2017;177:1265–1272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Von Korff M Dublin S Walker RL, et al. . The impact of opioid risk reduction initiatives on high-dose opioid prescribing for patients on chronic opioid therapy. J Pain. 2016;17:101–110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Von Korff M Saunders K Dublin S, et al. . Impact of chronic opioid therapy risk reduction initiatives on opioid overdose. J Pain. 2019;20:108–117. [DOI] [PubMed] [Google Scholar]
  • 22.Weimer MB Hartung DM Ahmed S, et al. . A chronic opioid therapy dose reduction policy in primary care. Subst Abus. 2016;37:141–147. [DOI] [PubMed] [Google Scholar]
  • 23.Weiner SG Price CN Atalay AJ, et al. . A health system-wide initiative to decrease opioid-related morbidity and mortality. Jt Comm J Qual Patient Saf. 2019;45:3–13. [DOI] [PubMed] [Google Scholar]
  • 24.Kahler ZP Musey PI Schaffer JT, et al. . Effect of a “no superuser opioid prescription” policy on ED visits and statewide opioid prescription. West J Emerg Med. 2017;18:894–902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Neven D Paulozzi L Howell D, et al. . A randomized controlled trial of a citywide emergency department care coordination program to reduce prescription opioid related emergency department visits. J Emerg Med. 2016;51:498–507. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Hartford LB Van Koughnett JAM Murphy PB, et al. . Standardization of outpatient procedure (STOP) narcotics: a prospective non-inferiority study to reduce opioid use in outpatient general surgical procedures. J Am Coll Surg. 2018;228:81–88. [DOI] [PubMed] [Google Scholar]
  • 27.Young LS, Crausman RS, Fulton JP. Suboptimal opioid prescribing: a practice change project. R I Med J (2013). 2018;101:41–44. [PubMed] [Google Scholar]
  • 28.Curran GM Bauer M Mittman B, et al. . Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217–226. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

SUPPLEMENTARY MATERIAL
pts-16-s36-s001.docx (40.4KB, docx)

Articles from Journal of Patient Safety are provided here courtesy of Wolters Kluwer Health

RESOURCES