Skip to main content
NCBI home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2017 Dec 1.
Published in final edited form as: Ann Emerg Med. 2016 Apr 29;68(6):719–728. doi: 10.1016/j.annemergmed.2016.03.007

A Randomized Trial Testing the Impact of Narrative Vignettes vs. Guideline Summaries on Provider Response to a Professional Organization Clinical Policy for Safe Opioid Prescribing

Zachary F Meisel 1, Joshua P Metlay 1, Lauren Sinnenberg 1, Austin S Kilaru 1, Anne Grossestreuer 1, Frances K Barg 1, Frances S Shofer 1, Karin V Rhodes 1, Jeanmarie Perrone 1
PMCID: PMC5086322  NIHMSID: NIHMS767180  PMID: 27133392

Abstract

Background

Clinical guidelines are known to be underused by practitioners. In response to the challenges of treating pain amidst a prescription opioid epidemic, the American College of Emergency Physicians published an evidence-based clinical policy for opioid prescribing in 2012. Evidence-based narratives, an effective method of communicating health information in a variety of settings, offer a novel strategy for disseminating guidelines to physicians and engaging providers with clinical evidence.

Objectives

To compare whether narrative vignettes embedded in the American College of Emergency Physician (ACEP) daily e-newsletter improved dissemination of the clinical policy to ACEP members, and engagement of members with the clinical policy, compared to traditional summary text.

Methods

A prospective randomized controlled study, entitled Stories to Promote Information using Narrative (SPIN) trial, was performed. Derived from qualitative interviews with 61 ACEP physicians, 4 narrative vignettes were selected and refined, using a consensus panel of clinical and implementation experts. All ACEP members were then block randomized by state of residence to receive alternative versions of a daily emailed newsletter for a total of 24 days during a 9 week period. Narrative newsletters contained a selection of vignettes that referenced opioid prescription dilemmas. Control newsletters contained a selection of descriptive text about the clinical policy using similar length and appearance to the narrative vignettes. Embedded in the newsletters were web links to the complete vignette or traditional summary text, as well as additional links to the full ACEP clinical policy and a website providing assistance with prescription drug monitoring program enrollment. The newsletters were otherwise identical. Outcomes measured were the percentage of subjects who visited any of the web pages that contained additional guideline related information and the odds of any unique physician visiting these web pages during the study.

Results

27,592 physicians were randomized and 21,226 received the newsletter during the study period. When counting each physician once over the study period, there were 509 unique visitors in the narrative group and 173 unique visitors in the control group (4.8% vs. 1.6%, difference 3.2% 95%CI 2.7%-3.7%). There were 744 gross visits from the e-newsletter to any of the three web pages in the narrative group compared to 248 in the control group (7.0% vs. 2.3%, OR 3.2 (95% CI 2.7-3.6). Over the course of the study, the odds ratio of any physician in the narrative group visiting one of the three informational web sites compared to the control group was 3.1 (95% CI 2.6-3.6).

Conclusion

Among a national sample of emergency physicians, narrative vignettes outperformed traditional guideline text in promoting engagement with an evidence-based clinical guideline related to opioid prescriptions.

INTRODUCTION

Background

The US and Canada are experiencing an epidemic of prescription drug abuse, with deaths from poisonings exceeding those from motor vehicle crashes since 2008.1,2 This epidemic is widely acknowledged to be driven by dramatic increases in opioid sales and prescribing, resulting in their expanded availability and frequent diversion for nonmedical use.3 Emergency Departments (EDs) are integral in the resuscitation of opioid poisoned patients and, paradoxically, are also among the top writers of opioid prescriptions for patients under age 40.3

Medical providers, particularly those who deliver acute unscheduled care, are challenged daily when facing patients with moderate to severe pain given the imperative to provide prompt and effective analgesia. Increasingly, the choices made for individuals with painful complaints are influenced by the societal risks of prescription drug abuse. Previous work finds that emergency physicians express frustration about how to best navigate the need to provide optimum pain relief to their patients while also serving as stewards of these controlled medications.4 Indeed, significant variations in opioid prescribing practices have been demonstrated.3,5,6 To address these variations amidst a public health crisis, physician organizations, government entities, and local hospitals and health systems have drafted guidelines to assist clinicians in the management of common pain complaints.4,7–11

Importance

The net impact of any clinical guideline depends on the extent to which it is disseminated to and adopted by providers.12 Many factors have been shown to impede the adoption of clinical guidelines, including unawareness of their existence, disagreement with their content, and the challenges of changing established practices.12,13 Strategies to promote the dissemination and implementation of clinical guidelines are acknowledged to have variable effectiveness and there is a limited evidence base to support any single approach.14–19 Little is known about how individual clinician specialists adopt and use existing opioid guidelines. Prior work found that emergency physicians use guidelines more often as a communication tool to inform patients about prescribing decisions.4 When queried specifically about opioid prescription guidelines, emergency physicians have expressed general favorability, but have varied in awareness of the specific content.4,20.

One potential approach to improving guideline dissemination and engagement includes the use of evidence-based narratives. Narratives are defined as cohesive stories with an identifiable beginning, middle and end that include characters, raise questions, and provide resolution.21 The theoretical underpinnings for narrative approaches to promote health behavior change include social cognitive theory and the theory of reasoned action.22,23 Many comparative studies of narratives on health have been conducted with the aim of modifying patient, not provider, behavior. Although physicians are known to use narrative sources to learn about health information, most programs for disseminating health care providers use summary methods (syntheses of data or recommendations presented without narrative context) alone.24,25 Guideline writers have been encouraged to use narrative based approaches to disseminate recommendations; to date, these methods have not been empirically tested.26 Based on previous studies comparing an evidence-based narrative to standard guideline text on emergency physician knowledge and recall of opioid prescribing guideline information, 27 we sought to test this approach on provider response among a national sample of emergency physicians.

Goals of the Investigation

In October 2012, the American College of Emergency Physicians, with support from the U.S. Centers for Disease Control and Prevention (CDC) published evidence-based guidelines to direct emergency physicians in the management of pain and to encourage use of state-based Prescription Drug Monitoring Programs (PDMPs) prior to opioid prescribing.8 We sought to compare whether guideline information presented in narrative format, compared to traditional summary form, improved the way physician members responded electronically to the ACEP practice guideline as a marker of dissemination and engagement.

MATERIALS AND METHODS

Study Design and Setting

This was a national prospective randomized controlled trial of emergency physicians (Stories to Promote Information Using Narrative, or SPIN). To develop the narrative intervention, vignettes were empirically generated using both established techniques in narrative communication28 as well as data collected during an earlier phase of this study. The earlier phase of the study took place in October of 2012, during which semi-structured interviews were conducted with emergency physicians attending the ACEP Scientific Assembly in Denver, CO, focusing on attitudes and beliefs related to pain control, opioid prescriptions, and perceived barriers and facilitators to the use of practice guidelines. The methods and findings from a qualitative analysis of these interviews have been described previously. 4,29 The common themes identified during the interviews were: local vs. national guidelines, lack of engagement, communication with patients, physician autonomy, standardization, public health perspectives, liability, hospital case-mix, word of mouth/spread of policy, and patient expectations.

Selection of Participants

All physician ACEP members (excepting students) as of September 1, 2014 were block randomized by state of residence to receive alternative versions of the ACEP Today newsletter for a total of 24 days between October 13, 2014 and December 19, 2014. The newsletter is provided free of charge to due paying members and members can opt out of receiving the newsletter at the time of joining or renewing membership. The newsletter, produced by a professional medical information firm, in cooperation with ACEP, contains general health and health policy news content as well as specific information for emergency medicine and ACEP members. Those who opted out of receiving the daily email newsletter were excluded. ACEP represents more than 33,000 emergency physicians, emergency medicine residents and medical students from across the United States (out of a total estimated 45,000 EM physicians in the US) with a small subset of international members. Block randomization was performed to ensure equal distribution of the interventions by state, for which policies related to opioid prescriptions and PDMP use vary. International members were coded as a single block in the randomization scheme

Intervention Development and Deployment

Narrative Intervention

The themes elicited in the first phase of the study served as the framework for a series of 10 written vignettes, constructed to optimize known elements of effective narrative communication. Each story contained at least 4 of the 10 identified themes, and their content was grounded in real stories collected during the interview process. The stories ranged from narratives about physicians, to others about patients. Some had a “scary” ending with negative outcomes. Others were deemed more uplifting. Some stories featured patients with legitimate analgesia needs and others featured patients with aberrant behavior. A consensus panel composed of clinical and implementation experts, professional journalists, and editors was used to measure, rank, and select four narratives vignettes for use in this guideline dissemination trial. The selection process used a validated scale for elements of effective narrative communication, focusing on the ability to “transport the reader”. 30 Each narrative vignette referenced the ACEP guideline and incorporated at least one of its core recommendations (e.g. for emergency providers to use a prescription drug monitoring program prior to prescribing opioids). The narrative vignettes described specific characters and contained conflict and resolution focusing on these theoretical elements of behavior change.

Each “narrative” newsletter contained a text box, headlined “Stories from ACEP Physicians” containing a short selection (or “teaser”) from one of the four narrative vignettes. Each text box contained 3 web hyperlinks as shown in Figure 1. The hyperlinks, when selected, led newsletter readers to one of three ACEP hosted websites containing respectively: 1) the complete narrative text associated with the text-box selection, 2) the official 2012 ACEP opioid clinical policy, or 3) a website containing a map of every U.S. state with additional hyperlinks to each state's PDMP enrollment site (Figure 2).

Figure 1.

Figure 1

Open in a new tab

Text boxes embedded in the “Emergency Medicine Today” newsletter for the narrative and control groups with experimental hyperlinks (see Appendix B for example of newsletter).

Figure 2.

Figure 2

Open in a new tab

ACEP PDMP enrollment webpage.

Control Intervention

The control text was written by the investigators and used descriptive language in a style similar to summaries that routinely accompany practice guidelines. It referenced specific elements of the guideline, including the core recommendation to use prescription monitoring programs, and was edited and vetted by the expert panel for clarity and similarity to other guideline summaries from ACEP. (See Appendix A for complete examples of narrative and control text displayed on the ACEP hosted webpages).

Control newsletters contained a text box with a selection of descriptive text about the guideline with similar length and appearance to the narrative newsletter text box. The control newsletters, otherwise identical to the narrative newsletters, likewise contained three web hyperlinks to an expanded version of the control text, the ACEP guideline and the PDMP enrollment map. The newsletters, sent to physicians randomized to either the narrative or control groups, were, beyond the experimental text-boxes, otherwise identical, including the subject line of the daily emails.

Deployment

For both the narrative and control groups, the newsletter-linked web pages also contained hyperlinks to the other engagement sites. For example, the “read more” pages (comprised of the full narrative vignettes or complete guideline summary text) each contained additional hyperlinks to the PDMP map and ACEP guideline website. The four narratives were published on a weekday rotation during the 9 weeks of the study period, broken into three time periods separated by two week “rest periods” where no study-related content was included in the newsletters. Control content did not vary over the course of the study. For each day that the experiment ran, all subjects received either a narrative or control newsletter, depending on the initial randomized allocation. Subjects were blinded to the presence of the experiment. (See Appendix B for examples of the “ACEP Today” newsletter with a narrative and control text box).

Methods of Measurement and Outcome Measures

Primary outcomes measured were unique visitors to the web pages involved in the study. Unique visitors counts each enrolled physician only once over the study period. The web pages, which were accessed via hyperlinks embedded in the newsletter, contained the full experimental text (narrative or control), complete guideline information or provided assistance with enrolling in PDMPs. Secondary outcomes included percentage of physicians who opened the newsletter email. Data collected from ACEP included visits to the complete guideline or PDMP enrollment page. ACEP's newsletter vendor provided additional data including gross and unique newsletter, guideline, and PDMP enrollment page views originating from the newsletter itself. Because the hyperlinks were embedded in emailed newsletters, the analytics program was able to assign a unique identifier to each member who visited the websites of interest. During analysis, we made sure to count each ACEP member number only once when measuring unique visitors. Therefore, if a user used 2 different computers, the overall count would not be impacted.

Primary Data Analysis

Data were summarized with means and percentages. Data were imported from Google Analytics (Google, Mountain View, CA) into an Excel spreadsheet (version 12.3.3; Microsoft, Redmond, WA) in which descriptive measures were calculated. Google Analytics (GA) is a web analytics tool that provides information and statistics about website visitors and their activity. GA has been used previously in project impact evaluations.31,32 Multivariable logistic regression, adjusting for state of residence, was used to compare the web visits from the narrative and control newsletters. Breslow-Day test was used to assess the homogeneity between the narratives. Analyses were completed in SAS (version 9.3, Cary, NC). IRB approval was obtained from the University of Pennsylvania to conduct this study.

RESULTS

Overall, 27,592 non-student ACEP physician members were randomized, 13,796 to the control group, and 13,796 to the narrative group. After randomization, members who opted out of receiving the daily email newsletters were not included in the study, leaving 10,631 physicians in the narrative group and 10,595 physicians in the control group (Figure 3).The total number of times the newsletters were viewed by ACEP members over the study period was similar between the control group and the narrative group (96,814 vs 96,173).

Figure 3.

Figure 3

Open in a new tab

CONSORT Diagram

During the study period, physicians randomized to the narrative group were significantly more likely to visit any of the web pages related to the expanded informational text, the official guideline or the PDMP enrollment sites (Table 1). There were 744 gross visits from the newsletter to any of the three web pages in the narrative group compared to 248 in the control group (7.0% vs. 2.3%, difference 4.7% 95%CI 4.1%-5.3%). When counting each physician only once over the study period (unique visits), there were 509 unique visitors in the narrative group and 173 unique visitors in the control group (4.8% vs. 1.6%, difference 3.2% 95%CI 2.7%-3.7%). When analyzing unique visitors to only the PDMP enrollment website or the original ACEP clinical guideline, 214 narrative subjects visited these sites compared to 151 control subjects (2.0% vs. 1.4%, difference 0.6% 95%CI 0.25%-0.95%). Over the course of the study, the odds ratio of any physician visiting one of the three informational web sites in the narrative group compared the control group was 3.06 (95% CI 2.57-3.66). After adjusting for state of residence, physicians in the narrative arm demonstrated similarly increased odds of visiting any of the guideline related websites (OR 3.07, 95%CI 2.57-3.67).

Table 1.

Website visitors in the narrative vs. control group. Gross visitors include all narrative or control text page visits. Unique visitors counts each participant only once over the entire study period.

Narrative (N=10595) Control (N=10631) Odds Ratio
Gross visits 744 (7.0%) 248 (2.3%) 3.2 (95% CI 2.7-3.6)
Unique visitors 509 (4.8%) 173 (1.6%) 3.1 (95% CI 2.6-3.6)
Unique PDMP or Clinical Policy visitors 214 (2.0%) 151 (1.4%) 1.4 (95% CI 1.2-1.8)
Open in a new tab

Subjects had multiple opportunities to visit the informational websites over the course of the study. Table 2 demonstrates website visitors sorted by each of the four narratives compared to the control on the days those narratives were published. Narrative subjects had higher visit rates than control subjects on any given study day (Figure 4). When the narratives were compared against one another, no significant difference was found in physician visits (p=.09).

Table 2.

Unique website visitors of each of the four narratives compared to the control on the days those narratives were published. Percentages represent the number of visits or visitors per person in the study group.

Narrative #1 (N=10595) Control (N=10631) Odds Ratio
Unique visitors by day 192 (1.81%) 50 (0.49%) 3.8 (95% CI 2.8-5.1)
Narrative #2 Control
Unique visitors by day 169 (1.60%) 34 (0.35%) 4.6 (95% CI 3.2-6.6)
Narrative #3 Control
Unique visitors by day 165 (1.56%) 62 (0.58%) 2.7 (95% CI 2.0-3.6)
Narrative #4 Control
Unique visitors by day 150 (1.42%) 37 (0.35%) 4.1 (95% CI 2.9-5.9)
Open in a new tab

Figure 4.

Figure 4

Open in a new tab

Visits and unique visitors by day for each newsletter in the narrative and control groups.

On two specific days (10/20/2014 and 11/12/2014), website visits were notably higher –among both the control and narrative groups – than on other days. These newsletters had unique features on these days compared to other days including: email subject lines that referenced a highly newsworthy topic at the time (Ebola virus in the U.S.) and the placement of the experimental opioid guideline text boxes immediately following the Ebola virus news articles. On these days, the narrative newsletters continued to perform well compared to the control newsletters in generating visits to the guideline information sites (52 and 68 unique physician visits in the narrative arm compared to 23 and 13 visits in the control arm).

LIMITATIONS

This study measured visits by physicians to web pages that contained guideline information. Implementation science frameworks, such as RE-AIM, identify Reach and Engagement as proximal to potentially more meaningful outcomes such as adoption, implementation and maintenance.33–35 In this study, despite the demonstrated comparative efficacy of evidenced based narratives over summary text in drawing physician members to visit additional web material (i.e. reach and engagement), we were unable to determine if the physicians assigned to either communication approach were more likely to adopt, implement, and maintain behaviors recommended by the guideline. Additional limitations include the presence of new policies, substantial controversy, and national interest related to opioid use in emergency medicine during the duration of the study, which may have interacted with the primary effect of the communication strategy. Although we had previously identified challenges with PDMP enrollment as key barriers to adoption during the interview phase of the study, the policy and clinical landscape had evolved during the course of the study (including mandatory PDMP use for physicians in some states and lack of available PDMPs in others) potentially making one of the study's outcome measures (focused on aiding physicians with the process of enrolling in their PDMP) less salient.36 Many of these policies are, however, specific to individual states and we were able to account for some of these potential influences by block randomizing by state and adjusting by state in the analysis.

Because ACEP does not collect complete individual demographic or professional level data on its members and because the data were anonymized prior to analysis, we were unable to determine if gender, experience level or other individual characteristics modified the effect of the intervention. Because the control content did not vary over the course of the study, the narrative newsletters did introduce more content variation than the control newsletters, a potential source of bias. However, we did not observe comparative trends in increased (or decreased) physician visits as the narrative content became more varied over the study period. Another limitation is that the generalizability of a narrative approach to other guideline implementation efforts (particularly for guidelines with lower baseline national attention) remains unknown.

DISCUSSION

Most guideline dissemination efforts use summary methods that synthesize data and provide recommendations without narrative context.37 Strategies to promote the dissemination and implementation of evidence-based clinical guidelines are acknowledged to have variable effectiveness and a limited evidence base.38 At a time when many providers are bombarded with information and daily email, the SPIN study adds to the literature by providing comparative evidence for a potentially low cost dissemination and engagement strategy for an evidence-based practice guideline. In this randomized trial, we found that while an equal number of physicians opened the daily electronic newsletter, narrative text about the opioid guideline was more likely to engage physicians, as measured by online visits to websites providing additional information, compared to standard summary text. The narrative vignettes not only engaged physicians at levels higher than the summary text, but also outperformed national marketing benchmarks for email “click-engagement” campaigns (2.9% average unique click rates for health).39

The findings of this study, the SPIN trial, align with previously described aspects of dissemination science. Research suggests that stories or narratives may be particularly effective ways to communicate information, particularly information that people might otherwise ignore or resist.40 The mechanisms underlying this process are multifactorial. The transportation-likelihood theory suggests that when information is communicated in stories, people are absorbed (or “transported”) into the narrative in a way that increases its persuasive potential.30 Stories also shape perceptions of norms. Narratives can allow people to observe behavior when it is modeled by others which may make them more likely to try the behavior. Social cognitive theory suggests that “observational learning” is an important contributor to behavioral change, and that being exposed to other people performing that behavior (even through media) is important.41 In this study, the narratives were gathered from the provider perspective—each contained descriptions of physicians using and acting on the guideline recommendations. Observing a colleague in a behavior could alter individuals’ perceptions of norms such that they believe norms are more supportive of that behavior.42 At their simplest, persuasive narratives have been defined as “coherent stories with an identifiable beginning, middle, and end that provides information about scene, characters, and conflict; raises unanswered questions or unresolved conflict; and provides resolution.”28 Our previous experimental work with emergency medicine resident physicians demonstrated that narratives can help clinicians remember and recall content from opioid prescribing guidelines, compared to standard guideline content alone.27 To our knowledge, narrative communication has not been tested, prior to this study, as a way to engage a national audience of providers with guideline content.

It is worth exploring the additional, and unexpected, findings that nearly one quarter of guideline related visits in both arms occurred on two days during the entire study period. As noted above, these specific newsletters had the unique feature of displaying the guideline related text boxes immediately adjacent to the high profile news article that had been referenced in the email subject line. On both of these high visit days, the narrative text outperformed the control text, suggesting that newsletter layout and subject line “hooks” may enhance the overall effect of narrative communication.

In this study, the absolute difference between the narrative and control newsletters as measured by unique web visits to the proxy sites of interest was a modest 3.2% over the course of the study. However, when multiplied by the number of ACEP members who regularly receive the newsletter, an estimated 680 additional physicians in the narrative group, over a 9 week period would be likely read/open the guideline or indicate an intention to pursue one of its recommendations.

There are substantial future opportunities to explore the use of narrative methods as a way to disseminate and implement evidence-based guidelines. We believe that emailed evidence-based narratives may represent an efficient and cost effective strategy to disseminate evidence and promote guideline adoption. Given the time, effort and expense that is devoted to guideline development and dissemination, and given the historically poor rates of guideline adoption, we believe that many would find these results, while modest, to be valuable for future guideline dissemination efforts.

Although substantial time and resources were committed to performing thorough interviews and rigorous qualitative analysis to build the intervention for this study, future studies could test emailed newsletter stories built on published principles of persuasive narratives without the use of formal stakeholder interviews and analysis. 4,20

In summary, narrative vignettes led to higher levels of engagement with opioid prescription guideline content among a national sample of emergency physicians. Organizations invested in producing and communicating clinical evidence might consider adopting narrative techniques to improve the dissemination and impact of evidence based polices.

Supplementary Material

1
2

Acknowledgements

We wish to thank Jim Steele, Nancy Steele, Charlotte Greensit, Hoag Levins, and Roy Rosin for their invaluable advice and review of components of this manuscript. We also wish to thank Cynthia Singh, Nancy Calloway and Kathryn Mensah from the American College of Emergency Physicians and Danielle Sturgis from Bulletin Media/ACEP Today for their invaluable support and assistance with this project. We also wish to express our deep appreciation to Jim Frederick who provided invaluable comradery and advice on the narrative portion of the study, but who died before the project was completed.

Funding and support: Agency for Healthcare Research and Quality (AHRQ) 5R18HS021956 and NIH Career Development in Comparative Effectiveness Research Award (1KM1CA156715).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Presented at the Society for Academic Emergency Medicine annual meeting, May 2015, San Diego, CA; and the Academy Health Annual Research Meeting (ARM),June 2015, Minneapolis, MN.

ZFM, JPM, JP, and FKB conceived the study, designed the trial, and obtained research funding. ZFM, ASK, AG supervised the conduct of the trial and data collection. FSS and FKB provided statistical advice on study design and analyzed the data. ZFM drafted the manuscript, and all authors contributed substantially to its revision. ZFM takes responsibility for the paper as a whole.

Bibliography

  • 1.Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in Drug and Opioid Overdose Deaths — United States, 2000–2014. MMWR Morb Mortal Wkly Rep. 2015;64:1–5. doi: 10.15585/mmwr.mm6450a3. [DOI] [PubMed] [Google Scholar]
  • 2.Paulozzi LJ, Kilbourne EM, Desai HA. Prescription drug monitoring programs and death rates from drug overdose. Pain Med. 2011;12:747–754. doi: 10.1111/j.1526-4637.2011.01062.x. doi:10.1111/j.1526-46372011.01062.x. [DOI] [PubMed] [Google Scholar]
  • 3.Volkow ND, McLellan TA, Cotto JH, Karithanom M, Weiss SRB. Characteristics of opioid prescriptions in 2009. JAMA. 2011;305:1299–1301. doi: 10.1001/jama.2011.401. doi:10.1001/jama.2011.401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Kilaru AS, Gadsden SM, Perrone J, Paciotti B, Barg FK, Meisel ZF. How Do Physicians Adopt and Apply Opioid Prescription Guidelines in the Emergency Department? A Qualitative Study. Ann Emerg Med. 2014 doi: 10.1016/j.annemergmed.2014.03.015. doi:S0196-0644(14)00221-2 [pii]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Chen JH, Humphreys K, Shah NH, Lembke A. Distribution of Opioids by Different Types of Medicare Prescribers. JAMA Intern Med. 2015:1–3. doi: 10.1001/jamainternmed.2015.6662. doi:10.1001/jamainternmed.2015.6662. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hoppe JA, Nelson LS, Perrone J, Weiner SG. Opioid Prescribing in a Cross Section of US Emergency Departments. Ann Emerg Med. 2015;66(3):253–259. e1. doi: 10.1016/j.annemergmed.2015.03.026. doi:10.1016/j.annemergmed.2015.03.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Del Portal DA, Healy ME, Satz WA, McNamara RM. Impact of an Opioid Prescribing Guideline in the Acute Care Setting. J Emerg Med. 2016;50(1):21–27. doi: 10.1016/j.jemermed.2015.06.014. doi:10.1016/j.jemermed.2015.06.014. [DOI] [PubMed] [Google Scholar]
  • 8.Cantrill S V, Brown MD, Carlisle RJ, et al. Clinical policy: Critical issues in the prescribing of opioids for adult patients in the emergency department. Ann Emerg Med. 2012;60:499–525. doi: 10.1016/j.annemergmed.2012.06.013. doi:10.1016/j.annemergmed.2012.06.013. [DOI] [PubMed] [Google Scholar]
  • 9.Franklin GM, Mai J, Turner J, Sullivan M, Wickizer T, Fulton-Kehoe D. Bending the prescription opioid dosing and mortality curves: Impact of the Washington State opioid dosing guideline. Am J Ind Med. 2012;55(4):325–331. doi: 10.1002/ajim.21998. doi:10.1002/ajim.21998. [DOI] [PubMed] [Google Scholar]
  • 10.Kunins HV, Farley T a., Dowell D. Guidelines for opioid prescription: Why emergency physicians need support. Ann Intern Med. 2013;158:841–842. doi: 10.7326/0003-4819-158-11-201306040-00631. doi:10.7326/0003-4819-158-11-201306040-00631. [DOI] [PubMed] [Google Scholar]
  • 11.Wattana MK, Nelson LS, Todd KH. Prescription opioid guidelines and the emergency department. J Pain Palliat Care Pharmacother. 2013;27(2):155–162. doi: 10.3109/15360288.2013.788602. doi:10.3109/15360288.2013.788602. [DOI] [PubMed] [Google Scholar]
  • 12.Cabana MD. Why don't physicians follow clinical practice guidelines. JAMA J Am Med Assoc. 2008;282(15):1458–1465. doi: 10.1001/jama.282.15.1458. doi:10.1001/jama.282.15.1458. [DOI] [PubMed] [Google Scholar]
  • 13.Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–1465. doi: 10.1001/jama.282.15.1458. doi:10.1001/jama.282.15.1458. [DOI] [PubMed] [Google Scholar]
  • 14.French SD, Green SE, O'Connor DA, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7:38. doi: 10.1186/1748-5908-7-38. doi:10.1186/1748-5908-7-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Green LW, Ottoson JM, García C, Hiatt R a. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health. 2009;30:151–174. doi: 10.1146/annurev.publhealth.031308.100049. doi:10.1146/annurev.publhealth.031308.100049. [DOI] [PubMed] [Google Scholar]
  • 16.Grimshaw J, Campbell M, Eccles M, Steen N. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract. 2000;17(Suppl 1):S11–S16. doi: 10.1093/fampra/17.suppl_1.s11. doi:10.1093/fampra/17.suppl_1.S11. [DOI] [PubMed] [Google Scholar]
  • 17.Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001;39(8 Suppl 2):II46–I54. doi: 10.1097/00005650-200108002-00003. doi:10.1097/00005650-200108002-00003. [DOI] [PubMed] [Google Scholar]
  • 18.Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342(8883):1317–1322. doi: 10.1016/0140-6736(93)92244-n. doi:10.1016/0140-6736(93)92244-N. [DOI] [PubMed] [Google Scholar]
  • 19.Moulding NT, Silagy CA, Weller DP. A framework for effective management of change in clinical practice: dissemination and implementation of clinical practice guidelines. Qual Saf Heal Care. 1999;8(3):177–183. doi: 10.1136/qshc.8.3.177. doi:10.1136/qshc.8.3.177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Smith RJ, Kilaru AS, Perrone J, et al. How, Why, and for Whom Do Emergency Medicine Providers Use Prescription Drug Monitoring Programs? Pain Med. 2015;16(6):1122–1131. doi: 10.1111/pme.12700. doi:10.1111/pme.12700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Hinyard LJ, Kreuter MW. Using Narrative Communciation as a Tool for Health Behaviour Change: A Conceptual, Theoretical, and Empirical Overview. Heal Educ Behav. 2007 doi: 10.1177/1090198106291963. [DOI] [PubMed] [Google Scholar]
  • 22.Montaño D, Kasprzyk D. Theory of Reasoned Action, Theory of Planned Behaviour, and the Integrated Behavioral Model. 2008 doi:10.1016/S0033-3506(49)81524-1. [Google Scholar]
  • 23.Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215. doi: 10.1037//0033-295x.84.2.191. doi:10.1037/0033-295X.84.2.191. [DOI] [PubMed] [Google Scholar]
  • 24.Fox TR, Li J, Stevens S, Tippie T. A performance improvement prescribing guideline reduces opioid prescriptions for emergency department dental pain patients. Ann Emerg Med. 2013;62(3):237–240. doi: 10.1016/j.annemergmed.2012.11.020. doi:10.1016/j.annemergmed.2012.11.020. [DOI] [PubMed] [Google Scholar]
  • 25.Bowen S, Zwi AB. Pathways to “evidence-informed” policy and practice: A framework for action. PLoS Med. 2005;2(7):0600–0605. doi: 10.1371/journal.pmed.0020166. doi:10.1371/journal.pmed.0020166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Newman TB. The Power of Stories Over Statistics: Lessons from Neonatal Jaundic and Airplane Safety. In: Hurwitz B, Greenhalgh T, Skultans V, editors. Narrative Research in Health and Illness. 1st ed. Blackwell; 2004. pp. 257–275. [Google Scholar]
  • 27.Kilaru AS, Perrone J, Auriemma CL, Shofer FS, Barg FK, Meisel ZF. Evidence- based narratives to improve recall of opioid prescribing guidelines: A randomized experiment. In: Academic Emergency Medicine. 2014;21:244–249. doi: 10.1111/acem.12326. doi:10.1111/acem.12326. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Hinyard LJ, Kreuter MW. Using narrative communication as a tool for health behavior change: a conceptual, theoretical, and empirical overview. Health Educ Behav. 2007;34:777–792. doi: 10.1177/1090198106291963. doi:10.1177/1090198106291963. [DOI] [PubMed] [Google Scholar]
  • 29.Smith RJ, Rhodes K, Paciotti B, Kelly S, Perrone J, Meisel ZF. Patient Perspectives of Acute Pain Management in the Era of the Opioid Epidemic. Ann Emerg Med. 2015:1–8. doi: 10.1016/j.annemergmed.2015.03.025. doi:10.1016/j.annemergmed.2015.03.025. [DOI] [PubMed] [Google Scholar]
  • 30.Green MC, Brock TC. The role of transportation in the persuasiveness of public narratives. J Pers Soc Psychol. 2000;79(5):701–721. doi: 10.1037//0022-3514.79.5.701. doi:10.1037/0022-351479.5.701. [DOI] [PubMed] [Google Scholar]
  • 31.Crutzen R, Roosjen JL, Poelman J. Using google analytics as a process evaluation method for internet-delivered interventions: An example on sexual health. Health Promot Int. 2013;28:36–42. doi: 10.1093/heapro/das008. doi:10.1093/heapro/das008. [DOI] [PubMed] [Google Scholar]
  • 32.Guckin CM, Crowley N, Couns D. Using Google Analytics to Evaluate the Impact of the CyberTraining Project. Cyberpsychology, Behav Soc Netw. 2012:121017061642006. doi: 10.1089/cyber.2011.0460. doi:10.1089/cyber.2011.0460. [DOI] [PubMed] [Google Scholar]
  • 33.Dzewaltowski D a, Glasgow RE, Klesges LM, Estabrooks P a, Brock E. RE-AIM: evidence-based standards and a Web resource to improve translation of research into practice. Ann Behav Med. 2004;28(2):75–80. doi: 10.1207/s15324796abm2802_1. doi:10.1207/s15324796abm2802_1. [DOI] [PubMed] [Google Scholar]
  • 34.Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What Does It Mean to Employ” the RE-AIM Model? Eval Health Prof. 2012;00(0):1–23. doi: 10.1177/0163278712446066. doi:10.1177/0163278712446066. [DOI] [PubMed] [Google Scholar]
  • 35.Glasgow R, Vogt T, Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327. doi: 10.2105/ajph.89.9.1322. doi:10.2105/AJPH.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Haffajee RL, Jena AB, Weiner SG. Mandatory use of prescription drug monitoring programs. JAMA. 2015;313(9):891–892. doi: 10.1001/jama.2014.18514. doi:10.1001/jama.2014.18514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Francke AL, Smit MC, de Veer AJE, Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38. doi: 10.1186/1472-6947-8-38. doi:10.1186/1472-6947-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Kreuter MW, Bernhardt JM. Reframing the dissemination challenge: A marketing and distribution perspective. Am J Public Health. 2009;99(12):2123–2127. doi: 10.2105/AJPH.2008.155218. doi:10.2105/AJPH.2008.155218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. [December 22, 2015];Email Marketing Statistics. 2015 http://mailchimp.com/resources/research/email-marketing-benchmarks/.
  • 40.Kreuter MW, Green MC, Cappella JN, et al. Narrative communication in cancer prevention and control: a framework to guide research and application. Ann Behav Med. 2007;33:221–235. doi: 10.1007/BF02879904. doi:10.1080/08836610701357922. [DOI] [PubMed] [Google Scholar]
  • 41.Fishbein M, Yzer MC. Using Theory to Design Effective Health Behavior Interventions. Commun Theory. 2003;13(2):164–183. doi:10.1111/j.1468-28852003.tb00287.x. [Google Scholar]
  • 42.Schultz PW, Nolan JM, Cialdini RB, Goldstein NJ, Griskevicius V. The constructive, destructive, and reconstructive power of social norms. Psychol Sci. 2007;18(5):429. doi: 10.1111/j.1467-9280.2007.01917.x. doi:10.1111/j.1467-9280.2007.01917.x. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS767180-supplement-1.docx (515.9KB, docx)
2
NIHMS767180-supplement-2.docx (3.1MB, docx)

RESOURCES