A mixed-methods evaluation of clinician education modules on reducing surgical opioid prescribing

Abstract

Background

We developed online interactive clinician education modules highlighting best practices to minimize opioid prescribing at discharge following surgery. The modules were implemented as part of a multi-component quality improvement initiative across a six-hospital health system. This paper describes the development and evaluation of this educational intervention.

Materials and Methods

Clinician education modules targeting surgical prescribers, nurses, and pharmacists were developed and implemented by an interdisciplinary team. Clinicians were invited to participate in an evaluation survey after completing the modules. Survey items assessed clinicians’ rating of the module and intention to change clinical practice due to the module. Quantitative and qualitative survey responses were analyzed by the study team.

Results

2,119 clinicians completed the module and 1,831 of these clinicians (86.4%) completed the survey. Of clinicians completing the survey, 65.6% reported that they intend to change clinical practice after completing the module. Intended changes were related to increased knowledge and awareness, provider empowerment, opioid prescribing practices, non-opioid prescribing practices, and patient education. Many clinicians who indicated they do not intend to change practice reported that their clinical practices were already in line with module recommendations. Some clinicians did not perceive the module to be relevant to their role.

Conclusions

Module completion was associated with intention to improve clinical practice in areas related to provider empowerment, opioid prescribing, non-opioid prescribing, and patient education. Evaluation data will inform future module improvements. There is an opportunity to ensure that all clinicians, including those who are not prescribers, recognize their role in opioid stewardship.

Introduction

The opioid crisis is a complex problem that necessitates comprehensive solutions implemented across social institutions.^1-4 Health systems are in an optimal position to implement solutions that (1) reduce the overprescribing of opioids to decrease unused opioids available for diversion and misuse and (2) improve the pain management education provided to patients and families. Surgical providers contribute to the opioid crisis by overprescribing opioids for postoperative pain management at discharge. In a 12-month period from 2016 to 2017, opioids prescribed by surgical providers accounted for 10.4 million filled prescriptions.⁵ On average, 67 to 92% of patients report unused opioids from these prescriptions,^6-8 putting the excess pills at high risk for misuse including diversion to individuals other than the intended patient.^{6, 9-12}

Numerous organizations, including the Joint Commission, Institute of Medicine, American Hospital Association, and National Academy of Medicine, have called for increased clinician education to change opioid prescribing habits and to promote a shift in the culture around pain management.^{2, 3, 13-15} We set out to develop an educational intervention to promote a culture shift in postoperative pain management within our health system.

As a first step, we conducted interviews with surgeons, nurses, pharmacists, and administrators to identify gaps in knowledge and skills related to opioid stewardship. We learned that clinicians typically had not received formal education on opioid stewardship or opioid prescribing recommendations during their training.¹⁶ Our interviews suggested that many surgeons likely prescribed specific quantities of opioids out of habit, lacking basis in the evidence. Furthermore, our interviews revealed opportunities for enhanced clinician education on topics such as how to set patient expectations and current legislation related to disposal of prescription opioids.

Drawing from existing evidence on best practices and qualitative work within our health system, we developed a multi-component intervention “Minimizing Opioid Prescribing in Surgery” with five components: (1) clinician education, (2) patient education, (3) electronic health record (EHR) optimizations, (4) automated Illinois Prescription Monitoring Program (IL-PMP) look-up, and (5) individualized dashboards with prescribing data for surgical prescribers.¹⁷

The aim of this paper is to describe the development and evaluate the outcomes of the clinician education component of our intervention to reduce opioid overprescribing in surgery. By including a qualitative thematic analysis of module evaluation data, our study also presents a methodology for optimal evaluation of clinician-focused opioid education interventions. This manuscript is primarily structured according to the Standards for QUality Improvement Reporting Excellence (SQUIRE) 2.0 guidelines.¹⁸

Materials and Methods

Context

This study was conducted from October 2018 to January 2020 across an Illinois-based health system comprising six hospitals (one academic, two large community, and three small community hospitals, two of which are located in rural areas). The clinician education intervention was implemented as part of a system-wide opioid prescription reduction initiative.¹⁷ The target study population included all clinicians involved in pain management prescribing and/or education for general surgery patients: all general surgeons; general surgery residents; hospitalists managing general surgery patients; advanced practice providers affiliated with general surgery; pharmacists; and nurses in surgical clinics, preoperative clinics, perioperative services, and inpatient surgical units across the health system.

Intervention Development

To provide education to the target audience system-wide, we developed interactive online education modules to be distributed through the health system’s learning management platform. Our core implementation team—responsible for development, implementation, and evaluation—consisted of an executive sponsor (K.Y.B.), a surgeon champion (J.J.S.), a performance improvement leader (B.A.B.), health services researchers (J.K.J., W.L.A.S., R.H.), and a project manager (M.L.S.). The education intervention, as one component of the larger system-wide quality improvement charter to minimize opioid prescribing at hospital discharge after surgery, had significant support and buy-in from hospital leadership. Baseline system-wide prescribing data had demonstrated that only 38.0% of discharge opioid prescriptions in general surgery were aligned with procedure-specific prescription quantity guidelines, so hospital leaders were supportive of efforts to improve guideline-based opioid prescribing. The implementation team assembled a team of subject matter experts in areas such as general surgery, outpatient and inpatient nursing, pain management, pain psychology, pharmacy, and instructional design to provide feedback throughout the content development process.

Based on gaps identified through our provider interviews¹⁶ and existing literature, we developed learning objectives for each of three provider audiences: (1) surgeons and advanced practice providers, (2) nurses, and (3) pharmacists (Appendix A). The learning objectives of the module were to educate the learner to: (1) recognize their role, as a clinician, in addressing the opioid crisis; (2) effectively set pain management expectations with patients; (3) reduce opioid prescribing while leveraging multimodal, non-opioid strategies; and (4) provide opioid safety education to patients and families. Draft learning objectives and content were shared with the subject matter experts and relevant clinician education committees, and revisions were made to reflect stakeholder feedback.

To optimize learner engagement, the education modules leveraged scenario-based learning (e.g., presentations of sample patient cases with questions that required the learner to apply module concepts to the case) and interactive formative assessments (e.g., quizzes on content, with feedback, during the course of the module to reinforce learning). The implementation team worked closely with the web design vendor and instructional design experts to ensure that the module design was conducive to effective delivery of the content. As a last step in the development phase, we conducted usability testing with clinicians who represented the target audiences before finalizing the modules.

Intervention Implementation

The modules were integrated into our health system’s learning management system and distributed to clinician learners via an email alerting them of a new “learning plan assignment”. These emails were sent by hospital Chief Medical Officers, surgical chairs, and Chief Nursing Executives, indicating endorsement of the activity by leadership and alignment with the system’s priorities. Nursing leadership opted to require completion of the module for nurses, while there was no requirement for the other target audiences. Clinicians who were outside the target audience and not assigned the modules through the learning management system could still access and complete the modules. The modules are freely available at https://www.isqic.org/opioid-reduction-initiatives.

Evaluation of the Intervention

To evaluate the impact of the module on learners’ intention to change practice, we included a link to a module evaluation survey housed in a secure web-based survey platform, Research Electronic Data Capture (REDCap),¹⁹ to be completed anonymously at the end of the module (Appendix B).

Measures

The evaluation survey included demographic items related to the respondent’s primary hospital and clinical role. Other items assessed the learner’s reason for completing the module and overall rating of the module (using a Likert scale). Additionally, the survey asked whether learners intended to do anything differently in their clinical practice as a result of the module. This intention to change clinical practice item was dichotomous (“yes” or “no”) with the option to elaborate in a free-text field. In addition, module completion rates were captured through the learning management system.

Analysis

First, we performed descriptive statistics for module completion, demographic items, reason for completing the module, intention to change practice, and overall module rating. Second, for a subsample of open-ended responses elaborating on whether or not respondents intended to change their clinical practice, we conducted a thematic qualitative analysis using an inductive, constant comparative approach. Two members of the study team independently identified themes in a subsample of responses. The team then conducted an affinity diagramming exercise to organize the identified themes and develop the codebook (Appendix C). Free-text responses provided between October 2018 and March 2019 were then coded by two members of the study team in ATLAS.ti. The coders met regularly to reconcile any differences.

Ethical Considerations

All research procedures for the Minimizing Opioid Prescribing in Surgery project were approved by the Northwestern University Institutional Review Board (STU00205053). As the implementation and evaluation of the clinician education modules were conducted for quality improvement, these activities were exempt from Northwestern University Institutional Review Board oversight.

Results

Participants

Between October 2018 and January 2020, 2,119 clinicians completed the module, including 1,668 nurses, 299 pharmacists, and 152 general surgery prescribers (surgeons, advanced practice providers, general surgery residents, and hospitalists). In total, 1,831 clinicians completed the module evaluation survey (86.4% response rate). As shown in Table 1, respondents were primarily from six different hospitals and two medical groups within the health system. The greatest portion of respondents were employed by the academic medical center (n=628, 34.3%). Nurses were the largest group among survey respondents (n=1,399, 76.4%), followed by pharmacists (n=263, 14.4%), advanced practice providers (n=92, 5.0%), and surgeons/general surgery residents (n=36, 2.0%). Hospitalists and other clinicians who completed the survey were included within the “other” clinician type category (n=41, 2.3%). Survey completion rate was the highest for pharmacists (88.0%) and slightly lower for advanced practice providers and surgeons (84.2%) and nurses (83.9%).

Table 1.

Module evaluation survey completion frequencies

	n	%		n	%
Total completed surveys	1831
Primary hospital/clinical area			Why did you complete this module? (Check all that apply)
Academic medical center	628	34.3	Wanted to learn more about this topic to improve patient care	463	25.3
Large community hospital #1	401	21.9	Asked by my department chair or supervisor to complete this module	317	17.3
Large community hospital #2	221	12.1	Required to complete this module	1566	85.5
Large community hospital #3	227	12.4	Recommended to me by one of my colleagues	66	3.6
Small community hospital #1	157	8.6	Other	5	0.3
Small community hospital #2	35	1.9
System medical group	28	1.5	Do you intend to do anything differently in your clinical practice as a result of this module?
Regional medical group	18	1.0	Yes	1202	65.6
Other	76	4.2	No	576	31.5
Missing	40	2.2	Missing	53	2.9
Role			How would you rate this module overall?
Surgeon	36	2.0	Excellent	507	27.7
Advanced Practice Provider	92	5.0	Good	932	50.9
Nurse	1399	76.4	Okay	316	17.3
Pharmacist	263	14.4	Poor	10	0.5
Other/missing	41	2.3	Missing	66	3.6

Reasons for completing the module

1,566 respondents (85.5%) reported completing the educational module because they were required, 463 respondents (25.3%) reported completing the module to learn more about the topic to improve patient care, 317 respondents (17.3%) reported that they had been asked by their department chair or supervisor to complete the module, and 66 respondents (3.6%) reported that the module was recommended to them by a colleague (Table 1). The third category may include clinicians who were not assigned the modules through the learning management system but heard about them from a supervisor. These responses were not mutually exclusive.

Intention to change clinical practice

1,202 respondents (65.6%) reported that they intend to make clinical practice changes as a result of the module. Qualitative analysis of free-text explanations for respondents’ choices demonstrated key themes related to how participants intend to change practice or why they do not intend to make changes (Table 2).

Table 2.

Qualitative themes and quotes

Code	Subcode	Illustrative quotes
Intend to change practice	Increased knowledge/awareness	"with this knowledge, I am able to provide more effective communication with my patients about the risks and benefits of opioid usage" – Nurse "Awareness increase If the pt hasn't used opiates during hospital stay they should not need a prescription on discharge" – Nurse
	Provider empowerment	"Advise physicians more confidently if I see an discrepancy with the amount of opioids prescribed as compared to the procedure" – Pharmacist "Extra communication with prescribers to ensure best pain management options are utilized and alternate options (aside from opioid use) for pain management are discussed" - Nurse
	Opioid prescribing practices	"More prepared to discuss how/where to properly dispose of opioid medications. Emphasis on PMH/ diagnoses to watch for when prescribing opioids." – APP "Discharge my postop patients with the recommended number/amount of opioids" - Surgeon
	Non-opioid prescribing practices	"Non narcotic pain control techniques: ice, heat, positioning, nonsteroidals" – Nurse "Use nonopioid treatments first" - Surgeon
	Patient education	"I plan on discussing with my patients proper disposal of unused prescription opioid pills" – Surgeon "educate pt. to set realistic goals when it comes to pain relief post surgery." - Nurse
Do not intend to change practice	Already implementing changes in clinical practice	"Prescribing practices are currently in line with the recommended quantities for surgery. Currently discuss nonopioid therapies with patients, risks of opioids, and postop expectations with my patients." – APP "As a pharmacist, I actively act as an opioid steward. The ideas presented are in line with current practice in appropriate patients." - Pharmacist
	Not a prescriber/not relevant to role	"I cannot prescribe opioids, this is outside of my scope of practice." – Nurse "I do not have a DEA license/cannot prescribe opioids" - APP
	Disagree with incorporating changes into practice	"I work in the ICU and treat people with short term surgical pain and believe that people should not be suffering from intolerable surgical pain just because some people have ruined it for the rest of those who need it after surgery" – Nurse "Pain is subjective. I can not change that behavior over night." - Nurse

Intended changes to practice

Increased knowledge/awareness

Respondents reported increased knowledge and/or awareness due to the module. Respondents gained knowledge of resources (e.g., safe drug disposal locators), opioid use risks, and signs and symptoms of opioid misuse. Clinicians indicated greater awareness of the opioid crisis and of how surgical opioid prescribing is connected with this larger public health issue.

Provider empowerment

Clinicians responded that they intend to advocate on behalf of their patients for nonopioid pain management and for lower opioid prescription quantities. Nurses expressed feeling more confident and prepared to speak up when they see that prescription quantities are above the recommended quantities for a specific procedure.

Opioid prescribing practices

Prescribers indicated they would change their own prescribing practices (e.g., by using the procedure-specific recommendations shared in the module) or would check the Prescription Monitoring Program routinely to review a patient’s history of filling scheduled drug prescriptions. Clinicians also reported that they would review patients’ medical histories, prior diagnoses, and current medications more carefully when prescribing opioids to minimize the likelihood of adverse interactions and risk for misuse.

Non-opioid prescribing practices

Respondents articulated plans to recommend and administer non-opioid pain management therapies more frequently and when appropriate, as the initial pain management strategy rather than starting with opioids. When possible and appropriate, they would use alternatives to opioids such as local anesthetics, relaxation techniques, ice, positioning, and non-steroidal anti-inflammatory medications. Respondents who provide patient education shared the instructions they plan to share with patients regarding post-discharge opioid-sparing pain management regimens.

Patient education

Clinicians indicated that they intend to change the education they provide to patients and families by addressing new topics in their conversations, modifying their scripting or recommendations, or being more thorough about certain points. Respondents reported that they plan to discuss topics such as where to dispose of unused prescription opioids, potential side effects of taking opioids, and realistic pain expectations during recovery more frequently and/or in greater detail with patients.

Do not intend to change practice

Already changed clinical practices

Many clinicians who responded that they do not intend to make practice changes clarified that their practices were already in line with the recommended strategies. Some clinicians indicated that their departments or units are already focused on changing opioid prescribing and patient education, and that they are individually modifying their behaviors accordingly. A few nurses expressed that they are already notifying prescribers when they think opioid prescription quantities are too high or when they identify a patient at high risk for substance misuse.

Not a prescriber/not relevant to role

Some clinicians viewed the educational content as not relevant to their role and therefore thought it would be outside of their scope to make practice changes. This theme primarily emerged in responses from nurses who do not prescribe opioids or who work in intraoperative settings that do not involve patient education.

Disagree with incorporating changes into practice

Very few clinicians indicated that they disagree with incorporating opioid reduction strategies into their practice. Reasons for disagreeing with making changes primarily focused on not wanting patients to suffer from surgical pain just because some people misuse opioid medications.

Discussion

Following completion of an online opioid education module, we found that the majority of learners intend to change clinical practice to support patients’ optimal post-surgical pain management. Intended practice changes are centered on themes including increased knowledge and awareness, provider empowerment, opioid prescribing practices, non-opioid prescribing practices, and patient education. These themes relate closely to the stated learning objectives of the module.

A significant portion of clinicians who completed the module were already implementing opioid reduction strategies in their practice. While some clinicians referred to their individual practice changes, others described opioid reduction initiatives occurring at the hospital or department level with which they were actively engaged. These existing practice changes and initiatives may be indications that a broader cultural shift around prescription opioid prescribing and use has occurred across the United States in recent years,^{20, 21} potentially owed in part to increased media attention on the overdose risks and addictive properties of opioids.^{22, 23} This shift has important implications for all ongoing opioid reduction quality improvement and research projects as it may enhance the effectiveness of opioid reduction strategies due to higher acceptability among clinicians and patients, but may also make targeted evaluation of intervention effectiveness more difficult.

Of those clinicians who do not intend to adopt the practices detailed in the module, primary reasons included feeling the topic is irrelevant to one’s clinical role and disagreeing with the recommended changes. The clinicians who reported they do not intend to change practice tended to be nurses practicing in the operating room (OR) or post-anesthesia care unit (PACU). This finding demonstrated a need to make the link between the opioid crisis and surgical clinical roles in these areas more clear so that learners feel the content is directly relevant to them and actionable. Although some OR and PACU nurses focused on how they do not prescribe medications or provide patient education, and therefore did not see how the content related to their role, many of the OR and PACU nurses completing the module identified ways they could apply the content to their practice (e.g., by advocating for minimal opioid administration during and immediately after surgery). Based on these evaluation findings, the study team plans to update the modules with scenarios targeted more directly toward different surgical clinicians’ roles. Additionally, the evaluation elucidated barriers and contextual factors that may need to be addressed to motivate behavior change in clinicians who reported they do not intend to change. Barriers included clinical cultural factors (e.g., an inpatient nurse not wanting to question an attending surgeon’s prescription order) and deeply-rooted habits (e.g., basing prescription quantities solely on the patient-reported pain score).

Importantly, the clinicians who completed the module practice across the continuum of surgical care and many of them indicated the specific time points at which they would make changes to affect patients’ pain management-related care. For instance, clinicians discussed enhancing patient education and expectation setting at preoperative appointments, improving monitoring of opioid administration and advocating for opioid minimization while patients are in surgery or inpatient postoperatively, and improving patient and family education at discharge. Because our modules targeted clinicians across all phases of surgical care, we anticipate improved consistency of pain management messaging across the surgical care continuum.

Our findings are consistent with previous studies on the impact of clinician education programs on opioid prescribing. In a 2015 study by Alford et al.,²⁴ 87% of clinician participants stated they intended to make practice changes after completing a 3-hour live or online activity focused on safe and effective opioid prescribing practices for patients with chronic pain. A 2019 evaluation of an online opioid education program for acute pain management²⁵ assessed likelihood of changing practice in a different manner. In this study, the authors asked participants six months after completing the module to indicate whether it was more likely they used specified practices, as a result of completing the module, on a scale of 1 (not at all true) to 7 (very true). Average “likelihood to use guideline-adherent practice” score averages ranged from 4.7 to 5.5, indicating that on the whole, it was “somewhat true” that learners were more likely to use the specified practices due to the module. These findings in conjunction with ours demonstrate a growing body of literature, which suggests targeted opioid education for clinicians is an important component of quality improvement programs addressing the opioid crisis.

Our study has several limitations that are worth noting. One limitation is that social desirability bias may have led to an artificially high measurement of intention to change practice. However, in an attempt to mitigate this bias, evaluation survey responses were anonymous. Additionally, because the module was mandatory only for one clinician group (nurses), the module and survey were accessible to clinicians outside the target audience, and the survey was optional, characteristics of clinicians who completed the module and the evaluation may not be representative of the target audience of clinicians initially assigned the module.

Finally, this study was not intended to determine whether the educational intervention alone led to actual clinician behavior change. While this is an important next step, the intention of this work was to evaluate the module to facilitate iterative improvement of the module. In addition, other components of the multi-component quality improvement initiative (e.g., optimizations in the electronic health record and distribution of a patient education brochure) were implemented simultaneously with the module, making it difficult to isolate the effect of the education modules on actual prescribing and patient education-related practice change. Recognizing this limitation, we have assessed intention to change practice immediately following module completion as a proxy indicator of actual behavior change. Evidence suggests that clinicians’ self-reported intention to change behavior may predict actual change.^26-30

Conclusions

A clinician education module about minimizing opioid prescribing in surgery increased most learners’ intention to incorporate best practices into their clinical care. Qualitative analysis of evaluation responses highlighted themes in the types of changes that clinicians intend to make and revealed that many clinicians who indicated they do not intend to change practice due to the module were already implementing the recommended strategies. This analysis highlighted an opportunity to improve the modules by ensuring the examples and recommended change strategies are immediately relevant to the learners’ clinical roles. Additionally, implementation of the module will be expanded into surgical specialties beyond general surgery. Module completion and evaluation data will continue to be tracked and reviewed by the study team to inform future module improvements.