Improving Residents' Safe Opioid Prescribing for Chronic Pain Using an Objective Structured Clinical Examination

Abstract

Background

Internal medicine residents care for a sizable number of patients with chronic pain. Programs need educational strategies to promote safe opioid prescribing.

Objective

To describe a safe opioid prescribing education program utilizing an objective structured clinical examination (OSCE) and report the resulting impact on residents' knowledge, confidence, and self-reported practices.

Methods

Using a quasi-experimental design, 39 internal medicine residents from an urban academic medical center were assigned to 1 of 4 groups: 1-hour lecture only, lecture followed by immediate OSCE, lecture followed by 4-month delayed OSCE, and control. Safe opioid prescribing knowledge, confidence, and self-reported practices were assessed at baseline and at 8 months.

Results

At 8 months, knowledge, confidence, and self-reported practices improved in the control and in all 3 intervention groups. The immediate OSCE group had the greatest improvements in combined confidence scores within group (0.74, P = .01) compared to controls (0.52, P = .05), using a 5-point scale. This group also had the greatest improvement in self-reported practice changes (1.04, P = .04), while other groups showed nonsignificant improvements—delayed OSCE (0.43, P = .44), lecture only (0.66, P = .24), and control (0.43, P = .19).

Conclusions

Safe opioid prescribing education that includes a lecture immediately followed by an OSCE had an impact on residents' confidence and self-reported practices greater than those for delayed OSCE or lecture only groups. There was no difference in knowledge improvement among the groups. Lecture followed by an OSCE was highly regarded by residents, but required additional resources.

Editor's Note: The online version of this article contains information for OSCE implementation, including educational objectives and procedures, station-specific resident instructions and clinical tasks, and faculty, standardized patient, and resident assessment tools, and a table of baseline participant characteristics.

What was known and gap

Internal medicine residents care for patients with chronic pain, yet they report low confidence and self-efficacy in safe opioid prescribing.

What is new

Residents were assigned to a control and 3 intervention groups for opioid prescribing education, with assessment at baseline and 8 months later.

Limitations

Single site, single specialty study; small sample; and lack of randomization may limit generalizability.

Bottom line

A lecture followed by an objective structured clinical examination had the largest impact on resident confidence and self-reported practices, and is well accepted, but requires added resources.

Introduction

Chronic pain is one of the most common reasons patients seek medical care.^1,2 During the past 2 decades, more aggressive chronic pain management with opioid analgesics³ has been associated with an increase in prescription opioid misuse.⁴ There are numerous safe opioid prescribing guidelines;⁵ however, adherence with these guidelines remains low.^6–8

Physicians struggle to balance benefits and harms of prescription opioids,⁹ which is exacerbated by inadequate education.¹⁰ Medical trainees report low levels of confidence managing chronic pain.^11,12 Safe opioid prescribing education can improve residents' attitudes,¹³ knowledge,^14,15 and confidence.¹⁴ Residents prefer skills-based opioid prescribing education.¹⁴

To improve residents' opioid prescribing practices, we developed a skills-based educational program that included a lecture followed by an objective structured clinical examination (OSCE), allowing residents to practice skills in a realistic setting. OSCEs utilize principles of cognitive apprenticeship¹⁶ through coaching, feedback, and allowing learners to reflect on their skills.¹⁷ They have been used to assess trainees' pain and addiction management practices,^18,19 yet we are not aware of prior reports of using OSCEs to teach safe opioid prescribing skills.

Methods

Setting and Participants

The setting was an urban, academic, hospital-based internal medicine residency program with approximately 150 residents. Prescription opioid monitoring tools (eg, agreements, urine drug testing [UDT]) were available to residents but were not widely used.⁸

Intervention

Utilizing a quasi-experimental design, residents were assigned to either a control or 1 of 3 intervention subgroups: 1-hour lecture with immediate OSCE, lecture with 4-month delayed OSCE, or lecture only (figure 1). A United States National Institute on Drug Abuse grant covered the costs associated with program development and implementation.

Educational Program Description

Lecture: The 1-hour lecture developed by experts (D.P.A., A.H.J.) in pain, addiction, and education was guideline-based and covered the assessment of pain and opioid misuse risk, the monitoring of benefits and harms, and the modifying of treatment plans when appropriate.

Objective Structured Clinical Examination: 

Two OSCE sessions were completed postlecture (immediate or a delay of 4 months). The total time for OSCE administration was 4 hours, 30 minutes. Each session had three 20-minute stations (table 1), which included (1) the resident reading the case summary and specific tasks (2 minutes); (2) interviewing the standardized patient (SP; 10 minutes); (3) verbal self-assessment (1 minute); (4) SP feedback (1 minute); (5) faculty feedback (5 minutes); and (6) poststation evaluation (1 minute). The OSCE materials are available as online supplemental material.

Table 1.

Objective Structured Clinical Examination Station Patient Profile and Clinical Tasks

Faculty Observers: 

All 5 faculty members proctoring the OSCE were experienced in primary care, safe opioid prescribing, and medical education. Faculty attended a 1-hour, 30-minute orientation to review station-specific tasks and logistics. During OSCE sessions, faculty observed residents and gave feedback based on an assessment tool that included communication skills and station-specific tasks. Faculty were paid $500 for 6 hours of participation during nonclinical protected time.

Standardized Patients: 

The SPs were actors who were given detailed patient roles and attended a 1-hour, 30-minute orientation with faculty, which included general approaches to the interview and how to give constructive feedback. Three SPs were paid $240 each for 6 hours of participation.

Resident Learners: 

Residents in the OSCE groups attended the 1-hour lecture, a 30-minute OSCE orientation, and all 3 OSCE stations. Three residents completed the 3 OSCE stations per hour. After residents completed the follow-up surveys, they were remunerated with a $50 gift card.

Outcomes

Resident Surveys: Residents completed baseline (prelecture) and 4- and 8-month (postlecture) follow-up surveys (figure 1), which assessed knowledge, confidence, and self-reported practices. The surveys were developed by experts. To allow for maximum time between the intervention and follow-up, we report 8-month outcomes. We used the 4-month survey to identify residents who were not opioid prescribers and focused our 8-month practice change survey on residents who prescribed opioids. Residents in the OSCE groups evaluated the OSCE.

The assessments included:

• Knowledge: 4 multiple-choice questions on opioid efficacy, misuse risk factors, and management of aberrant medication-taking behaviors. Score ranged from 0 to 4.

• Confidence: 8 items utilizing 5-point scales (1, not at all confident, to 5, very confident) in specific safe opioid prescribing practices (table 2). The confidence score was calculated as the average of the 8 items.

• Practice: 3 items utilizing 5-point scales (1, never/rarely, to 5, always) in frequency of self-reported practices, calculated as an average (table 3).

Table 2.

Change in Safe Opioid Prescribing Confidence

Table 3.

Change in Safe Opioid Prescribing Self-Reported Practices

The Boston University Medical Campus Institutional Review Board determined this evaluation to be exempt from further review. Residents signed a consent outlining the voluntary nature of this study.

Analysis

The internal consistency of the 8-item summary confidence and 3-item summary practice scales at baseline was described using Cronbach's α. Exploratory analyses with the 4 groups examined changes from baseline to 8-month follow-up on the knowledge score, individual confidence and practice items, and summary confidence and summary practice scales using a paired sample t test. Changes with 2-tailed P < .05 are reported as significant. Given our sample size, these within-group comparisons are not adjusted for multiple comparisons and should be interpreted with caution.

Primary analyses examined differences across the 4 groups on mean changes in confidence and practice scales from baseline to 8-month follow-up; they were made through 1-factor analysis of variance (ANOVA), followed by Tukey's pairwise comparison, which accounts for multiple comparisons to identify specific differences if the overall ANOVA found significant differences. We reported effect sizes using Cohen's d to describe the magnitude of observed differences, calculated as the difference in the mean change score for an intervention group versus the control group, divided by the pooled standard deviation of the change score from the 4 groups. Given the small sample, we focused on effect sizes and statistical significance in reporting differences among study groups.

Results

This study included 39 internal medicine residents assigned to 4 groups: lecture with immediate OSCE (L + iO, n = 9), lecture with 4-month delayed OSCE (L + dO, n = 8), lecture only (L, n = 12), or control (C, n = 10). All residents completed all surveys and were similar at baseline (provided as online supplemental material) for mean number of patients managed with chronic pain (9.1, SD = 7.00) and on opioid analgesics (3.9, SD = 3.53). They had similar prior opioid prescribing training (1, none; 2, some; and 3, a lot; mean = 1.68; SD = 0.47) and opioid prescribing confidence (1, not at all, to 5, very; mean = 2.5; SD = 0.72).

Knowledge

All 4 groups showed an increase in knowledge at 8 months (mean [SD] score increased from 2.8 [1.1] to 3.2 [0.7], P < .01), with no significant difference across the 4 groups (P = .74).

Confidence

The 8 confidence items had strong reliability (Cronbach's α = 0.86). Within-group analysis showed significant improvement at 8 months in the summary confidence score for the L + iO group only. The L + iO group had significant increases on 4 of 8 individual confidence items (interpreting UDT, discussing unexpected UDT, discussing aberrant medication taking behaviors, dealing with patients' possible anger; table 2). The L + dO and L groups each showed a significant increase only on a single item (interpreting UDT and discussing opioid risks/benefits, respectively), while there were no significant increases on any items for the control group. ANOVA found no significant difference in change in confidence across the 4 study groups (P = .36). Based on Cohen's d, there were moderate increases in confidence for L + iO and L + dO groups relative to controls and a small increase in confidence for the L group relative to control.

Although there were some significant improvements in confidence within intervention groups, no significant changes were observed between groups. However, further analysis (Cohen's d) suggests meaningful improvements for both OSCE groups compared to the control.

Self-Reported Practices

The 3 practice items had strong reliability (Cronbach's α = 0.80). We excluded 5 participants from this practice outcome (1 each from L + iO, L, and C groups and 2 from the L + dO group) as they indicated no opioid prescribing in the months prior to both 4- and 8-month assessments and were, therefore, unable to make opioid prescribing changes. In the remaining sample (n = 34), only the L + iO group showed significant improvement on the summary practice score at 8 months (figure 2), with a significant improvement on 1 item (conducting pill counts; table 3). ANOVA found no significant difference in improvement across the 4 groups (P = .54). Based on Cohen's d, there was a moderate increase relative to controls on the summary practice score for the L + iO group.

While there was significant improvement in self-reported practice within the immediate OSCE group, there were no significant changes in self-reported practice between groups. However, further analysis (Cohen's d) suggests meaningful improvement for the immediate OSCE group compared to the control.

OSCE Evaluation

All participants reported the OSCE helped identify strengths and weaknesses, and 94% (16 of 17) reported the OSCE taught them something new and stimulated further learning.

Figure 1.

Objective Structured Clinical Examination (OSCE) Study Design

Abbreviations: L + iO, 60-minute lecture followed by immediate OSCE; L + dO, 60-minute lecture followed by delayed OSCE 4 months later; L, 60-minute lecture only; C, control, no lecture, no OSCE.

Figure 2.

Combined Within-Group Change From Baseline to 8 Months in 3 Safe Opioid Prescribing Self-Reported Practices (n = 34)

Abbreviations: L + iO, 60-minute lecture followed by immediate OSCE; L + dO, 60-minute lecture followed by delayed OSCE 4 months later; L, 60-minute lecture only; C, control, no lecture, no OSCE; OSCE, objective, structured clinical examination.

^aP < .05.

Discussion

A safe opioid prescribing educational program that includes a lecture followed by an OSCE may improve internal medicine residents' safe opioid prescribing confidence and self-reported practices. Having the OSCE immediately follow the lecture appeared to have the greatest impact.

This improvement can be explained by learners having an opportunity to practice challenging skills and receive feedback allowing for rapid integration, refinement, and solidification of new skills. Previous resident safe opioid prescribing education using role playing has demonstrated the importance of skills practice.¹⁴ Trainees respond favorably to using SPs,^20–22 which are more realistic but are a logistically challenging skills practice method.^20,23 Our program findings are similar to a program that used OSCEs to teach addiction management skills.^19,24

While this grant-funded program was highly regarded by both residents and residency program leadership, it was not sustained after grant support ended. Understanding the specific and unique challenges internal medicine residents face when implementing new safe opioid prescribing practices helps put our results in context. Opioid prescribing guidelines, based on expert opinion^5,25–28 rather than scientific evidence, result in wide variation in clinical practice and inconsistent faculty preceptors' guidance to residents. Moreover, faculty preceptors may disregard residents' new opioid prescribing practices if they differ from their own. Finally, residents often inherit patients already on opioids and may feel pressured to continue the treatment plan.²⁹ Despite these challenges, our educational program improved residents' self-reported safe opioid prescribing practices.

Our study has several limitations. The nonrandom distribution of residents across groups could lead to confounding, since the L + iO and L + dO groups were more likely to favor primary care careers. The small sample makes it difficult to identify significant differences among groups. Self-reported data may have introduced a social desirability bias, and the follow-up times after OSCE completion varied between the 2 OSCE groups. It is unclear whether more or less time before follow-up would influence the likelihood of change. Our study did not assess improvements in patient-level outcomes. Although the survey instruments were developed by content experts, they did not undergo validity testing.

Future research might include a more robust practice change and patient-level outcomes, including chart reviews and/or patient interviews. Due to the importance of faculty preceptors' role on resident practices, future safe opioid prescribing interventions should include faculty development and/or faculty and resident co-training.

Conclusion

A safe opioid prescribing educational intervention that included a lecture followed by immediate skills practice using an OSCE appeared to improve residents' confidence and self-reported practices and was highly regarded by resident learners.