SCOPE of Pain: An Evaluation of an Opioid Risk Evaluation and Mitigation Strategy Continuing Education Program

Abstract

Objective. Due to the high prevalence of prescription opioid misuse, the US Food and Drug Administration (FDA) mandated a Risk Evaluation and Mitigation Strategy (REMS) requiring manufacturers of extended-release/long-acting (ER/LA) opioid analgesics to fund continuing education based on a FDA Blueprint. This article describes the Safe and Competent Opioid Prescribing Education (SCOPE of Pain) program, an ER/LA opioid analgesic REMS program, and its impact on clinician knowledge, confidence, attitudes, and self-reported clinical practice.

Method. Participants of the 3-h SCOPE of Pain training completed pre-, immediate post- and 2-month post-assessments.

Subjects. The primary target group (n = 2,850), and a subset (n = 476) who completed a 2-month post-assessment, consisted of clinicians licensed to prescribe ER/LA opioid analgesics, who care for patients with chronic pain and who completed the 3-h training between February 28, 2013 and June 13, 2014.

Results. Immediately post-program, there was a significant increase in correct responses to knowledge questions (60% to 84%, P ≤ 0.02) and 87% of participants planned to make practice changes. At 2-months post-program, there continued to be a significant increase in correct responses to knowledge questions (60% to 69%, P ≤ 0.03) and 67% reported increased confidence in applying safe opioid prescribing care and 86% reported implementing practice changes. There was also an improvement in alignment of desired attitudes toward safe opioid prescribing.

Conclusions. The SCOPE of Pain program improved knowledge, attitudes, confidence, and self-reported clinical practice in safe opioid prescribing. This national REMS program holds potential to improve the safe use of opioids for the treatment of chronic pain.

Introduction

Chronic pain affects approximately 100 million in the United States, making it one of the most common reasons patients seek medical care [1,2]. Undertreated chronic pain causes reduced function and quality of life [3], and is associated with increased rates of suicidality [4,5]. However, more aggressive chronic pain management with opioid analgesics over the past two decades has been associated with an increase in prescription opioid misuse including addiction, diversion, and overdose deaths [6–11]. Determinants for increased opioid-related mortality have been described including high-volume and high-dose prescribing [12]. Despite concerns over misuse, opioid analgesics remain an important treatment for some patients’ chronic severe pain [1,13–15]. According to the Institute of Medicine report, “regulatory, legal, educational, and cultural barriers inhibit the medically appropriate use of opioid analgesics [1].” Numerous safe opioid prescribing guidelines have been published [16–21], however, recent reports show that adherence with these guidelines is low [22–24].

Clinicians struggle to balance the benefits and harms associated with opioid prescribing [4,25]. While pain management education remains inadequate [26–30], it is a key strategy to address the prescription opioid misuse problem [31]. In July 2012, the US Food and Drug Administration (FDA) approved a single shared Risk Evaluation and Mitigation Strategy (REMS) required of manufacturers of extended-release/long-acting (ER/LA) opioid analgesics to promote safe use of these medications [32]. While most FDA-mandated REMS programs include medication guides and communication plans and are associated with a single medication, this REMS requires all manufacturers to jointly fund accredited continuing education for the approximately 320,000 ER/LA opioid prescribers in the United States [33]. The FDA created the Blueprint for Prescriber Education for Extended-Release and Long-Acting Opioid Analgesics (“FDA Blueprint”) to define the content that must be included in REMS educational programs [34,35]. Boston University School of Medicine (BUSM), the first Continuing Medical Education provider to receive ER/LA opioid REMS funding, launched its Safe and Competent Opioid Prescribing Education (SCOPE of Pain) program on February 28, 2013.

As a new national strategy, the effectiveness of requiring manufacturers to contribute funds to support independent education based on an FDA Blueprint is unknown. The purpose of this study is to describe the SCOPE of Pain program and report on its impact on participants’ knowledge, attitudes, confidence, and self-reported practice. As the first report on an ER/LA opioid REMS program, the data from this project can offer an initial assessment of effectiveness of this national strategy to improve practices.

Methods

SCOPE of Pain Description

SCOPE of Pain is based on the FDA Blueprint [36] and is offered as a 3-h live or online activity available at www.scopeofpain.org. The live programs included 20 half-day standalone meetings across the United States in 16 different states. The live and online curricula are identical and presented using a clinical case involving three separate visits: initial visit—assessing chronic pain and opioid misuse risk; one week later—initiating (continuing) opioid therapy safely and months later—assessing and managing aberrant medication taking behaviors. This allows participants to apply the ER/LA opioid REMS content to a common clinical scenario. SCOPE of Pain was created based on an existing online and live education program we developed in 2010 called “Safe and Effective Opioid Prescribing for Chronic Pain” (www.opioidprescribing.org) that had trained approximately 19,000 clinicians. A team of 13 faculty with expertise in pain management, addiction, primary care, and medical education created the original Opioid Prescribing program and a team of five experts tailored that content to cover all aspects of the FDA Blueprint to make the program REMS compliant. While the original content was well aligned with the FDA Blueprint, specific topics were expanded including opioid prescribing using a risk/benefit framework, effective communication skills for assessing and managing aberrant medication taking behaviors and strategies for team-based care. While the content was not formally tested, evaluation data from the over 5,000 participants of the original Opioid Prescribing program were used to inform the creation of the SCOPE of Pain program.

To ensure that the curriculum covered all FDA Blueprint elements, BUSM conducted both internal and external audit processes and an additional independent audit was conducted by the Accreditation Council for Continuing Medical Education (ACCME). The Boston University Medical Campus Institutional Review Board (IRB) determined this evaluation to be exempt from further IRB review.

Outcomes

A repeated measures design was used to assess the impact of SCOPE of Pain in changing clinicians’ knowledge, attitudes, confidence, and clinical practice. Data were collected from participants at three time points: 1) pre-program (PRE), 2) immediate post-program (IMMED), and 3) 2-months post-program (2MO) (Figure 1). This design assessed changes over time with specific attention to increased alignment with practices described in the FDA Blueprint.

Figure 1.

Evaluation of SCOPE of Pain: Data collection points and associated outcome metrics.

Items to assess participants’ changes were designed by a multidisciplinary team including: a faculty expert in opioid prescribing, primary care and addiction medicine (DPA), experts in educational design (LZ, JLW, IH) and experts in outcomes assessments (SMH, SP, PN). Items were developed with the four key metrics of change that SCOPE of Pain targets: 1) twenty (20) items to assess improvements in knowledge (of which only 10 were repeated at 2MO to minimize respondents’ burden and allow for additional questions about changes in performance), 2) six (6) items regarding change in participant confidence to manage patients with chronic pain, 3) thirteen (13) items assessing change in attitudes (motivation and willingness) when treating patients with chronic pain and using guideline-based care; and 4) multiple items addressing changes in clinical practice including: a) two (2) items assessing intention to change clinical practice; b) seven (7) items assessing participants’ reported changes in clinical performance; c) one (1) item assessing number of changes implemented; and d) one (1) item assessing barriers to implementing change in practice.

To be REMS compliant, the assessment was required to have knowledge-based questions from each of the six sections of the FDA Blueprint [36]. The course director (DPA) who specializes in primary care, pain management and addiction medicine and program education experts (LZ, IH, JLW) determined which elements from each section were best suited for knowledge-based questions and most relevant to practicing clinicians. Confidence and performance questions were based on guideline-based [17–21] safe opioid prescribing practices (e.g., risk and benefit assessments, monitoring and management strategies) and important communication skills. Each item was tested and retested for face validity, and linked explicitly to elements within the six sections of the FDA Blueprint for content validity. All questions were tested by primary care clinicians from general internal medicine and family medicine and pain and addiction medicine experts. The questionnaires used did not undergo validity testing as the evaluation was designed for a new educational program without a known gold standard or preexisting criterion by which to validate.

The PRE/IMMED/2MO items are quantitative using forced choice (drop-down) options. Knowledge-testing questions were a combination of multiple nominal choice responses (including dichotomous true/false questions and item-matching questions). Likert-type response formats were used for self-reported assessment of confidence, attitudes, and clinical practice.

Participant Recruitment

The primary target group included clinicians who manage patients with chronic pain longitudinally. This included primary care and other specialties that manage chronic pain such as hematology, oncology, rheumatology, rehabilitation medicine, sports medicine, neurology, orthopedics, and anesthesiology. While promotion for the program and collection of pre-assessment (PRE) and post-assessment (IMMED and 2MO) data extended beyond the primary target group, only participants whose specialty indicated a likelihood for managing chronic pain were included in this study.

All participants completed the pre-assessment on registration. Participants were required to complete the immediate post-assessment to receive continuing education credit. A drawing for an e-book reader was used to incentivize completion of the 2-month post-assessment. As an email address was collected for all participants, an email was automatically sent to all participants at 60 days, with a reminder at 63 days, and 66 days post-activity for those who did not complete the assessment.

Analyses

Using IBM SPSS 22.0 software (IBM Corporation, Armonk, NY), frequencies and cross-tabulations were calculated for each item. Paired t-tests were used to identify participant knowledge change (PRE vs IMMED) and knowledge maintenance (PRE vs 2MO). Paired t-tests were also used to compare participants’ attitudes and clinical practice (PRE vs 2MO) to establish change in clinical practice two months after participation.

Results

Participants

A total of 10,566 participants completed SCOPE of Pain between February 28, 2013 and June 13, 2014. Twenty-seven percent (2,850/10,566) were considered our primary target group (defined as being physicians, advanced practice nurses, or physician assistants licensed to prescribe opioid analgesics and a member of 13 specialties that routine manage patients with chronic pain (Table 1). The primary target group was made up of mostly physicians (69%), primary care specialties (75%), and clinicians practicing for greater than 10 years (60%). A majority of participants (77%) completed the training online rather than live. All 2,850 participants completed the PRE and IMMED assessments. Of those, 17% (476/2,850) completed the 2MO assessment. Table 1 presents the socio-demographics for the primary target group who completed SCOPE of Pain compared with the subset who also completed the 2MO assessment. The two groups were similar, except for a higher proportion of advanced practice nurses completing the 2MO assessment (P < 0.001).

Table 1.

SCOPE of Pain participant characteristics

	Primary Target Group (n = 2,850)	Completed 2-Month Post-Program Assessment (n = 476)
Profession n, (%)
Physician	1,955 (69%)	288 (61%)
Advance practice nurse*	706 (25%)	154 (32%)
Physician assistant	189 (6%)	34 (7%)
Specialty n, (%)
Family practice	1,179 (41%)	235 (49%)
Internal medicine	791 (28%)	117 (25%)
Anesthesiology	183 (6%)	26 (6%)
Pediatrics	159 (6%)	19 (4%)
Orthopedic surgery	105 (4%)	14 (3%)
Physical medicine and rehabilitation	115 (4%)	17 (4%)
Hematology and oncology	85 (3%)	12 (2%)
Obstetrics and gynecology	83 (3%)	12 (2%)
Neurology	63 (2%)	11 (2%)
Rheumatology	52 (2%)	5 (1%)
Infectious disease	25 (1%)	6 (1%)
Sports medicine	7 (0.2%)	1 (0.2%)
Adolescent medicine	3 (0.1%)	1 (0.2%)
Years of practice n, (%)
1–5 years	659 (23%)	118 (25%)
6–10 years	405 (14%)	74 (16%)
11–20 years	783 (27%)	116 (24)
>21 years	950 (33%)	160 (34)
Other	21 (2%)	8 (1%)
Participant type n, (%)
Online	2,203 (77%)	315 (66%)
Live	647 (23%)	161 (34%)

^*Significant difference between the group that completed the SCOPE of Pain program and those that completed the 2-MO postassessment at the P = 0.05 level.

The following section focuses on the findings divided into two sections 1) IMMED and 2) 2MO assessment. Findings are grouped by the type of expected impact of SCOPE of Pain on participants (knowledge, confidence, attitudes, and clinical practice).

IMMED: Immediate Post-Program Assessment (N = 2,850)

Knowledge

A significantly higher proportion of participants responded correctly to the 20 knowledge items in the IMMED compared with PRE, 84% vs 60% (P ≤ 0.02), respectively.

Intention to Change

Immediate post-program, 87% of participants stated they were planning to make at least one change to align their practice with guideline-based care. The most frequently stated changes were 1) to improve opioid prescribing documentation (56%); 2) to implement or improve opioid prescribing patient education or communication (53%); and 3) to institute or improve Patient-Prescriber Agreements (47%).

2MO: 2-Months Post-Program Assessment (N = 476)

Knowledge Maintenance

Compared with the PRE, the proportion of correct responses at 2MO was significantly (P ≤ 0.03) higher for 7 out of the 10 knowledge questions on opioid misuse risk factors and risk assessment. While the improvement in correct responses in the 2MO (69%) compared with PRE (60%) was modest, it was significant.

Confidence

Approximately two-thirds of participants reported increased confidence in guideline-based opioid prescribing practices including assessing pain and opioid misuse risk and assessing, monitoring and discussing opioid benefits, risks, and harms with their patients (Table 2).

Table 2.

Changes in confidence in performing guideline-based clinical practices

Statements	2-Months Post-Program Assessment (n = 476)
	Rate your confidence in your ability to accomplish each of the following as you attended the program:
	Increased	Remained the same	Decreased
Assess pain in a new patient?	65% (311)	32% (153)	3% (12)
Assess the potential benefit and risk of opioids for chronic pain in a new patient?	72% (341)	26% (126)	2% (9)
Communicate and collaborate with patients around opioid initiation?	71% (338)	28% (132)	1% (6)
Monitor patients on chronic opioid therapy for opioid misuse, including addiction and diversion?	63% (301)	34% (164)	2% (11)
Effectively and efficiently assess your patients for potential misuse of opioids?	67% (318)	32% (151)	1% (7)
Effectively communicate with your patients when treatment has shown no benefit	63% (300)	34% (160)	3% (16)

Attitudes

Participants reported on average an increase of 9% in alignment with increased trust in their patients and with guideline-based care (P ≤ 0.01). For example, to the statement I trust that available pain scales provide reliable assessment of pain in my patients, 48% of participants responded 4 or 5 on the agreement scale (1 is completely disagree and 5 is completely agree) at 2MO, as compared with 31% at PRE, a 17% increase (P < 0.01). For the items for which a decrease in agreement was desired, the proportion of participants who reported being in agreement decreased on average by 7% (P ≤ 0.02) (Table 3).

Table 3.

Changes in attitude in managing patients with chronic pain (n = 476)

		Percent (n) Reported ≥ 4 on the Agreement Scale Scale: 1-Strongly Disagree to 5-Completely Agree
Statement	Desired Change	Pre-Program	2-Month Post-Program	% Change	P value
Statements that should have MORE agreement
I trust that most of my patients with chronic pain are able to provide an accurate self-assessment of their pain	↑	48% (227)	50% (239)	+2%	0.314
I trust that available pain scales provide reliable assessment of pain in my patients	↑	31% (149)	48% (230)	+17%	<0.001
It is my responsibility and role to discuss with my patients not to give away their medications to relatives or friends	↑	92% (437)	96% (459)	+4%	0.001
I am comfortable responding to family calls about my patients’ possible misuse of opioids	↑	50% (237)	62% (296)	+12%	<0.001
Statements that should have LESS agreement
There is no reliable way to identify those of my patients who are drug-seekers	↓	29% (138)	21% (102)	−8%	0.020
Treating and managing patients with chronic pain is time-consuming and frustrating	↓	68% (326)	64% (304)	−4%	0.054
I will never prescribe ER/LA opioids to a patient with history of mental health issues	↓	16% (77)	17% (82)	+1%	0.564
I cannot get my patients to be truthful about illicit drug use	↓	29% (137)	22% (107)	−7%	0.004
I am uncomfortable communicating an unexpected urine drug test result to my patients	↓	24% (112)	20% (97)	−4%	0.187
I am unsure I am effectively assessing opioids misuse risk in my patients with chronic pain on ER/LA opioids	↓	48% (226)	31% (147)	−4%	<0.001
I suspect there is more I should be doing in the treatment and management of my patients who report chronic pain	↓	76% (360)	58% (275)	−18%	<0.001
I prefer to stop seeing/following a patient who has misused his/her opioid prescription	↓	57% (273)	51% (242)	−8%	0.007
I would only ask for a urine drug test from a patient that I thought was abusing the opioid prescription	↓	19% (90)	13% (63)	−6%	0.003

Clinical Practice (Patient Communication and Guideline-Based Care)

Patient Communication (Table 4)

Table 4.

Changes in patient communication (n = 476)

Improvements were made in all seven recommended communication skills with a significant increase from PRE to 2MO in participants reporting performing these behaviors with most/all of their patients with chronic pain from an average of 64% to 78% (P < 0.01), respectively.

Guideline-Based Care (Table 5)

Table 5.

Changes in guideline-based practices (n = 476)

Changes to Practice	2-Months Post-Program Assessment
	Have you made any changes in your practice, system care, and/or patient care as you participated the program entitled Scope of Pain: Safe and Competent Opioid Prescribing Education?
	% (n) who partially/fully implemented	% (n) who implemented before participating in this activity	% (n) who are planning on implementing in next 6–12 months or not planning to implement
Implement or improve …
Patient Prescriber “Agreements”	47% (225)	26% (143)	27% (128)
Informed consent procedures	45% (216)	18% (84)	37% (176)
Urine drug testing for monitoring	52% (246)	19% (92)	29% (138)
Pill counts for monitoring	43% (204)	10% (49)	47% (223)
Patient education or communication strategies	67% (319)	13% (63)	20% (94)
Office-wide policies/procedures	49% (233)	18% (86)	33% (157)
Multidisciplinary team approach	48% (227)	14% (65)	39% (184)
Documentation in patient medical records	68% (325)	17% (80)	15% (71)
Register/begin using the Prescription Drug Monitoring Program	45% (214)	26% (124)	23% (108)

When presented with nine specific clinical practice changes at 2MO: 68% had either partially or fully improved their opioid prescribing documentation in patient medical records, 67% reported having implemented or improved patient education and communication relating to opioid prescribing and 52% reported having implemented/improved urine drug testing for monitoring opioid adherence and misuse. Approximately 60% reported partially/fully implementing four or more changes in their practice with 35% implementing 7–9 changes.

Barriers to Change

Eighty-three percent of participants reported at least one barrier to making practice change. The most significant barriers reported were patients’ resistance to change (23%) followed by other providers’ or institutional resistance to change (17%).

Discussion

SCOPE of Pain, an ER/LA opioid REMS program, resulted in improvements in knowledge and attitudes about safe opioid prescribing, as well as increases in self-reported confidence and implementation of improved communication skills and guideline-based opioid prescribing practices. There were increases in clinician trust in patients with chronic pain and in the tools available to assess patients’ pain and to detect opioid misuse.

For the first time, an FDA REMS included the mandate for independent continuing education to be funded by commercial entities to help mitigate the risks of their medications. While education is a natural part of any REMS, whether you must teach about a mandated registry or how to document safe-use conditions (e.g., pregnancy tests), this REMS included an extensive, prescribed curriculum developed by the FDA and not the providers of the education. This is distinct from the usual process of how content for continuing education is created by the provider.

While the need for prescriber education is universally accepted, this REMS has been met with some skepticism [37]. This study is a first step in evaluating this national strategy of clinician continuing education as a way to improve safe opioid prescribing. The comparison among PRE, IMMED, and 2MO assessment data suggest that not only did clinicians learn more about safe opioid prescribing, but they have more confidence and were able to make changes to align with guideline-based practices. While knowledge gain did decrease in the 2MO, it did not return to baseline, and in fact continued to be significantly higher than the PRE-assessment. Without repeated exposure deterioration of knowledge is an expected outcome in education studies.

While the evaluation of this REMS education is based on self-reported data and does not include objective measures (e.g., decreases in prescription opioid misuse) to demonstrate the effectiveness of the training, it does demonstrate that education based on content from the FDA, developed by continuing education providers, and funded by commercial interests can still yield a positive impact on self-reported changes in behavior.

There are a growing number of state policy, systems-level, and payer interventions being promulgated to address the prescription opioid misuse problem [31]. While these interventions appear to be efficient solutions to controlling prescription opioid misuse, such blunt instruments risk the unintended consequences of making opioids inaccessible for those that currently or potentially may benefit. In contrast, quality, targeted education can empower clinicians to make appropriate and informed clinical decisions about whether or not to initiate, continue, change or discontinue opioids for each individual patient suffering from chronic pain based on a careful benefit vs risk/harm assessment [38,39]. Educational approaches will maintain access for patients who do, or can, benefit from such medications while mitigating the potential risks to those who are not benefiting or are being harmed. While there has been considerable skepticism about continuing medical education's (CME) ability to improve clinicians’ practices [40], recent meta-analyses have supported that, overall, CME, especially using serial educational interventions, is effective in changing clinician performance [41,42]. As opposed to regulations limiting clinician practice, education is a tool that can help clinicians develop the nuanced, informed approach necessary for individualizing patient care with regards to safe opioid prescribing.

Questions remain on next steps to enhance the current REMS education. This speaks to the need for a clinician awareness campaign regarding the availability of these REMS trainings. While the REMS program is mandatory for the ER/LA opioid manufacturers, it is not mandatory for clinicians [37]. In one primary care survey [43], less than 10% of physicians were “very familiar” with the REMS education. Since the first announcement by the FDA regarding the opioid REMS program there has been debate as to whether clinician education should be mandated and linked to US Drug Enforcement Administration (DEA) licensure [44]. A training requirement is not unprecedented, as there is such a requirement within the Drug Addiction Treatment Act of 2000 [45] (DATA 2000) which limits the prescribing of buprenorphine for the treatment of opioid use disorders to those that have completed an 8-h training. While the DATA 2000 training requirement is highly supported by addiction medicine/psychiatry societies, only a small number of physicians have taken the training, which has resulted in limited access to this life-saving treatment for those who need it [46,47]. Thus, it would be important to link mandated opioid prescribing training to DEA licensure to avoid having clinicians “opt out” of this requirement leading to decreased treatment access and burn-out for those clinicians that “opt in.” However, to make education mandatory there must be evidence that education would positively impact prescription opioid misuse without decreasing appropriate access to prescription opioids. Alternatively the goal could be mandatory demonstration of clinical competence allowing those clinicians well trained in this area to “test out” of the requirement. Finally, including practice-based performance improvement or quality improvement efforts following SCOPE of Pain education may lead to more robust clinical practice changes, but would require a more substantial investment in time and resources [48,49].

With any intervention, education or otherwise, it would be ideal to measure changes in clinical outcomes, such as fewer opioid overdoses and overdose deaths, and fewer emergency department visits. However, these important clinical outcomes would be difficult to attribute to any education alone as there are other concurrent efforts [31] that could also improve these outcomes including naloxone distribution [50], expansion of office-based opioid addiction treatment [51] with buprenorphine and naltrexone, and the availability of abuse-deterrent opioid formulations [52,53]. Evaluations focusing on decreasing the number of opioid prescriptions [54] are difficult to interpret as it is unclear what the correct amount of opioid prescribing should be to concurrently decrease opioid misuse while maintaining access to opioids for those who benefit.

The SCOPE of Pain evaluation has several limitations worth considering. Because our post-program assessments, with the exception of knowledge-testing questions, were self-reported by the participants there is risk of self-assessment bias and social desirability bias. To mitigate social desirability bias, participants completed their follow-up surveys anonymously to an independent evaluator. Program participants with a particular interest in the program objectives were potentially more likely to participate in the 2-month follow-up assessment. In addition, as this was a voluntary program, those that were interested in changing practice were more likely to enroll and, therefore, may have a greater change than the general population of practitioners. Therefore, there is the potential for participant self-selection bias. However, the demographics of those that completed the 2-month follow-up were similar to those that did not. The lack of a control group makes it difficult to attribute participant changes solely to SCOPE of Pain, however, many of the questions asked participants to attribute changes specifically to the program. While we found improvements in participant clinical knowledge, confidence, attitudes, and self-reported practice, we were unable by study design to detect if these improvements impacted patient care. Future research on ER/LA opioid REMS education should consider a more in-depth investigation on the impact on patients’ care [55].

There were a few areas where this model did not succeed. First, the FDA Blueprint is very comprehensive and requires up to 2–3 hours of education. Some participants, particularly for the web-based activity, started the program but did not complete it. For the live activity, participants were required to pass a post-test to be counted as a program completer. As clinicians are not accustomed to completing a post-test for live activities, some participants attended the entire meeting, but could not be counted as completers of the education because they did not take the post-test.

In summary, the ER/LA opioid REMS training SCOPE of Pain improved clinician-level safe opioid prescribing outcomes, however, its impact on mitigating opioid misuse risk and harm while maintaining access to opioids for those that are or would benefit remains an unanswered question. While education cannot be the only strategy to combat this national crisis, it can help improve clinician behaviors and be a major part of the solution.