In 2018, opioids were involved in 46 802 overdose deaths.1 To help address the opioid overdose epidemic, the Centers for Disease Control and Prevention (CDC) released its Guideline for Prescribing Opioids for Chronic Pain (CDC Prescribing Guideline) in March 2016. To support providers in implementing the guideline, a set of 16 quality improvement (QI) measures were developed that align with the CDC Prescribing Guideline. The goal of these QI measures is to provide health care systems with the means to track their progress in implementing recommended practices to improve care over time. As the QI adage states, “You cannot improve what you cannot measure.”
When the CDC Prescribing Guideline was published, only a few opioid-related measures existed.2 Since then, some have been developed but remain based on claims or pharmacy data, including measures of opioid misuse,3 high dosages, and co-prescribing of benzodiazepines (NQF 3389, NQF 3316e).3 Electronic health record (EHR)-based QI measures have the potential to provide timely information using dashboards for audit and feedback to clinicians.4
This article describes the development of the first EHR-based opioid QI measures aligned with the CDC Prescribing Guideline recommendations to support opioid QI efforts.
Approach to Developing the Opioid QI Measures
The development of the QI measures required operationalization of all 12 recommendations in the CDC Prescribing Guideline. Beginning with early versions of several measures the CDC drafted in collaboration with experts in the field, we searched measures databases, and peer-reviewed and gray literature for an initial set of 23 QI measures.
To evaluate each measure, we used existing measurement criteria from National Quality Forum,5 the Centers for Medicare & Medicaid Services Measures Management System Blueprint,6 and the National Quality Measures Clearinghouse7 (Table 1). We then created a template to evaluate each measure on each of the criteria. Then, 9 stakeholders familiar with chronic pain, opioid prescribing, and potential misuse among patients in primary care were recruited to evaluate and provide individual input on the measures on those criteria and in 3 stages:
Table 1.
Stakeholder Ratingsa for a Sample of QI Measures.
| Measure #: Measure Description | CDC Rec. # | Importance | Face Validity | Timeliness | Acceptable | Useful | Not Undue Burden | Overall Rating | Rank: Needed and Easy to Buildb |
|---|---|---|---|---|---|---|---|---|---|
| New opioid prescription measuresc | |||||||||
| Measure 1: The percentage of patients with a new opioid prescription for an immediate-release opioid. | 4 | 3.7 | 3.7 | 3.7 | 3.7 | 3.6 | 3.7 | 3.4 | 11% |
| Measure 2: The percentage of patients with a new opioid prescription for chronic pain with documentation that a PDMP was checked prior to prescribing. | 9 | 4.4 | 4.3 | 4.2 | 3.8 | 4.2 | 2.7 | 4 | 22% |
| Long-term opioid therapy measuresd | |||||||||
| Measure 7: The percentage of patients on LTOT who are taking ≥90 morphine milligram equivalents per day. | 5 | 3.9 | 3.6 | 3.8 | 3.7 | 3.6 | 3.2 | 3.6 | 55% |
| Measure 9: The percentage of patients on LTOT who had a follow-up visit at least quarterly. | 7 | 4.3 | 4.4 | 4 | 3.8 | 4.1 | 3.7 | 4.2 | 44% |
| Measure 10: The percentage of patients on LTOT who had at least quarterly pain and functional assessments. | 2 | 4.1 | 4.3 | 4 | 4 | 4.2 | 3.3 | 3.9 | 11% |
| Measure 11: The percentage of patients on LTOT who had documentation that a PDMP was checked at least quarterly. | 9 | 4.1 | 4.3 | 4 | 3.8 | 4.1 | 3 | 3.9 | 33% |
| Measure 13: The percentage of patients on LTOT with documentation that a urine drug test was performed at least annually. | 10 | 4.6 | 4.6 | 4.3 | 4.3 | 4.4 | 4.2 | 4.1 | 77% |
| Measure 14: The percentage of patients with chronic pain who had at least one referral or visit to nonpharmacologic therapy as a treatment for pain. | 3.4 | 3.9 | 3.1 | 3.2 | 3 | 2.3 | 2.8 | 0% | |
| Measure 15: The percentage of patients on LTOT who were counseled on the purpose and use of naloxone, and either prescribed or referred to obtain naloxone. | 8 | 4 | 4.1 | 4 | 3.4 | 4.1 | 2.7 | 3.3 | 22% |
Abbreviations: CDC, Centers for Disease Control and Prevention; LTOT, long-term opioid therapy; PDMP, prescription drug monitoring program; QI, quality improvement.
Based on a 5-point Likert-type scale (ie, 1 = strongly disagree, 5 = strongly agree; colors are as follows, average rating: green ≥4, yellow 3–3.9, and orange <3. Note: The ratings are based on the original draft measure posed to the stakeholders; the measure description reflects the final measure.
This is the percentage of stakeholders who indicated this measure was both needed and easy to produce (range 0% to 100%).
The denominator for new opioid prescription measures is prescribed an opioid with no opioid prescription in the previous 45 days.
The denominator for long-term opioid therapy measures is ≥60 days of an opioid within a quarter.
Stage 1. Online Survey: Stakeholders rated each measure on the criteria in the template and provided input on challenges to pulling the measures. Stakeholders selected the 10 most needed measures, and from those, ranked the 5 easiest to produce.
Stage 2. Semistructured Interviews: We conducted individual telephone interviews to discuss survey responses, and ask about their opioid QI experience.
Stage 3. Group Discussion: To further specify the measures, we discussed with stakeholders and the CDC, after which several measures were revised or dropped.
Survey results are based on the average ratings for each measure on each criterion. Qualitative data were synthesized to identify themes. Data collection was approved under an expedited review of Abt Associates’ Institutional Review Board.
Stakeholder Feedback
Table 1 presents the stakeholder ratings for a sample of QI measures. These specific 9 measures were chosen to illustrate the range of feedback for different types of clinical care (ie, laboratory tests, prescribing) and how this care is captured in EHRs, and to illustrate a range of ratings on different criteria. For example, Measure 13 (use of urine drug testing) was rated important, valid, provides timely information, useful, would not be too burdensome to produce, and was ranked by stakeholders as needed and easy to build, whereas Measure 14 (access nonpharmacologic therapy) was rated relatively high in terms of importance, but high burden.
The final set of 16 QI measures are publicly available in the CDC Quality Improvement and Care Coordination: Implementing the CDC Guideline for Prescribing Opioids for Chronic Pain.8
Anticipated Challenges to Producing the Measures and Potential Solutions
Stakeholders provided input on anticipated implementation challenges. The most critical challenges were related to limitations of EHR data to support the creation of denominators (eg, long-term opioid therapy [LTOT]). Many EHRs are unable to capture or calculate days’ supply. A potential solution identified was to use the number of opioid prescriptions in a period of time, or to build a registry to track patients. A similar challenge for establishing denominators included identification of a pain condition or diagnosis as part of the measure specification that is not readily available in EHRs. Solutions included prioritizing select conditions or using the opioid medication as a proxy for a pain diagnosis.
Other challenges involved identifying referrals related to pain care or opioids, or capturing referrals outside of a health care system. Similarly, reflecting recommended processes of care (eg, counseling) was identified as a challenge for which solutions such as building structured fields or check boxes could better identify the completion of these recommended processes.
Conclusion
The 16 QI measures are meant to support providers, practices, and systems in their QI efforts to provide safer care to patients on LTOT for chronic pain. Although some of the solutions to operationalize the measures may not be ideal for precision, they are critical for feasibility.9 Health care systems across the country can tailor the measures to their systems’ policies and monitor the measures over time to observe trends. The small sample size and purposive sampling of expert stakeholders are limitations to this work. The next phase for these measures is to have health systems build the measures in their EHRs to support their opioid QI initiatives as participants in the CDC Opioid QI Collaborative—a collaborative of health systems aiming to improve care of patients with chronic pain and on long-term opioids by pursuing QI efforts, using the QI measures to monitor changes over time, and engaging in shared learning with other systems. Results on how systems operationalized these measures, trends, and ultimately the effect of their QI efforts in changing practice are forthcoming.
Acknowledgments
We would like to thank the individuals who served as expert stakeholders to provide critical feedback on the draft measures: Daren Anderson, MD; Erin Krebs, MD, MPH; Joanna Starrels, MD; Benjamin Nordstrom, MD, PhD; Thomas Isaac, MD, MPH; David Labby, MD, PhD; Carol Havens, MD; Stephen “Miles” Rudd, MD, FAAFP; and Alicia Guilford, MD. A debt of gratitude is warranted for David Baker, MD, and Roger Chou, MD, who started a draft set of measure ideas aligned to the CDC Guideline recommendations. We would also like to acknowledge several individuals involved in the Dr. Robert Bree Collaborative with whom we shared several conversations about each of our approaches and thinking about measure specification, although we were relying on different data sources to produce the measures.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is led by Abt Associates and funded by the Centers for Disease Control and Prevention (Contract No. 200-2016-F-92356). The content, findings, and conclusions of this paper do not necessarily reflect the views, policies, or official position of the US Department of Health and Human Services, the Centers for Disease Control and Prevention, or Abt Associates, nor does the mention of trade names, commercial products, or organizations imply endorsement by the US government.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
- 1.Hedegaard H, Miniño AM, Warner M. Drug Overdose Deaths in the United States, 1999–2018. NCHS Data Brief, No. 356. National Center for Health Statistics; 2020. [PubMed] [Google Scholar]
- 2.Dr. Robert Bree Collaborative. Opioid prescribing metrics. July 2017. Accessed August 20, 2017. http://www.breecollaborative.org/wp-content/uploads/Bree-Opioid-Prescribing-Metrics-Final-2017.pdf
- 3.Pharmacy Quality Alliance. PQA Opioid Measure Set—2020. Accessed September 10, 2019. https://www.pqaalliance.org/assets/Measures/PQA_Opioid_Core_Measure_Set_Description.pdf
- 4.Anderson D, Zlateva I, Khatri K, Ciaburri N. Using health information technology to improve adherence to opioid prescribing guidelines in primary care. Clin J Pain. 2015;31:573–579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.National Quality Forum. Measure Evaluation Criteria and Guidance for Evaluating Measures for Endorsement. Version 6.5. Accessed September 10, 2019. http://www.qualityforum.org/Measuring_Performance/Submitting_Standards.aspx
- 6.Centers for Medicare & Medicaid Services. CMS Measures Management System Blueprint. Accessed September 10, 2019. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS-Blueprint
- 7.Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse (NQMC) inclusion criteria. Accessed September 10, 2019. https://www.ahrq.gov/gam/summaries/inclusion-criteria/index.html [DOI] [PubMed]
- 8.Centers for Disease Control and Prevention. Quality Improvement and Care Coordination: Implementing the CDC Guideline for Prescribing Opioids for Chronic Pain. National Center for Injury Prevention and Control, Division of Unintentional Injury Prevention; 2018. [Google Scholar]
- 9.Shoemaker-Hunt SJ. Observations from a QI collaborative of healthcare systems implementing the opioid QI measures for improvement purposes. As part of the panel “Opioid Measurement for Improving Care: Findings from the CDC Opioid Quality Improvement Collaborative.” AcademyHealth Annual Research Meeting; June 2019. Accessed January 20, 2020. https://academyhealth.confex.com/academyhealth/2019arm/meetingapp.cgi/Session/20607 [Google Scholar]