A New Era of Quality Measurement in Rheumatology: Electronic Clinical Quality Measures and National Registries

Abstract

Purpose of Review

We review the evolution of quality measurement in rheumatology, highlighting new health-information technology infrastructure and standards that are enabling unprecedented innovation in this field.

Recent Findings

Spurred by landmark legislation that ties physician payment to value, the widespread use of electronic health records (EHRs), and standards such as the Quality Data Model, quality measurement in rheumatology is rapidly evolving. Rather than relying on retrospective assessments of care gathered through administrative claims or manual chart abstraction, new electronic clinical quality measures (eCQMs) allow automated data capture from EHRs. At the same time, Qualified Clinical Data Registries, like the American College of Rheumatology’s RISE registry, are enabling large-scale implementation of eCQMs across national EHR networks with real-time performance feedback to clinicians. While successful examples of eCQM development and implementation in rheumatology and other fields exist, there also remain challenges, such as lack of health system data interoperability and problems with measure accuracy.

Summary

Quality measurement and improvement is increasingly an essential component of rheumatology practice. Advances in health information technology are likely to continue to make implementation of eCQMs easier and measurement more clinically meaningful and accurate in coming years.

INTRODUCTION

Physician payments in the U.S. are shifting from a fee-for-service to a value-based system. The 2015 Medicare Access and CHIP Reauthorization Act (MACRA) created a Merit-based Incentive Payment System that consolidates and enhances previous programs that attempted to address quality and value in healthcare, including the Physician Quality Reporting System, the Value Modifier Program, and the Medicare Electronic Health Record (EHR) Incentive Program.¹ The broadly bipartisan legislation has the unprecedented and ambitious goal of tying a majority of Medicare physician payments to quality measurement over the coming years. Not surprisingly, the magnitude of this payment reform has increased the need for robust and meaningful quality measures relevant to rheumatologists and other medical specialists.

Quality measures in rheumatology continue to evolve. Over the last decade, measures largely relied on two data sources: administrative billing claims and manual chart abstraction. Claims-based measures had the advantage of collecting information from all of the clinicians and entities that have submitted bills for clinical care, thus capturing a broad picture of health care received.^2,3 They were also relatively easy for payers to access and analyze. However, the clinical information contained in billing claims was limited in scope, and many relevant elements of the care that rheumatologists provided were not captured. Moreover, the use of codes in billing is often incomplete or inconsistent. Other quality measures in rheumatology have relied on manual medical chart abstraction.³ Although this method often yielded more clinically detailed data, it was labor intensive, incomplete, and difficult to implement. Moreover, while these approaches enabled retrospective performance measurement, they were less conducive to providing information to rheumatologists in real-time to support rapid cycle quality improvement.

To address these limitations, there is increasing interest in leveraging EHRs to develop electronic clinical quality measures (eCQMs) in rheumatology.^4,5 eCQMs are a new type of measure that rely on automated extraction of information from the EHR. Coupled with local data analytics and innovations such as nationally Qualified Clinical Data Registries, which centrally analyze and feed data back to practices, eCQMs have potential to reduce the burden of data collection for quality measurement and to serve as useful tools to drive continuous quality improvement. However, there are significant challenges ahead in developing, testing and implementing eCQMs. Data extraction from the EHR is complex and may have insufficient accuracy.⁶ Nevertheless, there have been important technological advances in building infrastructure for eCQM implementation that are likely to propel this field forward in rheumatology. These include the development of standards for eCQM development, such as the Quality Data Model, successes in aggregating data from different EHR systems into national registries, such as the Rheumatology Informatics System for Effectiveness (RISE), and the application of methods such as natural language processing to enhance the accuracy of eCQM data collection.^7,8

In this review, we describe the evolution of quality measurement in rheumatology and critical innovations within the past 10 years. We review the scientific literature on eCQMs development and testing between 2013 and 2016. Finally, we analyze the persistent challenges, promising technological advances, and future directions of eCQM development and implementation in rheumatology.

Early Efforts to Measure Healthcare Quality in Rheumatology

To understand the current developments in quality measurement in rheumatology, it is useful to briefly review early efforts to define and measure quality of care. A primary focus of early work was to develop evidence and consensus-based standards,^9,10 and to apply these measure concepts to readily accessible data.

For example, in 2005, the National Committee for Quality Assurance introduced a measure for health plans examining whether patients with rheumatoid arthritis (RA) had received disease-modifying anti-rheumatic drugs (DMARDs) in the previous year.² Data for this measure could be assessed entirely from billing claims, although studies using clinical cohorts and registries were also performed to assess performance on the measure.¹¹ These initial studies suggested significant variation between health plans on measure performance as part of the Health Effectiveness Data Information Set (HEDIS) program. Measurement created financial incentives for health plans to improve access to rheumatologists. National data, in fact, indicated improvements over time in DMARD use for RA patients, suggesting health plans specifically targeted this measure for improvement in the years after implementation.²

Additional measures put forth in 2008 by the American Medical Association’s Physician Consortium for Practice Improvement addressed care in rheumatoid arthritis, osteoarthritis and osteoporosis. For example, process measures regarding assessment of disease activity and functional status in RA were included. These measures were used in the Physician Quality Reporting System (PQRS), a pay-for-reporting program that represented the first attempt of government payers to tie physician payments to quality. Most participating rheumatologists reported these measures either through claims data or by performing manual chart reviews and entering data into the Rheumatology Clinical Registry, the first version of the American College of Rheumatology’s quality registry.³ While performance on these measures identified some areas with opportunities for quality improvement, participation in the PQRS program was limited: small financial incentives for participating and the time-intensive nature of reporting led most eligible rheumatologists to opt out. The rheumatologists who did participate identified shortcomings with several of the measures,³ such as unclear specifications that made abstracting information in a standardized way difficult. Other studies suggested insufficient capture of key clinical details that led to an underestimation of performance with claims-based reporting.¹² These limitations led to renewed efforts to develop improved measurement systems and more meaningful measures in rheumatology.

EHRs, the Meaningful Use Program and eCQMs

The limitations of quality measures relying on administrative claims data, the time-intensive nature of medical chart reviews, and the opportunities presented by the growing use of EHRs, led to calls to leverage new health-information technology enabled infrastructure to measure quality of care electronically through eCQMs.

A key development that made the advancement of eCQMs possible was the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009. The legislation invested $20 billion to advance the use of health information technology, including EHRs, in the United States. Key goals were to create uniform standards for electronic information in the EHR and to tie incentive payments to demonstrating “Meaningful Use” of the EHR.¹³ For rheumatologists, one component of the Meaningful Use program was to use the EHR for automated reporting of quality measures. Although this effort was controversial given debates about the importance of measures examined to rheumatology subspecialty care and the problems with accuracy of automated data extraction of quality measures, it laid the foundation for the development of eCQMs in the coming years.

Similarly, new data standards also enabled eCQM development. The Health Quality Measures Format set standards for electronic documentation in the EHR, including consistency in the eCQMs structure, metadata, definitions and logic.¹⁴ In addition, the Quality Data Model was introduced, an information model that defined relationships between patients and clinical concepts in a standardized format to enable eCQMs.⁷ The National Quality Forum developed criteria for establishing the feasibility and validity of eCQMs.¹⁵ Together, the growing infrastructure of EHRs in the United States and these data standards and models laid the foundation for an expansion of eCQM development, but also significantly increased the complexity and cost of measure development.

Lessons from the Development and Testing of eCQMs Across Specialties

The initial approach to the development of eCQMs was to retool quality measures originally developed for claims-based reporting or for manual chart review. Studies examined the feasibility and accuracy of eCQMs in pediatrics,¹⁶ asthma,¹⁷ cancer,^17,18 diabetes,¹⁷ coronary heart disease,¹⁹ cardiovascular disease,^20,21 and in primary care and preventive health screening.^17,21 Not surprisingly, many of these studies reported challenges with data extraction and accuracy.

For example, studies found wide measure-to-measure variation in the accuracy of eCQMs compared to manual chart review as the gold standard. The sensitivity of 12 eCQMs used in the Meaningful Use program ranged from 96% for influenza vaccine to 46% for use of appropriate asthma medications among patients in a qualified health center.¹⁷ Similarly, eCQM sensitivity ranged from 84% for immunization measures to 39% for counseling and risk behavior measures in a study examining adolescent well-care measures.¹⁶ A theme from these studies was that sensitivity was highest when eCQMs derived data from structured fields, such as immunization records. Care that was complex, or delivered across multiple specialties or settings, was harder to capture accurately in eCQMs.¹⁸ In other words, in cases where clinicians documented processes of care more commonly in unstructured formats, such as clinical notes, sensitivity was severely compromised.^16,19,21 This was particularly true for care that is almost entirely documented in narrative form, such as psychological screening and counseling.²² Across studies, initial work on eCQMs demonstrated that 39% to 65% of required data for existing measures resides in physician notes or medication administration records outside the EHR, suggesting that automated quality reporting based on the use of structured data is limited without systems to draw on these additional data sources.^23,24

Several studies also attempted to examine the feasibility of constructing new eCQMs to address high priority areas. For example, one group examined the feasibility of translating the American Board of Internal Medicine Foundation’s Choosing Wisely Recommendations regarding overuse into eCQMs: this study reported that 32 of the 45 measure concepts would require data elements unlikely to be found in most EHRs.²⁵ Another group examined the feasibility of developing cancer care eCQMs in the Veterans Health Administration EHR and found that only 11 of 18 (61%) lung cancer measures were feasible, 4 (22%) of which were considered as valid measures of the care desired construct.¹⁸ Similarly, of measures developed by the American College of Cardiology/American Heart Association Task Force on Performance Measures, only three were amenable to eCQM construction. These measures were based on LDL-C measurements and statin therapy and the data was extracted from laboratory and medication lists rather than from unstructured data.¹⁹

The American College of Rheumatology was among the first professional societies to begin an eCQM development program in 2012, resulting in National Quality Forum endorsement of measures for RA in 2014.^4,5 Like studies in other conditions, eCQM validation work in RA also found variable sensitivity and specificity between measures and between health systems. Health systems with mature quality measurement programs and established workflows to record key data elements in structured EHR fields achieved remarkably high sensitivity and specificity for the RA eCQMs. Other health systems, where data was still largely in clinical notes, had lower concurrent validity between the performance score calculated by the automated eCQM and manual chart abstraction.⁵ These results were interesting insofar as they demonstrated that the accuracy of eCQMs depends not only on the construction of the measure itself, but also on the local workflows and EHR configurations in place to accurately capture critical data elements.

Implementation of eCQMs in Health Systems

With the availability of eCQMs and the rising costs of reporting manually-extracted quality measures to payers, accreditation commissions, and for internal quality improvement purposes, some organizations have explored expanding eCQM implementation across their healthcare systems to streamline quality measurement. Kaiser Permanente, a large integrated health care system, was among the first to tackle this challenge. Initial work at Kaiser found that 4 out of 5 National Quality Forum-endorsed eCQMs had data availability issues that made implementation of the measure specifications challenging.²¹ Nevertheless, as of 2014, Kaiser was able to partially or fully automate 6 of 13 quality metrics defined by the Joint Commission with an average of 61% data availability.²⁴

In another example, the Mayo Clinic reported 100% sensitivity for 11 intensive care eCQMs derived from an ICU DataMart, a real-time relational database that allowed for construction of rules and algorithms for automated measure calculations in the intensive care unit.²⁶ In the Veterans Health Administration integrated health system, the final version of a Meaningful Use program eCQM regarding stroke had nearly 90% accuracy when local algorithms were iteratively improved by examining patterns of misclassifications.²⁰ These studies illustrate that although significant challenges exist with eCQM implementation, mature data systems that iteratively examine and address measure misclassifications can achieve high levels of accuracy. However, this customization is often time-consuming and requires specialized informatics expertise.

EHR-enabled Registries and Quality Measurement

The widespread use of EHRs, growing interest in eCQMs and the need to generate a quality measurement infrastructure relevant to diverse medical specialties, led the federal government to create a new mechanism for quality reporting in 2014: Qualified Clinical Data Registries (QCDRs). Such registries are entities approved by the Centers for Medicare and Medicaid Services to collect, analyze and report data on quality measures and outcomes on behalf of practicing clinicians. Because data collected through QCDRs is meant to improve healthcare quality, there is a waiver of individual patient informed consent for registry data capture. This allows aggregation of clinical data generated during the course of routine clinical care across practices. As of 2016, there are over sixty QCDRs in the United States covering a range of medical specialties.²⁷

A QCDR for rheumatology, RISE, was developed by the ACR in 2014.⁸ RISE passively extracts data to populate eCQMs from individual practices, aggregates and analyzes these data centrally, submits eCQMs to federal agencies for quality reporting, and feeds performance information back to clinicians using a web-based interface. The registry has made headway in addressing some of the challenges of interoperability that have previously made sharing quality measure performance data across health systems difficult. RISE’s clinical informatics structure was designed to be agnostic to the EHR system used by rheumatologists. The registry can be adapted to draw data from most certified systems through a mapping process that identifies both structured and unstructured data in each practice’s EHR implementation.⁵ Central mapping of eCQMs also creates efficiencies for practices that have limited information technology support or data analytic capabilities. Current measures cover areas such as RA, osteoporosis, preventive health, gout and patient safety. Automation of eCQM extraction permits submission of quality measure performance on a clinician’s entire population of patients, thus reducing biases in patient selection. The data in RISE therefore provides a unique and inclusive view of quality in rheumatology practices because patients with all medical conditions managed by rheumatologists and all types of insurance are included in a single registry.

By feeding eCQMs back to practices, RISE also provides useful data to facilitate rapid-cycle quality improvement to the individual contributing practice. Practices can use the RISE web-based data dashboard to benchmark their performance on eCQMs against all other participating practices. This functionality is especially important under the new Medicare Incentive Payment System, since the program creates increasing financial penalties for low performers, while providing greater reimbursement for top performers. The RISE dashboard allows practices to determine their relative position before financial penalties or rewards are levied, providing them with data to ensure success under new payment reform models.

Aggregated EHR data across different practices and health systems is a new and interesting policy and research resource. Both management and analyses of these data will require rapid innovation in research methods and practices. Work is currently ongoing to understand and improve the accuracy of data mapping for eCQMs in RISE. We also anticipate that new measures across a range of rheumatic diseases will be added to the registry.

Opportunities and Challenges for Future Development and Use of eCQMs

Although eCQMs hold promise in reducing reporting burden for individual rheumatologists and in expanding the potential content of quality measures, work to date suggests that substantial infrastructure and analytic support at the practice level or through QCDRs will be required to scale eCQMs across the U.S. health care system.

The functionality of EHRs remains a key barrier, including cumbersome documentation that increases physician workload, erroneous or repetitive data included in medication lists and clinical notes that decreases the accuracy of reports on the quality care, and continued lack of interoperability that results in the inability to share data among clinical entities.²⁸ In a study of 967 physicians, only 35% of primary care specialists and 26% of medical specialists reported believing that EHRs will help them improve the care that they deliver.²⁹ In another study of 1,793 office-based physicians found that while over 80% reported some clinical benefits of the EHR, such as the ability to access a patient’s chart remotely, less than 40% reported that EHRs have resulted in less test ordering or even in more efficiently identifying needed lab tests.³⁰ These realities have constrained our ability to build a more robust, accurate electronic quality measurement infrastructure in the United States. However, we anticipate as EHR technologies improve, and as quality measurement becomes a core function of medical practice, the accuracy of eCQM data capture is likely to rapidly improve as well.

In the short-term, practices that have adopted workflows for reliable capture of key data elements for eCQMs in structured EHR fields are likely to have a performance advantage in quality reporting.^31,32 In the longer-term, computer scientists, clinical informaticists, and clinicians will need to work together to develop new technologies and tools useful to advance eCQM accuracy and implementation. For example, several groups have used natural language processing and text mining algorithms to increase the accuracy of eCQM data and improve quality.^33–35 Like natural language processing, machine learning algorithms relying on the text of clinical notes also provide new opportunities to capture data not found in standard structured data.

Work with these new methods has demonstrated that despite difference in EHR infrastructures, it is possible to develop and validate algorithms using data across different health systems.^36,37 A systemic review of 67 studies that extracted information from the clinical notes in EHRs using text searches, natural language programs algorithms, or machine learning algorithms found that while no particular text extraction methods stood out as superior, the inclusion of the information from text in combination with structured codes consistently improved accuracy.³⁸

We anticipate that rapid advances in natural language processing and machine learning applications in EHRs are likely to significantly impact the methods available for eCQM extraction in the near future. In the RISE registry, the ACR has already started to explore the use of data from the text of clinical notes to accurately identify key disease outcomes, such as RA disease activity and functional status scores. Work is ongoing to refine and validate these approaches across different rheumatology practices participating in RISE.

Conclusions

Spurred by landmark legislation that ties physician payment to value and the widespread use of EHRs in the United States, quality measurement in rheumatology is rapidly evolving. Rather than relying on retrospective assessments of care gathered through administrative claims or manual chart abstraction, health information technology is enabling development of eCQMs and large-scale implementation of these measures across national registries, like RISE, as well as across health systems. However, to be successful, a number of challenges will need to be addressed, including the functionality of EHRs to support quality measurement, tackling novel methods to incorporate both structured and unstructured EHR data into eCQMs, and ensuring data accuracy will be required.

KEY POINTS.

• Bipartisan health reform will increasingly tie rheumatologists’ payments to measures of quality and cost.

• Electronic clinical quality measures (eCQMs), which rely on data within electronic health records to automatically calculate performance, are increasingly being used to measure quality of care in rheumatology.

• Implementation of eCQMs in the Rheumatology Informatics System for Effectiveness, a national EHR-enabled registry, is allowing central data analysis and performance feedback to practices to facilitate local quality improvement.

• Despite opportunities to use eCQMs to make quality measures more meaningful and useful for practicing rheumatologists, challenges like lack of data systems interoperability and performance measure accuracy will need to be tackled in coming years.