Achieving the Promise of Electronic Health Record-enabled Quality Measurement: a Measure Developer’s Perspective

Abstract

Electronic health record (EHR) systems support local quality improvement efforts by health care organizations and provide the opportunity to address national priority areas for quality measurement, such as specialty care, overuse and efficiency, coordination of care, change over time and patient- reported outcomes (PROs). However, variations in provider workflow and documentation habits, adoption of advanced EHR functions and exchange of interoperable data, and eMeasure specification standards affect the ability to develop and test measures that target these high priority areas for improvement. Measure developers are working with providers, national standards organizations, and other eMeasure stakeholders to address these challenges and support learning health organizations in using EHR-based measurement to improve quality.

Introduction

Electronic health records (EHR) offer great potential to move quality measurement and quality improvement forward. Quality measurement is an effective way for health care providers to improve the quality of care by assessing the gaps between evidence-based medicine and actual clinical practice. EHRs can capture clinical results at the level of detail necessary for valid quality measurement that are not available in claims data and have previously only been available through labor-intensive medical record review. EHRs have the potential to integrate data from multiple sources of information, including patient-reported outcomes (PRO) and care delivered by different providers, to provide a more complete picture of a patient’s state of health over time. Reporting on such quality measures that take advantage of these longitudinal, comprehensive sources of patient information can support timely quality improvement efforts.

We are in the infancy of using EHRs in this manner. Prior to the Health Information Technology for Economic and Clinical Health Act part of the American Recovery and Reinvestment Act (ARRA HITECH) in 2009, EHRs were primarily used to replicate existing day-to-day operations—use of individual patient records to inform patient care one patient at a time and for billing. The HITECH Act introduced the need for “meaningful use” of certified EHR technologies and for EHRs to support clinical quality measurement. Since then, quality measure developers have been learning how to develop and specify measures that make the most of this new environment. In this article, we use the term “EHR” to mean either: a complete EHR that meets the requirements for Certification established by the Office of the National Coordinator for Health Information Technology; or a combination of EHR modules in which each module has been tested and certified in accordance with the requirements for Certification established by the National Coordinator.9 That is, an EHR is a record system with specific functions designed to manage electronic demographic and health information. This information and functionality can be contained in one “EHR system” or in multiple “EHR modules” that exchange information.

To have measure results that are comparable and reliable, eMeasure specification standards support consistent definition and implementation of measures across EHRs. eMeasure specifications use the same interoperability standards that are used in other aspects of health IT, such as use of standardized health care vocabularies and structured clinical document templates. In contrast to measures that must be manually programmed into EHR systems, eMeasures are meant to be automatically computer processable by different EHR systems, which supports consistent implementation across practices and EHR systems. Most eMeasures to date target EHRs as the intended data source, so eMeasures can be considered EHR measures. However, eMeasures may have other data sources as well: billing data, patient registries, and health IT systems other than EHRs.

Initial work in EHR quality measure development began with converting the specifications for existing non-EHR measures, intended to be calculated from claims or paper records, to specifications that can be used by EHRs.5 A second phase of work involves developing new measures designed specifically to take advantage of the capabilities of the EHR (Table 1).8 This article reports on what we have learned in developing clinical quality measures for Stages 1, 2 and 3 of Meaningful Use requirements and ARRA incentives. Our experience is primarily, but not exclusively, with ambulatory EHRs.

Table 1.

Key features of EHR systems that are important for quality measurement and improvement

Data Capture/Collection

Integration of data from disparate sources of information, including PROs Support for interprovider communication and exchange of information Assurance of the completeness of data required for care and improvement, such as standardized assessment tools and provider order entry

Analysis and algorithms

Support for aggregation of information across individuals and providers for population assessment and management Ability to execute complex algorithms and mathematic operations using data across all patients in a population of interest

Clinical decision support

Reminders and alerts Ability to detect clinically meaningful patterns, such as improvement and worsening, in patient data

Standards

To realize the potential value of eMeasures and EHR-enabled quality improvement, providers need EHR systems to represent clinical information accurately and consistently. eMeasure specifications are based on the same standards that ONC-certified EHR systems use to represent and exchange health care data (Table 2).8 Health care vocabularies and exchange standards are developed and maintained by standards development organizations, such as International Health Terminology Standards Development Organisation and Health Level Seven. These organizations are working with measure developers and the ONC to enhance eMeasure specification standards, so that they can represent the logic and calculation of innovative measures.

Table 2.

Examples of health IT standards that are used to specify and report eMeasures

Part of Specification	Standard	Standards Development Organization
Data Element Criteria and Logic	Health Quality Measure Format	Health Level Seven
Measure Result Reports	Quality Reporting Data Architecture
Medications	RxNorm	NLM
Laboratory Test Names	LOINC	Regenstrief Institute
Condition/Diagnosis/Problem	SNOMED CT	IHTSDO, WHO

IHTSDO: International Health Terminology Standards Development Organisation

LOINC: Logical Observation Identifiers Names and Codes

NLM: National Library of Medicine

WHO: World Health Organization

Addressing Emerging Interests

Using EHR data for quality measurement offers the opportunity to develop quality measures in areas in which there is great interest and which have been challenging or impossible using other data sources. These areas include measures of specialty care, overuse and efficiency, coordination of care, change over time and PROs.1–2,6,9

Measures of specialty care and overuse are difficult to develop without clinical details that are crucial to knowing when specialized interventions are, or are not, appropriate. Previously this detail has been available only through manual medical record abstraction. Abstraction from paper records does not support efficient and ongoing analysis of results for quality improvement because samples of records need to be pulled and medical records abstracted each time to analyze and document results across patients. EHRs can hold defined data elements as structured data and efficiently support computation of quality measure results and frequent analysis across patients and over time. However, the challenge is to encourage the use of standard terminologies to capture these elements in structured fields in a manner that supports workflow, decision making and measurement.

Many quality issues result from a lack of coordination of care between providers and across settings.4,¹² Within provider systems that use the same EHR, different providers can see the same data and address issues due to polypharmacy or can avoid duplicate tests. In the future, structured electronic documents that contain standardized data elements from EHRs should be able to be transferred from one system to another, further addressing this important quality issue. Slow enhancement of technical standards to support this functionality has limited development, use of measures of care coordination and episodes, and is frustrating to those who would like to work on quality improvement in these areas. Further adoption, more rapid enhancement of data exchange standards, and standardization of EHRs across providers is needed.

Systematic data collection directly from patients about their symptoms and functional status is often absent from care delivery. Standardized PRO tools have been used for research purposes; however, since clinical history taking and interviews by clinicians are the predominant means of obtaining patient information, only in a few settings have standardized PRO tools been incorporated into clinical workflows.7 EHRs offer the potential to facilitate this workflow because the data can be gathered from patients in a variety of ways, e.g., paper and online, and then fed into the EHR as structured data that are available at the point of care to support treatment decisions, as well as for quality measurement and analysis of results over time.

What We Have Learned

As measure developers who have begun to work with EHR vendors and clinical sites with EHRs, we have confronted challenges implementing these types of measures. These challenges need to be recognized and specific strategies adopted to overcome them.

Structured data versus notes

In some cases, EHR technology itself has not been the barrier, but the challenge has been working with clinicians to adapt workflows to take advantage of the technology. Too often data are included in unstructured notes when the EHR has fields to accept structured data that can be analyzed. Data buried in unstructured notes do not support the analysis needed for ongoing quality improvement. Overcoming this challenge requires clear policies about using structured data fields and the education of providers as to why this is important.

Initial quality measure reporting that shows low performance due to lack of structured data will help demonstrate the need for structured data. This has proven true as performance measurement has been rolled out in other health care settings. Data completeness gets better over time when the data are used for a meaningful purpose. Further efforts by EHR vendors to align clinical workflows and data flows for EHRs can also facilitate synergy between providing, measuring and improving care.

Completing the picture of patient health

Since patient outcomes are affected by actions and events that occur in different parts of the health care system, significant improvements in care can result from measures that assess patient care across settings. Important examples of measures that rely on data from different settings are (1) providing patients with appropriate primary care follow up after emergency department visits or hospital discharges and (2) reducing redundant testing and delays in access to results from testing or referrals.

Longitudinal care management can be improved when data from different settings is available at the point of care, such as in integrated delivery networks.3 Limited interoperability and inconsistent data exchange in other settings are significant barriers to quality improvement. Meaningful Use functional objectives are motivating providers, implementers, and technology vendors to adopt and enhance information exchange standards and practices; however, the rate and degree of the adoption of health information exchange varies widely. Much like organization-specific workflow policies and provider habits, variations in data exchange practices affect the ability of measurement to support improvement.

Embarrassment of riches (data)

The theoretical ability of EHRs to capture and exchange unlimited structured data elements is a blessing and a curse. On the one hand, it is this capability that opens up quality measurement to address areas that have previously been unavailable because of difficulties accessing medical records or a lack of documentation in paper records. On the other hand, exploiting this capability can quickly lead to overwhelming providers with data entry needs, “click fatigue,” complex algorithms to understand and a series of granular measures that don’t provide a coherent picture of the quality of care. Providers, many of whom are new to viewing such performance results, are confused and frustrated.

While performance measurement of providers precedes the use of EHRs, the EHRs make it easier to present providers with a greater amount of performance data. It will be important to work with specialty societies and other stakeholders to gain some consensus on the most important areas for measurement by specialty and to strive for parsimony. Composite measures can also help because they summarize results across a number of measures, while allowing drilling down to specific measures for those interested.

Challenges in testing

Health care providers and quality reporting programs want measures that are meaningful, actionable, and trustworthy in accurately reflecting processes and outcomes of care. Therefore, quality measures designed for EHR reporting should be tested at health care organizations that have adopted EHR functions that are relevant to the measures. There are several challenges to finding test sites for EHR-reported quality measures that address areas of interest: the reporting capability of the EHR system, the personnel and resources needed for testing, and protection of patient privacy and confidentiality. However, perhaps the greatest challenge to the field testing of new measures is finding sites that have fully incorporated advanced functions of EHR systems into their routine.

Relatively few provider organizations have implemented robust EHR system functions that make EHR-enabled quality measurement so promising: namely, structured data capture and information exchange. The quality measures that are of great national interest—PROs, care coordination, overuse versus efficiency— require access to information that must be (1) documented, (2) documented in a structured manner, and (3) documented in a manner that is available for reporting. Thus, a test site may have adopted ONC-certified EHR technology, but the test site may not take advantage of the software functions in place that allow the data integration or reporting needed for new quality measures. A quality reporting module may not have access, for example, to the data collected—using a kiosk—about the patient-reported status of patients’ knee arthritis or functional impairment due to chronic pain. This is particularly difficult when developing measures to be reported from EHRs in the future, such as for Stage 3 Meaningful Use requirements. It may be possible to identify one or two test sites that currently have these processes and capabilities in place, but there is interest also in demonstrating feasibility for a variety of practices. However, until adoption of advanced EHR functions by practices increases, field testing opportunities for new types of quality measures will be limited.

Standards for quality measure specification

Current standards for specifying and reporting measures are based predominantly on existing measures that relied on administrative and billing data or manual chart abstraction. Most of these quality measures assess the occurrence of targeted health-related events and report out a rate (e.g., proportion) of their occurrence. We are interested in EHR–based quality measures because of the EHR systems’ ability to perform complex calculations and to identify patterns and trends in the clinical data, rather than relatively simpler counts of occurrences. Traditional outcome measures related to hypertension and diabetes mellitus care assess the achievement of therapeutic goals or threshold values for blood pressure and HbA1c, respectively, at a specific point in time during a 12-month measurement period. Also helpful are measures that look for improvement over time and measures that summarize the ever-changing health states of patients, such as those in which patients’ blood pressure or medication management goes in and out of range several times throughout the measurement period. Using current specification standards, relationships between data and observations are expressed as Boolean logic, relative timing between data elements, and attributes of data elements (e.g., clinical reason for an order). For new measures such as we have described, specification and reporting standards need the ability to incorporate other clinically meaningful relationships, such as linking an abnormal test result with the specific actions taken by the provider in response to the abnormal result. This does not exist currently.

Increased rigor of EHR certification

In order to receive federal EHR incentives, providers must use certified EHR systems to calculate and report quality measures. The certification process for EHR systems must be sufficiently rigorous to test the reliability of the clinical quality measures—that is, they are programmed according to the eMeasure specifications by certified EHR systems. Without sufficient rigor of testing the programming, the quality measures may not be implemented correctly or in an adequate manner for generating comparable results across providers and systems. Being able to compare the results of measures across organizations is valuable in a learning health care system: enabling it to identify benchmark performance and what improvement methods are working, and to determine the comparative effectiveness of improvement efforts in different settings.

Conclusions

Measure developers are in the early stages of learning how best to use EHRs for quality measurement and improvement. The ARRA legislation, incentives, certification and meaningful use requirements provide an important stimulus and infrastructure for using EHRs for this purpose. With that as context, we need to continue to explore how to express measures in a standardized format to support the production of reliable results and how to validate that results are reliable. We need to continue working with standards-setting organizations to evolve standards to accommodate measures that take advantage of the capabilities of EHR data, rather than reflect the quality measures of the past designed for different data sources (claims and paper records). And we need to interface better with the provider community to introduce them to this type of measurement and its use for quality improvement.

We look forward to reporting on successful approaches to take advantage of the many opportunities provided for measure developments using electronic clinical data for measure development and quality improvement. As strategies that address the challenges outlined are implemented, we encourage the community to share lessons learned to accelerate progress for the field.