Meaningful measurement: developing a measurement system to improve blood pressure control in patients with chronic kidney disease

Abstract

Objectives

To develop an electronic registry of patients with chronic kidney disease (CKD) treated in a nephrology practice in order to provide clinically meaningful measurement and population management to improve rates of blood pressure (BP) control.

Methods

We combined data from multiple electronic sources: the billing system, structured fields in the electronic health record (EHR), and free text physician notes using natural language processing (NLP). We also used point-of-care worksheets to capture clinical rationale.

Results

Nephrologist billing accurately identified patients with CKD. Using an algorithm that incorporated multiple BP readings increased the measured rate of control (130/80 mm Hg) from 37.1% to 42.3%. With the addition of NLP to capture BP readings from free text notes, the rate was 52.6%. Data from point-of-care worksheets indicated that in 52% of visits in which patients were identified as not having controlled BP, patients were actually at goal based on BP readings taken at home or on that day in the office.

Conclusions

Building a method for clinically meaningful continuous performance measurement of BP control is possible, but will require data from multiple sources. Electronic measurement systems need to grow to be able to capture and process performance data from patients as well as in real-time from physicians.

Objective

Our objective was to use electronic systems to create a registry with two main functions:

1. Accurately identify patients with chronic kidney disease (CKD) treated in a nephrology practice, whose blood pressure (BP) is above target.

2. Use a clinically meaningful method to assess rates of BP control in a population of patients with CKD.

Background and significance

With healthcare costs on the rise, policymakers, payers, physicians, and patients are demanding improvement of both costs and quality of care. Increasingly, newer models of provider reimbursement are emerging in both the public and private insurance markets that reward value, rather than volume. In this landscape, it will be important for physicians, both specialists and in primary care, to be able to identify cohorts of patients not meeting recommended quality targets as part of a ‘population management’ strategy.

One example is BP control in CKD, which affects 26 million people in the USA. Controlling BP in these patients can slow progression to end-stage renal disease and reduce cardiovascular morbidity and mortality.^1–7 Despite this well established benefit, recent studies suggest that only 46% of patients with CKD have a BP at goal defined as 130/80 mm Hg, with age, gender, race, and socioeconomic status all associated with the likelihood of adequate control.^8–10 In order to improve rates of BP control in patients with CKD, we need accurate, reliable, and valid data systems to perform two functions that are critical to population management: identify patients needing BP control, and measure rates of BP control.

There are several ways in which clinical performance is currently measured. Some metrics, such as those that are part of the Healthcare Effectiveness Data and Information Set (HEDIS) developed by the National Committee on Quality Assurance, rely chiefly on insurance claims, with limited input from electronic health records (EHRs). Others methods, such as the metrics developed by the American Medical Association's Physician Consortium on Performance Assessment (PCPI) and the Centers for Medicare and Medicaid Services Physician Quality Reporting System (PQRS), rely heavily on special billing codes that indicate a service was delivered. The American College of Surgery's National Surgery Quality Improvement Program (NSQIP) uses manual chart review by trained reviewers. The federal government's Meaningful Use (MU) program will incorporate quality measurement into EHRs.

All of these current methods have advantages and disadvantages, in terms of breadth of data available and the resource intensity of data collection. Insurance claims are clear and structured, but they have a time lag, frequently up to several months. Data must be aggregated from multiple payers, and is usually not made available from Medicare claims. Chart reviews can accurately reveal actual care given, but are expensive and time consuming. Aggregated EHR data may be more up-to-date and accurate, but only if data are in structured fields, rather than free text notes. Even as the MU program rolls out and physicians are incentivized to use structured fields more frequently, they may continue to document in free text as well.

We aimed to develop an accurate method to measure rates of BP control and identify patients not at goal in a population of patients with CKD by integrating elements from many data sources, and overcoming the inherent pitfalls of each alone. Overall project goals included: creating a clinically acceptable definition of BP control, developing an electronic method to identify patients with CKD, measuring performance regularly, and designing and implementing a system to use these reports to conduct patient outreach efforts. Here we report on the first part of this effort: developing a quick, accurate, and reliable system to measure performance and identify patients not at goal.

Methods

Intervention site

Renal Medicine is a division of the Department of Medicine (DOM) at the Brigham and Women's Hospital (BWH), a 793-bed tertiary care academic medical center in Boston, Massachusetts that is part of the larger Partners Healthcare system. The renal division has 22 physicians who see patients in an on-site ambulatory clinic. BWH utilizes an internally developed outpatient EHR that allows physicians to type or dictate visit notes. It contains a structured flow sheet and problem list in which providers can record vital signs and medical history, respectively.

In 2007, the BWH DOM instituted a Quality Program that has been previously described.¹¹ This program's mandate was to engage each clinical division in identifying a clinical improvement project, centered on one or more performance metrics that can be measured electronically. The leadership of the renal medicine division selected BP management in patients with CKD. The staff of the quality program was then charged with developing a method of measuring performance and identifying patients needing better control.

Assessing and monitoring BP control

The project encompassed three key steps:

• Identifying patients with CKD: Because the EHR problem list is not routinely populated, we could not rely on this source to identify patients. We therefore extracted administrative billing data to identify patients based on the International Statistical Classification of Diseases (ICD-9) code for CKD. No attempt was made to isolate a diagnosis of hypertension. Only billing codes entered by nephrologists were considered. The data from the administrative billing database were then linked with clinical data in order to create performance reports. We calculated the number of patients identified through the EHR problem list and billing, and sampled 25 charts to ensure the accuracy of using billing codes.

• Defining and measuring BP control: To match clinical practice and rationale, we developed an algorithm to use several BP readings to determine whether a patient was at goal (table 1). We included BPs taken at any EHR recorded outpatient visit throughout our health system. Goal BP was defined as less than 130/80 mm Hg, which was the recommended level for patients with stages 1–4 CKD when the study was initiated,¹² even though there was some controversy over whether these targets may be too aggressive.¹³ BP readings were extracted from the EHR vital sign flow sheet, with a 2–4 week data lag. Since nephrologists do not consistently use the structured flow sheet to document vital signs, we also used a previously validated natural language processing (NLP) software to extract BPs from free text notes.¹⁴ If multiple BPs were recorded on the same day, we used the reading with the lowest mean arterial pressure (MAP). We compared the measured rate of control using one BP, the algorithm to incorporate multiple BPs, and adding on NLP to capture more readings.

• Point-of-care worksheet: Physicians were given a paper worksheet whenever a patient with CKD whose BP was not at goal according to the new tool presented for a visit (see figure 1). The worksheet was electronically pre-populated with the three most recent BPs from the EHR, and served two purposes. First, it was a reminder to take action to improve BP control. Second, the worksheet also allowed physicians to fill in additional data such as updated BP readings that our electronic systems had not captured, home readings taken by patients, mitigating circumstances, and actions taken. If a physician did not fill out the worksheet, a research assistant conducted a chart review to gather the necessary data.

Table 1.

Algorithm for determining whether a patient with chronic kidney disease is at goal blood pressure (BP) of <130/80

Number of documented days with at least one blood pressure reading in the last 12 months	Logic
0	Not at target
1	At target if the BP is <130/80
2	At target if either BP is <130/80
>2	At target if the last BP is <130/80 OR At target if any two of the most recent three are <130/80

Both the systolic and diastolic BPs needed to be at goal for the entire BP to be at target. If multiple BPs were recorded in the same day, only the systolic and diastolic BPs with the lowest mean arterial pressure were used in the algorithm.

Figure 1.

Blood pressure worksheet for patients with chronic kidney disease.

Results

Identification of patients with CKD

Using ICD-9 codes from billing data, we identified 1377 patients with CKD who had been seen in the renal clinic during a 2-year period, which was greater than the 281 that were identified using the problem list alone. Of the 281 patients with CKD on their problem list, 280 (99.6%) were identified through billing codes. To check the specificity of billing data use, we randomly sampled charts for 25 patients identified through billing data and found that 24 (96%) were correctly identified as having CKD.

Rates of BP control

Table 2 displays the changes in the reported number of patients at target with each enhancement we made. Only 12.3% of patients who were considered controlled according to the algorithm had an elevated BP as their most recent reading (see online supplementary appendix table 4). In only 3% of patients was this most recent BP >140/90.

Table 2.

Rates of measured blood pressure (BP) control with additional data sources

Data source	Percentage of patients at target
Using only the most recent reading in the structured flow sheet	37.1%
Adding algorithm incorporating >1 BP reading	42.3%
Adding natural language processing in free-text notes	52.6%

Point-of-care worksheets

Over a 15-month period, 1276 patients were identified by the tool as not at BP goal who subsequently presented for office visits in the renal clinic. For each of these patients, a worksheet was generated and given to the treating physician—with the support of a research assistant, 95% of the worksheets were completed.

The worksheets revealed several reasons why many patients are not reported at goal (table 3). There were very few patients (2%) for whom physicians thought a BP of <130/80 mm Hg was medically inappropriate. Based on the worksheet, many patients who were not at goal according to the electronic report were either at goal in the office that day (36.2%), or had home readings showing they were at goal (15.8%). Furthermore, the worksheet showed that physicians were often taking appropriate actions to improve BP control, such as intensifying a medication regimen (20.6%) or providing education on lifestyle and behavior (3.8%). In 17.8% of cases, no action was taken nor reason given.

Table 3.

Findings from clinical query at point of care of patients identified as not at goal blood pressure (BP)

Total patients with chronic kidney disease presenting to clinic and not at goal according to electronic algorithm	1276
Updated BP readings
BP controlled at home	194 (15.8)
BP at goal during current visit	443 (36.2)
Actions taken
Medication regimen intensified during current visit	252 (20.6)
Provided patient education on lifestyle interventions	47 (3.8)
Reasons why BP not at target
Patient not taking prescribed medications	46 (3.8)
Target BP is not medically appropriate for this patient	25 (2.0)
Other/no reason given	218 (17.8)

Discussion

Identifying patients and continually measuring performance are critical first steps to effective quality improvement efforts. We report an effort to combine multiple sources of data to create an accurate, clinically meaningful population management tool at a nephrology practice. We found that billing claims, structured data fields from an EHR, free text notes using NLP, and physician worksheets were all needed to accurately assess performance and identify patients. In addition, we found that more data sources are still needed for accurate measurement on a population level, such as incorporating home BP readings.

Our main finding is that using data from multiple sources identifies more patients who are not at goal, and more accurately measures performance. This can overcome many present methods used to assess performance. We improved the accuracy of identifying patients with CKD by using data from billing systems. While CKD was a coded option in the EHR problem list, many providers do not use the structured problem list and instead used a section within their free-text notes to document a past medical history. The diagnoses listed in these notes cannot be reliably extracted electronically. Our choice of billing data to identify patients could be hindered by the accuracy with which physicians select ICD-9 codes at visits. We found, however, a high degree of agreement between ICD-9 codes and chart review. It is possible that some patients with CKD were missed, since we did not review charts of other patients who might have had CKD in order to test the sensitivity of billing codes for diagnosis. Since we were relying on accurate identification to target outreach to physicians (and potentially patients), we were more concerned with ensuring that patients identified in fact had CKD than with the potential exclusion of a limited number of patients.

Others have found that primary care physicians do not reliably identify CKD with billing codes or problem lists, and have used electronic glomerular filtration rates (eGFR) instead.¹⁵ However, nephrologists are much more accustomed to billing for CKD. Furthermore, eGFR can fluctuate, and a low value may not always signify CKD. Evidence of billing for CKD implies that a nephrologist has acknowledged the presence of the condition. One recent study used both methods to capture patients for a CKD registry: either ICD-9 codes from two separate visits or eGFR on two separate occasions. This strategy certainly has merit as well.¹⁶

We improved the accuracy of assessment of BP control by using an algorithm to capture multiple BP readings. National quality metrics that focus on hypertension, including CMS's PQRS and NCQA's HEDIS, use only the latest BP to determine if a patient is controlled.¹⁷ However, several studies have shown that variations in the definition of elevated BP can dramatically alter performance.^18–20 This initiative addresses these concerns by developing an algorithm that utilizes multiple BP measurements from the EHR. Relying on several BP measurements, rather than just one, more closely represents clinical decision making and avoided incorrectly labeling patients as uncontrolled. Prior studies have also shown that manually reviewing EHRs to find free text notes increased measured rates of performance on several clinical quality measures.²¹ While the algorithm may consider patients whose most recent BP reading is elevated to be controlled, this occurred in a minority of cases and usually the BP was only mildly elevated. If the next BP were also to be elevated, the patient could be considered uncontrolled. We used a validated NLP software to capture BPs documented in free text notes, resulting in a significantly higher measured rate of control. Prior research on the software we used has shown high rates of sensitivity, specificity, and precision.^{14 22} To our knowledge, our study is the first published report of the use of NLP to capture BP measurements from free text notes for the purpose of quality improvement.

Despite our goal of developing an electronic method to assess performance, we were limited in that we could not integrate all information necessary to fully assess BP control. We therefore had to use a paper worksheet to allow physicians to document if readings from the current office visit or home readings were normal and to provide other important qualifying data that are not available in the EHR, such as when patients refused medications, were non-adherent, or had medical reasons precluding the aggressive target of <130/80 mm Hg. Using the worksheet, we were also able to determine if a physician had taken appropriate action to address a patient's elevated BP. Prior studies have shown significant increases in measured performance when physicians are given credit for taking appropriate actions to address hypertension and other chronic diseases.^{20 23} The performance measures and chronic disease registries of the future will need to capture as many of these important qualifying data points electronically, to avoid providers having to document them twice.

Another potential limitation to this project is that we were not able to specify where BP measurements were taken, and by whom. Previous studies have reported substantial inter-observer variation in BP measurement.²⁴ However, our EHR does not display the identity of the person taking BPs listed in the flow sheet. In addition, published reports have demonstrated high rates of inaccuracies due to BP cuffs not being appropriately calibrated on a regular basis.²⁵ We do not have data on all the BP cuffs used in our institution, so the degree of inaccuracy cannot be assessed. When multiple BPs are recorded on the same day, some guidelines suggest using the mean BP of the day.⁷ We used the BP with the lowest MAP, since studies have shown that office visits tend to significantly overestimate BPs compared to ambulatory BP monitoring.^{26 27} It is possible that by doing this, we labeled some patients as controlled when in fact their BP was too high, but our method was chosen to most closely reflect what physicians are likely to use when deciding whether to intensify or initiate antihypertensive medications.

Our process has some limitations that may limit wider applicability. It is based at one institution that relies heavily on an EHR. However, with the advent of MU requirements, many health systems have (or will soon have) EHRs. It focuses on a nephrology practice, and may not be completely applicable to a primary care setting, where billing may not be an accurate way to identify patients with CKD. However, combining billing with structured data elements from an EHR may be very relevant to other specialists concerned with a myriad of conditions. It is critical that we find tailored solutions for specialty practices, which have not been as engaged in performance measurement and population management as primary care despite playing a large role in patient care.²⁸

Our project has several implications for the systems we use to capture clinical information. Performance measurement and population management are critically important components of healthcare delivery improvement. Our experience suggests that in order to be successful, we will need to gather data from multiple sources, in more robust and comprehensive ways than we do currently. The emphasis on structured fields in the MU program will help, but will not be sufficient. MU requires that at least some problems, medications, vital signs, and allergies be structured.²⁹ However, MU does not guarantee that all needed information will be in structured fields, and does not prevent providers from entering important qualifying data in free text. NLP will help capture free text data, but may not able to identify all of the nuances of care that are needed to classify patients as well treated. As we discovered through this project, we will also need to capture data from patients, such as home BP readings and medication adherence, home sugar readings for patients with diabetes, and home weights for patients with congestive heart failure. Electronic systems will need to be designed to capture both provider and patient-reported information as close to real-time as possible, and make it queryable and reportable.

BP control in patients with CKD and other chronic diseases is critical to improving outcomes, and data show that we can do much better. By leveraging existing data systems that many providers have or will soon have, it is possible to create solutions to many common measurement challenges. Still, important gaps remain that must be bridged in order to have accurate, usable systems to measure performance and conduct population management.