Making Audit Actionable: An Example Algorithm for Blood Pressure Management in Chronic Kidney Disease

Abstract

Despite widespread use of clinical guidelines, actual care often falls short of ideal standards. Electronic health records (EHR) can be analyzed to provide information on how to improve care, but this is seldom done in sufficient detail to guide specific action. We developed an algorithm to provide practical, actionable information for care quality improvement using blood pressure (BP) management in chronic kidney disease (CKD) as an exemplar. We used UK clinical guidelines and EHR data from 440 patients in Salford (UK) to develop the algorithm. We then applied it to 532,409 individual patient records, identifying 11,097 CKD patients, 3,766 (34%) of which showed room for improvement in their care: either through medication optimization or better BP monitoring. Manual record reviews to evaluate accuracy indicated a positive-predictive value of 90%. Such algorithms could help improve the management of chronic conditions by providing the missing link between clinical audit and decision support.

Introduction

Best practice clinical guidelines are widely used in health systems around the world, however, observed quality of care often falls short of these standards.¹ The gap between ideal and actual care often results in additional morbidity, mortality and preventable hospital admissions, each with human and financial costs.

Chronic kidney disease (CKD) is a prime example. In CKD, both declining kidney function (measured by estimated glomerular filtration rate: eGFR) and increasing urinary albumin:creatinine ratio (ACR) independently increase cardiovascular mortality risk 2–4 fold.² Clinical guidelines therefore recommend strict blood pressure (BP) control to reduce this risk.^3–8 Nevertheless, studies invariably demonstrate just 13–66% of CKD patients in the US^9–13 and 35–60% in Europe^14–19 actually achieve controlled BP levels. Not meeting these standards has substantial adverse consequences given the high global prevalence of CKD, estimated at 8–16%.²⁰

Data from electronic health records (EHRs) are abundant and ever increasing. They reflect the real-world care that patients receive and are often compared against clinical guidelines for the purposes of audit.²¹ EHRs also often capture reasons why patients may not have achieved quality standards in the first place (e.g. patient choice or contraindication to treatment) and practical steps as to how they might be achieved in future (e.g. how to optimize current management). This information is rarely exploited by existing informatics tools that perform audit and feedback – both in experimental studies and in routine practice.^{11,14,24–28} Feeding such information back to practitioners has been shown in meta-analyses to produce greater improvements in patient outcomes than simple audit because it is more actionable by practitioners.^22,23 Therefore adding this functionality to audit and feedback interventions is likely to lead to greater improvements in patient care by connecting it with consequent quality improvement actions.

Aim of this study

The aim of this study was to develop and estimate the accuracy of an algorithm that searches EHR data to feed back practical, actionable information to clinicians regarding how they could improve care for patients in accordance with clinical guidelines. We used the example of BP management in CKD to test the feasibility of this method.

Importance and relevance of this study

This algorithm could provide the missing link between clinical audit and decision support, thus reducing the latency between clinical quality information and actions to improve care. Focusing on CKD and BP management, the algorithm developed here can be applied directly to EHR data to help reduce adverse cardiovascular outcomes. The approach in general can also be abstracted to other chronic conditions where physiological parameters reflect, at least in part, the quality of care provided to patients (e.g. cholesterol in cardiovascular disease; glycated hemoglobin in diabetes).

Methods

Algorithm development

We used anonymized EHR data from the City of Salford (population 234k) in the UK to develop the algorithm. Since the early 1990s, most UK citizen’s primary healthcare records have been stored as machine-readable clinical codes (mainly Read codes v2 and CTV3). Over 300k different entries exist, covering care-processes, diagnoses and medications.²⁹ Practitioners can enter narrative free-text to supplement certain Read codes, though it is the codes themselves that are used for official purposes in the UK National Health Service (NHS) such as provider payment, public health programmes, health service planning, population health data, clinical performance assessment, and research.³⁰ Read code collections therefore provide important insights into the UK population’s health.

We extracted alphanumeric codes and associated rubrics from a central database of all patients that received care in Salford from 2001–2012. These were then transferred securely to The University of Manchester as text-based vector files for analysis in Microsoft SQL Server 2008. The Salford database has collated EHR data daily from 53 primary care providers and one secondary care provider since 2001. We designed the algorithm to make inferences about how to improve BP management for CKD patients using coded EHR data in the following sequence: 1) identify patients with CKD; 2) identify those from Step 1 in whom it was appropriate to pursue quality standards for BP management; 3) identify those from Step 2 who did not achieve the quality standards; 4) provide information on what action could be taken to enable patients from Step 3 to achieve the quality standards.

In accordance with accepted methodology,³¹ we developed and tested the algorithm iteratively. Initial algorithms were written as narrative based on clinical guidelines and authors’ (BB and TF) clinical knowledge. The narrative was then translated into machine readable form using existing codes within the EHR dataset and dictionaries of Read codes used for primary care performance related pay.³² To optimize the performance of the algorithm, we calibrated it with manual clinician review. Two clinicians (BB and TF) manually reviewed the coded data of randomly selected anonymized EHRs of patients receiving care in Salford between 2010 and 2012. For each of the 4 steps in the algorithm sequence, batches of 20 records were reviewed (10 identified positive and 10 identified negative). Based on these findings, the algorithm programming logic was modified as necessary. When additional codes were added to the algorithm, synonyms were identified within the full dataset (2001–2012) and also included. Further reviews of records were undertaken until the algorithm correctly identified all patients in the batch.

Algorithm accuracy study

Participants

We applied the final algorithm to the whole Salford EHR dataset (2001–2012) to make inferences about how to improve the quality of CKD BP management in Salford between 2011 and 2012.

Test methods

To test the performance of the algorithm, we randomly selected patient records it identified that had not been used in the development phase to calculate the positive predictive value (PPV). We chose to use PPV for: 1) clinical importance – in its real-world application, each patient identified by the algorithm will require further action by a clinician, so achieving a high PPV is essential to prevent over-burdening practitioners; and 2) pragmatism – to adequately assess specificity and sensitivity would require the manual review of thousands of records for each stage in the algorithm sequence, for which resources were not available. We defined the PPV as the number of patients identified by the algorithm where the clinicians (BB and TF) deemed they had CKD and that steps could be taken to improve the management of their BP, divided by the total number of records identified by the algorithm. For each patient tested, both BB and TF independently reviewed the anonymized coded data from their EHR and made a clinical judgment in accordance with national UK CKD clinical guidelines.⁴ Any disagreements were resolved through discussion.

Statistical methods

We used a worst-case scenario of a predicted 50% PPV to determine the necessary sample size to test the algorithm (though we anticipated the algorithm would perform much better). We calculated that to detect a PPV of 50+/-10% with a two-sided α of 0.05, a minimum sample size of 96 patient records were required. We undertook reviews of 100 records to account for the imperfect nature of the review process. Statistical analysis was performed in R.³³

Results

Algorithm development

In total, we manually reviewed 440 different patient records over 22 iterative cycles during the development process. Between 2001 and 2012, we found over 165 million individual codes in the Salford data, of which 151 million (92%) were Read code v2 and 14 million (8%) were codes specific to commercial EHR information systems. Most codes (133 million, 81%) originated in primary care and 32 million (19%) originated in secondary care. Often the rubric associated with codes was re-written by clinicians, which added a further layer of complexity when searching for synonyms. We used only the primary care data, as it is largely the responsibility of primary care providers in the UK to manage BP in CKD patients. The narrative description of the final algorithm is in Table 1. The full algorithm and list of codes used is available from www.clinicalcodes.org.

Table 1.

Narrative description of the fully developed algorithm sequence.

Step	Criteria
1	Patients with CKD • Include patients prior to the study period with:   ○ a CKD diagnosis code.   ○ a most recent mean eGFR<60 or ACR≥30 or PCR≥50 over a >90 day period. • Exclude patients with a death code at any time.
2	Patients in whom it is appropriate to achieve quality standards for BP management • Exclude patients with:   ○ a palliative care code or an orthostatic hypotension code at any time.   ○ an ‘informed dissent’ or ‘unsuitability’ code for CKD or BP quality indicators within 15 months of the study period.   ○ a ‘maximal BP therapy’ code within 15 months of the study period, or who were receiving 4 types of anti-hypertensive medication concurrently (or fewer if they were unable to receive 4 types due to allergies and/or contraindications), at the time of their latest BP measurement.
3	Patients who have not achieved BP management quality standards • Exclude patients with a DM diagnostic code prior to the study period:   ○ and microalbuminuria (defined as a microalbuminuria code at any time or most recent ACR≥3.5 [females] or ≥2.5 [males]) or proteinuria (defined as a proteinuria code at any time ACR≥70 or PCR≥100), whose latest BP is <130/80 and within 9 months of the study period end.4,34   ○ but no microalbuminuria or proteinuria (as defined above), whose latest BP is <140/80 and within 12 months of the study period end.35 • Exclude patients without a DM code and without proteinuria (as defined above) prior to the study period, whose latest BP is <140/90 and:   ○ within 12 months of the study period end if a hypertension or CVD diagnostic code is present prior to the study period.36–40   ○ or within 15 months of the study period end otherwise.4,32
4	Patients whose care could have been improved to achieve these quality standards • Count the number of direct and indirect contacts* each patient had with the primary care health system since their last BP measurement or during the period of interest defined in Step 3 (whichever is latest). • Count how many patients with a high BP reading whose:   ○ Last BP reading is within 2 months of the study period end.   ○ Medication did not change.   ○ Medication changed.

Key: ACR: albumin:creatinine ratio (mg/mmol); BP: blood pressure (mmHg); CKD: chronic kidney disease; CVD: cardiovascular disease (ischaemic heart disease, peripheral arterial disease, stroke, or transient ischaemic attack); eGFR: estimated glomerular filtration rate (ml/min/1.73m²); PCR: protein:creatinine ratio (mg/mmol).

^*Direct contact: face-to-face or telephone contact; indirect contact: medication prescription or administrative activity where the EHR has been accessed.

As we used UK EHR data, we developed the algorithm based on clinical guidelines and standards issued by the National Institute for Health and Care Excellence (NICE)⁴¹ – the authority for clinical guidelines in the UK. To determine whether a patient had a CKD diagnosis according to biochemical parameters in the guidelines (rather than simply a diagnostic code), we took an average of their most recent readings over a period of more than 3 months to account for any fluctuations, and by discounting readings of zero related to eGFR. When the NICE CKD guidelines did not cater for certain cohorts of patients (e.g. those with diabetes but without microalbuminuria, or those without diabetes), we identified appropriate clinical standards from other related NICE guidelines. Interestingly, we could not find NICE guidance on how often CKD patients without hypertension, diabetes or cardiovascular disease should have their BP measured, so we used the standard of 15 months in the UK Quality and Outcomes Framework.³² We discounted patients from these quality standards (Step 2) based on our clinical experience and on rules used in the Quality and Outcomes Framework.³²

We classified anti-hypertensive treatment according to the 6 groups in NICE guidelines: (1) ace-inhibitors and angiotensin-receptor blockers; (2) calcium channel blockers; (3) beta blockers; (4) thiazide-like diuretics; (5) other diuretics; (6) alpha-blockers. When a patient had a high BP reading, we ascertained whether there had been a change in medication by analysing the medication prescribed 63 days either side of the reading to account for bi-monthly prescriptions. We defined whether or not a patient’s treatment could be optimised based on the number of medication classes they were prescribed, taking into account their allergies and other contraindications: beta-blockers should not be prescribed to asthmatics; beta blockers and calcium-channel blockers should not be prescribed in moderate-severe aortic stenosis (unless surgically repaired); thiazide-like diuretics should not be prescribed where the latest eGFR<30 ml/min/1.73m²; diuretics should not be prescribed if the most recent serum sodium<130 mmol or potassium<2.5 mmol; all medications should be avoided in pregnancy except for labetalol or methyldopa; ace-inhibitors and angiotensin-receptor blockers should be prescribed if the most recent serum sodium <130 mmol or renovascular disease is present (unless surgically fixed). We defined maximal anti-hypertensive therapy as being prescribed four concurrent classes of medication,⁴ or fewer if patients were unable to receive additional medication due to allergies or the above contraindications.

To identify whether patients had on-going contact with the health system, representing opportunities to improve their care, we classified contacts on any given date as follows: the presence of a code denoting direct contact with a patient (e.g. ‘home visit’, ‘medication review with patient’), examination, symptom, history, diagnosis, or telephone contact counted as a ‘direct contact’; everything else was classified as an ‘indirect contact’ including ‘failed encounters’, issuing of medication, or administrative activities such as reviewing letters from the hospital.

We categorized patients with high BP measurements needing repeat measurement as those who experienced a change in anti-hypertensive medication following the high reading. We also categorized those who had their high reading within 2 months of the final date of the data extract as requiring a repeat measurement, as this was considered a clinically reasonable period to re-check BP after advising lifestyle changes and therefore did not necessarily indicate sub-optimal management.

Algorithm accuracy study

Participants

We applied the final algorithm to 532,409 individual patient records from the whole Salford dataset to assess the quality of BP management in CKD patients between 2011 and 2012. Please see Figure 1 for a detailed breakdown of results. A total of 11,097 patients had CKD: 8,606 (72%) determined by diagnostic code and 2,491 (21%) determined by biochemical parameters. Of these, 1,435 were excluded from the quality standards: 1,226 (85%) due to the presence of a relevant exclusion code and 229 (15%) due to inferences that they were already prescribed maximal anti-hypertensive medication. Following their removal, over a half the remaining patients (4,940 out of 9,662; 51%) did not achieve quality standards for BP management: 2,072 did not receive a BP measurement and 2,868 had a BP measurement above target. Due to word count limitations, clinical and demographic characteristics of the study population are not presented here but are available on request from the authors.

Figure 1.

Flow of patients identified by the algorithm.

Key: BP: blood pressure (mmHg); CKD: chronic kidney disease; Rx: medication prescription; shaded areas represent actionable care improvement information for practitioners.

Test results

Of the 2,072 without a BP measurement, 595 (29%) had ongoing direct contact with primary care, which suggests that their BP could be measured at their next visit. This could be used to create a reminder system for providers prior to the patient’s next consultation. A further 64 (3%) had ongoing indirect contacts with primary care, suggesting they could be invited by letter, telephone or text message to arrange a BP measurement. This could be used to automatically send letters or text messages to these patients, or present providers with a list of the patients to telephone. Over two-thirds of this cohort (1,413; 68%) had no contact with primary care. This group is more difficult to draw conclusions about. They may require a different approach than simply inviting by letter, telephone or text message; this may be best guided by local provider knowledge. Of the 2,868 with a BP measurement above target, 1329 (46%) patients did not receive any change in anti-hypertensive medication and may therefore benefit from optimization of their treatment. This could be used to create an alert system for providers to change these patients’ medications; they could be contacted via the appropriate methods described above to implement these changes. A further 365 (13%) experienced changes to their medication regime (which may have included patient non-adherence) but did not have a repeat BP measurement to check the consequences of the change. Of these patients, 305 (84%) and 56 (15%) of patients had ongoing direct and indirect contact with primary care respectively, which represented missed opportunities. This information could again be used as a basis for provider reminders, lists or automatic methods of communication to patients to facilitate BP monitoring. Finally, 1,174 (41%) had their BP measured in the last 2 months of the study period, who also require repeat measurements but may not necessarily represent sub-optimal care. This information could be used as described above when the 2-month period has expired, but in the meantime should not be used as an indicator of suboptimal care.

Estimates

3,766 patients were considered ‘positive’ by the algorithm (i.e. did not receive care in accordance with best practice but whose care could be improved upon). On manually reviewing the coded information in 100 randomly selected full patient records from 35 different primary care providers within this cohort (not used in the development of the algorithm), clinicians agreed that 90 were truly ‘positive’. No cases needed further discussion. The 10 false positives were deemed to have been incorrectly classified as CKD patients by the algorithm.

Discussion

Summary of findings

This study developed and estimated the accuracy of an algorithm that can search EHR data to feed back practical, actionable information to clinicians regarding how they could improve care for patients in accordance with clinical guidelines. We used BP management in CKD patients as an exemplar and achieved a PPV of 90% (95% CI 82%-95%). Following the application of our algorithm we found 4,490 (51%) CKD patients did not achieve BP quality standards, and 3,766^† (76%) of these patients had obvious room for improvement in their care – either by optimizing medication or taking opportunities to monitor BP. This information has greater utility than current informatics tools that provide audit and feedback using EHR data, because it provides actionable detail for practitioners to follow rather than simply reporting proportions of patients achieving standards of care^{11,14,24–28}. The digital artefacts of this kind of retrospective, continuous analysis could provide the missing link between clinical audit and decision support – reducing the latency between clinical quality information and practitioners’ improvement actions/tactics for patients with long-term conditions.

Strengths and weaknesses of this study

A major strength of this study is the thorough process through which the algorithm rules were developed. We based the rules on widely accepted clinical guidance, and used our clinical experience to ensure that the algorithm assessed care quality according to both robust research evidence and the clinical service context. These were further refined to ensure their accuracy by manually reviewing 440 different patient records. In addition, we used the totality of patient records to gain a representative picture of how care is delivered across the local population.

A weakness of this study is our reliance on the coded data within the EHRs to validate the algorithm. We were unable to gain access to the full patient EHR, and it is possible that free-text within the records not reflected in the coded data would detail reasons why some of the patients identified by the algorithm did not achieve the BP quality standards. However, many official processes in UK primary care such as health service planning and pay-for-performance rely on coded data, so any important information not entered as codes into EHRs is likely to be minimal.

We did not calculate the negative predictive value (NPV) of the algorithm due to restrictions on the number of patient records we could reasonably review manually. The prevalence of CKD patients that could have their BP management improved is relatively low in the general population and would therefore require large sample sizes to achieve an acceptably narrow confidence interval. However, as this algorithm is intended as a screening tool to lead to further investigation of patient records, it is arguable that a high PPV is the most important statistic to avoid overburdening clinicians with false positive results.

Comparison with existing literature

The algorithm we developed addresses a need identified in the literature that audit and feedback should provide information to clinicians that is actionable, in order to drive effective improvements in patient care.^22,23 Few informatics systems currently use EHR data in an automated fashion to provide audit and feedback in this way,^{11,14,24–28} making this algorithm a relatively unique quality improvement tool.

Our finding that 49% of UK CKD patients received NICE quality standards regarding BP management is in broad agreement with other studies. De Lusignan et al. found this proportion to be 48–50%,¹⁴ whereas Fraser et al. found it to be 58%.¹⁶ This agreement suggests our algorithm has external validity.

The PPV of our algorithm is higher than others that use EHR data reported in the literature. Algorithms developed by Singh et al. to detect diagnostic errors in primary care achieved a PPV of between 5–21%;⁴² Murphy et al. developed algorithms to detect delayed cancer diagnoses and reported PPVs of 58–70%;⁴³ and Brenner et al. reported a PPV of 15% in their algorithm to identify adverse drug events.⁴⁴ These discrepancies are likely explained by differing prevalence and ease of detection from EHR data of the events under examination. Furthermore, because we were unable to view the free-text within the patient records, the true PPV of our algorithm may be lower than 90%.

Implications for clinical practice and research

The algorithm developed in this study can form the basis of an audit and feedback tool to improve BP care for patients with CKD. Clinical performance systems that provide practitioners with actionable information on how to improve care, such as this, are more effective than those that simply provide proportions of patients meeting quality standards.^22,23 Thus the algorithm could enable practitioners to reduce cardiovascular morbidity and mortality – and as nearly all UK primary care providers use EHRs that capture Read codes, it could be rapidly deployed.

By relating care quality analyses to a practitioner’s individual patients in this way, such a system may also improve clinical coding. This effect may reach beyond the specific codes concerned to a more general increase in coding accuracy as the gap between care quality information and improvement-actions is closed.

This study has tested the feasibility of a method that draws clinical process and outcome measurement closer together, which is likely to improve BP control and CKD outcomes. Our approach could be abstracted to other chronic conditions that use both processes and outcomes, which include (but are not limited to) other longitudinal physiological parameters (e.g. cholesterol, glycated hemoglobin).⁴⁵ The method could also be applied to other healthcare settings that employ different coding languages in their EHRs, such as hospitals and different countries.

Future work

Our future work will initially focus on improving the performance of the algorithm (particularly with regard to the diagnosis of CKD) and assessing the accuracy of the actionable information it provides. We aim to compare this with manual reviews of full patient records that include free-text. We also hope to improve the functionality of the algorithm to identify total daily doses of medication to provide more specific advice on how treatment could be optimized, and to assess medication adherence. Ultimately our vision is to use this algorithm to form the basis of an audit and feedback informatics tool by integrating it into software previously developed by our group.⁴⁶ We anticipate that the tool could be linked to patients’ EHRs and provide clear and succinct messages to providers on how to improve care, taking into account comorbidities, allergies and other relevant patient characteristics. We believe that local clinical ownership of quality improvement projects is fundamental to success, but that clinicians may not have the expertise to develop and implement the informatics tools necessary. We will therefore work with local clinicians to understand the needs of their populations and ensure there is clinical engagement with our work. We are also mindful that implementing such tools into practice must align with existing clinical workflows to be effective and efficient. Therefore our approach to development and implementation of the audit and feedback tool will be iterative, and employ mixed methods techniques informed by relevant theoretical frameworks.⁴⁷ The effectiveness of the tool will eventually be tested in a randomized clinical trial.

Conclusion

We have developed an algorithm that uses EHR data to provide practical, actionable information to clinicians on how to improve care – a ‘rear view mirror’ for long-term conditions management. We used BP management in CKD patients as an exemplar and achieved a useful PPV of 90%. The utility of this information might be improved by mining the related EHR narrative, and by instrumenting clinical coding behavior. In future work we will develop this approach further in variety of conditions and settings, measuring its effects on clinical behavior and patient outcomes.