Impact of health information technology interventions to improve medication laboratory monitoring for ambulatory patients: a systematic review

Abstract

Medication errors are a major source of morbidity and mortality. Inadequate laboratory monitoring of high-risk medications after initial prescription is a medical error that contributes to preventable adverse drug events. Health information technology (HIT)-based clinical decision support may improve patient safety by improving the laboratory monitoring of high-risk medications, but the effectiveness of such interventions is unclear. Therefore, the authors conducted a systematic review to identify studies that evaluate the independent effect of HIT interventions on improving laboratory monitoring for high-risk medications in the ambulatory setting using a Medline search from January 1, 1980 through January 1, 2009 and a manual review of relevant bibliographies. All anticoagulation monitoring studies were excluded. Eight articles met the inclusion criteria, including six randomized controlled trials and two pre–post intervention studies. Six of the studies were conducted in two large, integrated healthcare delivery systems in the USA. Overall, five of the eight studies reported statistically significant, but small, improvements in laboratory monitoring; only half of the randomized controlled trials reported statistically significant improvements. Studies that found no improvement were more likely to have used analytic strategies that addressed clustering and confounding. Whether HIT improves laboratory monitoring of certain high-risk medications for ambulatory patients remains unclear, and further research is needed to clarify this important question.

Introduction

Since the Institute of Medicine highlighted the impact of medical errors on patient morbidity and mortality in To Err is Human,¹ significant effort has focused on reducing medical errors and improving patient safety in the USA. Medical errors result in 44 000–98 000 deaths per year, a large proportion of which are due to adverse drug events (ADEs).¹ Laboratory monitoring errors are a major cause of potential ADEs, occurring in 60.8% of preventable ADEs in ambulatory older adults² and in 45.4% of preventable ADEs requiring hospital admission.³ Baseline monitoring rates are low, with up to 58% of initial drug dispensings occurring without appropriate laboratory monitoring for ambulatory older adults.⁴ Because patients sometimes miss more than one test for a given drug and often take many drugs, the rate of all potential laboratory-monitoring errors was estimated to be extremely high (∼80%) among patients taking chronic medications in 2001.⁵ Because poor adherence to guidelines leads to hospitalizations and significant morbidity,^{3 6} and because basic human factors make it challenging for clinicians to adhere to complicated monitoring recommendations for a large number of medications, health information technology (HIT) holds promise for improving laboratory monitoring of high-risk medications and may potentially reduce medication errors.^{7 8}

Some experts estimate that up to 95% of potential ADEs can be avoided with the adoption of advanced computerized systems.⁸ As a result, tools to reduce errors continue to be developed, many of which are technology based. However, the actual impact of these systems is unclear. Technology and clinical decision support (CDS) systems have been shown to improve patient care and clinical outcomes in many clinical situations.⁹ For example, computerized physician order entry with decision support can reduce medication errors,¹⁰ and interventions to improve laboratory monitoring in the hospital setting can improve outcomes.¹¹ Furthermore, computer access to laboratory data improves the opportunity for pharmacists to monitor medications,^{12 13} and systematic reviews of interventions to improve monitoring in the hospital setting show that HIT can reduce errors.^{10 14–16} Unfortunately, it remains unclear whether HIT CDS alerts in the ambulatory setting are as effective.

To address this gap in the literature, we conducted a systematic review to identify studies that evaluated HIT interventions to improve laboratory monitoring of selected high-risk medications in the ambulatory setting. The specific aims of this review are to answer the following questions regarding high-risk medications (excluding anticoagulants) in the ambulatory setting. (1) Do HIT interventions improve laboratory monitoring? (2) What are the characteristics of HIT interventions that improve monitoring? This review should inform the planning of laboratory monitoring interventions and guide future researchers about important research design elements for HIT interventions.

Methods

Literature search

To identify journal articles for this systematic review, we performed a Medline search of English-language human studies published between January 1, 1980 and January 1, 2009 using keywords for HIT and drug monitoring. The search performed was as follows: ‘drug monitoring’ OR ‘laboratory monitoring’) AND (computerized OR electronic OR informatics OR reminder systems OR ‘Medical Records Systems, Computerized’ (MeSH) OR ‘Decision Support Systems, Clinical’ (MeSH) OR ‘Decision Making, Computer-Assisted’ (MeSH) OR ‘Database Management Systems’ (MeSH). We performed a manual review of relevant authors and journals including bibliographies from identified articles.

Inclusion criteria and selection of studies

We included studies that: were clinical trials, randomized controlled trials (RCTs), or comparative studies; were conducted in an ambulatory setting; had sufficient information about the HIT intervention for it to be assessed separately from other non-HIT interventions; and examined laboratory test monitoring rather than clinical tests (eg, pulmonary function tests). We included studies evaluating the effect of HIT interventions on laboratory test monitoring, defined as laboratory tests to evaluate efficacy, toxicity, or side effects. We excluded studies that examined laboratory testing to evaluate medication adherence, computerized order interventions that did not include laboratory monitoring, at-home patient testing, in-hospital interventions, and literature reviews, meta-analyses, case studies, and opinion pieces. We also excluded studies in which the HIT monitoring intervention was coupled with other interventions (eg, HIT-based medication dosing and appointment scheduling recommendations) because it was not possible to identify the independent effect of HIT on laboratory monitoring; this included all studies evaluating anticoagulation interventions and several multipronged diabetes interventions. Additional studies identified from bibliographies and author searches were also evaluated for inclusion based on the same criteria.

The literature search produced 347 abstracts, of which 314 were in English and published from January 1, 1980 to January 1, 2009 (figure 1). Each study was assessed independently by two investigators (SHF and JT) for inclusion. Disagreements were resolved by consensus. There were 256 studies after exclusion of review articles, and most were excluded after manual review for not meeting the inclusion criteria. Many studies were excluded for more than one reason, such as not being an actual trial and covering a topic other than laboratory monitoring.

Figure 1.

Flow diagram of included and excluded studies. HIT, health information technology.

Data abstraction and evaluation

We extracted data from the text and tables of the original publications and classified by clinical setting, targeted medications, time frame, HIT intervention type and duration, randomization, comparison group, and endpoint assessed. In one case, investigators contacted a study author for additional results.

Quality scores were assigned by two investigators (SHF and JT) using an approach outlined by Downs and Black¹⁷ to assess methodological quality. This approach standardizes and rates important aspects of study design and data presentation to assign an overall study quality rating score. The maximum score possible for an original investigation was 27. Disagreements were reconciled by consensus.

Results

A detailed review of the potentially eligible articles identified a total of eight articles for inclusion that evaluated the impact of HIT interventions on laboratory monitoring in the ambulatory settings published between 2003 and 2009. A brief description of these studies is presented in table 1.

Table 1.

Characteristics of studies included in the systematic review

Author	Date published	Study type	Intervention	Study location	Sample size	Score	Unit of analysis	Confounding and clustering in analysis	Outcome measured	Results (+: statistically significant change in monitoring rate / –: no effect of the intervention on the monitoring rate)
Feldstein18	2006	RCT–cluster–randomized by clinic	EMR reminder via email as well as two other interventions	OR: Kaiser Permanente HMO, 15 primary care clinics	44 PCPs with 196 patients in EMR arm	25	Patient	Yes	Completed monitoring	+: EMR reminder ↑ baseline laboratory monitoring of 10 medications, from 22.4% to 48.5%: 26.1% absolute ↑/116% relative ↑; HR of 2.5, but less effective than voice message or pharmacy outreach
Hoch19	2003	Pre–post no control	EMR reminder via email	Israel: HMO	504 physicians	18	Patient	No	Completed monitoring	+: Reminders to clinicians ↑ potassium testing (78.5→81.5%; 3.0% absolute effect; 9.8% relative; p<0.001)
Lo20	2009	RCT	EMR reminder	MA: Partners HealthCare	22 primary care clinics; 3673 events among 2765 patients	23	Clinic visit	Yes	Ordering	–: Reminders did not improve ordering of laboratory monitoring significantly
Matheny6	2008	RCT	EMR reminder	MA: Partners HealthCare	1922 patients seen by 303 physicians in 2507 clinic visits	24	Clinic visit	Yes	Completed monitoring	–: Reminders did not improve laboratory monitoring significantly
Palen21	2006	RCT	EMR reminder	CO: Kaiser HMO	207 PCPs with6 20 104 in the intervention arm caring for 26586 patients	22	Dispensing (first)	No	Completed monitoring	–: Reminders did not improve laboratory monitoring significantly. Significant improvement for selected medications.
Raebel22	2005	RCT	Pharmacists reminded electronically about missing tests and then ordered them and reminded patients	CO: Kaiser HMO	10169 drug dispensings for 9565 patients	23	Dispensing (each unique initial drug dispensing)	No	Completed monitoring	+: Statistically significant ↑ monitoring in the intervention group, varying widely by medication (70.2→79.1% overall; 8.9% absolute effect; 12.6% relative; p<0.001)
Raebel23	2006	RCT	Pharmacists reminded electronically about missing tests and then ordered them and reminded patients	CO: Kaiser HMO	9139 patients with 4871 patient–drug combinations	22	Patient–drug combination (ongoing therapy)	No	Completed monitoring	+: Statistically significant improved monitoring in the intervention group for only some of the medications (58→64% overall; 6% absolute effect; 10% relative; p<0.001)
Steele24	2005	Pre–post, no control	EMR reminder	CO: Safety net outpatient clinics	Rule processed 16291 times; 19076 patients seen during the time period	16	Orders	No	Ordering	+: Increased ordering of the rule-associated laboratory test when an alert was displayed (39→51%; 12% absolute effect; 31% relative; p < 0.001)

↑, increase; CO, Colorado; EMR, electronic medical record; HMO, health maintenance organization; HR, hazard ratio; MA, Massachusetts; OR, Oregon; PCP, primary care physician; RCT, randomized controlled trial.

Seven studies were conducted in the USA^{6 18 20–24} and one in Israel.¹⁹ Six of the eight studies were conducted in large, integrated healthcare delivery systems,^{6 18 20–23} including a series of studies by Raebel et al at Kaiser Permanente^{18 21–23} and two studies at Partners HealthCare.^{6 20} Five interventions sent electronic alerts to prescribing physicians alone.^{6 19–21 24} Three sent electronic alerts to a pharmacist who could then order the laboratory test and contact the patient.^{18 22 23} One of the three studies that involved pharmacists also included a comparison arm of computerized alerts to physicians only.¹⁸ Seven studies targeted a broad range of medications,^{6 18 20–24} while the eighth targeted a single medication.¹⁹ Six studies evaluated completion of laboratory test monitoring as the outcome measure,^{6 18 19 21–23} while two evaluated physician test ordering.^{20 24} A meta-analysis of the data reported was deemed inappropriate because of the differences between the studies.

Five of the eight studies reported statistically significant improvements in laboratory monitoring attributable to the study intervention,^{18 19 22–24} whether an improvement in appropriate tests ordered or an increase in the completion rate, with the absolute per cent improvement ranging from 3.0% to 26.1%. There was no consistent pattern of intervention efficacy based on outcome measurement. The number of patients enrolled in each study ranged from 196 to 26 586. The smallest study showed the largest absolute improvement in monitoring.¹⁸

Study quality and impact on laboratory monitoring

Six of the eight studies were RCTs, while two were pre–post intervention studies. A brief description of the study methodologies and quality rating score is included in table 1. The study quality rating scores ranged from 16 to 25 (possible score range 0–27). The RCTs were rated higher (quality score=22–25) than the pre–post intervention studies (quality score=16–18). Studies with the highest scores differed from lower quality studies in their analytic approaches by including adjustment for confounding and clustering.^{6 18 20} Interestingly, randomization failed in two of the highest quality studies,^{6 20} where the intervention and control groups were significantly different on key clinical characteristics such as gender, race, and insurance type.

Both pre–post studies showed statistically significant improvements,^{19 24} while only three of the six RCTs did.^{18 22 23} All of the RCTs that showed improvements involved pharmacist-based interventions; this included the only RCT that showed improvement by an alert targeting physicians, and this intervention was evaluated as the comparison arm for more intensive pharmacist-based intervention.¹⁸

All studies enrolled patients nested within providers; two multi-site studies were cluster randomized trials at the level of the clinic, nesting providers within each site.^{18 20} Three studies accounted for clustering at the level of the clinic or provider in the analyses or design,^{6 18 20} and two of these reported no improvements in monitoring with HIT intervention.^{6 20} While all studies listed some possible patient-level or facility-level confounders, only the same three studies adjusted for these possible confounders in their analyses,^{6 18 20} and two of these studies showed no intervention improvements.^{6 20} In addition, of the six RCTs, the three with failures in randomization reported no improvement in monitoring, after any adjustment.^{6 20 21}

Study site characteristics and impact on laboratory monitoring

Six of the eight studies were conducted in one of two large integrated healthcare delivery systems, Kaiser Permanente and Partners HealthCare; these included all of the RCTs.^{6 18 20–23} One study was conducted in a safety-net clinic,²⁴ and one study in multiple health maintenance organization sites in Israel.¹⁹ Baseline rates of appropriate laboratory monitoring varied between study sites, ranging from 14%⁶ to greater than 95%²² depending on the study drug. Sites with lower baseline rates of monitoring reported greater improvements associated with HIT interventions.^{18 23 24} Both studies from Partners HealthCare showed no improvements with HIT interventions, but had high baseline rates of monitoring prior to the intervention.^{6 20} The safety-net clinic study and the Israeli health maintenance organization study had different baseline levels (38.5% and 78.5%), but both showed significant monitoring improvements in their pre–post intervention assessments.^{19 24}

Intervention design and impact on laboratory monitoring

All the studies were conducted within healthcare systems with electronic records. Four interventions were based on homegrown electronic medical records programs,^{6 19–21} while four were based on modifications to proprietary systems.^{18 22–24} Six of the eight interventions were built-in systems with computerized physician order entry (CPOE), with the alert going to the physician. Of these, two sent messages via email,^{18 19} while four provided alerts during patient profile reviews.^{6 20 21 24}

Of the CPOE interventions, alerts within the electronic record system were either interruptive (requiring the provider to respond to the alert) or non-interruptive (not requiring action). In one study, the intervention was interruptive and required action on the part of the provider to dismiss an alert²⁴; however, this intervention did not shorten the process of test or medication ordering. Other studies had real-time alerts that appeared on the prescribing page as a warning, but they were non-interruptive and did not stop the workflow.^{6 20 21} No aspect of the CPOE design itself was found to be consistently more effective than any other. Interestingly, the two studies that alerted pharmacists directly, but not physicians, demonstrated significant improvements in monitoring.^{22 23}

Most interventions reviewed targeted multiple high-risk medications, while one involved only a single drug.¹⁹ When we examined the impact of interventions on the same drug, diuretics, across all the studies, we found no significant effect of the HIT intervention except in the study for which this was the only drug targeted¹⁹ (table 2). There was no consistency between the medications targeted and whether there was a significant intervention effect.

Table 2.

Comparison of serum potassium monitoring for diuretic use across reviewed studies

Study	Drug	Outcome measure	Effect measurement	Effect size	CI	p Value	Pre-intervention monitoring rate or control group	Post-intervention monitoring rate or intervention group
Feldstein18	All diuretics*	K testing	HR	0.9*	0.70 to 1.10	0.24
Hoch19	All diuretics	K testing	Absolute % increase prevalence of testing	3.0%		<0.001	78.5%	81.5%
Lo20	All diuretics	K	Adjusted OR†	1.32	0.87 to 2.023	0.20
Matheny6	Potassium-sparing diuretic	K	OR	0.82	0.12 to 5.60	0.84	60.7%	68.4%
Matheny6	Thiazide diuretic	K	OR	1.30	0.63 to 2.67	0.47	51.7%	64.5%
Palen21	All diuretics	K	Absolute % increase prevalence of testing	1.60%		0.11	44.0%	45.6%
Steele24	Diuretics not reported separately	NA	NA	NA	NA	NA	NA	NA

^*Not specific to diuretics, but embedded in composite measure for non-ACE/ARB drugs.

^†Corrected numbers based on correspondence with the authors.

ACE, ACE inhibitors; ARB, angiotensin II receptor blockers; HR, hazard ratio; K, potassium; OR, odds ratio.

Discussion

By 2009, eight studies reported the results of HIT interventions to improve laboratory monitoring of medications in the ambulatory setting, including six RCTs. Surprisingly, 50% of the RCTs reported significant improvements in monitoring while 50% did not. A detailed review of each of the studies identified important aspects of study quality, analysis and intervention design that help to explain these conflicting results.

Higher quality studies were less likely to show significant improvements in monitoring with HIT interventions compared with lower quality studies. Studies with lower quality scores^{19 24} used less rigorous study designs (such as pre–post intervention timing rather than RCT) and analytic approaches. These differences may explain some of the differences in intervention efficacy across studies. Because most of the HIT intervention studies were introduced in clinical systems with multiple clinical sites, it is important to account for non-independence of outcomes within each site due to local practice variations that can explain differences between different sites. Likewise, because clinicians cared for multiple patients within a site, it is important to consider non-independence of outcomes (ie, laboratory testing) between patients of the same provider because differences in care delivery between providers can also affect outcomes. Our review found that studies that addressed clustering in their design and analysis were less likely to show improvements in laboratory monitoring.^{6 20}

We also found that all of the RCTs were conducted in one of two large integrated healthcare systems in the USA. Study setting appears to be related to study results in two ways. First, both studies conducted outside of a large integrated healthcare system in the USA were less rigorous pre–post intervention trials,^{19 24} and each showed significant improvements. Second, one of the integrated healthcare systems had high baseline rates of monitoring,^{6 20} and our review indicates that studies in sites with lower baseline rates of monitoring reported greater improvements from HIT interventions than sites with higher baseline monitoring rates.^{18 23 24}

Intervention design features may also explain the conflicting study results. Our review revealed that the two interventions that targeted pharmacists were effective, ^{22 23} while only three of six interventions targeting physicians were effective.^{18 19 24} One study compared three arms, including an arm with electronic alerts to physicians, a second arm with voice mail messages to patients, and a third with pharmacy team outreach to patients, and found that the physician alert arm was the least effective.¹⁸ Past evidence suggests that changing physician behavior is challenging, and that passive approaches (non-interruptive alerts) to such physicians may not be effective.^25–27 It does not appear that the intrusiveness of the alert explains the difference in study findings, and this is consistent with several studies in which non-intrusive reminders did not improve physician adherence to alert recommendations.²⁸

It is helpful to consider our results in the context of other literature on the effectiveness of HIT interventions and their effects on prescribing errors and ADEs.^{8 10} Most reviews included a small number of studies, and many report that the studies reviewed were of low quality. For example, a 2003 review reporting error-rate improvement from CDS-only interventions included seven studies, many of which were under-powered.¹⁰ Another review of HIT interventions to improve drug dosing, mostly in the inpatient setting, found that many studies were of low quality.^{15 16} None of these studies addressed laboratory monitoring.

Variation in intervention effectiveness is also reported in other reviews of HIT interventions. For example, one review of the effect of CPOE and CDS on ADEs found that only half of the studies showed a reduction in ADEs²⁹, and another systematic review of CPOE and medical errors reported that, while more than half of studies found significant reductions in ADEs, the results varied widely. Although the investigators concluded that CPOE can reduce prescribing errors, they noted, ‘Reporting quality and study quality was often insufficient to exclude major sources of bias.’³⁰ The findings of our review are similar, with a slight majority of studies finding a positive impact of the interventions, but with variation in quality. As with other reviews, the number of studies addressing this issue is still limited.

There are several limitations to our review that should be noted. First, given the relatively small number of studies identified, it is difficult to draw conclusions about the overall impact of HIT intervention on rates of laboratory monitoring. By limiting our search to Medline English-language studies, we may have missed some non-US studies, but this allowed us to adequately review the study methodologies. Further, all studies regarding anticoagulation were excluded because it was not possible to identify the independent effect of interventions in improving laboratory monitoring (ie, INR testing) from dosing recommendations for warfarin. This limits the inferences we can make about HIT interventions on laboratory monitoring overall. Second, the studies were conducted in a limited number of clinical settings: three of the studies were conducted at one site and two at a second site. Further, all but one of the studies were conducted in large managed care organizations, limiting generalizability of the findings outside of these settings. Finally, differences in study design made it difficult to compare outcomes across studies. While we were unable to use meta-analysis to pool the effect sizes, we did compare the effects of the interventions across several studies on a single drug common to all studies, and did not find any consistent effect of HIT interventions on monitoring.

Even though the idea of using HIT to improve quality of care is not new,³¹ this goal has not yet been achieved. Many questions still remain, as posed by Kuperman et al³² in 2007: ‘To what extent does alerting impact on clinician behavior and patient outcomes? What is the optimal way to present alerts to prescribers? Which member of the healthcare team—for example, physician, nurse, pharmacist, other—is the best recipient of any kind of alert?’ These questions have yet to be answered. As more outpatient clinics adopt electronic records and electronic prescribing, it will be increasingly important to know the impact of decision support in this setting in supporting implementation of the most effective interventions. This is particularly true with regard to laboratory monitoring, which is often a locus for preventable adverse effects.

Although numerous reviews and studies have attempted to answer these questions, our systematic search identified more interventions in the inpatient setting than in the ambulatory setting. Of the studies identified, concerns about study quality and design could not exclude sources of bias in the reported results. Future studies of laboratory monitoring should better address patient and provider characteristics and account for fixed physician or clinical site effects by multilevel analysis. Studies can also better clarify outcomes (ie, improvements in test ordering versus test completion), and should also be expanded to include settings outside of the large integrated healthcare delivery systems.

While this systematic review found evidence suggesting that information technology interventions may improve laboratory monitoring for high-risk prescribed medications (exclusive of anticoagulants) in the ambulatory setting, the evidence is conflicting. The well-considered, well-designed studies reviewed appeared to find little improvement in laboratory monitoring for high-risk medications with HIT interventions targeting physicians only. However, five of eight studies found some positive effect, and this suggests that using HIT may be a promising avenue for improving laboratory monitoring. More research is needed to determine how to maximize the full potential benefit of HIT for monitoring high-risk medications and ultimately improving patient safety.