Computerized Physician Order Entry: Helpful or Harmful?

Abstract

Computerized physician order entry (CPOE) is touted as a major improvement in patient safety, primarily as a result of the Institute of Medicine's 1999 report on medical errors and the subsequent formation of the “Leapfrog Group” of companies to preferentially direct their employees' health care to those institutions that install such systems (as part of directives that “Leapfrog” feels will improve patient care). Although the literature suggests that such systems have the potential to improve patient outcomes through decrease of adverse drug events, actual improvements in medical outcomes have not been documented. Installation of such systems could actually increase the number of adverse drug events and result in higher overall medical costs, particularly in the first few years of their adoption.

In the last five years, hospitals, including our own, have begun to use computerized systems that require physicians and other health care providers to electronically enter patient care orders.¹ Before this time, only a handful of hospitals used such systems. These computer programs contain algorithms that alert health care providers to potentially harmful therapeutic decisions before orders are processed. The installation of these systems is costly (millions of dollars) and requires major behavioral changes, not only by physicians, but also by the entire health care organization.² In January 2003, Cedars-Sinai Health System in Los Angeles removed its recently installed computerized physician order entry (CPOE) system from use after almost unanimous protest from the medical staff. Why are hospitals and other health care organizations pursuing this avenue at this time? Does the literature support the premise that these systems are beneficial for patient care? Do such systems decrease total health care costs? The answers to these questions are still evolving. In this forum, we address these questions and describe some of the pertinent medical literature on this subject.

Why CPOE Now?

In 1999, the Institute of Medicine (IOM), a body formed by the National Academy of Sciences to “enlist distinguished members of the appropriate professions in the examination of policy matters pertaining to the health of the public,” published a report entitled To Err Is Human: Building a Safer Health System.³ Shortly after this report was made public, the Leapfrog Group was founded by the Business Roundtable (BRT), a national association of Fortune 500 chief executive officers (CEOs). The Leapfrog Group was created to “help save lives and reduce preventable medical mistakes by mobilizing employer purchasing power to initiate breakthrough improvements in the safety of health care and by giving consumers information to make more informed hospital choices.”⁴ The intent of the Leapfrog Group is to preferentially direct their corporate members' health care to those organizations that adhere to patient safety standards specified in their guideline documents based primarily on the IOM report. Interestingly, 10% of the Leapfrog Group members (as of December 2002) are directly involved in sales of hardware, software, or both to health care entities.⁵

The IOM report Executive Summary contains the following paragraphs:

Two large studies, one conducted in Colorado and Utah and the other in New York, found that adverse events occurred in 2.9 and 3.7 percent of hospitalizations, respectively. In Colorado and Utah hospitals, 8.8 percent of adverse events led to death, as compared with 13.6 percent in New York hospitals. In both of these studies, over half of these adverse events resulted from medical errors and could have been prevented.

When extrapolated to the over 33.6 million admissions to U.S. hospitals in 1997, the results of the study in Colorado and Utah imply that at least 44,000 Americans die each year as a result of medical errors. The results of the New York Study suggest the number may be as high as 98,000. Even when using the lower estimate, deaths due to medical errors exceed the number attributable to the 8th leading cause of death. More people die in a given year as a result of medical errors than from motor vehicle accidents (43,458), breast cancer (42,297), or AIDS (16,516).⁶

These two studies quoted by the IOM and the Leapfrog Group are based on data collected in the 1980s. The methodology of these studies has been challenged, even by its own authors, one of whom stated in a subsequent editorial: “Moreover, the reliability of identifying errors is methodologically suspect, and some astute observers have recommended that reviews based on implicit judgments by physicians, such as the reviews we used in the New York and Utah–Colorado studies, be replaced by reviews based on the use of explicit criteria.”^7,8

The New York study (based on New York State Hospitalization data collected in 1984), in particular, if corrected for a control group of equally ill patients who did not suffer a “preventable” adverse event, could well have shown no difference in mortality rates resulting from adverse events during a hospitalization.^9,10 These two studies, with their lack of control groups and other methodologic flaws, are part of the basis on which the IOM and Leapfrog form their conclusions on the use of CPOE in preventing death and other adverse medical outcomes.

Among other recommendations, the Leapfrog Group and the IOM report strongly recommend that hospitals and other health care entities institute CPOE. The Leapfrog Group in particular includes the following language in their materials distributed to prospective members: “Computer Physician Order Entry (CPOE). Hospitals that fulfill this standard will: (1) require physicians to enter medication orders via computer linked to prescribing error prevention software; (2) demonstrate that their CPOE system can intercept at least 50% of common serious prescribing errors, utilizing test cases and a testing protocol specified by the Institute for Safe Medication Practices (ISMP); (3) require documented acknowledgment by the prescribing physician of the interception prior to any override; and (4) post the test case interception rate on a Leapfrog-designated web site.”¹¹

Does the Literature Support the Premise That These Systems Are Beneficial for Patient Care?

There are very few studies in the medical literature addressing the question of whether CPOE changes patients' medical outcome. Several studies performed with systems designed in the 1970s and 1980s dealt only with antibiotic administration by CPOE and show some benefit in both cost savings and patient outcome.^12,13 More recently, a study done at a pediatric hospital in Canada, limited to one medical and two surgical inpatient units on which CPOE was installed, showed a 40% decrease in medication error rates (defined as “any event involving medication prescription, dispensing, administration, or monitoring of medications irrespective of outcome”). Despite this, no difference in actual patient morbidity or mortality was shown,¹⁴ a result similar to what was observed in the Harvard studies discussed subsequently.

The only study addressing CPOE of all hospital services (adult and pediatric, including all pharmaceutical agents and intravenous drips and mixtures), quoted extensively by Leapfrog and the IOM, was authored by Bates et al.¹⁵ and was performed at Brigham and Women's Hospital (part of the Harvard System) in the early 1990s. Since the IOM study and the initial formation of Leapfrog, Harvard's further experience with CPOE use by housestaff officers (the main users of such systems) over a four-year time span from 1993 through 1997 provided more information regarding patient outcomes.¹⁶ Both studies used a control group of hospital admissions to obtain a baseline for rates of “nonintercepted serious medication errors” followed by the same analysis after the institution of a CPOE system. The methods and outcome variables were different in each study and are discussed separately.

Both Harvard studies defined “nonintercepted serious medication errors” as a combination of “preventable adverse drug events (ADEs)” that actually occurred plus the total number of “nonintercepted potential ADEs.” The authors define “nonintercepted errors” as errors in drug dosage, interactions, and so on that were not identified by the ordering physician, the pharmacy system, or the administering nursing or pharmacy staff on the floors. “Potential ADEs” did not cause any adverse patient outcomes, and the “seriousness” of both the potential and real ADEs was determined by two blind reviewers with interobserver correlation. The first study used medical and surgical units as well as intensive-care unit settings for collection of baseline data over six months, followed by institution of CPOE (with a pause for training and acceptance) and recollection of data for nine months. The number of “nonintercepted serious medication errors” decreased 55% with the addition of CPOE. The number of “nonintercepted potential ADEs” decreased 84%. These numbers are very impressive and have been extensively quoted by the Leapfrog Group as well as software vendors and the lay press. However, the institution of CPOE had no statistical impact on the occurrence of actual serious ADEs, which decreased by only 17% after CPOE was instituted. Although the authors of the study comment that 42% of the preventable actual ADEs were attributable to “judgment” errors in the use of multiple sedating medications that the computer program would not have prevented, the reality is that no significant decrease in patient morbidity/mortality occurred as a result of the institution of CPOE.

The second study by the Harvard group used data collected on three medical services and designed to look at the same type of outcomes over a four-year period, during which time the CPOE system was constantly undergoing refinement. Data collection was done over two-month periods in 1992 (baseline), 1993 (period 1), 1995 (period 2), and 1997 (period 3). This study again showed an 86% decline in number of “nonintercepted potential ADEs” from baseline compared with period 3. Of major concern, however, was the increase in preventable ADEs (5 of 1,000 patient-days without CPOE to 15 of 1,000 patient-days with CPOE, an increase of 200%) that occurred during period 1 when CPOE was first instituted. This trended downward over time in the last two CPOE periods studied. However, the absolute numbers of true ADEs were too small to be of statistical significance (5, 15, 2, 2 for baseline, periods 1, 2, and 3, respectively). Additionally, the number of “intercepted potential life-threatening ADEs” (usually caught by nursing staff before administration of a drug or intravenous admixture) dramatically climbed in periods 1 and 2 after the institution of CPOE. These increases were tracked to “bugs” in the CPOE system's mechanism for ordering potassium infusions. After the software was refined in period 3, the problem disappeared. These negative effects of the institution of CPOE stimulated the authors to make the following comments in their discussion: “The increase in the number of intercepted potential ADEs that occurred post-POE during periods 1 and 2 illustrates the potential that any change, especially a systems change with profound effects such as POE, has for causing new errors, even though this particular error was always intercepted and the overall effect was clearly positive.”

“We conclude that computerized POE resulted in a very large decrease in the frequency of non-missed-dose medication errors, the errors that are most likely to harm patients. Systems such as these have the potential to both fix and cause problems, and require evaluation.”¹⁶

Review of the Harvard studies indicates that although CPOE does indeed decrease the potential for an ADE to occur, there is no decrease in actual serious ADE events with the institution of CPOE. In these Harvard studies, CPOE appeared to increase the incidence of actual serious ADEs, particularly during the early years of implementation of such systems.

Do CPOE Systems Decrease Total Health Care Costs?

The Leapfrog Group literature maintains that an additional $2 billion is spent in health care costs nationwide as a result of serious ADEs in hospitalized patients. This baseline dollar cost for ADEs nationally was calculated from a study by Classen et al. in 1997 attributing an additional medical care cost of $2,000 per ADE¹⁷ multiplied by the estimated number of ADEs that occur nationally.¹⁸ In the initial Harvard study, a direct savings of $480,000 per year to the institution was calculated based on an estimated annual ADE cost of $2.8 million and the not statistically significant decrease of 17% of serious ADEs in their initial study discussed previously. This figure (according to the authors) “does not include the costs of injuries borne by patients, of admissions due to drug errors, of malpractice suits, or of the extra work generated by the nonserious medication errors.”¹⁵ When these admittedly ethereal costs are added, the Harvard group estimated a savings to their institution of $5–10 million. This includes a deduction of $1.9 million installation cost for CPOE and a $500,000 yearly maintenance cost.

Leapfrog takes this estimate even further by using the 55% reduction in “serious ADEs” reported in the first Harvard study (even though these reductions were only potential ADEs, not actual ADEs, as discussed previously) to calculate that CPOE would avert 522,000 ADEs across the United States and, therefore, reduce the additional health care costs for ADEs of $2 billion, quoted previously, to $1.1 billion.

Similar to data on patient care and outcome, there is only a small body of literature on cost and resource utilization after installation of CPOE across an entire institution. An early study from Indiana University, limited to a single internal medicine inpatient service which installed CPOE, showed a 12.7% decrease in total hospital charges and a decrease in average length of stay by 0.89 day.¹⁹ More recently, Ohio State University published its experience with “housewide” CPOE at two hospitals and showed statistically significant and impressive improvements in medication turnaround times, radiology procedure completion times, and laboratory result reporting times. Patient length of stay had a small, statistically significant drop at one hospital but not at the other. However, total “cost per admission” had no significant change at either facility after implementation of CPOE.¹

The Leapfrog Group used the results of the first Harvard study as its entire basis for its theoretical calculations of decrease in health care costs as a result of institution of CPOE systems. However, these calculations can show an opposite trend when taking into account the follow-up Harvard study and adding the increased time required by housestaff (the primary users of the system) to enter their daily orders. Although the absolute numbers are small, the installation of CPOE created a 200% increase in actual serious ADEs during the first two years of implementation (from the second Harvard publication discussed previously, which showed an absolute increase from five ADEs to 15 ADEs in the first two years of use of CPOE). Therefore, if the Harvard studies are reflective of the effects of CPOE in general, national health care costs as a result of ADEs will increase transiently to $6 billion per year (using the Leapfrog numbers) during the first two years of CPOE implementation.

Because housestaff work hours have recently been limited to 80 hours per week, the additional time required for them to enter data in CPOE will almost certainly result in additional health care personnel costs to hospitals in the form of physician extenders to provide direct patient care. This additional time required to enter computer-based orders has been estimated at 5% of their total workweek hours.^20,21 If a large hospital employs 500 housestaff, an additional 25 full-time equivalent physicians or physician extenders would be required for direct patient care, assuming that before the institution of CPOE, a resident's 80-hour week was entirely spent doing direct patient care. This would add approximately $1.6–2 million to individual hospital budgets in addition to the costs of purchase and maintenance of a CPOE system itself.

Conclusion

The recent pressure for CPOE occurred after the IOM report, with its dramatic and methodologically questionable data on deaths and serious medical errors. A push from the business sector (Leapfrog), many of whose corporate members stand to financially benefit from installation of CPOE systems, has created further pressure on hospitals to install these systems.

The available objective data, which are scant, suggest that, at best, there is a potential for these systems to decrease ADEs and their additional medical costs. An initial study from one university health system showed a decrease in the potential for ADEs, but not a statistically significant decrease in the actual occurrence of ADEs with installation of a CPOE system. This potential benefit, which has yet to be shown to improve real outcomes statistically, could be offset by an initial increase in actual ADEs with their attendant dollar cost and patient morbidity, as newly installed CPOE systems are refined within the context of the operational and cultural environments of the large medical facilities that can afford the costs of implementation of such systems.

CPOE systems will prove to be more costly to institutions than just the purchase and maintenance of the hardware and software involved in such a purchase or development. The increased time required by physicians to enter data into CPOE products will result in increased personnel costs for direct patient care. The “requirements” by Leapfrog to validate CPOE add time and dollar costs to hospitals in addition to the major cost and behavioral change that CPOE entails. The medical malpractice cost implications of such systems (pro or con) have yet to be determined because the electronic tracking and storage of physician “overrides” of alerts that CPOE systems' provide will be a legal conundrum.

Medical facilities contemplating the institution of CPOE should understand that the literature on the medical and economic effects of such systems is still evolving. There are now numerous CPOE systems (both commercial and proprietary) available and, like all software, some could fit the needs of specific institutions better than others. CPOE is also just one part of overall software solutions designed to theoretically improve patient outcomes. The medical informatics community must continue to rigorously study CPOE systems as they become integrated, along with other medical software, into the daily delivery of clinical care. It is certainly possible that as CPOE systems mature in the future, true benefits can be shown from their implementation. For now, however, the jury is still out.