Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis

Mirela Prgomet; Ling Li; Zahra Niazkhani; Andrew Georgiou; Johanna I Westbrook

doi:10.1093/jamia/ocw145

. 2016 Oct 7;24(2):413–422. doi: 10.1093/jamia/ocw145

Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis

Mirela Prgomet ^1,^✉, Ling Li ¹, Zahra Niazkhani ^2,³, Andrew Georgiou ¹, Johanna I Westbrook ¹

PMCID: PMC7651905 PMID: 28395016

Abstract

Objective: To conduct a systematic review and meta-analysis of the impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay (LOS), and mortality in intensive care units (ICUs).

Methods: We searched for English-language literature published between January 2000 and January 2016 using Medline, Embase, and CINAHL. Titles and abstracts of 586 unique citations were screened. Studies were included if they: (1) reported results for an ICU population; (2) evaluated the impact of CPOE or the addition of CDSSs to an existing CPOE system; (3) reported quantitative data on medication errors, ICU LOS, hospital LOS, ICU mortality, and/or hospital mortality; and (4) used a randomized controlled trial or quasi-experimental study design.

Results: Twenty studies met our inclusion criteria. The transition from paper-based ordering to commercial CPOE systems in ICUs was associated with an 85% reduction in medication prescribing error rates and a 12% reduction in ICU mortality rates. Overall meta-analyses of LOS and hospital mortality did not demonstrate a significant change.

Discussion and Conclusion: Critical care settings, both adult and pediatric, involve unique complexities, making them vulnerable to medication errors and adverse patient outcomes. The currently limited evidence base requires research that has sufficient statistical power to identify the true effect of CPOE implementation. There is also a critical need to understand the nature of errors arising post-CPOE and how the addition of CDSSs can be used to provide greater benefit to delivering safe and effective patient care.

Keywords: medical order entry systems, decision support systems, clinical, medication errors, mortality, length of stay

Introduction

The high rate of medication errors in hospitals is a well-recognized and significant patient safety issue.¹ Medication errors have consistently been attributed to longer hospital stays, increased costs, significant morbidity, and even death.^1–4 In complex hospital wards, such as intensive care units (ICUs), the prevalence of errors and adverse patient outcomes is higher and of greater severity than in general wards.⁵ A clinical review of medical errors in critical care undertaken by Moyen et al.⁶ associated this increased prevalence of errors to risk factors related to the severity of illness of ICU patients, number and type of medications used (i.e., frequent use of boluses and infusions, which often require weight-based dose calculations), and complexity of the ICU environment.

Interventions aimed at preventing medication errors in hospitals, particularly at the prescribing stage, include computerized provider order entry (CPOE) and clinical decision support systems (CDSSs). Systematic reviews of the impact of CPOE and CDSSs across inpatient settings have reported significant reductions in medication errors,^7–11 while changes in mortality¹⁰^,¹² and length of stay (LOS)¹¹^,¹² have not been significant. However, these reviews combined results from homegrown and commercial CPOE systems. Homegrown systems are more likely to demonstrate positive effects on safety and quality of care, as they are under the local control of the implementing institution and have been highly customized for local conditions.⁹^,¹³ As homegrown systems become increasingly difficult for organizations to maintain, almost all future system implementations are likely to involve commercial systems.¹³

The lack of specific reviews to guide the large investments being made in sophisticated commercial systems highlights the need to collate and examine research that evaluates the impact of commercial CPOE systems on errors and patient outcomes,⁷^,⁸ particularly among populations most at risk of errors and adverse outcomes, such as patients in ICUs. Thus, our aim was to conduct a systematic review and meta-analysis of evidence of the impact of commercial CPOE and CDSSs on medication errors, LOS, and mortality in ICUs.

Method

Search strategy

We searched for English-language literature published between January 2000 and January 2016 using Medline and Embase via Ovid, and The Cumulative Index to Nursing and Allied Health Literature (CINAHL) via EBSCOhost. We restricted the date range, as studies conducted prior to 2000 tended to evaluate homegrown CPOE.⁷ We used a combination of MeSH terms and keywords related to the intervention (CPOE, CDSS), outcomes of interest (medication errors, LOS, mortality), and study setting (ICU). The complete database search strategy is provided in Appendix A (available as a Supplementary File). We also searched 2 specialized bibliographies—the Inventory of Health Information Evaluation Studies (https://evaldb.umit.at/) and the Health IT Bibliography (https://healthit.ahrq.gov/health-it-tools-and-resources/health-it-bibliography)—and hand searched the reference lists of all full-text articles that we assessed for potential inclusion. The protocol for this review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement¹⁴ and was registered with PROSPERO (registration number CRD42013004543).

Study selection

The above search strategy was executed on 17 February 2016 and resulted in the identification of 887 citations. After removing duplicates, the titles and abstracts of 586 unique citations were screened for eligibility. Screening of titles and abstracts was conducted independently by 2 researchers and compared for consistency. Where there was a discrepancy between researchers, the citation was assigned to full-text review. Two researchers also independently reviewed the full text of 124 studies. The authors of 13 studies were contacted in order to obtain clarification or additional data. Three researchers assessed the full text of a subset of 34 studies, which were discussed in depth against the eligibility criteria to determine the final set of included studies. Figure 1 shows the study selection process.

Figure 1. — Search and selection flow-diagram.

We defined CPOE as computer-based systems used for entering orders, including laboratory tests, imaging, nutrition, blood products, and medication prescriptions. Almost all CPOE systems have some level of decision support to assist ordering decisions; however, the degree of sophistication of CDSSs can vary from basic duplicate order alerts to complex algorithms based on patient-specific data.¹⁵ Where studies evaluated the addition of a specific CDSS to an existing CPOE system, such as algorithms developed in response to identified medical errors or quality improvement initiatives, we defined these as “targeted” CDSSs.¹⁶ For studies reporting medication errors, we focused on errors occurring at the prescribing stage, which can include incomplete, incorrect, or inappropriate drug orders.

Studies were eligible for inclusion if they: (1) reported results for an ICU population; (2) evaluated the impact of moving from paper-based ordering to CPOE or evaluated the addition of a targeted CDSS to an existing CPOE system; (3) reported quantitative data on medication errors, LOS, or mortality pre- and post-CPOE or CDSS; and (4) used a randomized controlled trial or quasi-experimental study design.

Studies were excluded if the CPOE system was not a commercial system, was implemented prior to the year 2000, or was implemented alongside other interventions making it difficult to assess the impact of CPOE (e.g., Abstoss et al.¹⁷). For studies assessing the addition of a CDSS to an existing CPOE system, if the CDSS was not integrated with the CPOE system (e.g., Sintchenko et al.¹⁸), the study was excluded. We also excluded studies where outcomes were voluntarily reported (e.g., nurses reporting errors in incident reporting systems); studies that were conducted in a simulated environment; qualitative studies or opinion pieces; and studies that were available only as abstracts or posters, as they provided insufficient information to systematically determine eligibility.

Data extraction

We extracted the following data variables into a spreadsheet: study author(s), year of publication, country in which the study was conducted, type of ICU (adult/pediatric), study design, number and type of patients included in the study (all ICU patients/only patients with a specific condition), description of the system implemented (CPOE/CDSS), and duration of baseline and intervention periods. For studies that reported mortality, we extracted the number of patient deaths in ICU (ICU mortality) and/or the number of ICU patient deaths in hospital (hospital mortality) at baseline and intervention. For studies that measured LOS, we extracted the mean patient LOS in ICU (ICU LOS) and/or the mean LOS of ICU patients in hospital (hospital LOS) at baseline and intervention. For studies that reported medication errors, we extracted the number of prescriptions, prescribing error definitions, total number of prescribing errors at baseline and intervention, and evidence of any new types of errors introduced by the CPOE system.

Quality assessment

We assessed the methodological quality of the included studies using the Effective Public Health Practice Project (EPHPP) quality assessment tool for quantitative studies.¹⁹ The tool was selected because it can be used to assess the methodological quality of both randomized and nonrandomized studies, and has been judged suitable for use in systematic reviews.²⁰ Using the tool, studies are attributed a rating of strong, moderate, or weak based on 6 components: (1) selection bias, (2) study design, (3) confounders, (4) blinding, (5) data collection methods, and (6) withdrawal and dropouts.

Statistical analysis

We categorized the included studies into those that evaluated the impact of moving from paper-based ordering to CPOE and those that evaluated the impact of adding a targeted CDSS to an existing CPOE system. We then grouped the studies by outcome measure (medication errors, ICU LOS, hospital LOS, ICU mortality, or hospital mortality). Three studies^21–23 reported ICU LOS using median and interquartile ranges. We contacted the authors of these studies and requested mean and standard deviation (SD) data. We received the results for 1 study;²³ however, the authors for the other 2 studies were no longer in possession of the raw data, so these studies could not be included in meta-analysis for this outcome measure. Mean and SD were estimated²⁴ for 1 study²⁵ that reported ICU LOS using median and range. One study²⁶ reported results on ICU LOS, ICU mortality, and hospital mortality from 4 overlapping study periods. The results from the longest study periods (i.e., 24-month baseline and 12-month intervention periods) were used for the meta-analyses, while results from the other study periods were used for sensitivity analysis. Three studies^27–29 reported results from 2 separate intervention periods (e.g., two 2-week periods). We combined the data from the 2 periods into 1 intervention period (e.g., one 4-week period). A study by Kadmon et al.³⁰ on the impact of CPOE on medication errors included 2 interventions: post-CPOE and post-CPOE with CDSS. We included the results from the latter in the meta-analysis and conducted sensitivity analysis for the results from the post-CPOE–only intervention period.

The included studies contained sufficient information to conduct meta-analyses for 4 outcome measures: medication errors, ICU LOS, ICU mortality, and hospital mortality. We calculated relative risks (RRs) for medication errors, ICU mortality, and hospital mortality, and mean difference for ICU LOS. A meta-analysis for each outcome measure was performed using random effects models to pool the results and, in order to be conservative, the Knapp-Hartung approach³¹ was applied to account for heterogeneity between studies. The meta-analyses results are presented using forest plots (Figures 2–5). Between-study heterogeneity was evaluated using chi-square tests and I² statistics.³² The potential for publication bias for each meta-analysis was assessed by inspection of funnel plots and statistical tests based on weighted linear regression of the intervention effect on its standard error.³³ Subgroup meta-analyses were also conducted by study quality and ICU type (adult/pediatric) when appropriate. Studies conducted in neonatal ICUs^34–36 were categorized as pediatric. Sensitivity analyses were conducted using different measurements for the outcomes, such as odds ratios and standardized mean differences. All statistical tests were 2-sided and were evaluated at a significance level of 0.05. Analyses were carried out using R version 3.2.1.³⁷

Figure 2. — Relative risk of medication errors (errors indicates number of errors and orders indicates number of orders audited; RE = random effect).

Figure 3. — Mean difference of ICU LOS (N indicates number of patients and mean indicates LOS in days; RE = random effect).

Figure 4. — Relative risk of ICU mortality (event indicates number of deaths and total indicates number of patients; RE = random effect).

Figure 5. — Relative risk of hospital mortality (event indicates number of deaths and total indicates number of patients; RE = random effect).

Results

Study characteristics

Twenty studies met our inclusion criteria: 16 that assessed the transition from paper-based ordering to CPOE²³^,^25–30^,^34–36^,^38–43 and 4 that examined the addition of a targeted CDSS to an existing CPOE system²¹^,²²^,⁴⁴^,⁴⁵ (Table 1). Eleven studies were conducted in the United States, 4 in the UK, and 1 each in Belgium, Canada, Israel, Saudi Arabia, and Spain. Study publication dates ranged from 2004 to 2014. All studies used a pre-post study design, except 1,²³ which was a prospective controlled study. The CPOE vendors included: Cerner³⁹^,⁴⁰^,^43–45, GE Centricity²¹^,²³^,³⁶^,⁴², MetaVision iMDsoft²⁷^,³⁰, Horizon Expert Orders³⁵^,⁴¹, Misys QuadraMed²⁶, Global Dominion Access²⁸, IntelliVue Philips²⁹, INVISION Siemens³⁴, and EPIC.³⁸ Based on the EPHPP quality assessment tool, 13 studies²²^,^26–28^,³⁴^,³⁶^,^39–45 were rated as being of a moderate methodological quality and 7 studies²¹^,²³^,²⁵^,²⁹^,³⁰^,³⁵^,³⁸ were rated as weak. No studies were rated as strong.

Table 1.

Characteristics of the included studies

	Author (year)	Country	ICU type	Sample	CPOE vendor	CPOE/CDSS description	Outcomes reported
Studies Comparing CPOE to Paper	Al-Dorzi et al. (2011)²⁶	Saudi Arabia	Adult	All patients	Misys, QuadraMed	CPOE with interaction and allergy alerts, order sets, dose checking, and protocols	ICU LOS, Hospital LOS, ICU Mortality, Hospital Mortality
	Ali et al. (2010)²⁷	UK	Adult	All patients	MetaVision, iMDsoft	CPOE with default prescriptions	Medication Errors
	Armada et al. (2014)²⁸	Spain	Adult	All patients	Global Dominion Access	CPOE with protocols, and duplicate, allergy and interaction alerts	ICU LOS, ICU Mortality, Medication Errors
	Carayon and Wood (2009)³⁸	United States	Adult	All patients	EPIC	CPOE	ICU LOS, Medication Errors
	Colpaert et al. (2006)²³	Belgium	Adult	All patients	GE Centricity	CPOE with protocols, interaction alerts, allergy status, drug-related complications	ICU LOS, Medication Errors
	Cordero et al. (2004)³⁴	United States	Pediatric	Very-low-birth-weight infants	INVISION, Siemens	CPOE with order sets, allergy, interaction and duplicate alerts, dose checking and calculation	ICU Mortality, Medication Errors
	Del Beccaro et al. (2006)³⁹	United States	Pediatric	All patients	Cerner	CPOE with order sets, allergy checking, and dose checking	ICU LOS, ICU Mortality
	Han et al. (2005)⁴⁰	United States	Pediatric	Patients admitted via inter-facility transport	Cerner	CPOE with allergy and drug interaction alerts	Hospital Mortality
	Jozefczyk et al. (2013)³⁵	United States	Pediatric	All patients	Horizon Expert Orders	CPOE	Medication Errors
	Kadmon et al. (2009)³⁰	Israel	Pediatric	All patients	MetaVision, iMDsoft	CPOE with dose checking and default prescriptions	ICU Mortality, Medication Errors
	Keene et al. (2007)³⁶	UK	Pediatric	All patients	GE Centricity	CPOE with order sets	Hospital Mortality
	Longhurst et al. (2010)⁴³	USA	Pediatric	All patients	Cerner	CPOE with clinical decision support	ICU Mortality
	Potts et al. (2004)⁴¹	United States	Pediatric	All patients	Horizon Expert Orders	CPOE with order sets, interaction and allergy alerts, and dose checking	ICU LOS, Medication Errors
	Shulman et al. (2005)⁴²	UK	Adult	All patients	GE Centricity	CPOE with links to drug information	Medication Errors
	Thompson et al. (2004)²⁵	Canada	Adult	Patients with urgent or stat orders	Not reported	CPOE with order sets	ICU LOS, Hospital LOS, Hospital Mortality
	Warrick et al. (2011)²⁹	UK	Pediatric	All patients	IntelliVue, Philips	CPOE with order sets and drug information	Medication Errors
Studies Evaluating Targeted CDSS	Adams et al. (2011)⁴⁴	United States	Pediatric	All patients	Cerner	Red blood cell transfusion alert	Hospital LOS, Hospital Mortality
	Fernandez-Perez et al. (2007)²¹	United States	Adult	Patients with anemia	GE Centricity	Red blood cell transfusion algorithm	ICU LOS, Hospital LOS, ICU Mortality, Hospital Mortality
	Pageler et al. (2013)⁴⁵	United States	Pediatric	All patients	Cerner	Rule that restricts scheduling repeat orders	ICU LOS, Hospital LOS, Hospital Mortality
	Rana et al. (2006)²²	United States	Adult	Patients with anemia	Not reported	Red blood cell transfusion algorithm	ICU LOS, ICU Mortality, Hospital Mortality

Open in a new tab

ICU = intensive care unit; CPOE = computerized provider order entry; CDSS = clinical decision support system; LOS = length of stay.

Studies comparing CPOE to paper

The 16 studies that assessed the transition from paper-based ordering to CPOE contained sufficient information to perform meta-analysis for 4 outcomes: medication errors, ICU LOS, ICU mortality, and hospital mortality. A summary of the findings of these 16 studies is provided in Appendix B (available as a Supplementary File).

Medication errors

Of the 10 studies examining the impact of CPOE on medication errors, 9^23,^27–30^,³⁵^,³⁸^,⁴¹^,⁴² were included in the meta-analysis. The broad definition of medication prescribing errors was similar across the studies and included illegible, erroneous, or omitted information (Appendix C, available as a Supplementary File). However, the level of detail regarding the specific elements that were included varied between the studies. Some studies, for example, indicated that missing weight⁴¹ or no signature²⁹^,⁴² constituted an error of omission and some included rule violations,³⁰^,⁴¹ while other studies did not list these elements in their error definitions. There were also different methods used to determine error rates. Subsequently, there was significant variation in the frequency of errors, with pre-CPOE error rates ranging between 4.5%³⁸ and 58.2%,³⁵ and post-CPOE error rates ranging between 0%²⁷ and 8.2%.²⁹ There was evidence of heterogeneity between studies (I²= 99.65%, P < .0001). Seven of the studies reported a significant reduction in medication errors following CPOE implementation, 1 study³⁰ reported no change, and 1 study³⁸ reported a significant increase in errors. Overall, there was evidence that the introduction of CPOE was associated with a significant reduction in the medication error rate by 85% (pooled RR: 0.15, 95% confidence interval (CI), 0.03–0.80, P = .03) (Figure 2). There was no evidence of publication bias (P = .07). Sensitivity analysis using results from the post-CPOE–only period (instead of the post-CPOE and CDSS period) in the Kadmon et al. study³⁰ still indicated an overall significant error reduction (pooled RR: 0.15, 95% CI, 0.03–0.72, P = .02).

Subgroup analysis by ICU type showed no evidence of significant error reduction following CPOE implementation for studies conducted in adult ICUs (pooled RR: 0.11, 95% CI, 0.00–3.41, P = .1) or in pediatric ICUs (pooled RR: 0.21, 95% CI, 0.02–2.65, P = .1). Likewise, subgroup analysis of 4 studies²⁷^,²⁸^,⁴¹^,⁴² with a quality rating of moderate showed no reduction in medication errors following CPOE implementation (pooled RR: 0.04, 95% CI, < 0.01–2.82, P = .1).

A study by Cordero et al.³⁴ provided data on medication errors that was not included for synthesis via meta-analysis. The study looked at errors within a subgroup of ICU patients (very low birth weight neonates receiving gentamycin on admission), while the studies included in the meta-analysis looked at all ICU patients during their respective study periods. Cordero et al.’s³⁴ study found 14 errors in 105 orders examined in the baseline period. In the intervention period, no errors were found among the 89 orders examined.

Of the types of errors reported across the studies, elimination of illegible orders was the most frequently reported benefit following CPOE implementation.²⁸^,²⁹^,³⁵^,⁴¹ Three studies reported new error types arising due to CPOE. Armada et al.²⁸ and Colpaert et al.²³ identified problems with duplicate prescriptions, while Armada et al.²⁸ and Shulman et al.⁴² both found problems with erroneous selection from dropdown menus, with Shulman et al.⁴² indicating that selection of wrong dose from a dropdown menu resulted in 1 potentially fatal intercepted error. However, Shulman et al.⁴² also found that the frequency of errors considered moderate/major decreased from 1.8% to 0.9% of audited orders. Colpaert et al.²³ similarly reported a decrease in serious medication prescribing errors, from 4.9% to 1.8% of audited orders.

ICU LOS

Seven studies²³^,²⁵^,²⁶^,²⁸^,³⁸^,³⁹^,⁴¹ were included in the meta-analysis examining the association between CPOE introduction and ICU LOS. There was evidence of heterogeneity between studies (I²= 54.42%, P = .02). Only 1 study reported a significant finding, with ICU LOS found to decrease from a mean of 7.44 days to 5.96 days following CPOE implementation.²³ Overall, there was no evidence of change in ICU LOS following the introduction of CPOE (pooled mean difference: −0.10, 95% CI, −0.81–0.60, P = .7; Figure 3). There was no evidence of publication bias (P = .2). Subgroup analysis of the 5 adult ICU studies²³^,²⁵^,²⁶^,²⁸^,³⁸ (pooled mean difference: −0.01, 95% CI, −1.16–1.13, P = .9) and the 4 studies with a quality rating of moderate²⁶^,²⁸^,³⁹^,⁴¹ (pooled mean difference: −0.40, 95% CI, −1.07–0.26, P = .1) showed no evidence of reduction in ICU LOS following CPOE implementation.

Two studies²⁵^,²⁶ reported on hospital LOS for ICU patients; however, the data were insufficient to perform meta-analysis. Neither study reported a significant change in hospital LOS following the implementation of CPOE.

ICU mortality

Six studies²⁶^,²⁸^,³⁰^,³⁴^,³⁹^,⁴³ were included in the meta-analysis examining the association between CPOE introduction and ICU mortality. The studies were found to be homogenous (heterogeneity between studies: I²= 0%, P = .8). Overall, there was evidence that the introduction of CPOE reduced ICU mortality by 12% (pooled RR: 0.89, 95% CI, 0.78–0.99, P = 0.04; Figure 4). There was no evidence of publication bias (P = .7). Subgroup analysis of the 4 pediatric studies³⁰^,³⁴^,³⁹^,⁴³ showed no significant change in ICU mortality (pooled RR: 0.84, 95% CI, 0.60–1.19, P = .2), while subgroup analysis of the 5 studies with a quality rating of moderate²⁶^,²⁸^,³⁴^,³⁹^,⁴³ revealed a 14% reduction in ICU mortality after CPOE implementation (pooled RR: 0.86, 95% CI, 0.78–0.96, P = .02).

Hospital mortality

Four studies²⁵^,²⁶^,³⁶^,⁴⁰ were included in the meta-analysis examining the impact of CPOE on hospital mortality for ICU patients. There was evidence of heterogeneity between studies (I²= 82.53%, P = .0006). Only 1 study reported a significant finding, with mortality found to increase from 39 deaths in 790 patients to 36 deaths in 312 patients following the introduction of CPOE.⁴⁰ Overall, however, there was no significant association between CPOE introduction and hospital mortality (pooled RR: 1.17, 95%CI, 0.53–2.54, P = .6; Figure 5). There was no evidence of publication bias (P = .5). Subgroup analysis of the 3 studies with a quality rating of moderate²⁶^,³⁶^,⁴⁰ revealed similar results to the overall finding (pooled RR: 1.20, 95%CI, 0.28–5.24, P = .6).

Studies evaluating targeted CDSSs

Four studies examined the addition of a targeted CDSS to an existing CPOE system²¹^,²²^,⁴⁴^,⁴⁵ and reported outcomes on ICU LOS, hospital LOS, ICU mortality, and hospital mortality (Table 2).

Table 2.

LOS and mortality findings in studies evaluating targeted CDSS

Author	Baseline			Intervention			P-value
Author	Duration, months	Patients (n)	LOS (day)	Duration	Patients (n)	LOS (day)
ICU LOS
Fernandez-Perez et al.²¹	12	1110	Median 1.7	12	1110	Median 1.7	.18
Fernandez-Perez et al.²¹	12	1110	IQR 0.9–2.9	12	1110	IQR 0.9–3.5	.18
Pageler et al.⁴⁵	12	818	Mean 5.1	12	1021	Mean 4.2	.05^*
Pageler et al.⁴⁵	12	818	SD 0.7	12	1021	SD 0.6	.05^*
Rana et al.²²	3	440	Median 1.9	3	403	Median 1.9	.36
Rana et al.²²	3	440	IQR 0.9–4.0	3	403	IQR 1.0–4.3	.36
Hospital LOS
Adams et al.⁴⁴	12	809	Mean 16.4	12	1044	Mean 11.5	.0002^*
Adams et al.⁴⁴	12	809	SD 31.3	12	1044	SD 25.8	.0002^*
Fernandez-Perez et al.²¹	12	1110	Median 9.3	12	1110	Median 9.5	.76
			Median 9.3			IQR 5.0–17.0
			IQR 6.0–17.0			IQR 5.0–17.0
Pageler et al.⁴⁵	12	818	Mean 16.8	12	1021	Mean 11.6	<.001^*
Pageler et al.⁴⁵	12	818	SD 2.1	12	1021	SD 1.6	<.001^*

	Duration	Patients (n)	Mortality^**	Duration	Patients (n)	Mortality^**	P-value
ICU Mortality
Fernandez-Perez et al.²¹	12	1110	0.05	12	1110	0.07	.21
Rana et al.²²	3	440	0.06	3	403	0.08	.25
Hospital Mortality
Adams et al.⁴⁴	12	809	0.04	12	1044	0.03	.38
Fernandez-Perez et al.²¹	12	1110	0.10	12	1110	0.13	.04^*
Pageler et al.⁴⁵	12	818	0.04	12	1021	0.03	.32
Rana et al.²²	3	440	0.12	3	403	0.15	.23

Open in a new tab

*Significant at 0.05 level.

**Mortality provided as rate per patient.

ICU = intensive care unit; LOS = length of stay; CDSS = clinical decision support system; n = number; IQR = interquartile range; SD = standard deviation.

A study that examined the impact of a rule restricting the scheduling of repeat orders (i.e., complete blood cell counts, chemistry, and coagulation studies within a 24-h interval) in a pediatric setting reported a significant decrease in both ICU LOS (from a mean of 5.1 days to 4.2 days) and hospital LOS (from a mean of 16.8 days to 11.6 days).⁴⁵ Another study in a pediatric setting also reported a significant decrease in hospital LOS (from a mean of 16.4 days to 11.5 days) following the introduction of a red blood cell transfusion algorithm.⁴⁴ The 2 studies conducted in adult ICUs did not find significant changes in ICU LOS²¹^,²² or hospital LOS²¹ following the addition of a targeted CDSS to an existing CPOE system.

Only 1 study reported a significant change in hospital mortality for ICU patients, from a rate of 0.10 deaths per patient to 0.13 deaths per patient, following the addition of a red blood cell transfusion algorithm to an existing CPOE system in an adult ICU.²¹ There were no significant findings among the other 3 studies²²^,⁴⁴^,⁴⁵ that examined hospital mortality, nor the 2 studies²¹^,²² that assessed ICU mortality.

Discussion

The transition from paper-based ordering to commercial CPOE systems in ICUs was found to be associated with an 85% reduction in medication prescribing error rates. This significant decrease in medication errors is consistent with reviews of CPOE implementation in other inpatient settings.^7–11 Overall meta-analysis of LOS and hospital mortality outcomes did not demonstrate a significant change following commercial CPOE implementation in ICU, which is also in line with other inpatient settings.¹⁰^,¹² However, analysis of ICU mortality showed CPOE implementation to be associated with a 12% mortality risk reduction in ICUs.

In 2005, Han et al.⁴⁰ reported the findings from a study that included 1102 pediatric patients admitted via interfacility transport directly to ICU: 790 patients during a 13-month baseline period and 312 patients during a 5-month post-CPOE implementation period. The findings revealed a significant increase in mortality among the cohort of patients following implementation of a commercial CPOE system; from 39 at baseline to 36 post-CPOE (P < .001).⁴⁰ While such findings are cause for concern, subsequent studies of critically ill patients, both in pediatric³⁰^,³⁶^,³⁹ and adult²⁶^,²⁸ ICUs, have not demonstrated any significant change in mortality following CPOE implementation. However, a consistent issue across all of these studies, including Han et al.,⁴⁰ is small sample sizes that may not be powered to detect a true effect.⁴⁶ Conversely, a 2010 study of a commercial CPOE by Longhurst et al.⁴³ implemented at a pediatric hospital found a significant decrease in mortality. The study included a substantial sample of 80 063 patient discharges spanning a 6-year baseline period and 17 432 patient discharges during the 18-month post-CPOE implementation period. They found that the mean monthly adjusted hospitalwide mortality rate decreased by 20%. Within the current review, the individual studies that assessed ICU mortality did not demonstrate an effect. Increasing the statistical power through meta-analysis found a positive effect (even with the use of a conservative approach) of ICU mortality reduction in critically ill populations following CPOE implementation. These findings highlight the importance of future studies that include larger sample sizes that are sufficiently powered to accurately and reliably detect clinically relevant rates of change in important indicators following CPOE system introduction.

Han et al.’s⁴⁰ study also served to demonstrate the need to monitor outcomes following system implementation, particularly as it is now well recognized that system implementations can result in unanticipated work process changes and unintended consequences.^47–49 Han et al., for example, suggested that the negative outcomes they identified were affected by a combination of order delays due to the inability to “pre-register” patients into the system, the increased time required to enter orders at computer terminals located away from the patient bedside, the reduction of staff interaction, and delays in medication administration due to the relocation of drugs from the ward to a centralized pharmacy service. Another unintended consequence of CPOE implementation can be the emergence of new system-related errors.⁵⁰^,⁵¹ Among the 10 studies included in this review that examined medication errors, only a few assessed the errors that occurred following the implementation of CPOE and identified duplicate prescriptions and erroneous selection from dropdown menus as new system-related errors. These few studies also found that the severity of errors changed, with the frequency of serious errors decreasing by > 50%. The limited evidence base of the types of errors and potential new risks occurring in critical care settings post-CPOE implementation underscores the importance of future research that not only quantifies the changes in error rates and patient outcomes but endeavors to understand the nature of these changes. Such information is critical to ongoing improvement in the design of CPOE systems and the delivery of safe patient care.

While we identified a significant overall reduction in medication-prescribing error rates following CPOE implementation, we also found that the frequency of medication errors found in each independent study varied substantially. This was particularly evident in baseline error rates, which ranged between 4.5% and 58.2%, whereas post-CPOE error rates ranged between 0% and 8.2%. Inherent differences between study settings may account for some of this variation. However, differences among definitions as to what constitutes a medication prescription error, as well the methods used to detect errors, are the likely cause of the majority of disparity between studies.⁶^,⁵² Some studies, for example, indicated that missing weight or no signature constituted an error of omission and some included rule violations, while other studies did not list these elements in their error definitions. These findings reiterate previous calls for the need to use a more standardized set of criteria when defining and reporting medication errors. Reckmann et al.,⁵² for example, suggest that future studies should include a clear definition of prescribing errors; absolute error rates pre- and post-CPOE; appropriate denominators, such as total number of orders; proportions of errors categorized according to a standardized severity scale; and appropriate significance testing. Such information would facilitate more accurate comparison between studies.

The significant reduction in medication error rates following CPOE implementation is not surprising. The automation and standardization of the format and structure of electronic orders intrinsically eliminates some error types, such as legibility errors. While eliminating these types of errors is important, the greater challenge is enabling appropriate, evidence-based care. It is here that the implementation of comprehensive CPOE applications that include sophisticated CDSSs are anticipated to have the greatest impact on errors and adverse outcomes.⁷ However, there is currently very limited evidence on the impact of adding targeted CDSSs into existing commercial CPOE systems in ICUs. Among the 4 studies we identified, the study findings of patient outcomes proved to be mixed. While studies found that the implementation of CDSSs enhanced the adoption of evidence-based recommendations, this positive impact on the process of care in ICUs did not necessarily translate into improved patient outcomes. Adams et al.⁴⁴ and Pageler et al.⁴⁵ reported significant decreases in LOS following the addition of CDSSs, while other studies found no change. With regard to mortality, Fernandez Perez et al.²¹ reported an increase in unadjusted hospital mortality, but no change in ICU mortality, while the other studies found no change in mortality following the addition of CDSSs. This suggests the need for more sophisticated multilevel statistical approaches in a much needed area of research, as examinations of mortality and LOS need to account for many complex variables, including acuity and patient demographics.

Limitations

An inherent limitation of systematic reviews is that the soundness of the review findings is reliant on the quality of the included studies. We rated the quality of the studies included in this review as either methodologically strong, moderate, or weak based on the EPHPP quality assessment tool criteria.¹⁹ While we found the majority of studies to be of moderate quality (13 of 20 studies), there were no studies rated as strong. As such, in addition to evaluating the findings from the current evidence base, our study also highlights key areas where there is a need for more robust studies with larger sample sizes in order to ascertain the true effect of the implementation of CPOE systems and CDSSs. While previous systematic reviews on the impact of CPOE in other hospital settings have found the evidence to be largely US-based as a result of including homegrown systems,¹¹ a strength of our review is that we focused on commercial systems. As such, the evidence base we identified was more global, with half of the included studies conducted outside the US, making it more applicable to international settings.

Conclusion

Critical care settings, both adult and pediatric, involve unique complexities that make them vulnerable to medication errors and adverse patient outcomes. While limited, the current evidence base suggests that the implementation of commercial CPOE systems can significantly decrease the frequency of medication prescribing error rates, as well as reducing the risk of mortality in ICUs. Future studies that aim to examine medication errors and patient outcomes should ensure they have sufficient sample sizes that are powered to identify the true effect of CPOE implementation. There is also a critical need to understand the nature of errors arising post-CPOE and how the addition of advanced CDSSs can be used to provide even greater benefit to delivering safe and effective patient care.

Author Contributions

MP led the systematic search, interpretation, and writing. LL led the meta-analysis. MP, ZN, LL, and JW reviewed articles for inclusion. All authors contributed to the study design, and protocol development, and participated in the writing and editing of several versions of this manuscript.

Conflicts of Interest

None to declare.

Funding

This work was funded by a National Health and Medical Research Council program grant (APP1054146).

SUPPLEMENTARY MATERIAL

Supplementary material are available at Journal of the American Medical Informatics Association online.

Supplementary Material

Supplementary Data

Click here for additional data file.^{(42.8KB, zip)}

REFERENCES

1. Institute of Medicine. Preventing Medication Errors. Washington, DC: The National Academies Press; 2007. [Google Scholar]
2. Classen DC, Pestotnik SL, Evans R, et al. Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA 1997;277(4):301–6. [PubMed] [Google Scholar]
3. Hug BL, Keohane C, Seger DL, et al. The costs of adverse drug events in community hospitals. Jt Comm J Qual Patient Saf 2012;38(3):120–6. [DOI] [PubMed] [Google Scholar]
4. Rothschild JM, Landrigan CP, Cronin JW, et al. The critical care safety study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med 2005;33(8):1694–700. [DOI] [PubMed] [Google Scholar]
5. Cullen DJ, Sweitzer BJ, Bates DW, et al. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med 1997;25(8):1289–97. [DOI] [PubMed] [Google Scholar]
6. Moyen E, Camiré E, Stelfox HT. Clinical review: medication errors in critical care. Crit Care 2008;12(2):208. [DOI] [PMC free article] [PubMed] [Google Scholar]
7. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163(12):1409–16. [DOI] [PubMed] [Google Scholar]
8. Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008;15(5):585–600. [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Nuckols TK, Smith-Spangler C, Morton SC, et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014;3:56. [DOI] [PMC free article] [PubMed] [Google Scholar]
10. van Rosse F, Maat B, Rademaker CM, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009;123(4):1184–90. [DOI] [PubMed] [Google Scholar]
11. Georgiou A, Prgomet M, Paoloni R, et al. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med 2013;61(6):644–53. e16. [DOI] [PubMed] [Google Scholar]
12. Thompson G, O’Horo JC, Pickering BW, et al. Impact of the electronic medical record on mortality, length of stay, and cost in the hospital and ICU: a systematic review and metaanalysis. Crit Care Med 2015;43(6):1276–82. [DOI] [PubMed] [Google Scholar]
13. Classen DC, Bates DW. Finding the meaning in meaningful use. N Engl J Med 2011;365(9):855–8. [DOI] [PubMed] [Google Scholar]
14. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009;62(10):1006–12. [DOI] [PubMed] [Google Scholar]
15. Wright A, Sittig DF, Ash JS, et al. Clinical decision support capabilities of commercially-available clinical information systems. J Am Med Inform Assoc 2009;16(5):637–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Levick DL, Stern G, Meyerhoefer CD, et al. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis Mak 2013;13(1):43. [DOI] [PMC free article] [PubMed] [Google Scholar]
17. Abstoss KM, Shaw BE, Owens TA, et al. Increasing medication error reporting rates while reducing harm through simultaneous cultural and system-level interventions in an intensive care unit. BMJ Qual Saf 2011;20(11):914–22. [DOI] [PubMed] [Google Scholar]
18. Sintchenko V, Iredell JR, Gilbert GL, et al. Handheld computer-based decision support reduces patient length of stay and antibiotic prescribing in critical care. J Am Med Inform Assoc 2005;12(4):398–402. [DOI] [PMC free article] [PubMed] [Google Scholar]
19. Thomas H. Quality Assessment Tool for Quantitative Studies. McMaster University, Toronto: Effective Public Health Practice Project; 2003. [Google Scholar]
20. Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7(27):1–173. [DOI] [PubMed] [Google Scholar]
21. Fernandez Perez ER, Winters JL, Gajic O. The addition of decision support into computerized physician order entry reduces red blood cell transfusion resource utilization in the intensive care unit. Am J Hematol 2007;82(7):631–3. [DOI] [PubMed] [Google Scholar]
22. Rana R, Afessa B, Keegan MT, et al. Evidence-based red cell transfusion in the critically ill: quality improvement using computerized physician order entry. Crit Care Med 2006;34(7):1892–7. [DOI] [PubMed] [Google Scholar]
23. Colpaert K, Claus B, Somers A, et al. Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care 2006;10(1):R21. [DOI] [PMC free article] [PubMed] [Google Scholar]
24. Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol 2005;5:13. [DOI] [PMC free article] [PubMed] [Google Scholar]
25. Thompson W, Dodek PM, Norena M, et al. Computerized physician order entry of diagnostic tests in an intensive care unit is associated with improved timeliness of service. Crit Care Med 2004;32(6):1306–9. [DOI] [PubMed] [Google Scholar]
26. Al-Dorzi HM, Tamim HM, Cherfan A, et al. Impact of computerized physician order entry (CPOE) system on the outcome of critically ill adult patients: a before-after study. BMC Med Inf Decis Mak 2011;11:71. [DOI] [PMC free article] [PubMed] [Google Scholar]
27. Ali J, Barrow L, Vuylsteke A. The impact of computerised physician order entry on prescribing practices in a cardiothoracic intensive care unit. Anaesthesia 2010;65(2):119–23. [DOI] [PubMed] [Google Scholar]
28. Armada ER, Villamanan E, Lopez-de-Sa E, et al. Computerized physician order entry in the cardiac intensive care unit: effects on prescription errors and workflow conditions. J Crit Care 2014;29(2):188–93. [DOI] [PubMed] [Google Scholar]
29. Warrick C, Naik H, Avis S, et al. A clinical information system reduces medication errors in paediatric intensive care. Intensive Care Med 2011;37(4):691–4. [DOI] [PubMed] [Google Scholar]
30. Kadmon G, Bron-Harlev E, Nahum E, et al. Computerized order entry with limited decision support to prevent prescription errors in a PICU. Pediatrics 2009;124(3):935–40. [DOI] [PubMed] [Google Scholar]
31. Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med 2003;22(17):2639–710. [DOI] [PubMed] [Google Scholar]
32. Higgins JPT, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327(7414):557–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
33. Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315(7109):629–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
34. Cordero L, Kuehn L, Kumar RR, et al. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol 2004;24(2):88–93. [DOI] [PubMed] [Google Scholar]
35. Jozefczyk KG, Kennedy WK, Lin MJ, et al. Computerized prescriber order entry and opportunities for medication errors: comparison to tradition paper-based order entry. J Pharm Pract 2013;26(4):434–7. [DOI] [PubMed] [Google Scholar]
36. Keene A, Ashton L, Shure D, et al. Mortality before and after initiation of a computerized physician order entry system in a critically ill pediatric population. Pediatr Crit Care Med 2007;8(3):268–71. [DOI] [PubMed] [Google Scholar]
37. R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. [Google Scholar]
38. Carayon P, Wood KE. CPOE Implementation in ICUs. Rockville, MD: The Agency for Healthcare Research and Quality; 2009. [Google Scholar]
39. Del Beccaro MA, Jeffries HE, Eisenberg MA, et al. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 2006;118(1):290–5. [DOI] [PubMed] [Google Scholar]
40. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116(6):1506–12. [DOI] [PubMed] [Google Scholar]
41. Potts AL, Barr FE, Gregory DF, et al. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics 2004;113(1 Pt 1):59–63. [DOI] [PubMed] [Google Scholar]
42. Shulman R, Singer M, Goldstone J, et al. Medication errors: a prospective cohort study of hand-written and computerised physician order entry in the intensive care unit. Crit Care 2005;9(5):R516–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
43. Longhurst CA, Parast L, Sandborg CI, et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics 2010;126(1):14–21. [DOI] [PubMed] [Google Scholar]
44. Adams ES, Longhurst CA, Pageler N, et al. Computerized physician order entry with decision support decreases blood transfusions in children. Pediatrics 2011;127(5):e1112–9. [DOI] [PubMed] [Google Scholar]
45. Pageler NM, Franzon D, Longhurst CA, et al. Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization. Pediatr Crit Care Med 2013;14(4):413–9. [DOI] [PubMed] [Google Scholar]
46. Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 2013;14(5):365–76. [DOI] [PubMed] [Google Scholar]
47. Ash JS, Sittig DF, Dykstra RH, et al. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007;76:S21–7. [DOI] [PubMed] [Google Scholar]
48. Bloomrosen M, Starren J, Lorenzi NM, et al. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc 2011;18(1):82–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
49. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care - an interactive sociotechnical analysis. J Am Med Inform Assoc 2007;14(5):542–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
50. Westbrook JI, Baysari MT, Li L, et al. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013;20(6):1159–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
51. Westbrook JI, Reckmann M, Li L, et al. Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients: a before and after study. PLoS Med 2012;9(1):e1001164. [DOI] [PMC free article] [PubMed] [Google Scholar]
52. Reckmann MH, Westbrook JI, Koh Y, et al. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009;16(5):613–23. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data

Click here for additional data file.^{(42.8KB, zip)}

[ocw145-B1] 1. Institute of Medicine. Preventing Medication Errors. Washington, DC: The National Academies Press; 2007. [Google Scholar]

[ocw145-B2] 2. Classen DC, Pestotnik SL, Evans R, et al. Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA 1997;277(4):301–6. [PubMed] [Google Scholar]

[ocw145-B3] 3. Hug BL, Keohane C, Seger DL, et al. The costs of adverse drug events in community hospitals. Jt Comm J Qual Patient Saf 2012;38(3):120–6. [DOI] [PubMed] [Google Scholar]

[ocw145-B4] 4. Rothschild JM, Landrigan CP, Cronin JW, et al. The critical care safety study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med 2005;33(8):1694–700. [DOI] [PubMed] [Google Scholar]

[ocw145-B5] 5. Cullen DJ, Sweitzer BJ, Bates DW, et al. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med 1997;25(8):1289–97. [DOI] [PubMed] [Google Scholar]

[ocw145-B6] 6. Moyen E, Camiré E, Stelfox HT. Clinical review: medication errors in critical care. Crit Care 2008;12(2):208. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B7] 7. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163(12):1409–16. [DOI] [PubMed] [Google Scholar]

[ocw145-B8] 8. Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008;15(5):585–600. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B9] 9. Nuckols TK, Smith-Spangler C, Morton SC, et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014;3:56. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B10] 10. van Rosse F, Maat B, Rademaker CM, et al. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009;123(4):1184–90. [DOI] [PubMed] [Google Scholar]

[ocw145-B11] 11. Georgiou A, Prgomet M, Paoloni R, et al. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med 2013;61(6):644–53. e16. [DOI] [PubMed] [Google Scholar]

[ocw145-B12] 12. Thompson G, O’Horo JC, Pickering BW, et al. Impact of the electronic medical record on mortality, length of stay, and cost in the hospital and ICU: a systematic review and metaanalysis. Crit Care Med 2015;43(6):1276–82. [DOI] [PubMed] [Google Scholar]

[ocw145-B13] 13. Classen DC, Bates DW. Finding the meaning in meaningful use. N Engl J Med 2011;365(9):855–8. [DOI] [PubMed] [Google Scholar]

[ocw145-B14] 14. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009;62(10):1006–12. [DOI] [PubMed] [Google Scholar]

[ocw145-B15] 15. Wright A, Sittig DF, Ash JS, et al. Clinical decision support capabilities of commercially-available clinical information systems. J Am Med Inform Assoc 2009;16(5):637–44. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B16] 16. Levick DL, Stern G, Meyerhoefer CD, et al. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis Mak 2013;13(1):43. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B17] 17. Abstoss KM, Shaw BE, Owens TA, et al. Increasing medication error reporting rates while reducing harm through simultaneous cultural and system-level interventions in an intensive care unit. BMJ Qual Saf 2011;20(11):914–22. [DOI] [PubMed] [Google Scholar]

[ocw145-B18] 18. Sintchenko V, Iredell JR, Gilbert GL, et al. Handheld computer-based decision support reduces patient length of stay and antibiotic prescribing in critical care. J Am Med Inform Assoc 2005;12(4):398–402. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B19] 19. Thomas H. Quality Assessment Tool for Quantitative Studies. McMaster University, Toronto: Effective Public Health Practice Project; 2003. [Google Scholar]

[ocw145-B20] 20. Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7(27):1–173. [DOI] [PubMed] [Google Scholar]

[ocw145-B21] 21. Fernandez Perez ER, Winters JL, Gajic O. The addition of decision support into computerized physician order entry reduces red blood cell transfusion resource utilization in the intensive care unit. Am J Hematol 2007;82(7):631–3. [DOI] [PubMed] [Google Scholar]

[ocw145-B22] 22. Rana R, Afessa B, Keegan MT, et al. Evidence-based red cell transfusion in the critically ill: quality improvement using computerized physician order entry. Crit Care Med 2006;34(7):1892–7. [DOI] [PubMed] [Google Scholar]

[ocw145-B23] 23. Colpaert K, Claus B, Somers A, et al. Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care 2006;10(1):R21. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B24] 24. Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol 2005;5:13. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B25] 25. Thompson W, Dodek PM, Norena M, et al. Computerized physician order entry of diagnostic tests in an intensive care unit is associated with improved timeliness of service. Crit Care Med 2004;32(6):1306–9. [DOI] [PubMed] [Google Scholar]

[ocw145-B26] 26. Al-Dorzi HM, Tamim HM, Cherfan A, et al. Impact of computerized physician order entry (CPOE) system on the outcome of critically ill adult patients: a before-after study. BMC Med Inf Decis Mak 2011;11:71. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B27] 27. Ali J, Barrow L, Vuylsteke A. The impact of computerised physician order entry on prescribing practices in a cardiothoracic intensive care unit. Anaesthesia 2010;65(2):119–23. [DOI] [PubMed] [Google Scholar]

[ocw145-B28] 28. Armada ER, Villamanan E, Lopez-de-Sa E, et al. Computerized physician order entry in the cardiac intensive care unit: effects on prescription errors and workflow conditions. J Crit Care 2014;29(2):188–93. [DOI] [PubMed] [Google Scholar]

[ocw145-B29] 29. Warrick C, Naik H, Avis S, et al. A clinical information system reduces medication errors in paediatric intensive care. Intensive Care Med 2011;37(4):691–4. [DOI] [PubMed] [Google Scholar]

[ocw145-B30] 30. Kadmon G, Bron-Harlev E, Nahum E, et al. Computerized order entry with limited decision support to prevent prescription errors in a PICU. Pediatrics 2009;124(3):935–40. [DOI] [PubMed] [Google Scholar]

[ocw145-B31] 31. Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med 2003;22(17):2639–710. [DOI] [PubMed] [Google Scholar]

[ocw145-B32] 32. Higgins JPT, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327(7414):557–60. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B33] 33. Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315(7109):629–34. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B34] 34. Cordero L, Kuehn L, Kumar RR, et al. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol 2004;24(2):88–93. [DOI] [PubMed] [Google Scholar]

[ocw145-B35] 35. Jozefczyk KG, Kennedy WK, Lin MJ, et al. Computerized prescriber order entry and opportunities for medication errors: comparison to tradition paper-based order entry. J Pharm Pract 2013;26(4):434–7. [DOI] [PubMed] [Google Scholar]

[ocw145-B36] 36. Keene A, Ashton L, Shure D, et al. Mortality before and after initiation of a computerized physician order entry system in a critically ill pediatric population. Pediatr Crit Care Med 2007;8(3):268–71. [DOI] [PubMed] [Google Scholar]

[ocw145-B37] 37. R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. [Google Scholar]

[ocw145-B38] 38. Carayon P, Wood KE. CPOE Implementation in ICUs. Rockville, MD: The Agency for Healthcare Research and Quality; 2009. [Google Scholar]

[ocw145-B39] 39. Del Beccaro MA, Jeffries HE, Eisenberg MA, et al. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 2006;118(1):290–5. [DOI] [PubMed] [Google Scholar]

[ocw145-B40] 40. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005;116(6):1506–12. [DOI] [PubMed] [Google Scholar]

[ocw145-B41] 41. Potts AL, Barr FE, Gregory DF, et al. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics 2004;113(1 Pt 1):59–63. [DOI] [PubMed] [Google Scholar]

[ocw145-B42] 42. Shulman R, Singer M, Goldstone J, et al. Medication errors: a prospective cohort study of hand-written and computerised physician order entry in the intensive care unit. Crit Care 2005;9(5):R516–21. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B43] 43. Longhurst CA, Parast L, Sandborg CI, et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics 2010;126(1):14–21. [DOI] [PubMed] [Google Scholar]

[ocw145-B44] 44. Adams ES, Longhurst CA, Pageler N, et al. Computerized physician order entry with decision support decreases blood transfusions in children. Pediatrics 2011;127(5):e1112–9. [DOI] [PubMed] [Google Scholar]

[ocw145-B45] 45. Pageler NM, Franzon D, Longhurst CA, et al. Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization. Pediatr Crit Care Med 2013;14(4):413–9. [DOI] [PubMed] [Google Scholar]

[ocw145-B46] 46. Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 2013;14(5):365–76. [DOI] [PubMed] [Google Scholar]

[ocw145-B47] 47. Ash JS, Sittig DF, Dykstra RH, et al. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007;76:S21–7. [DOI] [PubMed] [Google Scholar]

[ocw145-B48] 48. Bloomrosen M, Starren J, Lorenzi NM, et al. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 Health Policy Meeting. J Am Med Inform Assoc 2011;18(1):82–90. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B49] 49. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care - an interactive sociotechnical analysis. J Am Med Inform Assoc 2007;14(5):542–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B50] 50. Westbrook JI, Baysari MT, Li L, et al. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013;20(6):1159–67. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B51] 51. Westbrook JI, Reckmann M, Li L, et al. Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients: a before and after study. PLoS Med 2012;9(1):e1001164. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocw145-B52] 52. Reckmann MH, Westbrook JI, Koh Y, et al. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009;16(5):613–23. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis

Mirela Prgomet

Ling Li

Zahra Niazkhani

Andrew Georgiou

Johanna I Westbrook

Abstract

Introduction

Method

Search strategy

Study selection

Figure 1.

Data extraction

Quality assessment

Statistical analysis

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Results

Study characteristics

Table 1.

Studies comparing CPOE to paper

Medication errors

ICU LOS

ICU mortality

Hospital mortality

Studies evaluating targeted CDSSs

Table 2.

Discussion

Limitations

Conclusion

Author Contributions

Conflicts of Interest

Funding

SUPPLEMENTARY MATERIAL

Supplementary Material

REFERENCES

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases