Associations between the concurrent use of clinical decision support and computerized provider order entry and the rates of appropriate prescribing at discharge

Abstract

Introduction

Electronic health record systems used in conjunction with clinical decision support (CDS) or computerized provider order entry (CPOE) have shown potential in improving quality of care, yet less is known about the effects of combination use of CDS and CPOE on prescribing rates at discharge.

Objectives

This study investigates the effectiveness of combination use of CDS and CPOE on appropriate drug prescribing rates at discharge for AMI or HF patients.

Methods

Combination use of CDS and CPOE is defined as hospitals self-reporting full implementation across all hospital units of CDS reminders, CDS guidelines, and CPOE. Appropriate prescribing rates of aspirin, ACEI/ARBs, or beta blockers are defined using quality measures from Hospital Compare. Multivariate linear regressions are used to test for differences in mean appropriate prescribing rates between hospitals reporting combination use of CDS and CPOE, compared to those reporting the singular use of one or the other, or the absence of both. Covariates include hospital size, region, and ownership status.

Results

Approximately 10% of the sample reported full implementation of both CDS and CPOE, while 7% and 17% reported full use of only CPOE or only CDS, respectively. Hospitals reporting full use of CDS only reported between 0.2% (95% CI 0.04 – 1.0) and 1.6% (95% CI 0.6 – 2.6) higher appropriate prescribing rates compared to hospitals reporting use of neither system. Rates of prescribing by hospitals reporting full use of both CPOE and CDS did not significantly differ from the control group.

Conclusions

Although associations found between full implementation of CDS and appropriate prescribing rates suggest that clinical decision tools are sufficient compared to basic EHR systems in improving prescribing at discharge, the modest differences raise doubt about the clinical relevance of the findings. Future studies need to continue investigating the causal nature and clinical relevance of these associations.

1. Introduction

The optimism and promise surrounding the potential of electronic health record (EHR) systems to significantly improve the quality and delivery of healthcare in the United States continues to grow despite studies showing mixed results as to electronic health records’ (EHRs) clinical effectiveness. Some studies have found that EHR implementation significantly reduces medication errors [1–4], complications [5], mortality [6–7], or inappropriate prescribing [6]. However, more recent studies conducted in nationally representative samples of hospitals found that basic or comprehensive EHR use was not associated with higher quality measure scores for acute myocardial infarction (AMI), or heart failure (HF) [6], or appropriate prescribing rates [8]. These inconsistent findings deserve further explanation because EHR systems were defined differently, making it difficult to attribute associations to the use of EHR systems in general or to the use of specific functionalities within EHR such as computerized provider order entry (CPOE) or clinical decision support (CDS).

Another important question is whether the combination of use of CPOE and CDS is clinically effective. If the functions of CPOE and CDS are to improve ordering and clinical decision-making, respectively, examining the concurrent use versus the isolated use of these systems would provide insight as to which both of these processes are necessary and/or sufficient to improve appropriate prescribing. One study finding that 56% and 34% of inpatient ADEs occurred at the ordering and administrative stages respectively [9] implies that EHRs incorporating both CPOE and CDS should in turn improve prescribing better than one system alone.

Prior studies’ results are consistent with this argument. The use of both compared to one alone, for example, led to greater reduction of medication errors [10]. Furthermore, CPOE and CDS at point of care has been found to reduce medication errors and near misses [11], findings consistent with systematic reviews concluding that CDS improve patient outcomes [12], practitioner performance [12], and rates of preventive care [13]. Although prior studies using the American Hospital Association health IT survey data have examined associations between implementation levels of EHRs configured with either CPOE or CDS and quality of care [6] or prescribing rates [8], none using this data have directly compared the clinical effectiveness of EHR systems using CPOE and CDS in combination versus those with only one or the other, or neither.

2. Objectives

The goal of this analysis is to examine how concurrent use of CPOE for medications and CDS guidelines and reminders compared to the use of one or the other in isolation is related to appropriate prescribing rates at discharge. This goal is achieved by measuring the associations between hospitals’ combination versus isolated use of CPOE and CDS on appropriate prescribing rates of aspirin (ASA), angiotensin converting enzyme inhibitor (ACE), angiotensin II receptor blockers (ARBs) or beta blockers for HF and AMI patients at discharge. It is hypothesized that compared to hospitals not fully implementing in all units at least one or more CDS or CPOE systems, those fully implementing both CPOE and at least one CDS tool will have higher appropriate prescribing rates at discharge.

3. Methods

This cross sectional analysis uses survey data from the 2008 American Hospital Association (AHA) Hospital EHR Adoption Database [14] to obtain implementation levels of CDS clinical reminders, CDS clinical guidelines, or CPOE. The survey’s intent was to measure EHR functionality levels of member AHA hospitals, and was administered to an employee identified by the hospital’s CEO who was deemed knowledgeable about the health IT system implementation levels. The survey response rate among AHA members was approximately 63% [15].

Appropriate prescribing rate data was obtained from the Hospital Compare website [16]. The Department of Health and Human Services created this reporting system in 2004 in order to collect information from acute-care hospitals on conformance to key quality measures as endorsed by the National Committee of Quality Assurance (NCQA), the National Quality Forum (NQF), and The Joint Commission (TJC). Acute care hospitals voluntarily submit quarterly data in order to receive incentive payments from Medicare. In contrast, acute care hospitals classified as critical access hospitals are not eligible to receive incentive payments and as such may not elect to report data [17]. To ensure accurate performance data in the Hospital Compare database, CMS requires hospitals to submit data to a Quality Improvement Organization (QIO) by using CMS-endorsed medical record abstraction tools or by working with a TJC-certified vendor. Since both of these processes employ auditing procedures, outliers and missing data should be minimized. [18] Measures of interest for this analysis include prescribing rate performance measures for acute myocardial infarction (AMI) and heart failure (HF). Approximately 73% (N = 3,451) of AHA member hospitals (N = 4,726) also report data to Hospital Compare. Furthermore, approximately 78% (N = 2,719) of the health IT survey respondents also reported data to Hospital Compare.

Analytic datasets for each respective prescribing rate were formed by merging the AHA Health IT survey data with the Hospital Compare survey datasets. The AHA 2008 Health IT survey data, collected from hospitals between March 2008 and October 2008, were merged with the Hospital Compare data, collected during time period October 2008 through November 2009, resulting in 2,719 institutions. We included only general medical and surgical acute care hospitals, resulting in 2,646 hospitals. Hospitals having missing information on CDS or CPOE implementation levels or located in US territories were excluded from the analysis, resulting in 2,489 hospitals in the base sample.

A four-level categorical variable was constructed from the AHA health IT survey data to classify these 2,489 hospitals into 4 mutually exclusive groups, including those reporting full implementation of

• 1.both CDS and CPOE;
• 2.CDS only;
• 3.CPOE;
• 4.neither CDS nor CPOE.

Full implementation, according to the AHA survey, was defined as hospitals that had fully integrated these electronic systems and no longer relied on paper records [13]. This was captured for CDS by hospitals responding ‘Fully implemented across all units’ to both of the following questions:

• 1.Does your hospital have a computerized system for decision support? And
• 2.Does your hospital have computerized system for clinical reminders?

Full implementation was similarly captured for CPOE by hospitals responding ‘Fully implemented across all units’ to the following one question: “Does your hospital have a computerized system for computerized provider order entry for medications?” All other hospitals responding that their CPOE or CDS systems were either partially implemented, planning to be implemented, considering to be implemented, or not implemented were categorized into the control group.

The following quality measures obtained from the Hospital Compare website were used as appropriate prescribing rates:

• 1.ASA for AMI patients,
• 2.ACEI or ARBs prescribed for AMI patients;
• 3.beta blockers prescribed for AMI patients, and
• 4.ACEI or ARBs prescribed for HF patients.

These medication indicators were chosen based on evidence that these medications improve patient outcomes in AMI and HF. As such, associations found between EHR systems and evidence-based measures would provide indirect evidence of EHR impacting patient outcomes as well as appropriate prescribing rates. The hospital-level performance rates present in the Hospital Compare database are calculated by dividing the numerator by the denominator. While the denominator is the sum of all eligible cases, the numerator is the sum of all eligible cases submitted for the same reporting period where recommended care was provided [19]. For example, a heart failure patient with left ventricular systolic dysfunction without contraindications to ACE/ARBs who is prescribed an ACE/ARB at discharge would be classified as a patient appropriately receiving care, and hence would count toward both the numerator and the denominator. Subsequently, a hospital that appropriately prescribes at discharge ACE/ARBs to 98 out of 100 heart failure patients would receive a score of 98%.

Multivariate linear regressions were used for each of the pharmacy quality measures to estimate the mean differences of appropriate prescribing rates between hospitals reporting full implementation of both CDS and CPOE, either CDS or CPOE, or neither. These differences were adjusted by including hospital, census region, size, and ownership status as covariates in the multivariate models. In order to account for effects on appropriate prescribing of additional EHR components beyond CPOE for medications and CDS reminders and guidelines, a vector of binary indicators that captured hospitals’ additional EHR capabilities were also added to the model. These 4 binary indicators correspond to whether a hospital reported full implementation of

• 1.electronic documentation,
• 2.results viewing,
• 3.other CPOE or
• 4.CDS functionalities apart from those already included in the predictor.

These four domains together contain 24 specific EHR functionalities that comprise what has been defined in a previous study as a ‘comprehensive electronic health record’ [20]. For purposes of this analysis, these 4 indicators include only 21 of the 24 functionalities, since CDS reminders, CDS guidelines, and CPOE for medications are already included in the predictor of interest. A hospital will full implementation of either of these 4 dimensions had to have reported full implementation of each functionality contained within this category, details of which are displayed in ►Table 1 All statistical analyses were conducted using SASv9.2 (Cary, NC). Since this study does not directly involve human subjects, the Institutional Review Board (IRB) at University of Missouri-Kansas City School of Pharmacy classified this study as exempt.

Table 1.

Definitions of the EHR sophistication covariates

1. Hospitals classified as fully implementing electronic clinical documentation have reported full implementation across all units of the following components	a.Patient demographics b.Physician notes c.Nursing assessments d.Problem lists e.Medication lists f.Discharge summaries g.Advanced directives
2. Hospitals classified as fully implementing Results Viewing have reported full implementation across all units of the following components	a.Lab reports b.Radiology reports c.Radiology images d.Diagnostic test results e.Diagnostic test images f.Consultant reports
3. Hospitals classified as fully implementing CPOE beyond CPOE for medications have reported full implementation across all units of the following components	a.Laboratory tests b.Radiology tests c.Consultation requests d.Nursing orders
4. Hospitals classified as fully implementing CDS beyond CDS for reminders and guidelines have reported full implementation across all units of the following components	a.Drug-allergy results b.Drug-drug interaction results c.Drug-lab interaction alerts d.Drug-dosing support

4. Results

Across the different combinations of clinical functionality implementation reported by hospitals in the sample, the most and least prevalent forms were CDS only and CPOE only, respectively, while the combination of both CDS and CPOE fell between. Approximately 10% of the sample reported full implementation of both CDS and CPOE across all units, while 7%, and 17% reported full used of only CPOE, or only CDS, respectively. This indicates that 66% of the hospital sample did not fully implement either CPOE or CDS. Approximately 27% reported full implementation of EHR results viewing as well as of CDS components apart from reminders and guidelines. Only 6% and 12% of the sample reported full implementation of electronic clinical documentation, and of CPOE components apart from medication CPOE, respectively. Most of the hospital respondents were either small or medium-sized, non-government non-for-profit entities located in the Central or Atlantic regions of the United States (►Table 2).

Table 2.

Organizational characteristics of hospitals participating in both Hospital Compare and AHA Health IT survey (N = 2,489).

	N (%)
Implementation levels of clinical functions
Full implementation of CDS and CPOE across all units	238 (10)
Full implementation of CDS only across all units	418 (17)
Full implementation of CPOE only across all units	178 (7)
Neither CPOE nor CDSS fully implemented across any unit	1655 (66)
Sophistication level of EHR
Full implementation Electronic Clinical Documentation1	162 (6)
Full implementation of results viewing2	667 (27)
Full implementation of other CPOE components3	295 (12)
Full implementation of other CDS components4	663 (27)
Ownership Status
Government, non-federal	592 (23)
Nongovernment, not-for-profit	1705 (65)
Investor owned, for-profit	322 (12)
Government, federal	10 (0.4)
Census Region
New England	121 (5)
Mid Atlantic	245 (10)
South Atlantic	388 (15)
East North Central	453 (18)
East South Central	180 (7)
West North Central	365 (15)
West South Central	339 (14)
Mountain	165 (7)
Pacific	233 (9)
Size of Hospital
Small (up to 99 beds)	1072 (43)
Medium (100 to 399 beds)	1119 (45)
Large (400 plus beds)	298 (12)

¹ Patient demographics, physician notes, nursing assessments, problem and medication lists, discharge summaries, and advanced directives

² Lab report, radiology reports, radiology images, diagnostic test results, diagnostic test images, consultant reports.

³ Laboratory tests, radiology tests, consultation results, nursing orders

⁴ Drug allergy alerts, drug-drug interaction alerts, drug-lab interaction results, drug-dosing support

In the unadjusted analyses, hospitals reporting full implementation of both CDS and CPOE in general had consistently higher appropriate prescribing rates, while those reporting full implementation of CPOE alone had the lowest appropriate prescribing rates (►Table 3) Adjusting for covariates, though, led to a mixture of attenuation or magnification of associations, of which only those between hospitals with CDS only and appropriate prescribing rates remained statistically significant. Compared to hospitals in the control group, those fully implementing CDS only reported between 0.2% (95% CI 0.04 – 1.0) and 1.6% (95% CI 0.6 – 2.6) higher appropriate prescribing rates. Compared to hospitals in the control group, the hospitals fully implementing both CDS and CPOE, or CPOE in the absence of CDS, had comparable prescribing rates across all measures, showing no statistically significant differences. Comparing the associations across prescribing rate measures with the CDS only group, the largest and smallest associations were found in the ASA and ACE/ARBs for AMI patients, respectively (►Table 3). Significant associations in the CDS only group were present in only the AMI measures and not the heart failure measures.

Table 3.

Differences in appropriate prescribing rates across hospitals with different combinations of CDS or CPOE system implementation levels.

Appropriate Prescribing Rate Measure	Mean (SD)	Mean Unadjusted Difference (%)	Mean Adjusted Difference (%)	95% CI	p-value
Aspirin prescribed at discharge for AMI patients (N=1158)
Full implementation of CDS and CPOE across all units (N=153)	98.6 (1.8)	0.9	0.2	(-0.9 – 1.3)	0.7
Full implementation of CDS only across all units (N=250)	98.2 (3.4)	0.5	0.2	(0.04 – 1.0)	0.03
Full implementation of CPOE only across all units (N=76)	97.9 (2.6)	0.3	–0.3	(-1.3 – 0.8)	0.6
Neither CPOE nor CDSS fully implemented across any unit (N=679)	97.6 (3.3)	.	.	.	.
Overall	97.9 (3.1)
ACE/ARBs prescribed at discharge for AMI patients (N=681)
Full implementation of CDS and CPOE across all units (N=107)	96.2 (3.7)	1.1	1.5	(-0.7 – 3.7)	0.2
Full implementation of CDS only across all units (N=147)	96.4 (4.2)	1.2	1.6	(0.6 – 2.6)	0.002
Full implementation of CPOE only across all units (N=41)	94.6 (6.3)	–0.6	–0.6	(-2.8 – 1.5)	0.5
Neither CPOE nor CDSS fully implemented across any units (N=386)	95.2 (4.9)	.	.	.	.
Overall	95.6 (4.7)
Beta blockers prescribed at discharge for AMI patients (N=1176)
Full implementation of CDS and CPOE across all units (N=155)	98.4 (2.7)	0.6	–0.3	(-1.4 – 0.8)	0.64
Full implementation of CDS only across all units (N=250)	98.5 (3.0)	0.7	0.7	(0.2 – 1.1)	0.01
Full implementation of CPOE only across all units (N=74)	97.6 (3.0)	–0.2	–0.9	(-1.9 – 0.2)	0.10
Neither CPOE nor CDSS fully implemented across any unit (N=697)	97.8 (3.2)	.	.	.	.
Overall	98.0 (3.1)
ACE/ARBs prescribed at discharge for HF patients (N=1408)
Full implementation of CDS and CPOE across all units (N=168)	95.3 (4.2)	1.8	–0.1	(-2.3 – 2.2)	0.93
Full implementation of CDS only across all units (N=304)	94.3 (6.7)	0.8	0.8	(-0.2 – 1.8)	0.10
Full implementation of CPOE only across all units (N=95)	93.5 (7.0)	0.03	–1.5	(-3.4 – 0.44)	0.13
Neither CPOE nor CDSS fully implemented across any units (N=841)	93.5 (7.0)	.	.	.	.
Overall	93.9 (6.7)

5. Discussion

Compared to hospitals not fully implementing either CPOE or CDS systems, those hospitals reporting full implementation of CDS only reported consistently higher, albeit modest, appropriate prescribing rates of ASA and ACE/ARBs, and beta blockers for AMI patients.

On the other hand, the hospitals fully implementing both CDS and CPOE, or CPOE in the absence of CDS did not significantly differ from the control group with respect to appropriate prescribing rates.

These findings suggest full implementation of CDS systems improve prescribing rates for AMI patients at discharge, yet full implementation of combination CPOE/CDS or CPOE alone may not be sufficient to improve appropriate prescribing rates. These results are consistent with studies showing that CDS improved patient outcomes [12], practitioner performance [12], and rates of preventive care [13], or that implementing either basic or comprehensive EHR in conjunction with CDS led to 2.2% and 0.6% higher quality of care indicator rates for HF and AMI care, respectively [6]. In contrast, these results are inconsistent with studies showing that combination use of CDS and CPOE reduced medication related errors [10] or that basic EHR with CPOE compared to basic EHR alone led to 1.1% higher appropriate prescribing rates of ACE/ARBs to heart failure patients at discharge [8].

Although we hypothesized that full implementation a combination CPOE/CDS compared to other systems would best improve appropriate prescribing by decreasing ordering errors in conjunction with providing clinical decision support, results suggest that clinical decision support alone may be sufficient to improve prescribing at discharge. Furthermore, some of our results are inconsistent given the presence of significant associations among hospitals with CDS alone but not with CDS with CPOE. This inconsistency could possibly be explained by the CDS only hospitals compared to CDS/CPOE hospitals either implementing CDS systems more specific to heart failure medications, or having a cardiac specialty center. If these scenarios were associated with improved prescribing, selection bias may indeed explain these inconsistent results.

Given the limitations of the study reviewed below, these results should be interpreted with caution. Several limitations worth noting may compromise the validity or the clinical significance of our findings. First, the cross-sectional design limits our ability to infer causality, especially since we do not know whether CDS implementation levels predicted prescribing behavior or vice versa. We attempted to minimize the effects of this potential source of endogeneity by using 2009 Hospital Compare measures which were collected during a time period predicting that of the AHA Health IT survey collection.

Second, unobserved hospital-level heterogeneity could explain the observed associations between CDS levels and appropriate prescribing rates. For example, if overall hospital quality is correlated with appropriate prescribing rates as well as CDS implementation levels, the observed associations may be attributed to overall hospital quality instead of CDS implementation levels. This limitation could also explain the inconsistent results between the CDS and CDS/CPOE hospitals, which could possibly be explained by the omission of cardiac care centers of heart failure-specific CDS reminders or guidelines. We attempted to minimize these sources of heterogeneity by including hospitals’ implementation levels of additional EHR components beyond CPOE for medication or CDS reminders or guidelines.

Third, the current outcome variable is most likely inflated due to the under-representation of low-performing hospitals in the Hospital Compare website. Since participation in Hospital Compare is voluntary, low-performers such as critical care hospitals may choose not to submit data, resulting in a sample that over-represents high performers. Additionally, the absence of CMS audits may increase the likelihood of fraudulent inflated reporting in order to seek additional reimbursement. While under-representing low performing hospitals may compromise external validity and ignoring fraudulent rates may compromise internal validity, both of these issues most likely result an upwardly biased outcome. Additionally, the former issue limits the applicability of our results to higher performing hospitals, and the latter issue results in biased parameter estimates, assumedly downward if hospitals with different health IT implementation levels do not significantly differ with respect to fraudulent reporting.

Fourth, even if these associations reflect a true relationship between CDS implementation levels and appropriate prescribing rates, we cannot determine the clinical impact since we do not know whether the modest differences in appropriate prescribing truly impacts clinical care.

Finally, since self-reported implementation levels, a limitation of the survey design, do not necessarily correlate with actual utilization, associations may not reflect associations due to actual CPOE or CDS utilization patterns.

Despite these limitations, this study adds to the literature by being the first to examine within a comparative effectiveness framework the associations between CDS/CPOE combinations and appropriate prescribing rates. The predictor creates mutually exclusive groups of hospitals, facilitating the ability to quantify the differences between appropriate prescribing rates of CDS/CPOE together, versus either of these alone, using a control group as a comparator. Accounting for the EHR system comprehensiveness as a covariate helps account for other clinical functionalities which may affect prescribing rates, hence decreasing any other sources of error as alternative explanation for the associations. Furthermore, this analytic approach allows us to compare whether improving prescription ordering accuracy through CPOE, or improving clinical decision support through CDS are sufficient by themselves, or require combination use in order to impact prescribing quality. Although these results suggest that clinical decision support may be sufficient, future studies that control better for actual use and unobserved heterogeneity need to be conducted to further investigate the causal nature of these associations.

6. Conclusion

Although results from this study suggest that hospitals fully implementing CDS may be sufficient to improve appropriate prescribing rates at discharge, the inconsistencies across performance measures, the presence of ceiling effects that obscures our ability to detect differences, and the presence unobserved heterogeneity all warrant caution in concluding CDS clinically impacts appropriate prescribing. Future studies using longitudinal data should further examine the causal nature of these associations by incorporating additional hospital-level system variables, and measures of actual CDS and CPOE use. As Congress continues to implement the HITECH legislative provisions in the upcoming months and years, more evidence is needed regarding the effectiveness of EHR systems on patient care, evidence which will help link these policies to better outcomes.

7. Clinical Relevance Statement

Although modest yet statistically significant associations found between CDS systems and appropriate prescribing suggest that CDS alone sufficiently improves ASA, ACEI/ARBs, and beta blocker prescribing rates for AMI patients at discharge, the modest differences of prescribing rates ranging between 0.2% and 1.6% may not constitute sufficient clinical impact to justify clinical effectiveness. Despite this, the presence of statistically significant associations in the presence of ceiling effects may suggest true associations which need to be investigated further in studies which examine hospitals with greater difference in baseline performance measures and better control for hospital quality measures or CDS/CPOE actual use.

Conflicts of Interest

Dr. Mark Patterson has received a competitive grant from the University of Missouri Research Board (UMRB) for work on a separate but related project investigating associations between electronic medical record systems, medication adherence, and hospitalization rates. All authors report no conflicts of interest.

Protection of human subjects

Since this study does not directly involve human subjects, Institutional Review Board (IRB) at University of Missouri-Kansas City School of Pharmacy classified this study as exempt.