EHR/CPOE systems improve completeness of medical record and chemotherapy order documentation, as well as user satisfaction with the medical record system.
Abstract
Purpose:
Computerized physician order entry (CPOE) in electronic health records (EHR) has been recognized as an important tool in optimal health care provision that can reduce errors and improve safety. The objective of this study is to describe documentation completeness and user satisfaction of medical charts before and after implementation of an outpatient oncology EHR/ CPOE system in a hospital-based outpatient cancer center within three treatment sites.
Methods:
This study is a retrospective chart review of 90 patients who received one of the following regimens between 1999 and 2006: FOLFOX, AC, carboplatin + paclitaxel, ABVD, cisplatin + etoposide, R-CHOP, and clinical trials. Documentation completeness scores were assigned to each chart based on the number of documented data points found out of the total data points assessed. EHR/CPOE documentation completeness was compared with completeness of paper charts orders of the same regimens. A user satisfaction survey of the paper chart and EHR/CPOE system was conducted among the physicians, nurses, and pharmacists who worked with both systems.
Results:
The mean percentage of identified data points successfully found in the EHR/CPOE charts was 93% versus 67% in the paper charts (P < .001). Regimen complexity did not alter the number of data points found. The survey response rate was 64%, and the results showed that satisfaction was statistically significant in favor of the EHR/CPOE system.
Conclusion:
Using EHR/CPOE systems improves completeness of medical record and chemotherapy order documentation and improves user satisfaction with the medical record system. EHR/CPOE requires constant vigilance and maintenance to optimize patient safety.
Introduction
In the high-risk setting of oncology, medication error prevention is a top priority, but despite heightened focus, chemotherapy errors still occur.1 Computerized physician order entry (CPOE) has been recognized as an important tool in optimal health care provision that can reduce medication errors and improve patient safety.2,3 By incorporating a CPOE program with clinical decision support into an electronic health record (EHR), the health care team can effectively and safely provide quality patient care. CPOE systems designed for the outpatient oncology setting have only recently become available and widely used. The relative inexperience with ambulatory oncology CPOE has resulted in a knowledge deficit concerning their impact on patient care.
The reluctance to adopt CPOE is a result of the many challenges associated with chemotherapy ordering in the outpatient oncology setting. A combination of high toxicity profiles and narrow therapeutic indexes for chemotherapy medications makes tolerance for system failures in the oncology setting extremely low. In addition, chemotherapy regimens are complex, with varying doses and schedules. The outpatient oncology setting also has a large proportion of patients on clinical trial protocols that use highly complex monitoring parameters and dosing schemes. Furthermore, experience with chemotherapy medication ordering with inpatient-based CPOE systems has revealed that medication errors still occur; therefore, pharmacy and nursing review of orders is still a critical component in the prevention of medication errors.1,4 Documentation completeness is an important factor in error prevention because omission of pertinent information in the medication order or patient's chart has been identified as a potential source of medication errors.1,5
To ensure the safety of oncology patients, Kellogg Cancer Center (KCC; Evanston, IL) is involved in continuous assessment and improvement of the EHR/CPOE system. The objectives of this study, conducted a year after CPOE implementation, were to compare the completeness of chemotherapy documentation in the EHR/CPOE system with the completenss of historic paper charts and evaluate staff satisfaction with the EHR/CPOE system. Results from this study, along with ongoing experience with the system, have helped identify areas for improvements in patient safety and workflow and have increased EHR utility across the continuum of cancer care.
Site
NorthShore University HealthSystem (NorthShore) is a four-hospital not-for-profit corporation in the northern suburbs of Chicago that comprises 894 inpatient beds and 75 different physician offices. The KCC is located at three hospitals in the NorthShore network: Evanston Hospital, Glenbrook Hospital, and Highland Park Hospital. Currently the KCC dispenses and administers approximately 12,000 doses for more than 1,150 patients each month.
Over the past 10 years, NorthShore has been transitioning to a complete EHR system (Epic, Verona, WI). In 2003 the CPOE portion was implemented for the inpatient population, and very shortly after for all documentation and orders in the outpatient setting. It has been fully operational for 7 years and is currently used for 100% of all inpatient orders (medications, procedures, and diagnostics). A separate outpatient oncology module within Epic (Beacon) was developed in response to the special requirements of the oncology population and was fully implemented by September 2005. Since that time, NorthShore has been in the process of continually upgrading and enhancing the CPOE system as issues and concerns are uncovered.
EHR and CPOE System
The EHR/CPOE workflow at KCC is designed to have multiple checks in place. Common chemotherapy regimens are built into the system through multidisciplinary collaboration and are double-checked for accuracy. The physician enters the regimen and treatment parameters into the CPOE system in a treatment plan and signs electronically. A “chemotherapy note” serves as a verification of the overall plan and verifies treatment plan and order intent. If the patient meets the treatment plan parameters, the nurse releases the orders from the treatment plan to pharmacy. Pharmacist verification and double-check of chemotherapy orders is enhanced by real-time access to the complete EHR, including laboratory values, patient vital signs, treatment parameters, progress notes, and clinical trial protocols. After the medication is prepared and dispensed, the nurse uses dual sign-off and bar coding to administer the medication.
Methods
Subjects
The first portion of the study was a retrospective chart review to compare the chemotherapy documentation completeness in the EHR/CPOE system with the completeness of paper charts. All patients treated with one of seven different chemotherapy regimens at any KCC site from January 1, 1999 to December 31, 1999 were eligible for inclusion in the paper chart arm. Patients treated in the same regimens from October 1, 2005 to September 30, 2006 were eligible for inclusion in the EHR/CPOE arm. The seven eligible regimens were as follows: FOLFOX (fluorouracil, oxaliplatin, leucovorin), AC (doxorubicin, cyclophosphamide), carboplatin plus paclitaxel, ABVD (doxorubicin, bleomycin, vinblastine, dacarbazine), cisplatin plus etoposide, R-CHOP (rituximab, cyclophosphamide, vincristine, doxorubicin), and clinical trials. These regimens were selected because they were standard regimens during the study time period that included both paper and CPOE charts and offered a wide range of complexity and cancer types. Patients were excluded if they received fewer than three cycles of one of the study regimens. Cycle 3 was used for data collection to assess everyday use of the systems, recognizing that cycle 1 orders may be inherently more complete, but that each cycle is independently important and a safety risk.
The second portion consisted of a KCC provider satisfaction survey in which each chart type was distributed 1 year after the implementation of the outpatient module. Providers were excluded if they had not worked with both paper and CPOE charts at NorthShore. This study was reviewed and approved by the institutional review board of NorthShore.
Data Collection
Reports were generated from the old system (paper) and from the new system (EHR/CPOE) to identify potential patients. A total of 90 patient charts (45 EHR/CPOE charts and 45 regimen matched paper charts) were chosen at random and reviewed.
The data collected were grouped into two different categories: general information and chemotherapy order data. General information was considered appropriately documented if it appeared anywhere in the paper or EHR chart, whereas the chemotherapy order data were required to be on the order. The general information data points were as follows: date of birth, allergies, identifiable plan for treatment, number of cycles, cycle length, cumulative doses, laboratory results, treatment parameters, follow-up plan, pharmacy interventions, and administration. Four additional general information data points were collected for clinical trial patients: clinical trial note, identification of trial, treatment calendar, and study/protocol number. The following data points were required to be on the chemotherapy order: medical record number (MRN), height, weight, body-surface area (BSA), dose, dose per unit (eg, mg/m2), diluent, route, length/rate of infusion, frequency, pre and post medications (dose, route, frequency), and physician signature. On the basis of the number of data points and hypothesized differences, a total of 90 charts, 45 from each chart type, across seven different regimens were required to achieve statistical significance.
The satisfaction survey addressed accessibility and availability of information (Data Supplement, online only). Surveys were distributed to all KCC physicians, nurses, and pharmacists. The13-question survey addressed the following topics: chart availability, diagnosis, treatment plan, past medical history, completeness of chemotherapy orders, previous laboratory results, clinical trial identification, clinical trial notes, treatment calendar, next treatment date, identification of treatment physician, time to find all of the above information, and overall satisfaction. For each question, the respondents rated both chart types on a 5-point Likert scale.
Outcome Measures
Three outcome measures were evaluated: overall completeness score per chart type, completeness score per regimen, and staff satisfaction with EHR/CPOE versus paper charts. Completeness score was calculated by dividing correctly documented items in the chart by the total possible. t test was performed to assess differences between chart types, and a P value < .05 was considered statistically significant.
Results
Documentation Completeness
A total of 90 charts were reviewed to assess documentation completeness: 12 ABVD, 16 R-CHOP, 16 FOLFOX, 16 AC, 16 carboplatin plus paclitaxel, six cisplatin plus etoposide, and eight clinical trials. The overall completeness of documentation was 93% for EHR/CPOE compared with 67% for paper charts (P < .001). The EHR/CPOE system improved documentation over paper charts for the following items: MRN, cycle length, height/weight/BSA, dose per unit, diluent, length of infusion, pre and post medications, laboratory results, treatment parameters, pharmacy interventions, follow-up plan, and clinical trial notes (Table 1). Completeness of documentation did not vary significantly between different regimens (Figure 1).
Table 1.
Completeness of Documentation
| Group and Item | EHR/CPOE (%; n = 45) | Paper (%; n = 45) | P |
|---|---|---|---|
| All patients (n = 90) | |||
| Date of birth | 100 | 100 | — |
| MRN on order | 100 | 13 | < .001 |
| Allergies | 100 | 96 | .494 |
| Plan for treatment | 96 | 84 | .157 |
| No. of cycles planned | 47 | 60 | .205 |
| Cycle length | 100 | 69 | < .001 |
| Height | 96 | 9 | < .001 |
| Weight | 87 | 18 | < .001 |
| BSA | 96 | 20 | < .001 |
| Dose | 100 | 100 | — |
| Dose per unit | 96 | 49 | < .001 |
| Diluent | 100 | 47 | < .001 |
| Route | 100 | 89 | .056 |
| Infusion length | 100 | 56 | < .001 |
| Infusion rate | 11 | 0 | .056 |
| Pre meds (dose, route, frequency) | 100 | 84 | .012 |
| Post meds (dose, route, frequency) | 98 | 80 | .007 |
| Laboratory results | 100 | 84 | .012 |
| Treatment parameters | 87 | 13 | < .001 |
| Physician signature | 100 | 96 | .494 |
| Administration | 100 | 100 | — |
| Pharmacy interventions | 51 | 4 | < .001 |
| Follow-up plan | 100 | 71 | < .001 |
| Clinical trial patients only (n = 8) | |||
| Identified on study | 100 | 75 | 1 |
| Clinical trial note | 100 | 0 | .029 |
| Treatment calendar | 100 | 75 | 1 |
| Study/protocol No. | 100 | 75 | 1 |
Abbreviations: BSA, body surface area; CPOE, computerized physician order entry; EHR, electronic health record; MRN, medical record number.
Figure 1.
Average completeness score by regimen. Display of the average percent completeness of all assessed measures of the chemotherapy order, for each of the seven regimen types, for paper versus electronic health record/computerized physician order entry (EHR/CPOE). ABVD, doxorubicin, bleomycin, vinblastine, dacarbazine; R-CHOP, rituximab, cyclophosphamide, vincristine, doxorubicin; FOLFOX, fluorouracil, oxaliplatin, leucovorin; AC, doxorubicin, cyclophosphamide.
Staff Satisfaction
Of the 76 surveys distributed to KCC staff, 49 were returned: 33 nurses, eight pharmacists, five physicians, and three not identified. The average time worked at KCC by the survey respondents was 8.2 years. The respondents favored the EHR/CPOE system for all areas of the survey, including overall satisfaction (Figure 2). Aspects of the EHR/CPOE system with which the respondents were most satisfied were: chart availability, availability of previous laboratory results, identification of clinical trial patients, and completeness of chemotherapy orders (Appendix Table A1, online only). The lowest scores for the EHR/CPOE system were in past medical history and treatment calendar availability, but these were still higher than the respective scores for the paper chart.
Figure 2.
Average overall satisfaction, on a 5-point Likert scale, of the computerized physician order entry versus paper charts, displayed by practitioner type. EHR/CPOE, electronic health record/computerized physician order entry.
Discussion
Overall documentation was significantly increased after EHR/CPOE implementation. Many areas of potentially significant sources of medication errors were improved, including MRN on chemotherapy order, height/weight/BSA, and dose per unit. Every order in the EHR is tied to the patient's chart and therefore to the MRN, in contrast to paper charts, which tended to include only the patient's name. The chemotherapy orders in the EHR/CPOE system are preloaded as dose per unit, and the system does not allow orders to be processed without a height/weight/BSA. The patient's BSA was rarely on the paper order and was not commonly recalculated with each cycle. In the EHR/CPOE system, the BSA is automatically recalculated for each encounter. However, caution must still be exercised to detect inadvertent typographical errors of the patient's height or weight. In the EHR/CPOE system, pharmacy interventions (nausea and vomiting counseling, pain management consultations, etc) were documented more frequently, thus increasing communication across all disciplines. Completeness scores were consistent across regimens, indicating that regimen complexity did not change documentation completeness.
Results from the survey revealed that the staff was more satisfied with the EHR/CPOE system compared to paper charts. The increased availability and completeness of the EHR are key drivers of staff satisfaction. Despite the overwhelming preference for the EHR/CPOE system, the survey identified significant areas for improvement. After implementation of the EHR/CPOE system, NorthShore KCC has been working to continuously improve the system to provide safe, quality care across all aspects of oncology.
Implementing CPOE in the outpatient oncology setting has inherent unique challenges. After attempting to use the inpatient module in KCC for outpatient CPOE, it became evident that a new system needed to be developed. After development and implementation of the new outpatient module, documentation and staff satisfaction significantly increased, but the system was not complete. Over the past 5 years, numerous enhancements to the system have been made, each adding important advances in patient safety and/or ease of use.
Perhaps the most significant improvement has been the “big picture” report, a one-page summary of the most pertinent patient information. The biggest strength of EHRs is their comprehensiveness, which can also be a significant weakness. Sifting through the entire chart to locate the necessary information is not only time consuming but can be a source of error. To display all the information, a KCC provider needed to perform his or her duties, a big picture, flow sheet–based report was developed. Information from various places in the patient's chart is summarized on one page. Should the user need to see more detailed information, hyperlinks on the big picture report directly take the user to the full flow sheet. The one-page summary improves patient safety because it serves as a reminder of all relevant information needed to care for the patient. This also has improved efficiency in the system by not requiring the users to load multiple areas of the patient's EHR.
Treatment plan notes are an example of a functionality that existed at the implementation of the outpatient module but did not reach full potential until recently. After significant experience with the EHR/CPOE system, it was discovered that quick, concise communication about the patient's chemotherapy treatment would improve patient safety and communication between disciplines. After a period of emphasis and incentives, physician inclusion of a chemotherapy treatment plan note with the order increased from 11% in early 2009 to 79% in early 2010, and near 100% at the time of this report. The treatment plan note, which outlines the physician's intent, is now used to provide a quick summary of the treatment plan and any changes to the plan as they occur over time (dose reductions with reasons, supportive medications, patient route preferences, etc). The treatment plan note appears at the top of the big picture report, which further enhances communication. Other enhancements made to the system include (1) an alert that is issued when a medication is released on the wrong day, to decrease errors associated with using old orders; (2) pertinent laboratory values that are automatically populated within the medication order (eg, serum creatinine for cisplatin); and (3) a hypersensitivity flag with hyperlink to hypersensitivity management guideline for patients receiving medications with high incidence of reactions (eg, rituximab).
The results of the user satisfaction survey were somewhat surprising, as much of the staff had been using the paper chart system for a long time. In the initial year of implementation, a prolonged period of learning and adoption of a new documentation culture was necessary. Using EHR/CPOE for orders required interruption of workflow to log on to a computer, and some of the initial safety measures were cumbersome for staff. Despite the initial impression that the new system did not appear to save time, the completeness of orders and sense of safety for the patients likely contributed to the improved satisfaction that resulted from use of the CPOE tool.
The EHR/CPOE system has changed, for the better, the way KCC sites provide care for their patients. Although development and implementation can be challenging, the potential patient safety benefits are well worth the investment. It should be recognized that continuous vigilance and maintenance of the EHR/CPOE system is required as patient and institution needs evolve.
Supplementary Material
Acknowledgment
Presented in part at the Hematology/Oncology Pharmacy Association Annual Conference, Denver, CO, June 14-16, 2007, and at the 41st American Society of Health System Pharmacists Midyear Clinical Meeting, Anaheim, CA, December 3-7, 2006.
Appendix
Table A1.
Staff Satisfaction Survey Results
| Survey Item | Nurses (n = 33) | Pharmacists (n = 8) | Physicians (n = 5) | Unknown (n = 3) | All (n = 49) | P |
|---|---|---|---|---|---|---|
| Chart availability | ||||||
| EHR/CPOE | 4.7 | 4.88 | 5 | 5 | 4.78 | < .001 |
| Paper | 3.04 | 3.19 | 3.8 | 4 | 3.21 | |
| Previous lab results | ||||||
| EHR/CPOE | 4.73 | 4.5 | 5 | 4.33 | 4.69 | < .001 |
| Paper | 3.19 | 3.5 | 3.2 | 3 | 3.25 | |
| Clinical trial identification | ||||||
| EHR/CPOE | 4.03 | 4.12 | 4.4 | 5 | 4.14 | < .001 |
| Paper | 3.2 | 2.88 | 3.5 | 3 | 3.16 | |
| Past medical history | ||||||
| EHR/CPOE | 3.55 | 3.62 | 4.5 | 3 | 3.6 | .003 |
| Paper | 3.04 | 2.62 | 2.6 | 2 | 2.85 | |
| Order completeness | ||||||
| EHR/CPOE | 3.84 | 4.25 | 4.6 | 3.67 | 3.98 | < .001 |
| Paper | 3.36 | 3 | 3 | 3 | 3.24 | |
| Treatment calendar | ||||||
| EHR/CPOE | 3.68 | 3.75 | 3.8 | 4 | 3.72 | .0132 |
| Paper | 3.17 | 2.5 | 3.75 | 3 | 3.08 |
NOTE. 1 = poor, 2 = fair, 3 = good, 4 = very good, 5 = excellent.
Abbreviations: CPOE, computerized physician order entry; EHR, electronic health record.
Authors' Disclosures of Potential Conflicts of Interest
The authors indicated no potential conflicts of interest.
Author Contributions
Conception and design: Cara A. Harshberger, George W. Carro, Wayne E. Spath, Wendy C. Hui, Jessica M. Lawton, Bruce E. Brockstein
Administrative support: Cara A. Harshberger
Provision of study materials or patients: Cara A. Harshberger
Collection and assembly of data: Cara A. Harshberger, George W. Carro, Jessica M. Lawton
Data analysis and interpretation: Cara A. Harshberger, Abigail J. Harper, George W. Carro, Bruce E. Brockstein
Manuscript writing: Cara A. Harshberger, Abigail J. Harper, George W. Carro, Wendy C. Hui, Jessica M. Lawton, Bruce E. Brockstein
Final approval of manuscript: Cara A. Harshberger, Abigail J. Harper, George W. Carro, Wayne E. Spath, Wendy C. Hui, Jessica M. Lawton, Bruce E. Brockstein
References
- 1.Gandhi TK, Bartel SB, Shulman LN, et al. Medication safety in the ambulatory chemotherapy setting. Cancer. 2005;104:2477–2483. doi: 10.1002/cncr.21442. [DOI] [PubMed] [Google Scholar]
- 2.Galanter WL, Didomenico RJ, Polikaitis A. A trial of automated decision support alerts for contraindicated medications using computerized physician order entry. J Am Med Inform Assoc. 2005;12:269–274. doi: 10.1197/jamia.M1727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Zlabek JA, Wickus JW, Mathiason MA. Early cost and safety benefits of an inpatient electronic health record. J Am Med Inform Assoc. 2011;18:169–172. doi: 10.1136/jamia.2010.007229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Gandhi TK, Burstin HR, Cook EF, et al. Drug complications in outpatients. J Gen Intern Med. 2000;15:149–154. doi: 10.1046/j.1525-1497.2000.04199.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Walsh KE, Dodd KS, Seetharaman K, et al. Medication errors among adults and children with cancer in the outpatient setting. J Clin Oncol. 2009;27:891–896. doi: 10.1200/JCO.2008.18.6072. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.