Implementation of computerized provider order entry required extensive customization but improved patient safety in a highly complex pediatric oncology environment.
Abstract
Purpose:
Pediatric oncology is a challenging environment for computerized provider order entry (CPOE). Our goal was to build on the proven safety features of CPOE and facilitate input of expert clinicians.
Methods:
A standard, commercially available CPOE system was implemented throughout the hospital. The design of the pediatric oncology implementation was a collaborative effort by a multidisciplinary team of clinicians and information technology experts.
Results:
During 9 months of configuration effort, 30 medical logic modules and 110 order sets were developed to support pediatric oncology. The proportion of chemotherapy orders submitted using specific research protocol or standard-of-care order sets increased from 57% to 84% as the number of active order sets grew to 200. The number of medication-related patient safety events decreased 39% after implementation of CPOE in pediatric oncology. Acceptance of the system is high in all clinical disciplines.
Conclusion:
Implementation of CPOE required extensive customization but improved patient safety in this highly complex pediatric oncology environment.
Introduction
Pediatric oncology orders are among the most challenging to implement in a provider order entry system, complicated by medications with narrow therapeutic index and the need to individualize treatment regimens not only by age, weight and size, but also on the basis of prior response to treatment. Clinical criteria must be met before initiation of chemotherapy, and chemotherapy, protective, and rescue medications must be sequenced correctly, thus requiring a reschedule logic that links orders and their start time to prevent injury. Within oncology, pediatric practice is extraordinary in the proportion of patients treated with curative intent on highly complex, cooperative group clinical trials,1,2 each with low accrual per center, and acceptance of grade 4 hematologic and mucosal toxicity. Moreover, long-term adverse effects of therapy are common and serious.3,4 Therefore, the consequences of dosing and administration errors are potentially severe, requiring forcing of rescue medications, adequate hydration, and strict dose range checks.
Pediatric Chemotherapy Process at Johns Hopkins Before CPOE
The two cornerstones of chemotherapy safety in pediatric oncology are clarity and independent checks: clarity, so the prescriber's intent is carried out; and independent checks, so the process can recover from any single human error. In 1995, the Johns Hopkins Division of Pediatric Oncology adopted standard practices for specifying chemotherapy treatment plans and for writing chemotherapy orders that largely anticipated the 2009 ASCO/Oncology Nursing Society standards.5 Word processing fill-in-the-blank templates were generated for common regimens. However, between 1998 and 2004, we found that 10 of 26 drug-related sentinel events at The Johns Hopkins Hospital were associated with chemotherapy, of which five involved pediatric oncology patients, reflecting an extraordinary degree of risk. This motivated a failure modes and effects analysis of chemotherapy prescribing, dispensing and administration; the creation of refined policies and procedures; and the in-house development of a first-generation computerized order generation system used to create all printed chemotherapy orders in pediatric oncology.6 This system resulted in complete standardization of the format of orders and elimination of calculation errors, but it was limited by the absence of order set functionality, lack of an interface with the pharmacy system, and inability to improve and maintain the system.
We therefore decided to adapt the commercially available provider order entry system selected for use throughout the Johns Hopkins inpatient units for chemotherapy prescribing. The Eclipsys/Allscripts Sunrise system has a versatile security model, and functionality can be added by using Medical Logic Modules (MLMs).
Methods
Design
The design of the pediatric oncology ordering tool was based on the needs assessment of providers and nurses in the pediatric oncology section. They comprised the largest contingent of the implementation team and determined design decisions.
The design was based on the previous chemotherapy ordering system developed at Johns Hopkins, which lacked the ability to design order sets and to communicate with the pharmacy system. The design principles were to include all existing features found to enhance safety, such as height and weight check, automatic hydration, and automatic dose modification, while adding order set functionality to reduce omissions and improve safety and ease of use. A total of 30 MLMs were written to support pediatric oncology ordering (Appendix Table A1, online only).
Weight/height check.
Every chemotherapy order set requires that the provider enter height and weight anew with orders to avoid dosing errors. The system automatically calculates body surface area (BSA), body mass index, and ideal body weight and uses an MLM to alert the provider of weights outside the normal ranges published by the Centers for Disease Control and Prevention,7 (Appendix Figure A1A, online only) thus reducing measurement and data entry errors.
Hydration orders.
Adequate hydration is critical to prevent toxic adverse effects of certain chemotherapy drugs. We designed all order sets with medications that require such hydration to force the provider to include hydration orders (Appendix Figure A1B). An order set with suggested fluid volume allows the provider to automatically calculate hydration rates by weight or BSA, minimizing the risk for omissions or inaccuracies. All weight- or BSA-based calculations for fluids and medications remain attached to the order to allow downstream providers to understand how doses were derived.
Dose adjustment.
Dose adjustments are frequently required because of toxicity or changing metabolism during the course of treatment. Standard practice requires an increase or decrease in dose by a certain percentage from the standard dose. CPOE systems are generally not designed to accommodate this very important safety feature, but by using an order form–based MLM, we were able to implement a percentage dose adjustment that makes the logic explicit to downstream providers. Appendix Figure A1C shows the default dose calculation, and Appendix Figure A1D shows a 25% reduction in dose by the provider.
Order sets.
When orders cross the interface from CPOE to the pharmacy system, they appear in a queue with insufficient context for pharmacists to verify chemotherapy orders. To provide that context, we require that all oncology chemotherapy be written on an order set that includes a synopsis order. The synopsis order identifies the treatment regimen or protocol, point in therapy, basis for dose calculation, cumulative anthracycline dose, medication sequence, criteria to begin treatment, and other information pertinent to administration. A synopsis report supports the pharmacy workflow for verification.
At Johns Hopkins, chemotherapy is categorized as research protocols, standard of care, or individualized regimens. Research protocol and standard-of-care order sets cannot be interchanged, because the research protocol order sets may use study-specific medications and special research laboratories. A General Chemotherapy Order Template (GCOT) allows ordering of individualized chemotherapy, including nonformulary and investigational agents, and is also used for standard-of-care and research protocol chemotherapy for which specific order sets have not been constructed. Because of the scrutiny that specific research protocol and standard-of-care order sets receive, a single attending physician signature is sufficient for these orders, but we require that a second pediatric oncology physician review and sign GCOT orders.
Placing orders for one complete cycle of chemotherapy at a time is considered best practice. This approach balances the need for ongoing reassessment of the patient's body size, metabolic function, disease response, and drug toxicity with the need to understand the regimen as a whole, and minimizes the number of times orders are written. In our medium-sized practice, we see 172 newly diagnosed patients per year, and last year enrolled 58 patients on 79 active treatment protocols (Appendix Table A2, online only). To cover every cycle of every protocol or regimen with a specific order set would require several hundred order sets. Because of the low expected accrual per protocol in pediatric oncology, we have adopted a “just-in-time” approach for rarely used protocols. This approach requires an institutional commitment to provide rapid turnaround order sets; our standard is to build, review, test, approve, and make order sets available in the production system within 2 weeks of identifying an urgent patient need for a new regimen or a critical amendment. The process for prioritizing the development of order sets is shown in Figure 1.
Figure 1.
Process map for the prioritization and development of order sets. The research protocol team includes the principal investigator and research nurse. The information technology team includes an order set analyst, pharmacy information technology (IT), and ancillary IT. The clinical team includes a disease-specific physician leader, clinical nurse specialist, and pharmacy clinical specialist. The order set review committee is chaired by a physician member of the medical board and includes representatives from laboratory medicine, pharmacy, and nursing administration.
Implementation
Roll-out.
Because of the need for significant MLM programming and testing to support safe chemotherapy prescribing and administration, pediatric oncology was the last unit in the Children's Center to adopt CPOE. Therefore, the pediatric residents and intensive care unit staff were experienced users when CPOE was implemented in pediatric oncology, but chemotherapy prescribers were not.
Our analysis was that the CPOE system, with its electronic Medication Administration Record (eMAR), affects nursing workflow more than physician workflow, and there were major safety concerns with having more than one system in use on a unit. Therefore, adoption of the system was initiated in the nursing unit, not in the physician service.
A timeline of our development and implementation is shown in Appendix Figure A2 (online only). The largest commitment of time and effort was for configuration, primarily developing order sets, and involved five clinical (three nurses, one prescriber, and one pharmacist) and 10 information systems full-time-equivalent staff for 9 months. Implementation began with 110 chemotherapy order sets; 57% of chemotherapy orders were submitted on specific order sets, and the remainder on the GCOT. Currently, with 200 chemotherapy order sets, 84% of chemotherapy orders are submitted on specific order sets.
Training.
The effort of developing order sets and testing resulted in a cadre of “super-users” in each discipline. The ability of these super-users to design training scenarios and coach their peers in the use of the system was a key factor in the success of the implementation, along with the responsiveness of the information systems team, which held daily troubleshooting meetings upon activation of the system.
Prescriber training was done as a two-part process. The first hour consisted of the standard training required by the hospital for all prescribers. This was followed by 2 hours of oncology-specific training, led by a pediatric oncology attending physician involved in system design and configuration. An hour was allotted for each prescriber to work through scenarios in the training system specific to his or her role as an attending, fellow, or physician extender. For example, extenders had to enter orders for planned chemotherapy admissions, fellows had to enter orders for newly diagnosed patients and patients readmitted with complications, and attending physicians had to find and correct errors in orders submitted for their signature. Prescribers who can work from home were asked to test their remote access before implementation.
Nurses received 4 hours of standard training. Eight hours of oncology-specific training, led by pediatric oncology nurse super-users, included 4 hours of competency testing. Two crucial new processes for nurses were releasing order sets from a chemotherapy hold status and setting the schedule for all chemotherapy on the eMAR.
Pharmacists had used the system when covering other units and received 3 hours of oncology-specific training oriented around test scenarios. Nurses and pharmacists needed to be able to complete the chemotherapy checklist within the new system, including identifying all required elements within the order sets, particularly in the synopsis orders.
Maintenance.
A Web-based issues and bug tracking system (JIRA; Atlassian, San Francisco, CA) is used to track areas of concern as well as new requests for medications or order sets. By logging these issues, the provider initiates a cascade of events including analysis of the need, development of a solution, and prioritization, followed by development, testing, and implementation.
The pediatric oncology implementation team continues to meet frequently and prioritize pending requests; the top five are slated for development and testing efforts. By keeping the list small and keeping clinicians and developers abreast of the current status by weekly conference call, delays in critical changes remain at a minimum.
Order sets.
The JIRA system is also used to add order sets and maintain changes. Changes in protocols or participation in new study protocols trigger the development of new order sets (Figure 1). By developing “mini-order sets” that are used as modular building blocks for existing order sets (object-based design principle), development time for additional order sets is minimized.
Results
Acceptance
Acceptance of the system was excellent among all three clinical disciplines primarily affected. Importantly, the implementation of the system resulted in benefits for each discipline. Prescribers found that specific order sets reduced the time and effort needed to enter orders. Pharmacy found that workflow changed dramatically but orders were clearer and more complete, so fewer interventions were required. Nursing enjoyed the clarity of orders and consistency of their workflow and noted fewer prescribing errors and quicker pharmacy response.
Safety
When major changes are introduced in clinical systems and workflows, it is important to monitor for a possible increase in errors or the appearance of a new spectrum of errors. We collect and review all medication-related events that occur on our unit and service, and we observed an immediate and profound decrease in events after adoption of CPOE. In the last year before implementation, a total of 132 medication-related events were reported for patients admitted to the pediatric oncology/bone marrow transplantation service on our unit. In the first year after implementation, 80 medication-related events were reported, a decrease of 39%. Of these events, the number that reached the patient similarly decreased by 40%, from 84 to 50. Chemotherapy-related events declined 48%, from 33 to 17. At baseline, 23% of events involved prescribing, 19% transcription, 39% dispensing, and 50% administration (the total exceeds 100% because multiple processes could be involved in one event). Every process improved after implementation of the CPOE system: prescribing events decreased 67%, transcription events were eliminated, dispensing events decreased 42%, and administration events decreased 33%.
Discussion
This system has not yet been implemented in the outpatient setting. This is an area of current development in which we hope to achieve similar gains. Our orders system is not yet integrated with documentation of the treatment plan, so we still require the prescriber to select the correct order set and identify the point in therapy by using information from the eMAR and hard-copy treatment plans.
Implementation of a CPOE system improved patient safety in a highly complex inpatient pediatric oncology setting. However, considerable effort was required to analyze workflows and add functionality to the system to support the requirements of pediatric oncology. Several months were spent developing order sets and customizing the implementation, a process that depended on and fostered the development of super-users in each discipline who could champion the system and coach their peers in its use. The development of a common syntax for specifying treatment plans and order sets across systems and institutions would significantly advance the safety and efficiency of care.
Acknowledgment
Presented in part at the Second Annual Johns Hopkins Medicine Patient Safety Summit, June 21, 2011, Baltimore, MD.
We thank the many clinicians and information technology professionals who contributed to the design and implementation of the CPOE system in pediatric oncology. Alix Dabb, PharmD, was involved in system design and implementation, and provided information on order set utilization. Colleen Reid, RN, and Denise Daniel, RN, MSN, provided pediatric oncology nursing expertise in design, implementation, and training. Pat Zeller manages the system and provided the timeline information. Desiree Baldwin developed most of the MLMs.
Appendix
Figure A1.
Screen shots illustrating safety features of the system. (A) Current height and weight are required, and the system displays the body surface area, body mass index, and ideal body weight and alerts the user to abnormal values. (B) An alert that alkaline hydration is required. (C) Standard dose calculation. (D) Percentage dose modification.
Figure A2.
Timeline of design, implementation, and deployment of the computerized provider order entry system on the Pediatric Oncology Inpatient (ped onc ip) unit. A total of 125 research and 17 standard-of-care order sets, 250 chemotherapy inventory items, and 30 Medical Logic Modules (MLMs) were built for this deployment. GCOT, General Chemotherapy Order Template.
Table A1.
Medical Logic Modules Developed to Implement Pediatric Oncology Chemotherapy Ordering Safely in POE
| Clinical Need | MLM Name (type) | Description |
|---|---|---|
| Facilitate order verification and second checks | ||
| Make all order calculation logic visible, and highlight any changes from protocol or standard-of-care orders | Dose per course (form close) | Calculate the dose per course of all chemotherapy orders (formula: dose × frequency × duration = dose per course. |
| Defaulted protocol dose versus adjusted dose audit trail (form close) | Store and populate the defaulted dose and the final adjusted dose and display the calculation info for both on each order. (defaulted dose) x mg/kg/dose adjusted to x mg/kg/dose. | |
| Display original frequency and duration (form open) | Display the original frequency and duration defaulted in the order set in a protected field on the order form for reference by all disciplines. | |
| Prevent known pitfalls in chemotherapy ordering | ||
| Significant change in relevant results | Order entry/current relevant results (form open) | Store and populate the data from the “relevant results” field into a separate field called “relevant results at order entry,” so clinicians are aware of laboratory values that change significantly after order entry. |
| Too many or too few doses | Restrict duration to number of “doses” (field change) | Restrict the duration on chemotherapy orders to a value of “doses” to standardize on the most explicit way to specify duration. |
| Block > 1 “doses” with frequency of “once” (form close/field change) | Block any number of “doses” > 1 in the duration field when placed with a Frequency of “once.” | |
| Clearing fields on reorder (form open) | Clear the duration fields on reorder, based on original order session type (active v hold) so the prescriber must account for doses already administered. | |
| Inadvertent change between BSA and body weight dose calculation | Restrict dose basis UOMs (form open) | Restrict changes to a unit of measure dose basis outside of the specified group. A change from calculating dose by body weight to calculating by IBW is clinically rational, but changing between dosing by weight and dosing by BSA requires a different order. |
| Missing or incorrect base solution | Calculate base solution volume on the basis of standard concentration (form open/field change) | Calculate the base solution volume on the basis of a list of standard chemotherapy medication concentrations provided by pharmacy. |
| Orders on the wrong patient | Default system patient info on order form banner (form open) | Populate the patient's BIRTHDATE and MRN# fields on the banner at the top of the chemotherapy order form. Require the prescriber to enter a few characters of the patient's name at the start of an order session. |
| Facilitate protocol compliance | ||
| Dose cap | Dose cap (form close) | Reference the defined dose cap for all applicable chemotherapy medications and (1) alert the user of the dose cap if exceeded (2) set the medication dose to the dose cap value (this can be overridden by the user). |
| Pediatric IBW | IBW for patients < 18 yr (field change in the calculator or form open) | The system MLM SYS_CALC_WT has been enhanced to calculate IBW for pediatric patients on the basis of age, sex, and height. |
| Age-based dosing | Intrathecal dosing pediatric (form open) | Calculate and populate the patient's intrathecal chemotherapy dose based on age criteria. |
| Triple intrathecal dosing pediatric (form open) | Calculate and populate the patient's triple intrathecal doses on the basis of age criteria. | |
| Block system rounding when inappropriate | Ignore system standard 10% rounding rules (form open) | New routes have been created and referenced in to the system rounding MLM to block the system's standard 10% rounding rules as required by certain protocols. |
| Exact dose without rounding checkbox (checkbox field selection) | New routes have been created that the system rounding MLM ignores. The routes are defaulted specifically on medications in order sets where standard rounding should be ignored. A checkbox on all forms allows the user to manually prevent standard dose rounding. | |
| Prevent overdoses | Maximum dose (form close) | Reference the defined maximum dose for all applicable chemotherapy medications and (1) alert the user of the maximum dose if exceeded (field change), (2) provide a hard stop to prohibit users from submitting the order without reducing the dose below the maximum dose (field change and/or form close). |
| Restrict dose basis max dose logic (field change) | Prevent changes to the dose basis of specified items to a value more than than the defined maximum dose basis value. | |
| Extend system functionality | ||
| Multiple agents in one bag | Multi-ingredient (field change) | For all multi-ingredient orders, calculate the additive dose on the basis of specified dose basis, and populate the real “IV Additive” field to send to the pharmacy system. |
| Multi-ingredient additives % change (field change) | For all multi-ingredient orders, calculate the % Increase or decrease of additives in a multi-ingredient chemotherapy item. | |
| Calculate IVF rate | Calculate: mL/x/h (x = kg or m2) (field change) | Reference the ordered volume and the patient's weight or BSA and calculate mL/x/h × size in x = mL/h. |
| Standard form open;close MLMs (form open/close) | Modify/apply existing logic to oncology forms. |
Abbreviations: BSA, body surface area; IBW, ideal body weight; IVF, intravenous fluids; MLM, medical logic module; MRN, medical record number; POE, provider order entry; UOMs, units of measure.
Table A2.
Johns Hopkins Pediatric Oncology Program Statistics, 2010
| Statistic | No. |
|---|---|
| Number of newly diagnosed patients | 172 |
| Active COG treatment protocols | 57 |
| Active single institutional treatment protocols | 5 |
| Active non-COG multi-institutional treatment protocols | 17 |
| New protocols opened in the past year | 14 |
| New patient enrollments on study | 58 |
| Protocol amendments affecting therapy/yr | 25 |
Abbreviation: COG, Children's Oncology Group.
Authors' Disclosures of Potential Conflicts of Interest
Although all authors completed the disclosure declaration, the following authors indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.
Employment or Leadership Position: Christoph U. Lehmann, American Medical Informatics Association (U), Child Health Informatics Center, American Academy of Pediatrics (C), Applied Clinical Informatics (U), eNeonatal Review (C) Consultant or Advisory Role: Christoph U. Lehmann, Pediatric Informatics textbook (C) Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Other Remuneration: None
Author Contributions
Conception and design: Allen R. Chen, Christoph U. Lehmann
Provision of study materials or patients: Allen R. Chen
Collection and assembly of data: Allen R. Chen
Data analysis and interpretation: Allen R. Chen
Manuscript writing: Allen R. Chen, Christoph U. Lehmann
Final approval of manuscript: Allen R. Chen, Christoph U. Lehmann
References
- 1.Smith MA, Seibel NL, Altekruse SF, et al. Outcomes for children and adolescents with cancer: Challenges for the twenty-first century. J Clin Oncol. 2010;28:2625–2634. doi: 10.1200/JCO.2009.27.0421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Baquet CR, Ellison GL, Mishra SI. Analysis of Maryland cancer patient participation in National Cancer Institute–supported cancer treatment clinical trials. J Clin Oncol. 2008;26:3380–3386. doi: 10.1200/JCO.2007.14.6027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wasilewski-Masker K, Mertens AC, Patterson B, et al. Severity of health conditions identified in a pediatric cancer survivor program. Pediatr Blood Cancer. 2010;54:976–982. doi: 10.1002/pbc.22431. [DOI] [PubMed] [Google Scholar]
- 4.Yeh JM, Nekhlyudov L, Goldie SJ, et al. A model-based estimate of cumulative excess mortality in survivors of childhood cancer. Ann Intern Med. 2010;152:409–17. doi: 10.1059/0003-4819-152-7-201004060-00005. W131-W138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Jacobson JO, Polovich M, McNiff KK, et al. American Society Of Clinical Oncology/Oncology Nursing Society chemotherapy administration safety standards. J Clin Oncol. 2009;27:5469–5475. doi: 10.1200/JCO.2009.25.1264. [DOI] [PubMed] [Google Scholar]
- 6.Kim GR, Chen AR, Arceci RJ, et al. Error reduction in pediatric chemotherapy: Computerized order entry and failure modes and effects analysis. Arch Pediatr Adolesc Med. 2006;160:495–498. doi: 10.1001/archpedi.160.5.495. [DOI] [PubMed] [Google Scholar]
- 7.Centers for Disease Control and Prevention. Percentile Data Files with LMS Values,August 4, 2009 update. http://www.cdc.gov/growthcharts/percentile_data_files.htm.