Viewpoint: Controversies Surrounding Use of Order Sets for Clinical Decision Support in Computerized Provider Order Entry

Abstract

Order sets provide straightforward clinical decision support within computerized provider order entry systems. They make “the right thing” easier to do because they are much faster than writing single orders; they deliver real-time, evidence-based prompts; they are easy to update; and they support coverage of multiple patient problems through linkages among order sets. This viewpoint paper discusses controversies surrounding use of order sets—advantages and pitfalls, decision-making criteria, and organizational considerations, including suggestions for vendors. Order sets have the potential to improve clinician efficiency, provide real-time guidance, facilitate compliance with Joint Commission on Accreditation of Healthcare Organizations and Centers for Medicare & Medicaid Services performance measure sets, and encourage overall acceptance of computerized provider order entry, but may not do so unless these controversies are addressed.

Introduction

Promised benefits of computerized provider order entry (CPOE) with clinical decision support (CDS) include improved patient care quality and safety. ^1–4 Physicians often identify efficiency as the most important expected benefit of CPOE to their daily workflow. ⁵ Order sets, collections of pre-formed (i.e., pre-defined, containing specific values for fields within each order) “quick orders,” ⁶ can address needs of patients and clinicians by making the right thing easier to do (▶). ^6,7

Table 1.

Table 1. Benefits of Electronic Order Sets Compared to Paper Order Sets

Paper Order Sets	Electronic Order Sets
Difficult to find	Readily accessible from anywhere
Clinician unaware the order set exists	Real time prompts when appropriate
Physical change to order set lags behind practice change	Order sets can be updated more easily when practice changes
Difficult to remove ‘old version’ from patient care areas	Old version is replaced with the new version
Patient may need more than one order set	Ability to link order sets for standardized elements of care
Possibly evidence-based	Can be linked to the evidence-based literature

Both paper and electronic order sets within a CPOE system have the potential to improve provider efficiency by conveniently grouping orders together. ^6,8–10 This improves the speed of placing orders, which is particularly important as order entry in CPOE has been shown to take more time than writing orders on paper. ^11–13 This viewpoint paper, through reviewing current controversies related to order sets, discusses expected benefits of electronic order sets when used for CDS, along with some of the challenges to order set implementation, including recommendations for organizations using order sets.

Controversy #1: Even when quality of evidence-based order sets is excellent, most CPOE systems make the utilization of order sets for any given patient voluntary for clinician-users. This leads to a “you can lead a horse to water but you can’t make him drink” phenomenon, whereby patients do not benefit when their care providers bypass evidence-based order sets usage

One institution identified the lack of order sets as a factor in delayed care that potentially contributed to increased mortality in critically ill patients. ¹⁵ Because certain tests or medications may be difficult to prescribe due to limited search capabilities or difficult timing issues, the benefits to clinicians of grouped orders with appropriate defaulted details cannot be underestimated. ¹⁶ However, the presence of order sets in a system does not guarantee that clinicians will use them, in which case the increased morbidity and mortality due to “lack of order sets” occurs functionally—and possibly at increased legal risk to the clinician when existing order sets would prevent adverse outcomes.

Institutions implementing, and vendors developing CPOE systems must provide convenient mechanisms to make clinicians aware of existing, and especially “new” order sets. One solution recently described ¹⁷ is to provide a list of relevant evidence-based order sets at the time of writing admission orders—based on the new patient’s geographic ward location (if a specialty unit), or the patient’s clinical service (cardiology, general surgery, neonatal ICU, etc.).

More advanced approaches to recommending order sets for a given patient might involve capturing the admission diagnosis and linking it to relevant order sets, analyzing the problem list at time of admission to find pertinent order sets, or even comparing admission orders, as they are written, to existing order sets to “diagnose” which order sets might be applicable. Though all three of these approaches represent ways to push information to the clinician at the right time, the first two rely on entry of diagnosis and problem list information before writing orders. This may prove challenging for busy admitting clinicians who may write orders first.

Controversy #2: Both CPOE vendors and free-standing vendors supply “evidence-based” order sets for CPOE customers. However, local practices and circumstances may vary significantly from those envisioned by order set distributors. How can such differences be reconciled, especially at smaller hospitals with limited clinical, information technology, and financial resources? Even when an institution develops an order set and demonstrates through a controlled clinical trial its efficacy in improving care or reducing costs (while maintaining quality of care), it is difficult for other institutions to adopt the order set

Many electronic medical record (EMR) vendor products accommodate order sets in their systems, but all of the build and maintenance of the individual order sets takes place at the organization level. For organizations that lack personnel resources and time, companies such as Zynx Health supplies order set templates with linked evidence that may be customized to fit the hospital’s needs. ¹⁸ Unfortunately, to our knowledge to date, electronic exchange of order sets between sites is rarely possible. Thus, purchasing order sets from outside vendors may speed order set content development, but an arduous, sometimes painful process then follows to convert the orders suggested by the vendor into the “local order vernacular.” This is analogous to the “curly braces” problem of shareability and adaptability widely discussed a decade ago for local implementation of Arden Syntax. ¹⁴

Whether vendor supplied or organization specific, the initial amount of work required to design and build order sets will depend on various factors and can be overwhelming. VA Puget Sound reported that their “order configuration entities” were prepared over a three year period. ⁶ Experience at Northwestern Memorial Hospital and Hackensack University Medical Center has been similar. Though most hospitals have paper-based order sets to begin with, extensive changes may be required if standardization and consistency across order sets are part of the organization’s goals. For example, in response to evidenced-based performance measure sets based on national guidelines, we began a quality initiative within our organizations to standardize the choice and duration of surgical antimicrobial prophylaxis for the surgical infection prevention performance measure set. Antimicrobials were defined for every surgical procedure performed and those selected were the only antibiotics available from the post-op order sets and were defaulted to stop 24 hours from the pre-operative dose (▶). Similar efforts occurred for performance measure sets for acute myocardial infarction, heart failure, community-acquired pneumonia, coronary artery bypass graft surgery, and hip and knee replacement. High compliance was therefore facilitated for these measure sets from the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), the Centers for Medicare and Medicaid Services (CMS), and Premier, Inc.

Figure 1.

Standardized antimicrobial prophylaxis choices defaulted into a post operative order set.

A number of design features would increase the utility and safety of the care prescribed through order sets. Individual orders within order sets should be linked, if so desired by the client. For example, drug A is to begin at time zero, and linked orders for drug B and drug C begin 4 and 8 hours after drug A. When drug A is delayed by 2 hours, drug B and C are automatically moved back by 2 hours. This decreases the risk for error and amount of downstream re-work and is particularly useful for fully integrated EMRs with online electronic medication administration records. Linked orders should also prompt the clinician to discontinue all orders originating from an order set when appropriate. Patient Controlled Analgesia (PCA) orders typically include a number of additional orders for patient monitoring, rescue medications, and medication used to treat side effects. When the PCA order is discontinued, the user should be asked and given the ability to discontinue all associated orders with a single click.

Other desirable features of order sets would decrease the risk of clinician error during order entry. Linked orders in an order set should be mutually exclusive, if desired. This is particularly useful for high-risk medications, such as opioids to treat pain or anticoagulants for DVT prophylaxis. To support varied patient populations, order sets may contain more than one choice for a therapeutic indication. When one order is selected, the clinician should not be able to select the other in error. Finally, order sets could have logic to present only those options appropriate for the patient it is being used for. For example, though penicillin may be a standard choice in the order set, that option would be “dithered” or otherwise not available for a patient with a penicillin allergy. Similarly, a pregnancy test as part of a pre-procedure order set would not be available when the order set is used for a male.

During conversion from paper to electronic order sets, Northwestern Memorial Hospital developed standards (based on the literature, local consensus, and institution-specific drug formulary) for common care elements such as surgical antimicrobial prophylaxis, deep vein thrombosis (DVT) prophylaxis, glucose management, post-operative nausea and vomiting prophylaxis, pain management, prevention of contrast-induced nephropathy and others. Electronic order sets support efficient diffusion of standardized evidence based protocols, particularly when the protocol is appropriate for a number of areas, such as all intensive care units (▶). However, dozens of other institutions, including Regenstrief Institute, Brigham and Women’s Hospital, the Veterans Administration System, Vanderbilt, and many others, have each individually developed similar order sets de novo, without sharing them. Implementing and maintaining a CPOE system is a massive, institution-wide undertaking, and scarce resources should not be expended “re-inventing the wheel.” Until a national-standard set of defined CPOE orderables is developed, to which each vendor or institution can map their own “orderables”—at both the order name and individual fields’ levels—little progress may be made.

Figure 2.

Standardized protocols, such as DVT prophylaxis, glucose management, ICU sedation, and stress ulcer prophylaxis may be linked to appropriate admission order sets.

Controversy #3: While paper-based order sets, the precursor to CPOE order sets, had limitations that CPOE based order sets can overcome, there remain significant limitations to what CPOE-based order sets can do

Paper-based order sets have been used to standardize care and improve clinical quality in hospitals for many years. ^{8, 9} Order sets may be developed in response to available evidence and typically cover admission diagnoses and common surgical procedures. They have a close relationship to critical pathways which can often be represented as sequences of order sets. ¹⁹ To a large extent, paper-based order sets have achieved the goal of standardized care and decreased reliance on memory by providing a complete set of legible, structured orders. Electronic order sets offer these same benefits and are easier to integrate into care processes as information can be pushed to the clinician when it is needed. They also carry advantages of CPOE; automated order checking and electronic communication of orders to the filling service. Though paper-based order sets may be used extensively, they are often plagued by the shortcomings of paper (▶). The order set may not be available at the time the physician needs it, branching algorithms cannot be used, and the physical changes to paper-based order sets often lag well behind practice changes, formulary changes, and efforts to standardize care across the organization. For example, an effort to standardize the management of post-operative nausea and vomiting based on available evidence requires updates to every surgical post operative order set. Though still time consuming when electronic, near impossible when using paper order sets. Even after paper order sets are updated, the “old version” may be available in the patient care areas for months.

Conversion of existing paper-based order sets into an electronic version affords the organization an ideal opportunity to review all order sets to standardize common elements of care, as well as ensure the orders sets are supported by evidence.

Because order sets provide convenient mechanisms to execute existing sets of orders, the degree to which they can be customized to the current patient state is limited. Several CPOE developing institutions and vendors have implemented “pop-up” algorithmic “advisors” that go beyond the capabilities of order sets to take dynamic patient states into consideration. For example, implementing a heparin therapy advisor should ideally take into consideration the currently active medications of the patient (e.g., not adding heparin if the patient is already on streptokinase) and the most current laboratory results (preferentially adjusting suggested heparin drip rates based on most recent partial thromboplastin time results). To do so goes beyond the capabilities of most order set editors, which typically at most allow for weight, age, or body surface area based patient-specific dose calculations. It is not always easy to determine when to implement a given protocol as an order set or as a “pop-up” advisor. Order sets require less effort to maintain, due to their simplicity; programmers must work closely with clinical experts to maintain “pop-up” advisors.

Controversy #4: While evidence-based order sets can make it easier for CPOE clinician-users to “do the right thing,” clinical knowledge advances rapidly. When order sets are implemented without organization standards and clinical review or inadequately maintained, they become templates for efficiently practicing outdated medicine on a widespread basis

An order set template and standards are key to an end product that improves quality and is accepted by physicians. ^{8, 9} Standards will need to cover broad topics such as overall template design and details such as consistent naming conventions that make it easy for clinicians to locate the order sets in the system. An example of a patient safety standard is to not have any of the medications pre-selected when the order set is opened. Though the extra “clicks” may slow the physician down, Northwestern Memorial Hospital felt that reducing the risk of prescribing a medication in error outweighed the time required for extra “clicks.” On the other hand, we encourage pre-selected orders for nursing and lab orders where the same treatment is prescribed for virtually all patients (▶). Highlighting order categories followed by specific suggested orders facilitates readability. Finding order sets within the system may present a challenge. We discovered that strategies such as starting the name of the order set with “post-op,” as was used in paper order sets, makes it very difficult to locate the order set (▶). Our current standard is to start the name of the order set with the name of the service (i.e., Cardiac Surgery) followed by the procedure. We make the order sets searchable by service, procedure name, or pre/post-op, as well as available from within service specific folders (▶).

Figure 3.

Medications are never pre-selected upon order set initiation, but care elements common to all patients may be selected.

Figure 4.

Searching for an order set by typing in “post” versus locating one from within a service defined folder.

Figure 5.

Order sets also available in service specific folders for easy searching.

Multidisciplinary review with the order set author is invaluable, particularly for order sets developed and reviewed prior to any clinician’s experience with CPOE. It provides an opportunity to explain how the system works and to discuss any changes necessary for care standardization or orders that do not translate well into the electronic format. Most importantly, it provides the author an opportunity to explain the true intent of the specific orders contained in the order set. Misinterpretation in the build of these orders can at best lead to a great deal of re-work and at worst lead to error and patient harm.

Order sets should be reviewed by existing committees or a new committee prior to implementation into the system, and periodically thereafter. They should also be reviewed by the services (e.g., pharmacy, laboratory, radiology) who will receive orders when the order set is used. The organization’s process for order set request, approval, and routine review should be established and functional before CPOE go-live. As others have reported, ²⁰ we experienced an acute increase in demand when physicians began to use the system. One surgical service that had used two paper order sets for years developed an additional eight a week after go-live. Once order sets are live, periodic review is necessary to encourage use where appropriate or remove those order sets that are not being used. ^6,19 Though it is desirable to fit order set review into existing committee structure, the volume of work necessary makes this difficult. Our experience has been that a new multidisciplinary committee is required, with input from established oversight committees such as Pharmacy and Therapeutics, Critical Care, Blood Transfusion and Quality Committees. Committee responsibilities may include review for variance from organizational quality standards, consistency of care across order sets, potential for introducing new errors or adverse events, and to prioritize other decision support arising from the order sets, such as rules.

Order set maintenance remains a significant challenge in the face of rapidly advancing clinical knowledge. Many times clinical practice changes are recognized at the clinician level, but never communicated to the group managing order sets in the system. Clinicians create “standard work-arounds” to meet their needs and train their residents and others to use the work-around, thus reverting to reliance on memory and increasing the risk of error. Lack of communication between Formulary and Quality Committees can quickly lead to order sets inconsistent with recommended practices.

Controversy #5: While evidence-based order sets can make it easier for CPOE clinician-users to “do the right thing,” clinical knowledge advances rapidly and resource-challenged organizations may take some time to meet the needs for new order set requests. One solution to the problem is to allow individual clinicians to develop their own “private” order sets that they can use for their patients. However, doing so potentially removes the evidence-based nature of such order sets, and introduces other maintenance issues—e.g., will clinicians keep their own “private” order sets up to date with the frequency that the institution keeps global order sets current?

To realize the full benefits of standardized evidence-based care and to conserve resources for all necessary decision support, Northwestern Memorial Hospital, VA Puget Sound, and Hackensack University Medical Center have avoided building personal order sets. Others have reported a great deal of effort spent developing hundreds of order sets only to later realize that personal order sets were neither valued nor often used. ^5,6 One organization developed 513 order sets and reported that only 13% were used one or more times in a 6 month period. ⁶ In addition to the time spent on the initial build of the order sets, finding the correct one in the system, and routine maintenance or updates can quickly become overwhelming. Anyone at Northwestern Memorial Hospital may recommend an order set and provide content, but only one order set per indication is built and quality standards agreed on by the organization are a default part of the order set. Any disagreement over content is resolved by the Quality Committee or the departmental chair, though this is rarely necessary.

Summary

Upon initial implementation of CPOE, process change and interruptive decision support can be overwhelming to the clinician. Order sets may improve clinician efficiency and provide decision-making guidance, thereby increasing user acceptance of the system. Order sets can be viewed as basic building blocks of a decision support program, and should be developed and maintained with strong clinician and institutional support. Unless institutions undertaking CPOE plan an overall approach to order set acquisition and development, utilization, and maintenance, the controversies discussed above may partially degrade quality and use of order sets over time.