Effect of default order set settings on telemetry ordering

Abstract

Objective

To investigate the effects of adjusting the default order set settings on telemetry usage.

Materials and Methods

We performed a retrospective, controlled, before-after study of patients admitted to a house staff medicine service at an academic medical center examining the effect of changing whether the admission telemetry order was pre-selected or not. Telemetry orders on admission and subsequent orders for telemetry were monitored pre- and post-change. Two other order sets that had no change in their default settings were used as controls.

Results

Between January 1, 2017 and May 1, 2018, there were 1, 163 patients admitted using the residency-customized version of the admission order set which initially had telemetry pre-selected. In this group of patients, there was a significant decrease in telemetry ordering in the post-intervention period: from 79.1% of patients in the 8.5 months prior ordered to have telemetry to 21.3% of patients ordered in the 7.5 months after (χ2 = 382; P < .001). There was no significant change in telemetry usage among patients admitted using the two control order sets.

Discussion

Default settings have been shown to affect clinician ordering behavior in multiple domains. Consistent with prior findings, our study shows that changing the order set settings can significantly affect ordering practices. Our study was limited in that we were unable to determine if the change in ordering behavior had significant impact on patient care or safety.

Conclusion

Decisions about default selections in electronic health record order sets can have significant consequences on ordering behavior.

INTRODUCTION

Over the past 50 years, remote cardiac monitoring (“telemetry”) has become prevalent and routine for many patients admitted to the hospital. Telemetry has many clinical benefits, including diagnosis of malignant arrhythmia, prolonged QT interval, and myocardial ischemia when used in patients with appropriate indications.¹ The American Heart Association and the Choosing Wisely campaign recommend telemetry for a limited set of indications, including acute coronary syndrome, sepsis, and concern for malignant arrhythmia.^1,2 However, telemetry is currently being used for many other clinical situations. For example, at 1 academic community center, 20% of the telemetry orders were for nonguideline indications.³ This overutilization has been associated with unnecessary tests and procedures, such as catheterization⁴ and electrophysiology studies.⁵ Additionally, use of telemetry decreases emergency department throughput while patients are awaiting telemetry beds,⁶ contributes to alarm fatigue,^7,8 and can take a significant amount of nursing time to manage.⁹ It has been estimated that reducing inappropriate telemetry usage could save approximately $250 000 per year at a 400-bed hospital.¹⁰

Electronic health record (EHR) order configurations can impact the order rates of tests via multiple mechanisms. To encourage ordering, orders can be preselected, automated, or coupled (eg, isolation precautions coupled with orders for influenza testing). Orders can also be made available in order sets, but not be preselected when they are appropriate for certain patients but not others. Finally, to discourage ordering, orders can be hidden from order sets, restricted to subsets of ordering providers, or be associated with alerts. This final approach is commonly used to reduce unnecessary testing. Vitamin D orders, which have similarly high rates of overutilization to telemetry, have been reduced by alterations to order sets and preference lists.¹¹ Other studies have also looked at changing default order settings to influence clinician behavior and have shown dramatic changes in posttransfusion blood count ordering,¹² and generic medication¹³; and similar effects have been seen for nonclinicians, such as with organ donation. It is important to carefully and intentionally choose the preselected orders when creating order sets.

When we implemented a new EHR in May 2015, the general medicine admission order set had few orders preselected. To increase ordering efficiency, the internal medicine residency created a customized version of this order set, the Residency Order Set, with many common orders preselected, including telemetry. On September 21, 2017, at the request of the hospital Medicine division, which had noticed high rates of inappropriate telemetry usage, the order set was changed so that the telemetry order was still available, but no longer preselected (Figure 1). The internal medicine chief residents notified the residents of this change via email; no other education was conducted at this time.

Figure 1.

Screen shot of order set change. On Sept 21, 2017, the ECG Monitoring order (“telemetry”) was no longer preselected in Residency Order Set. Reproduced with permission from Epic.

We used this natural experiment to retrospectively evaluate the effect the order set change had on telemetry ordering patterns among the medicine house staff. Our hypothesis was that telemetry ordering would significantly decrease after the default settings in the order set were changed. We also looked at telemetry orders placed outside the admission order set as a way to monitor if the change caused a decrease in appropriate telemetry orders.

MATERIALS AND METHODS

This IRB-exempt, retrospective study was conducted at the Brigham and Women’s Hospital, a 793-bed academic medical center, and Brigham and Women’s Faulkner Hospital, a 162-bed community teaching hospital—both in Boston, Massachusetts. Our institutions currently use the Epic EHR system (Epic Corporation, Verona, WI). The order set change occurred on September 21, 2017. We obtained all orders for all patients admitted to a house staff medicine service between January 1, 2017 and May 1, 2018 from our enterprise data warehouse. This provided 8.5 months of order data prior to the order set change and 7.5 months of order data after the intervention.

A patient was defined as admitted to a house staff medicine service if their admission order was placed by a resident or fellow and the hospital service on admission was “Medicine.” We focused on patients who were admitted using the residency-customized order set, Residency Order Set.

We calculated the percentage of patients for whom telemetry was ordered from an admission order set during the time periods before and after the intervention. We used the chi-squared statistic to compare the proportion of patients with telemetry ordered on admission in the time period prior to intervention with the period after intervention. Additionally, we performed a change point analysis to compare pre- and postchange ordering patterns assuming at most 1 change point and assuming a normal distribution of ordering (R version 3.4.1)¹⁴ and an interrupted time series analysis; we performed both analyses, as we were unsure if the change would have an immediate effect or if there would be a lag. Graphs were produced using R and Tableau version 2018.3.

Due to the high rate of telemetry ordering and the retrospective evaluation, it was not possible to classify the telemetry as appropriate or not. Therefore, we used other natural controls to evaluate for potential negative effects on patients of removing the preselection of telemetry in the Residency Order Set. To do this, we calculated similar statistics for 2 other order sets used among medicine residents— 1 other customized order set, Alternate Order Set, which continued to have telemetry preselected and a second noncustomized order set, Noncustomized Order Set, which never had telemetry preselected. While not restricted by the EHR, the patients in our analysis were admitted using only 1 order set.

One risk of removing the default is that patients for whom telemetry is indicated might not have it ordered. To try to capture any inappropriate decrease in telemetry ordering from the order set (ie, patients who should have had telemetry ordered, but didn’t), we looked for a change in telemetry orders placed after hospital day 1 over the same time periods.

RESULTS

Between January 1, 2017, and May 1, 2018, there were 6131 patients admitted to a house staff medicine service. The residency-customized version of this order set, Residency Order Set, was used for 522 (16%) of patients before the change and 641 (22%) after. The Alternate Order Set and Noncustomized Order Set were used by Medicine residents in 9.5% and 52% prechange and 12% and 47% postchange, respectively; the remaining patients were admitted using additional order sets personally customized by users.

Among patients who were admitted using the Residency Order Set, there was a significant decrease in telemetry ordering in the postintervention period; from 79.1% of patients ordered to have telemetry in the 8.5 months prior to 21.3% of patients ordered in the 7.5 months after (χ² = 382; P < .001).

A 14-day smoothed time series analysis is shown in Figure 2. The change point analysis identified September 20, 2017 as the change point; the interrupted time series for the Resident Order Set was significant (β −0.5; P < .001).

Figure 2.

Percentage of order sets with telemetry ordered over the course of the study. A time series display of the 3 order sets used to admit patients to the medicine house staff service. The vertical line shows the date that the Residency Order Set (red) was changed to not preselect telemetry. There were no changes to the Alternate Order Set (gray) or the Noncustomized Order Set (gray).

There was no significant decrease in telemetry ordering for the same time periods among the Alternate Order Set or Noncustomized Order Set (Table 1). To evaluate whether the change in the order set led to delayed initiation of indicated telemetry, we looked at the number of telemetry orders placed after the first day of hospitalization. Figure 3 shows the rate of subsequent telemetry initiation, which shows that there is no increased ordering after the change went into effect.

Table 1.

Chi-Square results of difference in ordering pre- and postintervention

Order Set	Pre-intervention		Post-intervention
	Overall Order Set Usage (N)	Telemetry Usage, N (%)	Overall Order Set Usage (N)	Telemetry Usage, N (%)	Chi Square
Residency Order Set (Initially had telemetry preselected; preselection removed with intervention)	522	413 (79.1%)	641	137 (21.3%)	χ2 = 382; P < .001
Alternate Order Set (Continued to have telemetry preselected)	307	229 (74.6%)	350	241 (68.9%)	χ2 = 2.37; P = .123
Noncustomized Order Set (Never had telemetry preselected)	1687	483 (28.6%)	1370	353 (25.8%)	χ2 = 2.98; P = .084

This table shows the 3 studied order sets and the difference in telemetry usage rates on admission pre- and postintervention with associated chi-square statistics.

Figure 3.

New orders for telemetry after hospital day 1. A time series display of ordering for telemetry after hospital day 1. The color reflects the order set that was used to admit the patient (red is Residency Order Set; gray is Alternate and Noncustomized Order Set). There was no increase in the number of new orders for telemetry after hospital day 1 after the intervention.

DISCUSSION

In this retrospective study, we found that telemetry orders decreased in association with altering the preselected telemetry order in the most commonly used customized admission order set among medicine residents. Immediately following the change, telemetry ordering significantly decreased among residents using the Residency Order Set. Because there was no significant decrease in telemetry ordering when the 2 unaltered control order sets were used, we believe that the primary reason for changes in order frequency was due to the order set change rather than external factors.

Multiple interventions to reduce inappropriate telemetry ordering have been effective in the past, including special request forms and daily telemetry rounding,¹⁵ exclusion from order sets,¹⁶ and requiring the indication with electronic ordering.¹⁷

The reason this change in defaults led to a decrease in telemetry ordering is not clearly established from this retrospective study. Ideally, the admitting–ordering providers would determine the correct orders for the patient regardless of order set defaults and actively add or remove those orders from the set. Also, the order set defaults would be set to accurately parallel that thought process using either advanced (eg, machine learning) or basic (eg, frequency of ordering) methods. Based on these data and anecdotal experience, however, this is not how providers act.

Behavioral economics lends several explanations for this discrepancy. A likely explanation is the status quo bias, which leads people to remain with the presented state due to the higher cognitive energy required to change.¹⁸ Another explanation is the tendency toward inattention when people without strong preferences are pressed for time,¹⁹—as is the case in hospitals, which have high throughput. Trainees may feel pressured to quickly admit patients and rapidly go through the order set focusing on orders that will prevent patient harm, particularly those that present as required hard stops by the EHR. Additionally, the cognitive energy it takes to uncheck an order is likely much greater than that to check an order. Many factors likely contribute to this phenomena including recognizing that the order is selected; concern that this default was chosen by more experienced physicians and is, therefore, more accurate; an assumption that there is some logic at work that is suggesting this order; a feeling that doing more for a patient is beneficial; and unselecting an order means taking responsibility if the patient has an arrhythmia.^19,20

In our study, and with other order set changes, there is the potential for unintended consequences. Most obviously, as discussed later, is that patients who truly need telemetry may not have it ordered, leading to undermonitored patients. Additionally, although in theory telemetry is used for arrhythmia monitoring, it may also be used by busy clinicians to help monitor unstable patients who need vital signs checked more frequently than nurses and patient care assistants are able to do on their units. Order set default changes may also lead to a discrepancy between what is documented in the EHR and what is performed if a patient is placed on telemetry without an order in the chart.²¹ There is also the potential for downstream consequences and delays of care. An example is if a provider adjusts the type and screen order settings for surgical patients or diet orders for gastroenterology patients.

Order sets are increasingly allowing for more sophisticated logic and the incorporation of patient-specific information. With ongoing development, a likely future state would be to use risk-prediction models to preselect or leave orders unchecked based on other information to which the EHR has access.

Notably, although the order set change was publicized to the house staff, it was possible that they unintentionally did not order telemetry for patients. However, we did not see an increase in nonorder set telemetry orders in the postchange period likely due to ongoing appropriate telemetry orders for patients who truly needed it.

There were several limitations to this study. It was a retrospective study and, therefore, could not control for unmeasured confounding variables. Out of the scope of this investigation was performing chart reviews to determine whether this decrease occurred for inappropriate telemetry orders alone, or if it also caused a decrease in appropriate telemetry ordering. After the intervention, however, the rate of telemetry usage more closely approximated other published rates of appropriate usage.^9,22 There was also an increase in Residency Order Set usage after the change, which could be explained by users intentionally using this order set when admitting patients who did not need telemetry. However, the increase in Residency Order Set usage appears to be the result of a decrease in Default Order Set usage, which did not have telemetry selected; therefore, it is less likely that the default setting was driving residents to preferentially use the Residency Order Set.

CONCLUSION

This retrospective study shows that changes in default order set settings are associated with changes in provider behavior—in this case, decreases in telemetry ordering on admission. This study adds support for an approach to influence physician ordering behavior, which we can add to the growing repository of tools to employ when influencing ordering behavior is the goal for improved process and care outcomes.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

CONTRIBUTIONS

DR collected, cleaned, and analyzed the data, and drafted and revised the paper. AL assisted in design and implementation of the statistical plan and revised the draft paper. AW assisted in conception and design of the statistical plan and revised the draft paper. RB initiated the collaborative project and revised the draft paper. DR had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Conflict of interest statement

Dr Landman has received consulting fees from Abbott. No other authors have affiliations with or involvement in any organization or entity with any financial interest (such as honoraria, educational grants, participation in speakers’ bureaus, membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or nonfinancial interest (such as personal or professional relationships, affiliations, knowledge, or beliefs) in the subject matter or materials discussed in this manuscript.