Effectiveness of an information technology intervention to improve prophylactic antibacterial use in the postoperative period

Abstract

Background

A 2005 report from the Centers for Medicare and Medicaid Services and the Centers for Disease Control Surgical Infection Prevention program indicated that only 41% of prophylactic antibacterials were correctly stopped within 24 h of the end of surgery. Electronic order sets have shown promise as a means of integrating guideline information with electronic order entry systems and facilitating safer, more effective care.

Objective

The aim was to study the effectiveness of a computer-based antibacterial order set on increasing the proportion of patients who have antibacterial wound prophylaxis discontinued in the appropriate time frame.

Design

The authors conducted a quasi-experimental interrupted time-series analysis over an 8-month study period with the implementation of a computer-based order system designed to prevent excessive duration of surgical prophylaxis antibacterials.

Measurement

The primary outcome was the proportion of surgeries with antibacterials discontinued in the appropriate time frame. Additionally, we evaluated the percent of surgeries after implementation of the electronic intervention with chart documentation of infection among surgeries where the prescriber indicated the reason for antibacterial therapy was treatment.

Results

The computer-based order intervention significantly improved the proportion of surgeries with timely discontinuation of antibacterials from 38.8% to 55.7% (p<0.001) in the intervention hospital, while the control hospital remained at 56–57% (p=0.006 for the difference between treated and control hospitals). In surgeries after intervention implementation where a prescriber indicated the reason for antibacterial therapy was treatment, the prevalence of chart documented infection was only 14%.

Conclusions

A computer-based electronic order set intervention increased timely discontinuation of postoperative antibacterials.

Introduction

Surgical wound infections affect 2–5% of clean, non-contaminated surgeries, extra-abdominal procedures, and 20% of intra-abdominal operations.^{1 2} Antibacterial wound prophylaxis, when given properly, can reduce the risk of postoperative wound infection.^3–6 However, repeated doses of antibacterials given in the postoperative setting may increase hospital antibacterial resistance.^7–10 In 2002, the US Centers for Disease Control and Prevention initiated a national surgical infection project aimed at reducing the rate and severity of postoperative surgical wound infections by promoting appropriateness of perioperative prophylactic antibacterial therapy.¹¹ Further, the appropriate selection, use, and prompt discontinuation of prophylactic antibacterials surrounding surgery have all become national patient safety goals.^{11 12} Despite these goals, a 2005 report from the national surgical infection project indicated that only 56% of patients receive an antibacterial in the 1 h prior to incision, and only 41% of those receiving prophylactic antibacterials are then stopped within 24 h of the end of surgery.¹³ Two potential reasons for this poor performance are: (1) lack of knowledge or acceptance of the recommendations; and (2) the need for coordination of care between pre-, intra-, and postoperating room staff and physicians.

Electronic order entry systems with clinical decision support have been shown to improve clinical performance by reducing adverse drug events.¹⁴ There is also literature to suggest an increase in adverse events^{15 16} and several systematic reviews call for further analysis of health information technology interventions.^17–23 Electronic order sets have shown promise in integrating guideline information and facilitating safer, more effective care.²⁴ However, the effectiveness of electronic order sets needs to be demonstrated for prophylactic antibacterial administration before widespread implementation can be recommended. The aim of this study was to examine the effectiveness of a computer-based order set and ordering process on increasing the proportion of surgical patients who have antibacterial wound prophylaxis discontinued within the appropriate time frame.

Methods

Study design and study setting

We performed a quasi-experimental study to examine the effects of a computer-based intervention on the proportion of patients who have antibacterial wound prophylaxis discontinued in the appropriate time frame following surgery. The study protocol was approved by the University of Pennsylvania's Institutional Review Board, which waived the requirement for informed consent.

The study was conducted at two hospitals, an intervention hospital and a reference hospital, which served to control for potential secular trends in antibacterial prescribing. The intervention hospital was a 705-bed tertiary care teaching hospital located in West Philadelphia. The reference hospital is a 327-bed hospital also located in West Philadelphia, very close to the intervention hospital. Care is also delivered by resident physicians and faculty physicians. There is overlap in the resident physicians that practice at the intervention and reference hospitals with minimal non-resident practioner overlap.

Both hospitals have full use of computerized provider order entry (CPOE) and computerized decision support systems using Eclipsys Solutions Sunrise Clinical Manager. Anesthesia records at the intervention hospital were electronic and documented the start and stop times of surgery. Anesthesia records at the reference hospital were paper records and were abstracted manually for the selection of surgeries meeting eligibility. House staff at both hospitals used the Sunrise system to order medications postsurgery. The use of antibacterial therapy in the operating room is documented in the operating room electronic medical record, and patients who did not have any orders in the Eclipsys Solutions Sunrise Clinical Manager system had their antibacterial therapy discontinued within the appropriate time frame. The use of CPOE to enter antibacerial orders postoperatively is mandatory. The study consisted of two phases; phase 1 was before the implementation of the continued electronic intervention, and phase 2 was after implementation. Similarly the control hospital had two phases with no implementation of an electronic intervention at the start of phase 2.

Intervention

The intervention (phase 2) consisted of a requirement that physicians use a special order set to order antibacterials in specific postoperative locations, namely surgical intensive care and postsurgical recovery units. The order set contained all of the antibacterial orders used for postoperative surgical prophylaxis; additionally the intervention occurred without the physician specifically using the order set but intervened on all antibacterial therapy orders in specific postoperative locations. The order set also asked the user whether the antibacterial was being used for prophylaxis or for treatment. If the user answered prophylaxis, then the stop time for the order defaulted to either 20 h for non-cardiac surgery or 44 h for cardiothoracic surgery with sternotomy or bypass. The order set also disabled the ability to modify the stop time for prohpylatic antibacterials. The rationale for the 4 h discrepancy from the published guidelines²⁵ (ie, 20 rather than the published 24 h and 44 rather than the published 48 h) is the estimated time it takes for a patient to arrive at the postoperative hospital location, typically several hours after initiation of postoperative antibacterial therapy. Surgeries in patients who went from the operating room directly to non-postoperative locations were not exposed to the intervention order set. The order set was only implemented in specific postoperative locations, such as postanesthesia care units, surgical ICU locations, and ambulatory procedure units, so as not to disrupt the normal prescribing practice for antibacterials hospital wide. These surgeries are included in the primary analysis but excluded in a secondary analysis. The study was conducted over an 8-month period starting 4 months before the intervention in August 2007.

Data collection

Data were collected from the computerized order entry system, the electronic anesthesia record in the intervention hospital, and manual chart review of anesthesia records in the reference hospital. Because of the manual review required for the control hospital, a random sample of surgeries over the study period was selected. Surgeries with an antibacterial order more than 3 h prior to the start of surgery were excluded because this may have represented treatment of a pre-existing infection. However, a sensitivity analysis included these surgeries. Baseline characteristics included data on prescriber training level (interns versus more experienced providers), and whether the procedure was cardiac or non-cardiac surgery. The primary outcome of interest was whether patients received antibacterials beyond the recommended postoperative period, defined above. We also abstracted in the intervention hospital the medical records of the patients with postoperative antibacterial orders where the physician indicated that the use of the antibacterial was for treatment, to determine whether an infection was indeed documented in the chart. After implementation of the intervention (phase 2), where the prescriber indicated the reason for antibacterial administration was for treatment, we sought to determine the percent of surgeries with chart documentation, or microbiology laboratory records of infection.

Statistical analyses

Of all eligible surgical patients who received at least one dose of an antibacterial, the proportion whose course was discontinued in the appropriate time frame was compared between the preintervention and postintervention groups, using linear regression accounting for clustering within prescriber. The model was also adjusted for prescriber training level. A stratified analysis stratified on surgical type (cardiac vs non-cardiac) was also conducted. Additionally, we analyzed whether the patient spent any time in a non-postoperative location postsurgery as a potential effect modifier. Effect modification was analyzed by examining the significance of the interaction terms between the covariates and the binary pre-/postintervention variable. A sensitivity analysis included surgeries in patients who had received antibacterial therapy before 3 h prior to surgery.

Additionally, to account for potential preintervention trends, we analyzed the study using linear regression including time (month), intervention status (pre/post), hospital, and interaction terms of these parameters. This permitted comparisons between slopes over time, between hospitals for the pre- and postperiods, and intercepts for the pre- and postperiods.

All statistical analyses were performed using SAS 9.1.

Results

Figure 1 shows the classification of the surgeries over the study period. In the intervention hospital (figure 1A) there were 10 321 surgeries between April 19th 2007 and December 15th 2007, of which 5416 (52%) had a recorded antibacterial order (2608 preintervention and 2808 postintervention) in the CPOE system which required discontinuation. We excluded 308 preintervention and 355 postintervention surgeries because of prior antibacterial during the hospitalization.

Figure 1.

(A) Intervention hospital. (B) Control hospital. CPOE, computerized provider order entry.

In the control hospital (figure 1B), there were 7893 surgeries during the study period, of which 2511 (32%) had a recorded antibacterial order (1241 preintervention and 1270 postintervention) in the CPOE system which required discontinuation. Of these surgeries, we randomly selected 420 preintervention and 412 postintervention antibacterial orders for which to abstract charts to ascertain the details of the surgical procedure. We excluded 75 preintervention and 90 postintervention surgeries because of prior antibacterial utilization during the hospitalization.

There was no statistically significant change in the proportion of appropriate antibacterial discontinuation in the control hospital over the study period (55.9% pre compared to 56.9% post, p=0.830). The timely discontinuation of antibacterials in the intervention hospital increased from 38.8% to 55.7%, for an absolute increase of 16.9% (95% CI 10.1% to 23.7%) and p<0.001 comparing the change in the preintervention (phase 1) to that in postintervention (phase 2) periods. In the analysis controlling for secular trends utilizing a control hospital, there was a statistically significant difference of 15.9% (95% CI 4.6% to 27.3%) and p=0.006 in comparing the change between the preintervention and postintervention periods in the intervention hospital to that in the control group. Figure 2 shows the longitudinal data in which the preintervention and postintervention periods were split into 1-month intervals. The intervention appeared to have the greatest improvement in the first month. The absolute increase in proportion with timely discontinuation of antibacterials was nearly identical in an analysis controlling for prescriber experience (15.8%, 95% CI 9.1% to 22.6%). The increase in timely discontinuation was greater in cardiac surgeries than in non-cardiac surgeries. The absolute increase in the cardiac surgeries in the intervention hospital was 33.6% (95% CI 23.9% to 43.2%), and the absolute increase in the non-cardiac surgeries was 13.9% (95% CI 6.6% to 21.2%).

Figure 2.

Pre- and postintervention percent appropriate antibacterial discontinuation by study month (95% CIs).

The results were very similar in a sensitivity analysis in which we included surgeries with prior antibacterial therapy during the hospitalization. The results were also the same in a sensitivity analysis in which surgeries without a stop time were included and assumed to have excessive duration.

We performed a subgroup analysis in the 4738 surgeries in the intervention hospital, examining the effect of postoperative location on effectiveness of the intervention. In surgeries with antibacterials ordered only in postsurgical locations where the intervention would trigger (n=1592, 50.4%) we found an absolute increase in percent with timely antibacterial discontinuation of 26.4% (95% CI 17.9% to 35.0%). In surgeries with antibacterials ordered only in non-postoperative locations where the intervention would not trigger (n=524, 45.5%), we found an absolute increase in timely discontinuation of only 1.2% (95% CI –6.9% to 9.3%). The difference between these increases was statistically significantly different p<0.001. This analysis excluded 426 surgeries with antibacterials ordered in both location settings.

We reviewed medical records from 337 of the 403 postintervention surgeries at the intervention hospital where the prescriber indicated that the reason for administration was ‘treatment’ on the electronic order entry system. Of these 337 charts, 42 had either documented prior antibacterial therapy or a notation of prior infection. In the 295 remaining surgeries, only three had documented surgical site infection, and an additional three had positive blood cultures noted on or after the date of surgery. Thus, evidence of active infection was present in only 48 out of 337, or 14.2% (95% CI 10.7% to 18.4%) of the surgeries for which the prescriber indicated that treatment of infection was the reason for ordering the antibacterial.

Discussion

We found that the computer-based order set and ordering process increased the proportion of patients who had antibacterial wound prophylaxis discontinued in the appropriate time frame by an absolute increase of nearly 17 percentage points. This effect remained when controlling for prescriber experience. The intervention appeared to be more effective in cardiac surgeries than non-cardiac surgeries. Additionally, we found evidence of infection in only 14% of surgeries where the prescriber indicated treatment as the reason for ordering the antibacterial.

Excessive use of antibacterial agents contributes to bacterial resistance.^7–10 Although some evidence suggests that antibacterial prophylaxis after wound closure is unnecessary,^26–28 national consensus guidelines recommend that prophylactic antibacterial agents should be discontinued within 24 h of the end of surgery²⁵ for non-cardiac surgeries and 48 h for cardiac procedures.²⁹

Existing literature on the effectiveness of electronic interventions has demonstrated the ability to monitor for excessive antibacterial use.^{30 31} Our work extends previous work in utilizing the CPOE system to prospectively alter the order as oppose to retrospectively alerting pharmacists to update a ‘stop time’ to improve appropriate antibacterial discontinuation. Hsieh et al found only 20% of physicians changed prescriptions in response to drug allergy alerts³²; Payne et al found only 12% of critical drug-interaction alerts and 31% of drug allergy interaction alerts resulted in a changed prescription order.³³ However, our intervention automatically set the stop-date and time for postsurgical antibacterial prophylaxis, removing this choice from the prescriber. A prescriber who wanted to extend antibacterial wound prophylaxis prescribing had to choose the treatment option. This study suggests that information technology interventions that essentially force the discontinuation of a drug can be effective, albeit not completely. This is different from previous work, which only attempted to alert a prescriber to modify therapy by providing the prescriber with information and options or to notify pharmacy to review the patient.^{30 34} Computer interventions represent one method of medical policy implementation which, when used with other methods, such as provider education, can be effective.

A key strength of this study was the use of a concurrent control hospital to look for any local secular trends that may have occurred during the time of the implementation of the intervention. An additional strength is the large number of clinicians included in the study, approximately 570.

The intervention was limited to postoperative locations within the hospital, which was a constraint. We evaluated the possibility that location had an effect on the results in a subgroup analysis within the postintervention period of the intervention hospital and found that the absolute percent improvement was 27 percentage points higher in postoperative locations governed by the intervention than in locations where the intervention was not implemented. Therefore, extending the intervention to all hospital locations would be expected to improve the effectiveness of the intervention. Additionally, this would prevent physicians from reordering antibacterials when patients are transferred to non-postoperative locations which are not governed by the intervention. However, there are logistical barriers to this strategy, such as interrupting the normal provision of antibacterials to non-surgical patients.

We did not evaluate the incidence of surgical site infection before and after the intervention. We did examine the prevalence of documented infection in the surgeries in the postintervention period where the ordering physician indicated that the reason for antibacterial therapy was for treatment. The low prevalence of documented infection suggests that clinicians may have used the treatment option merely to prolong antibacterial therapy in postoperative patients.

Conclusion

We report on a computer-based electronic order set intervention which significantly increased timely discontinuation of postoperative antibacterial prescribing by 17% (absolute increase). The intervention was limited by a constraint to only activate in postoperative locations within the hospital. The results were not different based on prescriber experience. Greater improvements in antibacterial discontinuation were seen in cardiac surgeries as opposed to non-cardiac surgeries.