Handheld Computer-based Decision Support Reduces Patient Length of Stay and Antibiotic Prescribing in Critical Care

Abstract

Objective: This study assessed the effect of a handheld computer-based decision support system (DSS) on antibiotic use and patient outcomes in a critical care unit.

Design: A DSS containing four types of evidence (patient microbiology reports, local antibiotic guidelines, unit-specific antibiotic susceptibility data for common bacterial pathogens, and a clinical pulmonary infection score calculator) was developed and implemented on a handheld computer for use in the intensive care unit at a tertiary referral hospital. System impact was assessed in a prospective “before/after” cohort trial lasting 12 months. Outcome measures were defined daily doses (DDDs) of antibiotics per 1,000 patient-days, patient length of stay, and mortality.

Results: The number of admissions, APACHE (Acute Physiology, Age, and Chronic Health Evaluation) II and SAPS (Simplified Acute Physiology Score) II for patients in preintervention, and intervention (DSS use) periods were statistically comparable. The mean patient length of stay and the use of antibiotics in the unit during six months of the DSS use decreased from 7.15 to 6.22 bed-days (p = 0.02) and from 1,767 DDD to 1,458 DDD per 1,000 patient-days (p = 0.04), respectively, with no change in mortality. The DSS was accessed 674 times during 168 days of the trial. Microbiology reports and antibiotic guidelines were the two most commonly used (53% and 22.5%, respectively) types of evidence. The greatest reduction was observed in the use of β-lactamase–resistant penicillins and vancomycin.

Conclusion: Handheld computer-based decision support contributed to a significant reduction in patient length of stay and antibiotic prescribing in a critical care unit.

Evidence suggests that clinical decision support systems (DSSs) can lead to more appropriate clinical decision making and improve the quality of care.^1,2 However, the relationship between use of a DSS and patient outcomes remains uncertain.² A major difficulty has been providing the DSS access at the point where it is needed. Handheld computers, or personal digital assistants (PDAs), have been proposed as a way of delivering information to clinicians. Health care professionals are rapidly incorporating handheld computers into their practice. A survey by the American College of Physicians found that 47% of respondents currently use handheld electronic devices for their daily tasks.³ Clinicians use them to access patient information, medical textbooks, practice guidelines and drug databases; for writing prescriptions; and to perform medical computations. Though full of promise, the impact of this technology and associated models of clinical practice on patient management and outcomes has not been thoroughly studied.^4,5

Antibiotic prescribing in critical care represents a common, high-impact clinical decision with significant potential for improvement.^2,6 The demonstrated effect of antibiotic overuse on the development and spread of microbial antibiotic resistance in intensive care units (ICUs) led us to consider use of a DSS to promote more rational antibiotic prescribing.^6,7 In a previous study,⁸ we compared the impact of computerized decision support (with and without electronic access to clinical guidelines and laboratory data) on antibiotic prescribing decisions and demonstrated that a DSS provided a significant improvement in prescribing quality. The use of a DSS plus the microbiology report enhanced the agreement of care providers' decisions with those of an expert panel from 65% to 97% (p < 0.001) or to 67% (p = 0.02) when antibiotic guidelines only were accessed. The DSS plus microbiology reports had an even greater clinical impact.⁸ Importantly, in the evaluation of any DSS, both its effectiveness in improving decisions “in vitro” and its actual rate of adoption “in vivo” in the clinical environment need to be considered.^9,10 Therefore, a clinical trial was undertaken to assess the impact a handheld computer-based DSS had on empirical antibiotic prescribing in critical care.

Methods

System Design

A DSS was designed to provide “just-in-time” information to prescribers that included (a) a unit-specific, locally developed, antibiotic guideline for managing acute infections in critical care; (b) cumulative data from 2000 to 2003 on antibiotic resistance/susceptibility profiles for common bacteria isolated from patients in this ICU; (c) current inpatient microbiology laboratory reports, and (d) a clinical pulmonary infection score (CPIS)¹¹ calculator. The latter was included to allow clinicians to reevaluate and, if necessary, modify antibiotic therapy for ventilator-associated pneumonia three to four days after initiation.^7,11 Screenshots of these modules are presented in ▶. A pocket PC or PDA-based DSS containing antibiotic guidelines, patient microbiology reports and CPIS calculator was implemented on a Compaq iPAQ handheld device. Devices used in the study operated using Microsoft Windows CE version 3.0.9348 with ARM SA1110 processor and 31.25 MB memory.

Figure 1.

Screenshots of ventilator-associated pneumonia (VAP) risk assessment tool.

System Implementation

All content was either developed as HTML pages or translated into HTML for display in a Web browser. When large amounts of information were presented, they were displayed across several pages to minimize uncomfortable scrolling on long pages. JavaScript was used for active DSS pages, such as CPIS or ventilator-associated pneumonia (VAP) risk calculators. The iPAQ was also loaded with pathology reports for all current patients in the ICU. These reports were downloaded from a mainframe laboratory information system in HL7 format via File Transfer Protocol to a PC, translated into HTML with a Perl script, and transferred to the pocket PC during synchronization with the host PC (▶) located in the ICU. The system allowed the user to browse the content with Pocket Internet Explorer (Microsoft Corp., Redmond, WA).

Figure 2.

Antibiotic prescribing decision support infrastructure. 1, HL7 message, version 2.2 real-time after report validation; 2, HL7 message, version 2.3 every 10 minutes; 3, HL7 message file transfer every 10 minutes; 4, HTML pages via synchronization.

Access to the specific applications (antibiotic guidelines, microbiology reports, VAP risk assessment tool, or the local antibiotic prevalence data) generated log data that were temporarily stored on the handheld device and uploaded to the host PC when users docked their PDA on synchronization cradles. Therefore, synchronization served the double purpose of updating information stored on the devices and collecting usage logs that specified what decision support was used by clinicians and when.

Study Setting and Design

The trial was conducted in the ICU of Westmead Hospital, an 800-bed, university-affiliated tertiary center in Western Sydney, Australia. The ICU has 18 beds, provides medical and surgical services, and is staffed every day by a team consisting of an intensivist (usually trained in internal medicine), one or two postgraduate trainee(s), and two resident medical officers. No bedside computer terminals are available for information access. Participants were all senior medical officers in the ICU who were responsible for antibiotic prescribing decisions. In total, all 12 intensivists and advanced trainees employed in the unit at the time of the trial were recruited after signing informed consent forms. All participants were trained individually to use the system. Clinicians were given the device to use in the hospital as they wished, but there was no incentive or pressure to use the system. The study was approved by both the University of New South Wales and Western Sydney Area Health Service Human Ethics Committees.

The study was a prospective trial, with historical controls, of a handheld computer-based antibiotic prescribing DSS in an ICU. The control period lasted six months (24 weeks, April to September 2002), and no computerized decision support was available during this period to the prescribers in the unit. The intervention period also lasted six months (24 weeks, October 2002 to March 2003) when the system was available for routine use in the unit. There were only 12 hours of unscheduled downtime on one occasion due to failure of the hospital computer network.

Outcome Measurements

During the intervention period, electronic decision support usage was measured by the number of times any of its available functions were accessed on the handheld device. Data collected in the control and intervention periods included the number of admissions, the severity of illness indexes (APACHE [Acute Physiology, Age, and Chronic Health Evaluation] II and SAPS [Simplified Acute Physiology Score] II), and mortality and patient lengths of stay. Antibiotic consumption was calculated as the number of antibiotic courses in defined daily doses (DDDs) per 1,000 patient-days for each antibacterial agent based on data provided by the Pharmacy Department. The use of antiviral and antifungal agents was not included because it was unlikely to be affected by our intervention. Continuous variables were compared using the Student's t-test and chi-square statistics were used for categorical variables.

Results

The clinical characteristics of patients admitted to ICU during the study are summarized in ▶. The total number of ICU admissions, severity of clinical illness indices calculated on admission, and the mortality rates were statistically similar between the preintervention and intervention (or DSS use) periods. However, the mean patient length of stay decreased from 7.15 in preintervention to 6.22 bed-days during the DSS use (p = 0.02). The total numbers of multiresistant bacteria isolated from sterile sites of patients admitted to the ICU during the intervention were almost equal during the preimplementation and intervention periods. There were no documented outbreaks of hospital-acquired infection due to multiantibiotic-resistant bacteria during either study period. Retrospective analysis of patient length of stay in the unit for two years leading up to the study did not reveal significant seasonal variation.

Table 1.

Patient Outcomes and Characteristics of Pre- and Intervention Periods

Variable*	Preintervention	Intervention
No. of admissions
Mean	60.33	65.0
SD	5.46	4.29
95% CI	54.6–66.06	61.5–69.5
LOS
Mean	7.15	6.22†
SD	0.29	0.99
95% CI	6.85–7.45	5.18–7.26
Total mortality, %
Mean	11.5	13.17
SD	2.74	4.87
95% CI	8.63–14.37	8.05–18.29
Patient severity scores
APACHE II
Mean	20.0	20.3
SD	1.02	1.70
95% CI	18.93–21.07	18.22–22.08
SAPS II
Mean	33.53	34.85
SD	2.00	3.06
95% CI	31.43–35.63	31.64–38.06
No. of multiresistant bacteria isolated from sterile sites
Mean	18.3	18.8
SD	13.9	14.8
95% CI	3.72–32.94	3.23–34.43

LOS = length of stay (bed-days); CI = confidence interval; SD = standard deviation; APACHE II = Acute Physiology, Age, and Chronic Health Evaluation II; SAPS II = Simplified Acute Physiology Score II.

^*Average monthly figures.

^†p = 0.02 (chi-square test).

A total of 4,582 DDDs of broad-spectrum β-lactam antibiotics, fluoroquinolones, macrolides, carbapenems, and vancomycin were administered for 2,593 patient-days in the preintervention period (1,767 DDDs per 1,000 patient-days) and 3,766 DDDs for 2,583 patient-days (1,458 DDDs per 1,000 patient-days) in six months of the intervention period (p = 0.04). The data showed statistically significant decreases in consumption of two antibiotics most commonly used for broad-spectrum empirical therapy, during the intervention period (▶). Specifically, 546 and 261 DDDs per 1,000 patient-days of β-lactamase–resistant penicillins and vancomycin, respectively, were prescribed during the intervention period compared with 722 (p = 0.029) and 347 (p = 0.05) DDDs per 1,000 patient-days during the preintervention period. Fluoroquinolones and third-generation cephalosporins were also used significantly less during, than before, the DSS trial, whereas use of macrolides and cefepime increased slightly, but the difference was not statistically significant. First-generation cephalosporins, penicillin G, cotrimoxazole, teicoplanin, rifampicin, and metronidazole were used less frequently (158 and 148 DDDs per 1,000 bed days in total for respective periods). Their low utilization rates precluded significance testing, so those antibiotics were excluded from individual analysis.

Table 2.

Total Consumption of Most Commonly Used Antibiotic Classes in the ICU during the Study Expressed in DDD per 1,000 Patient-days

Characteristics	Preintervention	Intervention
β-lactamase–resistant penicillins†	722	546**
Third-generation cephalosporins‡	193	157
Cefepime	81	89
Fluoroquinolones§	171	146
Vancomycin	347	261††
Macrolides‖	115	130
Carbapenems¶	138	129
Subtotal	1,767	1,458‡‡
Others#	158	148
Total	1,925	1,606‡‡

ICU = Intensive care unit; DDD = defined daily dose.

^†Flucloxacillin, dicloxacillin, ticarcillin + clavulanate and piperacillin + tazobactam.

^‡Ceftriaxone, ceftazidime.

^§Ciprofloxacin, gatifloxacin, moxifloxacin.

^‖Erythromycin, clindamycin, roxithromycin.

^¶Imipenem, meropenem.

^#Penicillin G, cephalexin, cephalothin, cefazolin, cotrimoxazole, metronidazole, rifampicin, teicoplanin.

^**p = 0.029.

^††p = 0.05.

^‡‡p = 0.04.

Computer log files indicate that the DSS was used 674 times during 168 days of the trial, or four times per day, on average. Handheld devices were used to access recent microbiology reports between five and 15 times per week. Cumulative antibiotic resistance data and the VAP risk assessment tool were accessed less frequently, between one and ten times per week. Two peaks of usage of cumulative antibiotic susceptibility data during weeks 10 to 12 and 18 to 19 are correlated with the release of the 2003 annual statistics and the arrival of new registrars in January 2003, respectively.

Access to microbiology reports was the most common indication for use of the system: 53% of accesses on average were to look up laboratory data. Antibiotic guidelines were the second most commonly used feature (22.5%); antibiotic susceptibility data and VAP risk assessments (CPIS calculator) contributed only 16% and 9% of log-ins to the system, respectively. The majority (around 70%) of DSS use took place on weekdays, with little activity on weekends. After the DSS implementation, five of six registrars and five of six consultants (83%) used the system. However, the level of use was higher among registrars who were responsible for accessing 92% of microbiology data and antibiotic guidelines and 94% of use of the VAP risk assessment tool. Consultants, who were responsible for 24% of accesses, most frequently accessed the unit-specific antibiotic susceptibility data.

Discussion

Our results suggest that the use of the DSS contributed to the reduction of patient length of stay in the ICU, which is an important surrogate for overall costs. This significant impact of our system is plausible as there were neither significant differences in the patient mix nor outbreaks of infection due to multiresistant organisms between the preintervention and intervention periods. However, we were unable in this study to identify the specific contribution of using a handheld platform over fixed to this result.

The introduction of the DSS was associated with a reduction in antibiotic usage in the ICU and coincided with a change in patterns of antibiotic use in the ICU. The decrease in administration of β-lactamase–resistant penicillins (predominantly ticarcillin + clavulanate and piperacillin + tazobactam) and vancomycin is not surprising. These antibiotics are prescribed extensively in critical care units to provide broad-spectrum cover for suspected infection, and their use is likely to be susceptible to interventions designed to optimize antimicrobial usage.¹² As the numbers of patients treated with β-lactamase–resistant penicillins, vancomycin, and third-generation cephalosporins were similar in the preintervention and DSS trial periods, it was assumed that the overall reduction in their use was due to decreases in the average duration of antimicrobial therapy. This is in line with recent findings that the application of clinical guidelines in critical care can decrease the average duration of therapy.¹³

Another important observation was the difference in use of the DSS among clinicians with different roles. Senior clinicians accessed local antibiotic susceptibility data more often than any other DSS component. This is not surprising, given those clinicians' expertise and confidence in the management of infection. These data are the basis of antibiotic policy reviews and quality of health care assessments.

Our study has several limitations. First, it was carried out for a relatively short period of time in a single critical care unit with a specific decision-making environment and microbial ecology and a limited number of participants. Antimicrobial use in a busy ICU at a teaching hospital may differ from that in a nonteaching hospital, but previous studies have also shown the significant impact of antibiotic management protocols on antibiotic prescribing in different settings.^1,14 Although our findings may not be applicable to institutions with intensive antimicrobial control programs, the majority of hospitals lack such programs.

Second, the observed association between the intervention and the changes in outcomes and process measures does not necessarily prove a direct cause-and-effect relationship. It is possible that the effects reflect influences external to the study intervention, such as seasonal fluctuations in the incidence of infections, or a Hawthorne effect (temporary increase in the quality of work due to the stimulus of being singled out and observed). However, the severity of illness scores of patients presenting to the unit were similar during the preintervention and DSS trial periods. Furthermore, simultaneous increases in the administration of cefepime and macrolides would be unlikely if a Hawthorne effect had been solely responsible for the trend demonstrated in the study.

Third, this study is limited by the fact that we used a historical control group. However, the before/after approach is the most commonly applied design for evaluation of a clinical DSS because it controls the most important confounding variable—the innate characteristics of study participants.² The DSS usage was relatively infrequent compared with the number of prescribing decisions made by clinicians on a daily basis, but the effects observed suggest that even relatively small additional applications of information may lead to improvements in clinical decisions and patient outcomes. Our findings demonstrate the need for a randomized multicenter trial to more accurately quantify the impact of the DSS on practice and clinical end points.

Conclusion

Computer-based DSSs may help to significantly reduce the length of stay and antibiotic prescribing in critical care. Handheld computer-based DSSs can be useful for this purpose in environments lacking widely distributed, networked workstation-based systems. The study results contribute to our understanding of the role of point-of-care decision support in clinical practice and patient management and to identification of clinically relevant and useful information support tools to aid clinical decision making.