Use of Electronic Medical Record–Enhanced Checklist and Electronic Dashboard to Decrease CLABSIs

Abstract

OBJECTIVES:

We hypothesized that a checklist enhanced by the electronic medical record and a unit-wide dashboard would improve compliance with an evidence-based, pediatric-specific catheter care bundle and decrease central line–associated bloodstream infections (CLABSI).

METHODS:

We performed a cohort study with historical controls that included all patients with a central venous catheter in a 24-bed PICU in an academic children’s hospital. Postintervention CLABSI rates, compliance with bundle elements, and staff perceptions of communication were evaluated and compared with preintervention data.

RESULTS:

CLABSI rates decreased from 2.6 CLABSIs per 1000 line-days before intervention to 0.7 CLABSIs per 1000 line-days after intervention. Analysis of specific bundle elements demonstrated increased daily documentation of line necessity from 30% to 73% (P < .001), increased compliance with dressing changes from 87% to 90% (P = .003), increased compliance with cap changes from 87% to 93% (P < .001), increased compliance with port needle changes from 69% to 95% (P < .001), but decreased compliance with insertion bundle documentation from 67% to 62% (P = .001). Changes in the care plan were made during review of the electronic medical record checklist on 39% of patient rounds episodes.

CONCLUSIONS:

Use of an electronic medical record–enhanced CLABSI prevention checklist coupled with a unit-wide real-time display of adherence was associated with increased compliance with evidence-based catheter care and sustained decrease in CLABSI rates. These data underscore the potential for computerized interventions to promote compliance with proven best practices and prevent patient harm.

In 1999 and 2001, the Institute of Medicine published the landmark reports To Err Is Human and Crossing the Quality Chasm, which highlighted the excessive number of preventable medical errors and called for urgent changes to our health care system to improve patient safety and quality of care.^1,2 Among the highly prevalent and preventable sources of patient harm are hospital-acquired infections, which increase morbidity, mortality rates, and health care costs in both adults and children.^3–7 An estimated 250 000 hospital-acquired bloodstream infections occur each year,⁷ and central line–associated bloodstream infections (CLABSIs) are the most common hospital-acquired infection in children admitted to the PICU.^8,9 Although many evidence-based interventions have been demonstrated to reduce patient harm, including CLABSIs,¹⁰ implementation in critically ill patients is often confounded by the sheer volume of tasks that must be completed for each patient throughout the day.¹¹

To effectively address all the potential sources of patient harm, systems of care must be developed to standardize care, improve communication, and promote timely access to information at the point of care.^12–16 The electronic medical record (EMR), with integrated nursing documentation, computerized physician order entry, and clinical decision support, has the potential to facilitate achievement of these goals.^14,17 With thoughtful design and implementation, these tools can be used to automate practices and highlight compliance with interventions via continuously updated visual displays that extract the most up-to-date information directly from the EMR.¹¹

This study was designed to test the hypothesis that successful implementation and adoption of an EMR-enhanced checklist tethered to a real-time unit-wide dashboard would decrease CLABSIs in the PICU. The secondary hypothesis was that this intervention could improve care provider team communication and knowledge.

Methods

Setting

Lucile Packard Children’s Hospital (LPCH) is a 303-bed, freestanding, quaternary care academic children’s hospital with a 24-bed PICU. The PICU at LPCH cares for a wide range of critically ill patients, including solid organ and bone marrow transplant recipients, trauma patients, and a full complement of medical and surgical patients. Postoperative patients with congenital heart disease are managed in a separate cardiovascular ICU. There are ∼1300 admissions per year in the PICU. The PICU has implemented an EMR (Cerner Corporation, Kansas City, MO) with nursing documentation, biomedical device integration, physician documentation, and computerized provider order entry (Healthcare Information and Management Systems Society level 6).¹⁸ This project was reviewed and approved by the Stanford University School of Medicine Institutional Review Board. We followed the Standards for Quality Improvement Reporting Excellence publication guidelines for quality improvement in the implementation and reporting of this initiative.¹⁹

Prework

LPCH participated in the National Association of Children’s Hospitals and Related Institutions (NACHRI) collaborative efforts to reduce CLABSI rates from June 2009 to November 2010. This included implementation of standardized central line insertion and maintenance bundles in the PICU.²⁰ The collaborative showed that focused attention on adherence to pediatrics-specific central line insertion and maintenance care resulted in significant reductions in PICU CLABSI rates.²¹ Additional institution-specific measures were implemented, including development of a PICU-based CLABSI prevention workgroup, an executive oversight committee, and a process for root cause analysis of every CLABSI event. All CLABSIs were reviewed by the institutional infection control team.

Through the collaborative efforts, CLABSI rates in the PICU decreased to 2.4 per 1000 line-days, with a goal to reduce rates further. Institutional compliance with specific elements of the recommended central line maintenance bundle for catheter care remained as low as 30%, well below the collaborative goal of 80%. To increase the efficiency, sustainability, and educational impact of the CLABSI reduction efforts at LPCH, we developed the computerized intervention described in this study.

Intervention

In an effort to improve implementation of and compliance with best practice bundles at LPCH, we developed a computerized solution that hardwired relevant best practice bundles in the EMR, using existing documentation in the EMR to guide compliance. We developed a unique page in the EMR for each patient to highlight critical clinical data and ensure compliance with evidence-based best practice bundles (Fig 1). The page displayed actionable items, provided color-coded visual indicators to convey compliance, and included a checklist that combined components of the NACHRI pediatric-specific central line insertion and maintenance bundles to prevent CLABSI.^20,21 The checklist connected logic to nursing and physician documentation and provided pertinent information and educational links to aid clinician decision-making so that only relevant pieces of the checklist were displayed for consideration. For example, dressing change was highlighted by a red light when the change was due, and port needle change information was displayed only when port needles were in place. The checklist was further enhanced by the provision of pertinent information and education to aid clinician decision-making (eg, number of line-days, links to institutional policy and relevant literature on dressing changes, intravenous medications for possible conversion to oral formulations) (Fig 2).

FIGURE 1.

EMR-enhanced checklist. A unique tab was developed in the EMR for each individual patient to highlight critical clinical data and display EMR-enhanced checklists to ensure adherence to evidence-based protocols.

FIGURE 2.

Supplemental information and education on EMR-enhanced checklist. Upon clicking on the “Line Necessity” item in the EMR-enhance CLABSI prevention checklist, a second window opens with information about the line duration and signs of infection as well as educational links to aid clinician decision-making.

In addition, an electronic multipatient dashboard was displayed on a large monitor at a central location in the PICU (Fig 3). The dashboard provided a real-time unit-wide overview of critical patient data, specifically highlighting compliance with the maintenance bundle for CLABSI prevention for all eligible PICU patients.

FIGURE 3.

Unit-wide patient safety and quality dashboard. An electronic unit-wide patient safety and quality dashboard was displayed on a large monitor in a central location in the PICU. The dashboard was designed to enhance situational awareness by providing a unit-wide overview of critical patient data and specifically highlighting whether there was compliance with the CLABSI prevention guidelines. If there was compliance with all the items on the CLABSI prevention checklist, a green light was visible for that patient on the unit-wide dashboard. If any of the items were out of compliance or required attention, then a yellow or red light displayed for that patient.

The EMR-enhanced checklist was designed for use during rounds on each patient. All PICU attending physicians, fellows, and nurses received training on the use of the EMR-enhanced checklists and unit-wide dashboard before implementation. Throughout the implementation period (May 1–August 31, 2011), PICU nurse practitioners served as unit-based champions to promote use of the EMR-enhanced checklists on rounds.

Study Design

To determine the effect of the intervention on compliance with evidence-based guidelines, we conducted a cohort study using historical controls in the PICU at LPCH. Participants were included if they were admitted to the PICU during the preintervention or postintervention period. Data on compliance with the prevention bundle were collected during the preintervention period (November 1, 2010 to April 30, 2011). Data on CLABSI rates were consistently collected from the onset of LPCH participation in the NACHRI collaborative, so the preintervention period for CLABSI rates was June 2009 to April 30, 2011. The postintervention period was September 1, 2011 to December 31, 2012. The period from May 1, 2011 to August 31, 2011 was treated as the implementation rollout and optimization period and was not included in the analysis. CLABSI rates were included in the analysis starting June 1, 2009, when we began involvement with the NACHRI collaborative efforts to reduce CLABSI, so that the only major difference between the pre-evaluation and postevaluation period for CLABSI rates in this study was the implementation of the EMR-based checklists and dashboard. CLABSI rate data and compliance with the prevention guidelines were collected from data documented in the EMR.

Outcome Measures

The primary outcome measures were CLABSI rates and compliance with elements of the CLABSI prevention bundle. The CLABSI prevention best practices assessed were compliance with insertion bundle best practices (captured via a form in the EMR that is completed by provider after line insertion), documentation of line necessity, appropriate timing of dressing changes, appropriate timing of cap changes, and appropriate timing of port changes. Two checklist items were added to those in the NACHRI insertion and maintenance bundles: evaluation of possible transition of intravenous (IV) medications to oral medications and evaluation of appropriateness of laboratory frequency. These issues were not assessed before the intervention and therefore were not compared before and after intervention.

Secondary outcome measures included perceptions of team communication and CLABSI prevention knowledge based on survey. The anonymous survey was administered to all nurses, attending physicians, and pediatric critical care medicine fellows working in the LPCH PICU immediately before the intervention and 6 months after the end of the implementation phase (Table 1). Time and motion studies were conducted randomly once per week for 6 months after the intervention to collect descriptive data about dashboard use, to record the time needed to review the dashboard, and to track the rate at which dashboard review resulted in a change in the patient’s plan.

TABLE 1.

Summary of Outcome Measurements in Preintervention and Postintervention Populations

	Preintervention	Postintervention	P
CLABSI ratea	2.6	0.7	.02
Daily compliance with CLABSI prevention measuresb	n = 1942 line-days	n = 5041 line-days
Insertion bundle, %	67	62	.001
Line necessity review and documentation, %	30	73	<.001
Dressing change frequency, %	87	90	.003
Cap change frequency, %	87	93	<.001
Port needle change frequency, %	69	95	<.001
Transition from IV to oral medications, %	NA	71	NA
Laboratory frequency, %	NA	65	NA
Survey questions	n = 64 staff	n = 65 staff
How effective is the current rounding format in the PICU for communicating your patient’s needs to all team members?c	4.3	4.4	.73
Does the medical team address central line necessity on rounds?d	4.5	4.9	.02
Do you know the components of the maintenance bundle for CLABSI prevention?e	53 yes	49 yes	.39
	3 no	9 no
	7 unsure	5 unsure

NA, not applicable.

^aCLABSI per 1000 line-days. P represents analysis with 2-sample t test assuming equal variances.

^bPercentage compliance with CLABSI prevention best practices. Analyzed by using χ² tests.

^cPreintervention and postintervention results represent the average score on a scale of 1 to 6, where 1 means “very ineffective” and 6 means “very effective”. Analyzed with a Wilcoxon rank-sum test.

^dPreintervention and postintervention results represent the average score on a scale of 1 to 6, where 1 means “never” and 6 means “every time I work”. Analyzed with a Wilcoxon rank-sum test.

^eAnalyzed with Fisher’s exact test.

Audit and survey data were collected and managed by using REDCap (Harvard Catalyst, Boston, MA) electronic data capture tools.²²

Statistical Analysis

A quasiexperimental approach was used to determine the impact of the EMR-enhanced checklist on compliance with the maintenance bundle for CLABSI prevention and the effect on CLABSI rates. Initial analysis included a 2-sample t test assuming equal variances to assess differences in CLABSI rates. Changes in compliance with the components of the insertion and maintenance bundles in the preintervention and postintervention periods were analyzed by using χ² tests. To more rigorously evaluate the association between the intervention and the timing of the effect on CLABSI rates and to account for potential trends in the preintervention and postintervention periods, regression models were used to estimate CLABSI rates that incorporated month-to-month trends as well as the intervention indicator variable. Autoregressive integrated moving average (ARIMA) analysis was used to model the potential autocorrelations in time.²³ Akaike information criterion was used for covariance model selection to avoid model overparameterization.²⁴ All models were adjusted for severity of illness using Pediatric Risk of Mortality III (PRISM-III) 12-hour scores.²⁵

To evaluate the change in perception of team communication, Wilcoxon rank-sum tests were performed to test for a shift in the median response for the question with ordinal responses, and Fisher’s exact test was performed for the questions with yes–no responses. Descriptive statistics including mean and SD were calculated for the time added to rounds to review the dashboard.

R statistical software was used for statistical analysis.²⁶

Results

Rates of CLABSI decreased from 2.6 per 1000 line-days (19 CLABSIs/7322 total line-days) before intervention to 0.7 CLABSIs per 1000 line-days (7 CLABSIs/6155 total line-days) after intervention (Table 1, Fig 4). A regression analysis was performed to model CLABSI rates on both the intervention variable and severity of illness. The intervention variable remained statistically significant (P = .029) for the risk-adjusted CLABSI rates.

FIGURE 4.

CLABSI rates per 1000 line-days by month. CLABSI rates decreased from 2.6 CLABSIs per 1000 line-days before implementation of the EMR-enhanced checklists and the patient safety and quality dashboard (June 1, 2009–April 30, 2011) to 0.6 CLABSIs per 1000 line-days after intervention (September 1, 2011–December 31, 2012). UCL = upper control limit.

The ARIMA model includes estimates of correlation between the rate for any particular month with the rates for previous months. This allows us to take into account that rates that are observed over time may not be considered independent observations as in a typical regression analysis. In this case the model identified that the rate on any particular month is correlated with the values 3 and 4 months before that month. One possible explanation for this might be that the occurrence of an infection may increase the staff awareness and diligence in carefully following protocol the few months that follow. Regardless of the reason, we are able to account for this correlation over time in the model and still estimate the effect of the intervention in reducing CLABSI rates. Given this possible autocorrelation, our model was fit with an ARIMA model of order 4. In this model, the intervention variable (P < .0001) remained statistically significant. The estimated effect of the intervention is a rate reduction of 1.8 per 1000 line-days, with a 95% confidence interval of 0.32 to 2.55 per 1000 line-days.

For evaluation of compliance with components on the CLABSI prevention bundle, a total of 251 patients (1943 patient-days) were evaluated in the preintervention period and 609 patients (5041 patient-days) were evaluated in the postintervention period. Compliance improved for daily review and documentation of line necessity (P < .001), frequency of dressing changes (P = .003), frequency of cap changes (P < .001), and frequency of port needle changes (P = .001), but compliance with documentation of the insertion bundle decreased (P < .001) (Table 1). A logistic regression analysis was performed to evaluate the impact of severity of illness (as assessed by PRISM III scores) on the intervention effect. The changes in compliance with documentation for all variables remained statistically significant (all with P < .002) with PRISM scores incorporated into the model.

The time and motion studies were conducted 24 times, captured 193 separate episodes of patient rounds, and reflected 11 different attending intensivists. The dashboard was used in 84% of patient rounds episodes, and review of the checklists added an average of 37 seconds (SD 78 seconds) to rounds on each patient. A care change (eg, removal of a central line, change from IV to oral medications, reduction of laboratory frequency) was made during review of the checklist on 39% of the patient rounds episodes. Preimplementation and postimplementation survey results are also described in Table 1. There was a 78% survey response rate before intervention and 77% response rate after intervention. After intervention, there was a significant increase in the perception that the medical team addresses central line necessity during rounds.

Discussion

Implementation of automated clinical decision support, in the form of EMR-enhanced checklists and a unit-wide patient safety and quality dashboard, was associated with decreased CLABSI rates and improved compliance with CLABSI prevention best practices. The use of these real-time clinical decision support tools to promote ongoing quality improvement interventions was associated with significant improvements over the gains realized by previous CLABSI prevention efforts.

There are several potential reasons that we were able to improve our CLABSI rates using this intervention. First, checklists in general have been shown to improve compliance with best practices.^20,27–29 Second, linking the checklist electronically to nursing and provider documentation, clinical decision support, and immediately actionable orders resulted in real-time correction of any noncompliance. Third, integrating the checklist review into daily rounds ensured the regular, daily review of the checklist. Because the EMR was already being used in rounds at our institution, integrating the checklist into the EMR ensured ready access during rounds. Finally, continuous visual display of compliance on a unit-wide multipatient dashboard in the PICU increased awareness of gaps in compliance and facilitated unit-wide efforts to maintain compliance.

Checklists have the potential to be enhanced by information technology. There is a growing body of evidence that information technology and clinical decision support can facilitate compliance with evidence-based best practices and improve quality of care,^1,2,30–33 and specifically can promote compliance with checklists.³⁴ A study of 19 Michigan ICUs participating in the Keystone Project to reduce CLABSI rates suggested that the presence of more sophisticated information technology in the ICUs was associated with greater reductions in CLABSI rates within the collaborative.³⁴ Previous reports from our group offer additional support for the notion that computerized interventions in the PICU improved compliance with blood transfusion guidelines¹⁵ and reduced unnecessary laboratory use.¹⁶

As the number of evidence-based patient safety interventions continues to climb,¹⁰ a significant burden is placed on health care professionals to ensure compliance with all measures to prevent patient harm.¹¹ In fact, a previous effort to integrate paper checklists into PICU rounds at our own institution, though temporarily successful, proved unsustainable because of the additional work burden imposed on providers.³⁵ Integration and automation of these measures are critical determinants of durable prevention of patient harm.³⁶ This intervention demonstrated that CLABSI prevention efforts can be enhanced by embedding checklists in the EMR and enriching the checklists with data from the EMR and appropriate educational links. Although CLABSI prevention demonstrates proof of concept in this report, the tools are applicable to a wide array of quality improvement initiatives. We are developing EMR-enhanced checklists to facilitate compliance with prevention efforts for ventilator-associated pneumonia, catheter-associated urinary tract infections, pressure ulcers, and deep vein thrombosis.

The unit-wide, multipatient dashboard added an extra layer of visibility to our CLABSI prevention efforts. Patient dashboards represent a noninterruptive form of clinical decision support that provide real-time graphical representation of clinical data and can streamline information processing, facilitate situational awareness, and improve compliance with evidence-based guidelines.^37,38 Displaying the dashboard at the nursing station in the PICU enabled staff members to quickly review compliance problems and allocate resources appropriately.

The effects of this intervention on perceptions of team communication and knowledge are less clear. Although the audit data showed that the tool was incorporated into rounds at a high rate, and both the compliance data and survey data support the notion that there was more discussion of line necessity on rounds, the survey data did not reveal a perception of overall improved team communication or knowledge of the CLABSI compliance bundle. The primary goals of improved compliance with patient safety interventions and decreased CLABSI rates were met, but greater understanding of the impact these tools have on provider knowledge and attitudes might enable further refinement of these tools. More sensitive measures of knowledge and attitudes may provide better information about these effects with future interventions.

Although it was not within the scope of this study to do a formal cost analysis, we estimate a significant return on the initial investment to develop, program, and implement this intervention. This tool was developed with a $100 000 pilot study grant. The direct attributable cost of nosocomial bloodstream infections in the PICU is approximately $39 000.⁴ With an estimated decrease in CLABSIs of 1.8 per 1000 line-days attributable to this intervention and approximately 3700 line-days per year in our PICU, we estimated a cost savings of approximately $260 000 per year in our PICU.

There are several limitations of this study. First, we used a quasiexperimental pre–post study design to evaluate the effect of the intervention. Therefore, although there was an association between the intervention and the desired effects, causation cannot be established. Second, efforts to implement all components of insertion and maintenance bundles were under way before, during, and after implementation of the electronic checklist and dashboard. Thus, it is difficult to distinguish the impact of these efforts alone on compliance or CLABSI rates. Third, compliance assessments for several elements on the checklist (including dressing changes, cap changes, and port needle changes) were based directly on nursing documentation. We are not able to determine whether the improved compliance with these elements was secondary to improved performance of the interventions or simply improved documentation by the nurse as a result of the reminders from the dashboard. Fourth, the elements of the insertion bundle were not captured in routine nursing and physician clinical documentation but were captured via an electronic template completed by the provider after central line insertion. This process flow resulted in errors of omission and varied rates of provider documentation of central line insertion and may have contributed to the dashboard’s ineffectiveness for this element. Additionally, these data were not captured reliably for lines that were placed in the operating room due to different processes for documentation. Finally, these results reflect outcomes in a single PICU with a high severity of illness, which may render the intervention less generalizable to settings with lower acuity of illness.

Conclusions

An EMR-enhanced patient checklist in conjunction with a unit-wide dashboard led to improved compliance with CLABSI prevention bundles, decreased CLABSI rates, and improved staff communication. Using information technology to provide actionable checklists of evidence-based best practices and visible evidence of compliance can promote implementation of care initiatives. Future research should evaluate whether tools such as these can be extended to automate and facilitate compliance with multiple best practice bundles simultaneously, thereby promoting a system of care to prevent multiple sources of patient harm.

Glossary

ARIMA

autoregressive integrated moving average

CLABSI

central line–associated bloodstream infection

EMR

electronic medical record

IV

intravenous

LPCH

Lucile Packard Children’s Hospital

NACHRI

National Association of Children’s Hospitals and Related Institutions

PRISM III

Pediatric Risk of Mortality III