Beyond checklists: Using clinician prompts to achieve meaningful ICU quality improvement

Expanded abstract

Citation

Weiss CH, Moazed F, McEvoy CA, et al. Prompting physicians to address a daily checklist and process of care and clinical outcomes: a single-site study. Am J Respir Crit Care Med 2011, 184:680-686. Epub 2011 May 26. PubMed PMID: 21616996.

Background

Checklists may reduce errors of omission for critically ill patients.

Methods

Objective

To determine whether prompting to use a checklist improves process of care and clinical outcomes.

Design

Prospective, concurrently-controlled cohort study with additional historical controls.

Setting

Medical intensive Care Unit (MICU) of a tertiary care university hospital.

Subjects

All patients admitted to either of two independent MICU teams.

Intervention

Intervention team physicians were prompted to address six parameters from a daily rounding checklist if overlooked during morning work rounds. The second team (control) used the identical checklist without prompting. Pre-intervention patients received neither a checklist nor prompting.

Outcomes

Primary outcome included differences between the prompted and control groups related to several key quality indicators being investigated- ventilator-free days, duration of empirical antibiotics, duration of central venous catheters, duration of foley urinary catheter, pharmacological deep vein thrombosis (DVT) prophylaxis per eligible days and, stress ulcer prophylaxis per eligible days. Secondary outcome included ICU mortality, hospital mortality, ICU length of stay and ventilator associated pneumonia.

Results

One hundred and forty prompted group patients were compared with 125 concurrent control patients and 1283 pre-intervention patients. Compared with control patients, patients admitted to the service with prompting experienced increased ventilator-free days, decreased empirical antibiotic and central venous catheter duration, and increased rates of deep vein thrombosis and stress ulcer prophylaxis. Prompted group patients had lower risk-adjusted ICU mortality compared with the control group (odds ratio, 0.36; 95% confidence interval, 0.13-0.96; P = 0.041) and lower hospital mortality compared with the control group (10.0 vs. 20.8%; P = 0.014), which remained significant after risk adjustment (adjusted odds ratio, 0.34; 95% confidence interval, 0.15-0.76; P = 0.008). Observed-to-predicted ICU length of stay was lower in the prompted group compared with control (0.59 vs. 0.87; P = 0.02). Checklist availability alone was not associated with reductions in mortality or length of staycompared with the pre-intervention patients.

Conclusions

In this single-site, preliminary study, checklist-based prompting improved multiple processes of care, and may have improved mortality and length of stay, compared with a stand-alone checklist. The manner in which checklists are implemented is of substantial importance in the care of critically ill patients.

Commentary

Despite recent advances in the care of patients with critical illness, mortality in the intensive care unit (ICU) remains high. Overall mortality in the ICU is about 15% and this increases to about 50% for high risk syndromes such as acute lung injury and sepsis [1,2]. Some of the mortality and morbidity in the ICU are clearly avoidable. And although recent efforts have focused on patient safety in an effort to reduce mortality through reductions in medical errors, a much larger cause of preventable mortality is likely the medical community's failure to implement evidence-based practice in the ICU.

One approach to improving the translation of evidence into practice is through checklists. Checklists and, more broadly, protocol-driven care are advocated as a way to reduce errors of omission, simplify care, decrease in-appropriate variation and expedite the application of effective advances to everyday practice [3-5]. In theory, checklists work by overcoming many of the barriers to implementing guideline-based care, including errors based around knowledge, attitude and clinical behaviors [3].Yet checklists alone may not be sufficient to change practice since to be useful they must be implemented effectively and consistently [6].

To address strategies to improve checklist implementation, Weiss and colleagues conducted a prospective cohort study of ICU patients. The authors took advantage of a unique admitting process whereby two distinct ICU teams admitted patients on alternate days. The intervention in question was "regular verbal prompting" by a non-care-providing resident physician (the prompter) directed at the critical care attending physician or fellow of one of the ICU teams (prompted team), using scripted questions if any of the following six parameters were overlooked on daily work rounds: mechanical ventilation weaning, empirical antibiotics, central venous catheters, foley urinary catheters, deep vein thrombosis prophylaxis and stress ulcer prophylaxis. Patients admitted to an unprompted ICU team, with availability of an identical checklist, served as controls. The two groups were also compared to a cohort of more than 1200 patients admitted to the ICU one year prior to intervention and checklist introduction.

The authors hypothesized that prompting would lead to superior process of care and increased clinical outcomes compared to the unprompted use of an identical checklist.

A total of 265 patients were prospectively enrolled during the 82-day intervention period. For patients in the intervention group, the prompter was relatively busy, with prompting required on 64.7% of patient-days. Amount of prompting varied with each care practice investigated being highest with foley urinary catheter use.

Compared with the control group, the prompted group had significant improvement in all the key quality indicators investigated apart from foley urinary catheters duration. Despite the APACHE IV-predicted mortality being similar in both groups, both ICU and hospital mortality were lower in the prompted group compared with the control group. The mortality benefit was strongest in the second through fourth quartiles of predicted mortality and there was no difference in mortality in the lowest patient quartile or highest range of predicted mortality providing the study with some validity as ICU interventions would not be expected to affect mortality when risk of death is very high or very low. Interestingly, there was no difference in hospital mortality between the checklist-only group and the pre-intervention group, suggesting that the checklist alone had no impact on patient outcomes.

The strengths of the study include the fact that it was well designed and included a control group where an ICU team that was rounding on alternate days were encouraged to use a checklist but did not have a prompter to remind them to use it. Prior ICU organizational studies used only historical controls (i.e. before-after studies) and can be biased by coincident interventions, temporal trends and regression towards the mean [7,8]. By including concurrent controls, the authors strengthened the ability to make causal inference about the intervention. The study also targeted a broad range of care practices and examined patient-centered outcomes. Overall the investigators have introduced an innovative and potentially powerful way to improve outcomes.

However, there are some limitations that deserve consideration. Primarily, the generalizability and sustainability of the quality improvements are unknown, as the study was carried out in a single-center and over a short time period. Additionally, the intervention resulted in an implausibly large treatment effect reducing the odds of death by about 2/3 compared to the control arm. Considering the amount of prompting actually carried out, it is likely that the presence of a prompter and not prompting itself or some other factor may have contributed to outcomes seen. Lastly, it would be impractical to use a resident physician as a prompter during rounds in every ICU. Although face-to-face prompting might have been a positive attribute of this study, its applicability in the real world is unknown.

Recommendation

The study clearly shows that checklists or indeed any quality improvement measure requires a robust implementation and an accountability strategy to change behavior. While prompting is clearly a novel and probably an effective approach to checklist implementation, the jury is still out on the optimal approach to prompting. With the advent of multidisciplinary care model in the ICU in which daily rounds are done by the multidisciplinary care team [9], any member of the team could take the responsibility of being an effective prompter. Based on this study, ICUs using checklists as a means to quality improvement should consider using verbal or other forms of prompting to improve implementation. Although a confirmatory pragmatic trial would be helpful, this methodically sound study has created a benchmark for future quality improvement research especially as it relates to checklist implementation.

Competing interests

The authors declare that they have no competing interests.