Back to basics: checklists in aviation and healthcare

The checklist approach has the same potential to save lives and prevent morbidity in medicine that it did in aviation over 70 years ago by ensuring that simple standards are applied for every patient, every time.¹

Healthcare safety activists have looked to checklists to solve a myriad of problems, particularly with the current iteration of checklists that have been imported from aviation. Large-scale implementations with conflicting outcomes suggest that these tools are not as simple or effective as hoped. Scholars debating the efficacy of checklist implementation in healthcare have identified important reasons for varying results: that success requires complex, cultural and organisational change efforts, not just the checklist itself²; that results may be confounded by a mix of the technical and socioadaptive elements,³ and that local contexts may either augment or undermine the implementation's outcomes.⁴

When ideas are translated from one industry to another, the assumptions underlying the original concepts may be lost or diluted. As checklists are increasingly imposed through a variety of professional and regulatory mandates in North America,⁵ Europe⁶ and elsewhere,⁷ perhaps it is time to review the fundamental principles of checklist use, including why they might work and how we can implement them better.

Checklists in aviation and their analogues in medicine

Aviation checklists are designed for modern aircraft that are complicated, not complex; it is usually possible to define a single process path that offers optimum performance for each flight condition. These process paths are flight tested, endorsed (with minor modifications) by airlines when they purchase a new aircraft type, and published in procedural manuals and checklists. There are two categories of checklist used in the cockpit: normal and non-normal (or emergency) procedures.

Normal procedures

Normal checklists are completed whenever the aircraft configuration needs to be altered as part of an everyday flight. There are differing approaches to executing the checklist,⁸ with varying degrees of redundancy, but all methods include action and verification steps.

In healthcare, this type of operational check of equipment has evolved along with advanced medical technology. Since 1993, Anaesthesia Apparatus Checkout Recommendations have targeted the proper configuration of anaesthesia gas delivery systems.⁹ These recommendations are intended to be peer-reviewed, modified and updated for each specific type of manufactured anaesthesia equipment. Thus, prechecks are incorporated into manufacturing and inservicing of equipment and iteratively updated by the professionals using them. Operating room and anaesthesia workflows accommodate these prechecks.

‘Normal’ checklists are effective whenever there are advantages to standardising performance, time is not critical, the series of tasks is too long to be committed to memory (or there are likely to be interruptions to execution of the task that might interfere with memory retrieval), and the environment enables a physical list to be accessed and used.

Emergency (non-normal) procedures

Checklists developed to address emergency situations may contain boldface, non-boldface or flowchart items. The aircraft manufacturer selects one of the following formats based on the expected severity of the problem, and the time likely to be available to solve it.

1. Boldface items are for immediate action, when the aircraft may be lost if the items are not completed quickly and in the correct order. Correct and rapid execution of these steps is so critical and essential, that pilots must complete them from memory. Through training and repetition, the paired cognitive and motor activities required to perform the checklist are stored by the pilot as procedural memory (or ‘motor skills’).¹⁰ Despite notable exceptions (such as ‘choking’ under pressure), procedural memory retrieval is less affected by stress than declarative or episodic memory retrieval.¹¹ For this reason, aircrew practice time critical emergency procedures regularly to aid in forming the correct ‘habits’. However, as soon as time permits, the checklist is used to confirm that the steps were executed as required.⁸

In healthcare, this technique is used in resuscitation procedures. Healthcare workers are trained and recertified, in low and high fidelity simulators, to commence procedures such as Advanced Cardiac Life Support without referring to a checklist. When the patient is not readily revived or responding as expected, the team will refer to their checklists or algorithms to make sure the steps have been executed properly, and that they have not forgotten anything. For this reason, healthcare workers often keep a cognitive aid (a ‘checklist’ of sorts) posted on emergency carts, tucked into pockets or loaded onto mobile devices. ‘Boldface’ checklists can be effective whenever there is a critical sequence to be completed but time is short, or the situation does not enable a physical list to be immediately accessed and used.

• Non-boldface checklists are used to provide decision support when time is not critical. In complicated situations, such as multiple system failures, the checklist appears in the form of a flow chart or decision tree, helping the pilot(s) to navigate the process. In modern aircraft, the checklist is built into the electronic cockpit system, which leads the pilots through the appropriate steps on the screen. The steps are colour-coded for urgency and ranked in priority order. As steps are completed, they disappear from the screen. Checklist items are arranged in a systems operational sequence and are consistent with the patterns of motor and eye movements of the crew.

Clinicians have long relied on an analogous form of decision support such as lists or algorithms for differential diagnosis. When a patient presentation is unusual (non-normal but not emergent), differential diagnosis lists (whether in old-fashioned textbooks or new-fashioned handhelds) support clinical performance by serving as a cognitive aid. The practice of reviewing a complete differential helps overcome anchoring and confirmation biases and can be a forcing function to ensure that every critical, and treatable aetiology is ‘ruled out’. Unlike non-normal checklists that are built into cockpit workflow, differential lists are often not well-integrated into clinical workflow and this may undermine their use.¹²

‘Non-boldface’ checklists form part of the normal framework of ‘job aids’, which might also include mnemonics and other rote learning tools, task visibility, context-sensitive help functions, decision support and instruction manuals. Mnemonics (such as ‘ABC’ for ‘Airway, Breathing, Circulation’ in resuscitation), for example, are sometimes used to retrieve procedural items where participants are likely to be subject to high cognitive load; however, mnemonics are more critical in situations where there is no later access to a physical checklist for confirmation.

Recent checklist implementation in healthcare

WHO surgical safety checklist

Perhaps the most widely used checklist is the WHO Surgical Safety Checklist.⁶ There are three phases in the checklist, organised in a logical sequence and requiring participation by the surgeon, the anaesthetist and the nursing team. On examination, a number of issues are apparent when considering the WHO checklist.

First, the structure varies from the design of aviation checklists, in that it combines procedures with formal team discussion; these processes are not mixed in the cockpit but remain distinct because they serve different purposes. The WHO checklist consists of a checklist (Sign In), a briefing (Time Out) and a checklist with a short briefing at the end (Sign Out). Checklists are suited to verification of procedures for linear processes; whereas briefings are suited to support execution of complex processes that may require appropriate adaptation and variation. Briefings are important because surgical outcomes are complex and emergent, and optimal performance of surgical procedures may require flexibility to accommodate the unexpected, however briefings should be instituted separately from the checklist. If briefings are too closely coupled to checklist completion, teams may miss the cognitive shift required to move from linear or procedural work to complex or adaptive work.

Second, the roles of the team members in completing the checklist are not clear. Who will read the checklist? Who will verify that the actions have been completed? Each clinician's role in the checklist should be formalised for the surgical setting, so that when tempo is high, steps are not missed. Third, compliance requires that boxes be ticked. This means that at least one team member will be occupied with completing the checklist and thereby not be available for other tasks. Boxes are more suited to a shopping list format, where items must be completed but order is unimportant, rather than an aviation-style checklist. Problems arising from combining a memory support tool with an audit device are discussed below.

Lastly, the checklist involves a Time Out: this requires that everything stops and no one interrupts. In an emergency, or under extreme time pressure, it is difficult to get everyone on the team to stop what they are doing and attend completely. The loss of team discussion under time pressure has been described by some centres implementing the Safe Surgery checklist.^{6 13} These are the times when mistakes are most likely to occur, yet paradoxically also when the Time Out portion of the checklist (the briefing to support complex work) is least likely to be performed as intended.

When it comes to checklist implementation, it is important to recognise that aviation checklists are integral to the normal workflow. The aircraft does not stop while the checklist is completed, and the timing of checklist completion is arranged so that it does not conflict with other essential flight activities. To that end, the checklist does not impose an additional burden or workload, but is actually perceived by aircrew as something that makes the flight easier. In contrast, the Time Out is performed before the case can begin, so essentially stands independently of the workflow. To that end, the Time Out is likely to be seen as something additional, and, unless it results in obvious time-saving downstream, will be perceived as an increase in workload. This mixture of purpose between checklist and briefing, in combination with implementation issues, may explain the range of outcomes as well as the range of enthusiastic to skeptical opinions about the mandated use of checklists in surgery.^14–16

Concerns have recently been fuelled by the disappointing results after implementation of the checklist in Michigan¹⁷ and large-scale mandated implementation in Ontario Canada.⁵ Interpretations of results are also complicated by reported differences between perceived and actual application of the checklist. In a recent US study, hospital documentation indicated 100% compliance with checklists, but observers found that on average only 4 of 13 checklist items were actually completed.¹⁸ Even strong advocates for checklists admit that full implementation of the WHO checklist is difficult and that improvements require more than the checklist, including strong institutional leadership, data collection, and monitoring, and training in teamwork.^{4 14}

Checklist for reduction of catheter-related bloodstream infections: standardising evidence-based procedures

After impressive reductions of catheter-related blood stream infections (CLABSIs) were achieved with the implementation of a checklist bundle, checklists were promoted as evidence medicine should look to this safety solution.¹⁹ However, successful reduction of CLABSIs was not due to the checklist alone: multiple interventions addressing ICU safety were implemented at the same time, and it remains unclear what role the checklist specifically played in infection reduction.² For example, the CLABSI checklist relies on nurse oversight. The changes in nursing behaviour can improve physician performance of line insertion in ways that are unrelated to the checklist: through the ‘Hawthorne’ effect, because the physician knows they are being watched; through empowering nurses and levelling the power gradient between physician and nurse and improving the safety culture; or, through formation of best practice as a habit as physicians insert lines the same way each time.

Checklist completion as a metric for determining compliance

Checklist compliance is increasingly monitored in healthcare.⁵ Often, institutions conduct internal audits of checklist compliance in anticipation of regulatory inspections. Using ‘compliance with checklist’ audits as a measure of safety or quality, however, is problematic, as high checklist compliance is no guarantee that the task is well-executed,¹⁸ or that patient safety culture is high.²⁰ In addition, some of the benefits that have been found to be associated with checklist usage, such as enhanced team building and nurses speaking up, are likely to be negated if compliance audits lead to sanctions.

Reinforcement and sanctions surrounding tasks may distract performance from the intent of the checklist. In healthcare, there is often a need to adapt the procedure to the patient or the context. Recent findings show that the WHO checklist, for example, is often implemented differently within single organisations, depending on context. Clinicians may be discouraged from acting in a manner that is best for the patient if they perceive that they may be censured for not following the procedure ‘to the letter’.

Summary

In healthcare, we need to get back to the basics with checklists and reserve the tool for processes that are simple, easy to follow, standardised and (perhaps) time critical. Expanding the term to cover briefings and other tools more suited to complex and variable processes is confusing, and may require communication and advanced team skills to implement and sustain. It is appealing to embrace a single tool to improve safety, and checklists have been found to be effective in some settings.¹⁶ However, the complexity of quality and safety improvement in healthcare guarantees that solutions will never be singular, straightforward or simple to sustain.

Introduction of a new tool without full consideration of its purpose, benefits and limitations may actually increase risk to patients, providers and the system as a whole. Overimplementation of checklists may erode respect for long-standing healthcare cognitive aids that are effective, have been iteratively improved, and are well suited to specific purposes. Overreliance on checklists as a safety net can lead to omission of other safety practices that may better support safety through reliability and resilience. Checklists are excellent ‘aides memoire’ and directives to correct procedures, but they are not a panacea.