How to WHO: lessons from aviation in checklists and debriefs

Abstract

Introduction

The World Health Organization (WHO) surgical safety checklist (SSC) has had an overall positive impact; however, it has not completely prevented adverse events and compliance with the checklist varies. The aviation industry is considered to have better engagement with their safety checklists, reporting not only safety improvements, but also a cultural shift in their checklist philosophy over recent years.

Methods

We explored the personal attitudes of pilots working in the aviation industry to identify principles of an effective checklist philosophy that could be transposed to the healthcare setting to empower more effective, consistent and ultimately successful implementation of the WHO SSC. A questionnaire was developed by the authors. Three airline pilots were interviewed via telephone, and asked questions regarding the logistics of and attitudes to checklists in the aviation industry.

Results

Several key factors for successful checklist implementation were identified. These include regular training and education on human factors and the checklist’s purpose, and institution of an atmosphere that is receptive, engaged and welcoming. Much can be learned from the aviation industry, where not only has the incidence of adverse events decreased, but the attitudes of people working in the industry have also transformed.

Conclusion

The WHO SSC is an invaluable tool used in healthcare settings worldwide. However, it is not a standalone commodity. To be effective, it necessitates steadfast engagement from the team members involved in its implementation. Human and checklist must work in partnership, using each other’s strengths and fallibilities, to optimise outcomes and prevent risks to patient safety.

Introduction

The World Health Organization (WHO) surgical safety checklist (SSC) was introduced in 2007 ‘to decrease errors and adverse events, and increase teamwork and communication in surgery’, and since its introduction the SSC has successfully improved morbidity and mortality worldwide.¹ However, checklist compliance and assimilation into surgical practice varies widely between individuals, departments, hospitals and countries.²

Despite its overall positive impact, the SSC has not completely prevented adverse events in the operating room. Never events are a nationally set list of preventable patient safety incidents that should not occur when national guidance and safety infrastructure are implemented by the healthcare team; they can have devastating consequences for patients as well as psychological effects for the team responsible.³ NHS Improvement is a programme established in 2012 to ‘understand why surgical Never Events persist’ despite mandated use of WHO SSC in the NHS, because their incidence exposes ‘potential weaknesses’ in implementation of the SSC. Between 1 April 2016 and 31 March 2017, 445 never events were reported. Of these, 356 were surgical: 189 cases of wrong site surgery, 114 cases of retained foreign objects post procedure and 53 cases of the wrong implant/prosthesis being used. Identifying the reasons leading to these errors is paramount if high-quality care is to be delivered reliably.⁴

Variable administration of the SSC can ‘erode benefits’.⁵ Studies have identified factors hindering the SSC in practice, including:

• -allocation of roles to team members is unclear, who should call out each item;
• -who should verify each item is completed;
• -perceived hierarchical gradients among multidisciplinary team members;
• -the entire team may not stop all other activity at critical points of the checklist;
• -team members’ lack of understanding of the purpose of the SSC.⁶

As a consequence, staff may not fully engage with, or commit complete focus to, the checklist while it is being conducted.² This can lead to errors that the checklist was designed to prevent, and reduce task completion.⁷ Furthermore, coordination that would usually be brought about by simultaneously engaging all team members, giving time for introductions and instillation of a team spirit with a shared goal, is interrupted.⁸ Ultimately, the checklist may be not fulfilling its purpose, or gaining its desired benefits.

The aviation industry is considered to have better engagement with their safety checklists, and pilots engage with them because their own lives and safety, as well as those of their passengers, depend on it. In the 1970s and 1980s, airlines identified that ‘human factors were an important feature’ in several serious aircraft accidents during that period. Airlines swiftly implemented measures to prevent recurrence of the same errors by instilling an attitude of ‘learning from mistakes, accidents, and near misses’, in contrast to one of blame and retribution.⁹ Over recent years, the airline industry has noted not only safety improvements, but also a cultural shift in their checklist philosophy.¹⁰ Human factors are now an established component of a commercial pilot’s curriculum, training and assessment. Pilots are expected to have a robust appreciation of human factors, and are also assessed regularly in simulator checks on safe practice, teamworking, effective communication and workload management, among others.

We explored how aviation has initiated this cultural shift and maintained checklist compliance by exploring the personal attitudes of pilots working within the industry. Identifying principles of an effective checklist that could be transposed to the healthcare setting could be invaluable in empowering more effective, consistent and ultimately successful implementation of the WHO SSC.

Methods

A questionnaire was developed by the authors. This included questions regarding methods and procedures for conducting checklists in the aviation industry, and explored pilots’ attitudes towards its design and their knowledge of human factors. Questions were grouped into sections, namely ‘Logistics of the Checklist’, ‘Evolution of the Checklist’, ‘Human Factors’ and ‘Attitudes to the Checklist’. Three airline pilots were interviewed via telephone by one of the authors (IG), and asked the questions listed in Tables 1–4. All three pilots were known to the authors, and were approached because of their senior positions in the industry. All pilots responded to the request for an interview.

Table 1 .

Questions and pilots’ answers regarding ‘logistics of the checklist’

Question	Answers
How many checklists do you carry out?	• There is one main checklist . . . The main tool we use is interactive briefings . . . the checklists are more for wrapping things up. We always do briefings as a first step for every departure • There are also interactive briefings and debriefings. The pilots carry out a flight brief in the office which covers all the general aspects of the flight and issues including route, weather, turbulence. We then have a whole crew brief during which the issues which may affect them are covered • One, with different sections containing a certain number of checks for each stage of the flight
In what format is the checklist?	• The main checklist used is one side of A4, on a laminated sheet • On one page • It’s on one laminated card
How many pilots have to engage in the checklist?	• Two pilots • Two – the two pilots • Two pilots
Who leads the checklist?	• Of the two pilots involved, one is ‘flying’ and one is ‘monitoring’. A read and response method is used, where the ‘flying’ pilot will call for the checklist; the ‘monitoring’ pilot will call out each checklist item and the ‘flying’ pilot will confirm • One person calls for the checklist items, and the other person does all the actions • You read them out aloud and the other pilot acknowledges them or gives the answer
At what point in the flight is the checklist carried out?	• Before and after start, before and after take-off, before approach, landing. They also encourage us to do a post-flight review where you go through saying ‘oh did that work how we thought it would, could it have gone better’ • Before, during and after the flight • Before take-off, then once you’re airborne, and before you land
How long does the checklist usually take?	• Each section of the A4 sheet takes less than 30 seconds, it is all pretty quick • 30 seconds or so • Regarding the briefings: If we’re just flying in and out of our main base, you can just whizz through, but if you’re going somewhere that you haven’t been before, or somewhere that’s more challenging, it can be quite a bit longer

Table 2 .

Questions and pilots’ answers regarding ‘evolution of the checklist’

Question	Answers
When was the checklist introduced?	• In the mid-1980s, things started to get condensed, much more user-friendly and simplified with checklists • It has been in aviation for a long time – since the early days of flying, so is integral to the operation • About 40 years ago
Do you remember a time before these checklists were implemented?	• When I first started flying, everything was done from memory. We kept having mistakes; people get distracted and forget that they’ve done something or missed an item • When undergoing flying training 15 years ago on light aircraft, I was using a more extensive checklist • No – in the older days they would have been more extensive
What have been the changes in your industry since the checklist was introduced?	• The forefront of the flying operation is crew resource management •   A major change is crew resource management (CRM). As the captain, I need to make sure all my team is on board, and that they feel able to chip in [during briefings] if I’ve forgotten something. In the past, nobody would challenge the captain – the captain was god • The biggest change is the volume of flying, which has skyrocketed. If the volume of flying doubles, you’ve got to halve the incident rate just to have the same number of incidents
Has the checklist changed over time? If yes, how have these changes been implemented?	• The checklist used to be prescriptive, but the industry has tried to go away from that because it’s not very engaging. It also used to be that one person told the other what to do whereas now, we are very keen on having interactive briefings. It now works much better because it makes you pay more attention to it; in the old way, you could just sit there and switch off a bit • You used to do it on your own •  If we wanted to suggest a change to the checklist, a pilot could feedback suggestions to the company manager who would review the evidence. You just email the training department and make a suggestion, and they’ll always get back to you. There is a very, very open communication system
Do you have to undertake training in how to use the checklist?	• Not per se. It is so ingrained into the operation that checklists are a natural part of flying • In an airline operation, every 6 months you go in a simulator for two very intense training sessions. You are assessed both individually and how you work as part of a team. If your level of assessment falls below a predetermined threshold, you fail the test and are offered retraining. You can do everything right but fail your check if your interpersonal skills aren’t very good • In aviation, there’s a lot of stress management training. They talk about a bucket, saying everyone’s brain is like a bucket that you can put things in, but when it gets to the top, anything you try to put in falls out, and the bucket can’t take any more. Once you’re getting very stressed, the first thing that goes out of the window is hearing. In a situation where things are going horribly wrong, rather than telling someone to do something, instead you should ask them what their assessment of the situation is. Then you can tell how full that person’s bucket is

Table 3 .

Questions and pilots’ answers regarding ‘human factors’

Question	Answers
How do you promote and maintain engagement from those involved in the checklist?	• Everyone in the team must feel empowered to say something, so as the leader you have to be approachable and set the tone • For some elements of the checklist, they get you to read out what you’re actually checking. This has been a change since an incident in order to make the check more rigorous and minimise errors • Everyone knows it’s got to happen, so you don’t really need to grab people for that
What do you understand about human factors?	• The human element of the whole scenario, generally the weak link in the chain as far as flying is concerned. I have done more training over the years on a topic called crew resource management: how you interact with the other pilot, cabin crew and air traffic control. We also have training on the chimp effect, and how to manage this. Once a year there is a ground training day, which involves talking about human factors • It’s the psychology; understanding and recognising the performance of not only yourself, but the other individuals around you. It’s incredibly important in a safety-critical operation that you understand the human factors • It is a very interesting concept in aviation because in the old days, you’d rock up to work as a co-pilot and you’d be castigated for asking stupid questions. Now, in an airline environment, the psychology has changed . . . Human factors are the key to a very harmonious operation, and a way of life
How do you think the influence of human factors on the checklist is minimised?	• Training on human factors: There’s a course you have to attend where they go through the psychology and managing stress, for example if you ask a question in a non-accusatory, non-confrontational way, it works a lot better • Decision-making models, such as FNC: ‘Fly Navigate Communicate’ . . . make sure it is safe, then consider if you need to speak to someone. And TDODAR: ‘Time Diagnosis Options Decision Allocate Review’ which you are encouraged to use for every scenario • On paper it’s easy to forget where you are. With the electronic one, when the item is done it turns green

Table 4 .

Questions and pilots’ answers regarding ‘checklist attitudes’

Question	Answers
What is your attitude to the checklist?	• It is vital to the way we operate planes and fly safely. We wouldn’t be able to do the job without them and if you didn’t follow them, you’d be leaving yourself open to negligence. I fly with a different person every day, so everyone must be signing from the same hymn sheet; the checklist allows seamless working, otherwise it just doesn’t work. They are pretty useful because . . . you don’t want your colleague suddenly surprising you doing something you wouldn’t expect . . . so they give you situational awareness • I think they’re superb. Where we’ve come from over the last 40 years is a massive difference • It forms an absolutely essential part of the operation and cannot be done any other way
What is good about the checklist?	• They do work, they do catch errors • It’s all about identifying things before they happen • The job is much easier when the things you have to get done are written down. The checklist has had a remarkable effect on reducing accident rates
Do you think there are aspects which could be improved?	• Sometimes indentations and bullet points can be difficult to follow if you’re under pressure, so it’s not the content but rather how it’s laid out which could be improved • Some checklists have been translated from other languages, so can be difficult to understand clearly. • No doubt. This is evolution not revolution. We’re constantly striving to improve the way we do things
What advice would you give to somebody creating a checklist?	• Think about how the checklist will be run – who will say which bit, etc you need to test it out and practise it. You need feedback on its usability. Management have to make themselves approachable for the professionals doing the job, so they can be contacted if there any thoughts or concerns about the checklist • Set it down in the most concise, easy to follow way • Think about what you want to achieve from the checklist
What makes a good checklist?	• Easy to follow, clear and concise, not too long • It’s got to be well-written so there’s no ambiguity • By keeping everything absolutely simple and unambiguous
Are there ways you can suggest that would minimise the influence of human factors?	• Making yourself approachable. Wash up afterwards – we always debrief, after every flight: what went well, what could we do better, why did you do that; so it’s incremental learning and a loop of feedback whereby you can benefit from other people’s flying as well • Take it on your chin if you’ve done something not quite right • Decision making tools. Minimise people having to wing it; when you’re in a high stress situation you want rule-based thinking

Results

Three airline pilots were asked 20 questions, covering logistics (Table 1), evolution (Table 2), related human factors (Table 3) and attitudes (Table 4).

Two pilots carry out ‘one main checklist’, which is displayed on one side of a laminated piece of A4 paper, using a read-and-response method. The pilots reported the ‘main tools’ they use are interactive briefings and debriefings, whereby team members discuss those items that could not be addressed in appropriate detail if they were part of the checklist.

Checklists were introduced in aviation in the late 20th century. Since their introduction, checklists have evolved and been developed to increase engagement, because with earlier editions ‘you could just sit there and switch off’. A key development in practice is biannual mandatory training, in which pilots are assessed on both individual competencies and teamwork. There was a keen acknowledgement among all the pilots interviewed that ‘human factors are the key to a very harmonious operation’. Pilots reported novel training on ‘how you interact’ and how ‘recognising the performance of not only yourself, but the other individuals around you’ can make processes operate ‘a lot better’.

Pilots were asked about their attitudes to checklists. Some considered the checklist ‘vital’ to safety, having a ‘remarkable effect on reducing accident rates’ and allowing ‘seamless working’ among the team. The importance of an easy-to-use and convenient design, that accounts for human factors so that there is ‘no ambiguity’, was recognised as a key factor contributing to checklist compliance.

Key themes emerging from the interviews were: defined roles for team members; the role of briefings and debriefings; and mandatory human factors training.

Discussion

In the operating theatre, interdisciplinary staff and technology must work in a coordinated and efficient manner. Checklists aim to negate the dependence of such an intricate interplay on the hazards posed by human factors, including ‘cognitive biases, poor interpersonal skills, and substandard environmental factors’.¹¹ However, checklists themselves are liable to human factors and are effective only if implemented expertly.

Logistics of the checklist

The WHO SSC has three phases, Sign In, Time Out and Sign Out, each performed at discrete critical points during an operation to ensure key items are completed. Whereas some items only need a binary response (yes or no), others require more rigorous discussion and extensively worded answers that take more time and attention to address properly. The latter, more appropriately described as briefings, are less well-suited to the efficient linear verification system of a checklist, so can be rushed or overlooked as they may be felt to slow the pace of the SSC in comparison with the binary responses needing just a tick-box mark of completion. Combining briefings with simpler verification items can impede the momentum of the checklist, and disengage team members as a consequence.

Mayer et al demonstrated variability in checklist implementation and its effect on clinical outcomes. In only 62.1% of cases were all three components (Sign in, Time Out, Sign Out) of the WHO SSC completed. Not completing the checklist significantly increased the risk of postoperative complications.¹² In another study, operating team members were absent, and failed to pause, for checks in 40% and 70% of cases, respectively.⁷ Further research is essential to identify the factors behind these failures to complete and comply with the checklist, as ‘unlike a drug, a checklist will only ever be as effective as the personnel implementing it’.¹² Indeed, when all team members pause and are present, more information is shared, more items are checked and the WHO SSC takes less time.⁷

A key objective of checklists is to ‘facilitate optimum crew coordination’. The roles of the team members involved in the WHO SSC are not clear; notably, who should read the checklist, and who should confirm that the actions have been completed. This could inevitably lead to a confusion of roles, with multiple people speaking simultaneously, or disengagement of members who feel the question is not relevant to them. Roles should be formalised prior to starting the checklist. Indeed, Russ et al found greater checklist compliance when the SSC was led by the senior surgeon compared with another team member.⁷

In aviation, these roles are formally assigned, with one pilot calling out each checklist item and the other verifying its completion. This minimises communication errors and role confusion, enabling an adept and dynamic read-and-response method. In addition, having an allocated role can improve compliance because team members ‘feel acknowledged and important’.¹³ To increase engagement, each member of the operating theatre team could be asked to tick the item once it has been covered.

Briefings require formal team discussion, and are suited to the management of more complex, variable processes; as noted by one pilot, they ‘should be instituted separately from the checklist’.⁶ In aviation, briefings are not incorporated into the checklist, but instead undertaken independently. Once briefings have been completed, the checklist is used for ‘wrapping things up’.

Education and training

To maximise checklist engagement, those involved must appreciate why it is used and the adverse events it aims to prevent. A lack of understanding as to why an item is on the checklist can lead to ‘a decline in the perception of the task’s importance’, subsequent poor execution and rote task completion.¹³ This poses a ‘threat to patients’ safety and to high-quality care’ which the checklist has been designed and deployed to prevent.¹⁴ In aviation, there is regular mandatory training on resource management and human factors to educate airline staff on their responsibility to the checklist, and how optimising their engagement can make the checklist more effective. The checklist is also deemed ‘an absolutely essential part of the operation’. Levy et al noted ‘staff are more likely to use the checklist if they understand how it improves patient safety’.¹⁵ Team education of the checklist’s ‘existence and the reasons for and the importance of its implementation’ is vital.¹⁶ Raising awareness of real-life adverse events and near-misses that have occurred in clinical practice can help team members realise the importance of the checklist, and their responsibility in its implementation, such that it is not regarded a ‘nuisance task’.¹⁷ Team members should be educated on the origin of the SSC, and the grounds upon which it was introduced to surgical practice, as well as discussing barriers to implementation and advocating strategies to optimise team performance. This proposed training should be multidisciplinary to help remove the usual silos of healthcare teams.

Culture and attitudes

The WHO SSC aims to ‘increase teamwork and communication’.¹ Since the introduction of checklists, the aviation industry has ‘experienced a cultural shift that changed the way teams work together’.¹⁸ It is important to endorse an atmosphere that is transparent and receptive, in which all team members are both approachable and respected and regardless of position, feel confident to declare their concerns and queries. In their report ‘Opening the Door to Change’, the Care Quality Commission (CQC) acknowledged that hierarchical gradients, which can be associated with ‘other damaging behaviours in the workplace’ and negatively impact teamworking, are ‘inimical to safety’.¹⁹ Asked how engagement with the checklist is maintained, one pilot interviewed stated ‘everyone in the team must feel empowered to say something’. Team members should be educated on the importance, and benefits, of creating an atmosphere where this is the case.

Human factors is a discipline that encourages appreciation of the organisational, individual and environmental characteristics that affect performance. Whether it be personal stress, noisy distractions or physical obstacles, all can impact situational awareness, perception and cognition, and henceforth contribute to errors by precipitating suboptimal teamwork and communication. In one study, surgeons attributed 43% of adverse incidents to communication breakdowns among operating staff; communication failures were the strongest predictor of surgical errors in 31 observed cardiac surgery procedures.^20,21 Catchpole et al confirmed better leadership and management skills of operating theatre staff were associated with a reduced number of adverse events.²²

As stated by the pilots interviewed, ‘understanding and recognising the performance of not only yourself, but the other individuals around you’ is ‘incredibly important in a safety-critical operation’ and ‘key to a very harmonious operation’. Furthermore, in a study by Weller et al, senior clinicians reflected that if they asserted a positive attitude to the checklist, ‘particularly in terms of introductions and welcoming staff . . . their team functioned more effectively’.⁵ Finally, the pilots highlighted the importance of the ‘wash-up’ or debrief after the flight. There may be resistance to formal debriefs after operations or theatre sessions. This may be partly due to lack of training in how to do an effective debrief. They can, therefore, be considered tick-box exercises, as opposed to true learning opportunities.

Human factors training should be integrated into the process of checklist introduction to a workplace. Such training improves teamwork, increases checklist compliance and reduces errors, as well as transforming cultural attitudes that help sustain systemic changes.⁷ Neily et al demonstrated a significant reduction in surgical mortality and morbidity following implementation of such a training program for operating staff. This program, based upon aviation’s crew resource management theory, comprised teaching on effective communication strategies, team briefings and the use of checklists.²³ Education on human factors can thus be considered paramount to improving checklist implementation. The same importance should be placed on interpersonal skills as in the airline industry, with members not passing the training if their interpersonal skills are not good enough.

Conclusions

The WHO SSC is an invaluable tool used in healthcare settings worldwide. However, it is not a standalone commodity. To be effective, it necessitates steadfast engagement from the team members involved in its implementation. Understanding of human factors is maturing, with a growing wealth of strategies that acknowledge the inherent fallibilities human nature and encourage the design of systems that are less susceptible to human error. Through better appreciation of these factors, industries can improve equipment, processes and procedures to make them more compatible with the needs, compatibilities and limitations of people.

This exploration of attitudes and logistics within the aviation industry has identified key factors for successful checklist implementation: regular training and education on human factors and the checklist’s purpose, and institution of an atmosphere that is receptive, engaged and welcoming. The checklist needs to be considered an asset, rather than a burden or nuisance, by all team members; its design should account for human factors for it to function effectively as a cognitive net. As summarised by Clay-Williams and Colligan, ‘success requires complex, cultural and organisational change efforts, not just the checklist itself’.⁶