The endoscopy safety checklist: A longitudinal study of factors affecting compliance in a tertiary referral centre within the United Kingdom

Abstract

Gastrointestinal endoscopy is a widely used diagnostic and therapeutic procedure both within the United Kingdom and worldwide. With an increasingly older population the potential for complications is increased. The Wolfson Unit for Endoscopy at St. Mark's Hospital in London is a tertiary referral centre, which conducts over 14,000 endoscopic procedures annually. However, despite this high throughput, our baseline observations were that the procedure for safety checks was highly variable.

Over a seven-day period we conducted a questionnaire-based survey to all staff members involved with endoscopy within our unit. We found that there was little consensus between team members, both in terms of essential safety checks and designating responsibility for the checks. A panel of experts was convened in order to devise a safety checklist and a strategy for increasing compliance with the checklist among all staff members. Using a combination of electronic and physical reminders and incentives, we found that there was a significant increase in completed checklist (53% to 66%, p = 0.021) and decrease in the number of checklists left blank post intervention (10% to 2%, p=0.03). We believe that post implementation validation of safety checklists is an important method to ensure their proper use.

Problem

Gastrointestinal endoscopy is rapidly expanding and increasingly becoming a therapeutic procedure. Demand is high particularly with an ageing population, the rise of alcoholic liver disease and the development of national endoscopic screening programmes. The UK endoscopy joint advisory group (JAG) [1,2] has achieved high levels of quality assurance for endoscopy, through programmes such as the global rating scale (GRS) for endoscopy [3] and the national training certification process. Nevertheless, endoscopy is not without risk [4] and patient selection, sedation management and complications all need careful consideration for each individual patient.

The Wolfson Unit for Endoscopy at St Mark's hospital in London UK, is a tertiary referral and endoscopy-training centre conducting approximately 14,000 procedures annually. There are currently 30 endoscopists from medical, surgical, and nursing disciplines scoping within the unit. Given the expansion in endoscopy services required to accommodate future endoscopic screening programmes [5], measures to improve endoscopic procedural quality and safety are necessary. We therefore sought to evaluate safety checks and teamwork as anecdotal evidence suggested variability in safety practices.

Background

Errors in endoscopy are not uncommon. The National Confidential Enquiry into Patient Outcomes and Death (NCEPOD)[6] investigated 1818 deaths within 30 days of an endoscopic procedure in the UK. The majority of recommendations highlight failings in procedure planning, patient monitoring, and safe administration of sedation, as opposed to technical competencies of the endoscopist. Although this report was published a decade ago, there are no current data to suggest any improvement in practice. Furthermore, NHS England have conducted a recent review of surgical never events[7] citing 329 reported incidents in the period 2012-13 indicating further room for improvement.

The concept of avoidable error is corroborated by observations undertaken in the Wolfson unit. Prospective analyses of 90 procedures from 22 lists by 16 endoscopists identified 41 patient safety incidents (PSIs). 51% of PSIs were categorised as severe by expert consensus (between two Consultant endoscopists, one trainee endoscopist, and a patient safety expert). Examples include patient misidentification and wrong site procedure. Twenty-four percent had the potential to be full “never events” as defined by the Department of Health[8]. The total number of observed PSIs may appear high, but it is important to note that these included “minor” errors too, and not all errors had direct consequences for patients. Nevertheless, they represent weaknesses in the system and an opportunity to prevent future error.

Strategies to reduce error by introducing systematic safety checks at key points in a procedure are well established in other areas of medicine. The WHO surgical safety checklist has been shown to reduce morbidity and mortality in a global patient population.[9] Similarly Pronovost et al [10] have shown that central venous catheter associated sepsis can be abolished by adhering to best practice through use of a customised checklist. In view of this evidence, it is plausible that safety checklists have a place in interventional procedures such as GI endoscopy [11], bronchoscopy [12], and interventional radiology.[13]

Baseline measurement

Observations of endoscopies in our institution revealed that safety checks were highly variable. What safety checks were conducted, by which team member, and for which patients was user dependent and thus instituted variability. While a number of endoscopy units employ checklists, there is no clear evidence for their content, mode of employment, or benefit in endoscopy. To qualify this observed variability in safety checks, endoscopy team members’ opinions were sought.

For a seven-day period a “safety attitudes” questionnaire was distributed at the end of each endoscopy list to all team-members in the procedure room. This focused on obtaining opinions on safety checks in endoscopy. Data from 29 respondents (of the 30 approached, response rate 97%) were analysed. Respondents included endoscopy nurses (16), registrars (9), and consultants (4) across surgical and medical specialities.

Endoscopy team members were asked whether they were aware of any pre-procedural checks currently in place. Figure 1 illustrates the range of safety checks that were thought to occur prior to each procedure. A wide range of safety checks were thought to occur, with little consensus on what checks are essential for safety and should be performed in all procedures.

Respondents were also asked whose responsibility it was to complete these safety checks. The majority thought it was a joint responsibility between endoscopist and nurse (42%). Other responses were admitting nurse (23%), endoscopist (19%), bowel cancer screening specialist nurse (7%), administrative staff (5%), and referring clinician (4%).

The majority of respondents thought that an endoscopy safety checklist would be useful. For emergency and therapeutic procedures, 100% of respondents advocated its benefits, and for elective cases 93% agreed. This questionnaire suggests professional team performance may be optimised by standardising safety checks in order to enhance patient safety.

See supplementary file: ds3917.png - “Perceived safety checks”

Design

The aim of this quality improvement programme was to assess compliance with the endoscopy safety checklist, factors affecting this and measures to optimise compliance. An ongoing checklist training and feedback strategy was already in place as part of the implementation programme. In order to evaluate compliance with the checklist (assessed from the checklist documentation), we prospectively evaluated checklist completion for patients undergoing an endoscopic procedure 12 months post implementation. Targeted improvement strategies were devised and compliance re-assessed after a four-month interval.

Strategy

Considering the benefits of checklists in other specialties [9,10], an endoscopy safety checklist (appendix 1) was implemented across the trust in 2012 for all GI endoscopic procedures.[11] Having implemented the checklist across our entire unit, we sought to evaluate compliance with the checklist to identify problems and optimise its effectiveness.

Many lessons were learnt through this process. Importantly, implementation of a checklist needs a detailed long-term plan with consideration of likely obstacles to ensure sustainability.

PDSA cycle 1: Checklist compliance was prospectively evaluated for consecutive patients undergoing an endoscopic procedure during a seven-day period. This was initially completed in February 2013. The medical record was assessed for the following parameters:

- Section of checklist completed (Time out / Sign out / Both)

- Procedure time (AM or PM)

- Grade and gender of the endoscopist

- Admission type: elective or inpatient.

PDSA cycle 2: Following this initial assessment a panel of experts who were involved in the original design and implementation of the checklist convened to discuss and agree on realistic changes to target areas of weakness and raise compliance. In summary, the interventions included:

- Unit wide team communication highlighting the effectiveness of the checklist (ie errors averted)

- Posters and hospital-wide NHS computer screensavers to remind users to complete the checklist

- Mandating the use of the checklist as part of hospital policy (new unit manager and hospital medical director)

- Email to all staff from the hospital chief executive highlighting the importance of checklist use for endoscopic procedures

- Senior nursing team briefed and invited to motivate existing nurses and ensure appropriate training of new nurses / agency staff

- Registrar (trainee) focused training formalised in endoscopy specific unit induction

- Prospective observations of endoscopy teams and real time feedback on checklist use.

PDSA cycle 3: Prospective evaluation of endoscopy checklist compliance was repeated in the same fashion as PDSA 1 four months following the interventions. The findings from this evaluation will be presented at the hospital governance meeting and fed back to endoscopy teams in the unit through weekly departmental communications.

Results

The baseline compliance data was prospectively collected for 199 patients. After a four-month

interval, 151 patient checklists were evaluated following the improvement interventions.

Overall checklist compliance:

Post-intervention, there was a significant increase in the percentage of checklists fully completed: from 53% to 66% p = 0.021 (figure 2). There was a statistically significant decrease in the number of checklists left blank post intervention - from 10% to 2% p=0.03 (figure 2).

Factors affecting checklist compliance:

In the initial evaluation, factors associated with greater checklist compliance included morning procedure, consultant, or nurse-led procedures, as well as those conducted in the bowel cancer-screening programme. Possible explanations for this include operator fatigue during the afternoon, registrars lacking checklist training, and dedicated specialist screening practitioners responsible for documentation in the screening programme. Post intervention, there was no difference in checklist compliance comparing endoscopic procedures in the morning and the afternoon (p=0.704, two-tailed chi squared), suggesting that the checklist was being incorporated into routine practice by teams. Comparing the pre and post intervention data, consultants had the lowest checklist compliance (45%), with significantly fewer fully completed checklists in the consultant group compared to nurses (p=0.021) and registrars (p=0.0002). There was no difference in compliance between registrars and nurses (p=0.416). Operator grade was the only factor post intervention shown to influence checklist completion (table 1 and figure 3).

See supplementary file: ds4638.doc - “Figures 1-3, Table 1 & Appendix”

Lessons and limitations

This on-going quality improvement programme shows that even though staff favour the use of an endoscopy safety checklist, compliance and accurate documentation remain sub-optimal in practice. This illustrates the difficulties in changing established practice, particularly in a large volume tertiary endoscopy unit. Despite the detailed implementation strategy there are pitfalls to introducing sustainable change such as checklist use across large teams. This is mirrored in the surgical safety checklist.[14] This can be explained by differences between specialities and individuals in risk / safety management, variation in expertise and experience, pressure of perceived workload and dealing with hierarchy in medical teams.

The registrars initially had low rates of compliance but this was addressed with targeted training. However, over time the consultants, as compared with other endoscopy staff, had significantly lower rates of fully completed checklists suggesting the potential for further training targeted at more senior team members. Additionally, this may represent an issue with the checklist concept requiring further “buy in” from senior clinicians – a situation not dissimilar to occurrences in operating theatres and senior surgeons’ views of the WHO checklist.[14]

There are limitations to this evaluation: Our data represent a snapshot at two time points and this may reflect random fluctuations, for example in staffing levels, or other organisational pressures. Additionally, it would have been desirable to incorporate patient opinion to inform the improvements made in PDSA 2. Although there has been improvement in compliance, additional PDSA cycles would further enhance compliance.

Furthermore, documentation completion does not necessarily equate to effective use of the checklist: The checklist could still be a “tick box” exercise for some users and measured compliance may not reflect its effective use by the team. This limitation is being addressed in on-going work including a prospective observational study in real time of checklist use by teams.

Conclusion

Prior to the endoscopy safety checklist being implemented, there was no minimum standard for

procedural safety checks in a unit performing large volume endoscopy. The implementation of a simple, standardised checklist involving the whole multidisciplinary team performing the procedure has rectified this variability in standards. This quality improvement strategy in a large centre illustrates the challenges of implementing a checklist despite a thorough educational and feedback plan. However, this process also demonstrates that continued re-assessment and targeted feedback coupled with senior leadership can lead to successful outcomes; the proportion of checklists fully completed significantly increased and the number of unused checklists diminished within a four month period. We conclude that use of the endoscopy safety checklist is an opportunity for cost-effective quality improvement in endoscopy services. On-going attention needs to be given to encourage routine uptake of the checklist for its true potential on safety and outcomes to be realised.