Evaluation of the Patient Safety Leadership Walkabout programme of a hospital in Singapore

Abstract

INTRODUCTION

The Patient Safety Leadership Walkabout (PSLWA) programme is a commonly employed tool in the West, in which senior leaders visit sites within the hospital that are involved in patient care to talk to healthcare staff about patient safety issues. As there is a lack of perspective regarding PSLWA in Asia, we carried out an evaluation of its effectiveness in improving the patient safety culture in Tan Tock Seng Hospital, Singapore.

METHODS

A mixed methods analysis approach was used to review and evaluate all documents, protocols, meeting minutes, post-walkabout surveys, action plans and verbal feedback pertaining to the walkabouts conducted from January 2005 to October 2012.

RESULTS

A total of 321 patient safety issues were identified during the study period. Of these, 308 (96.0%) issues were resolved as of November 2012. Among the various categories of issues raised, issues related to work environment were the most common (45.2%). Of all the issues raised during the walkabouts, 72.9% were not identified through other conventional methods of error detection. With respect to the hospital's patient safety culture, 94.8% of the participants reported an increased awareness in patient safety and 90.2% expressed comfort in openly and honestly discussing patient safety issues.

CONCLUSION

PSLWA serves as a good tool to uncover latent errors before actual harm reaches the patient. If properly implemented, it is an effective method for engaging leadership, identifying patient safety issues, and supporting a culture of patient safety in the hospital setting.

INTRODUCTION

There are real risks associated with healthcare. The World Health Organization estimated that 10% of patients admitted into a hospital will be harmed, and that half of these incidences is preventable.(1) The release of the Institute of Medicine's report, titled To Err is Human: Building a Safer Health System, in 1999 had set off alarm bells, pointing out that adverse events and medical errors in healthcare settings have long been overlooked, and that such errors continue to threaten patient safety.(2) The report highlighted the pressing need for all healthcare staff to better understand the frequency and nature of safety issues faced by patients. Since its release, there has been great interest in employing various tools to promote and improve patient safety in hospitals. One such tool is the Patient Safety Leadership Walkabout (PSLWA), which was first conceptualised by the Institute for Healthcare Improvement in 2000 to connect senior leadership to patient safety.(3,4) PSLWA is a simple yet rigorous management tool that involves leaders visting a department or ward in the hospital to talk to healthcare staff about any patient safety issue that have caused, or may potentially cause, harm. Concerns raised during these conversations are addressed by implementing preventive measures.

PSLWA appears to be an effective tool for engaging leadership, identifying safety issues, and supporting a culture of patient safety. A recent systemic review by Morello et al(5) showed that the evidence for the impact of PSLWA in the effort to improve the patient safety climate in hospitals was stronger than that for other strategies, although there was no single tool that had a definitive impact. A study by Thomas et al(6) found that the positive impact PSLWA had on patient safety was limited to the nurses who had participated in the programme. Frankel et al's historically controlled study also supported the finding, reporting an increase in mean safety climate scores following the introduction of walkabouts in hospitals.(7) To date, there have been several reports regarding the implementation of PSLWA in several hospitals in the West.(3,8-10) However, a perspective from Asia is lacking. We therefore reviewed the PSLWA programme in Tan Tock Seng Hospital (TTSH), Singapore, and evaluated the effectiveness of the programme in improving the hospital's patient safety culture.

METHODS

The PSLWA programme was first launched in TTSH in November 2004. The programme is led by a core group of senior members from the hospital's management committee (i.e. the Chief Executive Officer, the Chairman of the Medical Board and the Chief Operating Officer), and secretariat support is provided by the Clinical Standards Improvement Department, Office of Clinical Governance. The rest of the core group consists of the Director of Nursing, the Heads of the various Clinical Departments and the Patient Safety Officers.

This core group conducts regular monthly visits to pre-identified patient care areas in the hospital. The site of visitation is typically announced a few weeks prior to the actual walk-about, and the Clinical Standards Improvement Department prepares the incident reporting trend of the site. A briefing on the facts obtained from the site is conducted for the core group immediately before the walkabout. Participation of healthcare staff at the site is voluntary. During the walkabout, discussions are held on-site in an open area or in a room in the patient care area. All patient safety issues raised and their concomitant responses are recorded. At the end of each visit, participants are asked to complete an anonymised questionnaire for the purpose of obtaining feedback and also for assessing the participant's perception on self-and organisational commitment to patient safety culture. Finally, recommendations are followed through by informing the relevant process owners and implementing action plans.

We reviewed the walkabouts that were conducted from January 2005 to October 2012 using mixed methods analysis. All documents, protocols, meeting minutes, post-walkabout surveys, action plans and verbal feedback pertaining to the walkabouts were reviewed and evaluated. For qualitative analysis, we adopted the framework set by the Health Research and Education Trust of the American Hospital Association.(11) The framework consists of seven domains for evaluation – preparation, scheduling, conducting, tracking, reporting, feed-back and measurement. For quantitative analysis, we adopted Charles Vincent's taxonomy of contributory causes(12) to classify the various categories of patient safety issues that were raised. The seven Vincent categories of contributory causes are institutional, organisation and management, work environment, team, individual staff member, task, and patient. In addition, the hospital's patient safety culture was assessed using an anonymised post-walkabout survey that was conducted on every participant. The survey was a four-item questionnaire modified from the safety climate scale of the Safety Attitudes Questionnaire.(7,13-15) Data entry was performed using the Statistical Package for the Social Sciences version 15.0 (SPSS Inc, Chicago, IL, USA). The tables and graphs in this study were constructed using the same software. No statistical test was applied. This study was approved by the National Healthcare Group, Domain Specific Review Board, Singapore (Reference code: 2013/00123).

RESULTS

A total of 70 walkabouts were conducted in TTSH from January 2005 to October 2012, giving an average of nine walkabouts annually. These walkabouts were conducted at different sites, with no repetition in the site of walkabout.

In all, 324 participants from various healthcare discipline groups and 491 hospital management personnel were involved in the walkabouts. This meant that for a single walkabout, an average of two participants and three hospital management personnel were involved. Fig. 1 illustrates the proportion of participants, categorised according to their healthcare discipline. Nurses and doctors constituted 57.7% and 15.7% of the total participants, respectively. Of the 187 nurses, 123 (65.8%) were of a senior position (i.e. senior staff nurse and above), while 64 (34.2%) were of a junior position (i.e. staff nurse and assistant staff nurse). Among the 51 doctors, 40 (78.4%) were of a senior position (i.e. registrar and above) and 11 (21.6%) were of a junior position (i.e. house officer and medical officer).

Fig. 1.

Pie chart of the PSLWA participants from January 2005 to October 2012, categorised according to healthcare discipline. The percentages do not add up to 100% as the figures were rounded off to the nearest decimal place.

The walkabouts were conducted in all units involved in patient care, including wards, outpatient clinics, intensive care units, surgical high-dependency units, emergency diagnostic and treatment centres, operating theatres, day surgery units, post-anaesthesia care units, endoscopy rooms, noninvasive cardiac labs, pharmacies, allied health units (i.e. nutrition and dietetics, physiotherapy, occupational therapy and speech therapy units), laboratories, and radiology departments.

During the study period, 321 patient safety issues were identified. Of these, 308 (96.0%) issues have been resolved as of November 2012. The remaining 13 (4.0%) issues were still pending as some issues require action from more than one department and some require more time in order to reach a meaningful resolution. Table I shows the Vincent classification(12) of the issues identified. Most of the patient safety issues raised fell under the work environment category (145 [45.2%] issues); this finding is similar to that of other studies.(3,7-9) We found that 234 of the 321 (72.9%) issues raised during the walk-abouts were not identified through other conventional methods of error detection such as incident reporting, morbidity and mortality rounds. Actions taken for certain issues identified during the walkabouts are highlighted in Table II.

Table I.

Vincent classification(12) of the issues raised during the Patient Safety Leadership Walkabout (PSLWA) programme from January 2005 to October 2012.

Table II.

Brief highlight of actions taken for certain issues identified during the PSLWA programme.

An anonymised post-walkabout survey, which was modified from the safety climate scale of the Safety Attitudes Questionnaire,(15) has been sent out to each PSLWA participant since 2006. As of November 2012, a total of 258 survey responses out of an expected return of 276 were received, achieving a response rate of 93.5%. Table III shows the percentage of positive responses of the participants to each question in the post-walkabout survey. Participants consistently expressed that the senior management viewed patient safety as a top priority in the hospital and was willing to take their feedback seriously (256 [99.1%] participants). In addition, 245 (94.8%) participants reported an increased awareness in patient safety, and 233 (90.2%) expressed comfort in openly and honestly discussing patient safety issues. Table IV highlights some of the feedback on the PSLWA programme, obtained from the post-walkabout surveys.

Table III.

Positive responses of participants to each question of the post-walkabout survey from 2006 to 2012.

Table IV.

Brief highlight of feedback obtained on the Patient Safety Leadership Walkabout (PSLWA) programme from the post-walkabout surveys.

The Health Research and Education Trust of the American Hospital Association has laid down a framework to serve as a guide for effective implementation of PSLWA in hospitals.(11) Table V shows the evaluation of the PSLWA programme in TTSH, in accordance with the criteria under each of the seven domains in the framework. In short, TTSH was found to fulfil the requirements for effective implementation of PSLWA.

Table V.

Evaluation of the implementation of the Patient Safety Leadership Walkabout (PSLWA) programme in Tan Tock Seng Hospital, Singapore, in accordance with the domains of the framework set by the Health Research and Education Trust of the American Hospital Association.

DISCUSSION

Evaluation of the PSLWA programme in TTSH has shown that among the various categories of patient safety issues raised, most (45.2%) were under the work environment category. Furthermore, 72.9% of all issues raised in the walkabouts were not identfied through other conventional methods of error detection. Thus, PSLWA appears to be a good tool for uncovering latent errors before actual harm reaches the patients. The concept of latent errors was first conceived by James Reason; it refers to defects (e.g. physical defects, and process-or manpower-related defects) in the design of systems that people work in, which would eventually contribute to patient harm.(16) These ‘accidents waiting to happen’ are often embedded in work environment issues, including poor design of workplaces or equipment, faulty maintenance of equipment, heavy work-load and inadequate staffing or equipment. As these defects lie dormant in the healthcare delivery system and usually have a long lag time before resulting in actual errors or adverse events, they are often difficult to measure.(17)

In our study, a majority of the patient safety issues raised via the PSLWA programme were not identified through other mechanisms, such as incident reporting, and morbidity and mortality rounds. This implies that PSLWA provides an important platform, other than the conventional methods of error detection, for healthcare staff on the ground to raise patient safety issues. Since the fundamental mechanism by which patient safety can be achieved is often multifaceted, no single method is able to detect all safety issues, adverse events or near misses.(18) Although there is some overlap between the various data sources or methods, most issues are found by only one method.(19)

One of the strengths of the PSLWA programme conducted in TTSH is the broad-based participation from all departments. All sites directly or indirectly involved in patient care were visited by the core team, and many process owners were involved in developing and implementing solutions to address the patient safety issues raised. All action items are stratified into different priority status to be resolved within a pre-agreed time period (red: within a month; orange: within three months; yellow: within six months; and green: beyond six months). This monitoring system is unique to our hospital, and has served to address critical patient safety issues in a prompt and effective manner. For instance, the allergy testing area in one clinic did not have a resuscitation trolley within close proximity. This issue was classified as a ‘red’ action item in view of the potential catastrophic consequences should a patient go into anaphylactic shock in the allergy testing area. In less than a week, a resuscitation trolley was made available in the afore-mentioned area. Thus, critical patient safety issues that are highly modifiable are addressed within the shortest possible time with the help of the system.

Another major strength of the PSLWA programme in TTSH is the hospital's strong commitment to patient safety. The top three senior executives (i.e. the Chief Executive Officer, the Chairman of the Medical Board and the Chief Operating Officer) are present at the walkabouts to understand the patient safety issues that occur on the ground. Due to the challenge of aligning the schedules of all three senior executives, the hospital's walkabout occurs monthly. This differs from most hospitals in the West, where walkabouts are conducted on a weekly basis.(3,8-10) We opine that monthly walk-abouts are optimal for TTSH as this gives the healthcare staff on the ground adequate time to reflect on the possible safety issues in their respective units, so that these issues can be raised during the walkabout. This will allow a more fruitful discussion between the participants and the core team. In addition, our hospital has a very high patient workload, and having weekly walkabouts may result in too many disruptions.

One of the limitations of the PSLWA programme in TTSH is the low level of participation on the ground as reflected by the participant to hospital management personnel ratio of 2:3. The walkabouts mainly involved nurses, and relatively fewer doctors. In addition, there was a lack of participation from junior staff; 78.4% and 65.8% of the participating doctors and nurses, respectively, were of senior ranks. It has been reported that the types of problems elicited during walkabouts are dependent on the types of participants. Nurses preferentially discuss operational problems, while doctors tend to focus on clinical decision-making issues.(7) Both aspects contribute to potential patient safety issues, and therefore, attempts should be made to gather more participants, especially doctors, in the walkabouts. A greater number of junior staff should also be encouraged to participate since these staff, compared to their senior counterparts, spend more time in direct contact with patients and have a higher likelihood of coming into contact with patient safety issues.

Evaluating the impact of walkabouts on the hospital's patient safety culture requires a validated tool, such as the safety climate scale of the Safety Attitudes Questionnaire.(7,13,14) Most hospitals use the Safety Attitudes Questionnaire at baseline (i.e. pre-walkabout) and at post-walkabout to monitor the progression of patient safety awareness on their participants.(2,8,9) Our current programme lacks a baseline (pre-walkabout) assessment. Hence, although our post-walkabout surveys showed a high level of patient safety awareness among participants (Table III), we do not have a baseline for comparison. A pre-walkabout survey would be useful to evaluate the actual effect the PSLWA programme has on raising patient safety awareness among frontline healthcare staff. Using the baseline assessment tool as a guide, we could be better stewards of healthcare resources by channelling more attention and assistance to sites with a low level of patient safety awareness and practices; the frequency of walkabouts to those sites could be increased as well. Another advantage of conducting a pre-walkabout assessment would be the ability to employ it as a tool to gather patient safety concerns from all staff, particularly junior staff who might not be available to participate in the actual walkabout.

We noted that some patient safety issues raised during the walkabouts were also faced by other units. As staff who had piloted new initiatives to address some of the safety concerns raised tend to share their experience with the core team during the walkabout, the way forward would be to create a platform for these common issues and successful initiatives to be shared across the hospital so that all staff, including nonparticipants, can be kept informed.

PSLWA has been directly linked to improvements in cultural perceptions and attitudes about teamwork, willingness to speak up, and the overall safety of the work environment.(19) As our hospital conducts patient safety climate surveys at regular intervals, future studies comparing the scores of PSLWA participants and non-participants to better assess the impact of the PSLWA programme on patient safety awareness would be ideal. Key questions from the patient safety climate survey could also be incorporated into the pre-and post-walkabout surveys to allow a more valid comparison. The scores of the different healthcare discipline groups could also be determined to evaluate the effects of the walkabouts on each group.

To conclude, PSLWA is not a remedy for all patient safety issues. Success is highly dependent on the strong commitment of senior leaders and the active participation of staff on the ground. Our study showed that the PSLWA programme in TTSH brought up several categories of patient safety issues, particularly those related to the work environment, which were not raised through other conventional methods of error detection. The PSLWA programme thus serves as a good tool to uncover latent errors before actual harm reaches the patient. If properly implemented, it is an effective tool for engaging leadership, identifying patient safety issues, and supporting a culture of patient safety in the hospital setting.