Improving patient safety for older people in acute admissions: implementation of the Frailsafe checklist in 12 hospitals across the UK

Abstract

Background

checklists are increasingly proposed as a means to enhance safety and quality of care. However, their use has been met with variable levels of success. The Frailsafe project focused on introducing a checklist with the aim to increase completion of key clinical assessments and to facilitate communication for the care of older patients in acute admissions.

Objectives

to examine the use of the Frailsafe checklist, including potential to contribute to improved safety, quality and reliability of care.

Methods

110 qualitative interviews and group discussions with healthcare professionals and other specialties, 172 h of ethnographic observation in 12 UK hospitals and reporting of high-level process data (completion of checklist and relevant frailty assessments). Qualitative analysis followed a thematic and theory-driven approach.

Results

through use of the checklist, hospital teams identified limitations in their existing assessments (e.g. absence of delirium protocols) and practices (e.g. unnecessary catheter use). This contributed to hospitals reporting just 24.0% of sampled patients as having received all clinical assessments across key domains for this population for the duration of the project (1,687/7,021 checklists as fully completed). Staff perceptions and experiences of using the checklist varied significantly, primarily driven by the extent to which the aims of this quality improvement project aligned with local service priorities and pre-existing team communications styles.

Conclusions

the Frailsafe checklist highlighted limitations with frailty assessment in acute care and motivated teams to review routine practices. Further work is needed to understand whether and how checklists can be embedded in complex, multidisciplinary care.

Introduction

Patient safety is often described as a hard or ‘wicked’ problem, resulting from the interaction of multiple influences in complex and unpredictable healthcare environments [1–3]. Countless attempts have been made to improve safety and reduce harm using different interventions and quality improvement programmes [4, 5]. Among these, checklists have achieved considerable traction, particularly since the success of the World Health Organisation (WHO) surgical safety programme [6, 7]. Derived from the aviation industry, checklists are deemed to contribute to improved reliability of care by supporting systematic completion of well-defined procedural tasks. However, their use has not always been met with success and transferability across settings has been questioned [7, 8]. This variability points to the need for in-depth study, and in particular ‘real time observations’, to assess how checklists can be used to influence reliability of care and harm reduction [9].

This article presents findings from a mixed methods study on Frailsafe, a patient safety intervention aiming to reduce harm and increase assessment reliability for older people admitted acutely to hospital. The care of older people with multiple co-morbidities is often described as involving an additional layer of complexity in the identification and management of geriatric syndromes, and requiring increased attention to medical errors and their impacts on patients [10–13]. This is driven by a recognition that older patients suffer from disproportionate levels of harm in their care due to insufficient attention to frailty in non-specialist settings [14]. Frailsafe aimed to address this problem by bridging the ‘know-do’ gap in the inconsistent implementation of accepted care standards for frailty [14, 15].

The Frailsafe project commenced in August 2014 and recruited 12 acute hospital sites from across the UK [14]. The project concentrated primarily on the introduction and use of a patient safety checklist. This consisted of two parts: a screening phase to identify patient characteristics related to a clinical definition of ‘frailty’ and, for those patients with a positive screen, an additional phase for consideration of key assessment domains: dementia, delirium, mobility, falls, pressure ulcers, resuscitation status, medication review and reconciliation, and use of equipment (catheter, cannula and bed rails) [14].

Hospital teams were encouraged to use the checklist as part of a ‘check-and-challenge’ process with two or more healthcare professionals working together. It was anticipated that conversations triggered by the checklist would contribute to the flattening of hierarchies and improve communication at critical hand-offs. To ensure timely, evidence-based care practices were followed, it was initially suggested that the checklist would need to be used within the first 24 h of admission [14]. In many settings, this meant that the primary users of the checklist would be acute care clinicians, rather than geriatric medicine teams (in the UK care of older patients often falls under acute care teams, although acute frailty units and geriatrician-led teams operating within acute admissions are increasingly adopted). In doing this, Frailsafe intended to extend awareness of harm avoidance for frail populations across specialties and to advance multidisciplinary thinking and conversation about frailty assessment in acute care.

The implementation of the checklist was supported by a ‘Breakthrough Collaborative’ [14]. This involved three learning events held over the course of 12 months, bringing together members of the improvement teams, alongside a team of geriatricians and improvement experts. The events were linked by ‘action period’ calls where teams discussed successes and ongoing challenges. As learning opportunities, these events received positive responses from participants, but their analysis remains beyond the scope of this article, which focuses more specifically on use of the checklist across hospital settings.

This article draws on qualitative and quantitative evaluation data to understand how health professionals used the checklist across the 12 participating NHS Trusts; how they perceived the role of the Frailsafe checklist in improving patient safety for older people admitted to hospital acutely; and how checklist implementation influenced quality of care and healthcare outcomes.

Methods

This article reports on the findings of an independent evaluation studying the implementation and use of the intervention. The study followed a mixed-methods design, with the summative stage preceded by iterative formative cycles, where preliminary findings were fed back to the Frailsafe project leads and local hospital improvement teams. Ethics approval was granted by the Office for Research Ethics Committees Northern Ireland (reference number 14/NI/1028).

Qualitative methods

We used qualitative methods to capture the narrative of change and understand Frailsafe use in hospital sites. A total of 139 people from different professional groups participated in 110 individual and repeat interviews (97 individual interactions [eight by phone and seven by e-mail], 10 joint interviews [2–3 people each] and three group discussions [4–10 participants]). Of these, 86 interviews were recorded with participant consent and transcribed fully, while concurrent notes were taken in the rest of the interviews. Participants comprised senior doctors from geriatric and acute care specialties (n = 31), doctors-in-training (n = 18), nurses and matrons (n = 31), ward managers and care co-ordinators (n = 7), physiotherapists and occupational therapists (n = 19), pharmacists (n = 15), managers (n = 5), carer representatives (n = 1), quality improvement coaches (n = 10), social workers (n = 1) and Clinical Commissioning Group representatives (n = 1). We attempted to interview all health professionals who had experience using the checklist or had been involved in discussions where implementation of the checklist was a key topic. This meant we were focusing on a purposive sample while trying to reach maximum variation, using snowballing techniques.

‘Real-time observations’ [9] across the 12 hospitals selected for the Frailsafe Collaborative (see Table 1 for details) totalled more than 100 h, with experienced qualitative researchers shadowing different health professionals and following ward rounds, board rounds, triage meetings, handover discussions and multidisciplinary team (MDT) meetings. Observation focused on how local teams were using the checklist but also on how the checklist was situated in the context of the wider clinical work in each setting. One visit per site was completed before and one after checklist implementation. All admissions documents, assessment charts, treatment plans and other documentation used within the first 24–48 h period from admission in the Acute Medical Unit (AMU), or equivalent structure, were collected for review.

Table 1.

Distribution of hospital sites according to their characteristics

Location	Type	Beds
Rural	District general hospital	400
Rural/Town	District general hospital	779
City	Teaching hospital	865
Rural/Town	District general hospital	600
Rural/Town	District general hospital	767
City	District general hospital	420
City	Teaching hospital	600
Rural/Town	District general hospital	1,000
Town	District general hospital	550
Rural/Town	District general hospital	785
City	District general hospital	1,200
City	Teaching hospital	200

Detailed ethnographic notes were kept during ‘real time observations’ and from over 50 informal discussions. Field notes were taken during the Breakthrough Collaborative learning events (56 h) and the eight ‘action period’ teleconference calls (16 h). The content of discussions on the project website and Twitter comments were also reviewed.

Interview transcripts and field notes were collected and analysed iteratively until theoretical saturation was achieved (i.e. no new themes emerged from the data) [16, 17]. NVivo 10 was used for qualitative analysis, which followed a thematic approach [17]. Themes emerged inductively from the data but also deductively by drawing on previous literature and theory. To increase trustworthiness findings were fed back in formative cycles and were confirmed with local improvement teams at the second and third Frailsafe learning events. We also sought negative cases—i.e. which disconfirmed our initial interpretations—to be able to build a richer picture of Frailsafe use. ‘Member-checking’ and analysis through disconfirming data are commonly used to increase the trustworthiness and quality of qualitative research [16].

In this article, we are further drawing on the distinction between ‘work-as-imagined’ and ‘work-as-done’ to understand the discrepancies between how work was formalised in the checklist and how it was performed in practice [18, 19]. This idea stems from literature on resilience and complexity in healthcare, which highlights the inherent tensions between work activities being fully planned and specified out in advance, and the reality of people having to respond and adapt to dynamic, unpredictable circumstances to get the work done safely in practice [18, 19].

Quantitative methods

Process data were collected on all elements of the checklist to investigate reliability of assessments: data were captured on whether delirium, dementia, mobility, falls and pressure ulcer assessments were reported as having been completed; whether resuscitation status was reported as having been considered; whether cannulas, catheters and bed rails were reported as present and whether they were still needed; and whether medicines reconciliation and review were reported as having been completed.

A data governance and collection protocol was provided to each site (available on request) with local clinical teams responsible for collecting and uploading the data to the Web Improvement Support for Healthcare (WISH), a bespoke online Quality Improvement software [20]. Clinical teams were asked to provide a minimum sample of 20 forms per week or a full census of the patient group within the unit. Sampling practices varied by site and over time.

Process measures were aggregated weekly to measure ‘compliance’ with the individual components of care mentioned above, where each item had to be recorded as completed to be ‘compliant’. Overall ‘compliance’ measured the extent to which all components of care were reported as complete to the standard expected. The denominators were the number of forms in the sample in a week, except in the case of equipment, where the denominator was instead whether equipment was recorded as present.

Quantitative data were used to strengthen the qualitative part of this study, by helping to guide the interview questions, focusing the ethnographic observation and informing the analysis. To strengthen the quantitative methods, we drew on the insights gained on local processes through the qualitative work, which allowed us to interpret process measures in-context.

Results

Volume and patterns of checklist use

Collectively the 12 Frailsafe sites uploaded 7,021 checklists, with the volume of reported use per site ranging between 249 and 1,208 checklists for the analytical period (29/09/2014–12/09/2015). Five sites reported relatively consistent use across all weeks, five had a delayed start before larger volumes were submitted, and two sites finished recording before the project ended.

Although a higher volume of sampled forms may be a preliminary indication of increased engagement with the Frailsafe project, this should be interpreted alongside qualitative data to gain an in-depth understanding. In the process of checklist implementation, tensions emerged between using the checklist as a quick audit tool (for recording assessments as completed or not) and using it as a tool for triggering interaction between staff members. Few MDTs were observed to be filling in the checklist as an active ‘check-and-challenge’ process, in the collaborative, interactive manner intended at the outset of the project. In most sites, the checklist was either completed in the background of the ward round by individual members of staff, or was filled in independently before or after the ward round by consulting patient notes (quotes A.1–3 in the Appendix).

Across sites, the degree of Frailsafe-triggered interaction and conversation during the ward round was largely driven by individual senior doctors. This was determined by their ownership of and engagement with Frailsafe, as well as pre-existing teamwork and communication styles. Where conversation was taking place, staff reported that this rarely involved verbal challenge from junior to senior members of staff, as initially intended, or meaningful exchange of opinion about optimal care (quotes B.1–2). Instead of verbal ‘check-and-challenge’, the researchers also observed some instances of written ‘check-and-challenge’, where outstanding actions were added to patient notes for other teams to follow, instead of directly communicating with responsible staff (e.g. quote B.3). In a few cases, the role of Frailsafe was reversed as senior doctors described it as an oversight mechanism (e.g. for registrars to challenge nurses or for senior nurses to challenge junior members of staff—quote B.4).

Perceived usefulness

Participants acknowledged the value of the Frailsafe checklist in identifying frail patients in acute admissions through the screening criteria (phase 1 of the checklist) and in helping them to focus on key frailty assessments requiring completion (quotes F.1). By being involved in the project, some sites discovered that they were not filling in their assessments as systematically as they previously assumed (e.g. quote F.2). The checklist was also considered valuable in prompting appropriate use of medical equipment (quotes G.1).

Participants explained how using a tool linked to a high-profile, national frailty project allowed them to build momentum in their local context and promote wider frailty work (quotes E.1–3). For example, sites were able to add dedicated physiotherapists and pharmacists to their teams to support frailty care in acute admissions. They appreciated having dedicated time to review deficiencies in related processes and benchmark their practices with other sites (quotes E.4).

Participants in the improvement teams, although often enthusiastic about how they used the tool in their own teams, also voiced concerns and reluctance for wider roll-out (quotes I.1). Some mentioned that the tool itself did not prompt any action or trigger completion of assessments, as decisions for follow-up action depended fully on the individual filling in the checklist (quotes H.1). Others thought Frailsafe overlapped with existing practices and was just ‘another piece of paper’ that would eventually be imposed on nursing staff, rather than a ‘check-and-challenge’ process (quotes H.2). Participants also commented on the potential for the Frailsafe tool to create ‘false certainties’ and perpetuate bad practice, when only used as a tick-box exercise rather than a reflective way to review frailty assessments.

Table 2 provides an overview of perceived benefits of the Frailsafe checklist, against areas where further improvements could be made.

Table 2.

Perceived influence on practice and areas of improvement discussed in qualitative interviews with hospital team members

Perceived influence on practice	Areas for improvement
Highlighted problems with assessments and processes	Little use of ‘check and challenge’ process
Changes primarily to delirium and mobility assessments, changes to equipment use	Overlap with existing practices—‘another piece of paper’
Space to reflect and focus attention on frailty	Difficulties spreading outside the initial improvement team
Worked as a prompt to identify things being missed	The tool itself did not prompt any action or close loops—depends on individual action
Easy and quick to use—some ambiguities allowed localisation	Some items still ambiguous and open to interpretation

Multidisciplinary working

Hospital teams often reported improved multidisciplinary working, but this was mostly observed in sites where hierarchies were already less prominent prior to the intervention. By providing a platform for bringing together staff members with an interest in frailty care, the Frailsafe project allowed hospital teams to re-think what constitutes good care for older patients in acute care. People found their participation in the local improvement team as one of the most rewarding aspects of the project (quotes C.1–2). However, in sites with strong, pre-existing hierarchical relationships, there was little indication that Frailsafe changed team communication styles and multidisciplinary working, but rather seemed to accentuate differences both within the MDTs and with other specialties (e.g. quote C.3). A significant obstacle to project implementation in acute admissions was the divide between acute care and geriatric medicine specialties. Local improvement teams (the vast majority with a geriatric medicine background) found it difficult to overcome specialty silos and introduce the project effectively to their acute care colleagues, in a way that would create buy-in and common ownership (quotes D.1). The use of the tool remained primarily in the domain of geriatricians, with one of the sites deciding to focus on their specialty care of older people ward, rather than acute admissions (quote D.2).

Reliability of frailty assessments

To understand whether use of the checklist led to increased ‘reliability’ with which frailty assessments were carried out and use of equipment was improved, quantitative data were collected on each of the different assessment categories.

Each check-and-challenge item referred to a defined task: completion of assessments (for delirium, dementia, mobility, falls, pressure ulcers, medicines reconciliation and medicines review), which could be answered as ‘completed’, ‘still to complete’ or ‘not relevant’; consideration of resuscitation status, which could be answered as ‘considered’ or ‘still to consider’; and consideration of equipment (cannula, catheter, bed rails) in use (‘yes’ or ‘no’) prompting follow-up questions of whether the equipment is still required (‘yes’, ‘no’, ‘not relevant’).

Combined overall ‘compliance’ (i.e. all assessments reviewed by the checklist would be deemed as completed, any single missed means not overall compliant) for the whole period was 24.0% (1,687/7,021 checklists reported for all 12 sites). By site the median value was 6.8% with a range of 0.3% (1/387)–60.8% (734/1,208) as minimum and maximum overall ‘compliance’ values for each site separately. There were vast differences between sites in terms of reporting delirium and dementia assessments as completed (range 18.9% [73/387]−95.7% [672/702] for the former and 3.1% [12/3,847]−96.0% [674/702] for the latter). Similar discrepancies were observed in relation to resuscitation status assessments and medications reconciliation (range 34.1% [139/408]−97.9% [687/702] for the former and 13.8% [51/370]−80.9% [330/408] for the latter). Good compliance was reported across the majority of sites for pressure ulcer risk assessments (range 66.1% [585/885]−98.5%[1190/1208]) and medications review (range 58.4% [316/541]−98.3% [690/702]).

Ethnographic observation showed that good use of the checklist often aligned with other contextual changes in the organisations. For example, front-door geriatricians started working in acute care at the same time as Frailsafe began in site I. In hospital E, a dedicated ‘frailty team’ led by a geriatrician also started looking after older patients admitted in acute care when Frailsafe was initiated. These organisational changes to the way frailty care was delivered ‘at the front door’ further reinforced the impact and relevance of the checklist.

Discussion

The analysis highlights the opportunities raised by the introduction of the Frailsafe checklist and the challenges encountered by hospital teams in implementing the intervention. The checklist was the primary focus of quality improvement and the core vehicle for the changes intended: its use would facilitate interactions between junior and senior members of staff, across specialties, and would encourage them to complete assessments to prevent harm from occurring. However, closing the ‘know-do’ gap proved challenging in the 12 participating hospital sites [21]. Focusing on the checklist as a ‘technical’ tool meant little attention was paid to addressing established and socio-culturally complex ways of working and communicating in different settings.

Reliability of good care

To a large extent, assumptions behind intended use of the checklist came at odds with the reality of admission practices for older people: delirium assessments were not part of admissions booklets; ward rounds were done by individual members of staff; resources did not allow for medicines reconciliation and mobility assessments to be performed in the first 24 h. This was reflected in the significant differences on the minimum and maximum values for assessment completion by site, as presented in Results.

Formalising tasks and work processes in the form of a checklist placed increased emphasis on ‘work-as-imagined’ (how things would be done in an ideal scenario), which some hospital teams found difficult to reconcile with ‘work-as-done’ in the messiness of everyday practice [18, 19]. The reality of achieving patient safety for older people included ad hoc communication through various routes, workarounds to establish good care in the absence of or in combination with formal assessments, ongoing effort to build relationships in transient healthcare teams, practising empathy and paying close attention to patient needs and exercising initiative to ensure care was safe and appropriate. This practical accomplishment of safety was not always reflected through the Frailsafe checklist, which made some care practices visible but obscured others.

Often the checklist was used as an audit tool to highlight variation in ‘work-as-done’ [18, 19], rather than as mediator and facilitator of safe care. For example, the checklist placed more emphasis on ensuring assessments were reported as complete, rather than comparing the quality of existing assessments with accepted care standards. In several instances, local teams took initiative to review their frailty assessments and care practices, but this process could have been more substantially embedded as part of the Collaborative to actively encourage good practice.

Professional and temporal diffuseness

It is recognised that checklists work best for clearly pre-defined tasks ‘for which the particulars of context are immaterial, reliance on human memory is a known problem and variations in those procedures are undesirable’ [22]. When care processes are loosely coupled and performed by different members of staff at different points in time, it becomes challenging to use a checklist to coordinate actions. Lack of geographical and temporal proximity means there is no immediate ‘stimulus-response’ from conducting the check, and this reduces the utility of the checklist in ensuring a necessary task is completed. Although hospital teams involved in Frailsafe found the checklist useful in identifying frailty and highlighting gaps in assessments, they recognised that assessment completion does not always translate to better patient care and non-completion does not necessarily lead to action.

Teamwork and communication

Assessment processes were not always in place and opportunities for complex clinical teams (i.e. where team members from different disciplines and professional groups had different working routines and varied responsibilities including in other parts of the hospital) to come together and coordinate on the basis of the Frailsafe checklist were often lacking. This meant the checklist was commonly used individually by members of staff without triggering conversation between health practitioners, highlighting the primacy of ‘paradigmatic’ against ‘narrative’ thinking [22]. Unless assessment results and pending actions are communicated appropriately, filling in the checklist has little influence on patient safety outcomes.

Catchpole and Russ [9] have discussed how good teamwork communication is a pre-requisite to successful implementation of checklists, rather than a result of their use. This manifested strongly in Frailsafe as teams with already well-developed collaboration styles (i.e. good and established working relationships) seemed to coalesce more strongly around the aims of the project, whereas individuals used to isolated working had more difficulties opening up communication channels. This was particularly relevant to acute care teams, which largely remained peripheral where the Frailsafe effort was led by geriatricians. Similar barriers are identified in previous studies discussing the role of hierarchies and professional boundaries in successful checklist implementation [8, 23, 24]. Studies arguing that checklists have the potential to improve communication and teamwork tend to focus on better defined processes (i.e. where steps are specific and clear) and more confined spaces, such as the operating room [25].

Intervention and implementation re-design

The intervention would have benefited from drawing more substantially on theory-driven design that recognises socio-cultural influences as critical for the implementation of quality improvement interventions. Possible re-design could take in account facilitators to implementation that have been extensively described elsewhere (e.g. local adaptation of the tool to respond to specific needs, alignment with pre-existing processes, attention to accountability structures, supporting teamwork and relationships, recognising leadership and providing constructive feedback on performance) [8]. There should be caution against transferring principles from other industries (e.g. aviation) without first tailoring these to match specific ways of working in healthcare [26]. Rather than focusing on checklists, it may be more useful to understand the bigger picture to which checklists can contribute, as only one component in the improvement of patient care [27, 28]. Involving patients and carers in co-design could have enhanced the value of the tool and played a role in reinforcing good care processes [29]. Attention to the practical, everyday accomplishment of work in different settings would help support context-sensitive solutions that reinforce patient safety for older people in acute care.

Limitations

Ethnographic work was limited in that it spanned 12 sites without the capacity to engage in long-term observations within each of these hospitals individually. The number of interviews partly compensates for the focused ethnographic data and theoretical saturation was reached as appropriate for qualitative research. Process measures were self-reported, which may have had an impact on the reliability of the data collected (e.g. over-estimation), and sampling practices and data capture varied by site. These limitations make quantitative comparisons between sites problematic.

Conclusion

This study highlights the importance of an in-depth understanding of the wider context in which a complex intervention is embedded and engagement with the critical role of implementation. A growing body of literature departs from a view of checklists as purely technical interventions, to highlight how they intervene in a complex socio-technical system of pre-existing patterns of working, communicating and sharing responsibility [30, 31]. Further work is needed for the Frailsafe Collaborative to extend beyond a checklist approach and develop interventions that allow health professionals to respond to their local circumstances in a sustainable manner that enhances multidisciplinary support.

Key points.

• This mixed methods study evaluates the implementation and use of the Frailsafe checklist for older people in acute admissions, including potential to contribute to improved safety, quality and reliability of care.

• Local hospital teams valued the opportunities afforded by the introduction of the checklist, in being able to focus their efforts on the acute care of older people and in identifying deficiencies in their existing assessment processes.

• However, introduction of the checklist was not sufficient to broker new ways of working across geriatric medicine and acute care, or to change communication patterns in hierarchical multidisciplinary teams.

• Findings are consistent with previous research that questions the role of checklists as isolated change agents in complex care processes. Further research should depart from a view of checklists as ‘technical’ tools to pay more attention to the socio-technical work required to alter established and socio-culturally complex ways of working and communicating in different settings.

Conflict of interest

None.

Funding

This work was supported by a Health Foundation ‘Closing the Gap in Patient Safety’ award and by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. J.R. was financially supported by an Improvement Science Fellowship from the Health Foundation. The views expressed in this publication are those of the authors and not necessarily those of the Health Foundation, the NHS, the NIHR or the Department of Health. The financial sponsors played no role in the design, execution, analysis and interpretation of data or writing of the study.