Does quality improvement improve quality?

ABSTRACT

Although quality improvement (QI) is frequently advocated as a way of addressing the problems with healthcare, evidence of its effectiveness has remained very mixed. The reasons for this are varied but the growing literature highlights particular challenges. Fidelity in the application of QI methods is often variable. QI work is often pursued through time-limited, small-scale projects, led by professionals who may lack the expertise, power or resources to instigate the changes required. There is insufficient attention to rigorous evaluation of improvement and to sharing the lessons of successes and failures. Too many QI interventions are seen as ‘magic bullets’ that will produce improvement in any situation, regardless of context. Too much improvement work is undertaken in isolation at a local level, failing to pool resources and develop collective solutions, and introducing new hazards in the process. This article considers these challenges and proposes four key ways in which QI might itself be improved.

Introduction

The quality and safety of healthcare worldwide remain problematic. Many of the basic operational systems and routines of work required to care for patients are not fit for purpose. Few have been purposefully designed or documented; instead, they are handed down through genealogies, sometimes mutating along the way so that processes intended to do the same thing may vary wildly across places, teams and shifts, and suboptimal functioning of processes to serve clinical work are the norm. As a result, the reliability of NHS clinical systems is poor, varying from 81% to 87%.¹ Processes for apparently simple tasks, such as ensuring the right equipment is available in operating theatres or that prescribed medication is administered on time, fail to function as intended with worrying frequency. When trained clinical teams use methods adapted from high-risk industries, they typically uncover multiple defects and hazards across their teams, units and organisations.² The associated risks are compounded when multiple systems and sectors interact, as is common in healthcare.³

These defects are highly consequential, impacting on efficiency, safety and the wellbeing of staff and patients.⁴ US studies suggest that nurses deal with an average of 8.4 work system failures per 8-hour shift, and they are continually interrupted.^5,6 The need for staff to learn and re-learn, associated with the variability in fundamental processes, is significant. Much professional time is consumed unproductively in learning anew how to undertake tasks as basic as ordering tests, knowing whether equipment has been cleaned, or how things are arranged in the resuscitation trolley in each setting. Personnel may also make errors as they move from place to place, either because they have not yet learned the new procedures or they apply previous learning to new but different contexts, sometimes with tragic outcomes.⁷

The problems with quality improvement

Healthcare has increasingly been encouraged to use quality improvement (QI) techniques to tackle these operational defects (clearly, healthcare faces many other challenges but they may require different approaches). Capacity to improve quality is clearly critical to healthcare organisations; every organisation needs to be able to detect its operational (and other) problems and solve them using structured methods. For many problems (although far from all), that may mean using methods adapted from other industries, such as Lean and Six Sigma, or approaches developed within healthcare, such as the Institute for Healthcare Improvement's Model for Improvement. This widely used model combines measurement – using statistical process control, for example – with small tests of change (plan-do-study-act (PDSA) cycles).⁸ But despite the widespread advocacy for QI, the evidence that it produces positive impacts in healthcare has been very mixed, with many of the better-designed studies producing disappointing results.^9–14 A 2016 review concluded that Lean interventions, for example, do not have a significant association with patient satisfaction or health outcomes, but do have a negative association with financial costs and worker satisfaction, and inconsistent effects on process outcomes.¹⁵

What explains these discouraging findings is now the focus of growing interest. One explanation appears to lie in poor fidelity in the use of QI methods. For example, a 2014 review found poor reporting and adherence to the basic tenets of PDSA cycles in QI reports.¹⁶ More generally, what may happen is that the superficial outer appearance of the intervention or QI method is reproduced, but not the internal mechanisms (or set of mechanisms) that produced the outcomes in the first instance.^17,18 These effects may arise because what is implemented in practice may be diluted, distorted or diminished versions of the intervention, as has been found, for example, in relation to leadership walkrounds.^19,20

Secondly, much QI work continues to be undertaken in the form of time-limited small-scale projects, perhaps conducted as part of professional accreditation requirements. Some of the achievements of this work are striking, but caution is needed. One risk is that QI becomes an activity largely assigned to professionals in training, who rarely have the skills, resources or power to affect the kinds of changes that may be required. For instance, a problem with crowding in oncology outpatients may have its origins in a complex tangle of poorly designed or functioning processes (eg ensuring blood results are available on time), but diagnosing the cause and redesigning the workflow accordingly might need a dedicated team with specialist training in ergonomics and the clout to support the changes needed; these are not resources usually available to junior doctors or small QI teams. They may therefore come up with a small fix or workaround that fails to solve the true problems and, in so doing, may introduce new risks.

Another risk is that of encouraging ‘projectness’²¹ – a sense that QI is a series of bounded, time-limited events rather than a continuous commitment, and overly focused on ‘innovation’ rather than replication. Treating QI as a series of local projects may increase the tendency for wheel reinvention – different ‘solutions’ to the same problem. Undoubtedly, this expansion of overlapping efforts in part reflects the relative novelty of QI in healthcare. But it requires urgent attention, not least because ill-coordinated improvement may, ironically, intensify the problem of locally-specific work processes, routines and tasks that only apply in their context of origin. Multiple ill-coordinated small-scale QI projects may, accordingly, degrade rather than improve the ability to achieve improvements across healthcare as a whole.²² Moreover, as attention shifts from one project to another, the gains achieved in the first project may attenuate, a phenomenon that has been termed the ‘improvement evaporation effect’.²³

A third, and linked, problem is the ongoing failure to cumulate and share learning from QI efforts. The NHS continually loses learning, and this is an urgent problem. Although proper evaluation is essential to advancing the science of improvement,²⁴ those who introduce local QI interventions are sometimes so convinced that the change introduced is positive that they may eschew evaluation.²⁵ When people do come up with good ideas and test them rigorously, the learning may be difficult to share and challenging for others to discover – in part because the learning is never reported or, if it is reported, it is not in an accessible form. When people come up with ideas that don't work, the learning is even more likely to remain obscured. These problems contribute significantly to wheel reinvention and to waste of time and energy. Yet traditional medical research funding mechanisms and publishing norms are poorly aligned with the imperative to evaluate, curate and make available experiences (positive and negative) and outcomes of both QI methods and QI interventions. Even when QI is reported, it tends to be poorly described.²⁶ It therefore remains difficult to even find out about a success or a failure elsewhere, let alone to know what was really done and with what outcomes.

A further challenge lies in the ongoing emphasis on specific interventions as the keys to QI, perhaps particularly when those interventions are valorised as magic bullets.²⁷ The dynamic interplay between intervention and context means that it is often difficult, and indeed not always helpful, to separate intervention from context²⁸ to the extent that transplanting a programme in its entirety from one setting to another is rarely straightforward.²⁹ Excessive attention to QI interventions in the narrow sense – eg huddles, bundles, checklists and other popular tools – risks overlooking the impact of context on intervention implementation and, perhaps more importantly, the critical role of context itself as generative of safety and quality. Very often, the kind of place that has come up with the idea for doing huddles and has been able to implement and sustain them is also the kind of place that has all of the other characteristics that facilitate quality and safety. The notion that the huddle – or anything else – is then a plug-and-play ‘solution’ is consequently misguided – the features of context (clarity of vision, infrastructure, organisational systems, values, skills and so on) that made it work in the first place need to be reproduced too. Healthcare organisations differ markedly from factory production lines, just as human bodies are not ‘widgets’. Acknowledging and attending to the social and cultural context is vital if improvement interventions are to work.

The tendency to attribute effects to interventions (rather than interventions and contexts working together) is further exacerbated by the problem that the forces that create positive conditions for quality and safety may be invisible to those who create them or may not be possible (or straightforward) to articulate. This makes it difficult for others to reproduce or recreate them. The intervention as described in published reports may offer only a partial account of the reasons why the success was achieved. Foregrounding a specific intervention, no matter how well characterised, as the explanation for the outcomes may risk rendering invisible the important mechanisms that contribute to the achievement of those outcomes. The result is a theoretically deficient approach to improvement that may rely on ‘magical thinking’.³⁰

Many of these challenges can be illustrated by looking at the example of sepsis management. For patients with suspected sepsis, organisations are encouraged to do a ‘bundle’ of six clinical activities within 1 hour:

1. deliver high-flow oxygen

2. take blood cultures

3. administer empiric intravenous antibiotics

4. measure serum lactate and send full blood count

5. start intravenous fluid resuscitation

6. commence accurate urine output measurement.

Delivering on each one of these goals requires a supporting infrastructure, ranging from role clarity through to sufficient well-maintained equipment. For example, obtaining a serum lactate with a rapid turnaround time requires optimised equipment and organisational systems, as well as staff with the right expertise available at the right time. Making all of these things happen requires high-level skills in operations design but may also require all kinds of other skills in implementation, including negotiating for clarity about roles and responsibilities, managing professional or managerial resistance to reconfigurations of tasks, delivering high-quality training and so on.

It is probably not necessary for each individual organisation to invest the effort in figuring out all of the tasks and activities needed to achieve each of the goals. Nor is it likely that all organisations will have all of the necessary expertise to come up with good solutions. However, if a good solution is found, it may help others because it can be shared and give them a head-start. Such a solution will need to go beyond the narrow specifics of a well-bounded, easily describable intervention and encompass the range of facilitating conditions – infrastructural, technological, social, and maybe even cultural – that have often been relegated to the category of ‘context’, but which are themselves vital to the success of efforts to improve. It is also important that the solutions reached are broadly similar across organisations, so that once a practitioner has learned the system once he or she will know broadly what to do next time. It may be disastrous, for example, if the system for alerting professionals of the availability of a test result varies from one setting to another because they may rely on being alerted in a particular way, with the potential for delay if it does not happen.

Overcoming the challenges

Where does this leave us and how can healthcare improve?

Several ways of addressing this can be proposed (Box 1);^4,22,31–36 all will require much more coordination of QI and a far more professionalised approach than has been evident so far. Healthcare should start by agreeing on the kinds of challenges for which full standardisation and interoperability are needed across the sector, and then which solutions can be agreed at the level of principle and left up to local customisation at implementation and which should be entirely locally developed. Healthcare leaders should identify the right kinds of structures for achieving these goals, ranging from international harmonisation mechanisms (similar, for example, to those used in the automobile industry) through to local innovation. Horizontal networks – including those enabled by the royal colleges, as well as initiatives such as the Health Foundation's Q – are likely to be especially valuable, as such structures can accommodate professional groupings who can work together to agree on solutions that are satisfying, workable, informed by professional values and clinical expertise, capable of being customised for specific situations, and enforceable through peers rather than harsh, externally imposed sanctions.^37,38 Finally, it should address the problem of many hands²² by identifying who has responsibility for solving problems for which no single actor in the system has responsibility, but which affect healthcare as a collective.

Box 1.

How to improve the quality of quality improvement

Act like a sector. Allowing a thousand flowers of quality improvement (QI) interventions to bloom is not a sensible or efficient way of going about fixing healthcare and it introduces new risks. As we have argued elsewhere, many of the quality challenges that confront healthcare need to be solved at the level of entire systems,22 not hospital by hospital, practice by practice, care home by care home. Healthcare needs to take itself seriously as a collective whole or sector-like entity capable of agreeing standard operating procedures and systems that are designed with the right expertise, tested properly, implemented with professional leadership at the core, and remain open to innovation. Where technology or external standardisation is the issue – for example, the ongoing failure to address issues of alarm fatigue, incompatible devices or drug-naming and packaging practices – political leadership will be needed, although professional advocacy and involvement will be essential. However, much can be achieved by coming together voluntarily; the key will be to find the right structures for enabling this. A key principle is that such structures should be properly inclusive and include patients, carers and multiple professional disciplines, as well as other sectors and other workers as appropriate. Stop looking for magic bullets – focus on organisational strengthening and learn from positive deviance. When healthcare has sought to learn from other industries, it has not always done so in thoughtful or well-informed ways. It has instead tended to adopt specific interventions (eg checklists) and tried to treat them as magic bullets that are then implemented with little fidelity. Too little has been spent on the organisational strengthening needed to make improvement. Once the search for magic bullet interventions is abandoned, much can be learned from the characteristics, practices and behaviours that are implicated in the performance of demonstrably safe and high-quality settings. This is the approach used, for example, in studies of high-reliability organisations.31,32 The increasingly popular positive deviance approach similarly seeks to learn from exceptionally good performance.33 Sometimes, this approach can help to identify processes that promote high-quality care;34,35 sometimes, it will identify characteristics of context (values, behaviours, structures and so on) that need to be propagated. What is clear already is that organisations need to develop clear goals, manage people and resources effectively, foster a sense of moral community, develop their information and intelligence systems, and ensure that they have the capacity to engage in problem solving.4,36 Build capacity for designing and testing solutions, and plan for replication and scaling from the start. Developing solutions to many quality and safety problems may require high-level skills and expertise from multiple disciplines, and highly sophisticated development processes. It is clear that we need to get better at developing or selecting interventions that have a high likelihood of success, testing them rigorously in different contexts, and offering organisations solutions (the technical and operational issues they need to tackle and the ‘hints and tips’ on the things they will need to do to make the change happen). Much more attention is needed to develop high-quality prototypes of possible solutions in laboratory-like conditions – which may be a designated hospital or network of hospitals that agrees to act as the lab – and undertake modelling and simulation before they are tested for real. The goal of such testing should be to identify, among other things, how the solution might work in different scenarios and conditions, and to work out what are the core, non-negotiable elements and what can be locally customised. Testing should also support intelligent replication and scaling. It is now clear that a simple description of the components of an intervention is not enough; what matters is likely to be the activation of mechanisms, even if precise activities undertaken to activate those mechanisms differ across contexts. Fidelity will lie in the mechanisms rather than fussy adherence to specific forms. Think programmes and resources, not projects. QI projects are sometimes the right answer – for example, where there is a specific, bounded problem to be solved, and particularly if it is one where experience and evidence suggest a plausible solution – but where they are undertaken it should be with a commitment to sharing. In general, thinking and planning long-term programmes of work that are coordinated through some central hub, and that doctors-in-training and others work on for particular periods of time as part of a contribution to a bigger effort (for instance, they might be involved in some of the testing activities described above or on data analysis), may be more productive than individual, short-term projects. Many people who do improvement work are not trained academics and the reports of their work are not traditional academic outputs. However, not being able to publish and share diminishes the attractiveness of improvement work in terms of career rewards and satisfaction. Healthcare needs to do for QI what it has done for research: build an infrastructure that enables learning about successful and less successful efforts to be curated and searched by others. An open-access, peer-reviewed curation model that provides a searchable database of improvement resources that people have developed or used in their organisations is one possibility worth exploring. Authors should be offered guidance on the aspects of the intervention, context and implementation process they should cover to make this resource as accessible, comprehensive and useful as possible.

Conflicts of interest

The authors have no conflicts of interests.