Adapting improvements to context: when, why and how?

Abstract

There is evidence that practitioners applying quality improvements often adapt the improvement method or the change they are implementing, either unknowingly, or intentionally to fit their service or situation. This has been observed especially in programs seeking to spread or ‘scale up’ an improvement change to other services. Sometimes their adaptations result in improved outcomes, sometimes they do not, and sometimes they do not have data make this assessment or to describe the adaptation. The purpose of this paper is to summarize key points about adaptation and context discussed at the Salzburg Global Seminar in order to help improvers judge when and how to adapt an improvement change. It aims also to encourage more research into such adaptations to develop our understanding of the when, why and how of effective adaptation and to provide more research informed guidance to improvers.

The paper gives examples to illustrate key issues in adaptation and to consider more systematic and purposeful adaptation of improvements so as to increase the chances of achieving improvements in different settings for different participants. We describe methods for assessing whether adaptation is necessary or likely to reduce the effectiveness of an improvement intervention, which adaptations might be required, and methods for collecting data to assess whether the adaptations are successful. We also note areas where research is most needed in order to enable more effective scale up of quality improvements changes and wider take up and use of the methods.

Introduction

A common approach to quality improvement is to take a change proven in one or more healthcare services, such as a surgical safety checklist or a care delivery model, and to implement that change in another healthcare service [1, 2]. The subject of this paper is whether or how a local project team adapts elements of a specific improvement change, originally tested somewhere else, so that it can be implemented in that local setting with local staff and patients.

Instead of ‘implementation’ we use the term ‘take-up’ to indicate the conscious choice that staff (and sometimes patients) make to take up a change to their everyday work and organization. Many quality improvement teams seek to enable staff to use the ‘new better way’ of working and to carefully copy elements that appeared to be important for the improved outcomes in the original test—for example, all parts of the surgical safety checklist intervention. The surgical safety checklist has been tested in many settings and, if implemented with fidelity in these settings, appears to be effective in many places. But is there scope for adaptation of the methods used to ensure that the surgical safety checklist is taken up as a routine practice (for example by giving fewer training sessions?), and if so, how could we develop guidance for local implementers to adapt either the implementation methods, or the content of the improvement changes?

The seminar shared observations that adaptations are often made because the local implementing site has a different ‘context’ to where the improvement change was tested. The context, elements of which are the number and type of staff in the service, local language, the way the service is paid or leadership, may make it difficult to copy the improvement change exactly. Adapting it may be the only way to implement the improvement change, or rather a version of it. But if, for example, some elements of the surgical safety checklist or clinical guidelines are not followed, is this improvement change not effective? Which elements of different improvement changes could be, or need to be adapted to be effective in different contexts? These questions about adapting the improvement change and/or the implementation method were addressed at the seminar: this paper seeks to share our answers to the questions from the discussions at the seminar and our work afterwards.

Definition of terms used in this article

Adaptation: modifying a change that has been tested and found to be effective in another care setting or patient population, such as an assessment method, a treatment method, a care practice (e.g. for sterilization), or a service delivery model.

Contextual factors: ‘may include the political, social, and organizational setting for the implementation of the intervention and include social support, legislations and regulations, social networks, and norms and culture’ [3]

Improvement change: a change to work practices or work organization that results in better patient outcomes and/or less waste, sometimes termed the ‘new better way of working’. Examples: increased compliance with best practice in hand hygiene; or a new way of organizing patient transitions from hospital to home.

Implementation action: actions to invite and enable people to take up and perform the new better way of working, such as training, performance feedback, or providing rewards or incentives.

Improvement method: a method to improve safety, quality and/or reduce waste, typically a systematic method or tool used by staff to collect and analyze data, plan and carry out a change, and spread the change if it is effective. An ‘improvement approach’, such as a quality breakthrough collaborative, combines a number of methods and tools.

Quality improvement investigation: using systematic methods to reduce bias in order to gather and analyze data that is useful for analyzing a quality or safety problem, or to describe or evaluate an improvement change or an improvement method.

Is adaptation inevitable or necessary?

Implementation science has studied the extent to which evidence-based guidelines, treatments or service delivery models are taken up by clinicians and healthcare services, and the results. The research has found that fidelity to the original improvement change is challenging to establish and to sustain, and that three factors appear to explain how completely the ‘new better way’ is implemented:

• – the complexity of the ‘new better way’: simple changes can be copied more easily than more complex changes, for example changes requiring more than one profession or service to change their practice or the organization of their work;

• – the structure and the strategy-actions of the program that are used to establish and support the take up of the new better way: appropriate facilitation, training and other resources are usually necessary to support the take up of the change. This has a cost, which may be recurring for some changes;

• – the context of the care practice that is to be changed: for example, if key healthcare organization leaders do not view the improvement as a high priority, then take up is likely to be low (senior leaders are sometimes considered part of the ‘internal context’); or if the healthcare organization loses money by taking up the new better way, then this improvement might not be sustained (the financial system is one part of the ‘external context’).

Example

Hospital ‘rapid response teams’ (RRTs) or ‘medical emergency teams’ (METs) provide examples to illustrate why, how, and with what effect local projects adapt improvement changes and some of the issues that improvers and researchers may need to consider.

Research in the late 1980s began to report an intervention that had been tested and which provided staff in hospital units with a way of seeking expert help for patients who appeared to be deteriorating and might otherwise need admitting to an intensive care unit (ICU). Normally, staff would contact the senior experienced nurse or doctor assigned to the unit to seek help. There were times when such staff were not available, and providing the option to call a rapid response team (RRT) for expert advice from these teams appeared to result in reduced admissions to intensive care units, and reduced morbidity.

A large body of research has been undertaken over the last 20 years into this intervention [4]. What this research reveals is instructive for answering the questions noted above about adapting an original tested improvement change, how context influenced the adaptations, and about whether adaptations are effective. It originally appeared that what was necessary for successful improvement was the composition of the team, the specification of signs of deterioration, when to call the RRT for advice and the training for staff in the nursing units. However, with publication and reports of this improvement change, many hospitals began to develop and implement RRTs over the next 20 years. The composition of the teams varied considerably as did the strategies for implementing them: the number of members and type of professions, whether physician or nurse-led, the hours available, the equipment and medications brought to the bedside, and access to other specialists or critical care units. Many response teams were formed using existing resources within hospital systems.

For the purpose of this paper we can note that one feature of this multifaceted improvement remained relatively constant across the many implementations were the statements about signs of deterioration that staff were trained about to guide their decisions to call the RRT. Most other features of the ‘RRT improvement change’ were adapted to the local setting, and to fit with the staffing, culture and other aspects of the hospital context: for example, whether to provide a 24 hour-7 day response team or just one at nights, and which arrangements for contacting the RRT rather than a senior clinician. The statements about signs of deterioration were similar across sites because human physiology is relatively similar. Methods for training and reminding staff about these signs were different across sites. Even more different were the other elements of the improvement intervention labeled by the term ‘rapid response team’.

Approaches to scaling up

The research and the literature on experience of scale up programs gives some guidance as to how successfully to enable the take up of evidence-based improvements that have to be copied exactly, for example by changing the context and or designing the implementation structure and strategy to ensure implementation achieves fidelity to the tested original [5–8].

For some well-researched improvement changes, it is possible systematically to plan and guide scale up of certain improvements that need to be implemented with fidelity. However, many improvement changes have only been tested in one setting and with one type of patient group: we do not know from research whether it might be effective in another setting or with another patient group. One way forward is to try to implement the improvement in different settings or with another patient group, and to study the results either using conventional research methods or quality improvement testing methods. This approach can also involve adaptation or iteration by the implementing team [9].

Different types of adaptations have been described by Stirman et al. 2013, such as modifying the training, how practices are performed (e.g. using different cleaning material that was used in the original trial because this material is not available locally), or changing a payment or reporting arrangement from the original [10]. This and other frameworks can be used to guide project teams and researchers in both planning and documenting adaptations of the improvement change.

Other approaches to scale up and learning about improvement

What did we learn from the seminar to answer some of the questions about fidelity and adaption in scale up and how to study these? First, the question about the effectiveness of adaptation: if some elements of an improvement change are not followed, is it the change no longer effective? One answer is that it depends on the type of improvement change, and how precisely the change is described. The seminar discussed many improvement changes: physicians prescribing medications more appropriately, changes to improve antenatal care in Uganda, and changes to improve parenting or violence prevention. Implementation of some may be much more dependent on context—some may be more ‘context sensitive’ [11].

We observe that some improvement changes are precisely described in a way that allows replication (‘specified improvement changes’), and that some are more conceptual, such as the chronic care model [2] (‘change models’). One challenge is that descriptions of quality improvement changes, even in research reports, may not be precise enough to allow replication which also means that it is not possible to judge whether others have or have not adapted it. Two conclusions are that we need better guidance for researchers and quality improvers to describe precisely an improvement change and that this guidance might be different for different types of improvement. Guidance for documentation for researchers is already available, but some do not distinguish the improvement change from the implementation methods in the guidance for description [12–16].

If the adaptation is well-described, a second consideration is how to evaluate an adaptation so as to judge if it is still effective. This relates to the question about understanding how context affects adaptation: which elements of different improvement changes could be, or need to be adapted to be effective in different contexts? One conclusion is that a variety of organizational and external context influences affect implementation [17–20]. These frameworks do not give guidance for implementers about how to adapt the improvement change to their context, but they do show factors that implementers need to consider when developing their implementation plans.

Another conclusion was that different context factors will influence implementation and effectiveness of improvement changes in different ways. Many of these research-based context assessment tools aggregate findings from a range of studies of different interventions and will only give general guidance. For example, an improvement that relies on adding a decision-support prompt to an electronic medical record needs the host organization to have the ‘context’ of an information system that can accommodate the decision-support software, as well as the availability of information technology experts who have the time and skills to add this software to the information system. The context influences for an improvement to prevent falls in a nursing home, or infections in an intensive care unit will be different.

It was clear from the seminar that educational material would need to be adapted if the language of the clients or staff were different to the language of clients or staff involved in the original test of the improvement change. In general, adaptation of different elements to be culturally appropriate would appear to be necessary, and there is some guidance about how to carry out our such adaptations [21, 22].

Conclusions

The seminar and this paper sought to contribute knowledge that would help both research and practical care improvement by identifying questions and giving some answers about adaptation and context. Adaptation of some improvements is likely to decrease their effectiveness. We know a little from research about for which improvements exact fidelity appears to be necessary, and also about the high cost of infrastructures to ensure implementation with fidelity. Less is known about sustainment. In some cases, it is clear that adaptations, such as translating educational materials, are necessary. Also, recent research has developed frameworks describing elements of context that may influence implementation and which may explain or require adaptations of improvement changes. In addition, it may be possible to improve descriptions of adaptations made by different project teams in order to assess whether these adaptations are more or less effective in different contexts [23, 24]. Project teams also can assess the effectiveness of their changes and this can provide additional data to develop answers to the question: when why and how to adapt improvements to context. Improvers, funders and researchers at the seminar were all agreed that improvements, especially in scale up programs could be more effective if we had better answers to these questions. In addition, there are now excellent opportunities to develop answers using recently developed research methods, tools and forms of partnership research.