Improving the sustainability of hospital-based interventions: a study protocol for a systematic review

Abstract

Introduction

Sustaining effective interventions in hospital environments is essential to improving health outcomes, and reducing research waste. Current evidence suggests many interventions are not sustained beyond their initial delivery. The reason for this failure remains unclear. Increasingly research is employing theoretical frameworks and models to identify critical factors that influence the implementation of interventions. However, little is known about the value of these frameworks on sustainability. The aim of this review is to examine the evidence regarding the use of theoretical frameworks to maximise effective intervention sustainability in hospital-based settings in order to better understand their role in supporting long-term intervention use.

Methods and analysis

Systematic review. We will systematically search the following databases: Medline, AMED, CINAHL, Embase and Cochrane Library (CENTRAL, CDSR, DARE, HTA). We will also hand search relevant journals and will check the bibliographies of all included studies. Language and date limitations will be applied. We will include empirical studies that have used a theoretical framework (or model) and have explicitly reported the sustainability of an intervention (or programme). One reviewer will remove obviously irrelevant titles. The remaining abstracts and full-text articles will be screened by two independent reviewers to determine their eligibility for inclusion. Disagreements will be resolved by discussion, and may involve a third reviewer if required. Key study characteristics will be extracted (study design, population demographics, setting, evidence of sustained change, use of theoretical frameworks and any barriers or facilitators data reported) by one reviewer and cross-checked by another reviewer. Descriptive data will be tabulated within evidence tables, and key findings will be brought together within a narrative synthesis.

Ethics and dissemination

Formal ethical approval is not required as no primary data will be collected. Dissemination of results will be through peer-reviewed journal publications, presentation at an international conference and social media.

PROSPERO registration number

CRD42017081992.

Strengths and limitations of this study.

• The findings from the review will provide a valuable addition to the evidence base around the use of frameworks to support sustainability.

• The review will identify the barriers and facilitators that influence the delivery of sustained healthcare interventions in hospital-based settings.

• A lack of consistent and complete reporting of sustainability in addition to a consensus on its definition may hinder our review findings.

Introduction

Sustaining the delivery of effective complex interventions is arguably one of the most significant challenges facing healthcare organisations today. The implementation of new interventions to improve both health and organisational outcomes has become commonplace.^1–3 These interventions frequently target professional behaviours.^{4 5} Some of these interventions are mandated by policy directives, but many are not. Instead, the impetus behind their development and implementation often appears reactive to identified systemic weaknesses and calls for remedial action. One recent example of this is the ward-based improvement interventions⁶ that were developed following a series of high-profile public enquiries into the quality of UK healthcare systems.⁷ However, there is growing evidence that the implementation of such interventions are frequently not sustained. A recent study estimated that up to 60% of all newly implemented interventions stop being delivered a few years after their initial funding ceases.⁸

Failure to sustain new interventions can have considerable consequences, including the failure to deliver best practice,⁹ wasting limited resources¹⁰ and practitioners becoming cynical about the value of implementing new interventions eroding the trust between the people who develop complex interventions and those who deliver them.¹¹

The importance of understanding how to improve the sustainability of intervention implementation is well documented.^12–14 Previous reviews have examined factors that improve sustainability of interventions in the health sector.7 10 13 15–17 Wiltsey-Stirman et al ¹³ identified innovation characteristics (eg, fit and effectiveness of the intervention), context (eg, culture and leadership), capacity (eg, funding and resources) and processes and interactions (eg, shared decision-making and adaptation/alignment) as influences on sustainability. More recently, Willis et al ¹⁸ identified factors that impact on culture change and its influence on sustainability within healthcare organisations. Other reviews have developed frameworks to support the sustainment of public health interventions.^{10 19} Such frameworks aim to address the factors identified in the reviews as areas that impact on sustainability. These bodies of work have provided significant contributions to the implementation field. However, to date, there has been no review conducted addressing sustainability of hospital-based interventions.

Well-developed theories about the factors that influence implementation outcomes already exist.²⁰ Such theoretical approaches provide an understanding of why the implementation of interventions can succeed or fail; and can provide strategies that can be adopted to improve the likelihood of successful implementation. Despite the contribution of theoretical frameworks to the understanding of some aspects of implementation, little attention has been given to issues of intervention sustainability; that is the enduring implementation of an intervention after its initial roll-out in practice.^{21 22} Therefore, we propose to conduct a systematic review of theoretical frameworks that have been used to address sustainability of interventions. We focus our review on hospital-based interventions as hospitals have been the focus for a series of implementation projects in recent years. They are challenging and complex environments. Understanding factors that lead to sustained implementation in hospital settings is therefore likely to be of considerable research and practice benefit.

Objectives

The overall aim of this review is to systematically search and synthesise evidence of the use of frameworks in the promotion of sustained intervention implementation in hospital-based interventions. We have three review objectives:

1. To identify, synthesise and appraise empirical studies that use/develop and test a theoretical framework to explore how hospital-based health (both physical and mental health) interventions are sustained in practice.

2. To identify and document the theories that are employed in implementing the sustained hospital-based interventions identified in (1).

3. To identify the barriers and facilitators that influence the delivery of sustained healthcare interventions in a hospital-based setting.

Methods and analysis

Design

We plan to conduct a systematic review and will report the data in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.²³ Our protocol was developed using the PRISMA Protocol (PRISMA-P) checklist.²⁴

Preliminary scoping work

In order to develop an informed, consensual research protocol, key preliminary scoping work was first completed. We employed an iterative-team approach while developing the protocol using the methodological steps outlined in Arksey and O’Malley.²⁵ Detailed notes were kept for each meeting and were circulated to the team afterwards in order to ensure transparency and consensus.

Discussion between the research team led to consensus over:

1. Search strategy.

2. Refining the selection criteria.

3. Key definitions (eg, framework, sustainability, hospital-based).

4. Identifying and agreeing the frameworks that would be used to synthesise the data (eg, barriers and facilitators and theoretical frameworks).

The outcome of each of these discussions are reported in the following sections.

Information sources and search strategy

Search strategy

We plan to generate search terms using the following process:

1. Discussion with the research team to identify relevant terms using background literature that was judged to have relevance to the review. The search strategies, used in previous reviews of sustainability10 13 14 16–18 26 27 will be used to provide a list of potential search terms which will then be discussed by the research team (figure 1).

2. We will examine the search strategies and terms that are published in high-quality empirical studies on sustainability in order to further refine the search strategy.^28–31 As part of our scoping work, the research team have agreed that terms such as ‘strategies’, ‘techniques’ and ‘approaches’ will not be included, as articles that use these terms, if omitted by the search strategy, will be identified through handsearching reference lists of included studies and key journals.

3. We will then combine key terms using a series of free-text terms and MEdical Subject Headings (MESH) for: (a) framework (eg, frameworks, theories, models), (b) sustainability (eg, durability, long-term implementation) and (c) hospital (eg, ward, patient).

4. Finally, we will trial the search using Medline database (Ovid) and refine the search terms if required before conducting the searches on the other electronic databases.

Figure 1.

A summary of the categories of implementation theories, models and frameworks described in Nilsen²⁰ and the level of theoretical visibility reported in Bradbury-Jones et al.³⁶

We will adapt the search strategy for each database as required, using Boolean operators and wildcards to account for variations across databases. Our search terms are detailed in table 1.

Table 1.

Search strategy

Topic	Search terms
Framework	theor* OR model OR models OR principle* OR construct* OR framework*
Sustainability	sustain* OR implement* OR long-term implement* OR long term implement* OR sustain* OR implement* OR long-term implement* OR long term implement* OR routini$ation OR discontinue* OR de-adoption OR deadoption OR durabil* OR institutionali$ation OR maintenance OR capacity building OR knowledge utili$ation
Hospital-based	hospital* OR inpatient* OR in-patient* OR ward* OR unit*

Electronic searches

We will systematically search the following electronic databases from 1 January 2008:

• Medline.

• AMED.

• CINAHL.

• Embase.

• Cochrane Library (eg, CENTRAL, CDSR, DARE, HTA).

We have applied a date restriction to the search, as the Medical Research Council (MRC) complex intervention framework was published in 2008,³² and it is likely that the most relevant studies to our review would have been conducted following the framework’s development.

Other searches

We will handsearch the Implementation Science journal. We will also hand-search the reference list of all included studies.

Eligibility criteria

Selection criteria for inclusion were purposefully wide. We plan to include studies that meet the following criteria:

1. Published in English from 1 January 2008.

2. Empirical studies relating to sustainability of an intervention or programme conducted in a secondary care hospital setting.

3. Studies should focus on sustainability intended to improve patient care and must incorporate a framework or theory or model.

We will exclude studies that are not published in English or are not peer-reviewed publications. We will not include any non-research study designs (eg, unstructured reviews or overviews, theoretical papers, commentaries or opinion papers, protocol, case study, editorial, audit, letter).

We plan to exclude studies that are not conducted in hospital-based settings. In the case of studies performed across multiple settings, studies will be excluded where results pertaining to the hospital setting are not clearly identifiable. In addition, if the service provided is regarded as an outpatient clinic, then the study will also be excluded.

We will not include any studies that do not discuss a specific intervention or programme (ie, solely reports programmes at a general systems level) or only discusses sustainability prospectively (ie, an empirical study has not been carried out). Finally, we will not include studies where sustainability is not a specific concern of the study (ie, it is concerned only with adoption and initial implementation of the intervention/programme) and does not make any reference to frameworks, theories or models that relate to sustainability.

Definition of key terms

We will use the following operational definitions to support the application of the selection criteria:

Sustainability

Over the last 20 years, significant effort has been invested in an attempt to summarise the use of various definitions of sustainability to produce a single definition of the term.^33–35 However, a universally accepted definition is still lacking. Consequently, sustainability has been variously described as maintenance, continuation, institutionalisation, routinisation, durability, maintenance and stability.^{13 16 27}

The review authors considered a number of definitions from the published literature. It was clear from initial discussions that we would need to employ a broad definition of sustainability, and the final consensus was that the review should be guided by the recent work described in Moore et al.³⁴

Moore et al identified 209 articles from four knowledge syntheses of sustainability. Definitions of sustainability were given in 24 (11.5%) of these articles which were used by Moore et al to create a revised definition comprising five key constructs:

• After a defined period of time.

• The programme, clinical intervention and/or implementation strategies continue to be delivered.

• Individual behaviour change is maintained.

• The programme and individual behaviour change may evolve or adapt.

• Continuing to produce benefits for individuals/systems.

We acknowledge that this definition may not be relevant or applicable to all of the different hospital settings. Furthermore, aspects of the definition may be difficult to operationalise particularly if the information is not fully reported within the study (eg, ‘defined’ period of time). Therefore, it was decided that the review team will judge each study against each of the constructs, and document where the information is missing, highlighting any gaps in the evidence base.

Hospital-based intervention

We defined a hospital-based intervention as any intervention that is delivered within a secondary care hospital environment, is aimed at improving patient care, and that directly involves care delivery to patients or staff, but not including ambulatory care, virtual or laboratory-based interventions.

Framework

Theories and models are widely used in implementation science. Consequently, we have decided to use the taxonomy of theories, models and frameworks developed by Nilsen, in which a framework is described as a structure that seeks to define factors which influence implementation outcomes.²⁰

Using Nilsen’s taxonomy, we will classify each framework into one of the following five categories:

• Process model.

• Determinant framework.

• Classic theory.

• Implementation theory.

• Evaluation framework.

We will also assess and report each paper on the degree of theoretical visibility as described by Bradbury-Jones et al.³⁶ Figure 1 summarises how theories, models and frameworks will be categorised and coded according to typology²⁰ and level of theoretical visibility.³⁶

Study selection

One reviewer will run the search strategy, read the titles of the references and eliminate any obviously irrelevant studies. Abstracts will be independently screened by two reviewers who will rank each abstract based on the selection criteria as relevant, irrelevant or unclear. Abstracts that are ranked as irrelevant by both reviewers will be excluded at this stage. Disagreements will be resolved through discussion with a third reviewer. The full text for the remaining (those ranked as relevant or unclear) will be obtained, and reviewed by two review authors against the predefined selection criteria. Consensus meetings will be organised to discuss any disagreements.

Data extraction and coding

Data extraction

A standardised, prepiloted form will be used to extract data from the included studies for assessment of study quality and evidence synthesis.

We will extract the following information:

• Study characteristics (author, date of publication, country, aims, study design).

• Study population.

• Participant demographics (sample size, patient group, details of healthcare professional/staffing groups involved including any details of their length of service, job role).

• Study setting and other relevant contextual information.

• Details of the intervention and interventionist (using the Template for Intervention Description and Replication guidelines³⁷).

• Any comparison conditions.

• Evidence of sustained change (eg, how long the intervention has been delivered on the ward; any associations that the authors report between intervention and sustained effectiveness).

• Framework employed (including the reason for selection).

• Outcomes measures.

• Key findings.

One member of the review team will extract the data from relevant publications into the spreadsheet, and a second will check the data entry. Any differences in the assessment of quality will be resolved via discussion between these authors, and if necessary, referred to a third reviewer for a final decision.

Barriers and facilitators that are identified as important in the delivery of the hospital-based intervention will be extracted by two independent reviewers, using predefined lists outlined in more detail below.

Coding for barriers and facilitators

A single, comprehensive tool for identifying the barriers and facilitators for sustained long-term intervention is currently lacking. Therefore, we propose to use a deductive approach to identify and code the barriers and facilitators using a predefined list of factors based on an amalgam of published frameworks in the fields of implementation science and patient safety. These include the Supporting the Use of Research Evidence (SURE) framework,³⁸ Barriers and Facilitators to Implementation,³⁹ Yorkshire Contributory Factors Framework⁴⁰ and Sustainability constructs in Healthcare.⁴¹ figures 2–5 show each of the domains that are currently described within each of the four implementation frameworks.^38–41 Barriers or facilitators that cannot be categorised using the predefined codes will be coded as ‘other’, and we will use an inductive coding approach to develop themes and subthemes from this additional data.

Figure 2.

A graphical summary of the different domains that are used to categorise the barriers and facilitators across the Geerligs et al ³⁹ framework.

Figure 3.

A graphical summary of the different domains that are used to categorise the barriers and facilitators across the Lawton et al ⁴⁰ framework.

Figure 4.

A graphical summary of the different domains that are used to categorise the barriers and facilitators across the Lennox et al ⁴¹ framework.

Figure 5.

A graphical summary of the different domains that are used to categorise barriers and facilitators across the SURE³⁸ framework.

Methodological quality assessment

Study quality will be assessed independently by two reviewers, using tools appropriate to the design of the study (ie, the Critical Appraisal Skills Programme,⁴² Mixed Methods Appraisal Tool⁴³ and Standards for Quality Improving Reporting Excellence⁴⁴).

Data synthesis

Descriptive data (ie, year, country, professional groups involved, hospital setting and other contextual factors) will be tabulated within evidence tables. Due to the potential heterogeneity between studies and outcomes, we do not plan to conduct a meta-analysis. Key findings will be brought together within a narrative synthesis.

Data specifically relating to the barriers and facilitators will be grouped to derive common themes, taking into consideration the theoretical framework and sustainability. These themes will be organised to provide tabular and narrative summarises of key characteristics.

Patient and public involvement

As the review is analysing secondary data, no patient and public involvement has been involved in the protocol. However, we do plan to involve patients in the later stages of the wider project this review will inform.

Ethics and dissemination

We intend to disseminate the review findings via social media and will present our work at an international peer-reviewed conference. In addition, we aim to publish two peer-reviewed journal publications: one detailing the review findings and a second, discussion piece, reflecting on barriers and facilitators and how these are addressed by frameworks aimed at supporting sustainability.

Conclusions

This systematic review will provide a better understanding of factors that promote or inhibit sustained intervention implementation in hospital-based settings. It is unlikely that there will be a singular contributing factor that is responsible for the longer-term success of any intervention across the diverse settings. However, it is essential to identify and analyse these factors in order to improve the design of hospital-based improvement interventions to include factors that are likely to increase their sustainability. Furthermore, learning about the factors that increase the likelihood of interventions being sustained in practice will provide valuable information for policy-makers, healthcare managers and clinicians.