Abstract
Background:
The primary purpose of this review is to synthesise the effect of strategies aiming to sustain the implementation of evidenced based interventions (EBIs) targeting key health behaviours associated with chronic disease (i.e., physical inactivity, poor diet, harmful alcohol use and tobacco smoking) in clinical and community settings. The field of implementation science is bereft of an evidence base of effective sustainment strategies, and as such this review will provide important evidence to advance the field of sustainability research.
Methods:
This systematic review protocol is reported in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocol (PRISMA-P) checklist (Additional file 1). Methods will follow Cochrane gold-standard review methodology. The search will be undertaken across multiple databases, adapting filters previously developed by the research team; data screening and extraction will be performed in duplicate; strategies will be coded using an adapted sustainability-explicit taxonomy; evidence will be synthesised using appropriate methods (i.e. meta-analytic following Cochrane or non-meta-analytic following SWiM guidelines). We will include any randomised controlled study that targets any staff or volunteers delivering interventions in clinical or community settings. Studies which report on any objective or subjective measure of the sustainment of a health prevention policy, practice, or program within any of the eligible settings will be included. Article screening, data extraction, risk of bias and quality assessment will be performed independently by two review authors. Risk of bias will be assessed using Version 2 of the Cochrane risk-of-bias tool for randomised trials (RoB 2). A random effect meta-analysis will be conducted to estimate the pooled effect of sustainment strategies separately by setting (i.e. clinical and community). Sub-group analyses will be undertaken to explore possible causes of statistical heterogeneity and may include: time period, single or multi strategy, type of setting and type of intervention. Differences between sub-groups will be statistically compared.
Discussion/Conclusion:
This will be the first systematic review to determine the effect of strategies designed to support sustainment on sustaining the implementation of EBIs in clinical and community settings. The findings of this review will directly inform the design of future sustainability-focused implementation trials. Further, these findings will inform the development of a sustainability practice guide for public health practitioners.
Registration:
This review was prospectively registered with PROSPERO (registration ID: CRD42022352333)
Keywords: Sustainment, Sustainability, Strategies, Chronic Disease Prevention
Background
Global burden of chronic disease
Preventable chronic diseases such as heart disease, diabetes, and respiratory disease account for a significant proportion of morbidity and mortality, attributing to 70% of all deaths internationally (1, 2). There are several key behavioural risk factors associated with the development of chronic diseases across the life course including: physical inactivity, poor diet, harmful alcohol use, and tobacco smoking (3, 4). Each of these behavioural risk factors is responsible for a considerable proportion (2.78–9.24%) of the total disease burden globally (5).
The World Health Organization (WHO) recommends the implementation of evidence-based interventions (EBIs) in clinical (e.g., hospitals, general practitioner (GP) surgeries, dental practices, community health centres and charity-based health programmes or initiatives) and community settings (e.g., schools, early childcare services, sporting clubs/organisations and community centres) to target and reduce the prevalence and severity of these behavioural risk factors (4). The routine and widespread implementation of EBIs (e.g., targeting physical activity (6) and alcohol reduction (7) to address the prevention of chronic disease in these settings is important, as they provide centralised points of access to reach a large proportion of the population, and they have existing infrastructure to support intervention delivery (8). Consequently, there have been substantial investments made by governments internationally in the development and implementation of EBIs to address behavioural risk factors for chronic disease in these settings (9–11).
There are two distinct outcomes within the field of sustainability: ‘sustainability’ and ‘sustainment’. There are several definitions in the literature for both sustainment and sustainability (12–14). For this review, we make a clear distinction between sustainment and sustainability. We view sustainment as an outcome, defined by Damschroder et al. (2022) as “the extent the innovation is in place or being delivered long-term” (15). Sustainability is defined by Moore et al. (2017) as “after a defined period of time, the program, clinical intervention, and/or implementation strategies continue to be delivered and/or individual behaviour change (i.e., clinician, patient) is maintained; the program and individual behaviour change may evolve or adapt while continuing to produce benefits for individuals/systems” (13). Recent research argues that sustainability should be viewed as a dynamic process with interventions updated according to new evidence and adapted to meet the changing needs of the context and population in which it is being delivered (12, 16).
Although many EBIs provide significant benefits when initially implemented, the effects of these interventions often diminish once initial implementation support or resources are withdrawn, and consequently the quality of intervention delivery decreases or is discontinued entirely (17). Therefore, long-term positive health impacts are often not realised (17–20) or are not achieved equitably across a range of settings and populations (21). Further, discontinuation of programs may also have important implications for wasted investments in time and resources, as well as community member and practitioner mistrust and wariness to engage in future implementation efforts (21). Even with multi-level implementation support and significant financial investment, ‘initiative decay’ is common (10). For example, a systematic review by Wiltsey Stirman, Kimberly (17) focusing on the sustainment of public health and clinical interventions, found that out of 125 studies included in the review, the majority of interventions were only partially sustained (i.e., continuation of some, but not all elements of the intervention), following full initial implementation. Overall, less than half of the interventions included in this review were sustained to high levels of fidelity. Another recent systematic review by Herlitz, MacIntyre (20) which aimed to determine the sustainment of school-based public health interventions, found that of the 18 included interventions, none continued to be delivered in their entirety (i.e., all components) once initial implementation support (start-up funding and/or other resources) had been withdrawn.
Accordingly, policymakers are increasingly concerned with the sustainability of EBIs, and highlight the importance of ensuring the sustained delivery of such interventions long term. To ensure that the positive effects of EBIs continue and health impact is realised, the public health investment in initial implementation is not wasted, and that community support, trust and engagement with such interventions are not lost, it is vital that the implementation of these EBIs be sustained (21).
What impacts on sustaining EBIs
Understanding the determinants of sustainment is essential to successfully design effective sustainment support strategies and reduce implementation decline (22). Theoretical frameworks, such as the Dynamic Sustainability Framework (12), the Program Sustainability Assessment Tool (23), and the Integrated Sustainability Framework (16), identify and categorise a range of factors that may impact the sustainment of EBIs. In general, most frameworks identify sustainability determinants at multiple levels; that is the salient outer contextual factors (e.g., external funding environment), inner contextual factors (e.g., program champions in the organisation), processes (e.g., strategic planning), intervention characteristics (e.g., fit with context and population), and implementer characteristics (e.g., staff attitude, motivation, and skills). Further, systematic reviews of determinants to sustaining EBIs in specific clinical and community settings have identified a number of factors perceived by stakeholders. The most frequently identified being: the availability of equipment, resources and facilities, continued executive or leadership support, and staff turnover (17, 19, 20, 22). Moreover, there are commonalities in factors that commonly impact sustainability across both clinical and community settings such as funding and external partnerships, organisational factors (e.g., alignment with values, needs, resources, and priorities of the organisation) and support (e.g., presence of program champions, leadership support), and practitioner/workforce characteristics (e.g., staff motivation and attitudes (16). The information gathered from these reviews can be utilised to determine which factors to prioritise when developing strategies to sustain EBI delivery.
The need for effective strategies to support sustainment
If policymakers and practitioners are to address determinants of sustaining EBIs, it is important to determine which strategies are most effective in supporting sustainment. It is also important to note that strategies designed to support sustainment may overlap with strategies designed to support initial implementation. Whilst there is a significant body of evidence regarding the effectiveness of strategies to support the initial implementation of EBIs (24–26), to our knowledge, only one review has aimed to collate strategies designed specifically to support sustainment (27).This review of strategies used within community-based settings found only six studies that reported the use of nine types of strategies designed to support sustainment. The most commonly reported strategies were funding and/or contracting for EBIs, continued use and maintenance of workforce skills through continued training, booster training sessions, supervision, and feedback. However, the review was descriptive and given the low number of studies conducted to date, did not synthesise any data relating to the effectiveness or impact of the strategies designed to support sustainment. Additionally, as this review only focused on community settings, there is a current gap which presents a need to synthesise strategies designed to support sustainment in a broader range of settings. Consequently, the field is bereft of an evidence base of effective strategies for sustainment. Research within sustainability science is rapidly increasing. Consequently, there are likely to be numerous new studies that may provide evidence of effective strategies designed to support sustainment. Therefore, the primary aim of this review is to determine the effect of strategies aiming to sustain the chronic disease prevention initiatives targeting key health behaviours (i.e., physical inactivity, poor diet, harmful alcohol use and tobacco smoking) in clinical and community settings.
The secondary aims of this review are to:
Examine the effectiveness of strategies designed to support sustainment on relevant health outcomes (including physical activity, healthy eating, obesity prevention, smoking cessation, or harmful alcohol use)
Describe the cost implications of strategies designed to support sustainment
Identify if there are any unintended/ adverse effects of strategies designed to support sustainment on end-users
Methods
This systematic review protocol was registered with PROSPERO on 20/08/2022 (Registration ID: CRD42022352333) and is reported in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols checklist (PRISMA-P) (28).
Eligibility Criteria
Types of studies
We will include any randomised study with a control group that aims to assess the effect of a strategy or group of strategies to sustain the implementation of a chronic disease prevention EBI in a clinical or community setting. We will include the following types of studies:
randomised controlled trial (RCT) (with a parallel control group)
cluster randomised controlled trial (C-RCT) (with a parallel control group and at least two clusters randomised to each group)
stepped-wedge trial
cross over (only data prior to crossover will be used in the analysis)
We will restrict the review to this set of designs for pragmatic reasons due to the size of this review. Further, these designs are considered as gold standard for assessing casual effects, so are most appropriate for addressing the research questions. We will only include studies that compare a strategy or group of strategies to improve sustainment of a physical activity, healthy eating, obesity prevention, smoking cessation or harmful alcohol use EBI (also termed as policy, practice, or program) with no sustainment intervention or ‘usual practice’. There will be no restriction on the length of the study follow-up period due to the varied definitions of sustainment within the literature. There will also be no restriction on country of origin or language. However, we will exclude studies that are not focused on assessing the effect of a sustainment strategy on the sustained implementation of a policy, practice, or program as a specific aim.
Types of participants:
We will include managers, policy makers, staff, clinicians or volunteers delivering, or supporting the delivery of, EBIs to patients in clinical settings including hospitals, GP surgeries, community health centres and charity-based health programs or initiatives (e.g., charity run smoking cessation and healthy eating interventions in low socioeconomic countries/areas).
We will also include managers, policy makers, staff or volunteers delivering, or supporting the delivery of, EBIs to end users in community settings including educational settings (i.e., primary and secondary schools, colleges and universities), childcare services (long day care, family day care, preschools and nurseries), elite or non-elite sports organisations and clubs (professional and amateur sports clubs, sporting governing bodies), and community centres (youth centres, community outreach centres).
Types of interventions:
We will include any study that employs a strategy or group of strategies with the explicit aim of sustaining the implementation of a smoking cessation, healthy eating, physical activity, alcohol or obesity prevention policy, practice, or program by usual staff, clinicians or volunteers within the setting, for example, managers, policy makers, nurses, doctors, teachers, and carers. Studies embedding principals of sustainability into strategies that have a primary aim of increasing adoption or implementation of EBIs will be excluded. Strategies designed to support sustainment will be classified based on the sustainability-explicit expert recommendations for implementing change (ERIC) glossary (29). To be eligible, strategies designed to support sustainment must be distinct from continuous quality improvement (CQI). Distinctions will be made between sustainment and CQI by recognising CQI as studies focused on making immediate improvements to an individual organisation (30). This is compared to sustainability trials which are typically designed based on theoretical frameworks or models and focused on making generalisable improvements, rather than being restricted on one individual organisation.
Types of outcome measures
Primary outcome measures
Studies that report on any objective or subjective measure of the sustainment of a health prevention policy, practice, or program within any of the eligible settings will be included. This may include the ongoing delivery of physical activity, dietary, alcohol or smoking cessation interventions in line with public health or clinical guidelines.
Sustained implementation must be a measure of usual staff or volunteer delivery of the policy, practice or program and not be externally supported by research personnel, except for the purposes of data collection. Individual outcomes such as sustained effects of patient’s participation in a program (e.g., their participation in a healthy eating program) are not considered sustainment outcomes.
Secondary outcome measures
Data on secondary outcomes will only be extracted for those studies that first meet the eligibility criteria for the primary review outcomes. For example, if a study aims to sustain the implementation of a physical activity policy practice, but reports on dietary outcomes and physical activity practices, only data regarding physical activity practices will the extracted.
Secondary outcomes include:
Health outcomes where an EBI or initiative is used to target modifiable health behaviour risks related to chronic disease. I.e., any objective or subjective measure of diet (e.g., fruit/vegetable intake), physical activity (e.g., minutes of physical activity during the school day), sedentary behaviour (e.g., daily minutes of sedentary time), weight status (e.g., BMI (body mass index)), alcohol consumption (e.g., number of standard drinks consumed on a typical drinking day) and smoking cessation (e.g., weekly number of cigarettes smoked). A hierarchy will be used to prioritise multiple measures of the same health outcome.
Cost outcomes relating to estimates of absolute costs, the assessment of the cost-effectiveness or budget impact of strategies designed to support sustainment.
Any reported adverse effects of strategies designed to support sustainment. This may include negative impact on health outcomes (e.g., an increase in injury rates following physical activity initiatives), disruption to service operation or staff attitudes (e.g., negative impact on staff motivation or cohesion), or negative consequences to other key programs or practices (e.g., lack of funding for other vital programs due to reallocation of funding).
Search methods for identification of studies
We will conduct searches for peer-reviewed articles in relevant electronic databases. We will also hand search relevant journals and reference lists of included studies and related reviews.
Electronic searches
We will conduct searches in the following electronic databases: the Cochrane Central Register of Controlled trials (CENTRAL) (2022) via Cochrane Library; MEDLINE (1946 to November, 2022), PsycINFO (1950 to November, 2022) and Embase (1947 to November, 2022) via OVID; CINAHL (November, 2022) via EBSCO; and SCOPUS (November, 2022) via SCOPUS; Education Research Complete (November, 2022) via EBSCO.
Search Strategy /Search Terms
Search terms will be developed based on reviews conducted by Shelton et al. (16) (maintenance/sustainability); Wolfenden et al. (9–11) (physical activity, nutrition, and obesity, implementation, and setting) and will cover the following four concepts:
Sustainability (other terms include maintenance, durability, continuation, institutionalization, routinization, normalization, integration, adherence)
Heath behaviours (e.g., physical activity, healthy eating, smoking cessation etc.)
Clinical settings (e.g., hospitals, general practice etc.)
Community settings (e.g., schools, workplaces, community centres etc.)
Data Collection and Analysis
Selection criteria
The search results from the electronic databases will be managed and duplicates identified using EndNote. The de-duplicated library will be imported into Covidence software, where article screening will occur. Both title and abstract and full-text screening will be conducted independently by two members of the research team, who will assess study eligibility according to the inclusion criteria. Any conflicts will be resolved by consensus. In instances where the study eligibility cannot be resolved via consensus, a third review author will make the final decision.
Data extraction and management
Two review authors unblinded to author and journal information will independently extract information from the included studies. We will record the information extracted from the included studies in a data extraction form, developed based on the recommendations of the Cochrane Public Health Group Guide for Developing a Cochrane Protocol (31). The data extraction form will be piloted before the initiation of the review. Data extraction discrepancies between review authors will be resolved by consensus or by a third review author if required.
We will extract the following information:
Study eligibility as well as the study design, date of publication, EBI, country, the demographic/socioeconomic characteristics of the program and participants, the number of experimental conditions, setting, overall study duration, and time points measured.
Characteristics of the strategy designed to support sustainment, including strategy description and duration of initial implementation support and length of time since withdrawn (if noted), duration of strategies (i.e., duration for which the sustainment strategy was in place), description of strategies, the theoretical underpinning of the strategy (if noted in the study), process evaluation measures (e.g., acceptability and appropriateness), information to allow classification against the sustainability-explicit ERIC glossary (29). Strategies will be described in line with Proctor et al. (14) recommendations for reporting implementation strategies.
Primary and secondary outcomes within each study, including the data collection method, validity of measures used, effect size and measures of outcome variability
Source(s) of research funding and potential conflicts of interest.
Assessment of risk of bias in included studies
Overall risk of bias
Two review authors will assess risk of bias independently for each review outcome using Version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB 2) described by Sterne et al. (32). Signalling questions will be used for the following domains: Bias arising from the randomisation process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in measurement of the outcome, bias in the selection of the reported result, and overall bias. The response options to the signalling questions will be: ‘yes’, ‘probably yes’, ‘probably no’, ‘no’ and ‘no information’. Once the signalling questions are answered, a risk-of-bias judgement and one of three levels (low risk of bias, some concerns, or high risk of bias) will be assigned to each domain. Stepped wedge trials will be assessed for risk of bias using RoB 2, with consideration given to time confounding. Crossover trials will be assessed using the RoB 2 extension for crossover designs, and only the initial segment prior to crossover will be used in the analysis. We will use the ROB2 extension for cluster trials for the assessment of the risk of bias for cluster RCTs, which includes consideration of the following additional domains: recruitment bias, baseline imbalances, loss of clusters, incorrect analysis, contamination, and compatibility with individually randomised trials. An overall risk of bias will be assigned to each study outcome giving consideration to all of the above domains. Overall risk of bias for study outcomes will be assessed against set criteria and judged as follows: ‘low risk of bias’ (“the trial is judged to be at low risk of bias for all domains”), ‘some concerns’ (the trial is judged to raise some concerns in at least one domain, but not be at high risk of bias for any domain), and high risk of bias (the trial is judged to be at high risk of bias in at least on domain OR the trial is judged to have some concerns for multiple domains in a way that substantially lowers confidence in the result) (Higgins et al., 2022). The risk of bias of the included studies will be documented in a ‘Risk of Bias’ table.
Synthesis methods
Study characteristics will be grouped as types of studies, participants (i.e., clinical or community) and strategies designed to support sustainment. Strategies designed to support sustainment will be classified using the sustainability-explicit ERIC glossary (29). The sustainability-explicit ERIC glossary is a taxonomy which categorises and defines strategies designed to support sustainment. It is an adapted version of the original ERIC (33), which has been extended with a specific focus on sustainability. The sustainability-explicit ERIC glossary will allow us to code the strategies in this review based on the standardised definitions included in the glossary. Deductive and inductive coding approaches will be used and any strategies that do not fit within the sustainability-explicit ERIC glossary will be added. The effect of interest will be intention to treat, and we will prioritise differences between groups at follow-up, rather than differences between groups in the change from baseline. Primary outcomes will be reported using odds ratios and any primary outcomes measured as means and standard deviations will be transformed into odds ratios (Higgins, et al., 2022). For secondary outcomes, the most appropriate effect type will be used, which will include odds ratios for dichotomous outcomes and means for continuous outcomes. Random effects meta-analyses will be undertaken to estimate a pooled treatment effect overall for the primary outcome and by health behaviour for secondary outcomes (i.e., physical activity, alcohol consumption, dietary outcomes, and tobacco use). If we are unable to conduct a meta-analysis due to insufficient or incomplete data (e.g., missing standard deviations) that cannot be estimated from the data reported by authors, we will synthesise results using vote counting based on the direction of effect (31), with such methods reporting in compliance with the synthesis without meta-analysis (SWIM) guidelines (34). For trials with multiple follow-up periods, we will use data from the final follow-up period reported. For studies that report multiple results for primary and secondary outcomes we will prioritise the most objectively measured. Results from cluster and individual-level RCTs will be combined. The standard error from cluster trials that do not adjust for clustering will be adjusted for unit of analysis errors following recommended procedures outlined by the Cochrane Handbook (31). Trials reporting multiple, relevant intervention arms will be combined into a single group following methods outlined in the Cochrane Handbook (31).
Sensitivity analyses
Where there are sufficient studies, a sensitivity analyses removing studies with high risk of bias will be undertaken. If imputation of intra-class correlation coefficient (ICC) values to adjust for clustered trials is required, a sensitivity analysis assessing different ICC values will also be conducted.
Assessing heterogeneity and subgroup analyses
Statistical heterogeneity will be assessed by reviewing the distribution of studies on the forest plots and assessing the I2 statistic. Pre-specified sub-group analyses will be undertaken to explore possible causes of statistical heterogeneity and will include time period classified as sustainability, and type of setting (i.e., clinical or community). Differences between sub-groups will be statistically compared following procedures recommended by the Cochrane Handbook; within subgroup differences will not be interpreted.
Discussion
This systematic review will synthesise current evidence on the effect of strategies designed to support sustainment of chronic disease prevention policies, practices and programs. This will be the first systematic review to determine the effect of strategies designed to support the sustainment of EBIs in both clinical and community settings. The findings of this review will directly inform the design of future sustainability and implementation trials. Further, these findings will help inform the development of a sustainability practice guide for public health practitioners.
Funding
This project is funded through the National Health and Medical Research Council (NHMRC) as part of Dr Nicole Nathan’s Medical Research Future Fund (MRFF) Investigator Grant (APP1194785) and was supported by work undertaken as part of an NHMRC Centre for Research Excellence National Centre of Implementation Science (NCOIS) grant (APP1153479). Dr Nicole Nathan is supported by a MRFF Investigator Grant (APP1194785); LW is supported by an NHMRC Investigator Grant (G1901360). Edward Riley-Gibson is supported by a University of Newcastle PhD scholarship. Adam Shoesmith is supported by a University of Newcastle PhD scholarship (ref. 3145402). Byron Powell is supported in part through grants from the Agency for Healthcare Research and Quality (R13HS025632) and the U.S. National Institutes of Health (R01CA262325, P50CA19006, and R25MH080916). Ramzi G. Salloum is supported by the University of Florida Clinical and Translational Science Institute, which is supported in part by the NIH National Center for Advancing Translational Sciences (UL1TR001427). The funders had no role in the study design, conduct of the study, analysis, or dissemination of findings.
List Of Abbreviations
- EBIs
evidenced based interventions
- PRISMA
Preferred Reporting Items for Systematic review and Meta-Analysis
- CQI
continuous quality improvement
- BMI
Body Mass Index
- RCT
randomised controlled trial
- C-RCT
cluster randomised controlled trial
- ICC
intra-class correlation coefficient
Footnotes
Competing interests
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Contributor Information
Edward Riley-Gibson, University of Newcastle.
Alix Hall, University of Newcastle.
Adam Shoesmith, University of Newcastle.
Luke Wolfenden, University of Newcastle.
Rachel C. Shelton, Columbia University Mailman School of Public Health
Emma Doherty, University of Newcastle.
Emma Pollock, University of Newcastle.
Debbie Booth, University of Newcastle.
Ramzi G. Salloum, University of Florida College of Medicine
Celia Laur, University of Toronto Institute of Health Policy Management and Evaluation.
Byron J. Powell, University of Washington
Melanie Kingsland, University of Newcastle.
Cassandra Lane, University of Newcastle.
Maji Hailemariam, Michigan State University College of Human Medicine.
Rachel Sutherland, University of Newcastle.
Nicole Nathan, University of Newcastle.
Availability of data and materials
Data and materials relating to this review are available from the corresponding author on reasonable request.
References
- 1.Schmidt H. Chronic disease prevention and health promotion. Public health ethics: Cases spanning the globe. 2016:137–76. [Google Scholar]
- 2.Alwan A. Global status report on noncommunicable diseases 2010: World Health Organization; 2011. [Google Scholar]
- 3.Jacob CM, Baird J, Barker M, Cooper C, Hanson M. The importance of a life-course approach to health: Chronic disease risk from preconception through adolescence and adulthood: White paper. 2017.
- 4.Organization WH. Global status report on noncommunicable diseases 2014: World Health Organization; 2014. [Google Scholar]
- 5.Roth GA, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017. The Lancet. 2018;392(10159):1736–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kahn EB, Ramsey LT, Brownson RC, Heath GW, Howze EH, Powell KE, et al. The effectiveness of interventions to increase physical activity: a systematic review. American journal of preventive medicine. 2002;22(4):73–107. [DOI] [PubMed] [Google Scholar]
- 7.Kaner EF, Beyer FR, Muirhead C, Campbell F, Pienaar ED, Bertholet N, et al. Effectiveness of brief alcohol interventions in primary care populations. Cochrane database of systematic reviews. 2018(2). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.McFadyen T, Chai LK, Wyse R, Kingsland M, Yoong SL, Clinton-McHarg T, et al. Strategies to improve the implementation of policies, practices or programmes in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use: a systematic review. BMJ open. 2018;8(9):e019151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wolfenden L, Nathan NK, Sutherland R, Yoong SL, Hodder RK, Wyse RJ, et al. Strategies for enhancing the implementation of school‐based policies or practices targeting risk factors for chronic disease. Cochrane database of systematic reviews. 2017(11). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wolfenden L, Barnes C, Jones J, Finch M, Wyse RJ, Kingsland M, et al. Strategies to improve the implementation of healthy eating, physical activity and obesity prevention policies, practices or programmes within childcare services. Cochrane Database of Systematic Reviews. 2020(2). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wolfenden L, Goldman S, Stacey FG, Grady A, Kingsland M, Williams CM, et al. Strategies to improve the implementation of workplace‐based policies or practices targeting tobacco, alcohol, diet, physical activity and obesity. Cochrane Database of Systematic Reviews. 2018(11). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implementation science. 2013;8(1):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implementation Science. 2017;12(1):1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation Science. 2013;8(1):1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science. 2022;17(1):1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annual review of public health. 2018. [DOI] [PubMed] [Google Scholar]
- 17.Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation science. 2012;7(1):1–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. American journal of public health. 2011;101(11):2059–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Cassar S, Salmon J, Timperio A, Naylor P-J, Van Nassau F, Contardo Ayala AM, et al. Adoption, implementation and sustainability of school-based physical activity and sedentary behaviour interventions in real-world settings: a systematic review. International Journal of Behavioral Nutrition and Physical Activity. 2019;16(1):1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implementation Science. 2020;15(1):1–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Weiner BJ, Lewis CC, Sherr K. Practical Implementation Science: Moving Evidence into Action: Springer Publishing Company; 2022. [Google Scholar]
- 22.Shoesmith A, Hall A, Wolfenden L, Shelton RC, Powell BJ, Brown H, et al. Barriers and facilitators influencing the sustainment of health behaviour interventions in schools and childcare services: a systematic review. Implementation Science. 2021;16(1):1–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. Peer reviewed: the program sustainability assessment tool: a new instrument for public health programs. Preventing chronic disease. 2014;11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Nathan N, Hall A, McCarthy N, Sutherland R, Wiggers J, Bauman AE, et al. Multi-strategy intervention increases school implementation and maintenance of a mandatory physical activity policy: outcomes of a cluster randomised controlled trial. British Journal of Sports Medicine. 2022;56(7):385–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jones S, Sloan D, Evans HE, Williams S. Improving the implementation of NICE public health workplace guidance: an evaluation of the effectiveness of action‐planning workshops in NHS trusts in England. Journal of evaluation in clinical practice. 2015;21(4):567–71. [DOI] [PubMed] [Google Scholar]
- 26.Seward K, Wolfenden L, Finch M, Wiggers J, Wyse R, Jones J, et al. Improving the implementation of nutrition guidelines in childcare centres improves child dietary intake: findings of a randomised trial of an implementation intervention. Public health nutrition. 2018;21(3):607–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implementation Science. 2019;14(1):1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic reviews. 2015;4(1):1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Nathan N, Powell BJ, Shelton RC, Laur CV, Wolfenden L, Hailemariam M, et al. Do the Expert Recommendations for Implementing Change (ERIC) strategies adequately address sustainment? Frontiers in Health Services. 2022;2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Joyce BL, Harmon MJ, Johnson RH, Hicks V, Brown‐Schott N, Pilling LB. Using a quality improvement model to enhance community/public health nursing education. Public Health Nursing. 2019;36(6):847–55. [DOI] [PubMed] [Google Scholar]
- 31.Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions: John Wiley & Sons; 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. bmj. 2019;366. [DOI] [PubMed] [Google Scholar]
- 33.Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 2015;10(1):1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Campbell M, McKenzie JE, Sowden A, Katikireddi SV, Brennan SE, Ellis S, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. bmj. 2020;368. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data and materials relating to this review are available from the corresponding author on reasonable request.