Sustainability of professionals’ adherence to clinical practice guidelines in medical care: a systematic review

Abstract

Objectives

To evaluate (1) the state of the art in sustainability research and (2) the outcomes of professionals’ adherence to guideline recommendations in medical practice.

Design

Systematic review.

Data sources

Searches were conducted until August 2015 in MEDLINE, CINAHL, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL) and the Guidelines International Network (GIN) library. A snowball strategy, in which reference sections of other reviews and of included papers were searched, was used to identify additional papers.

Eligibility criteria

Studies needed to be focused on sustainability and on professionals’ adherence to clinical practice guidelines in medical care. Studies had to include at least 2 measurements: 1 before (PRE) or immediately after implementation (EARLY POST) and 1 measurement longer than 1 year after active implementation (LATE POST).

Results

The search retrieved 4219 items, of which 14 studies met the inclusion criteria, involving 18 sustainability evaluations. The mean timeframe between the end of active implementation and the sustainability evaluation was 2.6 years (minimum 1.5–maximum 7.0). The studies were heterogeneous with respect to their methodology. Sustainability was considered to be successful if performance in terms of professionals’ adherence was fully maintained in the late postimplementation phase. Long-term sustainability of professionals’ adherence was reported in 7 out of 18 evaluations, adherence was not sustained in 6 evaluations, 4 evaluations showed mixed sustainability results and in 1 evaluation it was unclear whether the professional adherence was sustained.

Conclusions

(2) Professionals’ adherence to a clinical practice guideline in medical care decreased after more than 1 year after implementation in about half of the cases. (1) Owing to the limited number of studies, the absence of a uniform definition, the high risk of bias, and the mixed results of studies, no firm conclusion about the sustainability of professionals’ adherence to guidelines in medical practice can be drawn.

Strengths and limitations of this study.

• This is the first systematic review of the literature that has considered professionals’ adherence to clinical practice guidelines more than 1 year after active implementation. This review shows that in half of the sustainability studies professionals fully sustained in their adherence to a clinical practice guideline.

• This review showed that sustainability research is a relatively new and underexplored field in healthcare.

• Sustainability research is not well indexed in electronic databases, and text word searches are prone to high recall and low specificity. However, it is likely that the use of a broad variety of search terms that covered sustainability has downsized the number of relevant studies missed and is a strength of the review.

• The number of studies and the methodological quality of the studies focusing on the sustainability of professionals’ adherence are limited. This makes it difficult to draw firm conclusions.

Introduction

Quality of care can be improved by decreasing unwarranted practice variation between professionals. One way to reduce practice variation is by transferring evidence-based knowledge into daily practice. To facilitate the translation of the most recent evidence into practice, guidelines are developed and implemented. Following the Institute of Medicine (IOM), clinical practice guidelines are “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefit and harms of alternative care options”.¹ Guidelines contain practical evidence-based advice for professionals and patients and aim to improve the quality of care.² In general, uptake of guidelines does not happen spontaneously and often an active implementation approach is required.³ Moreover, once a guideline is successfully implemented in practice, it may be difficult to sustain the quality improvements over a longer period of time. People tend to fall back into old routines⁴ which may impact long-term adherence to a guideline.

The road towards sustainability of healthcare innovations into practice is suggested to be a dynamic process,⁵ and sustainable adherence may not be self-evident without continued efforts. Sustainable change of professionals’ behaviour has the potential to result in more optimal healthcare delivery and efficiency. Not sustaining quality improvements can result in nihilistic attitudes towards future innovation. In recent years, sustainability has gained attention in healthcare. Unfortunately, the concept of sustainability is still underdeveloped.^{6 7} Some existing reviews studied sustainability from a wide healthcare perspective, including studies varying from medical care to public health. Results showed that determinants of sustainability varied widely between healthcare areas^{8 9} and suggest that partial sustainability of healthcare innovations is more common than full sustainability.¹⁰

In this systematic review, the scope of sustainability research will be narrowed to professionals’ adherence to clinical practice guidelines in medical care. The aim of the current review was to evaluate the state of the art in sustainability research and the level of sustained professionals’ adherence to guideline recommendations in medical practice more than 1 year following the cessation of the implementation project.

Methods

Eligibility criteria

Studies needed to be focused on sustainability and on clinical practice guidelines. Sustainability was described as “Sustainability of change exists when a newly implemented innovation continues to deliver the benefits achieved over a longer period of time, certainly does not return to the usual processes and becomes ‘the way things are done around here’,¹¹ even after the implementation project is no longer actively carried out, until a better innovation comes along”.¹² Studies had to include at least two measurements: one before (PRE) or immediately after implementation (EARLY POST) and one measurement longer than 1 year after active implementation (LATE POST). All activities to facilitate the adherence to clinical practice guidelines were labelled as part of the implementation project. Studies needed to be focused on professionals’ adherence to a clinical practice guideline. Studies only using self-reported adherence were excluded to reduce the chance of social desirability bias and an overestimation of results.¹³ Lastly, studies had to focus on medical care. Participants had to be healthcare professionals who deliver direct patient care. There were no restrictions on study design of the research articles.

Search methods for identification of studies

Electronic searches

We searched MEDLINE (OvidSP; 1946 to February 2014), CINAHL (EBSCO Host; 1982 to February 2014), EMBASE (OvidSP; February 2014), Cochrane Central Register of Controlled Trials (CENTRAL) and the Guidelines International Network (GIN) library for studies. The electronic search strategy was designed to focus on sustainability of guideline recommendations. Free-text terms and MeSH terms regarding sustainability, quality improvement, impact and guideline recommendations were used. An information expert checked the developed search strategies (see online supplementary file 1). Before final analyses, update searches were performed to identify possible additional studies (26 June 2014 and 4 August 2015).

Searching other resources

A snowball strategy was performed, in which the reference sections of reviews^{6–10 14–16} and research papers on sustainability^{17 18} were searched. Also, databases such as PubMed and the Web of Knowledge Science Citation Index were used to locate publications and publications citing the original references. The process was repeated for any new relevant publication found.

Data collection and analysis

Selection of studies

All records were merged into a bibliographic database and screened independently by two reviewers (SMCA and JJAdG) based on title and abstract. Full-text screening was performed by two reviewers (SMCA and JJAdG). Disagreement on selection was resolved in consensus meetings with a third reviewer (TvdW). Reasons for exclusion were documented during the full-text screening. If more clarification or details of a study were needed, an author was contacted. Authors of conference abstracts were emailed and were asked to send the research protocol. Duplicate papers were identified, and all papers published on one study were used for retrieving information.

Data extraction and management

Data of the methodology and results were independently extracted by two reviewers (SMCA, JJAdG), guided by a predefined data extraction form. Effective Practice and Organisation of Care (EPOC) Data Collection Checklist¹⁹ items (eg, location of care, type of targeted behaviour, implementation interventions) were integrated in the data extraction form. The data extraction form was developed by the authors and was pilot tested. The following study characteristics were recorded: study design, publication year, whether the study was executed in a single centre or in multiple centres, type of targeted behaviour, location of care, the name of the clinical practice guideline, clinical specialty, the implementation activities used and whether or not the implementation strategy was externally guided. An externally guided implementation strategy is a strategy which is lead and supported by an external expert organisation. With respect to the methodology of the sustainability evaluation, the following data were extracted: the timeframe between the end of the implementation strategy and the sustainability evaluation, the applied definition of sustainability, the data collection method, whether the evaluation was performed on patient, hospital or multiple hospital level and whether the sustainability evaluation was performed on single or multiple centre level. With respect to the outcome measures of the studies, data on the professionals’ adherence rates before, early after implementation and longer than 1 year after implementation, and the authors’ comments with respect to the sustainability of professionals’ adherence were extracted. Adherence was presented in terms of proportion of patients receiving treatment according to the clinical practice guideline recommendations. If sustainability of professionals’ adherence to a clinical practice guideline was evaluated at multiple postimplementation moments, the latest evaluation was selected as LATE POST measurement. The authors (SMCA and JJAdG) checked if updates of the clinical practice guidelines had become available in the postimplementation phase (eg, between the EARLY POST and the LATE POST measurement), which may explain reduced professionals’ adherence. Disagreement on data extraction was resolved in consensus meetings with a third reviewer (TvdW).

Assessment of risk of bias in included studies

Risk of bias assessment was independently conducted by two authors using the Downs and Black checklist for randomised and non-randomised studies.²⁰ This is a checklist which can be used for checking the risk of bias of original research articles of various study designs. Results were interpreted under consideration of risk of bias. The assessments were also used for recommendations for further research by identifying elements of studies that can be improved in future studies. The checklist was adapted to the research question. Risk of bias of the studies was presented on reporting, external validity, internal validity (bias and confounding), power and overall level.

Analysis

The analysis was narrative. This included a summary of the methodological characteristics of the sustainability evaluations, descriptions of the level of sustainability as mentioned by the author, and the level of sustained professionals’ adherence compared with results achieved immediately after implementation. Sustainability was considered to be successful if performance in terms of professionals’ adherence was fully maintained in the late postimplementation phase. A sensitivity analysis was performed by applying a 90% instead of 100% adherence criterion of sustainability.

Results

Description of studies

For this review, 4219 items were retrieved and screened based on title and abstract, and 185 studies were assessed based on full-text reading. Figure 1 shows the study selection process as recommended by the PRISMA statement²¹ (see online supplementary file 2). Fourteen studies met the inclusion criteria for this review, describing 18 sustainability evaluations.^22–35 Table 1 presents the characteristics of the included studies. Two publications were published before and 12 after 2000.^{23 33} In six studies, the targeted behaviour was prescribing,^{24 25 28 30 33 35} in four studies procedures,^{29 31 32 34} in three studies general management of a problem^{22 26 27} and in one study²³ general management of a problem and prescribing. The location of care was inpatient in five studies,^{23 28 29 32 34} outpatient in four studies^{24–26 35} and mixed in five studies.^{22 27 30 31 33}

Figure 1.

PRISMA flow diagram.

Table 1.

Characteristics of the included studies

Study ID	Study design	Clinical practice guideline	Clinical specialty	Clinical practice guideline was updated in the postimplementation phase* (yes/no)	Time frame (years)
Ament et al,22 The Netherlands	Case series	Guideline to facilitate short stay for breast cancer surgery36	Surgery	Between 2007 and 2012: no42 43†	5
Benenson et al,23 UK	Case series	Clinical pathway for pneumonia44	Various	Between 1995 and 1997: no45 46	3
Cates,24 UK	Case series	Guideline for antibiotic prescription for children with earache and inflamed eardrums who are not unduly ill47	General practice	Between 1998 and 2001: no48†	Centre 1: 3; centre 2: 2
Enriquez-Puga et al,25 UK	RCT	(1) Antidepressant prescription guideline and49 (2) Antibiotic prescription guideline50 Control group: intervention groups were each other's control group	General practice	Guideline 1 between 2003 and 2004: yes51 Guideline 2 between 2003 and 2004: no50	1.5
Forsner et al,26 Sweden	RCT	Clinical guideline (1) for depression52 and (2) for suicidal behaviours48 Control group: received the guideline but were not included in the intervention	Psychiatry	UTD	1.5
Gerber et al,35 USA	Case series	Outpatient antimicrobial stewardship intervention53	Pediatric primary care	Between 2011 and 2014: no53	1.5
Higuchiet al,27 Canada	Case series	(1) Adult Asthma Care Best Practice Guideline54 and (2) Reducing Foot Complications for People with Diabetes Best Practice Guideline55	(1) Various (2) Various	Guideline 1 between 2002 and 2006: yes56 Guideline 2 between 2003 and 2006: yes57	(1) 4 (2) 3
Kelly,28 Australia	Case series	Guideline for nurse managed titrated narcotic analgesia58	Emergency medicine	UTD	2
Knops et al,29 The Netherlands	Case series	(1) A fluid balance guideline for oncology patients38 (2) A body temperature guideline for postoperative patients59	(1) Various (2) Surgery	Guideline 1 UTD (local guideline) Guideline 2 UTD (local guideline)	7
Loszadi et al,30 UK	Case series	Guidelines for the prevention and management of corticosteroid-induced osteoporosis60	Neurology	UTD	Unknown, >2
McLaws et al,31 Australia	Case series	Guidelines on Hand Hygiene in Health Care61	Various	Between 2007–2008: no61	1.5
Stéphan et al,32 Switzerland	Case series	Guideline for urine catheterization management for surgical procedures62	Orthopaedic / abdominal surgery	UTD (local guideline)	1.5
Wakefield et al,33 USA	Case series	Guideline for the use of transdermal fentanyl for chronic pain33	Various	UTD	1.5
Williams et al,34 UK	Case series	Guideline for the repair and follow-up of third degree tears63	Obstetrics and gynaecology	UTD (local guideline)	2

*The cpg was updated between the POST and LATE POST measurement (yes) or was not updated between the POST and LATE POST measurement (no). †Not updated with respect to the key recommendations of the guideline. The guideline was adopted in national guidelines in the postimplementation phase.

RCT, randomised controlled trial; UTD, unable to determine.

The implementation strategy was described in 13 studies (table 2).^{22–27 29–41} According to the EPOC checklist classification, in one study,²⁴ a single element implementation strategy was executed while in the other 12 studies a multifaceted implementation strategy was executed. Implementation activities were professional-targeted interventions (n=12) {Ament, 2014 #20; de Kok, 2010 #33; Ament, 2014 #34; Benenson, 1999 #21; Enriquez-Puga, 2009 #23; Forsner, 2010 #24; Higuchi, 2011 #25; Knops, 2010 #27; Mank, 2003 #35; Knops, 2010 #27; Lozsadi, 2006 #28; Storm-Versloot, 2012 #36; Pantle, 2009 #37; Stephan, 2006 #30; Wakefield, 1998 #31; Williams, 2003 #32; Gerber, 2013 #67; Gerber, 2014 #69}, followed by organisational interventions (n=6)^{22 23 26 33 34 36 37 39 40} and financial interventions (n=1).²⁷ In six studies, the implementation strategy was facilitated by external experts.^{22 25–27 31 35} In one study, it was unclear whether the implementation strategy was externally supported.²⁸

Table 2.

Implementation strategies as described by the authors

	Professional interventions
Author	Distribution of educational materials	Educational meetings	Local consensus processes	Educational outreach visits	Local opinion leaders	Patient-mediated interventions
Ament22	●	●	●	●	●
Benenson23			●
Cates24
Enriquez-Puga,25 antidepressant prescription guideline	●		●	●
Enriquez-Puga,25 antibiotic prescription guideline	●		●	●
Forsner,26 depression guideline	●	●	●	●	●
Forsner,26 suicidal behaviours guideline	●	●	●	●	●
Gerber35			●
Higuchi,27 Adult Asthma Care Best Practice Guideline	●	●		●	●
Higuchi,27 Reducing Foot Complications for People with Diabetes Best Practice Guideline	●	●		●	●
Kelly28*
Knops29 fluid balance guideline for oncology patients*
Knops29 body temperature guideline for postoperative patients	●	●
Loszadi30		●
McLaws31	●			●	●
Stéphan32	●	●	●
Wakefield33		●	●
Williams34

	Professional interventions					Financial interventions
Author	Audit and feedback	Reminders	Marketing	Mass media	Other	Other
Ament22	●
Benenson23
Cates24
Enriquez-Puga,25 antidepressant prescription guideline	●		●
Enriquez-Puga,25 antibiotic prescription guideline	●		●
Forsner,26 depression guideline	●				Participation in local network
Forsner26, suicidal behaviours guideline	●				Participation in local network
Gerber35	●
Higuchi,27 Adult Asthma Care Best Practice Guideline	●			●		Additional funding to replace nurses while they performed implementation activities
Higuchi,27 Reducing Foot Complications for People with Diabetes Best Practice Guideline	●			●		Additional funding to replace nurses while they performed implementation activities
Kelly28*
Knops29 fluid balance guideline for oncology patients*
Knops29 body temperature guideline for postoperative patients	●		●
Loszadi30	●
McLaws31	●	●		●
Stéphan32	●	●		●
Wakefield33		●
Williams34	●

	Organisational interventions
Author	Revision of professional roles	Clinical multidisciplinary team	Formal integration of services	Skill mix changes	Continuity of care	Changes in physical structure, facilities and equipment	Presence and organisation of quality monitoring	Other
Ament22		●	●		●
Benenson23		●	●		●			Standard antibiotic order sheet
Cates24								Evidence-based patient handout
Enriquez-Puga,25 antidepressant prescription guideline
Enriquez-Puga,25 antibiotic prescription guideline
Forsner,26 depression guideline		●
Forsner,26 suicidal behaviours guideline		●
Gerber35
Higuchi,27 Adult Asthma Care Best Practice Guideline								New documentation procedures
Higuchi,27 Reducing Foot Complications for People with Diabetes Best Practice Guideline								New documentation procedures
Kelly28*
Knops29 fluid balance guideline for oncology patients*
Knops29 body temperature guideline for postoperative patients
Loszadi30
McLaws31						●	●
Stéphan32
Wakefield33		●
Williams34						●

*No information about the implementation strategy provided.

● Item explicitly stated in one of the related articles of the study.

Characteristics of the sustainability evaluations

The mean timeframe between the end of the implementation strategy and the sustainability evaluation of 13 studies was 2.6 years (minimum 1.5–maximum 7.0). The actual timeframe of one evaluation was unclear, but was at least 2 years.³⁰ Two studies referred to a definition of sustainability.^{22 27} Eight studies used a retrospective data collection method,^{23–29 33} three studies used a prospective data collection method^{31 32 35} and three studies used both a prospective and a retrospective data collection method.^{22 30 34} Ten papers reported the level of sustained adherence of a single clinical practice guideline,^{22–24 28 30–35} while four reported the late postimplementation adherence of two clinical practice guidelines.^{25–27 29} Seven studies had a single-centre design^{23 28–30 32–34} and seven studies evaluated sustainability in multiple centres.^{22 24–27 31 35} Four out of six multiple centre studies evaluated the sustainability on multiple centre level.^{22 26 31 35} Two out of six multiple centre studies evaluated the sustainability of professionals’ adherence of two guidelines which were implemented in one centre each.^{24 27}

Sustainability of changed behaviour

The level of professionals’ adherence was fully sustained in 7 out of 18 evaluations (table 3, see online supplementary file 3). The adherence was not fully sustained in six evaluations, and four evaluations showed mixed sustainability results in the LATE POST measurement compared with the EARLY POST measurement. In one study, the EARLY POST measurement was not executed, while the authors reported sustained results.²⁸ After decreasing the sustainability level of professionals’ adherence to 90% or higher, 9 out of 18 evaluations showed sustained results, 3 evaluations showed no sustained results, 4 evaluations showed mixed results. In two evaluations, it was unclear whether the professionals’ adherence had been sustained at a level 90% or higher.

Table 3.

Sustainability of professionals’ adherence to clinical practice guidelines

Study ID	Authors’ comments in terms of sustainability of adherence to the clinical practice guideline*	Sustained compared with early implementation results (100%) (yes/no)†	Sustained compared with early implementation results (90%) (yes/no)‡
Ament et al22	“Adherence to the guideline recommendations was sustained in four early adopter hospitals”	Yes	Yes
Benenson et al23	“The observed pre pathway to post pathway differences were sustained over three years”	Yes	Yes
Cates24	(Centre 1 and 2) “our approach has brought about a sustained reduction in the use of antibiotics for children with acute otitis media, and after dissemination of our findings, similar results have been replicated at centre II using deferred prescribing of antibiotics for children who are not unduly ill”	Yes	Yes
Enriquez-Puga et al25	“There was a small change in the desired direction in the proportion of antidepressants prescribed according to guidelines that lasted for 24 months, although no change for antibiotics. A simple, group level educational outreach intervention, designed to take account of identified barriers to change, appears to have a small sustained effect on prescribing levels, but the effect is not consistent across different groups of drugs”	Guideline 1: no Guideline 2: yes	Guideline 1: no Guideline 2: yes
Forsner et al26	“This study suggested that the compliance to clinical guidelines, for treatment of depression and suicidal behaviour, was implemented and sustained over a two-year period after an active implementation”	Guideline 1: mixed Guideline 2: mixed	Guideline 1: mixed Guideline 2: mixed
Gerber et al35	Not mentioned	No	No
Higuchi et al27	(1) “The chart audit revealed that eleven nursing care indicators related to the asthma guideline recommendations showed a mixed pattern of sustainability” (2) Not mentioned	Guideline 1: mixed Guideline 2: mixed	Guideline 1: mixed Guideline 2: mixed
Kelly28	“The study demonstrated a significant and sustained change in analgesia administration practices away from the intramuscular (IM) route in favour of the IV route”	NA	NA
Knops et al29	(1) Not mentioned (2) Not mentioned	Guideline 1: yes Guideline 2: no	Guideline 1: yes Guideline 2: no
Loszadi et al30	Not mentioned	Yes	Yes
McLaws et al31	Not mentioned	No	Yes
Stéphan et al32	“One of the most important results of our intervention is its sustained impact. In particular, the frequency of catheter use decreased in the operating room not only immediately after guideline implementation, but also could be observed 2 years later”	No	Yes
Wakefield et al33	Not mentioned	No	Na
Williams et al34	Not mentioned	Yes	yes

*Citations of the authors of reviewed papers about the sustainability of adherence to the clinical practice guideline. †The same level or improved professionals’ adherence was achieved years after implementation compared with early postimplementation results (yes/no). ‡At least 90% of professionals’ adherence was achieved years after implementation, compared with early postimplementation results (yes/no).

Mixed, the overall professionals’ adherence was not presented, and both sustained and not sustained levels of professionals’ adherence to clinical practice guideline recommendations were achieved in the late post-implementation phase compared to early post-implementation results; NA, not applicable as the early post-implementation results were not measured.

Five of the 10 papers that reported about a single clinical practice guideline presented sustained professionals’ adherence to clinical practice guidelines in the LATE POST measurement.^{22–24 30 34} One of these five papers evaluated the sustainability of a single clinical practice guideline in two centres.²⁴ In both centres, professionals’ adherence had improved in the LATE POST measurement compared with the EARLY POST measurement. The four studies analysing the sustainability of two clinical practice guidelines showed mixed results. Two of these four studies^{25 29} presented the same level or improved adherence to one guideline and decreased adherence to the other guideline in the LATE POST measurement compared with the EARLY POST measurement. The other two of these four studies^{26 27} presented adherence results on guideline recommendation level and did not present overall adherence results on patient level. The adherence to the recommendations of the clinical practice guidelines showed decreased and improved levels in the LATE POST measurement compared with the EARLY POST measurement. In total, eight papers mentioned the term ‘sustainability’ in the conclusion (table 3).^{22–28 32} Five of these studies concluded to have sustained professionals’ adherence in the late postimplementation phase,^{22–24 28 32} three out of eight studies described to have a ‘mixed pattern’, ‘small desired’ or ‘almost’ sustained professionals’ adherence.^25–27

Risk of bias in included studies

All studies included in the present review had a high risk of bias, following the Downs and Black assessment tool²⁰ (table 4, see online supplementary file 4).

Table 4.

Results of the risk of bias assessment

Study ID	Reporting	External validity	Internal validity—bias	Internal validity— confounding	Total
Ament et al22	Unclear	High	High	High	High
Benenson et al23	Unclear	High	High	High	High
Cates24	High	Unclear	High	High	High
Enriquez-Puga et al25	Unclear	High	High	Unclear	High
Forsner et al26	Unclear	Low	High	High	High
Gerber et al35	Low	Low	High	High	High
Higuchi et al27	High	High	High	High	High
Kelly28	High	High	High	High	High
Knops et al29	High	Low	High	High	High
Loszadi et al30	Unclear	High	Unclear	High	High
McLaws et al31	High	Low	Unclear	High	High
Stéphan et al32	High	High	Unclear	High	High
Wakefield et al33	High	High	High	High	High
Williams et al34	High	High	Unclear	High	High
Total	High	High	High	High	High

Discussion

Our review focused on the level of sustainability of implementation success in terms of professionals’ adherence. Also, this systematic review described the state of the art in sustainability research. This systematic review identified 14 studies, including 18 evaluations that investigated the sustainability of professionals’ adherence to a clinical practice guideline more than 1 year after the implementation was finished. Of 18 analyses that focused on the extent of sustained professionals’ adherence to a clinical practice guideline, seven analyses revealed fully sustained results. After decreasing the sustainability level of professionals’ adherence to 90% or higher, 9 out of 18 evaluations showed sustained results. The current review showed that the number of sustainability studies is scarce and that the studies are heterogeneous with respect to their methodology. Furthermore, almost no study analysed or reflected on the updates of the guideline in the postimplementation phase. The results of this review suggest that updates of the clinical practice guidelines may have led to a warranted decrease in the adherence to the original clinical practice guideline.

In this systematic review, information was presented about how to search for sustainability evaluations, how sustainability research is defined and about the type and the methodological quality of studies that report on sustainability. As was confirmed in another systematic review,¹⁰ the sustainability studies showed to have limited methodological rigour. Two out of 14 studies used an experimental design. The lack of identified studies in the current review suggests that most teams do not focus on the long-term performance effect of quality improvements.⁶⁴ Owing to the limited number of studies focusing on this subject, the heterogeneity in studies, suboptimal reporting by authors and the revealed methodological weaknesses, no strong conclusions can be drawn based on the presented sustainability results. As also shown in other research, most sustainability studies used a single-case study design by focusing on a single type of programme or performed the evaluation at a single centre level.⁶⁵ The current review showed that in only two of the studies, a reference for the definition of sustainability was used. Other studies performed a sustainability evaluation without mentioning a definition. This shows the underdeveloped field of sustainability research. Also, a variety of timeframes to study the sustainability of professionals’ adherence to clinical practice guidelines was revealed, varying from 1½ to 7 years following implementation.

Optimal adherence to a clinical practice guideline as determined during implementation is not always desired; for example, clinical experience and evidence may change. This systematic review included all research designs and seems to be the first review with respect to sustainability of professionals’ adherence to clinical practice guidelines to date. Other reviews focused on healthcare from a broad perspective including multiple healthcare fields¹⁰ or reviewed studies performed specifically in public health.^{6 9} The sustainability of a health programme in public health may be influenced by other determinants than the sustainability of a clinical practice guideline in medical care. Also, the concept of the sustainability may differ between healthcare fields. For example, in public health sustainability of a health programme may be successfully sustained if health outcomes, for example, changed lifestyle, are maintained and financial support is still available.^{6 65} In medical care, the primary focus is on the quality and safety of care which is supposed to be captured in clinical practice guidelines. Owing to the specific focus on clinical practice guidelines in the current review, mainly other studies were included compared with the existing sustainability reviews.^{6–10 14}

Strengths and weaknesses

As yet, the term ‘sustainability’ is not consistently used for this area in the broader medical field, which presents a limitation to the electronic search strategy. The topic is not well indexed in electronic databases, and text word searches are prone to high recall and low specificity. However, it is likely that the use of a broad variety of search terms that covered sustainability has downsized the number of relevant studies missed and is a strength of the review.

In this systematic review, sustainability was assessed as successful if performance in terms of professionals’ adherence was fully maintained in the late postimplementation phase. This definition of sustainability may be too pragmatic as it could be undesirable to fully sustain the professionals’ adherence in the late postimplementation phase. Therefore, a sensitivity analysis was performed to analyse the sustainability at a level of 90% or higher. However, as mentioned before, a limitation of the review is the high risk of bias of all studies included. The majority of the studies used a retrospective data collection method. Nevertheless, results were interpreted under consideration of risk of bias, and the assessments were also used for recommendations for further research by identifying elements of studies that can be improved in new studies. Also, the question is what the best method is for evaluating sustainability. For example, retrospective data may be desired to prevent a Hawthorne effect when studying routine practice.

The results of the current review show more studies with sustained professionals’ adherence than might be expected without continuing efforts and support to promote the level of sustained adherence in the postimplementation phase. Possibly, studies with unfavourable results may not be published or unsuccessful implementation projects may not be evaluated, leading to an under-representation of the true amount of work carried out in the field.^{66 67}

Sustainability of professionals’ adherence may be influenced by the perceived quality of the guideline. However, we were not able to analyse the quality of the guidelines given the limited information in the manuscripts and the information on the internet on the specific guidelines. More information about the quality of the guidelines in sustainability evaluations may be helpful to analyse the sustainability of the guideline. Also, the potential effect of the specific implementation strategies was not analysed as part of the systematic review. Professionals’ adherence is an outcome measure used in implementation science, and it captures the behaviour change as a result of implementation strategies. The type of implementation strategy may have had an effect on the sustainability of the implementation results. The studies included used various implementation strategies and implemented different clinical practice guidelines.

Implications for practice

The current review showed that the level of the sustainability of professionals’ adherence to clinical practice guidelines varies on case study level and drops in more than half of the studies. Owing to the lack of sustainability research we think that sustainability failure as presented in this study is an underestimation. Unfortunately, implementation projects are primarily focused on short-term actions and short-term effect.⁶⁴

Future research

This review complements the existing sustainability research by focusing on sustained professionals’ adherence in medical practice. The current review showed that not many studies reported data on the sustainability of professionals’ adherence to clinical practice guidelines. Also, no strong conclusions can be drawn due to the high risk for bias and the heterogeneity of the studies. As shown in previous research, structural methods for sustainability evaluations are lacking.^{10 68} Furthermore, future implementation and sustainability evaluations may include information about the quality of the clinical practice guideline, such as described in the AGREE instrument.⁶⁹ More sustainability evaluation research and methodological guidance is needed to make future sustainability research more robust and generalisable and may be helpful in creating a general sustainability language.

Conclusion

This systematic review identified, reported and analysed studies that evaluated the level of sustainability of professionals’ adherence to guideline recommendations in medical practice more than 1 year following the cessation of the implementation project.² Seven out of 18 evaluations showed sustained professionals’ adherence on average 2.6 years after implementation.¹ Owing to the limited number and the lack of methodological quality of the identified studies, no firm conclusion about the sustainability of professionals’ adherence to guideline recommendations in medical practice can be drawn. More sustainability evaluations, methodological sustainability studies and reviews are needed in order to develop a general framework for sustainability measurement and to facilitate uniform language and communication within the sustainability science.