Patterns of ‘leakage’ in the utilisation of clinical guidelines: a systematic review

Abstract

Background

Research evidence is insufficient to change physicians' behaviour. In 1996, Pathman developed a four step model: that physicians need to be aware of, agree with, adopt, and adhere to guidelines.

Objective

To review evidence in different settings on the patterns of ‘leakage’ in the utilisation of clinical guidelines using Pathman's awareness-to-adherence model.

Methods

A systematic review was conducted in June 2010. Primary studies were included if they reported on rates of awareness and agreement and adoption and/or adherence.

Results

11 primary studies were identified, reporting on 29 recommendations. Descriptive analyses of patterns and causes of leakage were tabulated and graphed. Leakage was progressive across all four steps. Median adherence from all recommendations was 34%, suggesting that potential benefits for patients from health research may be lost. There was considerable variation across different types of guidelines. Recommendations for drug interventions, vaccination and health promotion activities showed high rates of awareness. Leakage was most pronounced between adoption and adherence for drug recommendations and between awareness and agreement for medical management recommendations. Barriers were reported differentially for all steps of the model.

Conclusion

Leakage from research publication to guideline utilisation occurs in a wide variety of clinical settings and at all steps of the awareness-to-adherence pathway. This review confirms that clinical guidelines are insufficient to implement research and suggests there may be different factors influencing clinicians at each step of this pathway. Recommendations to improve guideline adherence need to be tailored to each step.

Introduction

For healthcare professionals to avoid actions that are harmful or fail to do actions that are beneficial, they should ideally always make decisions informed by high quality research evidence. However, the transfer of research evidence into practice is often complex and incomplete.¹ The availability of research evidence alone is usually insufficient to change physicians' practice.² While a wealth of literature exists to assist physicians to understand and evaluate research, comparatively little is known about the processes of change.³ Evidence based clinical practice guidelines are one important tool for helping physicians access the best research evidence efficiently. Well developed guidelines can help physicians and patients make better decisions, and reduce the risk of negligent care. Theoretically, they promote consistent and quality care that is cost effective for the healthcare system.⁴ However, for a variety of reasons, known and unknown, publication of clinical practice guidelines has had a limited effect in changing physician behaviour.² Cabana et al's work showed that simply knowing about guidelines is rarely sufficient to change physicians' behaviour.²

In 1996, Pathman et al developed a four step model (1. awareness→ 2. agreement→ 3. adoption→ 4. adherence) to look at the utilisation of clinical practice guideline recommendations on paediatric vaccine usage. The underlying conceptual framework suggested that to conform to guidelines, physicians must be aware of them, agree with them, decide to follow (adopt) them for some patients, and then actually follow (adhere to) them at appropriate times for all patients. The authors showed progressive drop off with a proportion dropping off at each step. This tendency has also been described as a ‘pipeline’, in which research evidence ‘leaks’ at various steps and reduces the extent to which research findings are implemented in practice.¹

We wanted to know whether the patterns observed by Pathman for paediatric vaccination recommendations are similar in other professions and for different types of clinical recommendation. We wanted to explore the patterns of ‘leakage’ and summarise any identified barriers to guideline implementation. Given the uncertainty about how this model had been applied in practice, we did not begin with any preconceived notions about how patterns might be similar or different from each other. We decided, therefore, to systematically review all studies that had explicitly or implicitly applied the Pathman awareness-to-adherence model to the utilisation of a clinical guideline.

Question addressed

What are the patterns of ‘leakage’ in the utilisation of clinical guidelines in practice?

In particular, can the rates of leakage be quantified and compared between steps, and are there identified causes for this leakage?

Methods

Design

A systematic review of all studies that look at the utilisation of clinical practice guidelines using the awareness-to-adherence model.

Included studies

• Studies that look at the utilisation of one or more clinical practice guideline recommendation/s, that measure awareness and agreement and either adoption or adherence (or both)

• Design: any primary survey or cross-sectional study

• Response rate: not specified as we wished to include internet surveys, and determining the denominator is not always possible

• Outcome measures: both objective and self reported

• Specialty or area: any area of healthcare (eg, surgery, physiotherapy, public health)

• Healthcare objective: any (eg, diagnosis, treatment, prevention, screening).

Excluded studies

• Studies that failed to measure awareness AND agreement

• Studies that failed to measure at least one measure of practice (ie, adoption or adherence).

Search strategy

Four electronic databases were searched (AMED 1985–2010, Embase 1996–2010, Medline 1996–2010, and PsychInfo 1987–2010), at 30 June 2010, using the text words aware* AND agree* AND (adopt* OR adhere*) combined with a MESH and free-text search to identify clinical practice or health planning guidelines (full strategy available on request). A forward citation review of the original 1996 article ‘The Awareness-to-Adherence Model of the Steps to Clinical Guideline Compliance’³ by Pathman was performed using Web of Science, Scopus and PubMed's related articles.

Selection of papers

Two authors independently reviewed identified citations for eligibility. Disagreements were resolved by consensus.

Assessment of quality

Quality was assessed using a pro forma against the following criteria:

• Study design

• Pilot or validation of measurement method

• Appropriateness of sampling frame and method

• Number of participants and completeness of data

• Response rates (where measurable)

• Representativeness of responders

• Method of measurement of outcome (subjective or objective)

• Consistency of reporting across abstract, tables and full text.

The awareness-to-adherence pathway considers whether target users agree with the guidelines. It does not consider whether the guidelines were in fact appropriate or evidence based. Therefore we did not assess the quality of the guidelines and the evidence supporting recommendations.

Data extraction

Two authors independently extracted data. Disagreements were resolved by consensus. The following data were extracted: study design, date of guideline, date of study, the quality criteria given above, target condition of guideline and recommendations, target population, sampling frame, rates of awareness, agreement, adoption and adherence, measures of uncertainty if reported (eg, p values, confidence intervals). We also tabulated separately any identified barriers to and recommendations for guideline implementation reported by the authors.

Analysis

We calculated the response rate for each study. From those responding we calculated the proportion who were aware, agreed, adopted, and adhered to each recommendation (‘absolute rates’). We also calculated ‘conditional rates’—that is, the proportion of those who were aware and who agreed, who agreed and adopted, and who adopted and adhered to the recommendation.

We grouped recommendations by different intervention types (eg, drug interventions, medical management, health promotion, screening, vaccination). These categories were not pre-defined as we did not know in advance what guideline topics we would find but were chosen post-hoc, to reflect clinically related and relevant themes.

Although our protocol suggested we would perform a meta-analysis, the heterogeneity of the studies precluded this.

Protocol

A protocol was agreed in advance and can be obtained from the authors. No amendments to protocol were made.

Results

Studies identified

We identified 427 citations. From reading abstracts alone, 410 articles were excluded for not measuring rates of awareness and agreement and one or both of adoption and adherence (figure 1). For 17 studies, the abstracts were unclear and they had to be read in full before they were excluded for the same reason. Eleven studies met the inclusion criteria. The participants, intervention, sample frame and size, measurement method, guideline publication, and timing of each study are summarised in table 1.

Figure 1.

Study selection diagram.

Table 1.

Study descriptions

Author	Condition	Topic of guideline	Participants, sample frame	Sample size	Outcome measure	Measurement validation	Publication of guideline	Time of study
Beaulieu et al, 20055	Medical treatment of stable angina	Prescription for aspirin Favour β blockers as first choice anti-angina drug Control LDL cholesterol	Random, stratified sample of 1/3 family physicians, all cardiology and internal medicine specialists in Quebec, Canada	1228	Mailed self-report survey	Based on Pathman's questionnaire	February 1999 Quebec College of Physicians mailed one page summary	May 1999
Cabana et al, 20076	Medical treatment of children with asthma	Prescribing inhaled corticosteroids for children with persistent asthma symptoms	National, random cross-sectional survey of paediatricians and family physicians, from American Medical Association (AMA) master file of US physicians	694	Mailed self report survey, two follow-up surveys, telephone call	Based on literature review and physician focus groups	2002 update National Heart, Lung & Blood Institute Asthma Guidelines	September–November 2004
Cruz-Correa et al, 20017	Screening for Barrett's oesophagus	Screening for Barrett's oesophagus in patients with 1. no dysplasia 2. low grade dysplasia	Stratified random sampling of members of three specialty organisations, excluding those in training programmes and retired specialists	235	Mailed self-report survey with three case scenarios, two follow-up surveys	Survey and case scenarios used at baseline and follow-up 19–22 months later	August 1998 guidelines developed by American College of Gastroenterology	October 1999–January 2000
Erhardt et al, 20088	Medical management of patients with chronic heart failure	Agreement with guidelines for pharmacotherapy	Random identification from research database of cardiologists in seven western European countries, according to specific inclusion criteria, country quotas	484	On-line survey including three case scenarios, translated	Validated and piloted in face-to-face interviews	2005 European Society of Cardiology guidelines for chronic heart failure	Not recorded
Freed et al, 19949	Immunisation of infants	Universal hepatitis B immunisation of infants	All paediatricians and random sample of family physicians in North Carolina, USA	591	Mailed self-report survey, two follow-up surveys	First survey repeated, used as pilot	November 1991 ACIP February 1992 AAP August 1992 AAFP	September–October 1992
Gentile et al, 200410	Well child visits to paediatricians	Limit children's media time Discourage TV viewing among children <2 years Encourage alternative entertainment for children	All members of the Minnesota chapter of American Academy of Paediatrics (AAP)	899	Mailed self report survey, three reminders	Survey designed by authors	February 2001 AAP media use guidelines	July–October 2002
Heneghan et al, 200711	Management of hypertension	Adjust blood pressure for home/self monitoring Measure blood pressure both arms Provide verbal lifestyle advice Provide written lifestyle advice Annual review of patients with mild hypertension Statin therapy for primary prevention	Survey of all UK general practitioners registered on http://www.doctors.net.uk, an online organisation for UK doctors	800	Internet survey, presented as a link in a targeted electronic bulletin	Adapted and piloted from Pathman's original questionnaire	2004 Guidelines from British Hypertension Society Guidelines	May 2006
Pathman et al, 19963	Paediatric vaccine recommendations	Second dose of measles vaccine Haemophilus influenza type B (HIB) at 2 months Universal hepatitis B immunisation of all infants Acellular vaccine for 4th and 5th pertussis dose	Random selection from AMA's master file of family physicians and paediatricians in nine US locations, stratified for community size	2146	Mailed self-report survey, two follow-up p surveys	Survey instrument pilot tested with 100 paediatricians and family physicians	1988–2nd dose of measles 1990–HIB at 2 months 1992-universal Hep B 1992–acellular pertussis vaccine 4,5 dose	October 1993–January 1994
Pitimana-Aree et al, 200712	Anaesthetic practice	Conscious sedation Provision of anaesthesia Pre-anaesthetic evaluation Labour analgesia Emergency endotracheal intubation Spinal anaesthesia	National survey, based on college list of anaesthesiologists (600) and nurse anaesthetists (1300) in Thailand	1900	Mailed self-report survey	Not described	2006—Royal College of Anaesthesiologists of Thailand proposed and revised six guidelines	Not recorded
Weyman et al, 199513	Screening for Chlamydia trachomatis infection	Selective screening of asymptomatic women	All registered staff physicians and residents working in six hospital family practice teaching units in downtown Toronto	153	Mailed self-report survey with case scenarios, one reminder	Literature search guided development of questionnaire	November 1989 Canadian guidelines mailed to all practising physicians in Canada	December 1990
Zimmerman et al, 200414	Influenza vaccination for elderly adults	Recommend vaccination of elderly patients with asthma	Stratified random sample of primary care clinicians from Veterans Affairs (VA) practices in USA Random sample of approx 22 patients of each clinician, minimum 15 completed patient interviews per clinician	71 clinicians 925 patients aged 65 or over	Physicians: personal interview+self report survey Patients: telephone questionnaire+self report of 1999–2000 influenza vaccination	Based on Precede—Proceed model, Awareness-to-adherence model included as predisposing factors Piloting occurred	2000—CDC Guidelines for vaccinating elderly adults	August 1999–February 2000

Study characteristics

All studies were surveys, using a self-reported questionnaire. Eight surveys were distributed by mail, two by internet,^{8 11} and one was given to participants after a personal interview. Only two studies explicitly acknowledged using Pathman's original questionnaire.^{5 11} Only one survey compared physician reports with other data: namely patients' reports of their influenza vaccination within one calendar year.¹⁴

Sampling strategies were sometimes not well described. Most commonly, random samples were obtained from specialist or general practitioners within specific geographical regions or specialty organisations. Sample sizes of eligible participants ranged from 153–2146 with an average of 920 participants. Reported response rates varied from 33–97%.

While all studies reported the date of the publication of the guideline, only nine reported the timing of their study. Three studies surveyed physicians <1 year after guideline publication.^{5 9 14} Most surveyed practice between 1–2 years after publication.

The quality of studies was moderate to poor, though it was often difficult to distinguish whether studies were poorly conducted or simply poorly reported. A number of studies presented inconsistencies between tabulated and text figures, abstract and content, and absolute and calculated values. To maintain consistency of reporting, absolute response rates for awareness, agreement, adoption, and adherence were recalculated from raw data, where possible. Any inconsistencies within the reporting of studies did not substantively alter our conclusions. Conditional response rates were also calculated to measure the change per step: to capture the percentage of agreement, given awareness; adoption, given awareness and agreement; and adherence, given awareness, agreement and adoption. All conditional rates are included in table 2.

Table 2.

Absolute responder rates and conditional leakage rates for each recommendation

Study, date of publication	Survey response rate (%)	Recommendation and condition	Absolute responder rates and leakage rates (%)
			Aware		Agree		Adopt		Adhere
			Absolute responders	Conditional leakage	Absolute responders	Conditional leakage	Absolute responders	Conditional leakage	Absolute responders	Conditional leakage
Beaulieu et al, 20055	71.4	Aspirin for stable angina	96.6	3.4	96.1	0.5	86.1	10.0	84.1	2.0
		β blockers for stable angina	92.7	7.3	90.3	2.4	67.7	22.6	65.8	1.9
		Control LDL cholesterol	89.7	10.3	82.4	7.3	44.7	37.7	42.7	2.0
Cabana et al, 20076	49.4	Inhaled corticosteroids for children with asthma	91.8	8.2	87.1	4.7			53.9
Cruz-Correa et al, 20017	65.5	Screening for Barrett's oesophagus with no dysplasia	55.2	44.8	81.8	−26.6			76.0
		Screening for Barrett's oesophagus with low grade dysplasia	55.2	44.8	54.5	0.7			37.7
Erhardt et al, 20088	96.5	Pharmacotherapy for chronic heart failure	98.0	2.0	90.0	8.0	78.0	12.0	25.0	53.0
Freed et al, 19949	80.9	Universal Hep B infants	100	0	50.6	49.4	40.6	10.0
Gentile et al, 200410	40.6	Limit children's media time	92.0	8.0	87.0	5.0			52.0
		Discourage TV viewing among young children	91.0	9.0	71.0	20.0			33.0
		Encourage alternative entertainment	92.0	8.0	97.0	−5.0			76.0
Heneghan et al, 200711	50.1%	Home/self monitoring adjustment hypertension	52.1	47.9	21.4	30.7	16.7	4.7
		Measure blood pressure in both arms	76.1	23.9	31.9	44.2	22.4	9.5	16.7	5.7
		Verbal advice lifestyle re hypertension	98.0	2.0	91.5	6.5	94.3	−2.8
		Written lifestyle advice re hypertension	89.0	11.0	70.6	18.4	61.1	9.5	34.0	27.1
		Annual review mild hypertension	92.8	7.2	77.6	15.2	84.8	−7.2	61.0	23.8
		Statin therapy for hypertension	98.8	1.2	74.3	24.5	81.5	−7.2	43.0	38.5
Pathman et al, 19963	66.2	2nd dose measles vaccine	99.3	0.7	94.8	4.5	93.7	1.1
		Haemophilus influenzae type B at 2 months	98.3	1.7	96.6	1.7	97.4	−0.8
		Universal Hep B infants	98.4	1.6	70.3	28.1	77.7	−7.4	30.1	47.6
		Acellular pertussis vaccine 4,5 dose	89.8	10.2	66.5	23.3	46.3	20.2	35.2	11.1
Pitimana-Aree et al, 200712	33.4	Conscious anaesthetic sedation	46.0	54.0	28.2	17.8			27.8
		Provision of anaesthesia	46.0	54.0	32.3	13.7			30.9
		Labour analgesia	46.0	54.0	24.3	21.7			22.7
		Pre-anaesthetic evaluation	46.0	54.0	30.3	15.7			30.1
		Emergency intubation	46.0	54.0	30	16.0			30.1
		Spinal anaesthesia	46.0	54.0	46.2	−0.2			45.9
Weyman et al, 199513	77.1	Selective screening for chlamydia	28.0	72.0	12.7	15.3	11.0	1.7
Zimmerman et al, 200414	85.9	Influenza vaccination for elderly	81.7	18.3	83.3	−1.6			76.7

Participants and recommendations

Across the 11 studies there were 29 recommendations that reported at least three of the four steps of the model. Only 12 recommendations reported data at all four steps, 12 did not report adoption, and five did not report adherence. The topic of recommendations included advice regarding: drug interventions (six topics—stable angina,⁵ chronic heart failure (CHF)⁸ and hypertension (HT),¹¹ and prescriptions for children with asthma.⁶), medical management (nine topics—three on hypertension,¹¹ six on anaesthetic practice guidelines in Thailand¹²), vaccination schedules (two studies of vaccinations given by American paediatricians and family physicians for infants,⁹ children³ and elderly adults¹⁴), screening tests (American gastroenterologists for surveillance at different stages of Barrett's oesophagus⁷; and Canadian staff physicians and residents for screening for chlamydia.¹³) and health promotion (two studies reviewed five health promotion recommendations: three for American paediatricians for advising parents on media use¹⁰; and two for general practitioners in England for adults with hypertension¹¹).

Comparison of rates at each step

There was considerable variation in the pattern at different steps and for different recommendations (table 2). The leakage between steps is not simply a function of difference in specialties or sampling frames, but varied between recommendations from the same guideline within the same population of responders. This variation prompted a deeper level of comparison of patterns across different purposes for using clinical guidelines. We made visual comparisons across a range of different purposes. From these comparisons some interesting observations emerged.

Different patterns of leakage were apparent when recommendations were grouped according to their purpose of drug prescription, medical management, vaccination, screening, and health promotion activities. For the six drug interventions there are consistently high rates of awareness and agreement across recommendations (figure 2). However, adherence rates vary almost fourfold: from just above 20% to over 80%. Adoption and adherence rates are generally progressively lower, except for the recommendation of statin treatment for hypertension, where surprisingly physicians adopt the guideline slightly more often than they agree with it (the authors note that there are specific financial incentives for complying with this particular recommendation¹¹).

Figure 2.

Absolute responder rates for drug recommendations. LDL, low density lipoprotein.

For the nine medical management recommendations, awareness rates are lower and more variable than for drugs. Figure 3 documents six key recommendations; three similar patterns within the same study have been removed for ease of reading. The leakage between aware and agree is also more pronounced. There is less variation in rates of adoption and adherence which seem to reflect agreement. The recommendation for annual review of mild hypertension increases between agreement and adoption and again the authors note that there are specific financial incentives for compliance with this recommendation.¹¹

Figure 3.

Absolute responder rates for medical management recommendations.

The six vaccination recommendations demonstrate a different pattern (figure 4); there is high awareness of all recommendations, but agreement rates vary almost twofold. The step to adoption is highly variable. The authors of the study in which adoption was higher than agreement reported that adoption and adherence were more likely when physicians believed that parents know about and requested the vaccine and when physicians belong to specialist societies where peer expectations were likely to be greater.³ All three studies that report adherence show that there is leakage after the adoption stage.

Figure 4.

Absolute responder rates for vaccination recommendations.

For the five health promotion recommendations awareness is high (figure 5). Agreement was also relatively high, with the lowest rate being 70%. There is steady leakage across all four steps. The screening recommendations demonstrate most variability across the pathway (figure 6), with awareness of the chlamydia screening guidelines being the lowest across the 29 recommendations. It was noted that reasons for agreement and disagreement with screening guidelines for Barrett's oesophagus changed according to the severity of disease, perceived risk of cancer, and medico-legal liability. This could be in part explained by differences in interpretation of the literature, which does not include definitive evidence comparing surveillance intervals across different grades of dysplasia.⁷

Figure 5.

Absolute responder rates for health promotion recommendations.

Figure 6.

Absolute responder rates for screening recommendations.

Comparison of rates across all studies

Given the heterogeneity of the guidelines and their recommendations, it is difficult to combine results into a single summary statistic; therefore, we reviewed the median and IQRs. The median awareness of guidelines was high at 90%. The absolute rates (figure 7) show a steady fall in the median from aware to agree and from adopt to adhere, with a plateau between agree and adopt. The median proportion of respondents who report adhering to guideline recommendations is 36% (IQR 30–56%). In practice, the adherence to recommendations is probably even lower given that most adherence was self-reported. We also examined the median percentage reduction in the conditional rates as a proxy for the rate of leakage. The median rates of leakage—16% agree given that responders were aware, 13% adopt given agreement, and 17% adhere given adoption—are of all the same order, and suggest similar proportional leakage at each step.

Figure 7.

Box and whisker plot for all absolute rates.

Identification of barriers for guideline implementation

From the 11 included studies, different rates of awareness, agreement, adoption, and adherence have been reported within the same clinical population and group of clinicians, suggesting that differences depended on individual clinicians' training and experience and the clinical action required.¹¹ For example, specialists are more likely to adopt and adhere than their generalist peers, whereas clinicians working in solo or two physician practices were less likely to adopt guidelines than those in large practices and hospitals.^{3 6 9 12 13}

The credibility of a guideline's authors, its scientific rationale, and the method of dissemination impacted agreement and adherence. Physicians had more confidence in guidelines developed by their own specialty organisation.⁶ European cardiologists preferred European and national guidelines over American ones.⁸ Physician agreement was enhanced when information was shared among colleagues and adherence was enhanced when the Centres for Disease Control and Prevention (CDC) provided guideline information.³ Even when global agreement for a guideline was high, practical considerations and the perception of relative advantage for groups of patients⁵ impacted on adherence. Guidelines were seen as impractical in hospitals where there was limited equipment and staff.¹²

Methods of dissemination varied and influenced uptake. In Thailand, announcements of guidelines in the annual general meeting of the Royal College of Anaesthesiologists and in the monthly newsletter were considered more effective than formal documents sent to hospital directors, and announcements on websites and in journals.¹² In contrast, European cardiologists reported gaining knowledge of guidelines from congresses and medical journals.⁸ Most family physicians and paediatricians preferred receiving guideline information from their own specialty society.⁹

Finally, patients could influence clinicians' behaviour; those who believed that parents knew about and requested a vaccine were more likely to adopt the guideline.³ Adoption of influenza vaccination was also higher when patients planned to be vaccinated.¹⁴

Discussion

This systematic review—which included 29 guideline recommendations from 11 identified studies—shows an average 15% ‘leakage’ between each step in the Pathman awareness-to-adherence model. The heterogeneity of setting in these studies suggests that the Pathman model can be usefully applied to guidelines addressing very different healthcare objectives (eg, screening, anaesthetic management, drug prescribing), across a range of different clinician groups, and in different countries and healthcare systems. This pattern supports the conceptual model of a ‘pipeline’ from guideline publication to utilisation, with progressive leakage across stages which leads to a failure to get research findings into practice.¹ The cumulative leakage is substantial and suggests that guidelines may not be being adhered to about two-thirds of the time—a significant loss in potential health gain to patients.

In most cases awareness precedes agreement, as the model suggests. However, sometimes there is agreement with guideline recommendations even though the responders are not aware of the guidelines. This could occur with poor dissemination of guidelines, combined with either the existence of widespread consensus within the profession about appropriate practice, or limited research evidence supporting the recommendation, or both.⁷ For some recommendations there is a large drop between awareness and agreement, suggesting that failure to implement may due to active disagreement.

Following on with the pipeline metaphor, agreement usually precedes adoption. However, some recommendations have higher adoption than agreement rates, suggesting that physicians may adopt recommendations with which they do not agree. This pattern was noted by Pathman et al, who reported that 11% of physicians adopted a recommendation to provide hepatitis B vaccination without agreeing with it. They suggested this may be due to peer pressure, malpractice fears, patient demand, community norms, and practice organisation policies.³ Other external factors that may play a part include target setting, policy directives, and financial incentives. This suggests that winning hearts and minds of clinicians through the dissemination of evidence is not the only strategy to increase the adoption of research evidence into practice. Financial incentives and penalties may be levers for change, but may also invoke distrust and professional demoralisation.

Despite the small overall rate of adherence, rates of awareness and agreement are high for many recommendations, especially those for drugs and vaccinations. The higher agreement rates for drug treatments and vaccinations compared to medical management or health promotion may be a function of the strength of the evidence underlying the recommendations. Clearly, moving from awareness through agreement to adopt and ultimately adhere to guidelines is a complex process.

To understand better the causes for leakage along this pipeline of guideline implementation, we summarised factors that were reported as barriers, for each included study. We used the comprehensive systematic review conducted by Cabana et al which identified 293 potential barriers to guideline adherence to conceptually organise these factors.² We identified those barriers which influenced clinicians' beliefs of outcome expectancy and their perception of the inertia of current practice. External barriers were investigated as to whether they were related to the guidelines themselves, patient preferences, or environmental obstacles. We then reviewed all identified obstacles and attributed their contribution to one of the four steps of the awareness-to-adherence model. Table 3 provides a more generic summary of the identified barriers and recommendations to promote awareness, agreement, adoption, and adherence.

Table 3.

Barriers and recommendations influencing awareness, agreement, adoption, and adherence to clinical practice guidelines

Step of model	Identified barriers	Recommendations
Awareness being aware of clinical guidelines	Inertia of current practice—reluctant to change practice Lack of specialty society membership Lack of experience with patient group Lack of knowledge about alternatives Limited distribution of guidelines	Create clear and consistent guidelines via: liaison between clinical specialists local development and implementation pilot testing and monitoring Target dissemination of guidelines: to less experienced clinicians to clinicians working in small centres using specialists to influence decision making Utilise research evidence to clarify: expected outcomes—positive and adverse key points of decision making suitable patient population
Agreement agreeing with clinical guidelines	Genuine or perceived ambiguity in the underlying evidence Insufficient information to make a decision Personal beliefs about utility of recommendation Differential beliefs about applicability of guideline to the patient or population Disagreement with specific recommendations—they will not lead to desired outcome Confusing and complex recommendations
Adoption following clinical guidelines for some patients	Genuine or perceived ambiguity regarding side effects, contraindications and risks Belief that it is difficult to change personal habits Perception of inconsistency of recommendations with patient values and preferences Anticipated practical difficulties	Provide patients with information about: their condition recommended treatment side effects, contraindications, risks Manage clinical environments to: develop special purpose clinics include patient reminder/prompt systems display patient education information publicly monitor key outcomes promote time and cost efficient practices
Adherence following clinical guidelines for all appropriate patients	High costs for patients and/or practice Patient knowledge, expectations and compliance Patient motivation and support for recommendation Lack of time, materials, logistical support Issues of medico-legal liability High proportions of uninsured patients within a practice

From this analysis, recommendations can be inferred. By ensuring that guidelines are consistently and clearly understood across appropriate clinical groups, clinician awareness and agreement may be improved. Useful strategies could include: liaison across specialties regarding the content and timing of recommendations; promoting local development, pilot testing, implementation and monitoring of guidelines; targeting dissemination to clinicians who have less experience and who are less connected with their peers; and using specialist clinicians to influence the guideline compliance of their peers.

Through better recognition of patient needs and values, and strategies to manage environmental obstacles, clinician adoption and adherence to guidelines may be improved. It is important to consider providing patients with concise information about their condition and its recommended treatment. Clinicians also need short and relevant summaries within guidelines for possible side effects, contraindications and risks, to ensure they and their patients are appropriately informed. Environmental barriers can be identified and reduced with sufficient planning, organisation and piloting. Some practical strategies may include: developing special purpose clinics; using patient reminders or prompts; having accessible and relevant patient education posters; and using publicly visible monitoring tools such as the calculated practice wide immunisation rates. Therefore, it is apparent that to enhance guideline implementation, a better understanding of the leakage and its potential cause is required at each step of the awareness-to-adherence model.

This systematic review is limited by the validity of the included primary studies, which have a number of weaknesses in their design and reporting. All studies except one used self reporting of awareness, agreement, adoption, and adherence. We know that for patients, self reporting tends to overestimate compliance.¹⁵ Physicians are likely to be prone to a similar bias and are likely to overestimate their own adherence.^{6 13} Inconsistencies were noted between physicians' self reports and their performance on specific case studies, and confirms that self-reports overestimate.¹³ The likely overestimation of adherence rates is compounded when one considers that non-responders to these surveys probably have lower rates than responders.

This systematic review suggests that the awareness-to-adherence model is a useful theoretical model to inform clinical guideline development and evaluation. In the first instance it would be useful to ascertain if the patterns and causes of leakage identified across 11 published studies are meaningful beyond these studies. Simple audit and evaluations of guideline implementation could be made more systematic if this model was applied. For future research, it is important to design quality studies that measure and report all steps of awareness, agreement, adoption, and adherence using self-report and other objective measures. Further, it should not be assumed that barriers and facilitators influence all four steps equally and deeper investigation of change at each step may be enlightening.

Finally, it is valuable to briefly reflect on how this pipeline of leakage may impact on healthcare policy and practice. In the UK, the recent white paper Equity and Excellence: Liberating the NHS,¹⁶ proposes a move from targets to measurable evidence based outcomes. Relevant recommendations embedded in evidence based clinical guidelines may be able to identify appropriate outcomes that are meaningful and motivating to both clinicians and their patients. The Quality and Outcomes Framework contract for general practices in the UK provides a strategy for measuring and reporting clinical indicators in general practice. If these indicators are derived from research evidence and included in appropriate guidelines and clinical practice, they may serve to inform patients and enhance clinicians' rates of adoption and adherence. While financial incentives can be effective in increasing the rate of adoption beyond that expected from awareness and agreement alone, they have substantial costs and other downsides. There is a need to validate and evaluate suggested strategies to improve the implementation of guidelines in clinical practice.

Conclusion

Leakage from guideline publication to utilisation occurs among different specialties, across a range of recommendations, in different countries and healthcare systems, and at all steps of the awareness-to-adherence pathway. Leakage generally increases proportionally along the pipeline from awareness, through agreement, adoption, and adherence and we conclude that guideline recommendations may not be adhered to, up to two-thirds of the time. We identified different patterns of leakage across varying purposes of recommendations. Drug interventions showed high rates of awareness and agreement, but had varied and pronounced leakage to adoption and adherence. Recommendations for vaccination and health promotion activities also showed high rates of awareness, but leakage was more progressive between agreement, adoption, and adherence. For medical management recommendations, awareness rates were lower and more variable, and leakage was greatest between awareness and agreement. We investigated barriers for guideline implementation at each step of the awareness-to-adherence pathway and made differential recommendations for clinical practice and future research. This systematic review confirms that the production and dissemination of evidence based clinical guidelines is not sufficient to ensure that research evidence gets into practice. This is a complex process and requires deeper investigation at each step of clinician awareness, agreement, adoption, and adherence to guidelines.

Main messages.

• Research evidence progressively leaks out at each step of clinician awareness, agreement, adoption, and adherence to guidelines.

• It is suggested that only one third of the research evidence informing guidelines is being routinely adhered to.

• Rates and patterns of leakage vary across different types of clinical guidelines.

• Barriers to guideline implementation vary for each step of the awareness-to-adherence pathway.

• Recommendations to improve guideline adherence also need to consider strategies for improving clinicians' awareness, agreement, and adoption of guidelines.

Current research questions and suggested strategies.

• What causes the leakage of research evidence in implementing clinical guidelines?

  • conduct a case study, audit of a service's pattern of leakage of research evidence across a specific clinical guideline/s using the four steps of the awareness-to-adherence pathway

• How is research utilised in clinical practice?

  • use questionnaires/focus groups/interviews to evaluate barriers for guideline implementation between the steps of clinician awareness, agreement, adoption, and adherence

• How do self-report measures of awareness, agreement, adoption, and adherence compare with objective measures?

  • compare questionnaire responses with actual prescribing or vaccination rates