Multifaceted implementation of stroke prevention guidelines in primary care: cluster‐randomised evaluation of clinical and cost effectiveness

Abstract

Objective

To evaluate clinical and cost effectiveness of implementing evidence‐based guidelines for the prevention of stroke.

Design

Cluster‐randomised trial

Setting

Three primary care organisations in the North of England covering a population of 400 000.

Participants

Seventy six primary care teams in four clusters: North, South & West, City I and City II.

Intervention

Guidelines for the management of patients with atrial fibrillation and transient ischaemic attack (TIA) were developed and implemented using a multifaceted approach including evidence‐based recommendations, audit and feedback, interactive educational sessions, patient prompts and outreach visits.

Outcomes

Identification and appropriate treatment of patients with atrial fibrillation or TIA, and cost effectiveness.

Results

Implementation led to 36% increase (95% CI 4% to 78%) in diagnosis of atrial fibrillation, and improved treatment of TIA (odds ratio of complying with guidelines 1.8; 95% CI 1.1 to 2.8). Combined analysis of atrial fibrillation and TIA estimates that compliance was significantly greater (OR 1.46 95% CI 1.10 to 1.94) in the condition for which practices had received the implementation programme. The development and implementation of guidelines cost less than £1500 per practice. The estimated costs per quality‐adjusted life year gained by patients with atrial fibrillation or TIA were both less than £2000, very much less than the usual criterion for cost effectiveness.

Conclusions

Implementation of evidence‐based guidelines improved the quality of primary care for atrial fibrillation and TIA. The intervention was feasible and very cost effective. Key components of the model include contextual analysis, strong professional support, clear recommendations based on robust evidence, simplicity of adoption, good communication and use of established networks and opinion leaders.

Clinical guidelines have become an established part of routine medical practice. Both the National Health Service and professional groups have adopted them as a method of summarising complex and rapidly changing research evidence with the aim of speeding up the translation of evidence into practice and reducing variations in the quality of care.

Many methods have been developed to implement guidelines in everyday practice and many studies have been undertaken to evaluate their effectiveness.¹,²,³,⁴,⁵,⁶ Multifaceted approaches to changing professional practice appear to be the most effective though considerable uncertainty exists about the nature and reproducibility of what has worked in specific settings.¹ Few studies have examined the affordability of implementation programmes or the influence of different contexts on the effectiveness of interventions.

We developed evidence‐based guidelines for the management and prevention of stroke and implemented them using a tailored, multifaceted approach. Stroke is a common condition with a high personal and societal burden. Transient ischaemic attacks (TIA) offer an important opportunity for secondary prevention of strokes and atrial fibrillation is a key modifiable risk factor for stroke. We evaluated the effectiveness and cost effectiveness of an implementation strategy for the two guidelines on management of patients with these conditions across diverse primary care settings within an urban district.

Methods

Setting

We studied three primary care trusts (PCTs) in Bradford, a city in the north of England. Two of the PCTs (North and South & West Bradford, with populations of 98 000 and 147 000 respectively) consist mainly of large general practice partnerships (13 and 24 practices respectively) with a history of “fundholding” and “total fundholding”. Their populations are demographically and socioeconomically similar. In the third (Bradford City, population 147 000) 36 of 40 practices are single‐handed, and the population is more culturally diverse and materially deprived, with fewer elderly patients.

Design

Initially we chose two different designs to evaluate the implementation of the two guidelines in these different organisational contexts (fig 1). First, we undertook cluster randomisation of the two homogeneous primary care trusts (Bradford North and Bradford South & West PCTs) between TIA guidelines and atrial fibrillation guidelines. Based on the advice from general practioner leads (MW, JB), who were responsible for PCT policies on the development and dissemination of clinical guidelines, we judged that other designs risked substantial contamination between practices.

Figure 1 Flow diagram showing randomisation of primary care trusts and practice groups plus numbers of patient records included in audit of outcomes. CONSORT statement for the trial is available as supplementary material on the QSHC website http://qshc.bmj.com/supplemental

Second, because Bradford City PCT provides a different primary care context that is more fragmented and heterogeneous, we chose the practice as the unit of randomisation. We had planned to randomise half its practices to each guideline. Early in the study, however, it became clear that such randomisation was not feasible. Twenty practices had established a joint practice development programme on Wednesdays and the other 20 on Thursdays, and there would be logistical difficulties and contamination between practices if we altered this configuration. As the two groups were comparable in practice size and socioeconomic characteristics, the project steering group decided to randomise these two natural clusters between TIA guidelines and atrial fibrillation guidelines. Hence only two random allocations were necessary—between the two suburban PCTs and between the two central practice development groups. We therefore undertook these in public at the appropriate meeting of the project steering group using numbered balls concealed in an opaque bag.

In adopting a cluster‐randomised design, we could not estimate intraclass correlation coefficients (ICCs) in Bradford. Instead we relied on a previous study of clinical guidelines within practices within PCT‐sized areas in the North of England, in which ICCs did not exceed 0.1 for process measures like those we adopted for atrial fibrillation and TIA.⁷,⁸ To be conservative we assumed that clustering would halve the effective sample size. So we aimed to recruit a total of 400 patients to detect a change in adherence from 50% to 70% (or 30%) with 80% power and 5% significance level. In the event we recruited far more than 400.

Planning interventions

Change can be a complex and drawn‐out process that depends on a variety of contextual factors.²,⁹ Local ownership of clinical guidelines and involvement of relevant stakeholders were seen as key steps.²,¹⁰,¹¹ While developing clinical guidelines we undertook a contextual analysis based on interviews with patients and professionals in each PCT. This analysis provided an opportunity to gain local commitment, build coalitions and tailor the intervention to the local context by identifying gaps in current services, barriers to good practice and educational needs. This preliminary phase highlighted particular challenges in obtaining commitment and implementing the guidelines in City PCT. The large number of single‐handed practices and historical underdevelopment of training, audit and clinical governance systems suggested difficulties in engagement and the need for more active efforts in implementation. So we adopted a phased strategy of developing, disseminating and reinforcing interventions.

Developing interventions

The clinical evidence base and aspects of cost effectiveness were reviewed by a multiprofessional group who met on five occasions to develop evidence‐based guidelines for management of patients with atrial fibrillation and TIA.¹²,¹³,¹⁴,¹⁵,¹⁶ These meetings of stakeholders such as clinical governance leads, managers and influential professionals from primary and secondary care, were convened to:

• obtain ownership and commitment to improving quality of care using guidelines;

• adapt nationally recommended evidence‐based guidelines into local summary guidelines;

• identify barriers to and incentives for changing practice; and

• agree appropriate implementation strategies.

The consensus group was able to adapt national guidance to specific care pathways for local service delivery (for example referral to TIA clinics) and address gaps in the guidance (for example the risk stratification of patients with atrial fibrillation). One difficulty, of particular relevance to stroke, is that national guidance can be regarded as specialist but often the service gaps can be with non‐specialist services. Hence there is a need for local interpretation to help non‐specialist staff participate in reconfigured care pathways. The draft guidelines were reviewed in two meetings held in January 2003 with a broad range of stakeholders including staff, service users and carers. Two patient focus groups were run in March 2003.

Baseline data to audit referrals to the TIA clinic and the numbers and treatment of patients with atrial fibrillation in each practice were collected by practice from 2001.

Disseminating interventions

A stepwise approach to disseminating the guidelines was taken in order to achieve the greatest coverage of health professionals from the limited resources available.

Step 1: Education meetings

We held a “protected learning time” event in each PCT for doctors, nurses and practice managers working in secondary care and across the interface of primary and secondary care. An interactive approach was taken with small group discussions, problem‐based learning, case histories and worked examples. Local opinion leaders were identified by each PCT and were given specific training about leading the meetings.

The aims of the education strategy were to:

• use existing education systems and events to promote the guidelines;

• establish local, smaller “interactive” discussion groups adopting a variety of teaching methods, rather than larger didactic sessions;

• share local audit information and the expertise of local speakers; and

• provide events that were accessible and accredited, and met the needs of the audience.

Step 2: Educational outreach visits

Following the education meetings, each practice was contacted by the relevant clinical governance lead and offered an outreach visit. Thirty practices accepted the offer: nine “Wednesday practices” in City (City I) PCT; eight “Thursday practices” in City (City II) PCT; three practices in North Bradford PCT; and 10 in South & West PCT. These were visited by the project leader who presented key messages from the relevant guideline and discussed the practice's implementation strategy during a one‐hour session. An information pack containing a range of materials to support implementation was given out at each visit, including copies of the guideline, information leaflets, and electronic and paper copies of referral forms.

Step 3: Postal dissemination

A copy of the information pack was posted to any practice that did not request a visit.

Reinforcing interventions

The third phase of interventions were designed to reinforce development and dissemination:

• The guidelines were designed, piloted and presented in a clear and concise format to improve readability.

• For South & West PCT we were able to link the audit results to quality targets in contract agreements.

• Guideline reminders for clinicians included laminated posters, desktop coasters and electronic referral templates.

• Our marketing strategy included a tailored communication strategy to target health professionals through a variety of channels including internet and local intranets and meetings of key stakeholders

• Local service users and carers contributed to developing patient information, as well as reviewing the guidelines themselves.

Details of supporting materials are at http://www.learnonline.nhs.uk/pace/Guidance+Documents.

Outcome measures

The primary outcomes for TIA were: the age‐sex standardised referral rates to the rapid access TIA clinic; and guideline compliance measured by proportion of patients “treated” with an anti‐platelet drug (already prescribed, prescribed after the ischaemic event or contra‐indicated) and provided with driving advice by the time they attended the TIA clinic; the data on driving advice were available only after intervention.

The primary outcomes for atrial fibrillation were: the age‐sex standardised diagnosis rate for patients with atrial fibrillation; and guideline compliance measured by proportion of patients treated with aspirin or warfarin or recorded as having a contra‐indication. Audit of practice records 21 months after the training workshop used Read codes and repeat prescription registers to record how many patients in each practice had been diagnosed with atrial fibrillation, when atrial fibrillation was first diagnosed, when warfarin or aspirin was prescribed or contra‐indication recorded, and the presence of other risk factors.

Statistical analysis

There were large age differences between the PCTs, and both TIA and atrial fibrillation are much more common in the elderly. Hence incidence estimates were standardised for age and sex, using practice‐specific data from June 2003 and a standard population comprising all participating practices. Furthermore age and sex were treated as covariates in other analyses.

The PCT training workshops were defined as the start of the intervention. Moving average smoothing was used to estimate monthly standardised atrial fibrillation incidence rates and TIA clinic attendance rates for the two non‐City PCTs and the two clusters of practices within City PCT, and to compare pre‐intervention and post‐intervention rates between these clusters.

Though a previous paper reported results from the TIA audit,¹⁷ the main analysis of guideline compliance combined TIA and atrial fibrillation audit results. This strategy exploits the factorial‐like trial design in which every practice was trained in one of the two conditions and provided control data on the other condition. However pre‐intervention compliance was not recorded for TIA patients, and some practices refused an atrial fibrillation audit. Thus comparisons were restricted to patients diagnosed after the intervention in practices which carried out an atrial fibrillation audit. We fitted a logistic regression model to these data, with compliance as dependent variable, and age, sex, type of training (TIA or atrial fibrillation) and area (City or non‐City) as potential predictors.

To reflect the separate TIA analysis,¹⁷ we conducted two separate atrial fibrillation analyses. The first used only patients diagnosed but not yet appropriately treated by the start of the intervention, and compared the proportions in trained and untrained practices whose treatment became appropriate by the end of the follow‐up period. The second used controlled before‐and‐after analysis to compare compliance for newly diagnosed patients in the 18 months before intervention and the 18 months after intervention between trained and untrained practices. We used the statistical packages SPSS and Excel for data preparation, analysis (including confidence intervals for proportions and odds ratios) and presentation (including graphs). We specified all these statistical analyses, but no others, in a plan agreed before analysis began.

Economic evaluation

The high cost of stroke care makes cost effectiveness an important issue when evaluating an intervention aimed at preventing stroke. We estimated the resources consumed in developing and implementing local guidelines for TIA and atrial fibrillation under two headings:

Direct costs comprised staff time (mainly the project coordinator), reimbursements (for example, users' expenses in attending focus groups) and consumables used (for example, the costs of printing and circulating promotional materials). We used actual expenditure to value these costs.

Time costs comprised inputs from general practitioners and other clinical staff to developing and implementing the guidelines, mainly in stakeholder engagement, pilot groups, consensus groups, focus groups and educational events. We recorded the frequency and duration of these events, and valued the resources consumed by the “opportunity cost” of the time of each person attending. The cost of each of the main health professions was derived from the published Unit Costs of Health and Social Care.¹⁸ Other staff costs were imputed from the mid‐points of salary scales plus employers' costs for each type of staff.

Results

Table 1 shows the population and sample sizes for the two audits. Figure 2 shows changes in estimated standardised monthly atrial fibrillation diagnosis rates from January 2000 to the audit cut‐off date of December 2003; the first subtable of table 2 summarises these changes. Because these estimates are based on a single audit at the end of follow‐up, they do not include patients who died or moved practices after diagnosis. Even so the increase was significantly greater in trained than in untrained practices—by 36% (95% CI 4% to 78%). In the North PCT, however, the rise started a few months before implementation began (fig 2).

Table 1 Populations and samples.

	City I: AF training	City II: TIA training	North: AF training	South & West: TIA training	All: AF training	All: TIA training
Populations
TIA audit
Number of practices	20	20	13	24	33	44
Total population	74100	66700	92900	149400	167000	216100
Proportion aged >70	0.058	0.078	0.118	0.107	0.091	0.098
AF audit
Practices audited	17	11	12	23	29	34
Population	66600	41400	90500	147800	157200	189200
Proportion aged >70	0.060	0.077	0.112	0.108	0.090	0.101
Samples
TIA audit
Recorded contacts	61	114	317	414	378	528
Referrals to clinic	53	94	283	360	336	454
AF audit
Records examined	293	245	580	872	873	1117
Sampling fraction (range across practices)	1.0	1.0	0.15–1.0	0.12–1.0
AF diagnoses found	213	245	425	668	638	855
Estimated number AF	213	245	1049	1056	1262	1301

AF, atrial fibrillation; TIA, transient ichaemic attack.

Figure 2 Trends in atrial fibrillation diagnosis: standardised moving‐average monthly rates per million for each PCT cluster (intervention at 28 months). Sample sizes in table 2.

Table 2 Atrial fibrillation audit: incidence of diagnosed atrial fibrillation (January 2000 to December 2003) and guideline compliance.

Observed number (standardised rate per million per month)	City I: AF training		City II: TIA training (control)		North: AF training		South & West: TIA training (control)		Pooled significance	Combined odds ratio estimate (95% CI)
Diagnosis of AF
Before intervention (27 months)	62 (56*)		50 (49*)		129 (113*)		192 (69*)		0.026†‡	1.36*‡
After intervention (21 months)	58 (64*)		43 (56*)		166 (186*)		166 (77*)			(1.04 to 1.78)
Number (%) complying with best practice recommendations	City I: AF training	City II: TIA training (control)		Standardised difference in % (AF trained – untrained)	North: AF training	South & West: TIA training (control)		Standardised difference in % (AF trained – untrained)	Pooled significance	Combined odds ratio (95% CI)
Not complying at intervention	76	75			116	214
Complying by end of study (n)	14 (18%)	13 (17%)		+2.2%‡	42 (36%)	55 (26%)		+10.6%‡	0.103†‡	1.42‡ (0.93 to 2.17)
Diagnosed July 2000 to Mar 2002 (21 months)	42	44			107	145
Complying before intervention (n)	22 (52%)	22 (50%)		+4.7%‡	60 (56%)	76 (52%)		+3.4%‡	0.543†‡	1.14‡ (0.74 to 1.76)
Diagnosed April 2002 to Dec 2003 (21 months)	58	43			166	166
Complying by end of study (n)	38 (66%)	22 (51%)		+14.8%‡	101 (61%)	93 (56%)		+6.9%‡	0.141†‡	1.34‡ (0.91 to 1.96)

AF, atrial fibrillation; TIA, transient ischaemic attack.

*Adjusted for sampling fraction in each practice.

†Mantel–Haenzel test.

‡Standardised or adjusted for age and sex.

Together the combined odds ratios in the last subtable estimate the odds that training improves compliance at 1.18 (95% CI 0.96 to 1.45).

Figure 3 shows the changes in TIA clinic referrals. Again referral rates increased significantly more in trained practices than in untrained practices. Referrals from the trained South & West PCT, initially lower than in North PCT, rose to a similar level after the intervention. Referral rates declined in City practices trained in AF, but not in those trained in TIA.

Figure 3 Trends in appropriate referrals to transient ischaemic attack clinic: standardised moving‐average monthly rates per million for each primary care trust cluster (intervention at 28 months). Samples sizes in table 3.

While the intervention appears to be effective in improving atrial fibrillation case‐finding and diagnosis, its effect on atrial fibrillation guideline compliance is less clear. The second and third subtables of table 2 give two separate comparisons of the atrial fibrillation compliance rate in trained and untrained practices. Among patients diagnosed with atrial fibrillation but not appropriately treated before the intervention, treatment was significantly more likely to improve in non‐City practices (and in patients over 70 years), regardless of whether the practice had been trained! These patients were also more likely to be appropriately treated later if their practice had been trained, but this was not statistically significant. We compared compliance for patients diagnosed after the intervention with that for patients diagnosed in a similar 21‐month period before the intervention. The improvement was not significant in either trained or untrained practices.

In the TIA audit, in contrast, compliance was significantly higher in practices that had received the TIA training package (combined odds ratio 1.8; 95% CI 1.1 to 2.8).¹⁷ Combining the two audits improves the significance of the training effect in the model and narrows its confidence interval. Whether table 3 combines results from the atrial fibrillation and TIA audits by the Mantel‐Haenzel test or by logistic regression, it estimates that training improves compliance by a combined odds ratio of 1.5 with a 95% confidence interval from 1.1 to 1.9. The stepwise logistic regression to predict guideline compliance includes three predictor variables: the patient's age and sex, and whether the practice has been trained in the appropriate guideline. The area (City or non‐City) and the condition (atrial fibrillation or TIA) do not significantly affect compliance once these three variables have been included in the prediction equation. If the patient is over 70, the odds of complying with the guidelines are 1.8 times higher than for a younger patient of the same sex. Adding extra age categories did not improve the model. The odds of guideline compliance are 1.5 times higher for female patients than for males.

Table 3 Guideline compliance—modelling atrial fibrillation and transient ischaemic attack audits together (from intervention to Dec 2003).

Complying with guideline? Number (%)	City		Significance (comparing changes)	North and South & West		Significance (comparing changes)	Pooled significance	Combined odds ratio estimate (95% CI)
	No	Yes		No	Yes
TIA audit
Untrained	7 (47%)	8 (53%)	0.328	60 (46%)	70 (54%)	0.047
Trained	14 (33%)	29 (67%)		65 (35%)	122 (65%)			1.46†‡
AF audit							0.009	(1.10 to 1.94)
Untrained	21 (49%)	22 (51%)	0.146	73 (44%)	93 (56%)	0.328
Trained	20 (34%)	38 (66%)		65 (39%)	101 (61%)
Logistic regression	Coefficient	Significance level		Estimated effect on odds ratio			95% confidence interval
Constant	0.065			(1.07)
Old (>70)	0.569	<0.001		1.77			(1.32 to 2.37)
Sex (female)	0.395	0.007		1.49			(1.11 to 1.96)
Trained	0.375	0.010		1.46			(1.09 to 1.94)

AF, atrial fibrillation; TIA, transient ischaemic attack.

†Mantel–Haenzel test.

‡Standardised or adjusted for age and sex.

Economic evaluation

Table 4 shows the calculated costs for each of the main activities required to develop and implement the TIA and atrial fibrillation guidelines, respectively totalling £50,100 and £48,600. The totals exclude the costs of educational events since these were within the general practioners' routine protected educational time; opportunity costs were therefore zero.

Table 4 Guideline costing.

Input to guideline development and implementation	Guideline
	AF	TIA
Pilot groups	£530	£425
Focus groups	£306	£242
Stakeholder consultation	£161	£161
Consensus groups	£4940	£4940
Education events	£13,128	£12,810
Management/coordination of interventions	£29,644	£29,644
Audit of practices	£6995	£6995
Follow‐up of practices	£757	£2251
Miscellaneous costs	£1148	£1148
Total cost for all inputs	£57,610	£58,617
Total cost for guideline (excluding education events)	£44,482	£45,807
Uplift to 2003–4 prices	£48,618	£50,066

AF, atrial fibrillation; TIA, transient ischaemic attack.

Although the estimated cost approaches £100,000, this total needs to be interpreted in the light of the number of practices engaged and the effects obtained. Table 5 shows the unit costs. The cost per practice was £1430 for the atrial fibrillation guideline and £1160 for the TIA guideline. The estimated costs per unit of effect¹⁹ (that is, per extra patient receiving optimal care after guideline implementation) were £742 for atrial fibrillation and £858 for TIA (table 5), based on the results presented in tables 2 and 3, and previously published TIA results.¹⁸

Table 5 Unit costs (comparing trained practices with what they would be like untrained).

Guideline	Atrial fibrillation		Transient ischaemic attack
Cost of guideline intervention	£48,618		£50,066
Number of practices trained	34		43
Cost/practice trained	£1430		£1164
Extra patients treated correctly:
Over 21 month period of study	n (% gain)	Extra patients	n (% gain)	Extra patients
Previously diagnosed but untreated	192 (7.3%)	14.0	NA	NA
Newly diagnosed:
from improved diagnosis	59.3 (62.1%)	36.8	57.1 (65.7%)	37.5
from improved compliance	164.7 (8.9%)	14.7	172.9 (12.1%)	20.9
Total		65.5		58.4
Cost per unit of effect (extra patient correctly treated)		£742		£858

Numbers based on the observed patients in trained practices after training and the relative risks of compliance and diagnosis in trained compared to untrained practices.

To translate this result into cost per quality‐adjusted life year (QALY) gained requires estimates of both the QALY gain from treating a patient according to the guidelines and the treatments costs for patients following and not following guidelines. Although this study collected neither of these, approximate results can be deduced from the literature.

There is good economic evidence on the cost effectiveness of a variety of interventions designed to reduce the probability of health‐related adverse outcomes associated with stroke. In particular, for both TIA²⁰ and atrial fibrillation,²¹,²²,²³ prescription of aspirin produces QALY gains relative to no therapy, with a net saving in treatment costs. The small direct treatment cost and the cost of treating adverse effects such as bleeding are more than counterbalanced by the reduced cost of treating patients after stroke or myocardial infarction.

Eckman²³ estimated that in atrial fibrillation aspirin produces a QALY gain of 0.18 and a net cost saving of £464 per patient for a typical patient. When combined with the results in table 5, this would give a cost per QALY of £1540. Warfarin, although more expensive, is a cost‐effective addition for patients at high or medium risk, but not those at low risk,²¹,²³ with a cost per QALY of about £3000 relative to aspirin and £1100 relative to no treatment. These studies used US treatment costs and, although prices were converted to UK currency at 2003 prices, this may not estimate NHS prices accurately.

No equivalent estimates are available for TIA patients, but combining risks and estimates from Sarasin²⁰ implies that aspirin may deliver a QALY gain of about 0.3 per patient treated, with at least as large a cost saving as for atrial fibrillation patients. As the TIA guideline also included driving advice, for which no QALY estimates are available, the gain from guidelines associated with aspirin or warfarin will occur only through extra diagnoses. Nevertheless, table 5 suggests that these yield a cost per QALY of less than £2000. All of these estimates are well within acceptable cost‐effectiveness threshold values.²⁴,²⁵

Discussion

The study demonstrates that healthcare professionals are significantly more likely to comply with clinical guidelines following a tailored and multifaceted intervention. The costs of the implementation strategy were modest, and the resulting cost‐utility ratios of TIA and atrial fibrillation treatment were well within what is conventionally considered an acceptable range.²⁴,²⁵

Strengths and weaknesses of study

In North and South & West Bradford we chose the PCT as the natural focus of implementation. So many initiatives had already crossed practice boundaries that it would have been artificial to focus on individual practices or limit interventions to specified practices. In Bradford City PCT, however, the individual practice remained the natural focus of intervention and this provided the opportunity to evaluate the effect of guideline implementation in such a context. As we were unable to achieve our planned aim of allocating individual City practices at random, however, our pragmatic allocation of entire practice development groups, though random, may have introduced systematic differences between intervention groups. However we judged this preferable to the risk of contamination that would otherwise have accompanied artificially imposed groupings. Moreover the quality and training leaders in this PCT judged that systematic differences were unlikely since the major pre‐intervention differences were between City and non‐City practices.

We sought to include all practices within the three PCTs, both intervention and control, in the TIA audit. This reduced the potential for selection bias from including only volunteer practices, as occurs in many implementation studies. All practices were invited to participate in the atrial fibrillation audit; several City practices declined, but only one non‐City practice. Though missing data were thus limited, they may have biased our findings. We chose process measures rather than clinical outcomes to assess change in clinical practice, as the treatment recommended in the guidelines was of proven effectiveness.

Interpretation of findings

Both TIA and atrial fibrillation interventions increased significantly the numbers of patients identified by general practices, with trained practices improving more than untrained in each area. For both conditions, identification rates were much higher in non‐City practices, even after age adjustment. Surprisingly there was no evidence that the intervention decreased this difference, as the training might have been expected to have more effect on incidence where there were more undiagnosed cases. True rates of atrial fibrillation and TIA may vary between areas, and both conditions may be underdiagnosed even where the apparent rate is high. City practices may be less receptive to change: they were more likely to refuse to participate in the atrial fibrillation audit. TIA diagnosis may have declined in untrained City practices because those practices, many of which were single‐handed, were concentrating on improving their identification and treatment of atrial fibrillation (in which they had been trained).

In contrast guideline compliance among diagnosed patients was similar in City and non‐City practices. The intervention improved this compliance for both conditions, although the improvement was not statistically significant for AF alone. This latter finding may be because compliance with the AF guideline was more complex and involved the use of drugs with potentially serious side effects. Previous research has demonstrated that the attributes of clinical recommendations can modify the effect of change.²⁶ Clinicians are more likely to comply with guidance if they are compatible with their own values and the perceived risk of warfarin may have reduced this compliance.

The effect of the intervention on guideline compliance was not affected by the age or sex of the patients. However female patients and patients over 70 years are more likely to be treated in line with the guideline recommendations, regardless of whether their practice received the intervention. Possible explanations are that female patients may be more likely to accept treatment options recommended by their general practioner, or be more frequent attendees at clinic. The introduction within primary care contracts of screening and medication reviews for those aged over 70 may explain their better treatment and management.

Though we were unable accurately to estimate the power of this study beforehand, the many significant findings provide reassurance that it was appropriately powered.

Box 1 Components of an effective implementation strategy used in study intervention

• Initial contextual analysis at start of process to identify barriers and facilitators for change.

• Strong professional support.

• Clear recommendations about the clinical context.

• Robust research evidence base demonstrating effectiveness of guidance.

• Guidance that is not too complex or expensive to adopt.

• Good communication and use of established networks and opinion leaders.

Implications of findings for practice and future research

In a previous study we evaluated the effect of a similar tailored and multifaceted intervention for asthma and angina guidelines.²⁷,²⁸ We found significant increases in guideline compliance in both intervention and control groups and from our qualitative evaluation concluded that the effect of the intervention had been swamped by concurrent national policies. These included the introduction of clinical governance, national clinical guidelines from the National Institute for Health and Clinical Excellence (NICE) and national service frameworks. The effectiveness of our atrial fibrillation and TIA implementation programme in a more stable environment emphasises the importance of reviewing the context of implementation studies.

Previous reviews have described different approaches to implementation and summarised the imperfect evidence base about which implementation strategies are most effective and efficient.¹,²,²⁹ More recently an evaluation of the effect of NICE guidance in the UK identified factors that were important in successful implementation.³⁰ Box 1 highlights the components of effective implementation from this literature that were adopted for our multifaceted intervention.

Finally the study contributes to the scarce evidence base about the cost effectiveness of implementation research. The intervention was affordable, feasible and effective in very different primary care contexts across a whole health district. Further research is required to assess whether this is a generalisable and sustainable model for implementing the ever‐increasing volume of guidance in primary care and to evaluate which components are most cost effective.

Abbreviations

PCT - primary care trust

QALY - quality‐adjusted life year

TIA - transient ischaemic attack