Decision making, evidence, audit, and education: case study of antibiotic prescribing in general practice

Abstract

Objectives

To describe a group general practice's implementation of a decision to prescribe 3 day courses of 200 mg trimethoprim twice daily for urinary tract infections in women and to compare 3 day courses with 5 and 7 day courses.

Design

Record review, audit of trimethoprim prescribing for urinary tract infections, and critical appraisal of evidence originally presented in support of 3 day course.

Setting

Group general practice in Newcastle upon Tyne.

Data sources

The records of all female patients aged 12 years and older who were prescribed trimethoprim for uncomplicated urinary tract infections during a 12 month period were reviewed. 271 valid records were identified.

Data extraction

Prescribing rates for different courses of trimethoprim, rates of patients returning for second consultations, rates of urine cultures, results of cultures, results of critical appraisal of evidence.

Results

114 of 271 (42%) prescriptions written at the first visit were for 3 day courses. 16 of 114 (14%) patients who had had a 3 day course of treatment returned for a second consultation compared with 6/83 (7.2%) of those who had had a 5 day course and 8/74 (11%) who had had a 7 day course. The difference between 3 day and 5 day courses in rates of returning for second consultations was 6.8% (95% CI −1.7% to 12.6%) and between 3 day and 7 day courses was 3.2% (−3.6% to 10.0%). Appraisal of the original evidence on which the practice based its recommendations showed that it was flawed. Additional evidence was found in the Cochrane Library.

Conclusions

Our original decision, made by consensus at a meeting of the practice's partners, had not led to a consistent change in practice. We did not find a significant increase in treatment failures among patients treated with the 3 day regimen.

Introduction

Successful programmes for implementing effective healthcare interventions have often developed from projects on single topics that have been carried out in local areas but organised outside individual general practices.¹ Of necessity these programmes address only one aspect of health care at a time, yet practices need continuously to re-examine their policies on many topics if they want to improve clinical effectiveness and cost effectiveness. It is unrealistic to expect large scale projects to be implemented to assess each aspect of health care, therefore practices must rely on their own resources to make decisions about most changes in clinical management. Westerhope Medical Group benefited from the services of a qualified pharmacist (DP) for one session per week who facilitated this process. She presented reports on the practice's prescribing patterns and made recommendations based on local policy and evidence selected by local experts. Until the events described in this paper decisions about changes in prescribing were made at meetings of the practice's partners (which she attended), taking into account her reports and advice.

At one meeting we decided to use 3 day courses of trimethoprim (200 mg twice daily) as the first line treatment for uncomplicated lower urinary tract infection occurring in adult women. Evidence from local experts supported this regimen.² The partners did not critically appraise the evidence before implementing the decision; it was made by consensus after a brief discussion.

More than a year later, soon after we had introduced fortnightly multidisciplinary educational sessions,³ a practice nurse claimed, during a session devoted to questions from everyday practice, that the change to the 3 day course had led to more treatment failures. At that point some general practitioners claimed not to have heard of the new policy, and we realised that it was not being implemented consistently.

We decided to obtain and critically appraise the articles cited in support of the policy and to search for new evidence. At the same time we would also audit the prescribing of trimethoprim for urinary tract infections and the investigations and second visits associated with its use. Merely returning for a second visit does not necessarily indicate that a treatment has failed but was a practicable outcome measure given that patients in routine clinical practice are not likely to be followed up as systematically as in clinical trials. However, we felt that we would be justified in reviewing our policy if the audit found that there was a significantly higher rate of reconsultation among patients who had been given 3 day courses of trimethoprim compared with those who had been given 5 or 7 day courses.

Methods

Critical appraisal

Copies of all the articles cited were obtained and critically appraised using checklists from standard texts.^4–9 Critical appraisal took less than an hour, and most articles did not meet the essential criteria given in the checklists which would have allowed them to be classed as methodologically rigorous studies.

Literature search

A 10 minute search of the Cochrane Library found a meta-analysis of randomised trials of single dose treatment with antibiotics, randomised trials of 3 day courses of co-trimoxazole compared with 7 day courses for urinary tract infections in women, 3 day courses of co-trimoxazole and pivmecillinam compared with 10 day courses, treatment with a single dose of an antibiotic compared with 3 day and 7 day courses (including a single dose of co-trimoxazole compared with a 3 day course), and treatment with 3 days of co-trimoxazole compared with 10 days.^10–14

Audit

Electronic medical records were searched by DP to identify all prescriptions for trimethoprim written in the 12 months after the decision had been made to use a 3 day course as first line treatment. The records of children younger than 12 years were excluded. Both the paper and the electronic records were retrieved and examined. The length of the first prescription for trimethoprim in days (3, 5, or 7) was recorded, whether the patient returned with similar symptoms within 2 weeks, whether a urine culture was performed at the time of the first or second consultation, the result of the culture (no growth, sensitive to trimethoprim, sensitive to other antibiotic), the sex of the patient, and the general practitioner who wrote the prescription.

Results

Critical appraisal and literature search

Four reviews were cited and their critical appraisals are summarised in table 1.^6–9 None is a full systematic review: no details are given of search strategies and only one outlines the criteria used to evaluate the quality of trials (which are described as modest) for inclusion in the review.⁸ The other reviews are expert reviews which cannot be relied on.⁶,⁷,⁹,¹⁵

Table 1.

Critical appraisal of four reviews of treatment of urinary tract infections

Criteria	Study
	Bailey 19936	Neu 19927	Norrby 19908	Powers 19919
Was there an important clinical question?	Yes	Yes	Yes	Yes
Has there been a systematic search for evidence?	No	No	Can't tell	No
Has the methodological quality of the studies included been assessed?	No	No	Yes	No
Are the results sensitive to how the review was performed?	Very	Very	Moderately	Very
Have the numerical results been interpreted sensibly?	No	No	Moderately	No

Checklist adapted from Greenhalgh⁴ 

Norrby summarises and combines evidence from 28 trials.⁸ The trials are heterogeneous and no trials of trimethoprim alone are included, although there are 17 trials of combinations of trimethoprim and sulphonamide. The use of co-trimoxazole is discouraged in the United Kingdom but there is some evidence that trimethoprim alone is as effective as co-trimoxazole.¹⁶,¹⁷ The results seem to show that courses lasting longer than 5 days are more effective than single doses but that they are not more effective than 3 day courses. However the results must be interpreted with caution: they combine data from a heterogeneous selection of small clinical trials of different dosages and combinations of single dose, 3 day, and >5 day courses, and it is questionable whether the aggregation of these data is valid.

Three randomised controlled trials are cited, none of which analysed results by the intention to treat (table 2).^18–21 Two trials compared treatment of urinary tract infections with a single dose of a quinolone and treatment with multiple doses.¹⁸,²⁰ The third trial compared treatment of cystitis in women with a single dose of trimethoprim and 7 day treatment, finding the 7 day treatment significantly more effective.¹⁹ However, the lack of an intention to treat analysis undermines the validity of this result, and even if accepted it is not generalisable to 3 day courses of trimethoprim. The randomised trials were consistent in finding a 3 day course of co-trimoxazole was as effective as longer courses.

Table 2.

Assessment of three clinical trials of treatment of urinary tract infection

Appraisal	Clinical trial
	Iravani 199318	Osterberg et al 199019	Saginur et al 199220
Were participants randomly assigned to treatment?	Yes	Yes	Yes
Was the randomisation list concealed?	Yes	Yes	Yes
Was there an intention to treat analysis?	No	No	No
Was blinding maintained among patients and clinicians?	Yes	Yes	Yes
Aside from the experimental treatment were the groups treated equally?	Yes	Yes	Yes
Were the groups similar at the start of the trial?	Yes	No data provided	Yes

Checklist adapted from Sackett et al⁵ 

A review published in the Drug and Therapeutics Bulletin, which is distributed free to all general practitioners in England and Wales, recommended that uncomplicated cystitis should be managed empirically with trimethoprim for 3 days (or oral cephalosporin or nitrofurantoin in areas where resistance of urinary pathogens is common).²² This review too is non-systematic and adds nothing to the reviews already discussed.

Audit

Altogether, 302 prescriptions of trimethoprim written for symptoms of urinary tract infections were identified. Thirty one records were incomplete or missing or the patients had complex problems such as an indwelling catheter or pyelonephritis; these records were not included in the audit, leaving 271 records for analysis (table 3).

Table 3.

Treatment and follow up of urinary tract infections in women. Values are numbers (percentages) unless indicated otherwise

	Treatment duration			Difference between groups in rates of return visits (%; 95% CI)
	3 days	5 days	7 days	3 days v 5 days	3 days v 7 days
Consultation
First consultation	114/271 (42)	83/271 (31)	74/271 (27)	—	—
Second consultation	16/114 (14)	6/83 (7.2)	8/74 (11)	6.8 (−1.7 to 12.6)	3.2 (−3.6 to 10.0)
Second prescription written	13/114 (11)	2/83 (2.4)	5/74 (7)	9.0 (2.3 to 15.7)	4.6 (−3.6 to 12.8)
Urine culture
Not done at all	38/114 (33)	26/83 (31)	31/74 (42)	2.0 (−11.2 to 15.2)	−8.6 (−21.3 to 4.1)
Needed second consultation	3/38 (8)	0	1/31 (3)	7.9 (−0.7 to 16.5)	4.7 (−5.8 to 15.2)
Organism:
No growth	24/76 (32)	23/57 (40)	16/74 (22)	8.0 (−8.5 to 24.5)	5.0 (−4.9 to 14.9)
Trimethoprim sensitive	49/76 (64)	32/57 (56)	25/74 (34)	8.0 (−8.8 to 24.8)	6.0 (−12.3 to 24.3)
Second consultation if no growth	1/24 (4)	1/23 (4)	4/16 (25)	−0.1 (−11.1 to 11.3)	−20.8 (−43.4 to 1.8)
Second consultation if trimethoprim sensitive	10/49 (20)	4/32 (13)	1/25 (4)	7.9 (−8.2 to 24.0)	16.4 (2.7 to 30)

A total of 114 of 271 (42%) prescriptions written at the first visit were for 3 day courses; 16 of the 114 patients (14%) who had had a 3 day course returned for a second consultation compared with 6 of 83 patients (7%) who had had a 5 day course and 8 of 74 patients (11%) who had had a 7 day course. The differences between the results for treatment with a 3 day course compared with the 5 day and 7 day course are shown in table 3. The results of urine cultures are also presented in table 3.

Although more patients were given 3 day courses of trimethoprim than any other regimen, this still constituted only 42% of prescriptions for urinary tract infections, showing that the policy of prescribing 3 day courses was only partly implemented. The rates of patients returning for a second consultation did not differ significantly between treatment groups (table 4). Patients who had urine cultures performed on their first visit that grew organisms sensitive to trimethoprim and who had been treated with a 3 day course were most likely to return for a second consultation: 10/49 (20%) of these patients returned compared with 4/32 (13%) of those who had had a 5 day course and 1/25 (4%) of those who had had a 7 day course. However, only 1 patient out of 10 treated with a 3 day course received a second prescription. Their high rate of return visits may reflect a lack of confidence in the 3 day regimen on the part of patients or general practitioners.

Table 4.

Number (percentage) of urine cultures done at different visits

	First consultation	Second consultation	Difference between groups in rate of return visits (%; 95% CI)
Consultations with urine culture	176/271 (65)	26/176 (15)	10.6 (4.0 to 17.2)
Consultations without urine culture	95/271 (35)	4/95 (4)

There was an increase in the proportion of urine cultures taken at the second visit that were negative compared with the first visit and a decrease in the proportion of urine cultures growing organisms sensitive to trimethoprim. Patients were no more likely to have a urine culture performed at the second consultation than at the first but were significantly more likely to have a second consultation if a urine culture was performed at the first. All patients whose cultures grew organisms that were not sensitive to trimethoprim, received a prescription at the second consultation but only 7 of the 176 urine cultures performed at the first consultation grew organisms that were not sensitive to trimethoprim. This raises the issue of whether it is worth performing a urine culture at the first consultation for patients with uncomplicated urinary tract infections. Provided that patients are given clear information about the need for a urine culture if their symptoms do not resolve, we thought it unlikely that they would be harmed if they did not have a culture at their first visit.

Discussion

Although these results are reassuring in that 3 day courses of trimethoprim for uncomplicated lower urinary tract infections do not seem to increase the rate of treatment failure, they raise a number of important issues.

Quality of evidence

Firstly, we were disappointed by the quality of the evidence offered by apparently authoritative sources. The evidence cited to support the use of a 3 day course of trimethoprim was scanty, and using Eccles et al's classification, was rated category C at best: it was based on extrapolated evidence from randomised controlled trials (which were of mediocre quality).²³ The only way to be sure that 3 day courses of trimethoprim are as effective as 5 or 7 day courses would be if results from well designed randomised controlled trials in general practice were available; however, such trials have yet to be carried out.

Uncertainty and compromise

Secondly, given that the quality of the evidence was poor, would we make the same decision now given the same circumstances? It was easy to find evidence in the Cochrane Library suggesting that 3 day courses of co-trimoxazole are as effective as longer courses, that trimethoprim alone is as effective as co-trimoxazole,¹² that the recommendation is supported by expert consensus, and that it has biological plausibility. In the real world we have to accept compromises and, despite a degree of uncertainty, would probably have made the same decision. In future we would question any recommendation or guideline and, at the very least, assess its validity by critically appraising it before agreeing to implement it.

Dissemination and implementation

Thirdly, even in a practice that was well motivated enough to employ a pharmaceutical adviser and that had an explicit policy of reviewing its prescribing, the original decision was not well disseminated and only partially implemented. All of the general practitioners prescribed courses of 3, 5, and 7 days even though they had been party to and supportive of the original decision. In general practice doctors deal with hundreds of different problems in a short time and are not likely to remember every guideline or regimen for each condition that they treat.²⁴ Implementation of practice policies requires reinforcement using strategies such as computerised reminders, having regularly revised loose leaf practice manuals at each desk, and providing educational activities within the practice. At an educational session after the audit, the team used our findings to produce a practice guideline which is based both on the best external evidence (including evidence about diagnosis )²⁵ and the audit described here (box). Copies have been distributed to each team member and placed at the telephone triage desk, and the audit will be repeated in 12 months.

What is already known on this topic

For urinary tract infections in adult women some guidelines recommend 3 days of treatment with 200 mg trimethoprim twice daily, although there is some dispute over the most effective treatment

Practices need continuously to re-examine their policies on many aspects of care if they want to improve clinical effectiveness and cost effectiveness

What this study adds

An audit of outcomes at our surgery found no significant difference between women treated with the 3 day regimen and those treated with 5 or 7 day regimens

A more critical appraisal of the evidence on which the guideline for a 3 day course of treatment was based showed that much of the evidence was flawed

A small group educational process was more effective for making and implementing decisions in our practice than the administrative process of the partners' meeting

Guidelines for treatment of urinary tract infections(developed by Westerhope Medical Group, June 1999)

These guidelines apply only to adult women (older than 12 years) who have uncomplicated lower urinary tract infections.

Diagnosis

Diagnosis should be made if there is a history of urinary frequency and dysuria in the woman, if there is no suspicion of more serious problems (for example, pyelonephritis), and if the woman has no history of recurrent urinary tract infections (more than two in six months or three times in one year)

Dipsticks are not reliable in the practice setting

Analysis of a mid-stream urine specimen is not necessary for initial diagnosis

Appointments

Patients with a simple history may be treated without being seen

Patients should be offered an appointment, and a mid-stream urine specimen should be sent for analysis before treatment if they present with a complex history

Treatment

First line treatment is 200 mg trimethoprim twice daily for 3 days

A second course of treatment should be offered only if the first line treatment has failed, and it should be started after a mid-stream urine specimen has been sent for culture and sensitivity testing

Information for patients

If the symptoms have not resolved after 3 days of treatment with trimethoprim patients should make an appointment through the triage system and bring a mid-stream urine specimen in a plain urine container. The specimen should be sent as soon as possible but can be stored overnight in the refrigerator

Conclusions

Clinical audit is now firmly established as a key quality assurance method and our experience shows its value in answering the question “are we doing what we're supposed to be doing?” and “what is the effect of what we're doing?”²⁶ It is important that general practitioners identify and address relevant questions and implement changes according to the best evidence available. We have described the often messy and uncoordinated process whereby we have tried to improve our practice. We are always busy, always have to get through the next surgery, and struggle to find time and effective ways to evaluate and improve our performance. Decisions must be made quickly using the best evidence and data, and we must often trade methodological rigour for practicability and speed. Making this particular decision in a partners' meeting based on external advice did not lead to a consistent change in practice. However, we were able to use multidisciplinary educational methods in a small group to identify and address an important question. Active educational methods in the practice setting are more effective in achieving a sustained change in practice than passive methods (such as lectures or dissemination of information on paper).²⁷ The small group educational process worked better for us than the administrative process of the partners' meeting. It is now therefore our preferred method of identifying the need for review or change in our practice, examining the evidence, and addressing the details of implementation.