Pragmatic randomised controlled trial to evaluate guidelines for the management of infertility across the primary care-secondary care interface

Abstract

Objective

To investigate the effect of clinical guidelines on the management of infertility across the primary care-secondary care interface.

Design

Cluster randomised controlled trial.

Setting

General practices and NHS hospitals accepting referrals for infertility in the Greater Glasgow Health Board area.

Participants

All 221 general practices in Glasgow; 214 completed the trial.

Intervention

General practices in the intervention arm received clinical guidelines developed locally. Control practices received them one year later. Dissemination of the guidelines included educational meetings.

Main outcome measures

The time from presentation to referral, investigations completed in general practice, the number and content of visits as a hospital outpatient, the time to reach a management plan, and costs for referrals from the two groups.

Results

Data on 689 referrals were collected. No significant difference was found in referral rates for infertility. Fewer than 1% of couples were referred inappropriately early. Referrals from intervention practices were significantly more likely to have all relevant investigations carried out (odds ratio 1.32, 95% confidence interval 1.00 to 1.75, P=0.025). 70% of measurements of serum progesterone concentrations during the midluteal phase and 34% of semen analyses were repeated at least once in hospital, despite having been recorded as normal when checked in general practice. No difference was found in the proportion of referrals in which a management plan was reached within one year or in the mean duration between first appointment and date of management plan. NHS costs were not significantly affected.

Conclusions

Dissemination of infertility guidelines by commonly used methods results in a modest increase in referrals having recommended investigations completed in general practice, but there are no detectable differences in outcome for patients or reduction in costs. Clinicians in secondary care tended to fail to respond to changes in referral practice by doctors. Guidelines that aim to improve the referral process need to be disseminated and implemented so as to lead to changes in both primary care and secondary care.

What is already known on this topic

Most previous research into clinical guidelines has focused on their development and implementation

Evidence is lacking about the outcomes and costs associated with the use of clinical guidelines

What this study adds

Clinical guidelines that may alter the balance of care between general practice and hospital settings require more intensive implementation than guidelines aimed at either setting on its own

The cost effectiveness of clinical guidelines should not be assumed

Introduction

Clinical guidelines aim to promote evidence based practice, improve patient outcome, and allow more efficient use of resources, but there is still uncertainty about whether they achieve these aims.¹ About 15% of couples experience difficulty and delay in conceiving.² The Effective Health Care Bulletin on subfertility suggested that the clinical management of these couples could be improved by guidelines covering aspects of investigation and referral by general practices.³ A previous study in Aberdeen showed that such guidelines increased the numbers of couples with appropriate investigations completed before referral but did not provide data on outcomes or costs.⁴

Our main hypotheses were that clinical guidelines for the management and referral of patients with infertility by general practices would reduce the proportion of patients referred too soon after attempting to conceive and increase the number of appropriate investigations carried out in general practice. We thought this would lead to a reduction in hospital based investigations and outpatient visits, resulting in a shorter time to reach a management plan and reduced overall costs to the health service. As Glasgow is a complex referral setting with five secondary and tertiary referral sites and noticeable contrasts in socioeconomic deprivation, we also examined the effects of the hospital site and deprivation score on outcomes.⁵

Participants and methods

A multidisciplinary group with professional or personal experience of subfertility developed the consensus guidelines. Research evidence was incorporated where it existed—that is, if there was an area of uncertainty about best practice, one or more members of the group searched the literature and reviewed the evidence. A management pack for infertility was produced containing the detailed guidelines, quick reference guidelines (in box), a structured form for use by doctors at referral, and two information leaflets for patients. The guidelines were introduced as part of a pragmatic, randomised controlled trial. Approval for our study was given by local ethics committees.

Key recommendations in the guidelines for when a couple presents

Ensure appropriate knowledge about how to conceive, then:

• Confirm rubella immunity

• Update cervical smear

• Advise to take folic acid

Before referral, arrange:

• One semen analysis (if first result is abnormal, repeat)

• One measurement of serum progesterone concentration during the midluteal phase (if no evidence of ovulation from first result, repeat)

• Only perform additional endocrine tests (follicle stimulating hormone, luteinising hormone, thyroid stimulating hormone, testosterone, prolactin, oestradiol) if cycle is less than 21 days or greater than 35 days

Refer without delay if:

• The woman is aged 35 years or more

• The woman's cycle length is less than 21 days or greater than 35 days

• The woman's progesterone concentration is less than 20 nmol/l

• The woman has a history of ectopic pregnancy, pelvic infection, endometriosis, or anatomical abnormalities

• The man has two abnormal results for semen analyses

• Otherwise delay referral until couple have been trying to conceive for 12 months

• After referral continue to provide support and information as needed

The study population comprised patients resident and registered with doctors in the Greater Glasgow Health Board area. All 621 doctors and 47 registrars in 221 general practices in Glasgow were informed by letter about the study and invited to participate.

We stratified practices into small, medium, and large on the basis of number of partners (one or two, three or four, and five or more, respectively) and by the location of the catchment areas of the five large hospitals that manage infertility in the health board area. We randomised these into intervention and control practices. To reduce the risk of contamination, practices sharing the same premises were allocated to the same group.

Doctors in intervention practices were sent the management pack and invited to attend one of four meetings to discuss the implementation of the guidelines. Overall, 57 doctors (17%) attended a meeting. Individual visits to practices to discuss implementation were also offered, and two practices were visited. Doctors in the control practices were informed that they would be sent a copy of the guidelines after 12 months. Relevant professionals in the participating hospitals were informed about the project during individual meetings with members of the study team (JM and LC), and infertility specialists from three of the participating hospitals were members of the guideline development group.

In each of the five hospitals, referral letters for infertility were screened by the research assistant for one year from June 1996 to May 1997, using a system of key words. Information about the date of birth, postcode, and whether the referral was for a woman only, a man only, or a couple was recorded from the referral letter.

Management survey before referral

Shortly after we had identified referrals, the referring doctor was sent a questionnaire requesting information about the management of the couple before referral, including questions about the dates of presentation and referral and investigations carried out. A reminder was sent after three weeks if necessary.

Review of hospital management from case notes

Twelve months after we had identified referrals, we obtained data from hospital records about hospital care, including the investigations carried out and the time taken to reach a management plan. A management plan was defined as a definite decision about management and included: specific treatment such as ovulation induction with or without intrauterine insemination, tubal surgery, male surgery, or referral for assisted conception techniques; one partner only continuing with specific investigations; review appointment only; or discharged from further hospital management. To comply with the Human Fertilisation and Embryology Act, we collected no further data on couples referred for consideration of assisted conception techniques. This decision was recorded as “a management plan in place,” and these couples were included in the analysis.

Statistical analysis

We originally estimated that a sample size of 600 referrals would have 80% power to detect at the 5% significance level a 15% difference in the number of measurements of serum progesterone concentrations during the midluteal phase and a difference between groups greater than 0.275 of the standard deviation of continuous variables, such as time intervals.

For presentation in the tables, we aggregated data about referrals from each practice and proportions determined for items under investigation. We tested the effect of the intervention by comparing the mean proportions for items under investigation in the two groups of practices. We used multilevel modelling to account for the clustering effect of practices and to assess the effect of patient deprivation and site of hospital referral.⁶ All analyses were on an intention to treat basis.

Cost analysis

For tests and investigations before referral we extracted information on the use of resources from the prereferral questionnaire. For use of hospital resources in the 12 months after referral we extracted information relating to tests, investigations, and clinic attendances from case notes.

To reduce bias we included in the analysis only cases where full information was available. We then applied unit costs to each unit of resource use. To reduce bias as a result of using different costs from each hospital, we used a single set of unit costs. All costs are in 1998 prices. We compared mean costs with t tests.⁷ The costs of developing and disseminating the guidelines are not presented as part of this cost analysis as we wanted to focus on the effect of the guidelines on the cost of referral.

Results

Seven of the 221 practices withdrew from the study, leaving 214 (96%) in the trial (figure). We collected data on 689 referrals. Median referral rates were 3.25 per 1000 from 88 454 women aged 20-44 years in the intervention practices and 3.27 per 1000 from 95 141 women aged 20-44 in the control practices (P>0.05). The two groups of referred patients were similar for age of the women at the time of referral, the deprivation category of their postal address, and whether the referral was made for a woman only, a man only, or a couple (table 1).

Table 1.

Characteristics of referrals from intervention and control practices. Values are numbers (percentages) of referrals

Characteristic	Intervention practices	Control practices
No of referrals	322	367
Age (years) of female partner at referral:
<34	231 (72)	279 (76)
>35	70 (22)	79 (22)
Missing data	21 (7)	9 (2)
Carstairs deprivation score:
1 and 2	64 (20)	58 (16)
3, 4, and 5	86 (27)	96 (26)
6 and 7	144 (45)	198 (54)
Missing data	28 (9)	15 (4)
Type of referral:
Women only	272 (84)	311 (85)
Men only	30 (9)	26 (7)
Couple	20 (6)	30 (8)

Management by doctors before referral

The referring doctor could not be identified from 30 referral letters (17 intervention, 13 control). Overall, 553 of 659 (84%) prereferral questionnaires were returned: 249 of 305 from the 85 intervention practices (82%) and 304 of 354 from the 79 control practices (86%).

Overall, 71 referrals were made after less than 12 months' reported duration of infertility (30 intervention, 41 control). Reasons for early referral included the woman being aged over 35 years (n=12), a relevant medical history according to the guidelines (n=32), an abnormal result for serum progesterone concentration in the midluteal phase or semen analysis (n=15), a history of infertility (n=1), and a history of miscarriage or stillbirth (n=5). More than one reason was given in 11 referrals. On checking the referral letter for the 17 cases with no apparent reason for early referral, six had a positive history according to the guidelines, and five had a history of infertility. Only six referrals after less than 12 month's infertility did not have an appropriate reason for early referral given either in the questionnaire or in the referral letter (1 intervention, 5 control).

Each investigation was carried out more frequently in the intervention practices, but these differences were not significant (adjusted odds ratios in table 2). A difference was found, however, in the total number of investigations, with referrals from intervention practices having a significantly higher mean. Deprivation category and referral hospital did not influence these results.

Table 2.

Numbers (percentages) of referrals with each baseline investigation carried out before referral and mean number of investigations conducted per patient

Investigation	Intervention practices	Control practices	Odds ratio (95% CI)	P value
No of referrals	249	304
Midluteal progesterone concentration	140 (56)	147 (48)	1.46 (0.93 to 2.31)	>0.5
Semen analysis checked	93 (37)	93 (31)	1.34 (0.84 to 2.13)	>0.5
Cervical smear recorded as checked in past 3 years	214 (86)	26 (86)	1.05 (0.53 to 2.00)	>0.5
Advice given about folic acid supplementation	142 (57)	151 (50)	1.31 (0.85 to 2.02)	>0.5
Rubella immunity status checked	111 (45)	110 (36)	1.41 (0.93 to 2.15)	>0.5
Mean No of tests	2.81	2.50	1.32 (1.00 to 1.75)	0.05

Review of case notes

Case notes were available for inspection for 617 of 689 (90%) referrals (285 intervention, 332 control).

Outcome of referral

A management plan was in place within one year for 292 referrals (47%). No difference was found between intervention or control practices in the proportion of couples for whom a management plan was made (145 intervention practices, 147 control practices: odds ratio 1.239, 95% confidence interval 0.869 to 1.765: P=0.236).

The mean duration between first appointment and date of the management plan was 3.34 (SD 2.93) months for referrals from intervention practices and 2.98 (2.59) months for referrals from control practices (95% confidence interval for difference −6.9 to 47.9; P=0.236).

No difference was found in the mean number of outpatient visits before a management plan was reached (1.92 (0.87) intervention, 1.91 (0.90) control (95% confidence interval for difference −13.9 to 20.3; P=0.836)). The results relating to hospital management were not influenced by correcting for the patient's deprivation category or for hospital site.

Hospital management

Measurement of serum progesterone concentration during the midluteal phase was repeated in 159 of the 287 referrals (55%), including all 111 referrals with a normal test result in primary care before referral. Semen analysis was repeated in 95 of the 168 cases (57%), including 20 of 58 referrals with a normal test result in primary care before referral.

Cost analysis

Full information on which costs could be calculated was available for 159 referrals from intervention practices and 183 referrals from control practices. No significant differences were found between the two groups in the total costs to the NHS, although the median cost was lower in referrals from control than intervention practices (table 3).

Table 3.

Costs (£) to NHS of referrals from intervention and control practices. Values are means (medians) unless stated otherwise

Cost	Intervention practices	Control practices	P value
No of referrals	159	183
General practitioner	26.38 (23)	22.07 (15)	0.084
Hospital:	323.39 (214)	305.40 (196)	>0.5
Investigations	214.26 (60)	196.82 (39)	>0.5
Clinic	109.13 (72)	108.59 (72)	>0.5
Total	349.78 (251)	327.48 (215)	>0.5

Discussion

Principal findings

Following a comprehensive evaluation of the effect of clinical guidelines on the management of all cases of presumed infertility referred for hospital investigation in Glasgow over one year, few patients were referred inappropriately early for investigation of infertility. The guidelines resulted in a modest increase in the number of recommended investigations carried out before referral. There was no evidence that subsequent hospital management was influenced as a result of better investigation by doctors before referral. Many investigations were repeated, possibly representing an important opportunity cost to the NHS. There was no evidence that patient outcomes were improved or that the intervention led to cost savings for the NHS (in fact it may have led to increased costs owing to the unnecessary repetition of investigations).

Strengths and weaknesses

Our study is the largest one of referral guidelines to date. We involved 96% of general practices in the Greater Glasgow Health Board area. Response rates for surveys of care before and after referral were also high (84% general practices, 90% hospitals), and information on outcomes and costs to the NHS was obtained. A recent systematic review of interventions including guidelines to improve outpatient referrals from primary care to secondary care found only one other study in which most eligible providers participated, and a cost analysis was included (J Grimshaw et al, unpublished data 2000). The study was similar to the previous study in Grampian. This has enabled replication of an apparently successful intervention in a more complex referral setting, allowing the external validity of the intervention to be assessed.

Interpretation within context of setting and intervention

Our implementation strategy followed current practice at the time of the study, including simple distribution of the guidelines, educational meetings, and visits to practices, but uptake of educational meetings and visits to practices was low. Use of more intensive methods to encourage the guidelines to be used might have increased uptake but would have required additional resources.⁸

The increases in recommended investigations before referral are smaller than those observed in the evaluation of similar guidelines in Grampian, where the proportions of cases who had progesterone concentrations measured during the midluteal phase and semen analyses checked increased from 40% to 71% and from 41% to 51%, respectively.⁴ The reasons for the lower baseline levels and smaller guideline effects in Greater Glasgow are not known but could include the general effects of severe socioeconomic deprivation on general practice in Glasgow and specific problems in implementation of the guidelines.⁵ For example, some aspects of the guidelines, such as the recommendations for semen analysis, may have been more difficult to implement in an urban setting, where fewer couples are registered with the same doctor. In addition, Grampian is a geographically discrete referral setting where couples are referred to a single teaching hospital, and there is a strong history of research collaboration across the primary care-secondary care interface. Arguably, Glasgow provides a more typical setting for research of this nature.

Possible mechanisms and implications for clinicians or policymakers

Referral behaviour is potentially difficult to change. For most clinical problems the decision to refer is taken infrequently in general practice and is influenced by a range of complex factors, including psychosocial considerations, which are poorly recognised in referral guidelines. Guidelines continue to be produced, but the results of our study show that their simple dissemination is unlikely to be sufficient in complex referral settings. Furthermore, interventions will be undermined if management in secondary care is not coordinated with that of primary care. It is possible that the duplication of tests in hospital reflects an unnecessarily rigid application of local clinical protocols or a distrust of results obtained from unfamiliar laboratories, but clearly there is scope for better communication and coordination of care between hospitals and general practices. It is likely that referral guidelines such as those being prepared by the National Institute for Clinical Effectiveness will not succeed without important activities for local implementation across the interface. For example, in contrast to our findings Thomas et al found that patients referred for urological investigation by doctors who used guidelines were seen and had a management decision more rapidly than control patients.⁹ These guidelines, however, were part of a complex intervention that included reorganisation of the secondary care system to streamline the referral process.

Relatively few studies of interventions to improve referrals exist. Further studies are needed, especially to explore methods of engaging both primary care and secondary care. Our study shows the need to replicate implementation studies to test the generalisability of effects in other settings.

Figure.

Trial profile