Can guidelines improve referral to elective surgical specialties for adults? A systematic review

Abstract

Aim

To assess effectiveness of guidelines for referral for elective surgical assessment.

Method

Systematic review with descriptive synthesis.

Data sources

Medline, EMBASE, CINAHL and Cochrane database up to 2008. Hand searches of journals and websites.

Selection of studies

Studies evaluated guidelines for referral from primary to secondary care, for elective surgical assessment for adults.

Outcome measures

Appropriateness of referral (usually measured as guideline compliance) including clinical appropriateness, appropriateness of destination and of pre-referral management (eg, diagnostic investigations), general practitioner knowledge of referral appropriateness, referral rates, health outcomes and costs.

Results

24 eligible studies (5 randomised control trials, 6 cohort, 13 case series) included guidelines from UK, Europe, Canada and the USA for referral for musculoskeletal, urological, ENT, gynaecology, general surgical and ophthalmological conditions. Interventions varied from complex (“one-stop shops”) to simple guidelines. Four randomized control trials reported increases in appropriateness of pre-referral care (diagnostic investigations and treatment). No evidence was found for effects on practitioner knowledge. Mixed evidence was reported on rates of referral and costs (rates and costs increased, decreased or stayed the same). Two studies reported on health outcomes finding no change.

Conclusions

Guidelines for elective surgical referral can improve appropriateness of care by improving pre-referral investigation and treatment, but there is no strong evidence in favour of other beneficial effects.

In many health systems, the general practitioner or primary care gatekeeper role is considered a vital component of demand management, restricting access to secondary care based on identified need, thus restraining healthcare costs. However, evidence suggests substantial variation in referral rates from primary to secondary care, indicating that access to surgery is not equitable.^1–3 In the UK, guidance for the referral of common conditions to surgical specialties has been provided, with the potential to increase the equity of referral, while maintaining a “gatekeeper role”. This guidance has not as yet been rigorously implemented or evaluated.⁴ Two reviews have investigated interventions for referral from primary to secondary care in general.^{5 6} Faulkner et al⁶ reported little impact of referral guidelines on rates of referral or health outcomes. Grimshaw et al⁵ found that passively disseminated guidelines had less impact than structured referral sheets.

The standard definition of clinical guidelines is “systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances”.⁷ Although their principal intended benefit is to improve the quality of care received by patients, several other benefits to patients, clinicians and healthcare systems have been suggested, including improving consistency and equity of care, empowering patient participation and influencing public policy—for example, through highlighting neglected health problems.⁸

Nevertheless, the evidence that guidelines alone are effective at changing clinician behaviour is limited.^9–12 In light of concern about increasing referrals to secondary care, the introduction of “patient choice” and increasing plurality of elective surgical service providers, the UK Department of Health commissioned this work to answer the following questions: “do guidelines for referral by primary care practitioners for adults for elective surgical assessment improve appropriateness of referral or health outcomes? What is the effect of referral guidelines on healthcare costs?”

Referral guidelines were defined as “a set of rules, an algorithm or a protocol which gives guidance on the circumstances in which a formal elective request is made from primary care on behalf of a patient, for treatment or consultation by a surgical specialist”. (A surgical specialist was defined as a medical practitioner who has undertaken a recognised training to be able to undertake elective, non-urgent surgery). We aimed to investigate the key characteristics of guidelines including format, length, development method and any prescribed or implicit associated implementation strategies.

Methods

The search strategy was developed based on extensive literature scoping exercise, and advice from and clinical, health information and health management experts (including those involved in the Cochrane Collaboration Effective Practice an Organisation of Care Group, practising NHS managers and clinicians, and primary care practitioners/general practitioners with an interest in elective surgical referral). To maximise sensitivity, both text words and subject headings (MeSH terms) were included. Four topic areas (“primary healthcare or family practice”, “practice guidelines or algorithms”, “referral and consultation”, and “surgical procedures or “surgery”) were all combined using the “and” operator. The final search strategy is included in Box 1.

Box 1. Search strategy.

Family Practice (MeSH) OR Primary Health Care (MeSH) OR Physicians, Family (MeSH) OR Primary Care OR “managed care” OR general practi* OR GP OR GPs OR family doctor* OR family physician* OR generalist*

AND

Referral and Consultation (MeSH) OR refer OR referr* OR ((recommend* OR request*) AND (Surgical Procedures, Operative (MeSH) OR Surgery (MeSH) OR surgeon* OR surgery (Text Word) OR surgical (Text Word) OR operate (Text Word) OR operative (Text Word) OR operation (Text Word) OR operations (Text Word)))

AND

Practice Guidelines (MeSH) OR Algorithms (MeSH) OR guideline* OR guidance OR tool OR tools OR “algorithm” OR “algorithms” OR protocol OR protocols OR pathway* OR “care standards” OR “treatment standards” OR “preferred practice patterns” OR “decision tree” OR “decision trees” OR “decision aid” OR “decision aids” OR “decision modelling” OR “decision modelling”.

*denotes generic ending.

Searches were conducted in Medline, EMBASE, CINAHL and the Cochrane Database from inception to February 2008. Hand searches of relevant selected journals (British Journal of General Practice, Journal of Health Services Research & Policy, Quality and Safety in Health Care) and websites were undertaken, and these were supplemented with examination of reference lists and contact with experts. Titles and abstracts were inspected and full texts of potentially relevant publications were each assessed for inclusion by two reviewers. Table 1 shows inclusion and exclusion criteria.

Table 1.

Inclusion and exclusion criteria

Selection criteria	Inclusion criteria	Exclusion criteria
Population patients	▶Adults (aged 16 years+)	▶Children (aged <16 years)
	▶With a non-urgent condition	▶With an urgent condition requiring emergency or urgent referral ▶“Red flag” back symptoms ▶Any evaluation of cancer–eg, fitting UK “2-week rule” guidelines
	▶Guideline consultation –In primary care by a PCP/GP	▶Guideline consultation –In primary dental care by general dental practitioner –Undertaken by optician/optometrist –In Accident and Emergency department –In secondary care
	▶Condition should be amenable to surgical intervention if severe enough ▶Do include infertility if referral is from primary to secondary care; back pain if surgery is an option; glaucoma if referral is from primary to secondary care and surgery is an option; breast symptoms if non-urgent component	▶Condition not amenable to surgical intervention
Population practitioners	▶Referring practitioner is GP or PCP	▶Referring practitioner is not a PCP (eg, in referral to tertiary or high dependency care facilities)
	▶Receiving practitioner is a surgeon or practitioner in surgical specialty in secondary care	▶Receiving practitioner is a someone other than a surgeon or practitioner in a surgical specialty (eg, social services, complementary therapies, district nursing, etc)
Interventions	▶Any guideline(s) or set of rules or protocol that assists PCPs with a decision of whether or not to refer patients to a surgeon or surgical specialty in secondary care for further advice, consultation or treatment	▶There is no identifiable (repeatable, written) set of rules that could be generalised to GP/PCPs in—eg, another geographic area
	▶Referral for endoscopy or other diagnostic tests if referral is for management of symptoms, not just for investigation	▶Referral is for diagnostic tests only ▶Back pain if referral is for physiotherapy or imaging
Outcomes	Any assessments of ▶Appropriateness of referral ▶Change in GP/PCP knowledge ▶Change in disease status/change in health status/quality of life ▶Costs	▶Outcomes identified do not fall into the five identified categories of outcome
Study designs	▶No study design excluded	▶No study design excluded
	▶Evaluation of a referral guideline OR study measuring compliance with specific named guideline (comparison of actual practice with guideline)	▶No evaluation or comparison with guideline ▶No specific, clear, identifiable guideline(s) named
	▶Publication must be research based with original data	▶No original data or research (or duplicate data or research) are presented

GP, general practitioner; PCP, primary care practitioner.

A data extraction form was designed, based on guidance from the Scottish Intercollegiate Guidelines Network¹³ and the NHS Centre for Reviews and Dissemination guidance.¹⁴ Data were extracted by two reviewers with discrepancies resolved through discussion. Studies were appraised using recommended critical appraisal tools and two authors working together allocated a Scottish Intercollegiate Guidelines Network quality score.

Studies were not excluded based on methods or methodological quality. Outcomes of interest assessed were general practitioners' knowledge of appropriateness of referral, actual appropriateness of referral (including appropriateness of diagnostic tests, effects of guidelines on referral rates and on waiting times—often referred to as guidelines compliance in studies), health outcomes and costs. Because of the nature of the topic and the heterogeneity of the studies (interventions, patient groups and outcomes measures used), quantitative summary estimates of effect were not calculated. Data are therefore synthesised descriptively.

Results

Initial searches yielded 9398 papers after duplicates were removed. Nine thousand two hundred and seventy were excluded by reading title and abstract, with uncertainty resolved by discussion between two reviewers. Full texts of 128 papers were retrieved and examined by two reviewers. Discrepancies were resolved by discussion with a third reviewer and 24 papers were identified for inclusion. Figure 1 gives a flow diagram and table 2 summarises the characteristics of the included studies. Publication dates ranged from 1993 to 2007. Ten of the studies were conducted in the UK and the remainder in North America and Europe. Guidelines were variably developed for local, national or international use.

Figure 1.

Flow (QUORUM) diagram. Study identification and selection.

Table 2.

Levels of evidence

1++	High quality meta-analyses, systematic reviews of RCTs or RCTs with a very low risk of bias
1+	Well-conducted meta-analyses, systematic reviews or RCTs with a low risk of bias
1−	Meta-analyses, systematic reviews or RCTs with a high risk of bias
2++	High-quality systematic reviews of case control or cohort or studies, high-quality case control or cohort studies with a very low risk of confounding or bias, and a high probability that the relationship is causal
2+	Well-conducted case control or cohort studies with a low risk of confounding or bias and a moderate probability that the relationship is causal
2−	Case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal
3	Non-analytic studies—eg, case reports, case series
4	Expert opinion

RCT, randomized control trial.

Study design

Eleven controlled designs included data on >9000 referrals. Five studies were randomised controlled trials. Four used cluster randomisation and one used patient-level randomisation.^15–19 There were six cohort designs of which four used historical controls,^20–23 one used concurrent controls²⁴ and one was of mixed design, using a combination of historic and concurrent non-randomised controls.²⁵ Thirteen studies, covering >12 000 referrals, used an uncontrolled case series design.^{12 26–37}

Clinical area of interest

Guidelines covered elective surgical referral for patients to orthopaedic surgeons (low back pain and other orthopaedic conditions), urologists (lower urinary tract symptoms and microscopic haematuria), otolaryngology (ENT), gynaecologists and ophthalmologists. Some guidelines explicitly covered more than one clinical area of interest, particularly those for referral to general surgeons (eg, for hernia repair and other minor surgery).

Interventions

Only three studies provided a simple evaluation of referral guidelines. All these were descriptive designs with two reporting no effects and one reporting benefits. The other studies investigated guidelines that had an associated strategy aimed at enhancing dissemination. These strategies included the following:

1. overall disease management guidelines, which included explicit referral guidelines (13 studies)—for example, a structured management sheet. Some of these included an accompanying educational package for referrers. Of the seven controlled studies that reported on this type of intervention, five reported positive benefits.

2. referral guidelines as part of services explicitly changing and simplifying the patient care pathway usually with accompanying greater integration of primary and secondary care (often referred to as “one-stop shop services”) (six studies)—for example, guideline-based direct access to surgical waiting lists. Both controlled studies reporting on one-stop shop services reported benefits.

3. guideline-based referral triage (which included referral to a central secondary care referral management centre with redistribution to appropriate providers) (two studies). One of the latter was a management guideline where referrers had to obtain telephone-based previous authorisation for referral. The single controlled study of this intervention reported mixed effects.

Guideline development process

Methods of guideline development used in various combinations were consensus development panels, multidisciplinary panels, specialist experience, general practitioner consultation and literature review. None of the studies mentioned patient input either at the individual or the representative level. Nineteen of the studies evaluated locally or regionally developed guidelines, and of these, 18 reported benefits or mixed effects and one reported no benefit. Two descriptive and three controlled studies described local evaluations of nationally or internationally developed guidelines, of which only two reported overall positive results and three reported no benefits.

Table 3 shows study characteristics for descriptive and controlled designs.

Table 3.

Study characteristics and type of intervention

	Main author	Study date	Country	Subjects	Reference	Condition/specialty	Methods	Intervention type	SIGN quality score13	Outcomes measured	Findings
Descriptive designs
1	Fertig	1993	UK local	400 referrals (100 referrals per specialty)+a further 22 referrals by 21 PCPs	12	Orthopaedics ENT, gynaecology, ophthalmology	Descriptive case series	Referral guideline	3	Guideline compliance Referral rates	Most of referrals appropriate in comparison to guideline standards Guidelines unlikely to reduce referral rates
2	Kumar	1996–1997	UK local	100 referrals by ∼100 PCPs	26	ENT, tonsillectomy	Descriptive case series	Referral guideline for direct listing for surgery	3	Guideline compliance	Variable compliance by GPs with guidelines
3	Collins	1995	US national	Simulated case histories/vignettes;444 PCPs	27	Urology, benign prostatic hypertrophy	Cross-sectional survey	Referral guideline	3	Guideline compliance	Diagnostic evaluations vary from guideline standards
4	Fried-Lieb	1990–1991	US local	1796 preauthorisation requests to HRM	28	Orthopaedics, low back pain	Descriptive case series	Management guideline plus telephone based prior authorisation scheme	3	Appropriateness Cost	Use of conservative management increased Net savings
5	Rossi-Gnol	1988–1990	Canada international	456 referrals for 2147 randomly selected patients Quebec Workers Compensation Board	29	Orthopaedics, low back pain	Descriptive case series	Management guideline+referral guideline	3	Guideline compliance Referral rates Waiting times	Compliance with guideline, referral rates low and waiting times high in comparison to guideline standards.
6	Bishop	2003	Canada local	49 referrals	30	Orthopaedics, low back pain	Descriptive case series	Management guideline+referral guideline	3	Guideline compliance	Compliance variable—eg, good in examination; less good in imaging recommendations
7	Cerdan	2002–2004	Spain local	188 referrals made by 71 PCPs	31	General surgery	Descriptive case series	Guideline plus one-stop service	3	Guideline compliance Waiting times	Improved compliance Reduced waiting times
8	Arroyo	2000	Spain local	86 referrals by 26 PCPs	32	General surgery	Descriptive case series	Guideline plus one-stop service	3	Guideline compliance Waiting time	Guideline compliance high Waiting time reduced
9	Padilla	1995–1996	Spain local	400 referrals from five health centres	33	Urology, benign prostatic hypertrophy	Descriptive case series	Guideline plus one-stop service	3	Guideline compliance Waiting times	Compliance good Reduced waiting times
10	Norg	2005	Netherlands local	512 patients from 14 general practices	34	LUTS	Modelling study	Decision checklist	4	Compliance	Compliance estimated as high
11	Rao	2001–2002	UK local	8993 patients	35	Orthopaedics, musculoskeletal problems	Descriptive case series	Referral guidelines pro forma	3	Guideline compliance	High proportion of referrals compliant with guideline.
12	Lash	2005	UK local	62 referrals for cataract	36	Ophthalmology	Descriptive case series	National guidance	3	Guideline compliance	Compliance with “gold standard”
13	Fullen	2006	Ireland	54 referred patients	37	Orthopaedics, acute low back pain	Descriptive case series	Referral guideline	3	Guideline compliance	Referral practice not consistent with European guidelines
Controlled designs
1	Bradshaw	1997	UK local	147 referrals	25	General surgery	Mixed design	Referral guideline for direct listing for surgery	2−	Waiting times	Reduction in waiting time
2	Thomas	2003	UK local	959 referrals from 55 practices/health centres	15	Urology, LUTS; microscopic haematuria	Cluster RCT Allocation concealment+	Guideline plus “one-stop service”	1+	Guideline compliance Health outcomes Short Form Health Survey (SF-36) Waiting times Costs	Appropriate investigations increased in intervention group (0.5 (0.2–0.8) p<0.01) Reduced waiting times No change in patient outcome Reduction in hospital costs (£0.28–£43.00)
3	Emslie	1993	UK local	100 couples from 82 practices/health centres	19	Gynaecology, infertility	RCT Allocation concealment—not stated	Guideline plus structured management sheet	1−	Guideline compliance	Compliance with guideline increased for all targeted activities—eg, use of day 21 progesterone increased (72% intervention vs 41% control (p<0.001)
4	Morrison	1996/1997	UK local	689 referrals from 214 practices/health centres	16	Gynaecology Infertility	Cluster RCT Allocation concealment—not stated	Guideline plus structured management sheet	1−	Referral rates Guideline compliance Cost	No difference in referral rates. Appropriate investigations more likely to be carried out (OR 1.32(1.00–1.75) Non sig increase in costs.
5	Maddison	2001/2003	UK local	∼5000 referrals from ∼100 PCPs	20	Orthopaedics	Cohort study—historical controls	Guideline plus referral triage	2−	Referral rates Appropriateness Waiting time	Rates of referral increased (by 86%) Waiting times reduced Appropriateness (surgical conversion rate 37%) unchanged
6	Benninger	1994/1995	US local	163 referrals from 74 PCPs	21	ENT	Cohort study—historical controls	Management guideline+referral guideline	2−	Appropriateness of referral Waiting times	Appropriateness (45% (before), 70% (after; at 5 months), p<0.05) No change in waiting times
7	Rao	1994/1995	US national	550 referrals by PCPs referring to one secondary care centre	22	Orthopaedics, low back pain	Cohort study—historical controls	Management guideline+referral guideline	2−	Guideline compliance	No change in appropriateness of investigations
8	Goldberg	2001	US national	(No data on absolute numbers of referrals or referrers-only rates supplied)	17	Orthopaedics, low back pain	Cluster RCT (with time series analysis) No allocation concealment	Management guideline+referral guideline	1−	Surgery rates	Reduction of 8.9% surgical operations (20.9 per 100 000 population) over 30 months in intervention communities
9	Spata-fora	2000/2002	Italy national	1203 referrals (2465 patients) 450 PCPs	23	Urology, LUTS	Cohort study—historical controls	Management guideline+referral guideline	2−	Referral rates Guideline compliance Cost	Referral rates unchanged Compliance (eg, reduced use and cost of inappropriate investigation (33%–22%)) (p<0.001) Reduced costs=savings of 13.8%
10	Fender	1999	UK	1001 consultations (130 referrals)	18	Gynaecology, menorrhagia	Cluster RCT Allocation concealment+	“Educational package” based on principles of academic detailing	1+	Guideline compliance (with treatment recommendations) Referral rates	Recommended medication increased (OR 2.38 (1.61–3.49)) in intervention group and referral rate reduced (OR 0.64 (0.41–0.99))
11	Julian	2007	UK local	193 referrals from 157 general practices (99 intervention: 94 control)	24	Gynaecology, menorrhagia	Cohort study—concurrent controls	Management guideline+referral guideline	2+	Patient outcomes (surgery rates)	No difference in SF-36 scores Increased satisfaction in intervention group No difference in surgical rates

HRM, health risk management; LUTS, lower urinary tract symptoms; PCP, primary care practitioner; SIGN, Scottish Intercollegiate Guidelines Network.

Study findings—descriptive designs

Quality assessment

Thirteen of the studies used a descriptive (mostly case series) design. For all, outcome assessment was undertaken by un-blinded assessors (usually the authors who had designed the guidelines). One study used a cross-sectional design and one was a modelling study. None of these studies used comparators of controls. These studies all have a high risk of bias and confounding, and their quantitative results are open to misinterpretation. However, they are useful for delineating the area (eg, the range of outcomes considered) and for demonstrating the feasibility of use of referral guidelines in practice.

Summary of themes from descriptive designs

Eight of the thirteen studies reported guidelines as beneficial for patient care. Outcomes investigated included appropriateness of referral (often assessed as compliance with guidelines), reduction in waiting times or costs, and change in referral rates. None of the descriptive studies reported on health outcomes.

Study findings—controlled designs

Quality assessment

Eleven studies used controlled designs. Six studies were cohort designs. In these studies, subjects were not randomly selected and outcome assessors were not blinded to intervention status. Almost all of these studies have a high risk of confounding and bias, and there is a significant risk that the relationships found are not causal. One²⁴ with concurrent (non-randomised) controls was carefully conducted to minimise bias. Of the five randomized control trials, four were cluster randomised and one was randomised at the individual level. All described clear eligibility criteria, comparability of baseline measures and described an intention-to-treat analysis. Only two^{15 18} described an adequate approach to sequence generation and allocation concealment and were therefore considered to have a low risk of bias.

General practitioners' knowledge of appropriateness of referral

None of the controlled studies reported on this outcome.

Effect of referral guidelines on appropriateness of referral

Two cohort studies reported overall improvement in referral appropriateness as a result of guideline implementation. One study in otorhinolaryngology (ENT)²¹ reported a statistically significant improvement in the proportion of appropriate referrals (assessed by an independent specialist) after the intervention.

Four studies reported on the impact of referral guidelines on the appropriateness of investigations carried out by general practitioners before making a referral.^{15 16 19 23} One cluster randomised trial, evaluating a “one-stop service” for urological conditions, reported a significant improvement of 0.5 in a “compliance score” (range 0–5, based on the number of guideline-recommended investigations carried out before referral).¹⁵ Both randomized control trials of use of a guideline plus structured management sheet for patients with infertility^{16 19} reported significant improvements in the appropriateness of diagnostic investigations undertaken before referral and in the patient histories recorded (72% vs 41% (n=100) correct assessment of day 21 progesterone levels (p<0.001))¹⁹ (mean number of relevant tests 2.81 vs 2.50; odds ratio 1.32; 95% CI 1.00 to 1.75; p=0.025).¹⁶ A cohort study with historical controls of a management guideline for patients with lower urinary tract symptoms also reported an increase in appropriate use of investigations. (Digital rectal examination (recommended in the guideline) increased from 32% to 41.1% (n=1203, p<0.001) and transrectal and suprapubic ultrasonography (not recommended in the guideline) decreased from 33% to 23% and 53% to 44% (p<0.001), respectively.)²³

All the controlled studies that provided evidence on this variable reported that referral guidelines improved the appropriateness of diagnostic evaluations carried out by general practitioners before referral.

We can draw no overall conclusions about the impact of guidelines on the likelihood of referral because studies used a variety of measures (both rates and numbers of referrals). Four studies reported a reduction in crude numbers, three reported no change and one suggested that guidelines would increase the referral rate. Two well-designed randomised trials reported no change in rates and one reported a reduction in referral rates in intervention as compared to control groups.

Five studies reported the impact of interventions on waiting times. A variety of start and end points were used. Periods of time measured were time from referral to first specialist consultation, time from referral to surgery and time from first general practitioner visit to final diagnosis; and different methods of measuring waiting time were used (mean waiting time and percentage of eligible patients seen by a specialist within a given time from the date of referral).

Three studies reported reductions in waiting times; one reported mixed results and one reported no change. As an example, the cohort study examining impact of a direct access to surgical waiting list intervention with referral guideline reported that for patients with hernia, the median wait for intervention patients was reduced by 91 days compared to the median wait for control patients (p<0.0001).²⁵

Impact of referral guidelines on cost and health outcomes

1. Cost No studies reported a formal economic evaluation. One cohort study with historical controls of a management guideline for patients with lower urinary tract symptoms reported a reduction in costs of between £1.45 and £49.54 per patient (at 2000 prices).²⁴ In contrast, a cluster randomised trial of a guideline plus structured management sheet for infertility reported a non-significantly higher median cost per referral in intervention practices (increase of 18%) £215–251 (at 1998 prices).¹⁶

2. Health outcomes Two studies reported explicitly on the impact of guidelines on patient health outcomes using the Short Form Health Survey (SF-36), a set of generic quality-of-life measures. Both reported no difference between intervention and control groups.

Discussion

Summary of results

We undertook a systematic review of guidelines for elective referral to surgical specialist. We identified 24 studies, of which only five used randomised designs, although a further six included data from comparison or control groups. Studies were based on data from large numbers of patients, from a number of different countries and covered a range of conditions. Most of the studies we found (21/24) reported evaluations of complex interventions that included an associated implementation or management plan. Of these, overall disease management guidelines and one-stop shop arrangements appear to confer benefits, although the absolute number of studies of each is small.

Studies reached contradictory conclusions regarding the effect of guidelines on general practitioners' knowledge and awareness of appropriateness of referral for the conditions under investigation. Controlled studies measuring compliance with guideline referral criteria reported improvements. All studies that assessed appropriateness of diagnostic evaluations carried out before referral after use of a referral guideline reported improvements.

We could draw no overall conclusions about the impact of referral guidelines on rates of referral based on the studies identified. Although many studies reported data on reduction of overall waiting times to surgery, attribution to referral guidelines of the effects found is problematic. No formal evaluations of costs and benefits of referral guidelines were found. Only two studies assessed patient outcomes, with no effects found.

Strengths and weaknesses

In this review we concentrated on guidelines for referral to surgical specialties by general practitioners or primary care practitioners for adults with non-urgent conditions. Strengths of the review are that unlike the two previously published reviews,^{5 6} we focused solely on referral guidelines for elective surgical assessment and we did not restrict our searches to randomised controlled trials. Extensive work went into developing a search strategy, which accurately covered the full range of terms used to describe the concept of “referral.” As a result we have identified a number of relevant studies internationally in addition to those covered in existing reviews. We did not estimate publication bias systematically, although it is noticeable that authors evaluating guidelines that they had produced themselves or that had been produced locally were more likely to be evaluated positively. Five of the included studies covered national guidelines produced from “elsewhere” and disseminated passively, and these tended to be evaluated less positively. Generalisability is an issue because all the included studies we found were undertaken in high-income countries and, of these, 18 of 24 were from English-speaking countries.

We did not synthesise results to produce overall quantitative conclusions on the benefits of referral guidelines because of the disparities in interventions, definitions, measures and outcomes used in the current body of research. Summary estimates of effect (if indeed they were possible to derive) would at this stage be misleading. Instead, we undertook a descriptive synthesis of the findings,³⁸ taking account of the relative weight of evidence using a recognised strength of evidence tool.¹³

Implications for policy, practice and research

Guidelines for referral for elective surgical assessment by primary care practitioners appear to improve appropriateness of referral by improving appropriateness of pre-referral diagnostic investigation. However, there is no consistent evidence for effects on other measures of appropriateness of referral or on health outcomes. There is conflicting evidence on the effect of these guidelines on costs. Notwithstanding these problems, guideline production has consistently been encouraged as one of the means to implement research findings into clinical practice. And elective surgical referral is a well-defined area where guidelines should be of value. Our findings suggest that referral guidelines on their own are unlikely to improve referrals especially those where dissemination is passive, and the findings of this review confirms previous reports.

However, our findings about different methods of active implementation are interesting. It is clear that those wishing to adopt referral guidelines need to adopt and tailor an associated implementation strategy, which is most suitable to their local circumstances.

All but a few of the studies of effectiveness of guidelines for elective surgical referral that we found were not of high methodological quality. High-quality evaluations of referral guidelines are still clearly needed. Other issues for further research include:

• investigation into the complex management intervention(s) that best support referral guideline use and compliance

• teasing out of the effects of bias because of local championing of local guidelines and the legitimate benefits of a local development approach

Guidelines may only ever be a part of the story, and there may also be a need for some “back to the drawing board” research into the best methods for ensuring that a rapidly developing evidence base can be incorporated into these important, nodal “gateway” decisions made by primary care practitioners in healthcare.