Abstract
Background
Ninety-one per cent of primary care trusts were using some form of referral management in 2009, although evidence for its effectiveness is limited.
Aim
To assess the impact of three referral-management centres (RMCs) and two internal peer-review approaches to referral management on hospital outpatient attendance rates.
Design and setting
A retrospective time-series analysis of 376 000 outpatient attendances over 3 years from 85 practices divided into five groups, with 714 000 registered patients in one English primary care trust.
Method
The age-standardised GP-referred first outpatient monthly attendance rate was calculated for each group from April 2009 to March 2012. This was divided by the equivalent monthly England rate, to derive a rate ratio. Linear regression tested for association between the introduction of referral management and change in the outpatient attendance rate and rate ratio. Annual group budgets for referral management were obtained.
Results
Referral management was not associated with a reduction in the outpatient attendance rate in any group. There was a statistically significant increase in attendance rate in one group (a RMC), which had an increase of 1.05 attendances per 1000 persons per month (95% confidence interval = 0.46 to 1.64; attendance rate ratio increase of 0.07) after adjustment for autocorrelation. Mean annual budgets ranged from £0.55 to £6.23 per registered patient in 2011/2012. RMCs were more expensive (mean annual budget £5.18 per registered patient) than internal peer-review approaches (mean annual budget £0.97 per registered patient).
Conclusion
Referral-management schemes did not reduce outpatient attendance rates. RMCs were more expensive than internal peer review.
Keywords: general practice, outpatients, primary care, referral, referral management centre
INTRODUCTION
In the English NHS, total outpatient attendances from all referral sources increased by 35%, from about 50 000 000 in 2005/2006 to 67 000 000 in 2009/2010 and annual growth slowed to 4% in the following year. Overall, 52% of outpatient attendances were initiated as referrals from GPs in 2011/2012.1 Variation in GPs’ referral behaviour can be considerable,2 and GP referral rates to psychiatry have been shown to vary by up to 10-fold between practices.3 In allergy services, application of agreed referral criteria led to rejection of 48% of referrals.4 Thirty-seven per cent of non-urgent cancer referrals were assessed as being unnecessary according to published guidelines,5 while 27% of orthopaedic referrals would have been more appropriate for rheumatology.6 There is also evidence suggesting that, in certain specialties, some patients are not referred until their condition has reached an advanced stage.7–9
Referral-management schemes have been a frequent response to problems of rising outpatient attendances and the associated cost, growth of specialty waiting lists, inappropriate referral, and variation in clinicians’ referral rate and quality.10–12 Ninety-one per cent of primary care trusts were using some form of referral management in September 2009.13 Typically, referral management aims either to reduce the number of referrals by influencing GPs’ decision to refer, or to influence the referral destination, or to improve referral quality and appropriateness. Approaches to referral management range from active interventions, such as referral-management centres (RMCs) that receive referrals and review their quality and appropriateness, to passive methods, such as decision-support tools that aim to alter referral behaviour.13
Two reviews found limited evidence that peer review, when combined with specialist feedback and evidence-based referral guidelines, are effective in changing referral behaviour to single specialties.13,14 No systematic evaluations of RMCs that manage referrals for all specialties have been published.13,15,16 and there is a lack of studies that describe the impact of referral-management schemes on hospital outpatient attendances.
Between 2006 and June 2011, 85 practices in NHS Norfolk, with 714 000 registered patients, formed five groups to manage referrals. Three groups established RMCs. Two groups carried out internal peer review of referrals. This study aimed to establish whether the introduction of RMCs or internal peer review in Norfolk was associated with a reduction in hospital all-specialty outpatient attendance rates.
How this fits in
Ninety-one per cent of primary care trusts in England were using some form of referral management in 2009, yet the evidence for its effectiveness is limited. The impact of different referral-management approaches, such as internal peer review or referral-management centres, on hospital outpatient attendance rates for all specialties is poorly understood. The introduction of referral management in each of five general practice groups failed to reduce the all-specialty hospital outpatient attendance rate during 9 to 21 months of follow-up observation. Higher-cost referral-management centres were no more effective in reducing outpatient attendance rates than lower-cost approaches, and in one group using a referral-management centre, the hospital outpatient attendance rate significantly increased after referral management began.
METHOD
The study had a retrospective before and after design. The study population was all patients registered with general practices in each of the five groups. The primary outcome was the directly age-standardised monthly rate of first outpatient attendances referred by GPs for any speciality in NHS hospitals. Outpatient activity for learning disability, occupational therapy, and obstetrics was excluded from the analysis and also by RMCs, as, in Norfolk, referral to these specialties is not primarily by GPs.
Data sources
NHS Norfolk provided outpatient hospital episode statistics, numbers of GP-registered patients in 5-year age bands, and budgets for referral-management groups. The NHS Information Centre supplied equivalent outpatient attendances for England. Crude rates of new GP registrations from 2007/2008 to 2009/2010 for people with immigrant status, and population estimates for England, stratified by age, were obtained from the Office for National Statistics. Mean monthly waiting times for all-specialty non-admitted patient appointments for the period of study were extracted from Department of Health Referral to Treatment statistics for the two main acute providers in this study, and for England, to use as a reference. To define referral-management approaches, each group was asked to complete a short questionnaire; this was followed by a visit in February 2012.
Statistical analysis
The primary outcome, calculated for each group and for England from April 2009 to March 2012, was a directly age-standardised GP-referred first outpatient attendance rate per 1000 persons per month, using 5-year age bands as denominators. The English population was used as the age standard. Population data were only available quarterly, so the number of persons in each group’s 5-year age band for each month was estimated using linear regression fitted to 11 quarterly extracts of the number of registered patients. Group attendance rates were adjusted to the mean number of working days per month for the period of analysis. In Figure 1, rates have been smoothed using the 3-month rolling mean.
Figure 1.
Directly age standardised GP-referred first outpatient attendance rates as a 3-month rolling mean for Norfolk referral-management groups and England, per 1000 persons per month from April 2009 to March 2012. (The x axis label shows alternate month and year, for example 5 2009 = May 2009, 7 2009 = July 2009). The change from dashed to solid marker lines indicates the start of referral management in each group. Group referral-management approaches are abbreviated to internal peer review (IPR) and referral-management centre (RMC).
An attendance rate ratio was calculated as a secondary outcome, by dividing each group’s attendance rate by the equivalent England attendance rate, for every month.
Linear regression was used to test whether the introduction of referral management in each group was associated with a change in attendance rate (the dependent variable). Independent variables for each group were calendar month (representing monthly trend) and whether referral management had started (start of referral management). Because RMCs began incrementally, as recruitment of all practices took up to 5 months, the independent variable for the start of referral management represented the cumulative monthly proportion of practices that were participating in referral management for each group. Linear regression was used to test whether the monthly trend was significant for each group.
Autocorrelation effects such as seasonality frequently occur in time-series data and can affect data interpretation. Attendance rates for each group were tested for autocorrelation, by using white-noise tests of the residuals from linear regression models, which included the start of referral management. If significant autocorrelation was found, then a Box–Jenkins autoregressive moving-average ARIMA model that corrected for autocorrelation was used. Further white-noise tests of ARIMA model residuals were used to check whether autocorrelation correction was effective. Assumptions of normality were checked from visual inspection of histograms. The same approach was used for the secondary outcome.
Because concurrent changes in outpatient waiting lists could potentially have confounded the effect of referral management on outpatient attendances, linear regression was used to test whether the mean monthly waiting times for non-admitted appointments for England and for the two main acute trusts used by the five groups had changed during the period of analysis. Statistical analysis was conducted with STATA SE 11.1, using a 5% significance level.
RESULTS
Characteristics of interventions
The original primary goal of groups 3, 4, and 5 (using RMCs) and group 1 (using internal peer review) was to reduce the number of GP-referred first outpatient attendances across all specialties to 2008/2009 levels. Group 2 (using internal peer review) aimed to reduce the number of inappropriate referrals, but did not define criteria for inappropriateness. Two RMCs (groups 3 and 4) reviewed all referrals, and the other RMC (group 5) reviewed referrals to seven specialties (Table 1). Both groups using internal peer review (groups 1 and 2) reviewed all referrals to selected specialties, which were identified by each group from high numbers of referrals or variation in practice referral rate, or because of clinical concerns about the service pathways.
Table 1.
Characteristics of referral management in each group
| Group | Referral-management approach | Key interventions | Number of practices and registered population | Allocated budget; budget per registered patient (2011/2012) | Recruitment of first practice; length of follow-up |
|---|---|---|---|---|---|
| 1 | Internal peer review | Monthly peer review of selected referrals; use of guidelines; monthly feedback reports to referring GPs; pathway redesign evidence | 2 practices; 33 837 | £85 946; £2.54 | April 2011; 12 months |
| 2 | Internal peer review | Monthly peer review of selected referrals; use of guidelines; monthly feedback reports to referring GPs; pathway redesign evidence | 23 practices; 206 005 | £113 303; £0.55a | June 2011; 9 months |
| 3 | Referral-management | Creation of referral database; clinical and centre non-clinical review of all referrals; use of guidelines; 6-monthly feedback reports to referring clinician; pathway redesign evidence; booking of secondary care patient appointments for practices | 20 practices; 166 917 | £1 039 892; £6.23 | August 2010; 20 months |
| 4 | Referral-management | Creation of referral database; clinical and centre non-clinical review of all referrals; pathway redesign evidence; evidence-based education for GPs; bimonthly feedback reports to referring clinician | 18 practices; 146 928 | £900 670; £6.13 | July 2010; 21 months |
| 5 | Referral-management | Creation of referral database; clinical and centre non-clinical review of referral to seven specialties; pathway redesign evidence; 6-monthly feedback reports to referring clinician; booking of secondary care patient appointments for practices | 22 practices; 159 824 | £514 634; £3.22 | April 2011; 12 months |
Allocated budget for 2010/2011.
Referral-management activities included the creation of databases holding referral data; review of referrals by clinicians and non-clinicians; education and specialist feedback to referring GPs; collation of evidence for pathway redesign; and booking of secondary-care patient appointments on behalf of practices, using a national computer booking system. No group carried out all of these activities. RMC interventions were more complex and involved a wider range of activities than internal peer-review interventions. Referral-management schemes did not take clinical responsibility for patients.
The intensity of the GP behaviour-change component varied between groups. Among the RMCs, group 4 provided bimonthly feedback from clinical reviewers to GPs on their referral behaviour, whereas groups 3 and 5 provided feedback reports to GPs every 6 months. Both groups using internal peer review provided monthly feedback reports for referring GPs. These were written by a referral review team from a separate practice, which included a GP (group 1), or by a pair of GPs with relevant specialist knowledge (group 2).
There was more than tenfold variation in annual group budgets, ranging from £0.55 to £6.23 per registered patient in 2011/2012. Budgets were larger in groups with a wider ranges of activities. Groups using RMCs had a mean budget of £5.18 per registered patient, compared with £0.97 for groups using internal peer review. The periods of observation after the introduction of referral management in each group ranged from 9 to 21 months. Two NHS trusts were the main acute-care providers to the five groups.
Outpatient attendance rates
Attendance rates in groups 1, 2 (internal peer review), and 4 (RMC) remained below that of England during the period of study (Figure 1). The attendance rate for group 5 (RMC) remained above the rate for England, except for July 2010. After May 2011 the rate for group 3 (RMC) exceeded that of England, except for November 2011.
The monthly trend by calendar month was not significant at the 5% level and was excluded from all regression models. Significant autocorrelation was found, and corrected for, in all groups except for group 4. White-noise test of ARIMA model residuals showed that no significant autocorrelation remained after autocorrelation correction.
Four groups (1, 2, 3, and 4) showed statistically significant increases in attendance rate (primary outcome), ranging from 0.41 to 1.20 attendances per 1000 persons per month before autocorrelation correction (Table 2). After correcting for significant autocorrelation in four groups (1, 2, 3, and 5), the group effect size estimates were similar but the confidence intervals widened so that only group 3 (RMC) remained statistically significant (1.05 attendances per 1000 persons per month). Without autocorrelation correction, group 4 (RMC) showed a significant increase of 0.41 attendances per 1000 persons per month).
Table 2.
Changes in attendance rate (primary outcome) after the introduction of referral management in each group: linear regression and ARIMA regression with autocorrelation correction
| Group | Referral-management approach | Mean attendance rate before start of referral management per 1000 persons per month | Mean attendance rate after start of referral management per 1000 persons per month | Change in attendance rate (95% CI) associated with start of referral management per 1000 persons per montha | Change in attendance rate (95% CI) associated with start of referral management (ARIMA) per 1000 persons per montha |
|---|---|---|---|---|---|
| 1 | Internal peer review | 14.9 | 15.6 | 0.76c (0.12 to 1.40) | 0.72 (–0.18 to 1.61) |
| 2 | Internal peer review | 14.0 | 15.2 | 1.20c (0.60 to 1.80) | 0.80 (–0.11 to 1.71) |
| 3 | Referral-management centre | 15.1 | 16.1 | 1.10c (0.67 to 1.53) | 1.05c (0.46 to 1.64) |
| 4b | Referral-management centre | 14.3 | 14.6 | 0.41c (0.02 to 0.79) | |
| 5 | Referral-management centre | 17.0 | 17.3 | 0.27 (–0.36 to 0.90) | 0.27 (–0.37 to 0.90) |
aMay not equal the difference between mean attendance rate before and after start of referral management because of the effect of the incremental introduction of referral management.
Autocorrelation was not significant in group 4; therefore, an ARIMA model was not used in this group.
Statistically significant at 5% level.
Four groups showed statistically significant increases in attendance rate ratio (secondary outcome), ranging from 0.03 to 0.08 before autocorrelation correction (Table 3). This means that after referral management began, group attendance rates increased by 3% to 8% more than the England attendance rate. Group 3 (RMC) showed an attendance rate ratio increase that was 7% more than that for England, after correcting for autocorrelation. Without autocorrelation correction, group 4 (RMC) showed a significant increase of 3% more than England.
Table 3.
Changes in attendance rate ratio (secondary outcome) after the introduction of referral management in each group: linear regression and ARIMA regression with autocorrelation correction
| Group | Referral-management approach | Mean attendance rate before start of referral management per 1000 persons per month | Mean attendance rate after start of referral management per 1000 persons per month | Change in attendance rate ratio (95% CI) associated with start of referral management per 1000 persons per montha | Change in attendance rate ratio (95% CI) associated with start of referral management (ARIMA) per 1000 persons per montha |
|---|---|---|---|---|---|
| 1 | Internal peer review | 0.95 | 0.99 | 0.04c (0.00 to 0.08) | 0.04 (–0.00 to 0.08) |
| 2 | Internal peer review | 0.89 | 0.97 | 0.08c (0.04 to 0.13) | 0.06 (–0.06 to 0.18) |
| 3 | Referral-management centre | 0.96 | 1.02 | 0.07c (0.05 to 0.10) | 0.07c (0.05 to 0.10) |
| 4b | Referral-management centre | 0.90 | 0.93 | 0.03c (0.01 to 0.06) | |
| 5 | Referral-management centre | 1.08 | 1.10 | 0.03 (–0.02 to 0.07) | 0.27 (–0.03 to 0.09) |
May not equal the difference between mean attendance rate before and after start of referral management because of the effect of the incremental introduction of referral management.
Autocorrelation was not significant in group 4; therefore, an ARIMA model was not used in this group.
Statistically significant at 5% level.
Mean all-specialty non-admitted patient waiting times for the two acute trusts were 3.6 and 4.7 weeks (4.1 weeks for England). There was no significant trend in mean monthly waiting times for either trust, or for England.
DISCUSSION
Summary
The introduction of referral management was not associated with a reduction in the hospital outpatient attendance rate or attendance rate ratio in any group. Group 3, which had the largest budget and used a RMC to carry out the widest range of activities, showed a statistically significant increase in both measures.
Strengths and limitations
To the authors’ knowledge, this is the first study to measure the effects of RMCs on NHS hospital outpatient attendance rates for all specialties. By calculating outcome measures common to all groups and to England, it was possible to measure the effect of referral management locally and determine whether the rate of hospital outpatient attendance was reduced, compared with England. This study involved data from a large population in Norfolk, which collectively attended 376 000 outpatient appointments over 3 years.
Previous studies of referral-management effects have commonly used data on referral rather than outpatient attendance.2–5,12,14 However, referral rates are unsuitable for comparing the effects of referral-management schemes because they are not provided to an NHS minimum data standard, resulting in inconsistent data collection and reporting between data providers over time. Unlike referrals, changes in outpatient attendance have a direct consequence on health-resource consumption.
This study used a retrospective observational design and consequently it was not possible to control for all potentially confounding variables over time. The analysis enabled comparison of changes within and between groups, while accounting for changes in practice list sizes and age distributions over time.
The duration of follow-up, which ranged from 9 to 21 months, should be sufficient to observe the effects of referral management, as changes in GP-referral behaviour would not take more than 18 weeks to affect outpatient attendances under current NHS Referral to Treatment standards. Linear regression results showed that there were no significant trends in mean monthly waiting times for non-admitted appointments, so waiting list changes probably did not affect attendance rates in this study. The authors are not aware of population changes in Norfolk that have increased the need for health services. GP registration rates for people with immigrant status remained lower in Norfolk than in England from 2007/2008 to 2009/2010.
Strategies to reduce over-referral could also potentially reduce under-referral of other patients, and thus limit the potential for reducing overall demand.13 It was not possible to measure referral appropriateness or quality, owing to the lack of agreed standards, although group 2 primarily aimed to reduce inappropriate referrals rather than the number of outpatient attendances. The study did not assess attendance rates for specific specialties, as none of the groups aimed to affect a single specialty. Although referral-management schemes are widespread across England, the authors do not know when individual schemes were introduced.
Referral-management budgets could potentially be misleading. It is not known if all budgets were spent or how budgets were utilised. Also, some of the activities for which RMCs were funded were not directed at reducing outpatient attendances. For example, two RMCs administered booking of secondary care patient appointments on behalf of practices using the national IT Choose and Book system, but they were unable to report what proportion of their budget was allocated to this activity.
A prospective study was not possible in this case, because referral-management schemes were operating before the evaluation started, and would not, in any case, have changed the way in which outpatient attendances were routinely recorded.
Comparison with existing literature
A few studies suggest that a combination of peer review, use of structured guidelines, and specialist feedback can be effective in reducing referral rates when applied to either single or small numbers of specialties.13,14,17 Elsewhere, referral management has reduced referral rates when applied to 10 specialties, using a complex intervention including peer review, reporting, and benchmarking of referral rates and specialist feedback.18 However, the intervention was unsuccessful in one specialty when specialist feedback was excluded. The relative importance of each specific component in effective referral management is not well understood.
Previous research suggests that more active forms of referral management such as RMCs are less likely to offer value for money.13,16 The present study found that the effect of RMCs, which were the most expensive intervention type, was no better than lower-cost internal peer review. Of the five approaches studied, the least effective was a higher-cost RMC.
Implications for research and practice
The lack of effect found by this study suggests that a cautious approach should be taken to adopting referral management, particularly if undertaking a wide range of activities across multiple specialties. Given the widespread use of referral management, evaluation is needed to understand the relative importance of each component in referral-management interventions, if referral management is to be applied effectively to reduce outpatient attendances.
The introduction of referral management was not associated with a reduction in the outpatient attendance rate or attendance rate ratio in any group. Despite larger budgets, RMCs were no more effective in reducing the attendance rate than internal peer-review approaches. This observational study from one region showed that referral-management schemes are poor value. However, these findings need confirmation from larger randomised studies using robust methods.
Acknowledgments
We would like to thank the members of all referral management groups involved in this study, the Business Intelligence Team (NHS Norfolk & Waveney), Dr Carolyn Pegg (GP trainee), Ian Ayres (Deputy Chief Executive, NHS Norfolk & Waveney), Dr Jenny Harries (Director of Public Health, Norfolk County Council), and the Eastern Region Public Health Observatory for supporting this study.
Funding
No specific funding was provided for this study.
Ethical approval
Ethical approval for this study was sought from the University of East Anglia Faculty Research Ethics Committee (reference 2011/2012-26 NHS Service Evaluation of Referral Management in Norfolk & Waveney) and NHS Norfolk and Waveney Research Governance Office. In both instances, the study was classified as service evaluation and ethical approval was not required.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss
REFERENCES
- 1.NHS Information Centre Hospital episode statistics for England Outpatient statistics. http://www.hesonline.nhs.uk/Ease/servlet/ContentServer?siteID=1937&categoryID=890 (accessed 2 May 2013).
- 2.Creed F, Gowrisunkur J, Russell E, Kincey J. General practitioner referral rates to district psychiatry and psychology services. Br J Gen Pract. 1990;40(340):450–454. [PMC free article] [PubMed] [Google Scholar]
- 3.Ashworth M, Clement S, Sandhu J, et al. Psychiatric referral rates and the influence of on-site mental health workers in general practice. Br J Gen Pract. 2002;52(474):39–41. [PMC free article] [PubMed] [Google Scholar]
- 4.Kaminski E, Jones D, Bethune CA, Jones R. A successful demand management initiative of referrals to a regional allergy clinic. Clin Exp Allergy. 2011;41(12):1844. [Google Scholar]
- 5.Patel NN, D’Souza J, Rocker M, et al. Prioritisation of vascular outpatient appointments cannot be based on referral letters alone. Surgeon. 2008;6(3):140–143. doi: 10.1016/s1479-666x(08)80108-4. [DOI] [PubMed] [Google Scholar]
- 6.Speed CA, Crisp AJ. Referrals to hospital-based rheumatology and orthopaedic services: seeking directions. Rheumatology. 2005;44(2):469–471. doi: 10.1093/rheumatology/keh504. [DOI] [PubMed] [Google Scholar]
- 7.Roderick P, Jones C, Drey N, et al. Late referral for end-stage renal disease: a region-wide survey in the south west of England. Nephrol Dial Transplant. 2002;17(7):1252–1259. doi: 10.1093/ndt/17.7.1252. [DOI] [PubMed] [Google Scholar]
- 8.Clark MA, Thomas JM. Delay in referral to a specialist soft-tissue sarcoma unit. Eur J Surg Oncol. 2005;31(4):443–448. doi: 10.1016/j.ejso.2004.11.016. [DOI] [PubMed] [Google Scholar]
- 9.Khattak I, Eardley NJ, Rooney PS. Colorectal cancer — a prospective evaluation of symptom duration and GP referral patterns in an inner city teaching hospital. Colorectal Dis. 2006;8(6):518–521. doi: 10.1111/j.1463-1318.2006.00967.x. [DOI] [PubMed] [Google Scholar]
- 10.Roland M, McDonald R, Sibbald B. Outpatient services and primary care A scoping review of research into strategies for improving outpatient effectiveness and efficiency. Manchester: National Primary Care Research and Development Centre and Centre for Public Policy and Management of the University of Manchester; 2006. [Google Scholar]
- 11.Coleman A, Harrison S, Checkland K, Hiroeh U. Practice based commissioning: report of a survey of primary care trusts 2007. Manchester: National Primary Care Research and Development Centre; 2007. [Google Scholar]
- 12.Foot C, Naylor C, Imison C. The quality of GP diagnosis and referral. London: The King’s Fund; 2010. [Google Scholar]
- 13.Imison C, Naylor C. Referral management: lessons for success. London: The King’s Fund; 2010. [Google Scholar]
- 14.Akbari A, Mayhew A, Al-Alwai MA, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev. 2008;(4):CD005471. doi: 10.1002/14651858.CD005471.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Davies M, Elwyn G. Referral management centres: promising innovations or Trojan horses? BMJ. 2006;332(7545):844–846. doi: 10.1136/bmj.332.7545.844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.CRG Research/Cardiff University . Referral management pilots in Wales — follow up review Final report. Cardiff: National Leadership and Innovation Agency for Healthcare; 2007. [Google Scholar]
- 17.Evans E. The Torfaen Referral Evaluation Project. Qual Prim Care. 2009;17(6):423–429. [PubMed] [Google Scholar]
- 18.Evans E, Aiking H, Edwards A. Reducing variation in general practitioner referral rates through clinical engagement and peer review of referrals: a service improvement project. Qual Prim Care. 2011;19(4):263–272. [PubMed] [Google Scholar]