Influencing referral practice using feedback of adherence to NICE guidelines: a quality improvement report for dyspepsia

Abstract

Problem

Rising demand and increasing waiting times for upper gastrointestinal endoscopy (gastroscopy).

Design

Quality improvement study with pre‐ and post‐intervention data collection.

Setting

Three endoscopy units in two hospital trusts (Singleton, Morriston and Baglan Hospitals endoscopy units), UK.

Key measures for improvement

Number of gastroscopy requests from general practitioners (GPs) and hospital doctors; their adherence to dyspepsia referral guidelines and the referral‐to‐procedure interval for upper gastroscopy. Data collected for six months before and for five months after the intervention.

Strategy for change

Referrals were assessed against the National Institute for Health and Clinical Excellence (NICE) guidelines for the management of dyspepsia by two part‐time GPs and feedback sent to clinicians where requests did not adhere to the referrals criteria

Effects of change

Adherence to guideline criteria increased significantly among GPs after the intervention (from 55% to 75%). There was no similar effect for hospital doctors, although their adherence rate (70%) was at a higher level than that of GPs before the intervention. The number of gastroscopy referrals for dyspepsia declined after the intervention, particularly from hospital doctors where a drop of 31% was observed, from 26.6 to 18.4 referrals per week. With the inclusion of seasonal effects, an estimated drop of 3.2 referrals per week from general practice was not significant (p = 0.065) while an estimated drop of 10.0 referrals per week for hospital doctors was very significant (p<0.001).

Lessons learnt

Referral assessment can be successfully introduced and shows promise as a way of improving the quality of referrals and reducing demand. Hospital clinicians are more resistant than GPs to referral assessment but nevertheless responded to the feedback by reducing their endoscopy gastroscopy requests. Most such referrals are generated in hospitals rather than in primary care: this finding has important implications for demand management.

The three hospitals included in this quality improvement study were struggling to meet a rising demand for endoscopy, particularly for the investigation of upper gastrointestinal symptoms—a situation mirrored widely across the UK. This was leading to long waiting lists; in one of the endoscopy units the waiting time for a “non‐urgent” endoscopy was quoted to be in the region of 12 months. Reviews, supplemented by an individual patient date meta‐analysis, have shown that prompt endoscopy is not a cost‐effective strategy for the initial management of dyspepsia.¹,² In addition, the delay between referral and diagnostic procedure led to concerns that patients were not receiving a good service and that unacceptable delay in diagnosing problems such as oesophageal and gastric cancer would occur. The NHS Improvement Plan (June 2004) set a new overall goal for the NHS—that by March 2007, all scans and diagnostic procedures would have been accomplished within 13 weeks of general practitioner (GP) referral.³ When interviewed, the gastroenterologists performing the majority of the procedures felt that part of the problem was the lack of adherence to recognised guidelines. A specific example was the situation when endoscopy was requested for the investigation of dyspepsia in a clinical situation where guidelines, widely debated and released to the NHS by the National Institute for Health and Clinical Excellence (NICE),⁴ advised other measures, such as testing and eradication for H pylori.

Three endoscopy units were included in the study: two units, based at Morriston and Singleton hospitals, are part of the Swansea NHS Trust and the third unit, based at the Baglan Bay hospital, part of the Mid Glamorgan NHS Trust. The hospitals have 850, 600 and 270 beds respectively and the endoscopy units are managed by gastroenterologists, although surgeons also undertake endoscopies and arrange lists independently.

The focus of this study was on patients referred for an gastroscopy where the presenting complaint was dyspepsia. Although the evidence is sparse,⁵ there have been reports that referral assessment at the primary and secondary care interface can successfully influence both the quality and the volume of demand.⁶,⁷,⁸,⁹,¹⁰ We therefore designed a quality improvement intervention that aimed to introduce referral assessment in order to change the proportion of referrals that adhered to accepted guidelines, and to assess what impact this might have on demand for endoscopy and on the referral‐to‐procedure interval.

Key measures for improvement

We chose three metrics for the evaluation of the proposed intervention:

1. Adherence to NICE guidelines for the referral of patients with dyspepsia, by GPs and doctors working in the hospitals.

2. The number of referrals received for gastroscopy.

3. The referral‐to‐procedure interval (in days) for gastroscopy at the three endoscopy units.

Process of gathering information: methods used to assess problems

We conducted an uncontrolled before and after study. Copies of all requests for upper gastrointestinal endoscopy (irrespective of clinical problem) were collected at the three endoscopy units for six months before the intervention start date (1/11/04) and for five months afterwards. Data from all the referrals were recorded on a database and coded according to the problems described. Two GPs were employed on a part‐time basis to judge whether the requests for gastroscopy adhered to the NICE referral guidelines,⁴ using a standardised protocol (available from the author). The interval, between the date on the referral letter and the date of the endoscopy was calculated in days in order to assess the interval between request and procedure.

Analysis and interpretation

We analysed the data as follows:

1. Logistic regression modelling was used to analyse the data regarding adherence to guideline criteria, using period of referral, origin of the referral (GP and hospital doctors) as qualitative predictor variables and time as a quantitative predictor variable. While the data we report have been aggregated across the three units, we also conducted the same analysis at the level of the individual endoscopy units.

2. Poisson regression models were applied to analyse weekly rates of referrals for upper gastroenterological endoscopy by their origin (general practice or hospital clinician) where dyspepsia was the prime reason, and also all referrals, irrespective of presenting problem). By including the month of referral as a qualitative variable, seasonal effects were estimated and controlled.

3. The interval, in days, between the referral date and the date when endoscopy was performed (referral‐to‐procedure interval) and a two‐sample z‐statistic for the difference in means between the pre‐ and post‐intervention periods calculated.

Preliminary analysis revealed that the majority of referrals for upper gastroenterological endoscopy originated in hospitals and that multiple lists for gastroscopy were in existence—that is, surgeons commonly conducted independent gastroscopy lists, making it difficult to introduce standardised pathways to handle referrals within a provider organisation.

Strategy for change

The intervention was composed of the following elements. In October 2004, a letter was sent to all 215 GPs in the catchment area of the three endoscopy units. The same letter was also sent to the 359 hospital consultants and post holders at junior grades based at the three hospitals. The letter stated that as from 1 November 2004 two GPs would be employed part‐time to assess all endoscopy letters and that referrals where the prime problem was dyspepsia would be judged against the recently issued NICE guidelines for the management of dyspepsia in primary care. The letter went on to say that where referrals did not adhere to the guidelines, the referring doctor would be informed by letter, giving the reason for the non‐adherence accompanied by a synopsis of the relevant guidance. Requests for gastroscopy where dyspepsia was not the stated clinical problem were recorded but analysed separately. The correspondence made it clear however that although all referrals for endoscopy were assessed by a clinician there would be no interference to the referral pathway. Referrals were therefore processed in the usual way and no delay or diversion was instigated. An All Wales Dyspepsia Management Guideline, closely modelled on the NICE and SIGN guidelines,⁴,¹¹ had been circulated to all clinicians in NHS Wales two weeks before the start of the intervention. This intervention was in effect occurring against a background of increased awareness that there were changes in the criteria for what was considered “appropriate” referral for gastroscopy, and a general awareness that endoscopy waiting times were increasing.

Effects of change strategy

1. Adherence to NICE guidelines

Aggregated measures over the three units showed that GPs increased their adherence rates from a mean 55% before the intervention to 75% during the intervention period (95% confidence interval (CI) 13.6% to 26.4%, p<0.001). This was a significant period effect (p<0.001). This change for GPs was observed at all three endoscopy units (Singleton 52% to 71%, Morriston 66% to 80%, Baglan 52% to 71%). The mean adherence rate for doctors working in hospital was higher than that for GPs and stood at 70%. Although there appears to be no step‐change corresponding to the period change, there appears to be a significant trend upwards in the adherence rate for hospital doctors after the intervention started. Figure 1 illustrates the rates of adherence for the time period and fitted models.

Figure 1 Adherence of gastroscopy referrals to guidance: weekly proportions (%).

2. Numbers of referrals for gastroscopy to investigate dyspepsia

A decline in the number of referrals for gastroscopy was observed in each of the three units. Figure 2 shows weekly referral totals for GPs and hospital doctors (dyspepsia referrals and all referrals) and include 12‐week moving averages. For dyspepsia referrals, a decline in average weekly referral rate from 13.3 to 12.1 referrals per week for GPs and 26.6 to 18.4 referrals per week for hospital doctors (or the equivalent figures 14.5 to 14.1 and 51.9 to 51.0 for all referrals) may be confounded by seasonal effects. To control for this, a monthly categorical predictor variable was introduced and Poisson regression models were fitted to the data. For the dyspepsia referrals, an estimated reduction of 3.2 referrals per week for GPs was not significant (p = 0.065), while for hospital doctors, the corresponding estimated decrease of 10 referrals per week was very significant (p<0.001). For all referrals, there was no change in the weekly rate across the two time periods for GPs or hospital doctors. To summarise, the greatest impact on referral volume is on the number of referrals from hospital doctors for gastroscopy to investigate dyspepsia, a drop from 26.6 to 18.4 referrals per week, a decrease of 31%.

Figure 2 Referrals requesting gastroscopy: weekly totals.

3. The referral‐to‐procedure intervals

The data show that the intervention, although focused on referral for dyspepsia, had the effect of significantly reducing the referral‐to‐procedure interval for gastroscopy. The mean interval in the pre‐intervention period (for all referrals) was 52.1 (sample size 1188, SD 67.9) days, compared to a mean interval of 39.4 (sample size 612, SD 46.2) days in the post‐intervention period. A two sample z‐statistic for this difference in the means is 4.14, p<0.001. The 95% CI for the mean difference is 6.6 to 18.6 days.

Next steps

The results show that it is feasible to employ GPs to assess the referrals of their colleagues and to send feedback to individual clinicians where requests do not meet guidelines. Although we do not have control data, the results show a significant change in adherence rates by GPs after the intervention was introduced, which was sustained over our data collection period, and occurred in all three endoscopy units. We also observed a 31% drop in the referrals from hospital doctors for dyspepsia overall, with an estimated drop of 10 referrals per week when allowing for seasonal changes. We consider these to be important, if preliminary, findings that should be investigated further by a controlled trial.

The intervention provoked resistance from some clinicians: we received 22 letters, 21 from specialists (14 letters from 7 surgeons, and 7 letters from 5 physicians) and one from a GP. These letters were critical of the referral assessment strategy. The strongest concern was a perceived erosion of clinical freedom. This view was also commonly associated with an outright disagreement with the NICE guidelines,⁴ often viewing them as mechanisms to “ration” services. A few noted that the guidance used in the study was developed for primary care, and not for dyspepsia occurring in hospital settings.

We have learnt the following lessons. Firstly, although referral assessment can be introduced elements of resistance will emerge. Secondly, the method shows promise as a way to improve referral quality, reduce demand and reduce the referral‐to‐procedure interval. Thirdly, 1720 patient beds in three hospitals and 359 hospital doctors generated more gastroscopy referrals per month than 215 GPs serving a population of approximately 360 000 patients, an underresearched area of clinical practice.¹² Any system wishing to introduce demand management in the area of endoscopy referrals will need to consider the demand generated by the total healthcare system and not just referrals from primary care.¹³

Key messages

• Clinician referral assessment and feedback can be successfully introduced where it does not threaten clinical processes.

• Providing feedback from referral assessment shows promise as a way to both improve the quality of referrals from primary care and to reduce demand in general.

• There is a greater demand for gastroscopy in hospitals than in primary care. Demand management systems need to consider the work generated by secondary care and not just referrals from primary care.

• The results of this before and after study justify further investigation in the form of a randomised controlled trial.