Abstract
Objective
To measure the cost effectiveness of systematic photographic screening for sight threatening diabetic eye disease compared with existing practice.
Design
Cost effectiveness analysis
Setting
Liverpool.
Subjects
A target population of 5000 diabetic patients invited for screening.
Main outcome measures
Cost effectiveness (cost per true positive) of systematic and opportunistic programmes; incremental cost effectiveness of replacing opportunistic with systematic screening.
Results
Baseline prevalence of sight threatening eye disease was 14.1%. The cost effectiveness of the systematic programme was £209 (sensitivity 89%, specificity 86%, compliance 80%, annual cost £104 996) and of the opportunistic programme was £289 (combined sensitivity 63%, specificity 92%, compliance 78%, annual cost £99 981). The incremental cost effectiveness of completely replacing the opportunistic programme was £32. Absolute values of cost effectiveness were highly sensitive to varying prevalence, sensitivity and specificity, compliance, and programme size.
Conclusion
Replacing existing programmes with systematic screening for diabetic eye disease is justified.
Introduction
Various methods of screening for diabetic eye disease have been tested in recent years,1–12 but few studies have produced meaningful cost effectiveness data. A three centre study commissioned by the Department of Health reported relatively high costs per case of diabetic eye disease detected.1,2 Similar data are also available from the United States.5 The Department of Health study was undermined by suboptimal screening methods, and both studies disregarded the effect of pre-existing opportunistic screening on cost effectiveness. Foulds et al studied the potential savings of systematic screening, but the cost effectiveness data are difficult to verify in the absence of sensitivity data.13 Mathematical modelling has also been used to study the potential economic benefits of screening.4,6,14
Nationally coordinated screening of diabetic patients for sight threatening eye disease is being considered as part of the national service framework on diabetes, which is due to be published in spring 2001, and an economic evaluation of a programme with high sensitivity and specificity and known prevalence and compliance is therefore urgently needed.15 The Liverpool diabetic eye study was established in 1991 to investigate the efficacy of primary care based photographic screening for sight threatening eye disease and to set up a systematic service replacing the existing opportunistic programme. We present a detailed cost effectiveness analysis of the systematic and opportunistic programmes and the effect of varying disease prevalence, compliance, and sensitivity and specificity to allow generalisation of our results throughout the NHS.
Methods
The systematic screening programme uses a mobile screening unit that visits inner city general practices together with a dedicated hospital assessment clinic.16 Screening comprises three-field, non-stereoscopic photography using mydriasis; 35 mm transparencies; and validated grading. The pre-existing opportunistic service used direct ophthalmoscopy and was performed by general practitioners, optometrists, and diabetologists. There was no systematic training, central coordination, or audit, and patients with positive results were assessed in general hospital eye service clinics.
The outcome measure was the detection of sight threatening eye disease, defined as any of the following: moderate preproliferative retinopathy or worse; circinate exudates within the macula; any exudate within 1 disc diameter of the foveola; other diabetes related disease such as vascular occlusion.
Source data
Data for this analysis were taken from two studies within the Liverpool diabetic eye study. The first was a cross sectional observational study of 320 diabetic patients registered with four general practices who were examined by a consultant ophthalmologist specialising in medical retinal diseases using slit-lamp biomicroscopy (an accepted reference standard for determining need for treatment).8,16 The second study comprised an analysis of the implementation of systematic screening in Liverpool and included a structured, closed response questionnaire administered by trained observers to the first 1363 diabetic patients recruited.17 These two studies provided all data except the specificity of the opportunistic programme, which was calculated from a previous study.1 We adopted a health service perspective for measurement of costs and benefits.
Four key data variables are necessary to determine overall effectiveness in any screening programme: disease prevalence; compliance; sensitivity and specificity of the screening method; and cost. The disease prevalence applied across both the systematic programme and the opportunistic programme, but the other variables were analysed separately for the two programmes. The overall baseline prevalence of sight threatening eye disease in Liverpool was calculated as 14.1% from the cross sectional study.
Compliance with systematic screening during data collection (1995-6) was 80%. The Liverpool study has a current contracted activity of 4000 screen events a year. To achieve this with a compliance of 80%, 5000 screening invitations need to be sent. The cross sectional study showed that the systematic programme had a sensitivity of 89% (95% confidence interval 80% to 98%) and specificity of 86% (82% to 90%).16
Compliance for opportunistic screening was calculated from the questionnaire; 78% (1059/1363) reported screening in the 12 months before attending for systematic screening.16 To accurately compare systematic with opportunistic screening the costs and detection rates were also based on 5000 invitations, but a 78% compliance produces only 3900 screen events. The sensitivity of opportunistic screening was calculated from the cross sectional study and the questionnaire data.8 Seven per cent (22/320) of patients were already under the care of an ophthalmologist for eye disease detected opportunistically. This number was divided by the prevalence (14.1%) to give 49% of patients, and a sensitivity of 63% was derived by dividing by the proportion who reported opportunistic screening in the previous year (49/78).
The specificities for the various health professionals performing opportunistic screening were taken from Buxton et al (general practitioners 89%, diabetologists 96%, and optometrists 94%).1 The proportions of patients screened by each class of health professional (obtained from the study questionnaire) were used to derive an overall specificity for the opportunistic programme of 92%.
Costs
We used an ingredient approach because the costs in screening programmes are largely fixed or semifixed; recording individual patient based costings is not helpful in this situation. Capital was given a seven year life and discounted at the test discount rate of 6%. Overhead costs for hospital based activities—grading, administration, and follow up—were set at 10% (Royal Liverpool University Hospitals Trust finance data).
Costs of systematic screening were calculated on actual resource use at 1996-7 prices for 4000 screen events with additional administrative costs to call non-attendees. Costs of opportunistic screening were calculated for an activity of 3900; with no call-recall system, there are no additional administrative costs. The proportions of patients screened by their general practitioner, diabetologist, or optometrist were identified from the study questionnaire.17 Costs of the general practitioner and diabetologist components were calculated by averaging estimates of time spent on direct ophthalmoscopy by six practitioners who regularly did screening. General practitioners' costs per minute including overheads (5 minutes at £1.72/min) were taken from Netten and Dennett,18 with five minutes, additional nursing time per consultation for instilling drops and measuring visual acuity. Diabetologist costs were calculated as a percentage of the cost of a whole outpatient visit for the hospital (standard outpatient cost for 1996-7=£55). The optometrist cost was taken as the minimum sight test fee of £13.50, representing the full cost to the NHS including staff consumables and overheads. The cost of a standard outpatient appointment was also used to cost follow up assessment of patients who had positive results.
Cost effectiveness was calculated as total cost divided by the number of cases detected and incremental cost effectiveness as the extra cost needed to generate each additional true positive result after replacing opportunistic screening by systematic screening. To test the robustness of the study we conducted a sensitivity analysis to determine the effect on cost effectiveness of varying the key variables.
Results
A baseline prevalence of 14.1% and a cohort of 5000 patients yield an assumed 705 (14.1/100×5000) true cases of sight threatening eye disease in the target population. Table 1 shows the number of true and false positive and negative results calculated for each programme. Table 2 shows the costs for the components of systematic screening, and table 3 presents costs for opportunistic screening based on the percentage of the sample seen by each type of screener. Total costs were £104 996 for systematic screening and £99 981 for opportunistic screening. The cost effectiveness was £209 and £289 respectively, and incremental cost effectiveness was £32 (table 4).
Table 1.
Outcomes of offering screening to 5000 diabetic patients in systematic programme and of screening equivalent cohort opportunistically
| Outcome | Systematic screening | Opportunistic screening |
|---|---|---|
| True positive | 502 | 346 |
| False positive | 481 | 268 |
| True negative | 2955 | 3082 |
| False negative | 62 | 203 |
| Total number screened | 4000 | 3900 |
Table 2.
Annual costs of each stage of systematic photographic screening for a cohort of 5000 invited diabetic patients, 4000 of whom attended
| Annual cost (£) | |
|---|---|
| Photographic screening | 64 517 |
| 1.3 whole time equivalent photographer | 20 040 |
| 1.0 whole time equivalent auxillary nurse | 10 617 |
| 0.5 whole time equivalent clerical worker | 6 359 |
| Film | 3 994 |
| Stationery | 934 |
| Medical and surgical (drops, sundries, etc) | 2 710 |
| Capital charges and depreciation: | |
| Van | 7 571 |
| Camera | 5 451 |
| Computer | 1 908 |
| Camera maintenance contract | 500 |
| Van maintenance | 300 |
| Petrol | 2 310 |
| Addition for non-attenders (administration plus stationery) | 1 823 |
| Grading | 15 739 |
| 0.125 whole time equivalent clinical assistant | 3 918 |
| 0.135 whole time equivalent nurse grade F | 3 097 |
| 0.5 whole time equivalent clerical worker | 6 359 |
| Stationery | 934 |
| Overheads | 1 431 |
| Assessment clinic | 23 740 |
| 0.375 whole time equivalent clinical assistant | 11 755 |
| 0.135 whole time equivalent nurse grade F | 2 534 |
| 0.5 whole time equivalent clerical worker | 6 359 |
| Stationery | 934 |
| Overheads | 2 158 |
| External quality control | 1 000 |
| Total | 104 996 |
Table 3.
Costs of screening tests extrapolated for opportunistic screening programme for cohort of 3900 diabetic patients from a target population of 5000
| Screener
|
Total | |||
|---|---|---|---|---|
| General practitioner | Diabetologist | Optometrist | ||
| No (%) screened | 1 190 (23.8) | 1 710 (34.2) | 1 720 (34.4) | 3 900 |
| Cost of screening (£) | 12 211 | 30 780 | 23 220 | 66 211 |
| Cost of follow up (£) | 11 282 | 10 398 | 12 090 | 33 770 |
| Total cost (£) | 23 493 | 41 178 | 35 310 | 99 981 |
Table 4.
Cost effectiveness of opportunistic and systematic screening programmes
| Opportunistic screenng | Systematic screening | Difference | |
|---|---|---|---|
| Cost of screening (£) | 99 981 | 104 996 | 5015 |
| No of true cases detected | 346 | 502 | 156 |
| Cost effectiveness (£) | 289 | 209 | 32* |
Incremental cost effectiveness of systematic screening.
Sensitivity analysis
Figure 1 shows the effect of varying the prevalence of sight threatening eye disease on cost effectiveness. If the prevalence falls the cost effectiveness of both programmes falls. At all prevalences the opportunistic programme is less expensive, but the systematic programme is more cost effective than the opportunistic programme.
Figure 1.
Effect of changing prevalence of sight threatening diabetic eye disease on cost effectiveness of systematic and opportunistic screening programmes
A two way analysis of the effect on the systematic screening programme of varying sensitivity and specificity within previously reported 95% confidence limits16 gave a value of £237 for low sensitivity and low specificity (80%, 82%) and £186 for high sensitivity and specificity (98%, 90%). Systematic screening is more cost effective than opportunistic screening within the 95% confidence range. Figure 2 shows the effect of varying the sensitivity of opportunistic screening on its cost effectiveness. Cost effectiveness ranged from £350 to £202. Opportunistic screening is less cost effective than systematic screening at all levels of its sensitivity up to 95%.
Figure 2.
Effect of varying sensitivity of opportunistic screening on cost effectiveness. Cost effectiveness of systematic screening is shown as horizontal line
As compliance with systematic screening rises cost effectiveness improves, varying from £487 for 30% compliance to £176 for 100% compliance. At 54% compliance the cost effectiveness of systematic screening equals that of opportunistic screening at £289.
Increasing activity to 6000 screens a year raises the total cost for systematic screening to £139 856. The cost per screen event falls from £26 to £23, and cost effectiveness improves to £186. There is a saving of £43 per true positive case detected when screening systematically rather than opportunistically. Increasing activity to 6000 screens a year raises the total cost for opportunistic screening to £149 972 with no improvement in cost effectiveness. This makes opportunistic screening more expensive than a systematic programme.
Discussion
We directly compared the costs of pre-existing opportunistic screening with a newly introduced systematic programme. The systematic programme is slightly more expensive than the opportunistic programme but yields 157 extra cases at only £32 per case.
In an earlier cost effectiveness assessment, Buxton et al studied 3318 screen events in three UK centres using two methods: direct ophthalmoscopy by optometrists, physicians, and general practitioners and single field non-mydriatic polaroid photography.1,2 If their figures are adjusted to 1996-7 prices the cost effectiveness of direct ophthalmoscopy by optometrists is £1057, hospital physicians £1392, and general practitioners £853-£1454; the costs of hospital and community photographic screening ranged from £670 to £2084. Their disappointing results were largely due to suboptimal screening methods and a low prevalence (5.8%).2
Lairson et al studied the cost effectiveness of four screening methods in the United States.5 They also found a large difference between a photographic protocol similar to ours ($295 (£184)) and direct ophthalmoscopy by a technician ($794), with direct ophthalmoscopy over 2.5 times more expensive than photography.
Applicability
Our results can be generalised to other British photographic screening programmes. The baseline prevalence is likely to be similar throughout the country,19 as is the effectiveness of opportunistic screening. However, accurate data on sensitivity, specificity, and compliance are required to complete an analysis based on our model. Such an analysis would be valuable when applied to other current techniques including dual modality screening,20–22 optometry based programmes,23 digital photography,24,25 and automated neural net systems.26
Several factors may have influenced our results. Disease prevalence is an important determinant of cost effectiveness, but the systematic screening was always more cost effective than opportunistic screening at all values of sensitivity and specificity within the 95% confidence limits for our data (fig 1).16 Lairson et al reported similar findings.5 This is important as prevalence in the screened population will fall with each year of screening: each year the true positive group will comprise patients who develop sight threatening eye disease in that year and a number of people with false negative results from the previous year. After several years the prevalence should approach the underlying incidence of new cases a year.
Cost effectiveness is further influenced by the effectiveness of the screening tool. Varying sensitivity and specificity between the upper and lower 95% confidence limits in the systematic programme produced only a small variation in cost effectiveness. However, in the opportunistic programme the low sensitivity for direct ophthalmoscopy gave a poor cost effectiveness. Studies of the sensitivity of direct ophthalmoscopy have all reported low rates with general practitioners and untrained physicians,1,9–11,27 even after intensive training.28 Unacceptably high serious error rates have also been reported with a trained retina specialist.12 The best sensitivity reported to date is 65% with a trained ophthalmologist,16 and at this level systematic screening is more cost effective.
Compliance with screening greatly affects cost effectiveness, with higher rates of compliance increasing cost effectiveness. A compliance rate of 80% was achieved in the second year of the systematic programme, and this may improve with better targeting and education and the implementation of a district diabetes register. However, full coverage is probably impossible because of factors such as death, housebound patients, and high population turnover in an inner city.
Our analysis is based on an annual 5000 invitations yielding 4000 screen events. Increasing the annual activity to full coverage at 6000 screen events increases the cost of systematic screening but confers an 11% (£23) improvement in cost effectiveness because capital costs do not change. The incremental cost effectiveness becomes negative, indicating a real cost saving.
Outcome measures
We have used the number of detected cases as our measure of effectiveness. The use of this proxy measure depends on the inference that correctly and appropriately identified cases can be treated and blindness prevented. Although useful, this kind of measure does not necessarily show the full effectiveness of a programme as it reflects process rather than final outcome. Further work is required to measure cost effectiveness against long term end points such as numbers of patients treated, years of sight saved, quality of life, or numbers of blind registrations.5
In conclusion, our cost comparison implies that a purchaser can recover the costs of opportunistic screening by diverting them to systematic screening. In our opinion reallocation of resources is feasible, and purchasers can justify the small increase in costs entailed by introducing systematic screening.
What is already known on this topic
Screening for diabetic eye disease can prevent loss of sight
Screening in Britain is currently opportunistic
The cost effectiveness of systematic screening has not been properly evaluated
What this study adds
Cost effectiveness of systematic screening in primary care using a multiple 45 field photographic protocol was £209 compared with £289 for an existing opportunistic programme
The incremental cost effectiveness of replacing opportunistic screening with systematic screening was £32
Systematic screening remained more cost effective than opportunistic screening for all values of disease prevalence
Acknowledgments
We thank Mr P Kingham of Royal Liverpool University Hospitals Trust finance department for help with costs.
Footnotes
Funding: This study was funded by North West Regional grant DIF1.
Competing interests: None declared.
References
- 1.Buxton MJ, Sculpher MJ, Ferguson BA, Humphreys JE, Altman JF, Spiegelhalter DJ, et al. Screening for treatable diabetic retinopathy: a comparison of different methods. Diabet Med. 1991;8:371–377. doi: 10.1111/j.1464-5491.1991.tb01612.x. [DOI] [PubMed] [Google Scholar]
- 2.Sculpher MJ, Buxton MJ, Ferguson BA, Humphreys JE, Altman JFB, Spiegelhalter DJ, et al. A relative cost-effectiveness analysis of different methods of screening for diabeticretinopathy. Diabetic Med. 1991;8:644–650. doi: 10.1111/j.1464-5491.1991.tb01671.x. [DOI] [PubMed] [Google Scholar]
- 3.Fendrick AM, Javitt JC, Chiang YP. Cost-effectiveness of the screening and treatment of diabetic retinopathy. What are the costs of underutilization? Int J Technol Assess Health Care. 1992;8:694–707. doi: 10.1017/s0266462300002385. [DOI] [PubMed] [Google Scholar]
- 4.Javitt JC, Canner JK, Sommer A. Cost-effectiveness of current approaches to the control of retinopathy in type I diabetics. Ophthalmology. 1989;96:255–264. doi: 10.1016/s0161-6420(89)32923-x. [DOI] [PubMed] [Google Scholar]
- 5.Lairson DR, Pugh JA, Kapadia AS, Lorimor RJ, Jacobson J, Velez R. Cost-effectiveness of alternative methods for diabetic retinopathy screening. Diabetes Care. 1992;15:1369–1377. doi: 10.2337/diacare.15.10.1369. [DOI] [PubMed] [Google Scholar]
- 6.Dasbach EJ, Frybach DG, Newcomb PA, Klein R, Klein BEK. Cost-effectiveness of strategies for detecting diabetic retinopathy. Med Care. 1991;29:20–39. doi: 10.1097/00005650-199101000-00003. [DOI] [PubMed] [Google Scholar]
- 7.Sculpher MJ, Buxton MJ, Ferguson BA, Spiegelhalter DJ, Kirby AJ. Screening for diabetic retinopathy: a relative cost-effectiveness analysis of alternative modalities and strategies. Health Econ. 1992;1:39–51. doi: 10.1002/hec.4730010107. [DOI] [PubMed] [Google Scholar]
- 8.Scott JA, Harding SP, Broadbent DM, Vora J. Detecting sight threatening diabetic eye disease in an inner city setting in the UK—the Liverpool diabetic eye study. Invest Ophthalmol Vis Sci. 1996;37:S105. [Google Scholar]
- 9.Finlay R, Griffiths J, Jackson G, Law D. Can general practitioners screen their own patients for diabetic retinopathy? Health Trends. 1991;23:104–105. [Google Scholar]
- 10.Awh CC, Javitt JC, Chong LP, Gehrs KM, Gusman GI, Street DA, et al. Ophthalmoscopic diagnosis and referral of diabetic eye disease by Primary Care Physicians. ARVO abstract. Invest Ophthalmol Vis Sci. 1993;34:713. [Google Scholar]
- 11.Forrest RD, Jackson CA, Yudkin JS. Screening for diabetic retinopathy. Comparison of a nurse and doctor with retinal photography. Diabetes Res. 1987;5:39–42. [PubMed] [Google Scholar]
- 12.Kinyoun JL, Martin DC, Fujimoto WY, Leonetti DL. Ophthalmoscopy versus fundus photography for detecting and grading diabetic retinopathy. Invest Ophthalmol Vis Sci. 1992;33:1888–1893. [PubMed] [Google Scholar]
- 13.Foulds W, McCuish A, Barrie T, Green F, Scobie IN, Ghafour IM, et al. Diabetic retinopathy in the west of Scotland: its detection and prevalence, and cost-effectiveness of a proposed screening programme. Health Bull. 1983;41:318–326. [PubMed] [Google Scholar]
- 14.Rohan TE, Frost CD, Wald NJ. Prevention of blindness by screening for diabetic retinopathy: a quantitative assessment. BMJ. 1989;299:1198–1201. doi: 10.1136/bmj.299.6709.1198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Davies R, Sullivan P, Canning C. Simulation of eye disease to compare screening policies. Br J Ophthalmol. 1996;80:945–950. doi: 10.1136/bjo.80.11.945. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Harding SP, Broadbent DM, Neoh C, White MC, Vora J. Sensitivity and specificity of photography and direct ophthalmoscopy in screening for sight threatening eye disease: the Liverpool diabetic eye study. BMJ. 1995;311:1131–1135. doi: 10.1136/bmj.311.7013.1131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Briggs MC, Broadbent DM, Harding SP, Vora JP. Primary screening in an inner city—the Liverpool diabetic eye study. Diabet Med. 1995;12(suppl 2):S44. [Google Scholar]
- 18.Netten A, Dennett J. Unit costs of community care. Canterbury: Personal Social Services Research Unit, University of Kent; 1996. [Google Scholar]
- 19.Owens DR, Gibbins RL, Lewis PA, Wall S, Allen JC, Morton R. Screening for diabetic retinopathy by general practitioners: ophthalmoscopy or retinal photography as 35 mm transparencies? Diabet Med. 1998;15:170–175. doi: 10.1002/(SICI)1096-9136(199802)15:2<170::AID-DIA518>3.0.CO;2-H. [DOI] [PubMed] [Google Scholar]
- 20.Ryder B. Screening for diabetic retinopathy. BMJ. 1995;311:207–208. doi: 10.1136/bmj.311.6999.207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.O'Hare JP, Hopper A, Madhaven C, Charny M, Purewal TS, Harney B, et al. Adding retinal photography to screening for diabetic retinopathy: a prospective study in primary care. BMJ. 1996;312:679–682. doi: 10.1136/bmj.312.7032.679. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jacob J, Stead J, Sykes J, Taylor D, Tooke JE. A report on the use of technician ophthalmoscopy combined with the use of the Canon non-mydriatic camera in screening for diabetic retinopathy in the community. Diabet Med. 1995;12:419–425. doi: 10.1111/j.1464-5491.1995.tb00506.x. [DOI] [PubMed] [Google Scholar]
- 23.Leese GP, Tesfaye S, Dengler-Harles M, Laws F, Clark DI, Gill GV, et al. Screening for diabetic eye disease by optometrists using the slit-lamp. J R Coll Physicians. 1997;31:65–69. [PMC free article] [PubMed] [Google Scholar]
- 24.George LD, Halliwell M, Hill R, Aldington SJ, Lusty J, Dunstan F, et al. A comparison of digital and 35 mm colour transparencies in detecting and grading diabetic retinopathy. Diabetic Med. 1998;15:250–253. doi: 10.1002/(SICI)1096-9136(199803)15:3<250::AID-DIA565>3.0.CO;2-G. [DOI] [PubMed] [Google Scholar]
- 25.Ryder REJ, Kong N, Bates AS, Sim J, Welch J, Kritzinger EE. Instant electronic imaging systems are superior to polaroid at detecting sight threatening diabetic retinopathy. Diabet Med. 1998;15:254–258. doi: 10.1002/(SICI)1096-9136(199803)15:3<254::AID-DIA543>3.0.CO;2-C. [DOI] [PubMed] [Google Scholar]
- 26.Gardner GG, Keating D, Williamson TH, Elliott AT. Automatic detection of diabetic retinopathy using an artificial neural network: a screening tool. Br J Ophthalmol. 1996;80:940–944. doi: 10.1136/bjo.80.11.940. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Sussman EJ, Tsurias WG, Sarper KA. Diagnosis of diabetic eye disease. JAMA. 1982;247:3231–3234. [PubMed] [Google Scholar]
- 28.Gehrs KM, Chong LP, Gusman G, Street DA, Awh C, Cupples H, et al. Can we educate primary care physicians about diabetic retinopathy after graduation? Preliminary results of the diabetic retinopathy education study. Invest Ophthalmol Vis Sci. 1993;34:S1182. [Google Scholar]