Skip to main content
NCBI home page
Medline Book to support NIHPA logoLink to Medline Book to support NIHPA
. 2021 May;25(32):1–104. doi: 10.3310/hta25320

Multimodal imaging interpreted by graders to detect re-activation of diabetic eye disease in previously treated patients: the EMERALD diagnostic accuracy study.

Noemi Lois, Jonathan Cook, Ariel Wang, Stephen Aldington, Hema Mistry, Mandy Maredza, Danny McAuley, Tariq Aslam, Clare Bailey, Victor Chong, Faruque Ghanchi, Peter Scanlon, Sobha Sivaprasad, David Steel, Caroline Styles, Augusto Azuara-Blanco, Lindsay Prior, Norman Waugh
PMCID: PMC8200933  PMID: 34060440

Abstract

BACKGROUND

Owing to the increasing prevalence of diabetes, the workload related to diabetic macular oedema and proliferative diabetic retinopathy is rising, making it difficult for hospital eye services to meet demands.

OBJECTIVE

The objective was to evaluate the diagnostic performance, cost-effectiveness and acceptability of a new pathway using multimodal imaging interpreted by ophthalmic graders to detect reactivation of diabetic macular oedema/proliferative diabetic retinopathy in previously treated patients.

DESIGN

This was a prospective, case-referent, cross-sectional diagnostic study.

SETTING

The setting was ophthalmic clinics in 13 NHS hospitals.

PARTICIPANTS

Adults with type 1 or type 2 diabetes with previously successfully treated diabetic macular oedema/proliferative diabetic retinopathy in one/both eyes in whom, at the time of enrolment, diabetic macular oedema/proliferative diabetic retinopathy could be active or inactive.

METHODS

For the ophthalmic grader pathway, review of the spectral domain optical coherence tomography scans to detect diabetic macular oedema, and seven-field Early Treatment Diabetic Retinopathy Study/ultra-wide field fundus images to detect proliferative diabetic retinopathy, by trained ophthalmic graders. For the current standard care pathway (reference standard), ophthalmologists examined patients face to face by slit-lamp biomicroscopy for proliferative diabetic retinopathy and, in addition, spectral domain optical coherence tomography imaging for diabetic macular oedema.

OUTCOME MEASURES

The primary outcome measure was sensitivity of the ophthalmic grader pathway to detect active diabetic macular oedema/proliferative diabetic retinopathy. The secondary outcomes were specificity, agreement between pathways, cost-consequences, acceptability and the proportion of patients requiring subsequent ophthalmologist assessment, unable to undergo imaging and with inadequate quality images/indeterminate findings. It was assumed for the main analysis that all patients in whom graders diagnosed active disease or were 'unsure' or images were 'ungradable' required examination by an ophthalmologist.

RESULTS

Eligible participants with active and inactive diabetic macular oedema (152 and 120 participants, respectively) and active and inactive proliferative diabetic retinopathy (111 and 170 participants, respectively) were recruited. Under the main analysis, graders had a sensitivity of 97% (142/147) (95% confidence interval 92% to 99%) and specificity of 31% (35/113) (95% confidence interval 23% to 40%) to detect diabetic macular oedema. For proliferative diabetic retinopathy, graders had a similar sensitivity and specificity using seven-field Early Treatment Diabetic Retinopathy Study [sensitivity 85% (87/102), 95% confidence interval 77% to 91%; specificity 48% (77/160), 95% confidence interval 41% to 56%] or ultra-wide field imaging [sensitivity 83% (87/105), 95% confidence interval 75% to 89%; specificity 54% (86/160), 95% confidence interval 46% to 61%]. Participants attending focus groups expressed preference for face-to-face evaluations by ophthalmologists. In the ophthalmologists' absence, patients voiced the need for immediate feedback following grader's assessments, maintaining periodic evaluations by ophthalmologists. Graders and ophthalmologists were supportive of the new pathway. When compared with the reference standard (current standard pathway), the new grader pathway could save £1390 per 100 patients in the review of people with diabetic macular oedema and, depending on the imaging modality used, between £461 and £1189 per 100 patients in the review of people with proliferative diabetic retinopathy.

CONCLUSIONS

For people with diabetic macular oedema, the ophthalmic grader pathway appears safe and cost saving. The sensitivity of the new pathway to detect active proliferative diabetic retinopathy was lower, but may still be considered acceptable for patients with proliferative diabetic retinopathy previously treated with laser. Suggestions from focus group discussions should be taken into consideration if the new pathway is introduced to ensure its acceptability to users.

LIMITATIONS

Lack of fundus fluorescein angiography to confirm diagnosis of active proliferative diabetic retinopathy.

FUTURE WORK

Could refinement of the new pathway increase its sensitivity to detect proliferative diabetic retinopathy? Could artificial intelligence be used for automated reading of images in this previously treated population?

TRIAL REGISTRATION

Current Controlled Trials ISRCTN10856638 and ClinicalTrials.gov NCT03490318.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology AssessmentVol. 25, No. 32. See the NIHR Journals Library website for further project information.

Plain language summary

More and more people are developing diabetes. Diabetic macular oedema and proliferative diabetic retinopathy are complications of diabetes, which could cause blindness. Thus, people with diabetic macular oedema and proliferative diabetic retinopathy need to be treated in a timely manner and reviewed in clinic for life. The population in the world is ageing. As a result, there are more people with eye diseases. There are also more treatments now for people with eye diseases. The workload in hospitals is increasing, making it difficult for the NHS to cope with the demand. There are not enough ophthalmologists (eye doctors) to look after patients. Delayed appointments and treatment mean that patients may lose sight. The goal of EMERALD (Effectiveness of Multimodal imaging for the Evaluation of Retinal oedema And new vesseLs in Diabetic retinopathy) was to see if patients with treated and stable diabetic macular oedema or proliferative diabetic retinopathy could be followed by ‘ophthalmic graders’, who are not doctors but are trained to diagnose diabetic macular oedema and proliferative diabetic retinopathy. In EMERALD, trained ophthalmic graders examined photographs of the back of the eye of people with diabetic macular oedema and proliferative diabetic retinopathy. They checked if diabetic macular oedema and proliferative diabetic retinopathy remain inactive. If so, patients could continue follow-up with the ophthalmic graders. If diabetic macular oedema or proliferative diabetic retinopathy were active, graders would immediately refer patients to ophthalmologists. EMERALD found that graders were excellent at detecting diabetic macular oedema, and this could give ophthalmologists time to see other patients. Graders were not quite as good at detecting active proliferative diabetic retinopathy. However, considering that patients had already had treatment, this may still be safe. Patients participating in focus group discussions mentioned that they would prefer to see ophthalmologists, so they could ask questions about their eye condition. If this was not possible, they would like to have immediate results from graders and still see the ophthalmologist from time to time.

Full text of this article can be found in Bookshelf.

References

  1. Lois N, Cook J, Aldington S, Waugh N, Mistry H, Sones W, et al. Effectiveness of Multimodal imaging for the Evaluation of Retinal oedema And new vesseLs in Diabetic retinopathy (EMERALD). BMJ Open 2019;9:e027795. https://doi.org/10.1136/bmjopen-2018-027795 doi: 10.1136/bmjopen-2018-027795. [DOI] [PMC free article] [PubMed]
  2. Lois N, Cook JA, Wang A, Aldington S, Mistry H, Maredza W, et al. Evaluation of a new model of care for people with complications of diabetic retinopathy: the EMERALD study. Ophthalmology 2021;128:561–73. https://doi.org/10.1016/j.ophtha.2020.10.030 doi: 10.1016/j.ophtha.2020.10.030. [DOI] [PMC free article] [PubMed]
  3. Liew G, Michaelides M, Bunce C. A comparison of the causes of blindness certifications in England and Wales in working age adults (16–64 years), 1999–2000 with 2009–2010. BMJ Open 2014;4:e004015. https://doi.org/10.1136/bmjopen-2013-004015 doi: 10.1136/bmjopen-2013-004015. [DOI] [PMC free article] [PubMed]
  4. Minassian DC, Owens DR, Reidy A. Prevalence of diabetic macular oedema and related health and social care resource use in England. Br J Ophthalmol 2012;96:345–9. https://doi.org/10.1136/bjo.2011.204040 doi: 10.1136/bjo.2011.204040. [DOI] [PubMed]
  5. Yau JW, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, et al. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care 2012;35:556–64. https://doi.org/10.2337/dc11-1909 doi: 10.2337/dc11-1909. [DOI] [PMC free article] [PubMed]
  6. Diabetes UK. Diabetes UK. 2020. URL: www.diabetes.org.uk/ (accessed 27 February 2020).
  7. National Institute for Health and Care Excellence (NICE). Ranibizumab for Treating Diabetic Macular Oedema. London: NICE; 2013.
  8. National Institute for Health and Care Excellence (NICE). Aflibercept for Treating Diabetic Macular Oedema. London: NICE; 2015.
  9. National Institute for Health and Care Excellence (NICE). Fluocinolone Acetonide Intravitreal Implant for Treating Chronic Diabetic Macular Oedema after an Inadequate Response to Prior Therapy. London: NICE; 2013.
  10. The Royal College of Ophthalmologists. Diabetic Retinopathy Guidelines. London: The Royal College of Ophthalmologists; 2012.
  11. Elman MJ, Aiello LP, Beck RW, Bressler NM, Bressler SB, Edwards AR, et al. Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology 2010;117:1064–77.e35. https://doi.org/10.1016/j.ophtha.2010.02.031 doi: 10.1016/j.ophtha.2010.02.031. [DOI] [PMC free article] [PubMed]
  12. Mitchell P, Bandello F, Schmidt-Erfurth U, Lang GE, Massin P, Schlingemann RO, et al. The RESTORE study: ranibizumab monotherapy or combined with laser versus laser monotherapy for diabetic macular edema. Ophthalmology 2011;118:615–25. https://doi.org/10.1016/j.ophtha.2011.01.031 doi: 10.1016/j.ophtha.2011.01.031. [DOI] [PubMed]
  13. Nguyen QD, Brown DM, Marcus DM, Boyer DS, Patel S, Feiner L, et al. Ranibizumab for diabetic macular edema: results from 2 phase III randomized trials: RISE and RIDE. Ophthalmology 2012;119:789–801. https://doi.org/10.1016/j.ophtha.2011.12.039 doi: 10.1016/j.ophtha.2011.12.039. [DOI] [PubMed]
  14. Korobelnik JF, Do DV, Schmidt-Erfurth U, Boyer DS, Holz FG, Heier JS, et al. Intravitreal aflibercept for diabetic macular edema. Ophthalmology 2014;121:2247–54. https://doi.org/10.1016/j.ophtha.2014.05.006 doi: 10.1016/j.ophtha.2014.05.006. [DOI] [PubMed]
  15. Diabetic Retinopathy Clinical Research Network, Wells JA, Glassman AR, Ayala AR, Jampol LM, Aiello LP, et al. Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema. N Eng J Med 2015;372:1193–203. https://doi.org/10.1056/NEJMoa1414264 doi: 10.1056/NEJMoa1414264. [DOI] [PMC free article] [PubMed]
  16. Virgili G, Menchini F, Casazza G, Hogg R, Das RR, Wang X, Michelessi M. Optical coherence tomography (OCT) for detection of macular oedema in patients with diabetic retinopathy. Cochrane Database Syst Rev 2015;1:CD008081. https://doi.org/10.1002/14651858.CD008081.pub3 doi: 10.1002/14651858.CD008081.pub3. [DOI] [PMC free article] [PubMed]
  17. Early Treatment Diabetic Retinopathy Study Research Group. Early photocoagulation for diabetic retinopathy. ETDRS report number 9. Ophthalmology 1991;98:766–85. https://doi.org/10.1016/S0161-6420(13)38011-7 doi: 10.1016/S0161-6420(13)38011-7. [DOI] [PubMed]
  18. Royle P, Mistry H, Auguste P, Shyangdan D, Freeman K, Lois N, Waugh N. Pan-retinal photocoagulation and other forms of laser treatment and drug therapies for non-proliferative diabetic retinopathy: systematic review and economic evaluation. Health Technol Assess 2015;19(51). https://doi.org/10.3310/hta19510 doi: 10.3310/hta19510. [DOI] [PMC free article] [PubMed]
  19. Moutray T, Evans JR, Lois N, Armstrong DJ, Peto T, Azuara-Blanco A. Different lasers and techniques for proliferative diabetic retinopathy. Cochrane Database Syst Rev 2018;3:CD012314. https://doi.org/10.1002/14651858.CD012314.pub2 doi: 10.1002/14651858.CD012314.pub2. [DOI] [PMC free article] [PubMed]
  20. Bressler SB, Beaulieu WT, Glassman AR, Gross JG, Jampol LM, Melia M, et al. Factors associated with worsening proliferative diabetic retinopathy in eyes treated with panretinal photocoagulation or ranibizumab. Ophthalmology 2017;124:431–9. https://doi.org/10.1016/j.ophtha.2016.12.005 doi: 10.1016/j.ophtha.2016.12.005. [DOI] [PMC free article] [PubMed]
  21. Keenan TD, Johnston RL, Donachie PH, Sparrow JM, Stratton IM, Scanlon P. United Kingdom National Ophthalmology Database Study: diabetic retinopathy; report 1: prevalence of centre-involving diabetic macular oedema and other grades of maculopathy and retinopathy in hospital eye services. Eye 2013;27:1397–404. https://doi.org/10.1038/eye.2013.196 doi: 10.1038/eye.2013.196. [DOI] [PMC free article] [PubMed]
  22. Silva PS, Cavallerano JD, Haddad NM, Kwak H, Dyer KH, Omar AF, et al. Peripheral lesions identified on ultrawide field imaging predict increased risk of diabetic retinopathy progression over 4 years. Ophthalmology 2015;122:949–56. https://doi.org/10.1016/j.ophtha.2015.01.008 doi: 10.1016/j.ophtha.2015.01.008. [DOI] [PubMed]
  23. Royal College of Ophthalmologists. Eye Health Experts Call for Urgent Action on Hospital Eyecare to Reduce Delays and Improve Patient Safety. London: Royal College of Ophthalmologists; 2020. URL: www.rcophth.ac.uk/2020/02/eye-health-experts-call-for-urgent-action-on-hospital-eyecare-to-reduce-delays-and-improve-patient-safety/ (accessed 18 January 2020).
  24. Knottnerus JA, Muris JW. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol 2003;56:1118–28. https://doi.org/10.1016/S0895-4356(03)00206-3 doi: 10.1016/S0895-4356(03)00206-3. [DOI] [PubMed]
  25. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529–36. https://doi.org/10.7326/0003-4819-155-8-201110180-00009 doi: 10.7326/0003-4819-155-8-201110180-00009. [DOI] [PubMed]
  26. Dixon JR Jr. The International Conference on Harmonization Good Clinical Practice guideline. Qual Assur 1998;6:65–74. https://doi.org/10.1080/105294199277860 doi: 10.1080/105294199277860. [DOI] [PubMed]
  27. Obuchowski NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res 1998;7:371–92. https://doi.org/10.1177/096228029800700405 doi: 10.1177/096228029800700405. [DOI] [PubMed]
  28. Silva PS, Cavallerano JD, Tolson AM, Rodriguez J, Rodriguez S, Ajlan R, et al. Real-time ultrawide field image evaluation of retinopathy in a diabetes telemedicine program. Diabetes Care 2015;38:1643–9. https://doi.org/10.2337/dc15-0161 doi: 10.2337/dc15-0161. [DOI] [PubMed]
  29. Piegorsch WW. Sample sizes for improved binomial confidence intervals. Comput Stat Data Anal 2004;46:309–16. https://doi.org/10.1016/j.csda.2003.10.002 doi: 10.1016/j.csda.2003.10.002. [DOI]
  30. Newcombe RG. Improved confidence intervals for the difference between binomial proportions based on paired data. Stat Med 1998;17:2635–50. https://doi.org/10.1002/(SICI)1097-0258(19981130)17:22<2635::AID-SIM954>3.0.CO;2-C doi: 10.1002/(SICI)1097-0258(19981130)17:22&#x0003c;2635::AID-SIM954&#x0003e;3.0.CO;2-C. [DOI] [PubMed]
  31. McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 1947;12:153–7. https://doi.org/10.1007/BF02295996 doi: 10.1007/BF02295996. [DOI] [PubMed]
  32. de Nooy W, Mrvar A, Batagelj V. Exploratory Social Network Analysis with Pajek. Cambridge: Cambridge University Press; 2005. https://doi.org/10.1017/CBO9780511806452 doi: 10.1017/CBO9780511806452. [DOI]
  33. Curtis L, Burns A. Unit Costs of Health and Social Care 2019. Canterbury: Personal Social Services Research Unit, University of Kent; 2019.
  34. NHS England and NHS Improvement joint pricing team. 2019/20 National Tariff Payment System: National Prices and Prices for Emergency Care Services. 2019. URL: https://improvement.nhs.uk/resources/national-tariff/#h2-201920-national-tariff-payment-system (accessed 12 February 2020).
  35. NHS Improvement. Reference Costs 2017/2018: Highlights, Analysis and Introduction to the Data. 2018. URL: https://improvement.nhs.uk/resources/reference-costs/ (accessed 16 January 2021).
  36. University Hospital Southampton NHS Foundation Trust. Eye Unit. 2019. URL: www.uhs.nhs.uk/ourservices/eyes/eyeunit.aspx (accessed 30 January 2019).
  37. Topcon Ireland Medical. Equipment Price. URL: www.topcon-medical.ie/ie/categories/27-diagnostic/ (accessed 20 February 2020).
  38. Optos Plc. Equipment Price. URL: www.optos.com/en-gb/products/ (accessed 20 February 2020).
  39. Veatch Ophthalmic Instruments. Haag-Streit BM 900 Table LED Slit Lamp Equipment Price. URL: www.veatchinstruments.com/Haag-Streit-BM-900-Table-LED-Slit-Lamp (accessed 8 January 2020).
  40. National Institute for Health and Care Excellence (NICE). Guide to the Methods of Technology Appraisal. London: NICE; 2013. [PubMed]
  41. EuroQol Group. EQ-5D-5L. URL: https://euroqol.org/eq-5d-instruments/eq-5d-5l-about/valuation-standard-value-sets/crosswalk-index-value-calculator/ (accessed 20 March 2020).
  42. van Hout B, Janssen MF, Feng YS, Kohlmann T, Busschbach J, Golicki D, et al. Interim scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L value sets. Value Health 2012;15:708–15. https://doi.org/10.1016/j.jval.2012.02.008 doi: 10.1016/j.jval.2012.02.008. [DOI] [PubMed]
  43. Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD, National Eye Institute Visual Function Questionnaire Field Test Investigators. Development of the 25-item National Eye Institute Visual Function Questionnaire. Arch Ophthalmol 2001;119:1050–8. https://doi.org/10.1001/archopht.119.7.1050 doi: 10.1001/archopht.119.7.1050. [DOI] [PubMed]
  44. Peacock S, Misajon R, Iezzi A, Richardson J, Hawthorne G, Keeffe J. Vision and quality of life: development of methods for the VisQoL vision-related utility instrument. Ophthalmic Epidemiol 2008;15:218–23. https://doi.org/10.1080/09286580801979417 doi: 10.1080/09286580801979417. [DOI] [PMC free article] [PubMed]
  45. Centre for Economics. Scoring - Psychometric (unweighted) or Utility (weighted)? Australia: Monash University; 2020. URL: www.aqol.com.au/index.php/scoring-algorithms (accessed 13 February 2020).
  46. Brown GC. Vision and quality of life. Trans Am Ophthalmol Soc 1999;97:473–511. [PMC free article] [PubMed]
  47. Brown MM, Brown GC, Sharma S, Landy J, Bakal J. Quality of life with visual acuity loss from diabetic retinopathy and age-related macular degeneration. Arch Ophthalmol 2002;120:481–4. https://doi.org/10.1001/archopht.120.4.481 doi: 10.1001/archopht.120.4.481. [DOI] [PubMed]
  48. Wallace P. Pay and Conditions Circular (M&D) 2/2019 R2. NHS Employers; 2020. URL: www.nhsemployers.org/-/media/Employers/Documents/Pay-and-reward/Pay-and-Conditions-Circular-MD-22019.pdf (accessed 16 January 2021).
  49. British Medical Association. Pay Scales for Consultants in England. 2019. URL: www.bma.org.uk/advice/employment/pay/consultants-pay-england (accessed 2 February 2020).
  50. Prescott G, Sharp P, Goatman K, Scotland G, Fleming A, Philip S, et al. Improving the cost-effectiveness of photographic screening for diabetic macular oedema: a prospective, multi-centre, UK study. Br J Ophthalmol 2014;98:1042–9. https://doi.org/10.1136/bjophthalmol-2013-304338 doi: 10.1136/bjophthalmol-2013-304338. [DOI] [PubMed]
  51. Royal National Institute of Blind People. Report Warns of Crisis in Eye Clinic Capacity as Cancelled Appointments Risks Patients’ Sight. 2018. URL: www.rnib.org.uk/who-we-are/media-centre/latest-media-releases/report-warns-crisis-eye-clinic-capacity-cancelled (accessed 4 March 2020).
  52. Impairment A-PPGoEHaV. See the Light: Improving Capacity in NHS Eye Care in England. 2018. URL: www.rnib.org.uk/sites/default/files/See%20the%20light_Improving%20NHS%20eye%20care%20capacity%20in%20England.pdf (accessed 4 March 2020).
  53. Sharp PF, Olson J, Strachan F, Hipwell J, Ludbrook A, O’Donnell M, et al. The value of digital imaging in diabetic retinopathy. Health Technol Assess 2003;7(30). https://doi.org/10.3310/hta7300 doi: 10.3310/hta7300. [DOI] [PubMed]
  54. Olson J, Sharp P, Goatman K, Prescott G, Scotland G, Fleming A, et al. Improving the economic value of photographic screening for optical coherence tomography-detectable macular oedema: a prospective, multicentre, UK study. Health Technol Assess 2013;17(51). https://doi.org/10.3310/hta17510 doi: 10.3310/hta17510. [DOI] [PMC free article] [PubMed]
  55. Abràmoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M. Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci 2016;57:5200–6. https://doi.org/10.1167/iovs.16-19964 doi: 10.1167/iovs.16-19964. [DOI] [PubMed]
  56. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 2018;1:39. https://doi.org/10.1038/s41746-018-0040-6 doi: 10.1038/s41746-018-0040-6. [DOI] [PMC free article] [PubMed]
  57. Talks SJ, Manjunath V, Steel DH, Peto T, Taylor R. New vessels detected on wide-field imaging compared to two-field and seven-field imaging: implications for diabetic retinopathy screening image analysis. Br J Ophthalmol 2015;99:1606–9. https://doi.org/10.1136/bjophthalmol-2015-306719 doi: 10.1136/bjophthalmol-2015-306719. [DOI] [PMC free article] [PubMed]
  58. Tsaousis KT, Empeslidis T, Konidaris VE, Kapoor B, Deane J. The concept of virtual clinics in monitoring patients with age-related macular degeneration. Acta Ophthalmol 2016;94:e353–5. https://doi.org/10.1111/aos.12832 doi: 10.1111/aos.12832. [DOI] [PubMed]
  59. Lee JX, Manjunath V, Talks SJ. Expanding the role of medical retina virtual clinics using multimodal ultra-widefield and optical coherence tomography imaging. Clin Ophthalmol 2018;12:2337–45. https://doi.org/10.2147/OPTH.S181108 doi: 10.2147/OPTH.S181108. [DOI] [PMC free article] [PubMed]
  60. Kortuem K, Fasler K, Charnley A, Khambati H, Fasolo S, Katz M, et al. Implementation of medical retina virtual clinics in a tertiary eye care referral centre. Br J Ophthalmol 2018;102:1391–5. https://doi.org/10.1136/bjophthalmol-2017-311494 doi: 10.1136/bjophthalmol-2017-311494. [DOI] [PubMed]
  61. Ahnood D, Souriti A, Williams GS. Assessing patient acceptance of virtual clinics for diabetic retinopathy: a large scale postal survey. Can J Ophthalmol 2018;53:207–9. https://doi.org/10.1016/j.jcjo.2017.10.035 doi: 10.1016/j.jcjo.2017.10.035. [DOI] [PubMed]
  62. Kern C, Kortuem K, Hamilton R, Fasolo S, Cai Y, Balaskas K, et al. Clinical outcomes of a hospital-based teleophthalmology service: what happens to patients in a virtual clinic? Ophthalmol Retina 2019;3:422–8. https://doi.org/10.1016/j.oret.2019.01.011 doi: 10.1016/j.oret.2019.01.011. [DOI] [PubMed]
  63. Kotecha A, Baldwin A, Brookes J, Foster PJ. Experiences with developing and implementing a virtual clinic for glaucoma care in an NHS setting. Clin Ophthalmol 2015;9:1915–23. https://doi.org/10.2147/OPTH.S92409 doi: 10.2147/OPTH.S92409. [DOI] [PMC free article] [PubMed]
  64. Gunn PJG, Marks JR, Au L, Waterman H, Spry PGD, Harper RA. Acceptability and use of glaucoma virtual clinics in the UK: a national survey of clinical leads. BMJ Open Ophthalmol 2018;3:e000127. https://doi.org/10.1136/bmjophth-2017-000127 doi: 10.1136/bmjophth-2017-000127. [DOI] [PMC free article] [PubMed]
  65. Prior L. In praise of small N, and of N = 1 in particular. Critical Public Health 2016;26:115–17. https://doi.org/10.1080/09581596.2015.1130250 doi: 10.1080/09581596.2015.1130250. [DOI]
  66. Prior L, Scott D, Hunter R, Donnelly M, Tully MA, Cupples ME, Kee F. Exploring lay views on physical activity and their implications for public health policy. A case study from East Belfast. Soc Sci Med 2014;114:73–80. https://doi.org/10.1016/j.socscimed.2014.05.015 doi: 10.1016/j.socscimed.2014.05.015. [DOI] [PubMed]
  67. Evans MR, Prout H, Prior L, Tapper-Jones LM, Butler CC. A qualitative study of lay beliefs about influenza immunisation in older people. Br J Gen Pract 2007;57:352–8. [PMC free article] [PubMed]
  68. Morris PG, Prior L, Deb S, Lewis G, Mayle W, Burrow CE, Bryant E. Patients’ views on outcome following head injury: a qualitative study. BMC Fam Pract 2005;6:30. https://doi.org/10.1186/1471-2296-6-30 doi: 10.1186/1471-2296-6-30. [DOI] [PMC free article] [PubMed]

RESOURCES