Abstract
Background
To generate a practical and clinically useful consensus definition of ‘stable glaucoma’ to aid provision of glaucoma services in the UK and to provide guidance for the criteria that should be used for monitoring of glaucoma patients in primary care services.
Methods
A Delphi exercise was undertaken to derive consensus through an online questionnaire. Participants were asked to score their strength of agreement for a series of clinical parameters. Results and comments from each round were used to inform subsequent rounds. A total of 3 rounds were undertaken.
Results
Thirty-two glaucoma experts participated in the study with over 90% completion rate achieved over three rounds. The consensus was reached for the following parameters: IOP levels to be used for defining stability, visual field-testing techniques to define stability, the number of medication changes acceptable to define stability and the number of treatment medications allowed to define stability. No consensus was reached on the period of time over which stability was defined, however, there was considerable agreement that longer durations of follow up (36–48 months) were required. A combination of optic disc photos and ocular coherence topography (OCT) retinal nerve fibre layer (RNFL) assessment/ OCT disc structural evaluation are the preferred imaging methods for the assessment of structural stability. Oversight by a glaucoma consultant was considered important for glaucoma monitoring schemes.
Conclusion
The consensus definition of glaucoma stability generated through this Delphi exercise provides guidance for allocation of patients suitable for monitoring in primary care glaucoma monitoring schemes.
Subject terms: Health care, Glaucoma
Introduction
Over 172,000 referrals for patients with ‘suspect’ glaucoma are made to specialist Ophthalmology services in England annually, of which an estimated one third require long term follow up [1]. The referrals for suspect glaucoma in combination with ocular hypertension (OHT) account for over 30% of current ophthalmology outpatient activity [2]. The Royal College of Ophthalmologists (RCOphth) reports that over the next 20 years glaucoma cases are set to rise by 44% [3].
The increasing demand on hospital services has led to the development of alternative community-based services often run by optometrists for monitoring ‘stable’ and low-risk glaucoma patients [1, 3]. NICE estimates that approximately 56,320 patients out of the 169,500 currently being managed in secondary care with chronic open angle glaucoma (COAG), suspect COAG and OHT could be managed in the community [4]. The NICE guidelines for managing glaucoma outline the general principles of monitoring patients who have, or are suspected of having, COAG or OHT [4]. Intraocular pressure readings with Goldmann applanation tonometry, assessment of anterior chamber depth, assessment of the optic nerve head (including imaging) and visual field assessment should all be undertaken.
Despite the move to commission a greater number of community services for the monitoring of OHT and suspect glaucoma [1], there is no established consensus on the clinical definition of ‘Stable Glaucoma’ currently available in the literature. This definition is left to the discretion of local service providers and so it is likely that there is inconsistency in how patients are monitored in these community-based clinics. A definition of ‘stable glaucoma’ would not only inform the effective design and commissioning of glaucoma services in the NHS by identifying those patients who can safely be monitored outside a secondary care environment, but also contribute to developing standards for these patients to be managed safely within the community and aid in accurately identifying those who need to be re-referred back to secondary care allowing consistent delivery of glaucoma services.
The aim of this study is:
To establish a consensus on the definition of “stable” glaucoma amongst consultant ophthalmologists with a recognised expertise in glaucoma.
To evaluate which factors are important when discharging ‘stable glaucoma’ patients to different oversight models of community-based care.
Method
An expert panel, consisting of Ophthalmology consultants with glaucoma subspecialist interest in the UK, were consulted in an adapted (3-round) Delphi exercise [5–7] to establish consensus on the definition of stable glaucoma.
We approached 33 of the 150 glaucoma specialists registered with the RCOphth. The group was a representative mix of teaching and district general hospital consultants and geographical distribution within the UK. The experts were identified via their membership of the UK and Eire Glaucoma Society and initially approached via an email which described the purpose of the exercise. Thirty-two responded to confirm their interest in participating and this was deemed to be an appropriate number of respondents to undertake a valid Delphi process. They were provided with further information about the survey and were subsequently involved in the Delphi process. No incentives were offered to participants. Prior research has suggested that a panel with a minimum of twelve members is required for the findings of a Delphi exercise to be considered valid [5].
The University of Nottingham School of Medicine Ethics committee confirmed that this consultative survey did not require ethical approval.
The survey process was managed using the online survey tool Survey Monkey with each questionnaire designed to take around 15 min to complete. Participants were sent a personalised link to the questionnaires and asked to indicate their strength of agreement for each of a series of parameters using a 0–10 scoring scale, where 10 indicated strong agreement and 0 strong disagreement.
The clinical parameters examined in this way were:
Time Period: How long should a patient be monitored before being considered stable.
Visual Field Methods: Which Visual field (VF) assessment methods should be used to define stability.
Imaging Methods: Which imaging assessment methods should be used to define stability.
Intra-ocular pressure (IOP): What IOP level should be used to define stability?
Use of drops: Whether the total number of IOP lowering agents drops being used by a patient or a change in number of drops required should be used to define stability.
Consultant Oversight: the nature and clinical expertise of the consultants overseeing patients within community monitoring services.
After each round scores were synthesised and descriptive statistics for all (whole group) responses were generated for each parameter. A group median score 8–10 was considered to indicate ‘strong agreement’ with a parameter; a median score 0–2 strong disagreement. The use of median scores to summarise group responses in this way is common in Delphi research [7] however, median scores in isolation may disguise a broad range of scores which might be typical of panel disagreement. To counter this and to add rigour to our Delphi process, we combined a median score with an Interquartile Range (IQR) assessment [6–9]. An IQR score indicates the concentration of scores across the range of scoring options; an IQR of 2 indicates that 50% or more of responses are within 1 score of the median, an IQR of 8 indicates that scores are more broadly dispersed. To be confident that agreement about parameters had been reached we defined consensus as: a median score indicating strong agreement [8–10] or strong disagreement (0–2) in combination with an IQR of 2 points or less (demonstrating a concentration of scoring around the median). In all other circumstances, less strong agree/disagree (median 3–7) or dispersed scoring (IQR > 2), consensus was not considered to have been reached.
Alongside scoring, participants were given the opportunity to offer free-text comments which might contextualize or explain their responses.
Those parameters where scoring demonstrated consensus amongst the expert panel were either accepted as a characteristic of stable glaucoma or rejected from our process. These parameters were fixed and not scored in subsequent survey rounds.
Where consensus was not achieved, parameters were amended (in accordance with previous scoring and any relevant free-text comments) in such a way as to support the generation of consensus. For example, the duration of time for monitoring stable glaucoma was increased to support the generation of panel agreement about it. Revised parameters, along with summary scores from previous rounds and any indication of changes to the parameter, were included in the next iteration of the survey for scoring.
This process was repeated twice in this amended, 3-round Delphi exercise. In the final round, for parameters where consensus was not established, participants were asked to rank options in an attempt to find a weaker form of agreement about a parameter. Also, in the final round an additional question, quantifying visual field progression in stable glaucoma, was added to further our understanding of Visual Field stability.
Results
In round 1 there were 32 responses (100%), 31 in round 2 and 29 in round 3, giving a final response rate of 90.63% [10]. Out of the 21 questions in which a consensus was reached, 10 out of 21 questions reached consensus in the first round, 7 in round two and 4 in round three (Fig. 1). The results for each clinical parameter are presented in Table 1.
Table 1.
Clinical question | Results: median (M), interquartile range (IQR), round at which consensus was reached or the question was closed (R1/ R2/ R3) | ||
---|---|---|---|
Consensus reached Strong agreement M 8–10 IQR < 2 |
No agreement reached M 3–7 or IQR > 2 |
Consensus reached Strong disagreement M 0–2 IQR < 2 |
|
Assuming a patient is diagnosed with OHT/ POAG, if all the measured parameters are stable, after how many months of monitoring would you consider the patient as stable? | |||
12 months | R2: M 2, IQR 5 | ||
24 months | R2: M 7, IQR 5 | ||
36 months | R2: M 9, IQR 3 | ||
48 months | R2: M 9, IQR 4.5 | ||
Patients with ‘High risk’ glaucoma | |||
12 months | R3: M 2, IQR 5 | ||
24 months | R3: M 6, IQR 7 | ||
36 months | R3: M 8, IQR 5 | ||
48 months | R3: M 9, IQR 3 | ||
Patients with ‘Low risk’ glaucoma | |||
12 months | R3: M 7, IQR 8 | ||
24 months | R3: M 7.5, IQR 5.25 | ||
36 months | R3: M 9, IQR 3 | ||
48 months | R3: M 10, IQR 2.25 | ||
What method of assessment should be used to define VF stability? | |||
Summary measure VFI/MD progression | R2: M 8, IQR 2 | ||
Trend analysis | R2: M 8, IQR 2 | ||
Point wise progression | R2: M 8, IQR 3.5 | ||
Which combination should be used to define VF stability | |||
VFI/ MD and Point wise progression | R2: M 7, IQR 4.5 | ||
VFI/ MD and Trend analysis | R2: M 7, IQR 3.5 | ||
Point wise progression and Trend analysis | R2: M 6, IQR 5 | ||
VFI/ MD, Point wise progression and Trend analysis | R2: M 9, IQR 4 | ||
When deciding if a glaucoma patient is stable, what method should be used, in your opinion to define adequate IOP? | |||
IOP controlled below a target IOP defined by the patient’s clinician | R1: M 9, IQR 2 | ||
IOP control of a fixed percentage (%) reduction compared to the presenting IOP | R3: M 7, IQR 3 | ||
IOP controlled below a fixed reference IOP (i.e 21 mmHg/18 mmHg) | R3: M 3, IQR 4.25 | ||
IOP controlled below a target IOP generated by an independent target IOP algorithm i.e Canadian Consensus on target IOP setting | R2: M 6, IQR 4 | ||
In your opinion what statement(s) is(are) consistent with glaucoma stability: | |||
No treatment change during the stability assessment period | R1: M 10, IQR 1.5 | ||
1 drop change for optimisation of IOP control during the assessment period for stable glaucoma | R2: M 4, IQR 4.5 | ||
Less than 2 drop changes for optimisation of IOP control during the assessment period for stable glaucoma | R2: M 1, IQR 2.5 | ||
Less than 3 drop changes for optimisation of IOP control during the assessment period for stable glaucoma | R1: M 1, IQR 2 | ||
The use of 1 agent for optimisation of IOP control during the assessment period for stable glaucoma | R2: M 7, IQR 6 | ||
The use of 2 agents for optimisation of IOP control during the assessment period for stable glaucoma | R3: M 10, IQR 10 | ||
The use of 3 agents for optimisation of IOP control during the assessment period for stable glaucoma | R3: M 10, IQR 2.5 | ||
The number of agents required for the optimisation of IOP control is not important for defining glaucoma stability | R3: M 0, IQR 0 | ||
In your opinion what method of imaging or combination of imaging techniques should be used to define structural glaucoma stability? | |||
OCT RNFL assessment alone | R2: M 5, IQR 4 | ||
OCT Optic disc structural evaluation | R2: M 5 IQR 3.5 | ||
Stereoscopic optic disc photos | R2: M 5, IQR 4.5 | ||
Optic disc photos | R2: M 3, IQR 4 | ||
Combinationsa: | |||
Optic disc photos and OCT RNFL assessment | R2: M 8, IQR 3 | ||
OCT RNFL assessment and OCT Optic disc structural evaluation | R2: M 7, IQR 3 | ||
Optic disc photos and OCT Optic disc structural evaluation | R2: M 7, IQR 5 | ||
Independent glaucoma monitoring schemes should be overseen by: | |||
Consultant ophthalmologist with glaucoma specialty expertise Consultant overview not necessary | R1: M 10, IQR 1 | R2: M2, IQR 6 | R1: M 0, IQR 2 |
Consultant Ophthalmologist without glaucoma specialty expertise | |||
When referring patients to a glaucoma monitoring service, run by optometrists with Higher Certificate Glaucoma A or Glaucoma B or level III or IV qualifications [2]: | |||
An assessment of glaucoma stability should be made before patients are transferred to a glaucoma monitoring scheme | |||
With no consultant overview: | R1: M 10, IQR 1 | ||
Overseen by a consultant w/o glaucoma expertise: | R2: M 10, IQR 0 | ||
Overseen by a consultant with glaucoma expertise: | R2: M 10, IQR 1.25 | ||
Only patients with “stable” glaucoma should be transferred to a glaucoma monitoring scheme | |||
With no consultant overview: | R1: M 10, IQR 1 | ||
Overseen by a consultant w/o glaucoma expertise: | R2: M 9, IQR 2 | ||
Overseen by a consultant with glaucoma expertise: | R3: M 8.5, IQR 1.25 | ||
Glaucoma severity is not an important factor to consider when discharging patients to a glaucoma monitoring scheme | |||
With no consultant overview: | R1: M 0, IQR 1 | ||
Overseen by a consultant w/o glaucoma expertise: | R2: M 0, IQR 1 | ||
Overseen by a consultant with glaucoma expertise: | R3: M1, IQR 2 | ||
The type of glaucoma is not important when discharging patients to a community-based glaucoma monitoring scheme | |||
With no consultant overview: | R1: M 0, IQR 2 | ||
Overseen by a consultant w/o glaucoma expertise: | R1: M 1, IQR 2 | ||
Overseen by a consultant with glaucoma expertise: | R3: M 1, IQR 1.25 | ||
The patient’s age is not an important factor to consider when discharging patients to a community-based glaucoma monitoring scheme | |||
With no consultant overview: | R2: M 1, IQR 2.25 | R2: M 1, IQR 2 | |
Overseen by a consultant w/o glaucoma expertise: | R3: M 2, IQR 2.5 | ||
Overseen by a consultant with glaucoma expertise: |
aThe combination of disc photos and OCT RNFL assessment was the highest ranking combination of imaging modalities
Strong agreement consensus was achieved that visual field stability should be assessed by trend analysis or by summary measures of VFI/ MD progression. Other methods of assessment or combinations of assessment methods did not reach consensus agreement.
The supplementary question (scored 1–4) to quantify the amount of visual field progression (MD) that can be defined as ‘stable’ found strong agreement on 0 dB of change being stable (M4) and < 4 dB being unstable (M1) with stability scoring decreasing with greater change in MD. (<1 dB:M3, <2 dB:M2).
Strong consensus agreement was reached on the following (Fig. 2):
IOP level used to define stability should be based on a clinician defined target IOP tailored for individual patients.
Having no drop treatment change during the stability assessment period is considered stable.
An increasing number of drop changes indicates instability (3 drop changes for the optimisation of IOP control during the stability assessment period is not considered ‘stable’).
There was no consensus on the number of agents used for the optimisation of IOP when defining stability and ‘The number of agents required for the optimisation of IOP control is not important for defining glaucoma stability’ (M0, IQR0) (Fig. 3).
No consensus was reached on what method or combination of imaging techniques, should be used to define structural glaucoma stability. In round 3 when respondents were asked to rank combinations of methods in order of preference, the combination of Optic disc (OD) photos (including stereoscopic disc photos) and OCT RNFL assessment was the most preferable followed by the combination of OD photos and OCT disc structural evaluation, with the combination of OCT RNFL assessment and OCT structural evaluation being the least preferred.
No consensus was achieved for length of the monitoring period required to define stability for patients identified with ‘high’ and ‘low’ risk glaucoma. There was a trend of increasing agreement with longer time periods of 36 and 48 months. This is illustrated in Fig. 4a, b.
For independent community glaucoma monitoring schemes run by optometrists with Higher Certificate Glaucoma A or Glaucoma B or level III or IV qualifications [2] there was strong agreement consensus that they should be overseen by consultants with glaucoma speciality expertise.
For all community scheme models: (1) without consultant overview, (2) overseen by general ophthalmologists and (3) overseen by a consultant with glaucoma expertise—there was strong consensus agreement that an assessment of glaucoma stability should be made before patients are transferred to a glaucoma monitoring scheme and that only patients with “stable” glaucoma should be transferred to these schemes. Severity and type of glaucoma were regarded important factors to consider when discharging patients to a community-based monitoring scheme. There was no consensus on the importance of considering a patient’s age unless referring patients to a service with no consultant overview, when it is deemed important.
Discussion
Currently, there is no definition of ‘stable glaucoma’ and there has been no previous attempt to generate a consensus definition of ‘stable glaucoma’. The Delphi method originated in the 1950s when the US Air-Force commissioned the RAND project to reach a consensus amongst military experts [11]. It has since become an established method of consensus development in the health field and has specifically been used to establish consensus in the field of glaucoma in multiple settings from developing standards for glaucoma virtual clinics [12] to developing specifications of open angle glaucoma screening interventions in the United Kingdom [13–16].
The method focuses on measuring the consensus of a group of qualified participants and has demonstrated decision-making advantages over other traditional methods [17], allowing for the discussion of complex problems whilst giving participants sufficient time to respond at their own convenience.
It has been established that the selection of the participants is likely to have little impact on the group decision as long as the selection reflects the range of experience and characteristics of the population from which the participants are selected [18]. It is not possible to make any definite statement about whether similar groups will produce similar/ the same results. Having less than six participants has low reliability and with large groups (above twelve) the increase in reliability needs to be balanced with diminishing return rates [5]. Thus, reliable outcomes can be obtained with a relatively small Delphi panel size with a response rate of over 70% [10].
Taking this into account, our panel of 32 respondents and our response rate of over 90% on the definition of ‘stable glaucoma’ carries weight for the formation of clinical guidelines. Consensus was reached on the majority of key clinical parameters and where consensus was not reached there was a strong consensus trend. We have used the consensus agreement obtained to generate a definition of stable glaucoma as follows:
IOP control should be below a target defined by the patients’ clinician—This ensures a tailored approach for each patient and allows clinicians to incorporate important factors such as age, presenting IOP, extent of visual field loss and the known rate of visual field progression into this target [19, 20].
Visual field loss can be monitored by Visual field testing with trend analysis of VFI/ MD progression—This represents a simple and practical method of assessing visual field progression used in standard clinical practice. It is no surprise that 0 dB of change is considered stable as essentially this indicates no change. Questioning if clinicians were comfortable with small amounts of visual field loss in the context of stability, we found that as larger changes in VF loss are suggested—these changes are considered progressively unstable.
No change to the medication regime indicates stability
We were unable to generate a consensus on length of time required to define stability, but our data suggest assessment of stability should take place over an extended period of time at least 36–48 months. The lack of consensus on the exact duration of follow-up required before glaucoma can be defined as stable may be a reflection of nervousness amongst clinicians in considering glaucoma a stable disease, as one respondent commented – “glaucoma is by definition a progressive condition and may progress at any time during the patients’ lifetime, even after it has been stable for many years”.
Lack of Delphi consensus on imaging techniques may indicate that when considered on their own, no single imaging technique is currently seen as sufficient or reliable for indicating stability, this may change with the development of improved technologies. However, when asked to rank the available options the combination of OD photos and OCT RNFL assessment/OCT disc structural evaluation were the preferred imaging methods for the assessment of structural stability [21]. Again this may indicate unease with relying on a single technology at present and a move towards the use of multimodality imaging when organising a monitoring service.
Based on the findings of this Delphi process, we suggest that the following could be used as a practical, working definition of stable glaucoma:
Glaucoma may be defined as “stable” when the IOP remains below the target IOP defined by the patients’ clinician, on less than three medications and requiring no medication changes over a 48-month period during which no further visual field loss monitored by Visual field testing with trend analysis of VFI/ has occurred.
The aim of this project was to identify a consensus agreement for defining stable glaucoma to allow patient entry into ‘stable glaucoma’ monitoring schemes and to determine the oversight that would be necessary to run different models of such schemes.
Despite current governance around community glaucoma schemes and Glaucoma certificates, the consensus was that all community glaucoma monitoring schemes should be overseen by consultant ophthalmologists with glaucoma speciality expertise and it is not acceptable to have no consultant overview of the scheme. This may seem counter-intuitive in the context of established recognised higher-level qualifications for optometrists which acknowledge their expertise in the assessment and management of glaucoma and the development of prescribing qualifications which allow optometrists to actively treat patients—however, the consensus may simply reflect consultants erring on the side of caution and it may be that with time, as these schemes become more established and integrated into the continuous model of care—this attitude will change.
The decision of when to transfer patients to a community monitoring scheme varies between regions, some involving clinician’s acumen, others a set of criteria given by the community provider or a combination of the two. The criteria for monitoring and referral back to Hospital Eye Services (HES) is a generally not clear and reliant on the internal governance of community providers.
In our survey, there was consensus that an assessment of glaucoma “stability” should be made prior to transferring patients to community glaucoma monitoring schemes and only patients who are considered “stable” should be transferred. The use of our definition of glaucoma stability will increase consistency and transparency within glaucoma service provision.
Other important factors to consider on discharge include: glaucoma diagnosis, severity and the patients’ age. It is interesting that regardless of the level of oversight for the community scheme, there was little difference in the results for each parameter.
When assessing patients within the community monitoring schemes, the key is to identify patients who are stable and those who do not meet the parameters of stability. Patients who are not stable need to be referred back to HES for further management and intervention. Our consensus definition helps to refine this process by providing some parameters of stability which are important regardless of the level of oversight supporting a particular scheme model.
Limitations
Ensuring confidentiality is an important aspect of formal consensus development. However, the Delphi method can be criticised for losing the benefits of face-to-face interaction which other forms of consensus development such at the nominal group technique (NGT) allow. Although the NGT may have enabled a more sophisticated and nuanced consideration of stable glaucoma, it places a greater time demand upon participants and rests upon all members of an expert group being able to attend an extended meeting (a full day) - It is hard to imagine that we would have been able to achieve this with the 32 glaucoma specialists.
The survey sample of Ophthalmology consultants was selected from registered Glaucoma Specialist Consultants who are recognised as authorities in the clinical aspects of glaucoma. However, many clinicians who are non- glaucoma specialists and health care professionals with glaucoma expertise are involved in the delivery of Glaucoma services and further study of their understanding and consensus of the parameters which are used to define stability is warranted. Although it is accepted that the selection of different representative groups of participants is unlikely to have an impact on the consensus decision [18], as this was not a random sample of all glaucoma specialists there is a possibility that there could be an unmeasurable bias of those who agreed to participate which may be reflected in the consensus outcomes reported.
It could be argued that there is a potential for bias in asking consultant ophthalmologists with glaucoma speciality expertise whether their oversight is important in running stable glaucoma monitoring schemes. However, for a consensus exercise it is appropriate to approach those most knowledgeable in a specific field in this case glaucoma for their expert opinion. This consensus outcome can then be used to inform both specialist and non-specialist of consensus-driven best practise. At present, many general ophthalmologists manage this patient cohort already and there are established optometry–led glaucoma clinics managing stable glaucoma within the hospital setting without sub-specialist ophthalmic oversight. This consensus outcome will further inform the future structure of such services.
We are unable to address this possibility directly, however there is a recognition that glaucoma consultants are already overwhelmed and insufficient in numbers to provide a service sufficient to meet the needs of the aging population [3] and unlikely therefore to want to continue to contribute to a service that they did not believe requires their oversight. Further exploration of this would be helpful and seeking opinion of non-ophthalmologists would clarify whether this opinion is shared by other health care professionals providing glaucoma care.
Conclusion
We believe this study has achieved a practical, multifactorial consensus definition of “stable” glaucoma for evaluation of transfer of patients to primary care glaucoma monitoring schemes and a consensus that all such schemes should have glaucoma consultant oversight. This will aid planning and allow consistent modelling of future primary care glaucoma monitoring schemes.
Summary
What was known before
Currently, there is no definition of stable glaucoma and there has been no previous attempt to generate a consensus definition of stable glaucoma.
What this study adds
We believe this study has achieved a practical, multifactorial consensus definition of stable glaucoma for evaluation of transfer of patients to primary care glaucoma monitoring schemes and a consensus that all such schemes should have glaucoma consultant oversight.
This will aid planning and allow consistent modelling of future primary care glaucoma monitoring schemes.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Footnotes
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Ophthalmologists. The college of Optometrists and The Royal College of. Commissioning better health care: Glaucoma. [Online] February 2013. http://www.locsu.co.uk/uploads/enhanced_pathways_2013/joint_colleges_glaucoma_guidance.pdf.
- 2.Ophthalmologists. The Royal College of. Commissioning guide: Glaucoma. [Online] June 2016. [Cited: July 20, 2018]. https://www.rcophth.ac.uk/wp-content/uploads/2016/06/Glaucoma-Commissioning-Guide-Recommendations-June-2016-Final.pdf.
- 3.The Way Forward: Glaucoma. [Online] [Cited: July 20, 2018]. https://www.rcophth.ac.uk/wp-content/uploads/2015/10/RCOphth-The-Way-Forward-Glaucoma-300117.pdf.
- 4.NICE. Guideline. NICE guidelines. [Online] 2017. [Cited: July 20, 2018]. https://www.nice.org.uk/guidance/ng81/evidence/full-guideline-pdf-4660991389.
- 5.Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J, et al. Consensus development methods, and their use in clinical guideline development: a review. Health Technol Assess. 1998;2:1–88. [PubMed] [Google Scholar]
- 6.Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;64:401–9. doi: 10.1016/j.jclinepi.2013.12.002. [DOI] [PubMed] [Google Scholar]
- 7.Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review. PLoS ONE. 2011;6:e20476. doi: 10.1371/journal.pone.0020476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach. 2017;39:14–9. doi: 10.1080/0142159X.2017.1245856. [DOI] [PubMed] [Google Scholar]
- 9.HAvd., Gracht. s. Consensus measurement in Delphi studies Review and implications for future quality assurance. Technol Forecast Soc Change. 2012;79:1525–36. doi: 10.1016/j.techfore.2012.04.013. [DOI] [Google Scholar]
- 10.Hasson F, Keeney S, McKenna HS. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008–15. [PubMed] [Google Scholar]
- 11.Linstone HA, Turoff M. The Delphi Method: techniques and applications. s.l.: Reading, MA: Addison-Wesley Publishing Company, 1975.
- 12.Kotecha A, Longstaff S, Azuara-Blanco A, Kirwan JF, Morgan JE, Spencer AF, et al. Developing standards for the development of glaucoma virtual clinics using a modified Delphi approach. Br J Ophthalmol. 2018;102:531–34. doi: 10.1136/bjophthalmol-2017-310504. [DOI] [PubMed] [Google Scholar]
- 13.Campbell SE, Azuara-Blanco A, Campbell MK, Francis JJ, Greene AC, Ramsay CR, et al. Developing the specifications of an open angle glaucoma screening intervention in the United Kingdom: a Delphi approach. BMC Health Serv Res. 2012;12:447. doi: 10.1186/1472-6963-12-447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lee PP, Sultan MB, Grunden JW, Cioffi GA. IOP Consensus PanelAssessing the Importance of IOP Variables in Glaucoma Using a Modified Delphi Process. J Glaucoma. 2010;19:281–7. doi: 10.1097/IJG.0b013e3181ca7cbe. [DOI] [PubMed] [Google Scholar]
- 15.Wilson MR, Lee PP, Weinreb RN, Lee BL, Singh K, et al. A panel assessment of glaucoma management: modification of existing RAND-like methodology for consensus in ophthalmology. Part I: Methodology and design. Am J Ophthalmol. 2008;145:570–74. doi: 10.1016/j.ajo.2007.10.019. [DOI] [PubMed] [Google Scholar]
- 16.Ismail R, Azuara-Blanco A, Ramsay CR. Consensus on outcome measures for glaucoma effectiveness trials: results from a Delphi and nominal group technique approaches. J Glaucoma. 2016;25:539–46. doi: 10.1097/IJG.0000000000000301. [DOI] [PubMed] [Google Scholar]
- 17.Adler M, Ziglio E. Gazing into the Oracle: the Delphi method and its application to social policy and public health. Bristol: Jessica Kingsley Publishers; 1996. [Google Scholar]
- 18.Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J, et al. Consensus development methods, and their use in clinical guideline development: a review. Health Technol Assess. 1998;2:37–8. [PubMed] [Google Scholar]
- 19.Clement CI, Bhartiya S, Shaarawy T. New perspectives on target intraocular pressure. Surv Ophthalmol. 2014;59:615–26. doi: 10.1016/j.survophthal.2014.04.001. [DOI] [PubMed] [Google Scholar]
- 20.Jampel HD. Target pressure in glaucoma therapy. J Glaucoma. 1997;6:133–8. doi: 10.1097/00061198-199704000-00010. [DOI] [PubMed] [Google Scholar]
- 21.Azuara-Blanco A, Banister K, Boachie C, McMeekin P, Gray J, Burr J, et al. Automated imaging technologies for the diagnosis of glaucoma: a comparative diagnostic study for the evaluation of the diagnostic accuracy, performance as triage tests and cost-effectiveness (GATE study) Health Technol Assess. 2016;20:1–168. doi: 10.3310/hta20080. [DOI] [PMC free article] [PubMed] [Google Scholar]