Is ophthalmology evidence based? A clinical audit of the emergency unit of a regional eye hospital

Abstract

Aim: To evaluate the proportion of interventions that are evidence based in the acute care unit of a regional eye hospital.

Methods: A prospective clinical audit was carried out at Hong Kong Eye Hospital in July 2002 to investigate the extent to which ophthalmic practices were evidence based. The major diagnosis and intervention provided were identified through chart review. A corresponding literature search using Medline and the Cochrane Library was performed to assess the degree to which each intervention was based on current, best evidence. Each diagnosis intervention pair was accordingly analysed and graded. The level of best, current evidence supporting each intervention was graded and analysed.

Results: A total of 274 consecutive consultation episodes were examined. 22 cases were excluded since no diagnosis or intervention was made during the consultation. 108 (42.9%) patient interventions were found to be based on evidence from systematic reviews, meta-analyses, or randomised controlled trials (RCT). Evidence from prospective or retrospective observational studies supported the interventions in 86 (34.1%) patients. In 58 (23.0%) cases, no evidence or opposing evidence was found regarding the intervention. The proportion of evidence based on RCT or systematic reviews was higher for surgical interventions compared with non-surgical interventions (p=0.007). The proportion of interventions based on RCT or systematic reviews was higher for specialist ophthalmologists than trainee ophthalmologists (p=0.021).

Conclusion: This study demonstrated that the majority of interventions in the ophthalmic unit were evidence based and comparable to the experience of other specialties.

Evidence based medicine (EBM) is the “conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”¹ The concept of EBM has gained worldwide popularity in the past decade and aims at encouraging clinical practice to be grounded in scientific inquiry.² In addition to keeping physicians up to date with the most current medical knowledge, EBM also sets out to support the provision of quality care and minimisation of medical error and to facilitate a more equitable provision of services.^2–4

As the quality of scientific evidence varies in the literature, examining the study design may provide clues to determine whether and to what degree the findings are valid and conclusive. The Oxford Centre for Evidence-Based Medicine has developed reviews and a “levels of evidence” scheme for classifying studies based on the epidemiologic design.⁵ Systematic reviews and randomised controlled trials (RCT) are rated as the highest level of evidence followed by cohort studies and case-control studies. Case series and expert opinion give the lowest level of evidence.

Before the 1995 study by Ellis et al, it has been suggested that only 10% to 20% of current medical interventions were supported by scientific evidence.⁶ Ellis and colleagues demonstrated that 53% of interventions in a general medical unit were based on evidence from RCT, while an additional 29% were based on reliable non-experimental or observational evidence. Although criticisms have been raised about the generalisability of this study, a later study by Michaud et al reported similar findings.⁷ Subsequent studies in various other specialties including general practice,⁸ paediatric surgery,⁹ dermatology,¹⁰ anaesthesia,¹¹ and general surgery¹² have evaluated the extent to which common interventions were evidence based. Together, these studies estimated the proportion of interventions that were supported by observational and/or experimental studies ranged from 71% to 97%.

To our knowledge, there has been no formal study on the proportion of evidence based interventions in ophthalmology. Furthermore, this type of study has not been performed in any specialty in Asia. The purpose of this study was to investigate the proportion of ophthalmic interventions that were evidence based in a large, regional teaching eye hospital.

PATIENTS AND METHODS

All patients presenting to the acute care unit (that is, emergency department) of Hong Kong Eye Hospital during seven consecutive sessions in July 2002 were prospectively recruited. The hospital is a tertiary ophthalmic centre serving the Kowloon East cluster with a coverage population of approximately 1.6 million out of 6.8 million in Hong Kong. Medical staff consists of four consultants, 13 specialist ophthalmologists, and 21 trainee ophthalmologists. To minimise Hawthorne's phenomenon (the potential bias associated with subjects being aware that they were being observed), none of the participating ophthalmologists involved was aware of the study during the recruitment period. After each session, all medical charts were retrieved for analysis. A predefined, standardised set of data was collected for each consultation including patients' characteristics and the rank of the ophthalmologist who performed the consultation. The primary diagnosis and primary intervention were independently determined by two of the authors (TL and VW) who were not involved in attending the consultations during the study period. The primary diagnosis was defined as the problem recorded for the consultation that was most responsible for the patient's presentation and the primary intervention as the most important attempt to manage or treat the patient in respect to the primary diagnosis.⁶ The primary interventions were classified as medical interventions which may include conservative or expectant treatment or the use of pharmaceutical products, or as surgical interventions if procedures included open surgery or laser treatment. Cases were excluded when a primary diagnosis could not be ascertained. Patients who refused or did not consent to the prescribed interventions were also excluded. In cases where there was disagreement in assigning diagnosis or intervention, the authors resolved these by consensus.

Literature review was performed after all the diagnosis and intervention pairs were confirmed. The same two authors performed a comprehensive search through Medline (1966 to August 2002) and the Cochrane Library, Issue 3, 2002, independently and disagreement was again settled by consensus. Only publications in English were selected. The evidence obtained was then graded into five hierarchical categories as described by Kingston et al (Table 1).¹² This classification is similar to the classification by the Oxford Centre of Evidence-based Medicine, in which systematic reviews or RCT were given the highest level of evidence.⁵ However, expert opinion was not considered as proper scientific evidence in this classification. When there was more than one article for an intervention, the study with a higher level of evidence was chosen. For articles at the same level, the more recent article was selected.

Table 1.

Levels of evidence according to Kingston et al¹² (n=252)

Level	Study design	No of patients (%)
1	Systematic review	26 (10.3%)
2	Meta-analysis	2 (0.8%)
3	Randomised controlled trial	80 (31.7%)
4	Prospective study	34 (13.5%)
5	Retrospective study	52 (20.6%)
None	No evidence or evidence opposed the intervention	58 (23.0%)

Statistical analyses were performed using spss for Windows v 10.0. Categorical outcomes were compared using two tailed Pearson's χ² test.

RESULTS

During the study period, 274 consultations were performed by 16 different ophthalmologists (five specialist ophthalmologists and 11 trainee ophthalmologists). The mean number of consultations per ophthalmologist was 17.1 (range 9–36). Patients' mean age was 49.5 years (range 1–87 years). Males comprised 51.8% of the sample population. For the ascertainment of the primary diagnosis and the primary intervention, there was agreement between the two authors in 270 of 274 (98.5%) and in 251 of 252 (99.6%) patients respectively. Twenty two (8.0%) patients were excluded since no diagnosis were made during the consultation (Fig 1). In the remaining 252 cases, 26 (10.3%) and two (0.8%) of the interventions were supported by systematic reviews and meta-analyses, respectively (Table 2). Evidence from RCT supported the management of 80 (31.7%) patients (Table 3). Prospective and retrospective observational studies provided evidence for the interventions in 34 (13.5%) and 52 (20.6%) patients, respectively (Tables 4 and 5). The total number of interventions which were evidence based—that is, with level 1 to 5 evidence, was 194 (77.0%).¹² For the remaining 58 (23.0%) diagnosis-intervention pairs, no evidence could be identified to substantiate or refute the intervention rendered (Table 6).

Figure 1.

Study organisation.

Table 2.

Interventions based on systematic reviews or meta-analyses (n=28)

Primary diagnosis	Primary intervention	No of patients	Reference
Infective conjunctivitis	Topical antibiotics	19	13
Senile cataract	Phacoemulsification + implantation of intraocular lens	5	14
Dendritic ulcer due to herpes simplex virus	Topical aciclovir	2	15
Primary open angle glaucoma	Topical timolol	1	16
Primary pterygium	Pterygium excision + conjunctival autograft	1	17

Table 3.

Interventions based on randomised controlled trials (n=80)

Primary diagnosis	Primary intervention	No of patients	Reference
External eye foreign body/rust ring	Removal of foreign body	20	18
Chalazion	Incision and curettage	19	19
Anterior uveitis	Topical steroid	7	20
Allergic conjunctivitis	Topical emedastine	4	21
Corneal abrasion	Topical chloramphenicol	4	22
Keratoconjunctivitis sicca	Topical lubricants	4	23
Acute central serous retinopathy	Observation	2	24
Blepharitis	Topical chloramphenicol	2	25
Inflamed pterygium	Topical steroid	2	26
Acute angle closure glaucoma	Nd:YAG laser iridotomy	1	27
Allergic conjunctivitis	Topical lodoxamide	1	28
Blepharitis	Topical steroid-antibiotic	1	29
Branch retinal vein occlusion with vitreous haemorrhage	Panretinal photocoagulation	1	30
Central retinal vein occlusion with neovascular glaucoma	Panretinal photocoagulation	1	31
Chlamydial conjunctivitis	Topical chlorotetracycline	1	32
Corneal abrasion	Bandage contact lens	1	33
Giant cell arteritis	Oral prednisolone	1	34
Herpes simplex virus disciform keratitis	Topical aciclovir and topical steroid	1	35
Herpes simplex virus iridocyclitis	Topical aciclovir and topical steroid	1	36
Macular oedema due to diabetic retinopathy	Argon laser focal photocoagulation	1	37
Moderate non-proliferative diabetic retinopathy	Observation	1	38
Primary open angle glaucoma	Topical betaxolol	1	39
Postoperative viscoelastics induced rise in intraocular pressure	Systemic acetazolamide + topical β blocker	1	40
Recurrent corneal erosion syndrome	Topical antibiotics	1	41
Rhegmatogenous retinal detachment	Pars plana vitrectomy + encircling + intravitreal sulfur hexafluoride gas	1	42

Table 4.

Interventions based on prospective case series (n=34)

Primary diagnosis	Primary intervention	No of patients	Reference
Subconjunctival haemorrhage	Measurement of blood pressure	13	43
Exposed/loosened corneal sutures	Removal of corneal sutures	10	44
Filamentary keratitis	Debridement of corneal filaments	4	45
Posterior capsular opacity	Nd:YAG capsulotomy	2	46
Aponeurotic ptosis	Correction of ptosis	1	47
Blepharitis	Lid hygiene	1	48
Chalazion	Warm massage	1	49
Chronic angle closure glaucoma	Sequential argon/Nd:YAG laser iridotomy	1	50
Exposure keratopathy	Topical lubricants + eyes taping	1	51

Table 5.

Interventions based on retrospective case series (n=52)

Primary diagnosis	Primary intervention	No of patients	Reference
Acute posterior vitreous detachment/floaters	Subsequent reassessment	26	52
Corneal chemical injury	Topical antibiotics	6	53
Sectorial episcleritis	Topical steroid	4	54
Screening for ethambutol toxic optic neuropathy	Reassessment	2	55
Sixth nerve palsy due to microvasular disease	Observation	2	56
Lattice degeneration	Barrier laser	2	57
Acute angle closure glaucoma	Systemic acetazolamide	1	58
Exposed scleral buckle	Removal of buckle	1	59
Involutional lower lid entropion	Entropion correction	1	60
Mooren's ulcer	Topical + systemic steroid	1	61
Phacomorphic glaucoma	Antiglaucomatous followed by cataract extraction	1	62
Posner-Sclossman syndrome	Systemic acetazolamide + topical β blocker	1	63
Preseptal celluitis	Oral ampicillin	1	64
Recurrent corneal erosion syndrome	Debridement of loosened epithelium	1	65
Silicone oil induced glaucoma	Topical β blocker	1	66
Toxic keratopathy	Use of non-preserved lubricants	1	67

Table 6.

Interventions in which no evidence was found or evidence found to oppose the intervention (n=58)

Primary diagnosis	Primary intervention	No of patients	Reference
Corneal abrasion	Eye patching	7	68
Chalazion	Topical steroid	5	None
Blepharitis	Topical chlortetracycline	4	None
Blepharitis	Topical lubricants	4	69
Chalazion	Topical antibiotics	3	None
Corneal abrasion	Topical levofloxacin	3	None
Subconjunctival haemorrhage	Topical lubricants	3	None
Trichiasis	Epilation of trichiasis	3	70
Conjunctival abrasion	Topical antibiotics	2	None
Eroded conjunctival concretion	Pricking of concretion	2	None
Traumatic microhyphaema	Topical steroid	2	71
Blepharitis	Topical sulfacetamide	1	72
Blepharitis	Topical antazoline + tetrahydrozoline	1	None
Chemical injury	Topical lubricants	1	None
Chlamydial conjunctivitis	Topical sulfacetamide	1	None
Conjunctiva foreign body	Irrigation	1	None
Corneal abrasion	Removal of loose epithelium	1	None
Epiphora without nasolacrimal duct obstruction	Topical zinc sulfate + naphazoline	1	None
Follicular conjunctivitis	Topical chloramphenicol	1	None
Infective conjunctivitis	Topical lubricants	1	None
Limbitis with blepharitis	Topical steroid + chlortetracycline	1	None
Marginal keratitis	Topical steroid	1	None
Nasolacrimal duct obstruction	Topical sulfacetamide	1	None
Phlyctenulosis	Topical steroid + antibiotics	1	None
Pingueculitis	Topical steroid + antibiotics	1	None
Postoperative stitch infiltrate	Topical steroid + antibiotics	1	None
Recurrent corneal erosion syndrome	Lubricating ointment	1	73
Retained meibomian gland secretions	Pricking of secretions	1	None
Retrobulbar optic neuritis	Oral prednisolone	1	74
Sectorial episcleritis	Topical lubricants	1	None
Subconjunctivial haemorrhage	Topical antibiotics	1	None

The primary intervention was classified as a medical intervention in 168 (66.7%) cases, while the remaining 84 (33.3%) cases were surgical interventions. The proportion of interventions that were evidence based was higher in the surgical group compared to the medical group but was not statistically significant (p=0.09). Among the 194 evidence based interventions, the quality of evidence was better in the surgical group with 48 of 70 (68.6%) interventions being supported by RCT or better compared to 60 of 124 (48.4%) interventions in the medical group (p=0.007).

Among all 252 interventions, 65 (25.8%) were delivered by specialist ophthalmologists and 187 (74.2%) were performed by trainee ophthalmologists. There was no difference in the proportion of evidence based interventions performed by the two groups (p=0.50). The respective proportions of medical and surgical interventions performed by each group were also similar (p=0.88). However, the quality of evidence based interventions performed by the specialist group was better compared with the trainee group, with evidence based on RCT or better supporting 36 of 52 (69.2%) interventions and 72 of 142 (50.7%) interventions in the specialist ophthalmologist and the trainee ophthalmologist groups respectively (p=0.021).

A total of 86 different interventions could be identified. The eight most common interventions accounted for the treatment of 121 (48.0%) patients (Table 7). All but one of these interventions were evidence based.

Table 7.

The eight most common interventions accounted for the treatment of 121 patients

Primary diagnosis	Primary intervention	No of patients	Level of evidence
Acute posterior vitreous detachment/floaters	Subsequent reassessment	26	Retrospective study
External eye foreign body/rust ring	Removal of foreign body	20	RCT
Chalazion	Incision and curettage	19	RCT
Infective conjunctivitis	Topical antibiotics	19	Systematic review
Subconjunctival haemorrhage	Measurement of blood pressure	13	Prospective study
Exposed/loosened corneal sutures	Removal of corneal sutures	10	Prospective study
Anterior uveitis	Topical steroid	7	RCT
Corneal abrasion	Eye patching	7	No evidence

DISCUSSION

Our study demonstrated that the majority of interventions (77.0%) in our regional ophthalmic unit were evidence based. The proportion of evidence based interventions in our study was comparable to findings from studies in other specialties, where around 80% of interventions were found to be grounded in scientific evidence.^6–10,12 Furthermore, evidence from RCTs (level 3) or better could be identified in 108 (42.9%) patients. This was also within the range of previous studies, in which 11% to 53% of interventions were found to be based on evidence from RCT or better.^6–12

Ophthalmology is a unique specialty in which a mix of both medical and surgical interventions are routinely performed. It is well known that randomised controlled trials are generally harder to conduct for surgical treatments and previous studies have shown that the proportion of evidence from RCTs was lower in surgical studies compared with studies in internal medicine.¹² In our study, we found that the proportion of interventions based on RCT or better was significantly higher for surgical interventions compared to non-surgical ophthalmic interventions. This might have been because of the rapid advancement in ophthalmic surgical techniques and new instrumentations, thereby encouraging more RCTs being performed for surgical interventions. In addition, the non-surgical interventions in our study were generally prescribed for less serious and self limiting conditions (for example, subconjunctival haemorrhage, acute posterior vitreous detachment), where it may be difficult or even unnecessary, to carry out RCTs to generate evidence.

Of note, the proportion of interventions with evidence based on RCTs or better was higher in the specialist ophthalmologist group compared with the trainee ophthalmologist group. This may be due to the more experienced specialist group being more aware of the evidence available in the literature. The seniority of the specialist ophthalmologists may also allow them to be involved in the development of clinical guidelines and protocols for the unit and this process would have allowed them to be better equipped in practising EBM. Therefore, our results suggest that the more senior doctors are actually practising EBM rather than “eminence” based medicine as some have previously suggested.⁷⁵

In this study, we only considered evidence to be valid if it was published in the medical literature. We did not use any textbooks or expert panels as the source of evidence since these sources have been suggested not to constitute “good” evidence and therefore should be excluded from studies of this nature.¹² Other potential limitations in our study also bear mention. During the literature review, we may not have identified all the relevant literature since not all available scientific databases were searched, although Medline and the Cochrane library have often been cited as the “gold” standard for this type of study.^12,76 Also, language bias may have occurred as only publications in English were assessed. The reductionist approach of having a primary diagnosis and primary intervention pairing may have led to a loss of practical reality in the actual clinical setting. Our clinical setting also made the results less generalisable since the study was conducted within the acute care unit and the proportion of evidence based intervention might differ among different subspecialty clinics as well as between other hospitals. Future research should extend this study methodology to different hospital subspecialty clinics and other centres to increase the external validity and representativeness. Lastly, we only selected the single best study on the basis of the study design for analysis to preserve clarity and simplicity. However, one of the weaknesses of EBM is the heavy emphasis on RCT where other types of evidence may be devalued, making a poor quality RCT more influential than valid non-randomised studies.⁷⁷ Ideally, the quality of each individual randomised controlled trials should be examined and any conflicting results from various randomised controlled trial should be resolved through systematic reviews or meta-analyses.

In summary, our study has demonstrated that most of the current ophthalmic interventions in the acute care unit of the Hong Kong Eye Hospital were evidence based. Routine audits similar to our study may be beneficial since they can provide quality assurance to the provision of optimal patient care.⁷ Through this study, we have highlighted areas in which there was limited evidence and have identified areas of deficiency in the literature for future research.