Diagnostic accuracy and use of non-mydriatic ocular fundus photography by emergency department physicians: Phase II of the FOTO-ED study

Abstract

Objective

During the first phase of the FOTO-ED Study, 13% (44/350;95%CI:9–17%) of patients had an ocular fundus finding, such as papilledema, relevant to their emergency department (ED) management found by non-mydriatic ocular fundus photography reviewed by neuro-opthalmologists. All of these findings were missed by ED physicians (EPs), who only examined 14% of enrolled patients by direct ophthalmoscopy. In the present study, we evaluated the sensitivity of non-mydriatic ocular fundus photography, an alternative to direct ophthalmoscopy, for relevant findings when photographs were made available for use by EPs during routine clinical care.

Methods

354 patients presenting to our ED with headache, focal neurologic deficit, visual change, or diastolic blood pressure ≥120 mmHg had non-mydriatic fundus photography obtained (Kowa nonmyd-alpha-D). Photographs were placed on the electronic medical record for EPs review. Identification of relevant findings on photographs by EPs was compared to a reference standard of neuro-ophthalmologist review.

Results

EPs reviewed photographs of 239 patients (68%). 35 patients (10%;95%CI:7–13%) had relevant findings identified by neuro-ophthalmologist review (6 disc edema, 6 grade III/IV hypertensive retinopathy, 7 isolated hemorrhages, 15 optic disc pallor, and 1 retinal vascular occlusion). EPs identified 16/35 relevant findings (sensitivity:46%;95%CI:29–63%), and also identified 289/319 normal findings (specificity:96%; 95%CI:87–94%). EPs reported that photographs were helpful for 125 patients (35%).

Conclusions

EPs used non-mydriatic fundus photographs more frequently than they perform direct ophthalmoscopy, and their detection of relevant abnormalities improved. Ocular fundus photography often assisted ED care even when normal. Non-mydriatic ocular fundus photography offers a promising alternative to direct ophthalmoscopy.

Introduction

Background

During the first phase of the Fundus photography vs. Ophthalmoscopy Trial Outcomes in the Emergency Department (FOTO-ED) study, we found that emergency physicians (EPs) performed direct ophthalmoscopy on only 48 of 350 of patients (14%) who presented to our university hospital emergency department (ED) with complaints and conditions warranting ocular fundus examination.^1,2

Importance

Systematic fundus photographs of the 350 patients enrolled during the first phase of our study showed that 44 patients (13%;95%CI:9–17%) had a finding of potential relevance to their ED management and disposition, such as papilledema or grade III/IV hypertensive retinopathy. These photographs were not made available to the EPs during the first phase, and none of these funduscopic findings were identified solely by EPs performing direct ophthalmoscopy, emphasizing that direct ophthalmoscopy is not only rarely performed by EPs but is usually not reliable.^1,2

During the first phase of the FOTO-ED study, the EPs had to rely only on direct ophthalmoscopy and ophthalmology consultation to assist in their ocular fundus diagnoses. Herein we report the second phase of the FOTO-ED study in which we routinely provided non-mydriatic ocular fundus photography to EPs during their clinical evaluations.

Goals of This Investigation

Our aim was to determine whether the availability of non-mydriatic ocular fundus photography for interpretation by EPs would provide better sensitivity for relevant ocular fundus findings than the availability of direct ophthalmoscopy during the first phase of the FOTO-ED study.

Methods

(Detailed methodology is available in the associated Web Appendix.)

Study Design

The FOTO-ED study was a two-phase sequential, cross-sectional study designed to compare the routine clinical use of direct ophthalmoscopy performed by attending EPs (in phase I) to the routine clinical use of non-mydriatic ocular fundus photography interpreted by attending EPs (in phase II). In both phases, all patients had non-mydriatic ocular fundus photographs obtained, with concurrent interpretation of the fundus photography by neuro-ophthalmologists serving as the reference standard. EPs were masked to the interpretations of the neuro-ophthalmologists, and vice versa.

Study Setting and Population

Adult patients presenting to our university hospital ED between May and August 2011 with a predetermined set of triage chief complaints (i.e., headache, acute focal neurologic deficit, acute visual changes, or a triage diastolic blood pressure ≥120 mmHg) were included.

Protocol

Non-mydriatic photographs of the posterior pole of the ocular fundus (optic disc, macula and major retinal vessels) were obtained from both eyes of enrolled patients using a commercially available non-mydriatic ocular fundus camera (Kowa α-D, Torrence, CA) by a nurse practitioner or medical student according to a previously described method.²

Photographs were immediately placed in the patient’s electronic medical record (Figure 1) and attending EPs were notified that photographs were available. EPs recorded their findings on a standardized case report form. EPs were specifically instructed to fill out the case report forms before they obtained any knowledge of possible ocular fundus findings from a third party.

Figure 1.

Viewing a patient’s non-mydriatic photograph on the electronic medical record (Cerner PowerChart) during the FOTO-ED study.

Outcome Measures

The primary outcome measure was the EPs’ detection of ocular funduscopic findings relevant to acute patient care in the ED (i.e., findings that should or could change the patient’s acute evaluation, treatment, and disposition). Relevant ocular fundus abnormalities were defined a priori as optic disc edema, optic disc pallor, retinal vascular occlusions, intraocular hemorrhages, and Grade III/IV hypertensive retinopathy.¹ Photographs were reviewed for the presence or absence of relevant ocular fundus abnormalities by neuro-ophthalmologists masked to all patient data including chief complaints as a reference standard.

Statistical Analysis

Statistical analysis was performed using R: A language and environment for statistical computing (R Foundation for Statistical Computing, http://www.R-project.org) by BBB. The unit of analysis was the patient. Medians and interquartile ranges (IQR) were reported for continuous data and percentages were reported for categorical data. Two-tailed p-values <0.05 were considered statistically significant. Proportions were calculated with 95% confidence intervals by the exact binomial method. Proportions were compared using asymptotic normal confidence intervals of the absolute difference.

Results

Among 478 patients screened for eligibility, 354 patients were enrolled in phase two of the FOTO-ED study (Figure 2). Eighty-six patients were ineligible, and 33 patients refused participation. The automated process identified 345 enrolled subjects (97%), with the remainder identified by active surveillance by study personnel. Five eligible patients (three headache, two focal neurologic) who triggered the automated process were missed by study staff.

Figure 2.

Flow of patients through the study.

Two hundred six (58%) patients had headache, 123 (35%) had focal neurologic symptoms, 56 (16%) had acute visual changes, and 21 (6%) had diastolic blood pressure of ≥120 mmHg (patients were allowed to have more than one chief complaint). Patient demographics and photographic quality are reported in the Table.

Table.

Comparison of phase I and II of the FOTO-ED study.

	Phase I1,2,6 (Direct ophthalmoscopy)	Phase II (Non-mydriatic photography)
Number of patients	350	354
Median age, years (interquartile range)	44.5 (31–59)	45.9 (33–57)
Number of women (%)	220 (63)	251 (71)
Chief complaints (%)*
- Headache	228 (65)	206 (58)
- Focal neurologic deficit	100 (29)	123 (35)
- Visual changes	92 (26)	56 (16)
- DBP ≥ 120 mmHg	21 (6)	21 (6)
Quality of photographs (%)
- Patients with at least one high quality (grade 4 or 5) photograph	297 (85)	268 (76)
- Patients with high quality of both eyes	215 (61)	226 (64)
- Patients with no photographs of diagnostic value (grade 1)	12 (3)	4 (1)
Number of relevant findings (%)	44 (13)	35 (10)
- Optic disc edema	13	6
- Grade III/IV HTN retinopathy	10	6
- Isolated intraocular hemorrhage	13	7
- Optic disc pallor	4	15
- Retinal vascular occlusion	4	1
ED physician fundus examination technique	Direct ophthalmoscopy	Non-mydriatic photography
Number of patients whose ocular fundus was viewed by ED physicians (%)	48 (14)	239 (68)
Number of abnormalities correctly detected by ED physician examination (%)♦	0 (0)	16 (46)

^*Note these sum to more than 100% because patients were allowed to report more than one complaint,

^♦absolute difference: 46%; 95%CI: 29–62%

FOTO-ED=Fundus photography vs. Ophthalmoscopy Trial Outcomes in the Emergency Department; DBP=diastolic blood pressure; HTN=hypertensive

Thirty-five patients (10%; 95%CI: 7–13%) had relevant findings identified by neuro-ophthalmologist review of the photographs, including six patients with disc edema, six with grade III/IV HTN retinopathy, seven with isolated intraocular hemorrhages, 15 with optic disc pallor, and one with a retinal vascular occlusion.

Among the 354 enrolled patients, the EPs reviewed the photographs of 239 patients (68%), and reported that the photographs were helpful in their evaluation of 125 patients (35%; 95%CI: 30–41%). The EPs identified 16 of the 35 relevant findings (46%) during their review of the fundus photographs. Eight of the relevant findings (24%) occurred in patients whose photographs were not reviewed by EPs. The remaining 11 of the patients with relevant findings had their photographs reviewed by EPs but the abnormalities were recorded as “likely normal, but unsure” (seven patients) and “normal” (four patients). These 11 abnormal findings included six optic disc pallor, three isolated flame-shaped hemorrhages, one grade III/IV hypertensive retinopathy, and one disc edema. All of these findings occurred in patients with either a complaint of headache (six of 11 patients; one also with visual changes and one with severely elevated blood pressure), focal neurologic deficit (two patients), or both (three patients). The patients that were mislabeled as “likely normal” or “normal” had photographs of good quality (i.e., images of grade ≥3 quality for both eyes).

Thirty patients (8%) were reported as having “likely abnormal, but unsure” or “abnormal” photographs, but did not have relevant findings based on neuro-ophthalmologist review. Twenty-five of these 30 image-sets (83%) were either of poor quality (9 patients) or represented the misidentification of a camera artifact as pathology (16 patients). The artifacts were either smudges on the camera lens or reflections from ocular structures. All of the five misreads of high-quality image-sets incorrectly indicated optic nerve pathology in normal patients.

Compared to a reference standard of photograph review by neuro-ophthalmologists, the EPs’ using or not using fundus photographs had 46% sensitivity (16/35; 95%CI: 29–63%) and 96% specificity (289/319; 95%CI: 87–94%). When comparing these results to those of phase I of the FOTO-ED study in which photographs were not made available for EP review, EPs had significantly better sensitivity for relevant ocular fundus findings with non-mydriatic fundus photography than with their use or non-use of direct ophthalmoscopy alone (16/35 vs. 0/44^1,2, absolute difference: 46%; 95%CI: 29–62%). These calculations considered unperformed direct ophthalmoscopy (in phase I) and unread photographs (in phase II) as “normal” because this equates sensitivity to the overall EP detection rate for abnormal findings (the primary outcome). However, when limiting the analysis to the 239 patients whose photographs were actually reviewed by EPs, sensitivity was 59% (16/27; 95%CI: 39–78%) and specificity was 86% (182/212; 80–90%).

Limitations

There are several limitations to our study. First, in this second phase of the FOTO-ED study, the EPs were notified that photographs were available for their review both by case report forms and by a notification on the ED census screen in the electronic medical record. In the first phase of the study, the EPs had no stimulus to examine the ocular fundus beyond being aware that a study on the topic was ongoing in the ED. This may have led to a higher use of non-mydriatic fundus photography in the second phase (68%) compared to direct ophthalmoscopy in the first phase (14%). However, it seems unlikely that notification alone accounts for the EPs using photographs five times more frequently than direct ophthalmoscopy, nor can it completely explain their substantially improved ability to detect abnormalities (46% vs. 0%). Furthermore, since 32% of the patients did not have their photographs reviewed by EPs, despite notification alerts, including 23% of patients with relevant abnormalities, it is clear that our method of notification alone does not lead to consistent examination of the fundus. Second, while the EPs were instructed to fill out their case report forms before they were aware of the results of other testing or consultation, we cannot be certain that they were always filled out in a blinded fashion. If they were not, this could have inflated the EPs’ detection of relevant findings. Third, while we provided no specific education to the EPs, the Hawthorne effect (improved performance under observation) may have led some physicians to be more vigilant than usual, and some may have sought additional education on their own to assist them in the review of fundus photographs. Fourth, the specific reason the photograph was considered helpful, although solicited, was infrequently reported by EPs; even when reported, the direct impact of the photographs on a specific patient’s work-up, consultation, and disposition cannot be known. Fifth, although patients were photographed as early during their ED stay as possible, some patients were photographed later in their stay, potentially after disposition decisions were made. Finally, our study population had convenience aspects due to the limited availability of study staff. However, even if we were to assume that patients presenting during the nights and weekends were substantially different than patients presenting during weekdays (an assumption not supported by our nights and weekend data), our findings would still have important implications for the care of patients in the ED during the time periods studied.

Discussion

This study demonstrated that the routine use of non-mydriatic fundus photography, interpreted by the EPs themselves, had higher sensitivity for relevant ocular fundus findings than the EPs’ typical use of direct ophthalmoscopy. Indeed, EPs without any specific training in interpretation of fundus photography identified 46% of the relevant findings during this phase, compared to their identification of none of the relevant findings during the previous phase of the FOTO-ED study when only direct ophthalmoscopy was available (Table).^1,2 Additionally, EPs correctly identified as normal 86% of the patients with normal fundi whose photographs they reviewed.

We found that findings relevant to emergency care occurred in 10% of the patients, similar to the 13% we observed in the first phase of the FOTO-ED study.¹ However, the relevance of fundus photography should not be judged solely on the abnormalities detected. In fact, EPs reported that the photographs were helpful in over a third of cases overall and in nearly 50% of the cases in which they reviewed the photographs, indicating that the ability to verify the absence of certain findings on ocular fundus examination was as important, if not more important, to ED clinical care situations than identifying abnormalities. Examples of such types of cases include the correct recognition of absence of papilledema among headache patients and the absence of malignant hypertensive retinopathy in patients with hypertensive urgency.

In this second phase of the FOTO-ED study, the EPs did not have any additional training in interpreting the photographs. This was so designed to make the EPs’ use of the fundus photographs as comparable as possible to their prior use of routine direct ophthalmoscopy. In addition, the two phases were performed sequentially, rather than concurrently, to avoid influencing the physician’s routine use of direct ophthalmoscopy during the first phase. The EPs’ substantially better performance on non-mydriatic ocular fundus photography review compared to direct ophthalmoscopy was compatible with one small previous study in which three of 14 physicians correctly identified papilledema with the direct ophthalmoscope compared to 10 of 14 correct identifications based on review of fundus photographs.³

It is important to emphasize that ED screening of ocular fundus photography is primarily useful for triage and disposition decisions. It obviously cannot replace expert review of the photographs by ophthalmologists nor appropriate ophthalmologic consultation in cases of abnormal or questionable findings or in patients with primarily visual complaints. However, digital photographs can be easily transmitted to other locations for review, including onto mobile devices such as smartphones.⁴ Indeed, combining non-mydriatic photography with telemedicine has the potential not only to facilitate patient triage, but to provide underserved communities with greater access to specialty ophthalmic care in a timely fashion.⁵ The cost-effectiveness of non-mydriatic ocular fundus photography in the ED setting will require further study.

In conclusion, non-mydriatic ocular fundus photographs were used more frequently than direct ophthalmoscopy by EPs, and were significantly more sensitive for the detection of relevant abnormalities when interpreted by the EPs than their direct ophthalmoscopy. Non-mydriatic ocular fundus photography was deemed helpful in over a third of ED patient evaluations requiring examination of the ocular fundus, not only for identifying abnormalities but also for excluding relevant pathology. This second phase of the FOTO-ED study supports a role for non-mydriatic fundus photography in the routine care of relevant ED patients. In the next phase of the FOTO-ED study, we plan to investigate whether training in the interpretation of fundus photographs further improves EP performance and facilitates appropriate ED management.