Tele-ophthalmic Approach for Detection of Corneal Diseases: Accuracy and Reliability

Abstract

Purpose

Corneal and anterior segment diseases cause the majority of urgent visits to eye-care professionals. We evaluated the diagnostic accuracy of detecting corneal diseases using external photographs from two portable cameras for telemedicine purposes.

Methods

A prospective study of adults with a clinical diagnosis of corneal pathology including corneal abrasions, ulcers, scars, and pterygia. A corneal specialist provided the gold-standard diagnosis by slit-lamp examination. Images of both eyes were obtained using iTouch 5S and Nidek VersaCam cameras in multiple gazes and interpreted by 3 cornea specialists for presence of pathology. Accuracy to detect disease was compared to gold standard diagnosis, stratified by camera and grader. Reliability was evaluated with weighted kappa statistics. Graders assessed image quality on a Likert scale from 1 (poor) to 9 (optimal).

Results

198 eyes (110 subjects) were photographed. By gold standard diagnosis, 59 eyes (30%) had corneal scars, 34 (17%) had ulcers, 13 (7%) had abrasions, 10 (5%) had pterygia, and 82 (41%) were normal. Sensitivity to detect AS pathology ranged from 54–71% for iTouch and 66–75% for Nidek, across graders; specificity ranged from 82–96% for iTouch and 91–98% for Nidek. Inter-grader reliability was moderate to strong (kappa ranges: 0.54–0.71 for iTouch; 0.75–0.76 for Nidek. Quality ratings were variable between graders.

Conclusions

External photographs taken by standard, non-enhanced portable cameras and interpreted remotely by ophthalmologist graders yielded sensitivity values that are not yet suitable for telemedicine applications. Additional work is needed to improve the ability to detect AS pathology remotely.

Introduction

Remote eye care is a promising solution for millions of Americans with inadequate access to eye care services.^1–16 One-third to one-half of Americans are at high risk for visual loss and do not or rarely see an eye care provider.^1–4,17 In addition, the ophthalmic workforce will not keep up with the demand for eye care and may become even less accessible in many rural or low-income communities with limited transportation.^6,7,18–22 In much of the United States, when people develop eye and vision problems, they go to primary care providers or emergency departments.^{6,15,18,23,24} Constraints for optimal care from primary care providers include the lack of ophthalmic expertise, insufficient time, and inadequate referral systems. The National Eye Institute (NEI) states that corneal diseases cause the majority of visits for eye care.²⁵ The NEI’s strategic plan specifically identifies telemedicine as a modality for detection of eye disease and visual impairment and emphasizes the importance of research that can “establish the safety, reliability, and feasibility of telemedicine procedures through pilot studies.” The NEI mission also states that research should “determine which ophthalmic applications are amenable to telemedicine, while still preserving the accuracy, sensitivity, and specificity achieved with in-person examinations.”²⁶

A rigorous evaluation of corneal imaging for telemedicine is crucial in the evolution of ophthalmic telemedicine for several reasons. First, many current telemedicine programs for detection of diabetic retinopathy already capture two diffuse light photographs of the anterior eye. We do not know the diagnostic accuracy of diffuse light anterior eye photographs, and the photographs may be causing false assurances about the health of eye. Second, since people with corneal diseases often present to primary (non-eye) care providers, we need tools to help these providers guide treatment. Globally, bilateral corneal blindness affects 4.9 million people and unilateral blindness may affect 23 million people based on estimates from the World Health Organization and prevalence work in India.^27–29 A method to diagnose and guide management of corneal diseases remotely could help vision outcomes, especially for patients in low resource settings. We designed a study to assess the accuracy and reliability of images taken from two portable cameras to detect the presence of corneal disease, namely corneal abrasions, ulcers, scars, and pterygia.

Materials and Methods

This study was approved by the institutional review board at the University of Michigan, complied with the Health Insurance Portability and Accountability Act, and adhered to the tenets of the Declaration of Helsinki. We recruited adult patients at the University of Michigan Kellogg Eye Center in the cornea and comprehensive clinics from May 1, 2014 to June 1, 2016. Subjects were eligible to be included in the study if they were ≥18 years old, and a chart review of the study team member indicated that the subject had at least one eye that had a corneal abrasion or a corneal opacity, which included corneal ulcers, corneal scars, and pterygia. The final diagnosis was determined by a board-certified ophthalmologist’s clinical examination. The ophthalmologist determined the presence or absence of corneal disease, the clinical severity of the corneal disease (none, mild, moderate, severe), and the location of any corneal lesion (central, paracentral, peripheral). If the patient had a resolved cornea lesion, these patients were still photographed and included in the analysis. Written informed consent was obtained from all study subjects. Both eyes of a subject were included in the study when eligible. Eye-specific exclusion criteria were eyes that did not meet the inclusion criteria, had other diseases of the anterior segment (i.e., corneal degeneration or dystrophy, surface tumor, episcleritis, scleritis, corneal foreign body, severe dry eye syndrome, a current tarsorrhaphy, or corneal glue), or a history of corneal surgeries (including pterygia removal).

Photographic Series

At the time of enrollment, an ophthalmic photographer took a series of photographs of the cornea(s) with two cameras for investigational purposes (not FDA approved for this purpose) - the iTouch 5G (Apple, Cupertino, CA) and the Nidek VersaCam (Nidek, Fremont, CA). The iTouch is a portable technology that is equipped with a camera (1136-by-640-pixel resolution; 5.0 megapixels; 3.10 ounces) and uses the same software package as other smartphone technologies. The Nidek VersaCam is a portable ophthalmic camera capable of taking anterior eye segment and posterior eye segment photographs with separate attachments to its base. The Nidek has a 1,920 × 1,080 pixel resolution, 5.0 megapixels, and weighs 8.96 ounces. For the iTouch photographs, the camera timer was set to take the photographs after the photographer ensured the eyelids were held away from the cornea and adjusted the lighting. All subjects with painful corneal diseases received topical anesthesia during photography. All photographs were taken under ambient room lighting using the built-in light from the iTouch or the Nidek and supplemental lighting as needed. The photographer took seven images of each eye: straight gaze, right gaze, left gaze, up gaze, down gaze, with eyelids closed, and straight gaze with cobalt blue light after instillation of topical fluorescein dye using a sterile fluorescein strip moistened with balanced salt solution. At the start of enrollment, the study coordinator recruited 40 patients, and the ophthalmic photographer determined optimal lighting and created a reference sheet for the image quality. The eyes of these patients were not included in the final analysis. Subjects were not excluded from the study if photographs were of poor image quality.

Image Interpretation

Three corneal specialists (i.e., “graders”; CH, JG, MW) graded each photograph for corneal disease and quality of image. All photographs were de-identified to mask the subject’s identification and the particular camera used for the image (iTouch or Nidek). Graders received image sets from one eye taken with one camera as a photograph series and interpreted the image sets of the entire sample in a randomized order. All graders were masked to the grading of other graders. Graders determined the presence or absence of corneal disease for each eye, with an additional category of “suspicious for pathology present.” If disease was present, the grader recorded both severity of disease (mild, moderate, severe) and the location of disease (central, paracentral, peripheral). Graders rated photograph quality from 1 (lowest) to 9 (highest) using the reference sheet of sample images. Lastly, the graders reported if they required all photographs in the series to make the diagnosis or if they could make their determination from only two photographs - the straight gaze and the cobalt blue light images.

Statistical Analysis

Descriptive statistics of the sample were summarized with means and standard deviations (SD) for continuous measures and frequencies and percentages for categorical variables. The sensitivity and specificity to detect corneal pathology from photographs compared to gold standard diagnosis was calculated for each grader and stratified by camera modality (iTouch or Nidek). A grader diagnosis of “suspicious for pathology” was treated as presence of pathology because, in a telemedicine setting, these patients would be referred for evaluation. 95% confidence intervals (CI) for sensitivity and specificity were calculated using logistic regression with the generalized estimating equations (GEE) method to obtain robust standard errors in the presence of clustered data (two eyes per subject). In addition, sensitivity and specificity to detect corneal pathology was calculated when stratified by gold standard diagnosis category (corneal scar, ulcer, abrasion, or pterygium) or when stratified by image quality, but within a grader and camera modality. Within diagnosis categories where no or few clusters existed, Wilson 95% CIs were provided. Equality of sensitivities between the two cameras was evaluated with McNemar’s test. Equality of sensitivities between gold standard diagnosis categories, but within a grader and camera modality, was tested with logistic regression using the GEE method.

Inter-grader agreement on diagnosis was characterized with kappa statistics, and stratified by camera modality. The percentage of eyes with a three-grader consensus for presence of pathology, by each camera and by each disease category, was calculated.

Image quality based on the average quality of all 7 photos in a series was calculated and stratified by camera modality and grader and displayed with boxplots. Tests of image quality between cameras were performed with linear mixed regression modeling. These models accounted for the correlation between eyes of a subject and correlation between photos of the same eye by different cameras. Similar models were used to test for differences in image quality in different diagnosis categories. Descriptive statistics on the added value of interpreting photographs other than those taken in straight gaze are also reported.

Results

A total of 198 eyes of 110 subjects were analyzed (88 subjects contributed both eyes to the study, 22 subjects contributed a single eye). Subjects were on average 54.3±18.9 years old (mean ± SD) (range 18–87) and 62% (68/110) were female. The racial composition of the sample was 85% Caucasian, 8% Black, 5% Asian, and 1% Hispanic. The mean (SD) logMAR visual acuity was 0.49 (0.81), a Snellen equivalent mean of 20/63) with range −0.12–+3.00 (Snellen equivalent 20/16 to hand motion). By gold standard diagnosis, 59 eyes (30%) of 47 subjects had corneal scars, 34 eyes (17%) of 33 subjects had corneal ulcers, 13 eyes (7%) of 13 subjects had corneal abrasions, 10 eyes (5%) of 9 participants had pterygia, and 82 eyes (41%) of 76 participants had no corneal disease. For some participants, each eye was in a unique category of diagnoses (e.g., right eye with a corneal scar and left eye with a corneal ulcer), so participants could contribute to multiple categories.

Accuracy and Reliability

The sensitivity to detect corneal pathology, ranged from 54–71% for the iTouch and 66–75% for the Nidek, across graders. The specificity to detect corneal pathology ranged from 82–96% for the iTouch and 91–98% for the Nidek, across graders (Table 1). The sensitivity and specificity to detect corneal pathology stratified by gold standard disease categories are shown in Table 2. The sensitivity to detect corneal abrasions (n=13) ranged from 69–77% for iTouch and 69–92% for Nidek. Similarly, sensitivity to detect corneal scars (n=59), corneal ulcers (n=34), and pterygia (n=10) ranged from 29–54%, 82–91%, and 90–100%, respectively, for the iTouch; 42–58%, 88–94%, and 90–100%, respectively, for the Nidek. For both cameras and all graders, each grader had significantly greater sensitivity to detect corneal ulcers than that to detect corneal scars for each of the cameras (p values <0.003, data not shown).

Table 1.

Sensitivity and specificity to detect corneal pathology from iTouch and Nidek photo series compared to gold standard diagnosis

Diagnosis	No Pathology*	Pathology*	Sensitivity (CI) (%)	p-value**	Specificity (CI) (%)	p-value**
Grader 1, iTouch
No Pathology	67	34	70.7 (62.8, 79.4)	1.00	81.7 (70.9, 88.3)	0.04
Pathology	15	82
Grader 1, Nidek
No Pathology	77	35	69.8 (62.7, 79.5)		93.9 (86.0, 97.4)
Pathology	5	81
Grader 2, iTouch
No Pathology	79	53	54.3 (45.5, 63.9)	0.007	96.3 (89.0, 98.8)	1.00
Pathology	3	63
Grader 2, Nidek
No Pathology	80	40	65.5 (56.6, 73.8)		97.6 (90.8, 99.4)a
Pathology	2	76
Grader 3, iTouch
No Pathology	76	39	66.4 (57.7, 75.0)	0.04	92.7 (85.1, 97.2)	1.00
Pathology	6	77
Grader 3, Nidek
No Pathology	75	29	75.0 (68.2, 83.9)		91.5 (82.4, 95.5)
Pathology	7	87

CI = 95% Confidence Interval

^*Diagnosis by board-certified ophthalmologist’s clinical examination

^**McNemar’s test for comparing sensitivity, or specificity, between iTouch and Nidek photo series, within a Grader

Note: 95% confidence intervals are adjusted for the correlation between eyes of a subject by use of a logistic regression model with generalized estimating equations;

^aWald confidence interval reported due to convergence issues with the model

Table 2.

Sensitivity to detect corneal pathology from images, stratified by diagnosis, grader, and camera

Grader/Camera	Corneal Abrasion (n=13) Sensitivity (CIa)	Corneal Scar (n=59) Sensitivity (CIb)	Corneal Ulcer (n=34) Sensitivity (CIa)	Pterygium (n=10) Sensitivity (CIa)
Grader 1
iTouch	76.9 (49.7, 91.8)	54.2 (41.7, 67.4)	91.2 (77.0, 97.0)	90.0 (59.6, 98.2)
Nidek	92.3 (66.7, 98.6)	50.9 (38.7, 65.0)	88.2 (73.4, 95.3)	90.0 (59.6, 98.2)
Grader 2
iTouch	69.2 (42.4, 87.3)	28.8 (19.0, 39.7)	82.4 (66.5, 91.7)	90.0 (59.6, 98.2)
Nidek	69.2 (42.4, 87.3)	42.4 (30.6, 55.1)	94.1 (80.9, 98.4)	100.0 (72.3, 100.0)
Grader 3
iTouch	76.9 (49.7, 91.8)	44.1 (31.6, 57.1)	91.2 (77.0, 97.0)	100.0 (72.3, 100.0)
Nidek	84.6 (57.8, 95.7)	57.6 (45.0, 71.0)	94.1 (80.9, 98.4)	100.0 (72.3, 100.0)

CI: 95% confidence interval, includes

^aWilson CIs when diagnosis category contains no subjects contributing both eyes or only 1 subject contributing both eyes, and

^bCIs calculated from a logistic regression with generalized estimating equations adjustment to account for the correlation between eyes of a subject

The inter-grader reliability for presence of pathology between pairs of graders was moderate to strong with kappa values ranging from 0.54–0.71 for the iTouch and 0.75–0.76 for the Nidek (Table 3). All three graders had consensus about the presence of pathology in 49% of eyes that had any pathology (n=116) when determined from iTouch photographs, and 59% when determined from Nidek photographs.

Table 3.

Inter-rater reliability for the diagnosis of corneal pathology from iTouch and Nidek photo series

	Second Grader Diagnosis
First Grader Diagnosis	Normal	Pathology	Kappa (CI)
iTouch
Grader 1 vs. Grader 2
Normal	94	7	0.54 (0.43, 0.65)
Pathology	38	59
Grader 1 vs. Grader 3
Normal	90	11	0.63 (0.53, 0.74)
Pathology	25	72
Grader 2 vs. Grader 3
Normal	110	22	0.71 (0.61, 0.81)
Pathology	5	61
Nidek
Grader 1 vs. Grader 2
Normal	104	8	0.75 (0.66, 0.84)
Pathology	16	70
Grader 1 vs. Grader 3
Normal	96	16	0.76 (0.66, 0.85)
Pathology	8	78
Grader 2 vs. Grader 3
Normal	100	20	0.76 (0.66, 0.85)
Pathology	4	74

CI = 95% Confidence Interval

Clinical Impressions

In the 198 eyes, the overall image quality of a series (mean of all 7 photographs) for iTouch and Nidek photographs was 5.3±1.2 and 5.7±1.1 for grader 1, 2.6±1.0 and 2.8±0.9 for grader 2, and 4.7±1.0 and 4.7±0.8 for grader 3, respectively (Figure 1). Two of the three graders rated images from the iTouch as having significant lower quality than those from the Nidek (both p<0.004). Sensitivity to detect corneal pathology did not improve with better rating of the photographic quality (data not shown). Differences in image quality were not consistent between diagnoses, camera type, or graders (data not shown).

Figure 1.

Boxplots displaying mean image quality of a photographic series for 198 eyes, stratified by grader. Higher ratings indicate better photo quality. Boxplot displays: the interquartile range (IQR), including the lower quartile/25th percentile (bottom of the box) and the upper quartile/75th percentile (top of the box), the median/50th percentile (line within the box), the mean (dot within the box), the lower fence/whisker (smallest observation within 1.5*IQR), the upper fence/whisker (largest observation within 1.5*IQR), and outliers (stars; observations located outside 1.5*IQR). P-values are from a linear mixed regression model of image quality, testing the effect of camera.

Graders 1 and 3 reported that for the majority of eyes additional images (5 photographs in addition to the 2 straight gaze photographs) were helpful in detecting the presence/absence of pathology (grader 1: 73% of eyes from Nidek, 74% from iTouch; grader 3: 71% of eyes from Nidek, 70% from iTouch). Grader 2 reported that additional photographs were helpful for 15.7% of eyes from Nidek images and 10.1% of eyes from iTouch images.

Discussion

This study assessed the accuracy and reliability of diagnosing corneal pathology by three cornea-trained ophthalmologists based on images obtained from portable cameras. A standard for the accuracy of telemedicine screening has been suggested to be 80% sensitivity for diabetic retinopathy and potentially other ophthalmic diseases.³⁰ Accuracy of remote diagnosis of corneal pathology for this study did not achieve these standards, implying that portable camera technology requires further development for corneal disease telemedicine applications.

Our findings indicate that interpretation of images from portable cameras had high specificity, but lower sensitivity. The ability to screen for pathology accurately is necessary when relying on imaging for remote evaluations. Our results bring caution to a common clinical practice. Many clinicians and patients use portable cameras, especially smartphone cameras, and send ophthalmologists images for interpretation.³¹ In our 2015 survey of eye providers and telemedicine, 56% percent of eye care providers said that they had received a photograph from a patient in the past 3 months for the purpose of evaluating the eye.³¹ The results of our study suggest that interpreting photographs from a diffuse light image will miss corneal pathology in some cases (eFigure 1). It is important to note that we did include a photograph with cobalt blue light and fluorescein dye, such as may be taken in a medical setting (i.e., an emergency department), and so our accuracy findings may be better than interpretations that rely on photographs taken without dye application. It is moderately reassuring to note that our study found that corneal ulcers, which would need more urgent follow-up, were detected with adequate sensitivity (range: 82.4–94.1%).

Graders had good agreement on the presence of corneal disease in photographs. Moderate to strong reliability scores in grader interpretations is consistent with previous ophthalmic telemedicine studies.³² One disadvantage to our study design was the lack of a standardized protocol from prior studies to use for image interpretation. For diabetic retinopathy, guidelines and protocols for photograph imaging and interpretation existed from clinical trial data, such as the Early Treatment Diabetic Retinopathy Study.³³ This standardization provided a framework for early telemedicine work in this area.³⁴ Clinical trials for corneal diseases have included specular imaging of the corneal endothelium^35,36 but have relied on clinical examination, not standardized photography. Some trials using photographs include a dry eye study photographing superficial punctate keratopathy³⁷ and the Collaborative Longitudinal Evaluation of Keratoconus study photographic corneal scarring in keratoconus.³⁸ We created a protocol based on the American Telemedicine Association guidelines² and other clinical trial protocols.^37–39 An opportunity exists to create standardized photography protocols for corneal diseases to serve as an image-based gold-standard. This standardization could open new avenues for image-based research in corneal disease, as has been done in numerous retina clinical trials.

Limited publications exist on anterior segment telemedicine imaging. We conducted searches in PubMed and EMBASE, using the following three groups of MESH terms, and additional similar and synonymous title and abstract words: (1) Anterior Eye Segment OR Cornea OR Eye Manifestations OR Eye Diseases OR Vision, Ocular OR Ophthalmology OR Retina OR Vision Disorders/diagnosis (2) Photography OR Cellular Phone OR Diagnosis, Computer-Assisted (3)Telemedicine OR Telepathology OR Remote Consultation OR Mobile Health Units OR Population Surveillance. The first two groups were also searched together in a number of telemedicine journals. The final search was done on 1/24/17, and only articles written in English were included. We identified 16 publications related to remote imaging for anterior segment diseases reported in the literature.^39–54 The standard slit-lamp biomicroscope is the gold standard for corneal examination. Kumar and colleagues created a portable digital slit-beam device and, compared to a clinical examination, found that gross corneal signs were detected with modest to excellent sensitivity (67–100%), but subtle corneal signs, such as epitheliopathy, were not detected at all (sensitivity 0%).⁴⁰ Other research has focused on telemedicine using the standing slit-lamp photography, but this method lacks feasibility to implement in non-ophthalmic settings because of personnel and equipment requirements.^{38–40,42,45,47–49,53} Some research did not include sensitivity analyses^{43,46,51,53,54} and some research describes implementation of technologies rather than validation.^{43,46,48,52,53} Portable anterior segment photography with mobile phone camera devices has been evaluated.^41,44 Barsam and colleagues provided a descriptive technique of anterior segment slit lamp photography with the iPhone.⁴⁵ Bhosai and colleagues found moderate reliability, but not always high accuracy, when using iPhone technology when compared to SLR photos for detecting clinically active trachoma.⁴² The results of the published work indicate that a hardware and software solution is still needed to photographically capture anterior segment pathology.

We chose to use commercially available camera technology in order to provide results that pertain to available devices that can be implemented. We identified four such cameras based on a review of the literature and on-site demonstration of the cameras. A camera attached to a phone (iTouch using the same hardware and software as iPhone devices) was chosen because of high availability and prevalent use of smartphone cameras. However, smartphone cameras are designed to capture faces and landscapes, not small objects. We selected the Nidek VersaCam device because the anterior segment attachment had diffuse and cobalt blue light settings and did not have direct light reflections that would make corneal disease detection difficult. We attempted to use a magnification lens, as has been published,⁵⁵ for the iTouch, but found that it decreased ambient light to the camera and thus affected image quality. To maximize image quality, a certified ophthalmic photographer took all photographs and practiced using both cameras prior to study enrollment. We chose to employ an ophthalmic photographer for maximizing our image quality for the purpose of this study; however, this would not be realistic in a non-ophthalmic setting.

Despite good agreement on pathology between graders, graders’ ratings of image quality were highly variable. One grader, in particular, rated most images low quality for both cameras. On further discussion (after data analysis), this grader reported that the camera images did not adequately compare to slit-lamp photographs or a clinical examination (despite the grader instructions to use the reference sheet). This finding has implications on the design of further studies, as conduct of a pilot period of grading may have enabled identifying and addressing such differences in grader expectations. Despite the lower quality ratings, this grader’s accuracy in detecting pathology was comparable to the other graders. The disconnect of accuracy and quality scores may be part of why National Health Services guidelines (United Kingdom) for diabetic retinopathy telemedicine rate image quality as adequate or inadequate rather than rating quality on a Likert scale.⁵⁶

The results of our study indicate that using portable cameras for corneal telemedicine in primary care or remote settings is likely premature. However, other opportunities to implement portable cameras for anterior eye diseases may exist. In our study, the sensitivity and specificity for smartphones to detect corneal ulcers and pterygia achieved the standards for ophthalmology telemedicine, albeit with wide confidence intervals. There may be a role for using portable cameras to monitor corneal ulcers or pterygia with portable imaging. While we did not evaluate this use, eye care providers could in the future use images to track disease progression between clinical appointments. Image-based monitoring has been a highly successful approach for retinopathy of prematurity between clinical encounters.⁵⁷ Image-based monitoring could be enhanced with quantifiable methods, such as signal processing, feature extraction, or artificial intelligence.⁵⁸

Although the accuracy of interpretation was unable to meet telemedicine ophthalmic standards, studies like ours should spur innovation in image capture to overcome current limitations. Possible innovations include better magnification while maintaining resolution, addition of a parallel pipette (slit-beam) feature, table-mounted stabilization, wireless or voice-activated camera prompts, and the ability to fine-focus (as auto-focus often focuses on the iris, not cornea). The goal of screening modalities should remain to serve patients with limited access to eye care with equal accuracy to current screening methods and at an affordable cost.

Supplementary Material

Supplemental Data File _.doc_ .tif_ pdf_ etc._

eFigure 1: Two examples of corneal ulcers taken under diffuse light images. In Case A, all of the ophthalmologist graders of the images considered the photograph normal, likely assuming that the mid-peripheral corneal lesion was the light reflex and the lack of conjunctival injection indicated no disease. In Case B, two of the ophthalmologist graders considered the photograph to be suspicious of corneal disease, likely given the conjunctival injection although no corneal lesion was visualized in the photographs.