Diagnostic accuracy of smartphone corneal photography for detection of corneal opacities in a resource-limited setting: a community-based study

Abstract

Purpose:

To evaluate the diagnostic accuracy of smartphone corneal photography in detecting corneal opacities in a community-based setting.

Methods:

A case-control, diagnostic accuracy study was nested in a cluster-randomized trial of a corneal ulcer prevention intervention in Nepal. Smartphone corneal photography was performed annually on community members self-reporting a potential risk factor for a corneal infection. Corneal photographs were graded for the presence or absence of an opacity. All cases with an opacity on smartphone photography and an equal number of controls were invited for a comprehensive eye examination with a slit lamp biomicroscope at an eye hospital. A mobile team visited participants unable to come to the hospital, conducting a limited examination with a penlight.

Results:

Of 1332 study participants (666 cases and 666 controls), 1097 had a penlight examination (535 cases and 562 controls) and 191 had a slit lamp examination (120 cases and 71 controls). When penlight examination was considered the reference standard, smartphone diagnosis of a corneal opacity had a positive predictive value (PPV) of 47% (95% confidence interval 43–52%) and negative predictive value (NPV) of 95% (93–97%). When slit lamp examination was considered the reference standard, the overall PPV and NPV were 71% (62–78%) and 80% (70–88%), respectively. The NPV was greater for detection of opacities > 1mm, estimated at 95% (90–98%).

Conclusions:

Corneal photography performed in a resource-limited community-based setting using a smartphone coupled to an external attachment had acceptable diagnostic accuracy for detection of corneal opacities large enough to be clinically meaningful.

Introduction

Corneal photography holds promise as a tool for assessing corneal ulcer and corneal opacity in communities at increased risk for eye infections. Previous research has suggested that smartphone cameras may have adequate diagnostic accuracy if paired with an external attachment to provide improved magnification and illumination.^1,2 However, previous studies have been limited by a relatively low number of opacities and the potential for selection bias since all participants were enrolled at an eye clinic. Moreover, photographs in previous studies were taken by ophthalmologists, but in reality, smartphones would probably be used as a diagnostic tool by non-ophthalmic personnel.

The Village-Integrated Eye Worker (VIEW) trial was a cluster-randomized trial in which staff without any health sciences background were trained to use a smartphone to image the cornea during a door-to-door census. A subset of participants subsequently received a comprehensive eye examination, in part to confirm the diagnoses provided by the photographs. The objective of the present analysis was to assess the diagnostic accuracy of a smartphone coupled to an external attachment for detecting corneal opacity when used by non-specialists in a resource-limited community-based setting.

Materials and Methods

General study design

The design and methods of the VIEW trial have been reported in detail elsewhere.³ Briefly, VIEW was a cluster-randomized trial in which 24 village development committees (VDCs) surrounding the city of Bharatpur in low-lying Nepal were randomized to a community health worker-based corneal ulcer prevention intervention or to no intervention. Corneal photographs were taken each year by trained non-ophthalmic personnel during a door-to-door census using a smartphone coupled to an external attachment, and were subsequently graded for corneal opacity by remote photo-graders. Study participants judged to have an incident corneal opacity in either eye based on the photo-grading were contacted by telephone or in person up to 3 times to schedule a comprehensive eye examination approximately 12 months following their reported corneal trauma or infection. Similarly, a random sample of age- (± 2 years), sex-, and community-matched controls was invited for a comprehensive eye examination in the same time window. The timing of the visit at 12 months was pre-specified in the trial protocol, and chosen because of the relatively lengthy photo-grading process as well as the desire to perform a clinical assessment at approximately the same follow-up time for each participant (which would have been difficult to implement with a shorter follow-up period since corneal ulcers happen throughout the year but the census only once per year).³ Visits were preferably conducted at Bharatpur Eye Hospital or an affiliated primary eye care center; if participants could not attend one of these facilities, then a mobile team visited the participant at their home. The eye examination was performed for each eye by an optometrist or ophthalmic assistant to determine the presence, location, and size of corneal opacity. Examiners were masked to case-control status of the participant. In the present analysis, the results of the smartphone photo-grades are compared with the results of the eye examination to provide estimates of the diagnostic accuracy of smartphone photo-grading for corneal opacities when done in a community-based setting.

Ethical approval for the trial was obtained from the University of California San Francisco Committee on Human Research, Nepal Netra Jyoti Sangh, and the Nepal Health Research Council. The trial adhered to the tenets of the Declaration of Helsinki.

Corneal photography

All corneal photographs were taken at the door-to-door census with a Nexus 5 smartphone (LG, Seoul, South Korea) coupled to a Cornea CellScope. The Cornea CellScope is a custom-made smartphone attachment consisting of a 3D-printed housing, a +25D lens, and 2 light emitting diode (LED) light sources; the attachment provides magnification and illumination and ensures the proper working distance for corneal photography (Development Impact Lab, Berkeley, California, USA).^1,2

Census workers were trained to take photographs before each census, including classroom training and practice. Training emphasized taking photographs in which the cornea was in focus, centered, perpendicular to the camera’s optical axis, sized appropriately, and free from flash artifacts and reflections. Images were reviewed by experienced personnel at training sessions and field visits throughout the study, and photographers were given feedback about how to improve their photographs. Census workers took two or three photographs per eye to increase the chances of obtaining a high-quality image.

Photo-grading

All photographs per eye were graded together as a set after each census visit, and a single grade was assigned to each eye per census. Photographs were initially screened for corneal opacities by 2 independent trained photo-graders, and any image sets graded as possible, probable, or definite opacity by either grader were subsequently graded by an ophthalmologist to determine the presence of an opacity. A panel of 3 study ophthalmologists eventually performed a review of the photographs before the final classification. For this study, individuals were designated as cases if the initial ophthalmologist review classified the eye as having an incident opacity in either eye (i.e., a photograph from one census visit classified as a probable or definite opacity and a photograph from a previous census visit graded as no or probably no opacity). A subset of images was assessed for the likely cause of opacity, categorized as due to a bacterial or fungal infection (i.e., typically characterized by a single scar with rounded borders—or a main scar with satellite lesions, Figure 1A), nummular keratitis (i.e., typically characterized by small multifocal scars with rounded borders distributed across the cornea, Figure 1B), or other patterns. The size of the opacity on photography was estimated using the measurement tools in ImageJ (version 2.9.0).⁴ For each eye, the highest-quality photograph (i.e., in focus, centered, perpendicular to the camera’s optical axis, sized appropriately, and minimal flash artifacts and reflections) was chosen by a co-author (B.C.) for the measurements. The total cornea area was estimated in ImageJ by drawing a circle that covered the entire cornea, and the area of the opacity (or opacities) was determined by tracing the opacity border using the oval or freeform drawing tools in ImageJ. The opacity size, which was expressed as a geometric mean, was estimated by dividing the opacity area by the total corneal area, taking the square root, and multiplying by 11 (i.e., under the assumption that the geometric mean of an average cornea was 11 mm). Opacity size was subsequently classified as tiny (<0.5 mm), small (0.5–3 mm), medium (3–6 mm), and large (≥6 mm) (Figure 2).

Figure 1.

Representative images taken with a smartphone coupled to an external attachment showing opacity due to (A) infection and (B) nummular keratitis.

Figure 2.

Representative images taken with a smartphone coupled to an external attachment showing (A) tiny, (B) small, (C) medium, and (D) large opacities.

Clinical examination

The eye examination was performed with a slit lamp biomicroscope in the eye clinics, and with a penlight at the mobile examinations. The size and location of corneal opacities were noted during the penlight and slit lamp examinations. Scar size was estimated on penlight examination as small (<3 mm in diameter), medium (3–6 mm) or large (>6 mm), and on slit lamp examination as the geometric mean of the longest diameter and its longest perpendicular diameter. Location was classified as any involvement of the central 4 mm, versus opacification only peripherally.

Statistical analysis

Positive predictive value (PPV) and negative predictive value (NPV) for the smartphone-based corneal photographs were determined at the participant level and eye level relative to the penlight or slit lamp examination reference standard. Sensitivity analyses were conducted based on opacity size, type, and location. The Wilson method was used to calculate the 95% confidence interval around estimates of predictive values. P-values less than 0.05 were considered statistically significant in this exploratory study. All analyses were performed with R version 4. For eye-level analyses we computed bootstrapped 95% confidence intervals (CIs), with resampling of participants (N = 1000 replications) to account for non-independence of two eyes from the same person.

Results

Figure 3 shows the flow of participants. A total of 666 study participants with a corneal opacity on photo-grading were referred for a comprehensive eye exam, as were 666 randomly selected controls. Of those referred, 535 (80%) participants classified as cases and 562 (84%) classified as controls participated in an eye exam. Most participants could not make the trip to the eye hospital, and thus 906 participants (i.e., 415 cases and 491 controls) had a penlight examination only, and 191 (i.e., 120 cases and 71 controls) had both a penlight and slit lamp examination. Of participants completing a penlight examination, the mean age was 41.7 ± 18.6 years in the case group and 41.9 ± 19.0 years in the control group, and the female fraction was 60% in the case group and 61% in the control group (Table 1).

Figure 3.

Study flow.

Table 1. Demographic characteristics of participants completing an eye examination.

Age and sex are shown for the subpopulation of participants who had a penlight examination only, and the subpopulation of participants who had both a penlight examination and slit lamp examination.

	Cases			Controls
Characteristic	Penlight only N=415	Slit lamp N=120	p-value	Penlight only N=491	Slit lamp N=71	p-value
Age, mean ± SD	42 ± 20	42 ± 15	0.99	42 ± 19	44 ± 16	0.06
Female, N (%)	261 (63)	61 (51)	0.001	313 (64)	32 (45)	<0.001

A corneal opacity was diagnosed on penlight examination in 254 of the 535 participants with an opacity detected by photography (PPV 47%, 95%CI 43–52%) and for 26 of the 562 controls (NPV 95%, 95%CI 93–97%). Penlight examination was less sensitive than slit lamp examination for detection of corneal opacities but just as specific: of the 129 eyes with a slit-lamp-detected opacity only 90 had an opacity detected by penlight examination (sensitivity 70%, 95%CI 61–78%), but all 253 eyes with no opacity detected on slit lamp were also classified as opacity-negative on penlight examination (specificity 100%, 95%CI 99–100%). Of the 120 participants with a photographic opacity who completed a slit lamp examination, 85 (PPV 71%, 95%CI 62–78%) had an opacity detected on slit lamp, compared with 14 of 71 controls (NPV 80%, 95%CI 70–88%). Diagnostic accuracy estimates were similar when using eye-level data instead of person-level data (Table 2).

Table 2. Predictive value of corneal photographs using eye-level data of cases.

Data are shown for the 1070 eyes of cases that had a penlight examination, and from the 240 eyes of cases that had a slit lamp examination. Controls are not included because not all the controls had photo-grades available.

	Penlight examination			Slit lamp examination
Photo-grade	Opacity	No opacity	Predictive value	Opacity	No opacity	Predictive value
Opacity	246	330	PPV= 43% (39–47%)	85	46	PPV= 65% (57–73%)
No opacity	47	447	NPV= 90% (88–93%)	28	81	NPV= 74% (66–82%)

PPV: positive predictive value; NPV: negative predictive value.

Of 131 eyes with an opacity detected on photography that had a slit lamp examination, photo-graders classified 84 (64%) as likely due to infection, 34 (26%) as likely due to nummular keratitis, and 13 (10%) as likely due to some other condition. The PPV did not differ stratified by likely cause (56/84 [67%] for scars due to infection, 23/34 [68%] for scars due to nummular keratitis, 6/13 [46%] for scars due to another cause; P=0.35) (Table 3).

Table 3. Photo-diagnosis of corneal opacity: slit lamp subgroups.

Eye-level data are shown for cases with a slit lamp examination, stratified by the photo-grader’s assessment of (a) the suspected cause of the scar and (b) the size of the scar on photography, and by the slit lamp examination assessment of (a) size and (b) location. For assessing the suspected cause, the typical infectious keratitis scar was considered to be a single, round lesion and the typical nummular keratitis scar was considered to be comprised of numerous small round opacities distributed across the cornea. Controls are not included because not all the controls had photo-grades available.

	Slit lamp examination
	Sizea					Location
Photo-grade	Tiny	Small	Medium	Large	None	Central	Peripheral	None
No opacity	17	10	1	0	81	13	15	81
Suspected cause
Infection	23	21	8	4	28	36	20	28
Nummular	8	12	3	0	11	13	10	11
Other	0	1	2	3	7	4	2	7
Size categoryb
Tiny	2	0	0	0	4	1	1	4
Small	25	25	5	0	31	30	25	31
Medium	4	8	5	0	9	12	5	9
Large	0	1	3	7	2	10	1	2

^aSize measured at the slit lamp as geometric mean of longest length and longest perpendicular width; categories defined as tiny (<1 mm), small (1–3 mm), medium (3–6 mm), and large (≥6 mm).

^bSize measured on photographs as the proportion of the total cornea area occupied by the scar, which was subsequently classified as tiny (<0.5 mm), small (0.5–3 mm), medium (3–6 mm), and large (≥6 mm) assuming a circular scar and 11 mm corneal diameter.

Eyes in which photography missed the diagnosis were more likely to have small opacities. For example, photography missed 47 opacities that were detected on penlight examination, of which 39 (83%) were smaller than 3 mm in diameter. In contrast, of the 246 eyes detected on both photography and penlight examination, only 125 (51%) were smaller than 3 mm (Table 4). Similarly, photography missed 28 eyes that were detected on slit lamp examination, and the mean scar size was smaller in these false negatives compared to the 85 true positives (0.75 ± 0.79 mm vs 2.17 ± 2.43 mm; P<0.001; Table 3). NPV was higher when the reference standard was considered to be detection of a larger corneal scar (Figure 4). For example, corneal photography had a NPV of 95% (95%CI 90–98%) for detection of corneal scars measuring > 1 mm in size, and 100% (95%CI 97–100%) for detection of corneal scars > 4 mm – although at the cost of reduced PPV (34% [95%CI 27–43%] for opacities > 1 mm and 8% [95%CI 5–14%] for opacities > 4 mm).

Table 4. Photo-diagnosis of corneal opacity: penlight subgroups.

Data are shown for cases only, stratified by the penlight examination’s assessment of (a) size and (b) location. Controls are not included because not all the controls had photo-grades available.

	Penlight examination
	Sizea				Location
Photo-grade	Large	Medium	Small	None	Central	Peripheral	Missing	None
Opacity	38	83	125	330	126	109	11	330
No opacity	2	6	39	447	18	23	6	447

^aSize estimated by penlight examination; categories defined as small (<3 mm), medium (3–6 mm), and large (≥6 mm).

Figure 4.

Positive and negative predictive values for photographic diagnosis of corneal opacity when reference standard was considered to be detection of corneal scars of different sizes (0, 1, 2, 3, and 4 mm).

Discussion

This study found that smartphone-based corneal photography using the Cornea CellScope had a PPV of 71% and NPV of 80% for detection of a corneal opacity when used in field conditions. The main factor affecting the predictive value of photography was the size of the opacity, with most of the false negative photo-grades coming from eyes with smaller scars. The NPV was considerably higher for detection of larger scars, with a NPV of 95% for detection of scars > 1 mm. The PPV was not affected by the likely cause of the opacity, with similar accuracy for scars due to infections and scars due to nummular keratitis. The PPV observed in this study is likely acceptable for community-based corneal opacity detection, especially given the low prevalence of disease in the community. The NPV was modest for detection of an opacity of any size, but much better for detection of opacities likely to be large enough to be clinically meaningful (i.e., opacities > 1mm in size).

Corneal photography could be used for several applications in a community-based setting. Smartphone photographs were used as the primary outcome for the VIEW trial, which is the parent study in which the present study was nested. Photography has advantages for use as an outcome measure for research since photo-graders can easily be masked to the allocated intervention, and the photographs can be audited and used for secondary analyses. Corneal photography could also easily be incorporated into existing community-based blindness prevalence surveys such as the Rapid Assessment of Avoidable Blindness (RAAB) which seek to determine the most common causes of blindness.⁵ Moreover, corneal photography may have telemedicine applications, since non-ophthalmic staff at a primary health clinic could be trained to take images for use in real-time diagnosis. While photographs require interpretation, currently by trained graders, it is likely that machine learning algorithms could be developed for this task, which could expand the potential uses of corneal photography.⁶

The present study found that smartphone-based corneal photography with a Cornea CellScope had a NPV of 71% and PPV of 80% for detecting an opacity of any size, but a NPV of 95% and PPV of 34% for detecting an opacity greater than 1 mm in size. Thus, a positive result detected most opacities, and a negative result did not miss many opacities of a size that would be likely to be clinically meaningful. The minimum acceptable PPV and NPV for a diagnostic test depend on whether the test is intended to rule in or rule out disease, and also the prevalence of disease. When used in the community for research or blindness prevalence surveys, a high NPV may be desired since a goal may be to minimize the number of missed cases (i.e., false negatives). In contrast, when used as a screening test to drive referrals to an eye hospital, a high PPV may be desired in order to minimize the number of false positive cases, which could become a burden on the referral institution. The diagnostic accuracy of smartphone corneal photography observed in this study is comparable to a previous study in which corneal photographs were taken by ophthalmologists in a hospital-based setting.² Both the present study and the previous study found that diagnostic accuracy was lower for the smallest opacities. Small scars may be challenging to detect on corneal photographs due to confusion with flash artifacts, iris markings, or reflections of the photographer’s fingernails or the participant’s nose or eyelashes. While it would be desirable for corneal photography to have greater PPV for small scars, it is likely that many of the smallest opacities are not visually significant. Given the results of this study, a smartphone coupled to a CellScope might be most useful for assessing clinically meaningful corneal opacities for research purposes or blindness prevalence surveys in a community-based setting, where the low prevalence of disease will inherently limit the PPV of any diagnostic test.

While this study intended to perform slit lamp examinations, in practice it was challenging for many of the study participants to reach the eye hospital due to financial and time constraints. The majority of examinations were thus performed with a penlight. The penlight examinations were less sensitive than slit lamp examinations, leading to underestimates of PPV. The results from the slit lamp examinations are probably more accurate than that of the penlight examination, although it should be noted that those who were able to come to the eye hospital for a slit lamp examination may be systematically different from those who only had a penlight examination.

This study has limitations. Participants were selected based on the results of the index test, and so we focused on the positive and negative predictive value of the index tests as opposed to the sensitivity and specificity of the tests. There was a lapse of time between the index test and the reference tests, which could have caused misclassification error since participants without an opacity at the time of photography may have subsequently developed a corneal scar by the time of the eye examination, and likewise some opacities present on photography—especially those due to nummular keratitis—may have resolved by the time of the examination.^7,8 However, we did not observe any differences in PPV among the opacities thought due to nummular keratitis. As mentioned above, a minority of participants presented to the eye hospital for a slit lamp examination, potentially introducing bias if the opacity characteristics were different among those who had a slit lamp examination. Finally, the study was performed in a lower middle-income country of South Asia with a specific smartphone and a custom-made CellScope attachment that is not widely available. The results may not be generalizable to other study populations (e.g., more developed settings with less prevalent or less severe corneal opacities) or to surveys using a different smartphone or smartphone attachment.

In summary, the CellScope-smartphone device had acceptable diagnostic accuracy for detection of corneal scars large enough to be clinically meaningful when used in a resource-limited setting. Corneal smartphone photography is well-suited for community-based studies given its portability, low cost, and ease of use, and may have telemedicine applications in resource-limited settings where ophthalmologists and slit lamp biomicroscopes are not readily available.