Comparing image quality and trachoma detection across three camera types from a survey in Kongwa, Tanzania

Abstract

Background

There is an increasing demand for photography for trachoma prevalence surveys. In previous studies, digital single lens reflex (DSLR) images were superior to smartphone images, but newer-model smartphones and/or lens attachments may be able to bridge this gap. This study compares the image quality and ability to detect trachomatous inflammation – follicular (TF) of three camera types: a DSLR Nikon camera, an iPhone SE and an iPhone 13 Pro with a cell scope.

Methods

We surveyed 62 children ages 1–7 y from two Tanzanian communities. Upper tarsal conjunctiva images of both eyes were graded for TF by two standardized graders. The McNemar's test and a logistic regression model were used for analyses.

Results

The DSLR camera malfunctioned during the study, thus the iPhone SE and iPhone 13 Pro with cell scope were both more likely to take high-quality, gradable photographs (88% and 86%, respectively) compared with the DSLR camera (69%) (p<0.001 and p=0.02, respectively). TF was detected in gradable images from the iPhone SE (8.8%) and iPhone 13 Pro with cell scope (9.0%) at the same rate (p=1.0) as images from the DSLR camera (9.7%).

Conclusion

Smartphones with high-quality image capture, like the iPhone SE/13 Pro, have the potential for use in trachoma surveys if the proportion of gradable images can be improved.

Introduction

Trachoma is an eye disease caused by repeated episodes with the bacteria Chlamydia trachomatis and is the leading infectious cause of blindness. The inflammatory stage of trachoma (trachomatous inflammation – follicular [TF]) can be widespread among preschool-age children in trachoma-endemic locations, with prevalence rates as high as 60–90%.¹ Trachoma is declining worldwide, as country programs strive to eliminate trachoma as a public health problem. The World Health Organization (WHO) will validate the claim of a country for elimination of trachoma as a public health problem if the following criteria are met: a prevalence of trachomatous trichiasis ‘unknown to the health system’ of <0.2% in adults >15 y of age, the presence of a system capable of identifying and managing incident trachomatous trichiasis cases as well as proof of enough financial resources to put those procedures in place and a prevalence of TF in children ages 1–9 y of <5% sustained for at least 2 y without ongoing antibiotic mass treatment.¹ The latter is documented in prevalence surveys involving field graders who observe the everted eyelids of random samples of children.²

As the prevalence decreases in many districts and the difficulties and cost of ensuring consistency of graders become increasingly complicated, there is a growing need for an alternative to field graders.³ Images of everted lids taken using digital single-lens reflex (DSLR) cameras and graded by experts is one option that has long been employed in research projects.³ Intergrader agreement on photographs has been demonstrated to be as good as agreement in the field between field graders.³ DSLR cameras are likely not practical for field use due to their high cost, so there are studies looking at other alternatives. However, previous literature suggests that DSLR cameras take more gradable images than earlier smartphones.⁴ Interestingly, the estimated prevalence of TF in children from the field and from the gradable images was similar in one study, despite the high number of ungradable images.⁵ Newer smartphones have much higher image quality and the cell scope attachment (a device that allows stabilization of the smartphone on the child's head, magnification and a set distance for focusing)⁵ may provide improved performance in trachoma surveys.

The aim of our comparative study was to investigate the image quality and ability to capture TF across three camera types: DSLR, iPhone SE and iPhone 13 Pro with a cell scope.

Methods

The National Institute for Medical Research in Tanzania and the Institutional Review Board of the Johns Hopkins School of Medicine approved the study protocols and methods. Guardians of the study participants provided informed written consent.

Population

We surveyed 63 children, ages 1–7 y, from two of the five communities in Kongwa, Tanzania, randomly selected in the context of a larger survey on methods of face washing carried out from August to December 2022. Both eyes of each child were eligible. Data were collected on age and gender. Images of each upper eyelid of each eye were taken using the three camera systems as described below. Field grades for TF were captured at the level of the child and not eye specific.

Camera systems

Three camera types were used in this study to take photographs of the eyelids. The first camera was a handheld Nikon D40 DSLR camera using a manual setting, with a 105 mm f/2.8D Auto Focus Micro Nikkor lens. This system has been used in numerous research projects in the past. During the survey the DSLR malfunctioned, resulting in a shallower depth of field, which was not detected until the images were reviewed. The smartphones were selected on the basis of availability and successful previous pilot field use during training for everted eyelids. The second camera was a standard iPhone SE (third generation, 2022). The third camera was an iPhone 13 Pro (2022) coupled to a stabilization device with a magnifying attachment (cell scope).⁶ It was noted that once the cell scope was attached it could not be easily removed without possible damage, so we do not have a comparison of the iPhone 13 Pro without the use of the cell scope. Four camera types were initially planned to be used in the study; however, during the pilot and training phase the Samsung Galaxy A53 camera was seldom able to take quality images of the upper tarsal conjunctiva and was excluded prior to the current study.

Photographer training

Three trainees underwent a standard photography training course, modelled on a manual for training photographers for trachoma prevalence surveys that will be published as a preferred practice manual by the International Coalition for Trachoma Control. The course was designed by SW and HM for smartphone use to train photographers. The three trainees used either the iPhone SE or the iPhone 13 Pro. Two passed the final exam, successfully taking gradable images of the upper lid in five successive children <7 y of age. One of the two photographers was given the iPhone SE to use and the other was given the iPhone 13 Pro with the cell scope. The fourth photographer for the study was already experienced using the DSLR for research purposes and continued to use that system; he was not part of the training.

Photograph grading

The photographs were graded for the presence or absence of TF by two trachoma photo graders at Johns Hopkins (UA and SW). A simplified system for grading image quality was used prior to grading the image.⁷ The two graders used the WHO simplified grading scheme to assess all images for TF.⁸ Agreement between the two graders prior to undertaking formal grading, based on a test set of 100 images, was κ=0.87. If there was disagreement between the grades for an image, the final grade of the image was determined by open adjudication of the two graders with each other.

Data analysis

Contingency tables were used to compare the proportion of images with acceptable photograph quality and TF among all three camera types. The McNemar's test was performed to test the significance of associations between the systems for photograph quality and the proportion with TF. We tested the hypothesis that age, gender and eye laterality were associated with ungradable images in each of the camera systems, using logistic regression. All analyses were performed using SAS software version 9.4 (SAS Institute, Cary, NC, USA).

Results

A total of 63 children were surveyed, but 1 child refused to have images taken after one camera system was used, so was not included. The remaining 62 children were ages 1–7 y (Table 1) and contributed 122 eyes (two children had unilateral images and refused the second eye). For nine children, both eyes were taken by the DSLR, only the left eye was taken by the iPhone 13 Pro with cell scope and only the right eye was taken by the iPhone SE. The rest of the children had eyes taken by all three camera systems. Females in the study tended to be older than the males, but the difference was not significant.

Table 1.

Characteristics of children included in the study

Age group (years)	Male, n (%)	Female, n (%)	Total, n (%)
1–3	17 (60.7)	17 (57.1)	34 (56.7)
4–7	11 (39.3)	15 (46.9)	26 (43.3)
Total	28 (100.0)	32 (100.0)	60 (100.0)a

^aA total of 62 children contributed with 122 eyes: 60 bilateral and 2 unilateral. Data for age and gender for two children are missing.

The iPhone SE had the most gradable images (89%), followed by the iPhone 13 Pro with cell scope (86%) and the DSLR (71%) (Table 2). Compared with the DSLR camera, the iPhone SE and iPhone 13 Pro with cell scope were both more likely to capture high-quality, gradable photos (p<0.001 and p=0.02, respectively). There was no significant difference in the quality of photos between the iPhone SE and iPhone 13 Pro with cell scope (p=0.18).

Table 2.

Comparison of image quality between the three camera systems

		iPhone SE
Camera type		Gradable	Ungradable	Total	McNemar's test
	Eyes with photos by DSLR and iPhone SE (n=113)a
DSLR**	Gradable	72	7	79	p<0.001
	Ungradable	28	6	34
	Total	100	13	113
	Eyes with photos by iPhone 13 with cell scope and iPhone SE (n=104)b
iPhone 13 Pro with cell scope	Gradable	79	10	89	p=0.18
	Ungradable	12	3	15
	Total	91	13	104
		Gradable	Ungradable
	Eyes with photos by DSLR and iPhone 13 Pro with cell scope (N=113)
DSLR	Gradable	73	7	80	p=0.02
	Ungradable	24	9	33
	Total	97	16	113

^aFor nine children only one eye had images from both camera systems.

^bFor nine children, neither eye had images from both the iPhone SE and the iPhone13 Pro with cell scope.

The proportion of eyes with TF was greater in the DSLR gradable images compared with the eyes in either the iPhone SE images or the iPhone 13 Pro with cell scope images (Table 3), but the differences were not significant. Since the DSLR had more ungradable images than the other two systems, we explored the rate of TF in the DSLR ungradable images as detected in the same eyes but with gradable iPhone SE and iPhone 13 Pro with cell scope images (Table 4). There was a tendency for the eyes in the ungradable DSLR images to have less TF compared with the images from the other camera systems. Due to the small number of those with TF, the differences were not significant, but the tendency may explain the slightly higher rate of TF in the gradable DSLR images.

Table 3.

Among the gradable images, the proportion that had TF by camera type

	Camera type
Images	DSLR (n=122)	iPhone SE (n=113)a	iPhone 13 Pro with cell scope (n=113)
TF gradable, n/N (%)	87/122 (71.3)	100/113 (88.5)	97/113 (85.8)
TF among gradable, n/N (%)	9/87 (10.3)	9/100 (9.0)	8/97 (8.3)

^aNine children had both eyes taken with the DSLR but only the right eye taken with the iPhone SE and the left eye taken with the iPhone13 Pro with cell scope. The rest of the children had eyes taken with all three systems.

Table 4.

Image grade for presence of TF in DSLR images that were gradable and ungradable for the iPhone SE and iPhone 13 Pro with cell scope

Camera	Image	n	iPhone SE TF absent, n (%)	iPhone SE TF present, n (%)	iPhone SE ungradable, n (%)
DSLR quality grade	Ungradable	34	27 (79.4)	1 (2.9)	6 (17.7)
	Gradable	79	64 (81.0)	8 (10.1)	7 (8.9)
			iPhone13 Pro with cell scope TF absent, n (%)	iPhone13 Pro with cell scope TF present, n (%)	iPhone13 Pro with cell scope ungradable, n (%)
DSLR quality grade	Ungradable	33	23 (69.7)	1 (3.0)	9 (27.3)
	Gradable	80	66 (82.5)	7 (8.7)	7 (8.7)

Among the gradable images, there was excellent concordance between grading TF and no TF in images from all three camera systems (Table 5). There was no difference by age, gender or laterality in the prediction of gradeability by camera type (Table 6).

Table 5.

Comparison of TF detection among eyes with gradable photographs by both camera types

	iPhone SE
Camera type		TF absent	TF present	κ statistic
Eyes with gradable photos by DSLR and iPhone SE (n=72)
DSLR	TF absent	64	0	1.0
	TF present	0	8
Eyes with gradable photos by iPhone 13 Pro with cell scope and iPhone SE (n=104)
iPhone 13 Pro with cell scope	TF absent	71	0	1.0
	TF present	0	8
		iPhone 13 Pro with cell scope		κ statistic (95% CI)
		TF absent	TF present
Eyes with gradable photos by DSLR and iPhone 13 Pro with cell scope (n=73)
DSLR	TF absent	65	0	0.93 (0.78 to 1.0)
	TF present	1	7

CI: confidence interval.

Table 6.

Association of age, gender and laterality with gradable images by camera type with ORs and 95% CIs

	Camera type
Characteristic	DSLR, OR (95% CI)	iPhone SE, OR (95% CI)	iPhone 13 Pro with cell scope, OR (95% CI)
Age (per year increase)	1.11 (0.86 to 1.44)	1.82 (0.97 to 3.41)	0.87 (0.65 to 1.16)
Female (male reference)	1.02 (0.42 to 2.50)	2.96 (0.91 to 9.63)	0.56 (0.15 to 2.06)
Right eye (left reference)	0.73 (0.37 to 1.46)	1.22 (0.33 to 4.52)	0.84 (0.34 to 2.11)

CI: confidence interval; OR: odds ratio.

Discussion

In our prior work with relatively inexpensive, widely available smartphones, we found very high rates of ungradable images.⁹ In large part this was due to issues of proprietary post-processing software in the phones or speed of focus and the time between pressing the ‘shutter’ and image capture. We sought to determine if using high-quality, flagship smartphones would improve gradeability. We initially sought to use a Samsung Galaxy A53, an iPhone SE (third generation) and an iPhone 13 Pro, compared with a DSLR. During initial training it was clear that the Samsung was incapable of reliably taking gradable images of the upper everted eyelid, even in the hands of the trainers. The issue appeared to be Samsung's proprietary post-processing software rather than any other feature, and it was unclear what aspect of the eyelid (tears, movement, etc.) rendered the subsequent image ungradable. In any case, we abandoned the Samsung smartphone from further study.

We found a high rate of gradable images, 89% and 86%,respectively, using the iPhone SE and iPhone 13 Pro with cell scope. This is much higher than we found in our previous study, where ungradable images ranged between 14% (Samsung Galaxy S8) and 71% (Tecno Camon 17 Pro).⁹ The proportion of gradable images was not as high as that reported for the iPhone 4S with cell scope (99%).⁶ The latter article did not discuss the training undergone by the photographers or previous experience, which would have been helpful in understanding differences. Our smartphone results were significantly better than those for the DSLR, but that is likely due to a malfunction in the aperture setting of the DSLR, which unfortunately was not detected during the survey. Typically, in the hands of good photographers, one can capture up to 100% gradable images using the DSLR.³ The photographers using the iPhones were also second or third in line for taking images of the everted eyelid after the DSLR, because the DSLR images were required for the primary research survey and could not be compromised by order. If the iPhone images had been the first images to be taken, it may have resulted in improved rates of gradeability. In our previous study, we found that the order of the cameras used influenced the gradeability of the images to a small but significant extent.⁷

The photographer using the iPhone 13 Pro with cell scope reported being able to obtain images much quicker because of the forehead stabilization feature, but the cell scope had no impact on the proportion of gradable images compared with the simple iPhone SE. The cell scope light feature was also finicky, and battery replacement was not easy once the cell scope was attached to the smartphone. The photographer reported the wire came loose linking the light to the battery, so there was no reliance on the light of the cell scope as the image light. These findings differ from those of Nesemann et al.,⁶ who performed a survey of 412 children ages 1–9 y to see if there was a difference in grading sensitivity for the SLR compared with the iPhone 4S with cell scope. They found that the sensitivity of the smartphone with cell scope was significantly higher when compared with the smartphone camera without attachment that was used in a previous study.⁴ The differences might be explained by the better camera system in the iPhone SE compared with the iPhone 4S without the cell scope. Nesemann et al.⁶ also had a very high rate of gradable images using the cell scope, where at best we had 11% ungradable images. This might be explained by the inclusion of children 7–9 years of age in the former study, who are less troublesome to photograph compared with younger children, whereas more than half our study children were <4 y of age. Of note, both studies support the reliability of the iPhone smartphone technology (especially later models) for producing gradable images and detection of TF. Furthermore, our research suggests that iPhone technology is sufficient without the attachment, although the cell scope head stabilization component made image acquisition much faster due to rapid and sustained focus. This comparison should be considered for future studies.

There is always an issue with defining the ‘gold standard’ for comparison of TF detection. Many studies use clinical field examination. In a prior study by Roper and Taylor,¹⁰ photographic examination proved to be a good tool, but when compared with clinical assessment it resulted in an ‘overestimation’ of active trachoma. In other studies, the sensitivity of early-technology smartphone cameras for grading of TF compared with field grades was reported to be low.^3,4 However, field grading is also not perfect, and especially in low-prevalence settings, overestimation has been shown to be an issue.³ The advantage of images is that graders can take time to scrutinize the images for five follicles of appropriate size, where pressure to grade quickly and relieve the burden on children may result in over- or underestimating TF in the field. We did not have eye-specific field examination results for comparison with the camera systems, but the concordance for TF among the gradable images from the three camera systems was almost perfect.

Ungradable photos are one of the major difficulties affecting the use of photography instead of field graders.³ Ungradable photos are caused by a variety of factors, including insufficient training of photographers, older smartphone camera technology and poor cooperation of subjects.³ Ungradable photos were present across all camera types utilized in our study and there is still room for improvement to reach <5% ungradable images. The higher rate of ungradable images with the DSLR system was due to focus issues that were not detectable during the survey. The fact that 18% of DSLR ungradable images were from eyes with images taken by the iPhone SE that were also ungradable suggested that other factors, such as poor subject cooperation, were involved. We did not find that gender, age (within the 1- to 7-year-olds) or laterality of the eye made any difference in the gradeability of the images from any system. We are encouraged by these results and the next steps would be to use the iPhone SE or 13 Pro with cell scope in a survey of children ages 1–9 y and to use the camera systems in random order to assess if the rate of ungradable images decreases further.

One limitation of our study was the sample size of 62 children, which limited our ability to detect small differences between the camera types. The sample size was dictated by the availability of photographers and the design of the larger study. In addition, at the outset of the study we were worried about parental consent to have all three cameras take images of both eyes, but after the first nine children, where photography went quickly with two camera systems per eye, we added all three systems per eye with minimal additional time and no concerns expressed by the parents. However, it did mean that for nine children we only had either the right or left eye for the iPhone SE or 13 Pro systems. We also did not rotate the camera systems between photographers, which would balance out any photographer effect. We chose not to train all photographers to use the DSLR system, as the primary research protocol specified using the experienced photographer who routinely uses that camera system. The other two photographers were trained and certified on both the iPhone SE and iPhone 13 Pro, so we expected minimal differences between them, but that could still be an issue.

In summary, our findings suggest that for prevalence surveys, TF detection utilizing smartphone technology and standardized remote graders could be a reliable alternative to field graders,¹¹ providing the proportion of ungradable images can be further reduced. In this study we opted for smartphones with better technology, albeit at slightly higher prices than the previous study, but much better image quality resulted.⁹ Our findings demonstrate a much-improved rate of ungradable images over our previous studies of smartphones in a survey of largely young (<4 y of age) children. Further studies using the iPhone SE or 13 Pro that have a higher proportion of older children, as would be the case in trachoma prevalence surveys, should improve the proportion of gradable images. This supports the idea that moving toward smartphone camera systems requires investment in models that can demonstrate repeated acquisition of high-quality images of everted eyelids in children. While the cost of higher-technology smartphone systems may pose a barrier, we were encouraged that the lower-cost iPhone SE (current price 429) performed as well as the higher-priced iPhone 13 Pro (price at time of purchase 900). Moreover, we anticipate the price of new and pre-owned models to decrease as newer models are introduced, which would improve accessibility.

Authors’ contributions:

MW, CB, and SKW provided funding for the project. MW and SKW conceived and designed the project. HM, MW and SKW collected data. UA, BM and SKW contributed to data analyses. UA wrote the paper. HM, BM, CB and SKW provided review and re-wrote sections of the paper.

Funding:

Funding was provided to SKW by the El Maghraby Chair in Preventive Ophthalmology and funding for the survey was provided by a grant from the Task Force for Global Health to MAW. CJB was supported by the National Institute of General Medical Sciences of the National Institutes of Health (award number P20GM125498). The funders had no role in the design, implementation, data analyses or decision to publish this study. We are grateful to Dr Jeremy Keenan and his team for supplying the cell scope for use in this study.

Competing interests:

None of the authors have declared any competing or conflict of interest.

Ethical approval:

Ethical approval was obtained from the Johns Hopkins Institutional Review Board and the National Institute for Medical Research in Tanzania.

Data availability

De-identified data are available upon request and approval by Johns Hopkins Institutional Review Board by qualified researchers directed to Beatriz Munoz, Senior Data Manager, Dana Center for preventive ophthalmology, Wilmer Eye Institute, Wilmer rm 129 600 N Wolfe St Baltimore MD, 21205.