Comparison of the effectiveness of three retinal camera technologies for malarial retinopathy detection in Malawi

Abstract

The purpose of this study was to test the suitability of three available camera technologies (desktop, portable, and i-phone based) for imaging comatose children who presented with clinical symptoms of malaria. Ultimately, the results of the project would form the basis for a design of a future camera to screen for malaria retinopathy (MR) in a resource challenged environment. The desktop, portable, and i-phone based cameras were represented by the Topcon, Pictor Plus, and Peek cameras, respectively. These cameras were tested on N=23 children presenting with symptoms of cerebral malaria (CM) at a malaria clinic, Queen Elizabeth Teaching Hospital in Malawi, Africa. Each patient was dilated for binocular indirect ophthalmoscopy (BIO) exam by an ophthalmologist followed by imaging with all three cameras. Each of the cases was graded according to an internationally established protocol and compared to the BIO as the clinical ground truth. The reader used three principal retinal lesions as markers for MR: hemorrhages, retinal whitening, and vessel discoloration.

The study found that the mid-priced Pictor Plus hand-held camera performed considerably better than the lower price mobile phone-based camera, and slightly the higher priced table top camera. When comparing the readings of digital images against the clinical reference standard (BIO), the Pictor Plus camera had sensitivity and specificity for MR of 100% and 87%, respectively. This compares to a sensitivity and specificity of 87% and 75% for the i-phone based camera and 100% and 75% for the desktop camera. The drawback of all the cameras were their limited field of view which did not allow complete view of the periphery where vessel discoloration occurs most frequently. The consequence was that vessel discoloration was not addressed in this study. None of the cameras offered real-time image quality assessment to ensure high quality images to afford the best possible opportunity for reading by a remotely located specialist.

1. INTRODUCTION

1.1. Background

Making an accurate diagnosis of cerebral malaria (CM) remains a clinical challenge in many parts of Africa. In areas where malaria is endemic, malaria parasitemia is common, even among children without symptoms. Thus when a child becomes comatose from any cause, simply detecting malaria parasites in the blood is not specific to CM. In fact, a large autopsy study revealed that 23% of children with clinically defined CM were found to have died of unsuspected non-malaria causes [1]. Moreover, the presence of highly specific lesions in the eye, known as malarial retinopathy, which was identified in the comatose children before death, was the best indicator of CM at autopsy. MR occurs in a spectrum of severity and among children with clinically defined CM the severity of MR correlates well with severity of various features of CM.

Although examination by an ophthalmologist remains the reference standard for detection of MR, this form of examination depends upon the availability of technical resources as well as qualified and highly-skilled ophthalmic expertise. In the regions of Africa where CM is highly prevalent there is a scarcity of above mentioned resources including ophthalmic expertise. Similarly, in this resource limited region of the world, the medical diagnosis must be easily accessible and affordable to the affected population. These requirements demand a low-cost retinal imaging device that is easy to use and produces sufficient image quality to perform screening or diagnosis.

As a result of relatively recent improvements in digital retinal imaging technology, the retinal imaging process is now possible at sites other than the malaria care clinics, thus permitting diagnoses using image visualization by ophthalmic experts located remotely, through tele-retinal screening services. The aim of the study was to test and compare the performance of three retinal imaging cameras (two handheld cameras and one desktop camera) against BIO (reference standard) for the detection of specific signs of MR. The ultimate aim was to identify design and technological shortfalls in today’s existing devices and their associated technologies in order to influence designs of a future, low-cost retinal imaging device that can be used to image and diagnose MR at the point of care.

Studies comparing the sensitivity of different types of digital retinal cameras to BIO for detecting MR have not been carried out in a clinical setting such as the Blantyre, Malawi Hospital. Such a study could lead to broader availability digital imaging for appropriate intervention and in the future, such studies could be useful to researchers seeking further understanding of the pathophysiology of CM and MR. In this work, we address the important questions regarding the clinical performance of retinal cameras in capturing signs of MR, which would provide the information needed to determine whether handheld and desktop retinal imaging cameras might be able to supplant or complement BIO as a means for diagnosing CM. Such a low-cost retinal imaging device would be useful and could be made widely available to clinicians who routinely face the challenge of diagnosing and treating CM.

2. METHODS

2.1. Description of the Devices

Two of the design and technical factors affecting the results of the inter camera comparison are the resolution and field of view (FOV) of each camera (See Table 1). While the optimum set of images would include the posterior pole as well as the periphery of the retina, the limitations of the camera’s FOV required the ophthalmologist to acquire multiple images to maximize the area imaged by the camera. For example, the Peek employed a different implementation of the image capture process. Peek acquires a video rather than a single image. In principle, this allows the user to scan the retina and collect multiple frames of the posterior pole and periphery. The challenge is keeping the camera aligned with the pupil and the target area on the retina, and focused on the retinal surface. Misalignments result in different kinds of light reflection artifacts that significantly affect image quality to the point of marking important MR features on the retina.

Table 1.

Camera specifications.

Camera	FOV	Pixel format	Operational mode
Topcon	50 deg	96 DPI, 8 bits	Single frame
Pictor Plus	45 deg	72 DPI, 8 bits	Single frame
Peek	20 deg	72 DPI, 8 bits	Video (30 frames/sec)

2.2. Study population

The study consisted of prospective data collection in the context of an ongoing descriptive study of pediatric CM at the Queen Elizabeth’s Hospital in Blantyre, Malawi. An ophthalmologist performed MR assessment through BIO examination as well as the retinal imaging using three retinal cameras (Topcon, Pictor Plus, and Peek) on 23 comatose children admitted to the Hospital. Parents of the children provided consent according to international research standards and with institutional review board (IRB) approval from the University of Michigan (IRB# 06–1012M). Of the N=23 children enrolled in the study, N=8 were controls, that is, they had a clinical diagnosis of CM but did not present with signs of MR, and N=17 were clinically diagnosed with CM and presented signs of MR. Figure 1 shows those signs: white-centered retinal hemorrhages (left), vessel discoloration (middle), and retinal whitening (right).

Figure 1.

Retinal features associated with malaria retinopathy and cerebral malaria. a) white centered hemorrhages; b) vessel discoloration; c) retinal whitening. These are images acquired with the TopCon retinal camera. As seen in 1c, a patient can present with multiple types of lesions (retinal whitening and white centered hemorrhages.

2.3. Data grading

Using an internationally recognized grading system developed by the MR grading consensus group at the University of Liverpool, the retinal images were graded by an independent certified retinal reader for the detection of the specific signs of MR. The reader’s grading was masked to previous reads from other cameras, ophthalmologist’s findings in the BIO exam, and clinical status. The BIO exams revealed twelve cases with whitening, eight cases with hemorrhages, and five cases with vessel discoloration. Several cases had multiple MR lesions.

The reader assessed the two different imaging modalities, still frames from the Pictor Plus and Topcon, and video from the Peek. The Peek camera, which is cellphone-based, captured an average of four videos per subject (two per eye); the Pictor Plus, a portable retinal camera, eight images per subject (four per eye); and finally, six images per subject were acquired with the Topcon, a tabletop camera. The reader graded each eye individually, masked to previous reads from other cameras as well as the BIO results.

2.4. Data analysis

The retinal image assessments by the certified grader for each camera were compared to the BIO examination findings for detection of MR-specific pathologies as shown in Figure 1. Sensitivity and specificity for MR and for specific lesion detection as well as statistical agreement coefficient (Kappa) in defining the grade of each lesion were obtained to evaluate the performance of each camera and its usability in detecting MR. The specific comparisons were in the form of:

1. Determining the inter-device agreement as judged by a single certified grader between, 1) the hand-held Pictor Plus and the BIO examination, 2) handheld Peek imager and BIO, and 3) Topcon and BIO, in classifying a clinically diagnosed CM patient into MR positive or MR negative (as determined by the BIO exam). The features graded in identifying and grading the various features of malarial retinopathy were: hemorrhages in macula, peripheral hemorrhages, macular whitening, and peripheral whitening. (Vessel discoloration was found in the BIO examination, but only in the far periphery where none of the cameras were able to image.)

2. Determining subjective and qualitative evaluations of each camera by the ophthalmologist in terms of factors such as usability, ergonomics, portability, view on the quality of images and suggestions for improvements to the user interface. Cost was also considered.

3. RESULTS

3.1. Image quality

Apart from calculating the performance of each camera in detecting MR lesions, the cameras were also evaluated for image quality by quantifying the fraction of the total retinal area that was available for grading (captured by the camera) and gradable (adequate image quality as determined by the retinal grader). The results shown in Table 2 demonstrate that all three cameras could capture only a partial view of the peripheral retina.

Table 2.

Image quality assessment given as a percent of the image area that was gradable.

Image quality	Peek	Pictor	Topcon
Macular region	72%	83%	87%
Out of macula (Peripheral)	3%	47%	69%

Figure 2 presents sample images from each of the three cameras during the same examination for the same subject. This subject had retinal hemorrhages, the majority of which were white centered, and retinal whitening. A large hemorrhage is centered over the fovea/macula. For the Peek, a sample of three still frames from the video captured are presented in Figure 2. However, for the reading of the cases using Peek, the entire set of four 20–30 second videos was used. Patches of retinal whitening are visible temporally from the macula in both the Topcon and Pictor Plus views. As seen in the images, the Pictor Plus captures the complete hemorrhages in the far temporal region of the retina, while the Topcon mask has cut them in half.

Figure 2.

Images captured for a subject: (Top) Sample frames from Peek video, (Bottom) Left: Pictor Plus, Right: Topcon.

The large central macula hemorrhage (Figure 2, red arrows) and several smaller ones can be seen in the Peek video. Peek also captures signs of whitening, but is frequently masked by glare or reflection artifacts, which makes the whitening detection difficult and unreliable. In addition to masking the true whitening (Figure 3, white arrows), these artifacts can be mistakenly misidentified as whitening. Retinal reflexes, different from reflection reflexes are natural occurrences due to the nature of the retina and can be differentiated from the glare artifacts mentioned previously. The retinal reflexes were more common with Peek and were at minimum a distracting, but often masked the presence of one of the MR pathologies. The artifacts in Peek can be seen in Figure 2 and Figure 3.

Figure 3.

Images showing retinal whitening: (Top) Sample frames from Peek video. (Bottom) Left: Pictor Plus, Right: Topcon. Whitening was not evident in the Peek image which led to incorrect evaluation of the subject.

The Peek has a limited FOV of approximately 20°, and the views captured depend on examiner’s skill. Furthermore, the design of Peek’s camera attachment to a cell phone makes it difficult to bring the camera-optics close to the patient’s cornea, which affects the image quality. The flat, rectangular shape of the Peek makes it difficult to align and requires an unnatural handling of the device. Its small FOV and its flat rectangular shape make the Peek a challenge to align in order to capture the intended region of the retina. Because of this difficulty in aligning the Peek, peripheral regions were almost always absent in Peek’s field of view.

Figure 3 (next page) shows an example of a patient with retinal whitening in the temporal region of the retina (near the macula), which is clearly seen on both the Pictor Plus and Topcon images. However, using the Peek video, the physician could only image the area surrounding the optic disc, but could not image the temporal region where the retinal whitening was present. Additionally, when the macula was captured, there was significant reflection from cornea of the camera illumination source. Focusing the Peek was problematic. Being able to both maintain the correct focus on the retina while at the same time moving the camera in the direction needed to image all retinal regions is one aspect that adds difficulty when trying to image all the areas of interest in the retina. The Peek camera may perform adequately in the hands of an ophthalmologist experienced with this device and when used for immediate diagnosis, but it is more challenging to master than the Pictor Plus or Topcon for a non-ophthalmic technician.

Table 3 shows that the Pictor Plus produced images with quality and resolution to allow the best overall performance, sensitivity of 100% in detecting integrated signs of MR, with a specificity of 87%. Hemorrhages and whitening were all detected in the Pictor Plus images. These results for the Pictor Plus were all superior to the other two cameras. None of the cameras were able to detect the vessel discoloration seen in the BIO examination by the ophthalmologist. This is likely to be a result of the lesions being in the periphery where none of the cameras collected imagery.

Table 3.

Sensitivity and specificity for detecting MR, hemorrhages, and whitening using the three cameras.

Category	Peek		Pictor Plus		Topcon
	Sensitivity	Specificity	Sensitivity	Specificity	Sensitivity	Specificity
Malaria Retinopathy	87%	75%	100%	87%	100%	75%
Hemorrhages	80%	100%	100%	100%	100%	87%
Whitening	82%	87%	100%	100%	100%	67%

Apart from determining the accuracy of each camera in detecting MR lesions, the cameras were also evaluated by the user (ophthalmologist) in terms of usability, ergonomics and relevant factors, as described in table 4.

Table 4:

Factors evaluating the camera usability

Factor	Peek	Pictor Plus	Topcon
Set-up	Cellphone	Handheld	Desktop
Field of view	20°	45°	50°
Ergonomic	Handy	Handy	Bulky
Size, weight	Small, light	Portable	Cumbersome
Imaging stability	Highly unstable	Stable	Stable
Image quality	Poor, artifacts	Adequate	Adequate
Cost	$500	$10,000	>$25,000

4. DISCUSSION

Although BIO has been the reference standard for identifying MR, it is important to recognize that may not necessarily be optimum for locations where ophthalmic expertise is not readily available. Fundus photography has the advantage of enabling a telemedicine environment and allowing the grader to study an area more closely and can provide higher magnification than BIO. The table top cameras such as the Topcon pose a significant challenge to the imager when trying to maneuver the camera and perform alignment when the subject is comatose and in a supine position as shown in Figure 4. This was the major disadvantage of the Topcon. The handheld and easy-to-use retinal imaging systems such as the Pictor Plus and the Peek do provide portability and usability advantages, but produce varying quality color fundus images or videos. As demonstrated in Table 3, all of these cameras do provide some capability for MR screening.

Figure 4.

Imaging malarial retinopathy in an infant using Topcon camera.

Although these cameras allow the collection of images from comatose children diagnosed for CM, all three of them have design or capability shortfalls for MR examination. The Topcon’s design becomes a cumbersome attribute for the MR application given the state and position of the typical malaria patient (Figure 4). None of the three cameras offer real-time image quality feedback to the user. In these resource limited application, the imager will likely be a minimally trained healthcare worker who may not recognize an inadequate image prior to transmitting the image to the specialist who will make the diagnosis. In a rapidly progressing disease like cerebral malaria, the time taken to re-image the patient could significantly impact the outcome of the patient’s recovery.

The lack of single shot wide field (≥ 120°) limits the utility given that some MR lesions occur in the periphery where multiple images would be required to capture the full area of the posterior pole. Finally, an automatic MR screening device is needed in these regions where immediate care is required, but accurate diagnosis by a specialist is not easily available.