Speed of Telemedicine versus Ophthalmoscopy for Retinopathy of Prematurity Diagnosis

Introduction

Retinopathy of prematurity (ROP) is a vasoproliferative retinal disease which affects low birth weight infants. Diagnostic and treatment criteria have been validated through the Cryotherapy for Retinopathy of Prematurity (CRYO-ROP) and Early Treatment for Retinopathy of Prematurity (ETROP) trials,^1-4 yet ROP remains a leading cause of childhood blindness in the United States and around the world.^5,6 In developed countries, the number of infants at risk for ROP is rising because of increasing premature birth rates from assisted conception, increasing maternal age, socioeconomic factors, and possible genetic etiologies.^7,8 In Asia, Latin America, and Eastern Europe, the number of ROP cases has increased due to higher overall birth rates, as well as improved neonatal survival from greater availability of neonatal care.^9,10 Concerns have been raised about an emerging international ROP “epidemic” due to persistent variability in neonatal care and a lack of adequately-trained ophthalmologists.^10-13

There are many logistical barriers to providing optimal ROP care. Serial ophthalmoscopy is typically performed by retinal specialists or pediatric ophthalmologists at the neonatal intensive care unit (NICU) bedside. This is time-consuming, and requires significant coordination between ophthalmologists and NICU staff. Findings are documented by recording location, severity, and nature of disease, and traditionally include a hand-drawn picture. This is potentially subjective, and can be a factor in medicolegal liability.¹⁴ Because of these and other concerns, the number of ophthalmologists who are willing to manage ROP is decreasing.¹⁵

Store-and-forward telemedicine is an emerging technology which involves the capture of patient data for subsequent interpretation by a remote medical specialist.^16-18 This has potential to address many of the challenges associated with ROP care. Previous studies have shown that telemedicine has high accuracy and reliability, particularly for detection of clinically-significant ROP,^19-31 and that trained neonatal nurses can acquire wide-angle retinal images of good quality.^21,24,27,29 Telemedicine has also been shown to be more cost-effective than standard ophthalmoscopy for ROP care.³² Based on these findings, small-scale ROP telemedicine systems have been implemented in the United States and internationally.

However, far less is known about the impact of telemedicine on physician efficiency and workflow. In particular, widespread implementation of these systems will require understanding of time commitment requirements for ophthalmologists. The purpose of this study was to evaluate the time required by ophthalmologists for ROP diagnosis using telemedicine compared to standard indirect ophthalmoscopy at the NICU bedside. This study was specifically designed to evaluate ophthalmologist time commitments, and did not analyze processes such as capture of telemedical retinal images by NICU nurses.

Methods

Study Examiners

All ophthalmoscopic and telemedicine diagnoses in this study were performed by two pediatric retinal specialists (RVPC, TCL) and one pediatric ophthalmologist (MFC). Each study examiner was experienced in ROP diagnosis and management, and was responsible for performing regular ROP examinations at an academic medical center. Each examiner had previously published peer-reviewed studies involving ROP, and two (TCL, MFC) had served as certified investigators in the ETROP study.

Ophthalmoscopic Diagnosis

Study examiners performed standard ROP diagnoses in the normal manner at their respective institutions (RVPC at Weill Cornell Medical College, TCL at Childrens Hospital Los Angeles, MFC at Columbia University Medical Center). Infants who met published guidelines for requiring examination⁴ were identified by a designated NICU staff member. Patients were dilated with either 0.2% cyclopentolate and 1.0% phenylephrine eyedrops (Cyclomydril; Alcon, Fort Worth, TX), or with 0.5% cyclopentolate and 2.5% phenylephrine eyedrops (Alcon, Fort Worth, TX) at least 30 minutes prior to ophthalmoscopy.

Examiners walked from their outpatient offices to NICUs at a scheduled time each week, and performed serial examinations according to their usual routines. During a four-month period, examiners were followed by a NICU staff member, pediatric fellow, or ophthalmology resident who recorded the times associated with all ROP examinations. The following times were noted on a standardized template: (1) time leaving the outpatient office; (2) time of arrival at the NICU; (3) start time of every examination, defined as arrival at the infant bedside; (4) end time of every examination, defined as completion of ophthalmoscopy and chart documentation; (5) time of departure from the NICU; and (6) time of return to the outpatient office.

Telemedicine Diagnosis

A web-based ROP telemedicine application was previously developed for research studies.^21,27 In these studies, parents provided written informed consent for retinal photography. This included 125 de-identified image sets from both eyes of 67 consecutively-recruited infants at Columbia University. During a one-year study recruitment period, this represented 62% of all inpatients who required ROP screening examinations based on published guidelines.^2,4 No infants were excluded because of perceived poor image quality or inability to capture images. Using this data set, telemedicine diagnosis has previously been shown to have high accuracy, inter-grader reliability, and intra-grader reliability.²¹ Birth weight, gestational age, and post-menstrual age at imaging were displayed for each infant. Image sets consisting of 3-5 wide-angle photographs from each retina were displayed both eyes at a time (Figure). The system permitted examiners to magnify images as needed to support diagnosis.

Figure. Web-based telemedicine system for retinopathy of prematurity (ROP) diagnosis.

Wide-angle posterior, nasal, and temporal images of each retina are displayed for examiners, along with up to 2 additional images per eye if felt by photographer to contribute diagnostic value. Examiners are asked to submit diagnosis for each eye, recommended follow-up interval, and evaluations of image quality.

Each examiner (RVPC, TCL, MFC) used this system to provide diagnoses for each eye as: (1) No ROP; (2) Mild ROP, defined as any ROP less than type-2 disease; (3) type-2 ROP (zone 1, stage 1 or 2, without plus; or zone 2, stage 3, without plus); (4) treatment-requiring ROP, defined as type-1 ROP (zone 1, any stage, with plus disease; zone 1, stage 3, without plus disease; or zone 2, stage 2 or 3, with plus disease) or worse; or (5) Unknown, meaning the examiner was uncomfortable making a diagnosis based on the data provided.

The web-based telemedicine system was programmed to record timestamps reflecting start and stop times for diagnosis of each patient, including time for documentation of findings. For each patient, examiners were asked to submit a diagnosis for each eye as described above, a recommended follow-up interval for the patient, a three-level evaluation of image quality, and a three-level evaluation of retinal image coverage. Telemedicine diagnosis time was considered to start when a new patient webpage was opened, and to end when the examiner submitted all required data for that patient.

Data Analysis

Time for ophthalmoscopic diagnosis of each patient was calculated in two ways: (a) time spent by examiner at the infant bedside (i.e. “end time” – “start time” of exams), and (b) mean total time commitment per infant (i.e. [“time returning to outpatient office” – “time leaving outpatient office”] / “total number of infants examined”).

Time for telemedicine diagnosis of each patient was recorded by computer timestamps in the web-based system. “Unknown” diagnoses were excluded from calculations. Because the telemedical diagnosis data in this study were used for validation research evaluating the accuracy of telemedicine,²¹ we believe that they serve a reasonable proxy for real-world diagnosis.

For each examiner, non-parametric statistical analysis (Mann-Whitney) was used to compare the distribution of times for diagnosis using ophthalmoscopy versus telemedicine.

Results

The time required for telemedicine diagnosis was significantly lower than the time required for diagnosis at the infant bedside, for all examiners (p<0.0001 for each examiner) (Table 1). Mean (±SD) times for telemedicine diagnosis ranged from 1.02 (±0.27) minutes to 1.75 (±0.80) minutes. Mean (±SD) times spent by examiners at the infant bedside for ophthalmoscopic diagnosis ranged from 4.17 (±1.34) minutes to 6.63 (±2.28) minutes. The latter times included all of the examiners' activities occurring at the infant bedside, such as positioning of the infant for examination, actual indirect ophthalmoscopy, repositioning of the infant, and documentation of examination findings.

Table 1. Time comparison of ophthalmoscopy vs. telemedicine for retinopathy of prematurity (ROP) diagnosis by 3 experienced examiners.

Ophthalmoscopic diagnosis time was measured beginning at arrival to the infant bedside, and ending after documentation of findings. Telemedicine diagnosis time was measured using computer-based timestamps beginning at opening of examination page, and ending after documentation of findings.

	Ophthalmoscopy		Telemedicine
Examiner	n	Mean (range) ±SD time, minutes	n	Mean (range) ±SD time, minutes	P-value
A	73	4.42 (1-11) ±2.09	125	1.02 (1-4) ±0.268	<0.0001
B	72	6.63 (2-13) ±2.28	125	1.34 (1-5) ±0.742	<0.0001
C	150	4.17 (1-11) ±1.34	122a	1.75 (1-7) ±0.796	<0.0001

^aExaminer C reported 3 diagnoses as “unknown.”

When considering the examiner's total time commitment per infant, the disparity between ophthalmoscopy and telemedicine increased (p<0.0001 for each examiner) (Table 2). Among the three examiners, the mean (±SD) total time commitment per infant ranged from 10.08 (±2.53) minutes to 14.42 (±2.64) minutes. In addition to time spent by the examiner at the bedside, this included travel, communication with families and staff, and other logistical factors.

Table 2. Mean total time commitment associated with bedside ophthalmoscopic retinopathy of prematurity (ROP) diagnosis by 3 experienced examiners.

This was calculated as the total time away from the ophthalmology outpatient office during ROP examinations each week, divided by the total number of patients. P-values are determined compared to telemedicine diagnosis by each examiner.

Examiner	n	Mean time (±SD), minutes	P-value
A	73	10.08 (±2.53)	<0.0001
B	72	14.42 (±2.64)	<0.0001
C	150	11.22 (±2.62)	<0.0001

Discussion

This study was designed to compare the ophthalmologist's speed of ROP diagnosis using telemedicine versus traditional bedside ophthalmoscopy. Our findings revealed that: (a) ROP diagnosis by the ophthalmologist is significantly faster via telemedicine, and (b) There are significant time requirements by ophthalmologists associated with ROP diagnosis at the NICU bedside beyond ophthalmoscopy. In this study, ophthalmoscopic diagnoses were performed during routine clinical care, whereas telemedicine images were captured from infants during clinical care but were interpreted specifically for a research protocol. It could be argued that the ophthalmoscopic diagnoses may have been performed more carefully, given that real-life visual outcomes were at stake. However, we also note that all telemedicine findings were analyzed carefully, and that these accuracy results were shared among examiners and published separately in the peer-reviewed literature.²¹ We therefore feel that telemedicine study examiners had strong incentives to review images carefully, and believe that these telemedicine diagnoses were performed with comparable detail to that of real-world ophthalmoscopic examinations.

Traditional ROP diagnosis with indirect ophthalmoscopy is logistically-difficult and time-consuming. This has led to a decrease in the number of ophthalmologists who are willing to manage ROP, despite a rising number of at-risk infants. A 2006 American Academy of Ophthalmology survey found that only half of retinal specialists and pediatric ophthalmologists are willing to manage ROP, and that one-fifth plan to stop because of concerns including logistical complexities, medical liability, and low reimbursements.¹⁵ In a nationwide survey, Kemper et al. demonstrated that 36% of neonatologists have had the experience of delaying the transfer of a child to a closer NICU due to lack of specialists available for ROP management.³³ Telemedicine might address these problems, both by increasing efficiency and by reducing the need for patient and ophthalmologist to be at the same location.

This time-savings from telemedicine pertains specifically to ophthalmologists. We examined the time required by ophthalmologists for diagnosis and documentation using ophthalmoscopy compared to telemedicine (Table 1). In this study, we made the decision to analyze only physician effort because it is the most costly and limited component of ROP evaluation. Specifically, the time savings from telemedicine could be attributed to avoidance of infant positioning, infant monitoring, scleral depression and manipulation, and other activities associated with bedside diagnosis. This time savings benefit, from ophthalmologists being able to perform accurate diagnosis without leaving their outpatient offices, might be expected to further improve the cost-effectiveness of telemedicine for ROP management.³² However, we note that a full analysis of telemedicine systems may also need to incorporate the cost of wide-angle retinal cameras (approximately $60,000 USD), as well as qualitative effects including patient and provider satisfaction.

Furthermore, there are substantial time requirements associated with bedside ROP diagnosis for ophthalmologists beyond ophthalmoscopy itself (Table 2). These additional activities include: (1) travel to, from, and within the NICU; (2) interactions with neonatologists and other NICU staff; (3) interactions with patients' families; (4) problems such as inadequate pupillary dilation and infants who are medically unstable for ophthalmoscopic examination;^4,34 and (5) logistical issues such as identifying nurses to assist with bedside examinations, coordinating with infant feeding schedules or other diagnostic tests, and locating infants who have moved to different rooms. These factors would not be encountered by ophthalmologists during telemedical diagnosis.

To implement telemedical systems for ROP, significant support from other healthcare personnel would be required. Neonatologists would be responsible for identifying infants requiring ROP evaluation based on national and local guidelines, as is often currently done. Trained nurses, or other personnel available at the point of care, would need to capture, select, and upload high-quality images for remote diagnosis. We have previously shown that this is technically feasible.^21,27,31 This study was not designed to examine nursing time requirements for image capture. However, we note that one previous study reported mean RetCam imaging time (when performed by a pediatric ophthalmologist) of 7.8 minutes per infant, compared to mean ophthalmoscopy time of 3.9 minutes per infant.³⁴ Other studies have estimated image capture times of “usually <1 minute [per eye]” (when performed by an ophthalmologist or technician),³⁰ and “3-6 minutes per eye” (when performed by a neonatal nurse).³¹ Finally, telemedicine images that were judged to be unreadable would need either to be re-taken by nurses in a timely manner, or to be referred for traditional ophthalmoscopic examination. Although telemedicine diagnoses of “unknown” were excluded from this study, they comprised <1% of all responses (Table 1). The time and cost involved in these additional duties, and their impact on neonatal workflow, may deserve additional study.

Telemedicine systems would also require that novel mechanisms be developed for communicating the results of diagnostic findings to neonatologists and families. Access to retinal images, accompanied by interpretations which are automatically transmitted to neonatal staff, could improve communication and knowledge compared to current methods.³⁶ Appropriate web-based consumer health resources for families, as well as context-sensitive online information resources for clinicians, could provide additional information.^36-38 This electronic infrastructure could be used to save time, while improving communication among ophthalmologists, neonatal staff, and families. That said, telemedicine systems will diminish the number of direct personal interactions among physicians, NICU staff members, and parents. For telemedicine to succeed, this paradigm shift must be implemented in a way that is acceptable to patients and health care providers.

In the face of lower reimbursements and rising pressures for clinicians to provide more services in less time, there is a growing body of literature investigating the speed and efficiency of healthcare delivery. These studies suggest that: computer-based medical record systems can improve efficiency of data availability and healthcare provider satisfaction;³⁹ telemedicine can improve quality, efficiency, and patient satisfaction of healthcare;^40-42 and telemedical systems can address barriers related to lack of regional healthcare specialists.⁴³ While many of these technologies have been validated in local settings, some reports suggest that the full impact of their benefits will only be felt if such technologies are utilized more systematically and on larger scales.⁴⁴ For ROP, telemedicine has been shown to have high diagnostic accuracy and reliability.^19-31 In fact, it has been suggested that telemedicine may offer diagnostic advantages over traditional ophthalmoscopy.^14,27

There are several additional factors and limitations that should be considered when interpreting study findings: (1) The routines of ophthalmologists performing ROP diagnosis in the NICU vary from hospital to hospital. The three examiners in this study examined infants with an assistant on most days, and Examiner B's assistant also helped document ophthalmoscopic findings. Also, Examiners A and C used scleral depression and eyelid speculums on most patients, whereas Examiner B did not. These factors may affect the magnitude of the time difference between ophthalmoscopy and telemedicine, although we note that telemedicine was significantly faster by all 3 examiners. (2) Examiners knew they were being monitored when they performed both telemedical and ophthalmoscopic diagnosis in this study, and this may have affected the speed of diagnosis. This might have created a bias because study investigators presumably had an interest in telemedicine systems. In addition, the Hawthorne effect, which refers to improved performance when being observed, has been reported to affect healthcare measurements such as hand-washing compliance and medication error rates.^45,46 Follow-up studies of examiners unaware that they are being monitored could be used to determine the magnitude of these effects. (3) For practical reasons, the number of examinations performed at each session was not standardized. For example, examiners typically performed 5-15 ophthalmoscopic diagnoses per session, while the mean number of infants diagnosed during each telemedicine session was 62 for Examiner A, 14 for Examiner B, and 9 for Examiner C. These differences may have influenced the measured speed of examiners. (4) When measuring speed of telemedicine, this study did not consider factors such as travel time to telemedicine rooms and computer login time. Although we do not believe that these times are significant compared to the measured differences (Tables 1 and 2), this may warrant further analysis. (5) As described above, ophthalmoscopic diagnoses in this study were performed during routine clinical care, whereas telemedicine images were captured from infants during clinical care but interpreted only for a research protocol. This may have biased toward more rapid telemedicine diagnosis, and follow-up studies of real-world telemedicine systems may be informative.

In summary, the implementation of telemedicine for ROP management has potential to decrease the time commitment for examining ophthalmologists. Previous studies have shown that telemedical ROP diagnosis is highly accurate and reliable compared to ophthalmoscopy, and future work is required to address workflow questions in more detail. Given the increased number of at-risk infants in the United States and worldwide, improved utilization of ophthalmology resources may help prevent cases of avoidable childhood blindness.

Biographies

Grace M. Richter will graduate in May 2009 from Columbia University College of Physicians and Surgeons and Mailman School of Public Health with an MD/MPH dual degree. She graduated summa cum laude from Washington University in St. Louis in 2004 with a B.A. in Chemistry and International Studies. Grace plans to pursue an ophthalmology residency, and her current special interests include public health, ocular epidemiology, and international ophthalmology.

Michael F. Chiang is Irving Assistant Professor of Ophthalmology and Biomedical Informatics at Columbia University. His research involves implementation and evaluation of telemedicine and electronic health record systems. Dr. Chiang received a B.S. in Electrical Engineering and Biology from Stanford University, an M.D. from Harvard Medical School and Harvard-MIT Division of Health Sciences and Technology, and an M.A. in Biomedical Informatics from Columbia University. He completed residency and pediatric ophthalmology fellowship training at the Johns Hopkins Wilmer Eye Institute.