Evaluation of pediatric ophthalmologists’ perspectives of artificial intelligence in ophthalmology

Abstract

Purpose

To survey pediatric ophthalmologists on their perspectives of artificial intelligence (AI) in ophthalmology.

Methods

This is a subgroup analysis of a study previously reported. In March 2019, members of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) were recruited via the online AAPOS discussion board to voluntarily complete a Web-based survey consisting of 15 items. Survey items assessed the extent participants “agreed” or “disagreed” with statements on the perceived benefits and concerns of AI in ophthalmology. Responses were analyzed using descriptive statistics.

Results

A total of 80 pediatric ophthalmologists who are members of AAPOS completed the survey. The mean number of years since graduating residency was 21 years (range, 0–46). Overall, 91% (73/80) reported understanding the concept of AI, 70% (56/80) believed AI will improve the practice of ophthalmology, 68% (54/80) reported willingness to incorporate AI into their clinical practice, 65% (52/80) did not believe AI will replace physicians, and 71% (57/80) believed AI should be incorporated into medical school and residency curricula. However, 15% (12/80) were concerned that AI will replace physicians, 26% (21/80) believed AI will harm the patient-physician relationship, and 46% (37/80) reported concern over the diagnostic accuracy of AI.

Conclusions

Most pediatric ophthalmologists in this survey viewed the role of AI in ophthalmology positively.

Over the last few decades, artificial intelligence (AI) has demonstrated the ability to advance screening and diagnostic methods in medicine, including ophthalmology.¹ The applications of AI in ophthalmology range across subspecialties, and in particular, AI systems have been developed to address pediatric ophthalmic conditions.² Failure to address ocular disease states in a timely manner during childhood can result in significant long-term visual morbidity, impeding educational pursuit and daily function.² In order to streamline the ability to detect and treat severe disease in pediatric patients, AI systems have been designed for retinopathy of prematurity screening and monitoring, cataract and strabismus evaluation, and for prediction of visual outcomes.² Outside the realm of pediatric ophthalmology, AI has been used to evaluate glaucoma by assessing visual field, optical coherence tomography, and fundus photography findings.¹ It has been used, for example, to monitor age-related macular degeneration and diagnose diabetic retinopathy.¹ The published studies on AI programs in ophthalmology have demonstrated promising results with regard to their diagnostic performance.^2–4 The IDx-DR (IDx Technologies Inc, Coralville, IA) AI system for screening of diabetic retinopathy has been recently approved by the US Food and Drug Administration (FDA) as a device for detection of referral-warranted disease, and the i-ROP Deep Learning system for evaluation of retinopathy of prematurity has been granted breakthrough status by the FDA.^5,6 Ultimately, the field of ophthalmology is advancing toward the integration of automated tools into clinical practice.

Previous reports have evaluated physicians’ perceptions toward AI in medicine.^7–10 Although some studies have demonstrated an overall positive outlook among participants, there are reports of uncertainty regarding the impact of AI on the physician’s role.^7–10 Our group has previously reported on ophthalmologists’ views toward AI in a survey of national and regional ophthalmological society members (Al-Khaled T, et al. Invest Ophthalmol Vis Sci 2020;61: Abstract 2023). We found that the majority of participants viewed AI as a valuable clinical tool that would improve health care delivery. Pediatric ophthalmology cases may be exigent in nature and may require immediate attention in order to mitigate the risks of long-term vision loss and disability. Given the current AI technology available to address pediatric ophthalmic conditions and the high response rate from pediatric ophthalmologists in our original study, we conducted a subgroup analysis to evaluate pediatric ophthalmologists’ perceptions of AI in their field.

Participants and Methods

This study received approval from the Institutional Review Board at the University of Illinois at Chicago and conformed to the requirements of the US Health Insurance Portability and Accountability Act of 1990. This study is a subgroup analysis of a study previously conducted by the authors of this paper (Al-Khaled T, et al). Participants included in this study were those who identify as pediatric ophthalmologists and are also members of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS).

A 15-item Web-based survey including questions regarding physician demographics and perceptions of AI applications was developed. Demographic questions included gender, year of graduation from ophthalmology residency, and subspecialty training. Questions regarding the impact of AI on physicians, patients, and the field of ophthalmology were assessed using a 5-point Likert scale ranging from “strongly disagree” to “strongly agree.” The survey also included questions regarding accuracy of AI and its incorporation into medical school or residency curricula. Free text questions were included to assess perceived advantages and concerns of AI. In March 2019, the survey was made electronically available to members of AAPOS through the AAPOS discussion board. Participation was voluntary, and participants were deidentified. The survey was designed to allow only one submission per electronic device. Descriptive statistics using variable distributions, frequencies, and percentages were analyzed to characterize the overall views of the participants.

Results

A total of 80 pediatric ophthalmologists who are members of AAPOS agreed to partake in the study and completed the survey. In the original study conducted, 97 pediatric ophthalmologists participated; however, 17 were excluded from this subgroup analysis because they were not AAPOS members (Al-Khaled T, et al.). Based on the total number of active AAPOS members, the response rate was 8.6% (80/926). In this subgroup analysis study of pediatric ophthalmologists, the mean number of years since graduating residency was 21 years (range, 0–46). Of the 80 participants, 42 were male (53%). The participants’ responses to each of the survey items are shown in Figure 1.

FIG 1.

Pediatric ophthalmologists’ perceptions of artificial intelligence (n = 80). In this survey, pediatric ophthalmologists who are members of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) indicated the extent they agreed or disagreed with statements on the potential impact of artificial intelligence on ophthalmology.

Of the 80 survey participants, 73 (91%) reported understanding the concept of AI (21 [26%] strongly agreed; 52 [65%] agreed), 56 (70%) believed AI will improve the practice of ophthalmology (18 [23%] strongly agreed; 38 [48%] agreed), 54 (68%) reported willingness to incorporate AI into their clinical practice (16 [20%] strongly agreed; 38 [48%] agreed), 12 (15%) reported believing that AI will replace physicians, but 52 (65%) did not believe AI will replace physicians (37 [46%] disagreed; 15 [19%] strongly disagreed), 37 (46%) were concerned about the diagnostic accuracy of AI (7 [9%] strongly agreed; 30 [38%] agreed), 21 (26%) believed that AI will harm the physician-patient relationship (7 [9%] strongly agreed; 14 [18%] agreed), and 57 (71%) believed that AI should be incorporated into medical school and residency curricula (14 [18%] strongly agreed; 43 [54%] agreed).

Of the 22 participants who reported being “neutral” with regard to willingness to incorporate AI into clinical practice, 14 (64%) were concerned about the diagnostic accuracy of AI, 11 (50%) believed that AI will harm the patient-physician relationship, and 14 (64%) were “neutral” to the idea that AI will improve the practice of ophthalmology. There were no notable associations between participants’ willingness to incorporate AI into clinical practice and gender or year of graduation.

Advantages reported by the participants included improved efficiency, standardization in the interpretation of results, and ability to extend care to remote areas. Reported concerns included overdependence on AI, administration by nonphysicians, and financial and time costs to the physician.

Discussion

This study has several key findings: (1) most pediatric ophthalmologists in this survey believed AI will improve ophthalmology; (2) participants expressed some concerns regarding AI in ophthalmology such as replacing physicians, diagnostic accuracy, and harming the patient-physician relationship; and (3) participants believed that an AI curriculum should be developed for medical school and residency.

The first key finding is that most of the pediatric ophthalmologists who were surveyed (70%) believed that AI will enhance the field of ophthalmology. Several participants highlighted the use of AI as a supplemental tool in clinical practice as an advantage. There have been a range of applications of AI in ophthalmology, including accurately predicting a patient’s vision-related quality of life using visual acuity and visual field data, providing immediate feedback during intraoperative evaluation of ocular surface squamous neoplasia margins, determining the presence of keratoconus based on automated analysis of topography, as well identifying anterior segment pathology, such as pterygia and infectious keratitis, based on photographs.^11–14 In the field of pediatric ophthalmology, AI has been used to develop the CC-Cruiser for automated evaluation of cataracts, to predict dyslexia and postoperative ophthalmic complications in children, and to predict the time patients spend waiting in a pediatric ophthalmology clinic.^2,15,16 Despite some concerns regarding the performance of AI systems compared to experts, AI in ophthalmology has been shown to be accurate and potentially effective for various disease states.¹ For example, the i-ROP Deep Learning system that uses a vascular severity score for plus disease has demonstrated high diagnostic performance for predicting retinopathy of prematurity and is equivalent, and at times superior, to expert evaluation.^3,17,18 An objective assessment of plus disease allows for improved surveillance of disease progression and regression, as well as response to treatment.^17,18 These currently established validated systems reflect the role AI may have in facilitating diagnosis and management of diseases, particularly those subject to inter-expert discrepancy.^17,18 Given the increased role of telehealth, existing software, particularly imaged-based programs, may be adapted to incorporate AI algorithms that automatically assess alphanumeric and imaging data already being shared virtually and almost instantaneously. For example, this can be applied to currently available applications that have been used in lieu of a formal ocular examination during the COVID-19 pandemic, such as “9 Gaze” (See Vision LLC) which captures eye motility and creates a composite image that can be virtually shared with providers.¹⁹ Overall, AI may better tailor patient care by (1) validating physicians’ diagnoses, (2) providing supplemental information that can aid in formulating an assessment and plan, and (3) contributing to improved access to care and time efficiency.

Moreover, Wolf and colleagues²⁰ explored the cost-effectiveness of automated diabetic retinopathy screening in children with type 1 and type 2 diabetes mellitus. The authors found that AI was cost effective compared with traditional screening methods when there was a higher rate of patients presenting for the advised screening. Ultimately, there is a trend toward integrating automated technology into existing telemedicine platforms. Xie and colleagues²¹ found that semiautomated and deep learning platforms for telescreening of diabetic retinopathy in Singapore yielded notable reductions in cost compared to standard examination (cost per patient per year: US$62 vs US$66 vs US$72, resp.). Looking at modalities to enhance ophthalmic care of pediatric patients is especially important, and this study demonstrated that pediatric ophthalmologists are overall receptive to these new advancements that utilize AI.

The second key finding is that although most participants in this study had a positive outlook on AI, there was concern over its potential negative implications. Several participants noted that AI may lead to overdependence on automated systems and may result in decreased active clinical decision making. Future integration of AI into clinical practice may shift decision making responsibilities to the machine and reshape the standard of care, leading to downstream effects on medicolegal liability.²² Furthermore, 26% of survey respondents were concerned that AI will negatively affect the physician-patient relationship. There is apprehension that AI is unable to emulate a physician’s ability to provide a clinical assessment and establish rapport.⁹ Ethical challenges include the lack of transparency of the “black-box” algorithms, where steps of analysis and interpretation of data may not be apparent to developers themselves.^23–25 Robust datasets consisting of high quality data as input are necessary for the generation of accurate algorithms to avoid the “garbage in, garbage out” phenomenon.²⁶ It is also important to consider that it may be difficult to develop algorithms for some pediatric ocular diseases given the unique circumstances of pediatric eye conditions. In addition, there are concerns that AI systems, particularly those that rely on data extraction from the electronic health record, may inaccurately recognize disease trends among lower socioeconomic populations and perpetuate health disparities with regard to receiving quality healthcare.²⁷ Other issues raised included privacy and data protection of patients.²³ Fenech and colleagues²³ found that 49% of surveyed adults in the United Kingdom were not comfortable with their data being used to improve healthcare, but 45% believed AI should be used for diagnosis.

The third key finding is that 71% of participants believed that a curriculum in AI should be established for medical school and residency training. Of note, 68% of medical students surveyed by Pinto Dos Santos and colleagues²⁸ reported having no understanding of what AI entails. Radiology residents also reported limited knowledge of AI.⁸ Ting and colleagues²⁶ published recommendations on how to approach AI studies with an emphasis on understanding a stepwise overview of the system and evaluating the quality of the datasets and the outcomes.

Our group previously surveyed ophthalmologists from a range of subspecialties and reported on their perceptions of AI (Al-Khaled T, et al. Investig Ophthalmol Vis Sci 2020; 61: Abstract 2023). Overall, we found that approximately 75% of ophthalmologists believed AI would improve the field of ophthalmology, were interested in utilizing AI in clinical practice, and felt it would be beneficial to have an AI curriculum established for trainees. The pediatric ophthalmologists affiliated with AAPOS shared attitudes towards AI that were consistent with non-pediatric ophthalmologists’ views. Similar attitudes have also been reported by medical trainees and physicians in other fields. 73% of medical students surveyed by Pintos dos Santos and colleagues²⁸ believed AI will play an integral role in the management of patients, and 97% agreed that the physician will not be sidelined by these advancements. Most pathologists surveyed by Sarwar and colleagues⁷ and most Korean physicians and medical students surveyed by Oh and colleagues¹⁰ reported low concern over replacement by AI. In both studies, approximately half of participants believed that liability resides with the physician when an AI-automated system is used.^7,10 Moreover, many of the pathologists surveyed by Sarwar and colleagues⁷ believed that AI will allow for greater dedicated research time.

There were several limitations to this study. First, the Web-based survey was set up to allow for only one survey form submission per electronic device. Though unlikely, there is a possibility that a participant could have submitted multiple survey forms if different devices were utilized. The risk for duplicated responses was mitigated by preventing repeat submission through the same device. Second, this subgroup analysis study was directed toward pediatric ophthalmologists who are members of AAPOS, which resulted in a sample size of 80 participants. These results may lack generalizability among other pediatric ophthalmologists, particularly those who are not members of AAPOS. In addition, a range of views were collected in the free text section, providing substantial insight into pediatric ophthalmologists’ views toward AI. Future studies should be aimed at increasing the sample size by recruiting members from other pediatric ophthalmologic societies in the United States, as well as by inviting ophthalmologists who are members of societies in countries outside of the United States, as their perspectives may fundamentally differ. Third, other inherent survey limitations include selection bias or response bias. Since this study was conducted on a voluntary basis, it is possible that ophthalmologists who elected to complete the survey were either more or less informed about the topic of AI in ophthalmology compared to those who did not complete the survey. In addition, ophthalmologists who are more interested in AI may have chosen to participate in an AI-related study, which led to more overall positive findings. However, concerns regarding the implementation of AI in ophthalmology were captured in the survey, indicating that ophthalmologists who were apprehensive about AI participated in the study. Fourth, the survey items were not validated. Despite the lack of a formal validation process, questions were designed to understand the perspectives of a group of ophthalmologists. If a similar study is conducted in other populations, the survey items can be validated and adapted to account for reading level and health literacy. Finally, there was a low response rate in our study. Per AAPOS protocol at the time of the study, the survey was posted to the discussion board in order to recruit members to participate. It is possible that this delivery method yielded low participation in the study, and directly contacting members through the AAPOS directory via email or phone may have resulted in a higher response rate.

Most pediatric ophthalmologists who participated in this survey perceived AI positively and recognized its widespread potential to improve the practice of ophthalmology. Pediatric ophthalmologists’ responses were consistent with the perspectives of ophthalmologists in other subspecialties. AI may aid in delivering efficient and tailored patient care. Protection of patient data and active medical decision making must be maintained when using AI. Evaluating patient perspectives of AI in ophthalmology would be insightful in light of the reported concerns over protection of healthcare information. Furthermore, it is crucial to incorporate topics of AI into the medical school and residency curricula to educate future physicians as AI becomes integrated into medicine and ophthalmology.