Applications of artificial intelligence-enabled robots and chatbots in ophthalmology: recent advances and future trends

Abstract

PURPOSE OF REVIEW

Recent advances in artificial intelligence (AI), robotics, and chatbots have brought these technologies to the forefront of medicine, particularly ophthalmology. These technologies have been applied in diagnosis, prognosis, surgical operations, and patient-specific care in ophthalmology. It is thus both timely and pertinent to assess the existing landscape, recent advances, and trajectory of trends of AI, AI-enabled robots, and chatbots in ophthalmology.

RECENT FINDINGS

Some recent developments have integrated AI enabled robotics with diagnosis, and surgical procedures in ophthalmology. More recently, large language models (LLMs) like ChatGPT have shown promise in augmenting research capabilities and diagnosing ophthalmic diseases. These developments may portend a new era of doctor-patient-machine collaboration.

SUMMERY

Ophthalmology is undergoing a revolutionary change in research, clinical practice, and surgical interventions. Ophthalmic AI-enabled robotics and chatbot technologies based on LLMs are converging to create a new era of digital ophthalmology. Collectively, these developments portend a future in which conventional ophthalmic knowledge will be seamlessly integrated with AI to improve the patient experience and enhance therapeutic outcomes.

INTRODUCTION

Technological advances have continually pushed the boundaries of healthcare, resulting in profound improvements in disease prediction, diagnosis, treatment, and surveillance. The discipline of ophthalmology, which is highly dependent on advanced technologies, has benefited significantly from the adoption of cutting edge digital and surgical innovations over past decades. Recent advances in artificial intelligence (AI) as well as the use of large language models (LLMs) seem poised to have an even greater impact on ophthalmic care globally. More specifically, the convergence between technology and eye care has culminated in the adoption of anthropomorphic robots equipped with AI, which represents a newly recognized area of research with rapid developments.

AI is a scientific and engineering technology that primarily utilizes machine learning (ML) to rapidly and more precisely discover intricate solutions while emulating some capabilities of human intelligence [1]. Scientists have also specifically utilized ML, deep learning (DL), and neural networks for modeling intelligent robots that mimic human intelligence [2, 3]. The exceptional precision demonstrated by ML, especially DL, in tasks such as image recognition has positioned this emerging technology as an important tool for medical specialties like ophthalmology that rely heavily on imaging for diagnosis and prognosis. One significant hurdle to reaching the potential of AI in healthcare is the need for thousands to millions of annotated images to train algorithms [4]. Recently, novel new techniques, such as language and vision transformers are partially overcoming these hurdles [5**].

Parallel to the rise of AI in medicine has been the development of robotics for healthcare applications [6]. Humanoid robots, easily recognizable by their human-like characteristics, are receiving more attention in healthcare research [7–11]. When paired with AI, humanoid robots can provide a more natural patient interface, thus may benefit both clinical operations and patient engagement. However, with these advancements come challenges. The dependability of AI decisions on training datasets, particularly in crucial situations, and the ethical implications of robotic interventions in healthcare are hotly debated topics [12]. Furthermore, the integration of AI with robots necessitates large investments in infrastructure, training, and maintenance [13]. Teleoperated medical robotic systems have the capability to facilitate surgical procedures via tethered and/or wireless communication networks, spanning short or long distances [14]. Telemanipulation systems for minimally invasive laparoscopic surgery like the Da Vinci system are currently in practice for laparoscopic surgery. This system offers the surgeon a realistic working environment and high-quality visualization that transmits the movements of the instrument tip within the patient in direct proportion to the surgeon’s hand movements. In ophthalmology, early iterations of experimental models for robotic surgery continue to evolve [15]. Despite these challenges, the potential benefits of AI combined with robots in ophthalmology are undeniable and their combination ability promises a future where eye care is more accurate, efficient, and accessible [16].

The capabilities of AI applications in ophthalmology can be further enhanced by the use of emerging LLMs such as ChatGPT [17*], which perform exceptionally well in comprehending and creating conversational text in ways that are convincingly human-like [18]. For instance, several LLM models such as ChatGPT have been applied to various ophthalmology problems [19**–27].

This review explores recent applications of AI-enabled robots and chatbots in ophthalmology outlining the advantages and disadvantages and providing a glimpse into the potential future promise within this arena. Figure 1 shows an overview of technologies have made significant advancements in ophthalmology.

Figure 1.

Overview of technologies that have made significant advancements in ophthalmology.

AI-ENABLED ROBOT TECHNOLOGIES IN OPHTHALMOLOGY

In recent years, AI, particularly DL [28] has made considerable advancements in medicine [29], and the application of these technologies within the realm of ophthalmology [30] has opened up ground-breaking possibilities, including diagnosing retinal disorders with accuracy similar to expert professionals that lead to the introduction of first AI-based autonomic devices in medicine [31, 32]. Among various AI technologies, AI-enabled ophthalmic robots are aimed to improve the accuracy and speed of ocular surgical procedures [33]. These robotic systems (e.g., robotic arms in surgery), integrated with DL models based on massive amount of data obtained via color fundus photography, optical coherence tomography (OCT), or other imaging modalities, may spot patterns and anomalies with an accuracy comparable and at some cases better than that of human experts [34]. For instance, in retinal procedures, where precision is of the utmost importance, robotic assistance can limit the possibility of human error, hence potentially reducing the likelihood of surgical complications [35].

AI models can improve precision of robotic surgery. For instance, new fundus reconstruction techniques have enabled real-time visualization intraoperatively primarily by precisely measuring the distance between endoscope pixels and matching fundus anatomy based on DL models. This, along with robotic control of surgical movements can help accurately reach targeted tissues while decreasing the chance for collateral damage [36]. Another example of AI enabled Robotics leading to precise interventions involves image guided subretinal injections [37]. The ability to safely and reproducibly inject fluids into potential spaces of the retina without injuring sensitive cells may only be possible with smart robotics as the tool [38, 39]. Similarly, AI-powered video analysis can identify delicate movements and properties of surgical instruments in the eye, such as vitreous. Tools’ names, positions, depths, and insertion directions are detailed in each video frame thus ML models can boost surgical accuracy and safety by performing real-time analysis of data from several sources based on underlying surgery circumstances [40]. As an example, symmetrical threshold noise reduction combiner (STIC) has been developed to address the prevalent issues of hand tremors and interference from environmental noise, which are inherent challenges in surgical procedures [41]. Another area that gained increasing interest, particularly during the COVID pandemic, was the use of telepresence robots for communication with patients so called chatbots [42]. Relying on robots to communicate with patients was generally well accepted, cost-effective, and often avoided the need for physical presence. In ophthalmology, some studies have explored the use of robotics for patient communications and education. Robotics can be deployed to navigate clinical areas, communicate with patients in multiple languages, and deliver disease specific education and complete specific task assessments [43].

Integrating AI models with robotic technologies has the potential to alter the future of ophthalmology. However, large, and high-quality datasets are essential for developing accurate and robust AI models such as DL and cognitive robots in surgery [44]. Future developments in these technologies may allow for remote surgeries with AI-driven predictive analytics through real-time data analysis which are supervised by professionals from around the world, as well as automated screening on a massive scale. This rapid expansion, however, highlights numerous concerns regarding ethical and regulatory frameworks to protect patient interests.

AI-ENABLED LARGE LANGUAGE MODELS IN OPHTHALMOLOGY

The healthcare industry is currently undergoing a transition from small-scale AI models to large-scale foundation AI models. Notably, in 2022, LLMs such as Chatbot Generative Pre-trained Transformer (ChatGPT) [45] have exhibited reasonable performance in various domains [46, 47**]. Typically, small-scale AI systems are engineered to execute precise and limited-scope operations, such as health data screening or medical image analysis [48]. On the other hand, physicians express discontent with the rigid, repetitive, and simplistic classification labels or instructions furnished by AI models. Rather, they desire access to cognitive labor that can generate innovative insights securely and cheaply [49]. Although small-scale AI systems have the potential to be beneficial in various contexts, they frequently lack interfaces that resemble those of humans and are unable to interact with or receive online human feedback. This deficiency hinders their ability to comprehend and learn intellectual tasks to the same extent as a human physician [50]. As such, LLMs have received significant attention, and their applicability is being explored extensively.

The most usable and famous large-scale language models for 2023 are ChatGPT 3.5¹, ChatGPT 4², BARD³, LlaMA⁴, Falcon⁵, Cohere⁶, PaLM⁷, and Claude⁸. The Generative Pre-Trained Transformer (GPT) is a cutting-edge language model that has advanced the state-of-the-art in natural language processing (NLP). OpenAI’s newest version of generative AI, GPT 4, is the largest language model available on the market that was reportedly trained on a trillion parameters. GPT 4 is more than just a linear improvement over GPT 3.5 when it comes to NLP in terms of a comparison of their respective performances and GPT 4 has superior text understanding and generation capabilities as well as the ability to analyze photos and videos [51]. We will thus focus more on ChatGPT research works in ophthalmology.

Several studies have investigated the applicability of ChatGPT in ophthalmology. For instance, two recent studies [19**, 24*] examined ChatGPT versions 3.5 and 4 ophthalmic capabilities based on two widely used question banks, OphthoQuestions online question bank and the Basic and Clinical Science Course (BCSC) Self-Assessment Program, and reported ChatGPT 4 performed better than ChatGPT 3.5. Moreover, research [23, 52] revealed that GPT4.0 exhibits a marked improvement over GPT-3.5 when it comes to handling ophthalmic case challenges, particularly those related to neuro-ophthalmology. To evaluate the effectiveness of ChatGPT in the context of myopia, a recent study [21*] gathered 31 common questions regarding myopia care and organized them into six distinct categories: causation; risk factors; diagnosis; clinical presentation; prognosis; treatment and prevention. The responses generated by ChatGPT were assessed by three expert pediatric ophthalmologists independently, and ChatGPT 4 demonstrated a better result in comparison to GPT-3.5. The findings emphasize the evolution and potential of AI in providing accurate information in specialized fields like ophthalmology. Another study [53] tested ChatGPT’s myopia-related abilities. For this purpose, 11 questions were asked in nine categories: general summary, symptom, cause, complication, onset, prevention, therapy, prognosis, and natural history, then, a panel of five optometry professionals from academia and research reviewed ChatGPT responses. The findings demonstrated ChatGPT’s ability to provide relevant information. To assess ChatGPT’s glaucoma diagnosis accuracy, research compared it to senior ophthalmology resident trainees [20]. A publicly available online database of case reports was used to select 11 primary and secondary glaucoma cases. Out of three ophthalmology residents who participated in this study, one had worse diagnostic accuracy while two had the same diagnostic accuracy compared to ChatGPT 3.5. The same team selected 20 case reports detailing corneal diseases such as dystrophies, infections, traumas, and degenerations from the same database. ChatGPT 4 and ChatGPT 3.5 provided 85% and 60% accuracy, respectively, while three cornea specialists correctly diagnosed 100%, 90%, and 90% of cases. These studies present novel investigations of ChatGPT capabilities for diagnosing glaucoma and corneal conditions. Another study examined ChatGPT 4’s ophthalmic triage capabilities [22] and showed that ChatGPT correctly identified triage urgency in 86% of cases while ophthalmology trainees identified 98% of cases. A recent study compared ChatGPT to five uveitis-specialized ophthalmologists [25]. The ophthalmologists had a 76%–100% success rate in 25 standard cases using the newest Uveitis Nomenclature criteria, and ChatGPT had 72% success rate. ChatGPT capabilities have been investigated in question comprehending and providing relevant responses as well. A study revealed that ChatGPT delivers responses that are 80% accurate and comprehensive in addressing the 16 most frequently asked questions about AMD, providing valuable information that individuals with AMD would find beneficial [54]. Additionally, another recent study showed the ChatGPT’s ability to handle ophthalmic discharge summaries and operative notes. ChatGPT effectively crafted ophthalmic discharge summaries [26]. Such studies highlight the fact that AI technologies like ChatGPT may function like medical specialists in specialized fields in the future if further enhancements are included. Integration of LLMs with vision models can be highly promising. Such integration can open exciting possibilities such as enhancing diagnostic accuracy in imaging, automating routine tasks in medical imaging analysis, and providing more precise and personalized care recommendations. Future variations of LMM models may analyze medical images (like retinal scans) alongside clinical data, offering a more comprehensive assessment of disease and establishing a better diagnosis, improved treatment plans, and individualized patient care.

CURRENT LIMITATIONS OF AI-ENABLED ROBOTS AND LLMs IN OPHTHALMOLOGY

Despite the remarkable advancements in AI, ML, and DL technologies, and their integration with robotics in ophthalmology, there are significant challenges and limitations. The primary issue lies in the heavy reliance on extensive, annotated datasets for training AI algorithms, which are not always readily available or diverse enough to ensure accuracy across different demographics [5*]. The ethical implications and the need for substantial investment in infrastructure, training, and maintenance are also major concerns [12]. Additionally, while humanoid robots provide a natural interface for patient interactions, integrating these systems seamlessly into existing healthcare frameworks poses a logistical challenge. Furthermore, while LLMs like ChatGPT show promise in understanding and creating human-like conversational text, their practical application in clinical settings is still in its nascent stages. One of the current limitations of the LLMs lies in their inability to interpret images. However, as these LLMs continue to evolve, there is promising potential for image interpretation capabilities to be integrated. Most importantly, ChatGPT can produce responses that seem coherent and plausible, it is important to note that it may also include factual errors, a phenomenon commonly referred to as hallucination [55]. The current models, although improving, lack the depth of understanding and contextual awareness needed to fully emulate the expertise of a human physician in complex medical fields like ophthalmology [50].

THE FUTURE: POTENTIAL DEVELOPMENT AND CHALLENGES IN OPHTHALMOLOGY

The future of ophthalmology with AI and robotics is poised for significant developments. The integration of AI with robotics is expected to enhance diagnostic accuracy, patient-specific treatment, and surgical precision, transforming the landscape of ophthalmic care [33]. The advancements in AI, particularly in DL and neural networks, are likely to yield more sophisticated models capable of precise medical interventions and early disease detection [56]. Moreover, the incorporation of LLMs, such as ChatGPT, presents an exciting frontier in patient education and AI-assisted diagnostic processes. However, this potential is tempered by challenges such as ensuring the quality and diversity of training data, addressing ethical and privacy concerns, and maintaining a balance between technological reliance and human judgment in clinical decision-making. Future developments will likely focus on creating AI models that complement human expertise, fostering a synergistic relationship for patient-centric care, while navigating the ethical and regulatory landscapes that are still evolving [47**].

CONCLUSION

Ophthalmology continues to be impacted by AI, particularly DL models. For instance, AI-enabled robotic devices have been employed in teleoperated medical robotics primarily for surgical procedures. These technologies may improve surgical accuracy and promote safety by integrating latest AI technologies trained based on massive datasets to improve patient outcomes. The convergence of AI-enabled robotic technologies and cutting-edge LLMs like ChatGPT can also have a significant impact on the future of ophthalmology. These innovations usher in a new era in patient care, with applications ranging from improved interaction with patients to aiding physicians in making more informed decisions. Due to their universal availability and ease-of-use, future variations of LLMs that can employ multi-modal data have a great potential to augment ophthalmic clinics ranging from diagnosis, prognosis, and monitoring. Despite many advantages, AI-enabled applications should be viewed as complementary rather than substitutes for the role of physicians. When AI technologies and human expertise come together, it will set a new standard for patient-centric care in the field of ophthalmology.

KEY POINTS.

• The promise of improved diagnostic precision and cutting-edge surgical procedures in ophthalmology is within reach assisted by the convergence of artificial intelligence (AI), machine learning and deep learning with robotic technology.

• ChatGPT is a large-scale language models (LLMs) that has been explored for use in the field of ophthalmology. LLMs may enhance patient education and access to information, as well as data collection and analysis by researchers.

• The future of ophthalmology may be shaped by the integration of time-honored knowledge with the most recent advances in AI, allowing for more streamlined, efficient, and effective diagnosis and therapeutic procedures thus potentially enhancing patient outcomes.