A novelty route for smartphone-based artificial intelligence approach to ophthalmic screening

Abstract

Artificial intelligence (AI) has been widely applied in the medical field and achieved enormous milestones in helping specialists to make diagnosis and remedy decisions, particularly in the field of eye diseases and ophthalmic screening. With the development of AI-based systems, the enormous hardware and software resources are required for optimal performance. In reality, there are many places on the planet where such resources are highly limited. Hence, the smartphone-based AI systems can be used to provide a remote control route to quickly screen eye diseases such as diabetic-related retinopathy or diabetic macular edema. However, the performance of such mobile-based AI systems is still uncharted territory. In this article, we discuss the issues of computing resource consumption and performance of the mobile device-based AI systems and highlight recent research on the feasibility and future potential of application of the mobile device-based AI systems in telemedicine.

1. INTRODUCTION

Artificial intelligence (AI) has recently evolved into a powerful and ubiquitously used technology and an international trend. In the medical field, AI is widely applied to analyze medical images and help specialists to make diagnosis and treatment decisions. The AI-based platforms can facilitate the work of experts by reducing the burden of reading medical images and enhancing their efficiency. Currently, medical image analysis is a rapidly evolving area of medical expertise with great development potential and benefits. For example, several teams have developed useful AI models for detecting diabetic macular edema (DME) through analyzing optical coherence tomography (OCT) images.^1–3 Chan et al⁴ have developed an AI model based on AlexNet convolutional neural network (CNN), which showed a higher accuracy after preprocessing the images with denoising and cropping. Moura et al⁵ also designed an automatic identification system with high diagnostic accuracy by extracting deep features from OCT images using various CNN architectures. These studies demonstrated the possibility and achievability of the AI-based diagnostic aid in identifying DME. Although many architectures could reach the similar sensitivity of disease identification, most of them are required to be built in a high-tech computer system containing advanced graphic processing units. Besides, these programs are hard to be applied rapidly in ubiquitous locations, especially in those areas with insufficient Internet or power supply. Furthermore, with the development of the deep learning, the architectures of AI models become more complex and require heavy computing resources to support their performance. However, the mobile or portable devices could not provide so much resources to match these requirements.

2. THE INNOVATIVE ROUTE FOR MOBILE DEVICE TO APPROACH THE AI APPLICATION

In the past years, there were many studies focusing on applying AI concepts into mobile devices in ophthalmology territory. Prasanna et al⁶ used the machine learning to develop a mobile-based system for DME detection with 86% accuracy. In their system, support vector machine (SVM), adaptive boosting (AdaBoost), and logistic regression classifiers have been applied to recognize the color retinal photographs. Moreover, Bourouis et al⁷ used the method of deep learning and applied feedforward neural network to establish a mobile-based AI model for diagnosing the retinopathy with the accuracy of 87%. Their study discovered an innovative route of AI modeling for medical application and presented a potential development of smartphone-based AI platforms. As for now, the MobileNet CNN has been developed by Google for smartphone-based AI application.⁸ The significant difference between feedforward neural network and CNN is that the latter uses the convolution layers, which can implement two-dimensional (2D) convolution, thus significantly facilitating the inference of images.⁹ MobileNet-based models have been applied as the main AI architecture and have a great potential to become an ideal solution for smartphone applications with low computational cost and high speed. Howard et al¹⁰ compared the performance of MobileNet and personal computer-based VGG16 CNN using ImageNet database. It was found that MobileNet performed with only 1% less accuracy than VGG16; however, the computational cost for calculation was enormously reduced by 30 times. In the same study, the Stanford Dogs data set has been used to verify the performance of MobileNet and InceptionV3. The Stanford Dogs data set contains images of 120 breeds of dogs from around the world, built using images and annotation from ImageNet for the task of fine-grained image categorization. MobileNet not only showed the same performance of accuracy but also reduced the requirements for calculation resource by about nine times. In spite of this evidence, many researchers still face challenges with MobileNet application as it does not demonstrate satisfactory performance compared with other traditional CNN architectures such as InceptionV3 or VGG16.^11,12

As mentioned above, most of developers criticize MobileNet for being less accurate than the popular, large-capacity neural networks such as VGG16 or InceptionV3. However, Hwang et al provide evidence to illustrate the strength of MobileNet in the ability to achieve a good balance between computing resource consumption and model performance. First, their AI models showed similar results to Howard et al¹⁰ experiment, being only marginally less accurate than the conventional CNNs. In the past experience, accuracy was the most critical disadvantage of MobileNet, but the performance of the AI model in discriminating the DME from non-DME was close to that of specialists. The decision accuracy of the mobile phone AI model was 90.02%, which we believe can be good enough to be applied for quick screening. Second, Hwang et al verified the performance of MobileNet in medical application, which was not only comparable with the accuracy of the personal computer-based AI models published in the literature, but could also be implemented in a smartphone-based application program. In such a way, the AI-based mobile diagnosis system is not just a concept, but is rather a realistic prospect. Third, the demanding AI application tasks run on the mobile device are often based on the cloud computing.^6,13–15 In such a way, when user wants to use the smartphone or other mobile device to recognize an image, the program first sends it over the network to a remote server for classification, and then sends the result back to the mobile device. However, Hwang et al discovered a different route to implement the local device with offline AI diagnosis system and developed the application for smartphone to accomplish the concept of telemedicine, the application is available at https://aicl.ddns.net/DME.apk. Users can download the Android application package (APK) from the link and install it on an Android-operated phone.

3. THE FUTURE OF THE SMARTPHONE-BASED AI PLATFORM AND TELEMEDICINE

In conclusion, AI and telemedicine represent the promising trend in medical diagnostics. At the same time, the offline smartphone-based AI platforms is a novel development in the field of medicine. The major advantage of offline smartphone-based AI platforms for DME screening over previous deep learning systems is that user can upload their OCT images and get the diagnosis and suggestions in only few seconds. The benefit of the application is in providing a quick screen and medical service for people who live in remote places with scarce medical services. Furthermore, the AI diagnosis app only needs Internet when downloading, then user can use the deep learning service without a network connection, which means that the app can be used in any location. With the offline app, users are able to get prompt suggestion in seconds without Internet service. The potential of the AI-based diagnosis will be enormously enhanced, especially in remote areas or outlying islands.

ACKNOWLEDGMENTS

We thank Ying-Hsuan Wu, Hou-Kai Hu, and Feng-Yuan Yang in data collection and thank Hsin-Yi Huang served as scientific advisors of the manuscript. This study was assisted in part by Big Data Center of Taipei Veterans General Hospital. This research was funded by the Taiwan Ministry of Science and Technology (MOST-108-2314-B-010-042-MY3, MOST-108-2811-B-010-541, MOST-108-2314-B-075-055), Taipei Veterans General Hospital (CI-109-19).