Key Points
Question
How well does an ultra-widefield (UWF) fundus image-based deep learning system (DLS) screen for retinal lesions in rural China?
Findings
In this diagnostic study, a previously developed DLS was used to screen for 5 retinal lesions among all images of 3149 participants in rural China; the DLS achieved a mean area under the receiver operating characteristic curve of 0.918. Potential factors associated with the model’s performance included image quality, lesion proportion, and complexity of lesion composition.
Meaning
These results support consideration of a UWF fundus image-based DLS to screen for retinal lesions in patients in rural settings.
Abstract
Importance
Retinal diseases are the leading cause of irreversible blindness worldwide, and timely detection contributes to prevention of permanent vision loss, especially for patients in rural areas with limited medical resources. Deep learning systems (DLSs) based on fundus images with a 45° field of view have been extensively applied in population screening, while the feasibility of using ultra-widefield (UWF) fundus image–based DLSs to detect retinal lesions in patients in rural areas warrants exploration.
Objective
To explore the performance of a DLS for multiple retinal lesion screening using UWF fundus images from patients in rural areas.
Design, Setting, and Participants
In this diagnostic study, a previously developed DLS based on UWF fundus images was used to screen for 5 retinal lesions (retinal exudates or drusen, glaucomatous optic neuropathy, retinal hemorrhage, lattice degeneration or retinal breaks, and retinal detachment) in 24 villages of Yangxi County, China, between November 17, 2020, and March 30, 2021.
Interventions
The captured images were analyzed by the DLS and ophthalmologists.
Main Outcomes and Measures
The performance of the DLS in rural screening was compared with that of the internal validation in the previous model development stage. The image quality, lesion proportion, and complexity of lesion composition were compared between the model development stage and the rural screening stage.
Results
A total of 6222 eyes in 3149 participants (1685 women [53.5%]; mean [SD] age, 70.9 [9.1] years) were screened. The DLS achieved a mean (SD) area under the receiver operating characteristic curve (AUC) of 0.918 (0.021) (95% CI, 0.892-0.944) for detecting 5 retinal lesions in the entire data set when applied for patients in rural areas, which was lower than that reported at the model development stage (AUC, 0.998 [0.002] [95% CI, 0.995-1.000]; P < .001). Compared with the fundus images in the model development stage, the fundus images in this rural screening study had an increased frequency of poor quality (13.8% [860 of 6222] vs 0%), increased variation in lesion proportions (0.1% [6 of 6222]-36.5% [2271 of 6222] vs 14.0% [2793 of 19 891]-21.3% [3433 of 16 138]), and an increased complexity of lesion composition.
Conclusions and Relevance
This diagnostic study suggests that the DLS exhibited excellent performance using UWF fundus images as a screening tool for 5 retinal lesions in patients in a rural setting. However, poor image quality, diverse lesion proportions, and a complex set of lesions may have reduced the performance of the DLS; these factors in targeted screening scenarios should be taken into consideration in the model development stage to ensure good performance.
This diagnostic study explores the performance of a deep learning system for multiple retinal lesion screening using ultra-widefield fundus (UWF) images from patients in rural areas of China.
Introduction
Retinal diseases are the leading cause of irreversible blindness worldwide.1 Early detection and treatment of retinal abnormalities are critical to preventing permanent vision loss. Residents in rural areas are especially susceptible to retinal diseases due to a relatively low socioeconomic status,2 limited medical knowledge, and low awareness of eye care.3 However, accessibility to and availability of timely screening and treatment were found to be low in rural populations due to shortages of experienced ophthalmologists and relevant medical devices4; hence, those in the greatest need were less likely to be screened. Recently, artificial intelligence (AI) has become an appealing means to address these challenges. Deep learning systems (DLSs) with high accuracy have been developed to screen for diabetic retinopathy (DR),5 age-related macular degeneration,6 glaucomatous optic neuropathy,7 and other abnormalities using fundus images. Several DLSs have been applied in clinical settings, including the DeepMind system by Google in 11 clinics in Thailand to assist nurses in screening and referring patients with DR8 and the Comprehensive AI Retinal Expert (CARE) by our team to identify 14 retinal abnormalities, whose robust performance and adaptivity were verified in clinical settings across China.9 However, most of the current screening models for retinal disease were developed using fundus images with a 45° to 55° field of view (FOV) that focused on the posterior pole (25% of the retina area); hence, the peripheral retina, where early signs of some severe retinal diseases might first appear, was left unscreened. For example, lattice degeneration or retinal breaks in the peripheral retina are likely to be missed; without timely intervention, these lesions may progress into retinal detachment. Moreover, to observe potential lesions in the peripheral retina, pupillary dilation and repeated image capture are needed, which are time-consuming and carry the risk of adverse drug effects, making them unsuitable for large-scale population screening.
Ultra-widefield (UWF) fundus imaging has a 200° FOV that covers 80% of the retinal area in a single capture without pupillary dilation.10 We have developed a DLS that can accurately detect 5 common retinal lesions from UWF fundus images. Its performance was initially validated using external data sets from different ethnic populations and from different hospitals; its accuracy was comparable to that of ophthalmologists (area under the receiver operating characteristic curve [AUC]: 0.990 for the DLS vs 0.989 for the ophthalmologist).11,12,13,14,15 However, its effectiveness for population screening in rural areas warrants exploration before large-scale implementation. In the present study, we further applied this UWF fundus image–based DLS to screen for retinal lesions and evaluated its effectiveness in rural populations with limited medical resources. The model performance in the rural screening stage was compared with that in the previous model development stage to inform further model performance improvements.
Methods
Study Design
This was a prospective rural screening diagnostic study conducted between November 17, 2020, and March 30, 2021, in 24 villages in Yangxi, Guangdong Province, China (eTable 1 in Supplement 1). A previously developed DLS was applied to screen elderly participants in rural areas for 5 retinal lesions: retinal exudates or drusen, glaucomatous optic neuropathy, retinal hemorrhage, lattice degeneration or retinal breaks, and retinal detachment (eFigure 1 in Supplement 1). Model performance in rural screening was tested using prospectively collected UWF fundus images and compared with that in the previous model development stage. Image quality, lesion proportion, and complexity of lesion composition were also compared between the rural screening and model development stages. The overall study design is shown in eFigure 2 in Supplement 1. This study was registered with ClincalTrials.gov (NCT04859634) and was approved by the institutional review board of the Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China. All procedures followed the tenets of the Declaration of Helsinki.16 All UWF fundus images were anonymized and deidentified prior to analysis. Written informed consent was obtained from the participants. Participants would receive comprehensive ophthalmic examinations and medical advice from ophthalmologists for free if they agreed to participate in the study. The Standards for Reporting of Diagnostic Accuracy (STARD) reporting guideline was followed in this article.
Study Setting and Participants
Yangxi County roughly reflects the eye health condition of rural residents and medical resources in low-income rural areas in China.17,18 Participants in this study were enrolled from the 6-year follow-up study of the Yangxi Eye Study, which was initiated in 2014.19 Rural residents aged 50 years or older were selected by random cluster sampling from 268 geographically defined sampling units in Yangxi County. Detailed methods have been described previously.19 In the present study, all registered participants at baseline were reinvited to undergo eye disease screening at local community facilities between November 17, 2020, and March 30, 2021. Participant enrollment is shown in eFigure 2 in Supplement 1. The screening locations and the number of participants are shown in eTable 1 in Supplement 1. The baseline characteristics between participants and nonparticipants are shown in eTable 2 in Supplement 1.
DLS Architecture
A DLS was previously developed to detect 5 types of retinal lesions on UWF fundus images (eFigure 1 in Supplement 1).11,12,13,14,15,20 The DLS consists of 2 modules: an image quality control module that generates outputs of good or poor image quality (eFigure 3 in Supplement 1) and a disease screening module with 5 binary classification models generating outputs of the presence or absence of 5 lesions (eFigure 4 in Supplement 1). The DLS is deployed on a cloud-based platform, enabling image upload, AI report generation, and online review across regions.
Screening Procedure
We conducted retinal disease screenings in geographical order of villages. Examination personnel and equipment were transported from 1 village to another. Initially, we used a 45° FOV fundus camera but found a large proportion of poor-quality images (203 of 399 images [50.9%]). Therefore, we subsequently applied a UWF imaging device (Daytona [P200T]; Optos) with greater light penetration and FOV. One image centered on the macula of each eye was captured by a trained examiner up to 3 times. Throughout the screening process, UWF fundus images were captured using the same camera by the same examiner. Images were uploaded to the cloud-based platform via a 4G Wi-fi hotspot device (E5576; Huawei Device Co Ltd). Image quality was first checked by the quality control module, and the DLS would notify the examiner of poor-quality images with an alert that read, “Poor image quality.” The examiner would recapture the images and choose one with adequate quality and proceed to the disease screening module. In cases in which images in the same eye were classified as poor quality 3 times, a notice would appear that read, “Poor-quality image 3 times. Please consult an ophthalmologist.” Even if they did not consult an ophthalmologist, the examiner could choose the image with the best quality and proceed to the disease screening module. One image of each eye was used for disease screening. The DLS provided a primary AI report in real time to indicate if any of the 5 retinal lesions were present in each eye (eFigure 4 in Supplement 1). Primary AI screening results of each participant were reviewed online within a specific time period. Two ophthalmologists (S.Y. and X.Z.) with 5 years of experience with retinal diseases from a tertiary ophthalmic hospital (Zhongshan Ophthalmic Center), who had no access to the output of the DLS, annotated the 5 types of retinal lesions if present in the images according to the reference standard (eAppendix 1 in Supplement 1). Any disagreement was arbitrated by a senior retinal expert (X.L.) with more than 20 years of clinical experience (eFigure 2 in Supplement 1). They also anotated any other abnormality in the image according to a predetermined standard grading list (eTable 3 in Supplement 1) to avoid misdiagnosis. A third ophthalmologist (T.C.) integrated the grading results of the 2 ophthalmologists and the arbitration results from the retinal expert and adjusted the primary AI screening report accordingly to create a final screening report. The final screening results were sent to the participants via a short message within 1 day.
DLS Performance Validation and Comparison With the Model Development Stage
The results showing the presence or absence of 5 types of retinal lesions in the primary AI report were used to validate the DLS performance (accuracy, sensitivity, specificity, receiver operating characteristic [ROC] curve, and AUC with 95% CI) in rural screening, as measured against the revised results in the final report as the criterion standard. The performance of the DLS in rural screening was compared with that of the internal validation in the previous model development stage. The image quality, lesion proportion, and complexity of lesion composition were also compared. Image quality was represented by the proportion of poor-quality images among the total images; reasons for poor image quality were categorized. Lesion proportion was represented by the proportion of images with retinal lesions among the total images. Complexity of lesion composition was represented by the number of the 5 specific lesions that simultaneously occurred in 1 image. The model performance for groups with different image qualities, various lesion proportions, and complexity of lesion composition was also analyzed.
Statistical Analysis
The statistical analysis was performed using R, version 4.1.3 (R Group for Statistical Computing), with the pROC_1.18.0 package for model performance calculation; Python, version 3.7.2 (Python Software Foundation), with the Scikit-learn_0.22.1 package for ROC drawing; and SPSS, version 22.0 (IBM SPSS Inc) for comparison of the differences between groups. Accuracy, sensitivity, and specificity were calculated using formulas in eAppendix 2 in Supplement 1. The 95% CIs of the AUC were calculated with the bootstrap method (1000 random resamplings with replacement), and the 95% CIs of accuracy, sensitivity, and specificity of the DLS were calculated with Wilson score intervals to evaluate the DLS performance. Comparisons of AUCs between the previous model development stage and the rural screening stage were performed using the t test. The image quality and complexity of lesion composition between the model development stage and rural screening were compared using the Pearson χ2 test (or the Fisher exact test, if appropriate). Lesion proportions were compared using a 2-proportions z test with continuity correction. All tests were 2 tailed, and there was no adjustment for multiple analyses. P < .05 was considered significant.
Results
A total of 6222 UWF fundus images from 6222 eyes in 3149 participants (1685 women [53.5%] and 1464 men [46.5%]; mean [SD] age, 70.9 [9.1] years) were included. More than 1 image was taken for 2630 eyes, accounting for 42.3% of the total eyes. Baseline characteristics and the proportions of the 5 retinal lesions are shown in eTable 4 in Supplement 1. Retinal exudates or drusen had the highest proportion among the 5 retinal lesions (1461 of 3149 participants [46.4%]).
The model performance in rural screening is shown in Figure 1. The mean (SD) AUC of the 5 binary classification models for identifying 5 retinal lesions in the entire data set was 0.918 (0.021) (95% CI, 0.892-0.944). The AUCs for detecting retinal exudates or drusen, glaucomatous optic neuropathy, retinal hemorrhage, lattice degeneration or retinal breaks, and retinal detachment were 0.934 (95% CI, 0.928-0.940), 0.905 (95% CI, 0.890-0.919), 0.945 (95% CI, 0.921-0.969), 0.896 (95% CI, 0.853-0.940), and 0.910 (95% CI, 0.743-1.000), respectively. The comparisons of the model performance between the previous model development stage and the rural screening stage are shown in Table 1. The mean (SD) AUC of the model in the rural screening stage was lower than that in the model development stage (0.918 [0.021] [95% CI, 0.892-0.944] vs 0.998 [0.002] [95% CI, 0.995-1.000]; P < .001). The DLS exhibited reduced sensitivity, especially for retinal detachment, a retinal lesion with a low prevalence in rural settings. The image quality, lesion proportion, and complexity of lesion composition between the model development stage and rural screening stage were compared to enable an understanding of possible reasons for the decreased performance in rural screening.
Figure 1. Receiver Operating Characteristic Curves and Area Under the ROC Curves (AUC) of All Images, Good-Quality Images, and Poor-Quality Images.
Receiver operating characteristic curves and AUCs of good- and poor-quality images for detecting retinal detachment were not drawn or calculated because of the limited sample size, which did not support detecting trends with statistical significance.
Table 1. Model Performance Comparison Between the Model Development Stage and Rural Screening Stage.
| Retinal lesion type | AUC (95% CI) | Accuracy (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | ||||
|---|---|---|---|---|---|---|---|---|
| Model development | Rural screening | Model development | Rural screening | Model development | Rural screening | Model development | Rural screening | |
| Retinal exudates or drusen (n = 2271) | 0.994 (0.991-0.996) | 0.934 (0.928-0.940) | 0.969 (0.963-0.975) | 0.877 (0.870-0.883) | 0.942 (0.920-0.964) | 0.820 (0.808-0.832) | 0.974 (0.968-0.980) | 0.909 (0.902-0.916) |
| Glaucomatous optic neuropathy (n = 672) | 0.999 (0.998-1.000) | 0.905 (0.890-0.919) | 0.982 (0.973-0.991) | 0.883 (0.877-0.890) | 0.975 (0.947-1.000) | 0.802 (0.778-0.825) | 0.984 (0.974-0.994) | 0.893 (0.887-0.899) |
| Retinal hemorrhage (n = 106) | 0.999 (0.999-1.000) | 0.945 (0.921-0.969) | 0.993 (0.990-0.996) | 0.966 (0.962-0.969) | 0.989 (0.980-0.998) | 0.736 (0.665-0.806) | 0.994 (0.991-0.997) | 0.969 (0.966-0.973) |
| Lattice degeneration or retinal breaks (n = 81) | 0.999 (0.997-1.000) | 0.896 (0.853-0.940) | 0.991 (0.984-0.998) | 0.923 (0.918-0.929) | 0.987 (0.969-1.000) | 0.778 (0.700-0.846) | 0.992 (0.985-0.999) | 0.925 (0.920-0.930) |
| Retinal detachment (n = 6) | 1.000 (0.999-1.000) | 0.910 (0.743-1.000) | 0.995 (0.988-1.000) | 0.994 (0.993-0.996) | 0.995 (0.985-1.000) | 0.500 (0.222-0.778) | 0.995 (0.985-1.000) | 0.995 (0.993-0.996) |
Abbreviation: AUC, area under the receiver operating characteristic curve.
In the rural screening, 860 of 6222 images (13.8%) were classified by the ophthalmologists as poor quality. The most common cause of poor-quality images was opacity of refractive media caused by cataracts (714 of 860 eyes [83.0%]). Other reasons, including blepharophimosis, pterygium, and vitreous opacity, are listed in eFigure 5 in Supplement 1. In contrast, all the UWF fundus images in the model development stage were of high quality, as those with poor quality were removed before model development. Image quality was different between the model development stage (0 poor-quality images of 39815 total images [0%]) and the rural screening stage (860 poor-quality images of 6222 total images [13.8%]) (Fisher exact test, P < .001). The mean (SD) AUC for detecting 5 retinal lesions using the poor-quality images data set (0.782 [0.180]) was lower than that using the good-quality images data set (0.934 [0.023]) (eTable 5 in Supplement 1). ROCs and AUCs of good- and poor-quality images are shown in Figure 1 and eTable 5 in Supplement 1.
The lesion proportions of the model development stage and the rural screening are shown in Table 2 and eFigure 6 in Supplement 1. In the model development stage, the lesion proportions of the 5 retinal lesions for binary-classification model development ranged from 14.0% (2793 of 19 891) for retinal exudates or drusen to 21.3% (3433 of 16 138) for retinal hemorrhage. In contrast, the lesion proportions in the rural screening varied from 0.1% (6 of 6222) for retinal detachment to 36.5% (2271 of 6222) for retinal exudates or drusen. The lesion proportions of the 5 retinal lesions were different between the 2 stages (2-proportions z test with continuity correction; P < .001).
Table 2. Comparison of Lesion Proportions Between the Model Development Stage and the Rural Screening Stage.
| Retinal lesion type | Model development | Rural screening | Difference, % (95% CI) | P value | ||||
|---|---|---|---|---|---|---|---|---|
| With lesion | Total | % | With lesion | Total | % | |||
| Retinal exudates or drusen | 2793 | 19 891 | 14.0 | 2271 | 6222 | 36.5 | −22.5 (−23.8 to −21.2) | <.001 |
| Glaucomatous optic neuropathy | 834 | 5263 | 15.9 | 672 | 6222 | 10.8 | 5.1 (3.8 to 6.3) | <.001 |
| Retinal hemorrhage | 3433 | 16 138 | 21.3 | 106 | 6222 | 1.7 | 19.6 (18.9 to 20.3) | <.001 |
| Lattice degeneration or retinal breaks | 1004 | 5005 | 20.1 | 81 | 6222 | 1.3 | 18.8 (17.6 to 19.9) | <.001 |
| Retinal detachment | 2009 | 10 451 | 19.2 | 6 | 6222 | 0.1 | 19.1 (18.4 to 19.9) | <.001 |
In the present study, up to 3 types of retinal lesions appeared in the same image. A total of 45.5% of the images (2830 of 6222) had 1 or more types of specific retinal lesions. In contrast, we used images from eyes with only 1 retinal lesion labeled in the model development stage. A large proportion of fundus images from eyes without the 5 specific lesions (78.8% [8893 of 11 291] to 86.0% [12 013 of 13 985]) were used in the model development stage. The complexity of lesion composition in 1 image between model development and rural screening is shown in eFigure 7 and eTable 6 in Supplement 1, with a difference. The mean (SD) detection accuracies for the 5 lesions decreased gradually from 0.934 (0.043) to 0.743 (0.212) as the number of lesions in a single image increased. The association between complexity of lesion composition and accuracy are shown in Figure 2 and eTable 7 in Supplement 1.
Figure 2. Association of Model Accuracy and Complexity of Lesion Composition.
As complexity (ie, number of types of retinal lesions) increased, the model accuracy decreased.
Discussion
This study explored a DLS for multiple retinal lesions using UWF fundus images for patients in rural areas. The DLS exhibited relatively good performance for detecting 5 common retinal lesions. However, the model performance decreased to some extent compared with that in the model development stage. The differences in image quality, lesion proportion, and complexity of lesion composition between the model development stage and the rural screening stage might be the possible reasons for the decreased model performance.
Previous studies, such as Google Health in Thailand and the CARE application in China, have adopted DLSs based on fundus images with 45° FOV to screen retinal diseases. Although DR, glaucomatous optic neuropathy, and other retinal diseases that occur in the posterior pole can be detected by models based on a 45° FOV fundus image, some blinding retinal diseases whose early signs often appear in the peripheral retina are likely to be missed with that FOV. Therefore, a DLS trained by UWF fundus images was applied to screen for common retinal lesions to reduce missed diagnoses to the greatest extent. Furthermore, UWF fundus imaging did not require pupil dilation or repeated image capture, supporting its feasibility in large-scale eye disease screening.
Medical AI models are developed mainly for clinical application to fulfill unmet clinical needs, especially in rural areas. Recently, a few studies have investigated the incidence and prevalence of vision loss in rural areas, such as the Yangxi Eye Study18 and the Handan Eye Study21 in China and the Andhra Pradesh Eye Disease Study in India.22 They found that patients in rural areas have great health care demands but experience the consequences of a lack of health awareness and poor access to high-quality and affordable eye services. Some previous studies have been conducted to screen for retinal diseases in hospitals and specific disease cohorts.8,23 Google Health performed DR screening in Thailand among a national cohort with diabetes with a much higher DR prevalence (31.5%)8 than that of rural populations (8.2%).19 However, the applicability of DLSs for retinal disease screening in rural populations remained to be investigated. In our study, the DLS was applied for common retinal lesion screening among patients in rural areas, and the results were reviewed by ophthalmologists from tertiary ophthalmic hospitals to ensure accuracy. With this DLS, retinal disease screening can be performed by trained operators and can reduce dependence on ophthalmologists for screening.
First, although the DLS presented relatively good performance for detecting 5 common retinal lesions in patients in rural settings, the model performance decreased to some extent compared with that in the model development stage. This decrease is probably associated with the differences in image quality, lesion proportion, and complexity of lesion composition between the 2 stages. Poor image quality is frequently considered a main reason for the decreased performance of DLS in other studies.8,9,23 In our study, although the quality control module was applied to filter out poor-quality images caused by defects in image capture, unavoidable poor-quality images caused mainly by refractive media opacity still made up nearly one-sixth of the total images. The mean AUC for detecting 5 retinal lesions using poor-quality images was lower than that using good-quality images. In contrast, the images in the model development stage were well curated, and all poor-quality images were excluded. Because poor image quality could affect the lesion detection of DLS, the model output based on images with refractive media opacity should be interpreted with caution.
Second, different lesion proportions may also affect DLS performance. The proportions of 5 retinal lesions in patients in rural settings were different from those in the model development stage. In the model development stage, the lesion proportions of the 5 retinal lesions for binary classification models were designed artificially, ranging from 14.0% to 21.3%. In the rural area examined in the present study, age-related retinal diseases were more prevalent among the elderly population. Retinal exudates or drusen related to age-related macular degeneration and metabolic diseases (eg, diabetes) were the most common retinal abnormality (36.5%), while retinal detachment was not common in the rural population. We also noted that the sensitivities decreased markedly in the detection of retinal hemorrhage, lattice degeneration or retinal breaks, and retinal detachment, which had low prevalence among patients in rural screening. Also, the DLS performance still presented similar results after controlling for image quality and complexity of lesion composition (eTable 8 in Supplement 1). It is necessary to train and validate the DLS using images with disease proportions similar to the disease prevalence in the population in targeted screening scenarios.24,25
Third, complexity of lesion composition may be another factor accounting for the decreased model performance. We found that the accuracy of DLS decreased gradually as the number of retinal lesions increased in the same fundus image. Complex disease features may affect each other, and relatively minor pathologic changes can be obscured by obvious features, leading to a certain degree of misdiagnosis and missed diagnosis. Further study is needed to interpret the association between complexity of lesion composition and model performance.
Limitations
This study has several limitations. First, retinal disease screening was performed among elderly individuals in a limited number of villages within a certain area in Southern China. The capability of AI retinal disease screening in other regions and populations with diverse disease characteristics should be explored in the future. Second, cost-effectiveness, participant experience, and medical staff satisfaction were not investigated. Third, only 5 representative retinal lesions were screened in the present study. In the future, we will expand the spectrum of retinal diseases, such as pathologic myopia and macular edema.
Conclusion
This prospective diagnostic study applied a DLS to screen for 5 retinal lesions in patients in rural areas and exhibited good robustness, demonstrating the potential for DLS-based screening for vulnerable populations in remote areas with poor medical resources. Furthermore, poor image quality, diverse lesion proportions, and complexity of lesion composition could be associated with a decrease in DLS performance; these factors from targeted screening scenarios should be considered at the model development stage to ensure model performance.
eFigure 1. Normal Fundus and Five Types of Retinal Lesions Detected by the DLS
eFigure 2. Overall Study Design
eFigure 3. Online Retinal Disease Screening System Based on Ultra-Widefield Fundus Images
eFigure 4. Artificial Intelligence Retinal Diseases Screening Report
eFigure 5. Causes of Poor-Quality Images in Rural Screening
eFigure 6. Comparison of Lesion Proportions Between the Model Development Stage and Rural Screening
eFigure 7. Comparison of Complexity of Lesion Composition Between the Model Development Stage and Rural Screening Stage
eTable 1. Screening Location and Number of Participants
eTable 2. Baseline Characteristics in YES (2014) Between Participants and Non-Participants in This Study (2020)
eTable 3. Other Retinal Abnormalities Annotated in Rural Screening
eTable 4. Baseline Characteristics of Participants in 2020
eTable 5. AUCs of Images With Good and Poor Quality
eTable 6. Complexity of Lesion Composition Between Model Development Stage and Rural Screening Stage
eTable 7. Model Accuracy for Image Sets With Increasing Complexity of Lesion Composition
eTable 8. DLS Performance of Good-Quality Image Sets With Only One Retinal Lesion
eAppendix 1. Reference Standard for Image Quality and Five Types of Retinal Lesions
eAppendix 2. Accuracy, Sensitivity and Specificity Calculation
Data Sharing Statement
References
- 1.GBD 2019 Blindness and Vision Impairment Collaborators; Vision Loss Expert Group of the Global Burden of Disease Study . Causes of blindness and vision impairment in 2020 and trends over 30 years, and prevalence of avoidable blindness in relation to VISION 2020: the Right to Sight: an analysis for the Global Burden of Disease Study. Lancet Glob Health. 2021;9(2):e144-e160. doi: 10.1016/S2214-109X(20)30489-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Yan X, Chen L, Yan H. Socio-economic status, visual impairment and the mediating role of lifestyles in developed rural areas of China. PLoS One. 2019;14(4):e0215329. doi: 10.1371/journal.pone.0215329 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Liu Y, Zupan NJ, Shiyanbola OO, et al. Factors influencing patient adherence with diabetic eye screening in rural communities: a qualitative study. PLoS One. 2018;13(11):e0206742. doi: 10.1371/journal.pone.0206742 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.GBD 2019 Human Resources for Health Collaborators . Measuring the availability of human resources for health and its relationship to universal health coverage for 204 countries and territories from 1990 to 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2022;399(10341):2129-2154. doi: 10.1016/S0140-6736(22)00532-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ting DSW, Cheung CY, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211-2223. doi: 10.1001/jama.2017.18152 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Burlina PM, Joshi N, Pacheco KD, Liu TYA, Bressler NM. Assessment of deep generative models for high-resolution synthetic retinal image generation of age-related macular degeneration. JAMA Ophthalmol. 2019;137(3):258-264. doi: 10.1001/jamaophthalmol.2018.6156 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Dong L, He W, Zhang R, et al. Artificial intelligence for screening of multiple retinal and optic nerve diseases. JAMA Netw Open. 2022;5(5):e229960. doi: 10.1001/jamanetworkopen.2022.9960 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ruamviboonsuk P, Tiwari R, Sayres R, et al. Real-time diabetic retinopathy screening by deep learning in a multisite national screening programme: a prospective interventional cohort study. Lancet Digit Health. 2022;4(4):e235-e244. doi: 10.1016/S2589-7500(22)00017-6 [DOI] [PubMed] [Google Scholar]
- 9.Lin D, Xiong J, Liu C, et al. Application of Comprehensive Artificial intelligence Retinal Expert (CARE) system: a national real-world evidence study. Lancet Digit Health. 2021;3(8):e486-e495. doi: 10.1016/S2589-7500(21)00086-8 [DOI] [PubMed] [Google Scholar]
- 10.Nagiel A, Lalane RA, Sadda SR, Schwartz SD. Ultra-widefield fundus imaging: a review of clinical applications and future trends. Retina. 2016;36(4):660-678. doi: 10.1097/IAE.0000000000000937 [DOI] [PubMed] [Google Scholar]
- 11.Li Z, Guo C, Nie D, et al. Deep learning for detecting retinal detachment and discerning macular status using ultra-widefield fundus images. Commun Biol. 2020;3(1):15. doi: 10.1038/s42003-019-0730-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Li Z, Guo C, Nie D, et al. Automated detection of retinal exudates and drusen in ultra-widefield fundus images based on deep learning. Eye (Lond). 2022;36(8):1681-1686. doi: 10.1038/s41433-021-01715-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Li Z, Guo C, Lin D, et al. Deep learning for automated glaucomatous optic neuropathy detection from ultra-widefield fundus images. Br J Ophthalmol. 2021;105(11):1548-1554. doi: 10.1136/bjophthalmol-2020-317327 [DOI] [PubMed] [Google Scholar]
- 14.Li Z, Guo C, Nie D, et al. Development and evaluation of a deep learning system for screening retinal hemorrhage based on ultra-widefield fundus images. Transl Vis Sci Technol. 2020;9(2):3. doi: 10.1167/tvst.9.2.3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Li Z, Guo C, Nie D, et al. A deep learning system for identifying lattice degeneration and retinal breaks using ultra-widefield fundus images. Ann Transl Med. 2019;7(22):618. doi: 10.21037/atm.2019.11.28 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.World Medical Association . World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194. doi: 10.1001/jama.2013.281053 [DOI] [PubMed] [Google Scholar]
- 17.Report on the implementation of the 2019 plan for national economic and social development and on the 2020 draft plan for national economic and social development. Accessed August 21, 2023. https://s.wsj.net/public/resources/documents/2020%20NPC%20NDRC%20Report%20EN.pdf
- 18.Zhao X, Lin J, Yu S, et al. Incidence, causes and risk factors of vision loss in rural Southern China: 6-year follow-up of the Yangxi Eye Study. Br J Ophthalmol. 2023;107(8):1190-1196. doi: 10.1136/bjophthalmol-2021-320892 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jin G, Xiao W, Ding X, et al. Prevalence of and risk factors for diabetic retinopathy in a rural Chinese population: the Yangxi Eye Study. Invest Ophthalmol Vis Sci. 2018;59(12):5067-5073. doi: 10.1167/iovs.18-24280 [DOI] [PubMed] [Google Scholar]
- 20.Li Z, Guo C, Nie D, et al. Deep learning from “passive feeding” to “selective eating” of real-world data. NPJ Digit Med. 2020;3:143. doi: 10.1038/s41746-020-00350-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hu A, Gu SZ, Friedman DS, Cao K, Wang N. Six-year incidence and causes of low vision and blindness in a rural Chinese adult population: the Handan Eye Study. Ophthalmic Epidemiol. 2021;28(2):160-168. doi: 10.1080/09286586.2020.1795886 [DOI] [PubMed] [Google Scholar]
- 22.Khanna RC, Marmamula S, Pendri P, et al. ; Andhra Pradesh Eye Disease Study Group . Incidence, incident causes, and risk factors of visual impairment and blindness in a rural population in India: 15-year follow-up of the Andhra Pradesh Eye Disease Study. Am J Ophthalmol. 2021;223:322-332. doi: 10.1016/j.ajo.2020.09.035 [DOI] [PubMed] [Google Scholar]
- 23.Lee AY, Yanagihara RT, Lee CS, et al. Multicenter, head-to-head, real-world validation study of seven automated artificial intelligence diabetic retinopathy screening systems. Diabetes Care. 2021;44(5):1168-1175. doi: 10.2337/dc20-1877 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lin D, Chen J, Lin Z, et al. A practical model for the identification of congenital cataracts using machine learning. EBioMedicine. 2020;51:102621. doi: 10.1016/j.ebiom.2019.102621 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16(9):981-991. doi: [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eFigure 1. Normal Fundus and Five Types of Retinal Lesions Detected by the DLS
eFigure 2. Overall Study Design
eFigure 3. Online Retinal Disease Screening System Based on Ultra-Widefield Fundus Images
eFigure 4. Artificial Intelligence Retinal Diseases Screening Report
eFigure 5. Causes of Poor-Quality Images in Rural Screening
eFigure 6. Comparison of Lesion Proportions Between the Model Development Stage and Rural Screening
eFigure 7. Comparison of Complexity of Lesion Composition Between the Model Development Stage and Rural Screening Stage
eTable 1. Screening Location and Number of Participants
eTable 2. Baseline Characteristics in YES (2014) Between Participants and Non-Participants in This Study (2020)
eTable 3. Other Retinal Abnormalities Annotated in Rural Screening
eTable 4. Baseline Characteristics of Participants in 2020
eTable 5. AUCs of Images With Good and Poor Quality
eTable 6. Complexity of Lesion Composition Between Model Development Stage and Rural Screening Stage
eTable 7. Model Accuracy for Image Sets With Increasing Complexity of Lesion Composition
eTable 8. DLS Performance of Good-Quality Image Sets With Only One Retinal Lesion
eAppendix 1. Reference Standard for Image Quality and Five Types of Retinal Lesions
eAppendix 2. Accuracy, Sensitivity and Specificity Calculation
Data Sharing Statement