Systematic review and meta-analysis of regulator-approved deep learning systems for fundus diabetic retinopathy detections

Abstract

To clarify the real-world performance of regulator-approved deep-learning (DL) systems for autonomous diabetic retinopathy (DR) screening, we systematically searched PubMed, Embase, and ClinicalTrials.gov to 3 April 2025, identifying 82 studies (887,244 examinations) covering 25 devices in 28 countries. Hierarchical bivariate meta-analysis yielded pooled sensitivity/specificity of 0.93/0.90 on a per-patient basis and 0.92/0.93 per eye, closely paralleling expert grading. Meta-regression showed that DR severity threshold, national-income level, image gradability, pupil dilation, reference standard, and diagnostic criteria collectively explained most between-study heterogeneity; any-DR screening, low-income settings, or ungradable images increased false-positive rates, whereas dilated pupils, portable cameras, and adjudicated references improved specificity. Publication bias was minimal. Overall, regulator-approved DL algorithms provide accurate, scalable DR detection, but programs must tailor deployment and reimbursement to disease threshold, image quality, and local resources, and post-market audits with standardized gradability metrics are needed to ensure safe, equitable global adoption.

Introduction

Diabetic retinopathy (DR) is one of the most common microvascular complications of diabetes mellitus and a leading cause of preventable blindness among working-age adults worldwide^1,2. A landmark meta-analysis estimated that 93 million individuals had DR in 2010, including 28 million with vision-threatening stages¹. With global diabetes prevalence projected to reach 783 million by 2045³, the number living with DR will rise commensurately. Timely identification and treatment can avert up to 90% of severe vision loss^4,5.

Regular retinal screening is, therefore, essential. Current guidelines recommend annual or biennial dilated fundus examinations for people with diabetes⁶. Yet real-world coverage remains sub-optimal: a recent meta-analysis found that only about 66% of patients in high-income countries and 39% in low- and middle-income countries receive guideline-recommended eye examinations⁷. Barriers include limited awareness, travel costs, and persistent shortages of trained graders and ophthalmologists, particularly in resource-constrained settings⁸. In the United Kingdom, the national diabetic eye screening program has contributed to substantial reductions in DR-related blindness; while DR is no longer the leading cause of certifiable blindness in working-age adults, it remains a major contributor⁹.

Advances in artificial intelligence (AI) and deep learning offer a scalable solution. Gulshan et al. first reported expert-level performance, achieving sensitivity > 90% and specificity ≈ 98% for referable DR on >100 000 images¹⁰. Subsequent systems validated in multi-ethnic cohorts¹¹ and prospective trials across Africa and Asia¹² confirm that deep learning can detect DR lesions reliably. A 2025 meta-analysis pooling 21 studies estimated pooled sensitivity of 88% and specificity of 91% for deep-learning models versus human grading¹³, suggesting that autonomous AI could markedly expand screening capacity and accessibility^11,12.

Real-world deployment, however, demands rigorous external validation and regulatory oversight. In 2018 the U.S. Food and Drug Administration (FDA) granted de-novo clearance to the first autonomous DR-screening device (IDx-DR) after a pivotal prospective trial demonstrated 87% sensitivity and 91% specificity for more-than-mild DR in primary-care settings¹⁴. Other systems—including EyeArt and SELENA+—have since obtained FDA 510(k) clearance or Conformité Européenne (CE), marking the strength of similarly robust trials^15,16. Early implementation studies show that these tools integrate smoothly into routine workflows and maintain high diagnostic accuracy at scale^16,17. Nevertheless, concerns persist about generalizability across populations, algorithmic bias, and post-market performance drift^18,19. Regulator-approval is thus a critical milestone—but not the final step—in ensuring safe and equitable adoption.

Previous reviews have largely evaluated experimental AI algorithms, many of which are not yet available for clinical use¹³. Clinicians, payers, and policymakers, therefore, lack consolidated evidence focused exclusively on regulator-cleared, commercially available AI devices. A recent review centered on a single cleared system (IDx-DR) confirmed high diagnostic performance in multiple studies²⁰, but no synthesis has yet examined the full suite of approved tools. This systematic review and meta-analysis, therefore, aim to (i) catalog every FDA-, CE-, or other authority-cleared AI system for diabetic-retinopathy detection that is currently marketed; (ii) quantify their pooled diagnostic accuracy across independent validation and real-world implementation studies; and (iii) probe study- and device-level factors—such as imaging protocol, disease prevalence, camera type, patient ethnicity, and management of ungradable images—that may influence diagnostic accuracy.

Results

Study identification and selection

Our systematic literature review, depicted in the PRISMA flowchart (Fig. 1), began with a search across PubMed, EMBASE, and ClinicalTrials.gov, yielding 8404 studies. After removing 1723 duplicates, we screened 6681 articles using EndNote, excluding 5518 for insufficient relevance. Further scrutiny of 1163 full-text articles led to additional exclusions for various reasons. Ultimately, this process resulted in selecting 82 studies^{10,11,14–16,21–97} for our meta-analysis.

Fig. 1.

PRISMA flowchart of included studies.

Overview of study characteristics

Table 1 aggregates 82 evaluation datasets of regulator-approved deep-learning systems for diabetic-retinopathy screening conducted between 2016 and 2025 in 30 countries. Roughly two-thirds of studies arose from high-income economies—most often the United States^{14–17,41,49,55,56,66,70,92,96}, Poland^27,28,42,67, Italy^{32,34,35,45,82}, the United Kingdom^46,68,81,95, Germany^57,62,76 and Australia^38,91—while one-third came from middle-income settings such as India^{21,33,59,60,69,83,84,86,92,93}, China^{24,31,39,43,51–54,58,65,71,73,78,80,85}, Thailand^22,61,89, Mexico²⁵, Chile^29,50, Turkey²⁶ and the Philippines⁴⁸, with only a single dataset from a low-income environment in Vietnam³³. Just over half of the evaluations were prospective real-world or clinical-trial designs, the remainder retrospective or mixed. Primary-care and community screening clinics provided the largest share of data (≈58%), followed by tertiary (31%) and secondary (9%) centers, with a handful of mixed-site or nationwide implementations. Adult participants with type 1 or type 2 diabetes were typically in their sixth decade of life, female representation ranged from 31% to 100%, and sample sizes spanned four orders of magnitude—from a 33-patient pilot³⁰ to a national registry of 207,228 examinations⁷³. All clinically relevant endpoints were covered (any DR, referable DR and vision-threatening DR), with 26 datasets reporting on all three thresholds^{21,24,27,28,31,33,49,50,55,59,63,67,71,73,74,76–78,81–83,85–87,90,91}, enabling comprehensive stratified performance analyses. For external validation, 36/82 studies (44%) used at least one fully external test set collected in a different country and/or ethnic population from the training data. These external cohorts spanned 22 countries across all World Bank income strata, including four lower-middle-income settings (India, Thailand, Philippines, Dominica) and one low-income cohort (Vietnam). The remaining 46 studies (56%) performed only internal or same-center testing.

Table 1.

Basic characteristics of included studies

Author	Country	Study design	Study	Study period	Healthcare setting	Population	Age	Female %	Sample size (patients)	Economic status	DR type	External validation (country)
Brant et al. (2025)21	India	Cross-sectional	Retrospective	2019–2023	Mixed	Adults with diabetes referred or self-presenting for DR screening	Mean 55.2 ± 11.9 years	49.9	4537	Middle	vtDR	No
Chotcomwongse et al. (2025)22	Thailand	Cross-sectional	Prospective	2022	Mixed	Adults with diabetes attending yearly DR screening; exclusions: already under eye-care follow-up or unable to be photographed	Mean 63.0 ± 10.6 y (AI arm); 62.0 ± 10.9 y (manual arm)	61.7	1454	Middle	rDR	Yes
Tan-Torres III et al. (2025)33	India	Cross-sectional	Prospective	2021–2022	Tertiary	Adults 18–45 y with diabetes ( ≈ 99% type 1)	Younger cohort 21.3 ± 2.3 y; older cohort 31.7 ± 4.2 y (overall 18–45 y)	50.5	321	Middle	rDR	No
Wei et al. (2025)24	China	Cross-sectional	Retrospective	2024	Tertiary	Consecutive patients (any reason for visit) undergoing routine fundus examination; exclusions: media opacity, mental disorder, pregnancy	Median 45 y (range 6 – 95)	50.4	3076	Middle	DR	No
Wroblewski et al. (2025)25	Mexico	Cross-sectional	Retrospective	2019	Primary	Adults with a known history of diabetes.	Females: mean 56.4 years (range 5–80 years) Males: mean 55.9 years (range 12–73 years)	85.5	248	Middle	DR	Yes
Doğan et al. (2024)26	Turkey	Cross-sectional	Prospective	NR	Tertiary	Adults with diabetes, no prior DR diagnosis	Mean: 58.3 years	49.11	865	Middle	rDR/vtDR	No
Grzybowski et al. (2024)27	Poland	Cross-sectional	Retrospective	2020–2021	Secondary	Adults with diabetes mellitus	NR	NR	758	High	rDR	Yes
Grzybowski et al. (2024)28	Poland	Cross-sectional	Retrospective	2020–2021	Secondary	Adults with diabetes mellitus, no previous DR diagnosis	Mean 57 years (range: 18–83 years)	58.5	695	High	DR/rDR/vtDR	Yes
Ibañez-Bruron et al. (2024)29	Chile	Cross-sectional	Retrospective	2019	Secondary	Consecutive diabetic patients attending for DR screening	Mean age: 61 years ( ± 14 years)	61	366	Middle	DR/rDR/vtDR	Yes
Karabeg et al. (2024)30	Norway	Cross-sectional	Retrospective	2017	Primary	33 women (66 eyes), self-reported diabetes mellitus (type 1 or type 2)	≥18 years	100	33	High	rDR	Yes
Li et al. (2024)31	China	Cross-sectional	Prospective	2020–2021	Primary	Adults ≥30 years with type 2 diabetes	Mean: 65 ± 12.5 years	55.78	1358	Middle	DR/rDR/vtDR	No
Musetti et al. (2024)32	Italy	Cross-sectional	Retrospective	2019	Tertiary	Adults (age > 18 years) with diabetes mellitus (Type 1 or Type 2)	Mean 65 ± 13 years	46.3	201	High	rDR	Yes
Nguyen et al. (2024)33	Vietnam	Cross-sectional	Retrospective	2021–2022	Primary	583 patients	Mean age: 61.8 years (range 16–90)	59.3	583	Low	DR/rDR/vtDR	Yes
Piatti et al. (2024)34	Italy	Cross-sectional	Prospective	2021–2023	Primary	Adults with type 1, type 2, or other diabetes (excluding gestational)	Mean: 65 years (range 18–92);	40	637	High	DR/rDR	Yes
Piatti et al. (2024)35	Italy	Cross-sectional	Prospective	2022–2023	Primary	Adults with diabetes (type 1, type 2, or other; excluding gestational)	Mean 68.02 ± 11.78 years	43	506	High	rDR	Yes
Ritto et al. (2024)36	Switzerland	Cross-sectional	Retrospective	2021–2023	Tertiary	Adults with diabetes mellitus	age ≥18 years	NR	1141	High	rDR	Yes
Šín et al. (2024)37	Czech Republic	Cross-sectional	Prospective	2022	Primary	Adults (≥18 years) with diabetes (type 1 or 2)	Mean 59.7 ± 14.2 years	48.5	1154	High	DR	Yes
Taylor et al. (2024)38	Australia	Cross-sectional	Prospective	2020–2022	Tertiary	Adults with pre-existing DR and/or AMD diagnoses	Mean: 64.0 ± 12.8 years	51	82	High	rDR	Yes
Yao et al. (2024)39	China	Cross-sectional	Prospective	2023–2024	Tertiary	474 adults (natural & diabetic cohorts) — no prior fundus-disease selection	Mean 50.04 ± 11.98 y	65.4	474	Middle	DR/rDR/vtDR	No
Chia et al. (2023)40	Austria	Cross-sectional	Retrospective	2013–2020	Primary	Indigenous Australians with diabetes	Mean 54.9 years (SD ± 15.0)	52.4	864	High	rDR/vtDR	Yes
Dow et al. (2023)41	USA	Cohort	Retrospective	NR	Mixed	Adults aged ≥18 years with diabetes, without prior DR diagnosis or exam within the past year	NR	NR	790	High	rDR	No
Grzybowski et al. (2023)42	Poland	Cross-sectional	Retrospective	2020–2021	Secondary	Adults with diabetes mellitus (Type 1 or Type 2)	NR	NR	811	High	rDR	Yes
He et al. (2023)43	China	Cross-sectional	Prospective	2021–2022	Tertiary	Adults > 18 y with/without DR, AMD or GON who agreed to double imaging	NR	NR	504	Middle	rDR	No
Kemp et al. (2023)44	Dominica	Cross-sectional	Prospective	2021	Primary	Adults with diabetes ≥18 years	Mean: 64 years (range 26–94)	72.6	587	Middle	rDR	No
Lim et al. (2023)17	USA	Cross-sectional	Prospective	2017–2018	Tertiary	Adults aged ≥18 years with diabetes mellitus	71% aged <65 years, 29% aged ≥65 years	53.2	521	High	rDR/vtDR	No
Lupidi et al. (2023)45	Italy	Cross-sectional	Prospective	2022	Primary	Adults >18 years; both diabetic and non-diabetic	Mean: 60.1 ± 16.2 years	NR	251	High	DR	Yes
Meredith et al. (2023)46	United Kingdom	Cross-sectional	Retrospective	2021	Primary	Patients attending routine diabetic eye screening	Mean: 62 years (range: 12–100)	46.6	9817	High	DR/rDR	Yes
Roubelat et al. (2023)47	Scotland	Cross-sectional	Retrospective	2006–2016	Primary	All people with diabetes ≥12 years invited for screening; single 45° macula-centered image per eye	NR	NR	179944	High	DR/rDR/vtDR	No
Salongcay et al. (2023)48	Philippines	Cross-sectional	Prospective	2021–2022	Primary	Adults ≥18 years with type 1 or 2 diabetes	Mean: 59.1 ± 10.4 years	64.4	2793	Middle	rDR/vtDR	Yes
Vought et al. (2023)49	USA	Cross-sectional	Retrospective	2022	Primary	Adults (age >18) with and without known diabetes	Mean age: 64 years	55	62	High	rDR/vtDR	No
Arenas-Cavalli et al. (2022)50	Chile	Cross-sectional	Retrospective	2014–2016	Primary	Adults with diabetes enrolled in the national DR program	Mean: 63 years (SD: 12.7)	59.8	1123	Middle	DR	No
Dong et al. (2022)51	China	Cross-sectional	Prospective	2019–2021	Primary	Adults (≥18 years) with type 1 or type 2 diabetes	Mean: 52.09 ± 11.51 years	36.12	443	Middle	DR/rDR/vtDR	No
Han et al. (2022)52	China	Cross-sectional	Prospective	2018, 2020	Mixed	Adults aged 23–86 years (community: healthy; hospital: varied diseases)	Median: 63 (PUMCH); 27 (NSDE)	NR	300	Middle	rDR	No
Hao et al. (2022)53	China	Cross-sectional	Prospective	2021	Primary	Adults with diabetes (type 1 or 2), aged ≥18 years	Mean: 63.03 ± 8.72 years	65.9	3796	Middle	DR/rDR	No
Li et al. (2022)54	China	Cross-sectional	Prospective	2020	Primary	Adults ≥18 years able to cooperate with fundus photography	Mean: 53.2 ± 15.6 years	43.2	1585	Middle	rDR	No
Mehra et al. (2022)55	USA	Cross-sectional	Retrospective	2019–2021	Primary	Adult patients with DM	age ≥18 years	40	1052	High	rDR	No
Mokhashi et al. (2022)56	USA	Cross-sectional	Retrospective	2019	Primary	Adults (age ≥18) with diabetes mellitus	Mean age: 60.7 years (range 21–90 years)	60.8	260	High	rDR	No
Paul et al. (2022)57	Germany	Cross-sectional	Prospective	2020	Tertiary	Patients with diabetes mellitus (Type 1 and Type 2)	Mean: 54.2 years (range: 9–92 years)	42	503	High	DR/rDR/vtDR	Yes
Pei et al. (2022)58	China	Cross-sectional	Retrospective	2018–2020	Tertiary	Adults with type 2 diabetes mellitus (T2DM)	Mean: 61.2 ± 11.8 years	50.5	549	Middle	DR	No
Rao et al. (2022)59	India	Cross-sectional	Retrospective	2015–2016	Tertiary	Adults with diabetes, aged >21 years	Mean: 54.1 ± 8.3 years	35	135	Middle	DR/rDR/vtDR	No
Rio et al. (2022)60	India	Cross-sectional	Prospective	NR	Primary	Adults with known or screen-detected diabetes	Mean: 57.7 years (SD: 11.1)	52.2	11199	Middle	rDR	Yes
Ruamviboonsuk et al. (2022)61	Thailand	Cohort	Prospective	2018–2020	Primary	Adults (≥18) with diabetes, listed on national registry	Mean: 59.9 ± 10.3 years	63.1	7651	Middle	vtDR	Yes
Skevas et al. (2022)62	Germany	Cross-sectional	Prospective	2020	Tertiary	Adults ≥18 years, not pre-selected for DR/AMD	NR	NR	630	High	rDR	Yes
Sedova et al. (2022)63	Austria	Cross-sectional	Prospective	NR	Tertiary	Adults with diabetes mellitus (Type 1 or Type 2), no previous DR diagnosis	Mean 55 ± 15.5 years	38.9	54	High	rDR	No
Tsai et al. (2022)64	Taiwan	Cross-sectional	Retrospective	NR	Primary	Adults with diabetes (no specific type given)	NR	NR	1407	High	rDR	No
Yang et al. (2022)65	China	Cross-sectional	Prospective	2018–2019	Tertiary	Adults ≥ 18 y with type 1 or type 2 diabetes mellitus (one eye/participant)	Median 60.6 y (range 20.2 – 85.8)	51.8	1001	Middle	rDR	No
Bhuiyan et al. (2021)66	USA	Cross-sectional	Prospective	2019–2020	Primary	Adults ≥50 years; diabetic and non-diabetic (mixed) patients	Over 50 years	NR	340	High	rDR	No
Grzybowski et al. (2021)67	Poland	Cross-sectional	Retrospective	2011–2012	Secondary	Diabetic patients who visited a community-based secondary care	Adults with diabetes mellitus ( ≥ 18 years old)	NR	170	High	rDR	Yes
Heydon et al. (2021)68	United Kingdom	Cross-sectional	Prospective	2017–2018	Primary	All patients who attended routine diabetic eye screening in these programs, with no additional editing or selection of images	60/66/61 mean	46/43/45.6	30405	High	rDR	Yes
Ipp et al. (2021)15	USA	Cross-sectional	Prospective	2017–2018	Primary, Tertiary	Adults (≥18 years) with diabetes (type 1 or type 2)	Mean 53.9 ± 15.2 years	49	893	High	rDR	No
Jain et al. (2021)69	India	Cross-sectional	Prospective	2018–2019	Primary	Adults with diabetes visiting public clinics	Mean: 54.9 ± 10.4 years	46.87	1294	Middle	DR/rDR	No
Kim et al. (2021)70	USA	Cross-sectional	Prospective	2016–2017	Tertiary	Patients with diabetes	Mean 57.0 ± 15.7 years	37.7	69	High	rDR	No
Li et al. (2021)71	China	Cross-sectional	Prospective	2018–2019	Tertiary	Adults with diabetes (Type 1 and Type 2)	Mean: 50 ± 12 years	31.6	1147	Middle	rDR	No
Li et al. (2021)79	Taiwan	Cross-sectional	Prospective	2019	Tertiary	Adults with type 2 diabetes	NR	NR	468	High	rDR	No
Lin et al. (2021)73	China	Cross-sectional	Prospective	2018–2019	Mixed	General Chinese population undergoing fundus screening	NR	47	207228	Middle	rDR	No
Ming et al. (2021)74	China	Cross-sectional	Prospective	2018–2019	Primary	Adults with diabetes aged ≥40 years, no prior DR diagnosis	Mean: 69.3 ± 9.0 years	54.3	173	Middle	DR/rDR	No
Shah et al. (2021)75	Spain	Cross-sectional	Prospective	2011–2012	Primary	Adults with diabetes mellitus (Type 1 and Type 2)	Medianage = 74 years	55	2680	High	rDR	Yes
Wintergerst et al. (2021)76	Germany	Cross-sectional	Prospective	2018	Primary	Patients with DM,	mean ~69	NR	72	High	rDR	Yes
Gonzalez-Gonzalo et al. (2020)77	Netherlands, Spain, Sweden, Denmark	Cross-sectional	Retrospective	2011–2016	Tertiary	Adults undergoing fundus imaging (with or without DR/AMD)	NR	NR	600	High	rDR	No
He et al. (2020)78	China	Cross-sectional	Prospective	2018	Primary	Adults with diabetes (Type 2 or Type 1), aged ≥18 years	Mean: 68.5 ± 7.2 years	53	889	Middle	DR/rDR	No
Hsieh et al. (2020)79	Taiwan	Cross-sectional	Retrospective	2007–2017	Tertiary	Adults with diabetes undergoing DR screening	NR	NR	7524	High	DR/rDR	No
Li et al. (2020)80	China	Cross-sectional	Prospective	2018	Primary	Adults with diabetes (diagnosed by fasting glucose ≥7.0 mmol/L, RBG ≥ 11.1 mmol/L, or on diabetes treatment)	Mean: 58.3 ± 10.6 years	59.8	8005	Middle	DR/rDR	No
Olvera-barrios et al. (2020)81	United Kingdom	Cross-sectional	Prospective	NR	Primary	Final human grade of standard NDESP images	NR	NR	1257	High	DR/rDR/vtDR	Yes
Sarao et al. (2020)82	Italy	Cross-sectional	Prospective	NR	Tertiary	diabetic participants (type 1 or 2)	Mean ≈ 64.7 ± 14	47	165	High	rDR	Yes
Sosale et al. (2020)83	India	Cross-sectional	Prospective	2018	Tertiary	Adults with type 1, type 2, or secondary diabetes	Mean: 55 ± 11 years	42	297	Middle	DR/rDR	No
Sosale et al. (2020)84	India	Cross-sectional	Prospective	2018	Tertiary	Adults (≥18 years) with type 1 or type 2 diabetes	Mean: 55 ± 11 years	42	900	Middle	DR/rDR	No
Zhang et al. (2020)85	China	Cross-sectional	Prospective	2018–2019	Primary	Adults ≥18 years with diabetes (mostly type 2, ~98%)	Mean: 54.3 ± 11.6 years	42.6	15805	Middle	DR/rDR/vtDR	No
Bhaskaranand et al. (2019)16	United States	Cohort	Retrospective	2014–2015	Primary	Adults with diabetes mellitus	NR	NR	107001	High	rDR	No
Gulshan et al. (2019)86	India	Cross-sectional	Prospective	2016–2017	Tertiary	Adults >40 years with known diabetes	Aravind: 56.6 ± 9.0; Sankara: 56.0 ± 10.0 years	41.9/32.8	3049	Middle	rDR	No
Natarajan et al. (2019)87	India	Cross-sectional	Prospective	2018	Primary	Adults with diabetes visiting government clinics	NR	51.6	213	Middle	DR/rDR	No
Quellec et al. (2019)88	France	Cross-sectional	Retrospective	2004–2017	Mixed	Routine DR screening patients in OPHDIAT	NR	NR	164660	High	rDR	Yes
Ruamviboonsuk et al. (2019)89	Thailand	Cross-sectional	Retrospective	2015–2017	Primary	Adults with diabetes (random sample from national registry)	Mean: 61.1 ± 10.96 years	67.5	7515	Middle	rDR	Yes
Verbraak et al. (2019)90	Netherlands	Cohort Study	Retrospective	2015	Primary	Adults with Type 2 diabetes mellitus	Mean age: 63 years (SD 11.3)	47	1616	High	rDR	Yes
Abràmoff et al. (2018)14	USA	Cross-sectional	Prospective	2017	Primary	Adults aged ≥22 years with diabetes, no known diabetic retinopathy	Median age: 59 years (range 22–84 years)	52.5	819	High	rDR	No
Keel et al. (2018)91	Australia	Cross-sectional	Prospective	2017	Tertiary	Adults ≥18 with diabetes	Mean: 44.3 ± 16.6 years (range 20–90)	43	96	High	rDR	No
Krause et al. (2018)92	USA, India	Cross-sectional	Retrospective	2015–2016	Mixed	Adults with diabetes	mean 54.9 ± 10.9 years	60.5	1958	High, Middle	rDR	No
Rajalakshmi et al. (2018)93	India	Cross-sectional	Prospective	NR	Tertiary	Adults (≥18 years) with type 2 diabetes (T2DM)	≥18	NR	301	Middle	DR/rDR/vtDR	Yes
van et al. (2018)94	Netherlands	Cross-sectional	Retrospective	NR	Primary	Adults with Type 2 diabetes	Mean 65 ± 11.9 years	43.9	1415	High	rDR	Yes
Ting et al. (2017)11	Singapore, China, USA, Australia, Mexico, Hong Kong	Cross-sectional	Retrospective	2010–2015	Primary	Adults with diabetes mellitus	Mean age in primary validation set: 60.2 years	45.4	38256	High, Middle	rDR	Yes
Tufail et al. (2017)95	United Kingdom	Cross-sectional	Prospective	2012–2013	Primary	Adults with known diabetes mellitus (any type), attending their annual DR screening	Median 60 years (range 10–98	NR	20258	High	rDR	No
Abràmoff et al. (2016)96	USA, France	Cross-sectional	Retrospective	NR	Secondary	Adults with diabetes mellitus	Mean 57.6 years ( ± 15.9 years)	43	874	High	rDR	Yes
Gulshan et al. (2016)10	USA, India	Cross-sectional	Retrospective	2015–2016	Mixed	Adults with diabetes	EyePACS-1: mean 54.4; Messidor-2: mean 57.6	EyePACS-1: 62.2%; Messidor-2: 42.6%	9963/1748	High, Middle	rDR	No

DR diabetic retinopathy, rDR referable diabetic retinopathy, vtDR vision-threatening diabetic retinopathy, T1DM type 1 diabetes mellitus, T2DM type 2 diabetes mellitus, DM diabetes mellitus, AMD age-related macular degeneration, SD standard deviation, NR not reported.

*A study may appear in more than one panel when it reports distinct 2 × 2 tables for different disease thresholds or datasets—counts remain mutually exclusive and weights are calculated within each panel only.

Comparative overview of deep learning algorithms in diagnostic studies

Table S4 synthesizes the methodological foundations of the 82 diagnostic-accuracy studies of regulator-approved deep-learning screeners for diabetic retinopathy. Human-graded color fundus photography remains the gold standard: approximately 75 datasets (91%) used double-masked or consensus reads by retina specialists, with a third expert arbitrating any discordance. Clinically anchored reference standards are rarer: only about one study in eight verified ground truth through in-person dilated biomicroscopy or optical coherence tomography (OCT)-aided examination^{25,29,41,45,58,59,70}. Image acquisition is remarkably homogeneous—most studies relied exclusively on 45° desktop fundus cameras—yet 12 incorporated wide-field, ultra-widefield or OCT inputs to probe peripheral or macular disease missed by single fields^{14,25,28,32,36,38,57,59,62,63,69,70,76,83,93}. Pupil-management strategies varied: 15 datasets were fully mydriatic^{25,29,33,44,48,59,68–70,75,81–83,87,93}, 39 strictly non-mydriatic^{24,27,28,30,31,33,34,37,40–43,45,46,49,51–54,56,57,60,62–64,66,67,71–74,77–80,84,86,91,94}, and 26 adopted “dilate-if-needed” contingency protocols^{10,11,14,15,17,21,22,26,32,35,36,38,39,47,50,55,58,61,65,76,88–90,92,95,96}. Single-field imaging predominated (≈60%), while two-field schemes were routine in hospital-based or smartphone studies and ≥3-field/montage approaches appeared in 12 investigations^{16,25,28,57,59,62,69,70,76,83,87,93}. Vendor participation was declared in 30 datasets (37%), chiefly for IDx-DR, EyeArt, ARDA and RetCAD^{10,14,16,21,22,30,33,37,40,43,50,52,54,59,61,62,64,66,71,73,75,85,86,88–90,92,94–96}, whereas independent validations—particularly of Medios, VeriSee, CARE and smartphone implementations—reported no developer involvement^{25,42,44,49,69,72,79,83,84,87}. Collectively, the evidence demonstrates that cleared DL algorithms have been benchmarked across a spectrum of camera platforms, dilation protocols and adjudication workflows, yet still anchor their performance against rigorous expert interpretation.

Regulatory clearances and underlying deep-learning architectures of AI-based diabetic-retinopathy screening systems

Table 2 maps the current regulatory and technical landscape for the 25 leading commercial DL algorithms for diabetic retinopathy. EyeArt (510(k) K200667/K223357 + EU-MDR Class IIb) is the only product that holds both FDA clearance and an EU-MDR Class IIb certificate, making it the most broadly approved solution worldwide. IDx-DR (De Novo DEN180001 + successive 510(k) updates) holds FDA clearance and CE-MDD Class IIa certification. Europe’s CE roster has expanded to include Medios AI™, LuxIA, and Aireen® under EU-MDR Class IIb, plus RetCAD™, EyeCheckup, VISUHEALTH-AI DR, iPredict®, Retmarker, OphtAI, and the VGG-based SELENA + ™ under MDR/MDD Class IIa certifications, as well as Autograder (CE Class I, deployed in NHS Scotland DES since 2011). Additional international clearances include Health Canada approvals for IDx-DR, EyeArt, and OphtAI, and Australia TGA certification for Retmarker and iPredict®. Asian regulators have been equally active: China’s NMPA has issued Class III licenses for CARE, EyeWisdom, VoxelCloud Retina, AIDRScreening, Smart Eye Sugar Net, and AI-100, while Taiwan’s TFDA has cleared VeriSee DR™, and Singapore’s HSA has approved Medios AI™ and SELENA + ™; Malaysia (MDA), Thailand (Thai FDA), and Saudi Arabia (SFDA) have each granted additional market authorizations.

Table 2.

Regulatory clearances and underlying deep-learning architectures of AI-based diabetic-retinopathy screening systems

AI system	Vendor/Manufacturer	Key regulatory clearances	DL architecture	Training Data (Countries/Ethnicities)	Primary Retinal Disease(s) Detected/Indication
ARDA® DL	Verily (Google Health), USA	CE-MDD IIa (2018)	10 × Inception-v3 ensemble	≈130k EyePACS (USA, Hispanic/White/Black/Asian) + 3 Indian hospitals (South Asian)	DR incl. DME
EyeArt®	Eyenuk, Inc., USA	FDA 510(k): K200667 (2020, EyeArt) & K223357 (2023, EyeArt v2.2.0) · EU-MDR IIb (2023) · Health Canada MDL 104401 (2021)	multi-CNN ensemble	>78k EyePACS (USA; ~55% Hispanic, White, Black, Asian)	>mild DR, vtDR; also AMD, GON
Medios AI™	Remidio Innovative Solutions, India	EU-MDR IIb (2023) · Singapore HSA Class B (2022) · India CDSCO license (2024)	2 × Inception-v3 + MobileNet QC	50k: EyePACS + proprietary Indian smartphone images	Referable DR (mobile/desktop)
LumineticsCore®(IDx-DR)	Digital Diagnostics Inc., USA	FDA De Novo: DEN180001 (2018 IDx-DR) + 510(k): K213037 (2021 IDX-DR) & K203669 (2022 IDX-DR v2.3) · CE-MDD IIa · Health Canada MDL 104238 (2021)	5-CNN Inception-like ensemble	US + Indian data; pivotal trial (60% White, 28% Black, 12% Hispanic)	Autonomous >mild DR ± DME
RetCAD™	Thirona / Delft Imaging, Netherlands	CE-MDR IIa (NB 0344, 2022)	5-CNN deep-conv ensemble	Messidor (France) + Retisalud (Spain); European ancestry	DR + AMD
Retmarker	Retmarker SA (formerly Critical Health), Portugal	CE‑MDD Class IIa (2010); manufacturer notes TGA (Australia) approval	deep-learning classifier for DR screening	DL variant training data not publicly specified.	DR screening (disease/no disease) + MA‑turnover biomarker for progression/DME risk
EyeCheckup	EyeCheckup Ltd / URAL Telekom, Türkiye	CE-MDR IIa (2023)	single Inception-v4-style CNN	Turkish clinic data; presumed Anatolian ethnicity	DR, AMD, glaucoma + other lesions
LuxIA	RetinAI & Fundación Ver Salud (Ikerian AG), Switzerland/Spain	EU-MDR Class IIb certificate (May 2025)	EfficientNet-based CNN (single) (Modern Retina, PubMed)	Retisalud (Canary Islands); Spanish/N-African admixture	>mild DR
EyeWisdom®	Visionary Intelligence Ltd, China	China NMPA Class III SaMD (CQD2200108, 2024)	ResNet / EfficientNet mix (ensemble)	Chinese hospital images; likely majority Han Chinese	DR (all grades)
Aireen®	Aireen Health s.r.o., Czech Rep.	EU-MDR IIb (2023)	single EfficientNet-V2 CNN	Czech cohort: training data not disclosed	DR screening (Czech setting)
SELENA + ™	EyRIS Pte Ltd, Singapore	CE-MDD IIa · Singapore HSA Class B · Malaysia MDA Class B	8 × VGG-19 family ensemble	274k Singaporean (Chinese, Malay, Indian) + 200k global (multi-ethnic)	DR, AMD, GON (tri-disease)
DART	Chile MoH screening program	Chile MoH adoption (2018); clinical validation accepted 2020	two fully convolutional networks	Chile national program; Hispanic/Mapuche admixture	DR (national program)
CARE	Beijing Airdoc Tech, China	China NMPA Class III SaMD (2020)	single Inception-ResNet-v2 CNN	>400k from 301 Chinese clinics; majority Han Chinese	DR (auxiliary diagnosis)
VeriSee DR™	Acer Medical Inc., Taiwan	TFDA Class II · Thai FDA Class 2 · Philippines CMDR 2024-02319	modified Inception-v4 CNN	7.5k from Taiwan (Han Chinese) + EyePACS	Referable DR; “VeriSee AMD” in pilot
VISUHEALTH-AI DR	VisuHealth Medical Tech Co., China	CE-MDR IIa (Rule 11 SaMD)	single Inception-v3 CNN	Likely Chinese multi-center dataset	DR screening
VoxelCloud Retina	VoxelCloud Inc., China/USA	China NMPA Class III · Saudi SFDA (2024)	6 × Inception-ResNet-v2 + FPN	15k+ patients from 155 China clinics (multi-regional Han)	DR, DME, AMD, glaucoma
Smart Eye Sugar Net	Shenzhen Silicon Intelligence Tech., China	China NMPA Class III SaMD (2020)	single proprietary CNN	Not disclosed; presumed Chinese data	DR (NMPA Class III)
iPredict® (DR module)	iHealthScreen Inc., USA	CE-MDR IIa (2021) · TGA IIa (2022) · UAE MoH (2022) FDA 510(k) pending	single CNN (undisclosed)	US urban datasets: ethnicity not published	DR; also AMD, GON (regional)
BigVision MIAS 2000	BigVision Medical Tech Co., China	China Class II SaMD (2018) – assistive	single proprietary CNN	No public info	Assistive DR, AMD, glaucoma (18 lesions)
AIDRScreening	Shenzhen SiBright Co. Ltd, China	China NMPA Class III (innovative device, 2019)	3-CNN ensemble	>40k Chinese images; ethnic details not given	Referable DR only
Eyetelligence Assure™	Eyetelligence Pty Ltd, Australia	CE-MDD Class I (2019) · Australia ARTG Class I (IDs 325343, 355170)	single Inception-v4 derivative	EyePACS, Messidor + Melbourne data (Caucasian & Indigenous Australians)	DR, AMD, glaucoma, CV risk markers
AI-100 algorithm	Baidu Intelligent Healthcare Unit, China	China NMPA Class III SaMD (2022) — DR + RMD + GCS	DenseNet-121 backbone with bilinear-pooling & quality-assessment modules	>700k images from 179k Chinese patients (multi-regional Han)	DR, RMD, GON
Huiyan AI (SG-DR System)	Shang Gong Medical Info Tech Co., China	Guangdong MPA Class II SaMD (May 2022)	proprietary CNN (undisclosed)	Not disclosed; presumed Chinese community data	DR (regulatory indication); details not published
OphtAI	Evolucare & ADCIS (France)	CE Class IIa (EU); Health Canada; UKCA (per vendor)	CNN‑based (proprietary) with heat‑map lesion localization	AP‑HP Ophdiat tele‑screening network (France) as primary data source (per vendor)	DR (severity grading), AMD, Glaucoma
Autograder	Medalytix / EMIS Health, UK (NHS Scotland DES)	CE Class I (MDD); routine use in Scottish DES since 2011	Dual deep neural networks using ResNet-101 architecture	Scottish DES datasets; validation on 78,601 images / 33,535 patients	DR triage: “no DR & gradable” vs “manual grading required”; ungradable detection; DLAG aims to further reduce manual grading

AI artificial intelligence, CE Conformité Européenne, CDSCO Central Drugs Standard Control Organization (India), CNN convolutional neural network, DL deep learning, FDA Food and Drug Administration (USA), HSA Health Sciences Authority (Singapore), MDD Medical Device Directive, MDR Medical Device Regulation (EU), ML machine learning, NMPA National Medical Products Administration (China), SaMD software as a medical device, SFDA Saudi Food and Drug Authority, TFDA Taiwan Food and Drug Administration, TGA Therapeutic Goods Administration (Australia).

Technically, the ecosystem remains dominated by Inception-family backbones (v3, v4, Inception-ResNet-v2) deployed either as single models (EyeCheckup, CARE, VeriSee DR, VISUHEALTH-AI DR) or as multi-CNN ensembles (IDx-DR with 5-CNN, EyeArt multi-CNN, ARDA® DL with 10× Inception-v3, RetCAD™ with 5-CNN, AIDRScreening with 3-CNN). Newer products increasingly adopt EfficientNet (Aireen®, LuxIA, EyeWisdom) or EfficientNet-V2 hybrids for improved parameter-efficiency, whereas Medios AI™ employs a MobileNet-lite branch to support on-device smartphone inference. Alternative strategies include DenseNet-121 + bilinear pooling (AI-100), VGG-19 ensemble (SELENA + ™ with 8× VGG-19), ResNet-101 dual networks (Autograder), feature-pyramid ensembles (VoxelCloud Retina with 6× Inception-ResNet-v2 + FPN), and proprietary CNN architectures with heat-map lesion localization (OphtAI). Retmarker employs a deep-learning classifier with MA-turnover biomarker analysis for progression and DME risk assessment. In short, although cleared screeners now span a broad geography and a widening range of regulators, most still converge on convolutional backbones from the Inception/EfficientNet families, often boosted by ensemble voting to maximize robustness across diverse imaging hardware and care settings.

Quality assessment

According to QUADAS-2 criteria, 56.1% of studies were rated low risk of bias for patient selection, 57.3% for the index test, 52.4% for the reference standard, and 41.5% for flow and timing. Flow and timing remained a frequent issue, with 17.1% of studies showing high risk—often due to substantial image exclusion or delays between clinical evaluation and AI testing. Unclear risk of bias was also common, particularly for the index test (42.7%) and patient selection (41.5%), with the reference standard showing 37.8% unclear ratings. Applicability concerns were generally minor: 76.8% of studies had low concern for patient spectrum, 92.7% for the index test, and 52.4% for the reference standard. However, 45.1% had unclear applicability for the reference standard, reflecting inconsistency with ETDRS or Wisconsin Reading Center protocols. In summary, most evaluations were methodologically sound and clinically representative, though approximately 1 in 6 exhibited high risk in either flow/timing (17.1%) or reference standard (9.8%) domains. These results underscore the need for prospectively registered, fully masked diagnostic-accuracy studies (Supplementary Fig. 1; Table 5).

Overall meta-analysis of patient-wise regulator-approved deep learning system

The hierarchical random-effects bivariate model revealed high between-study heterogeneity, particularly for specificity (τ² = 1.02; I² = 86.81%) and, to a lesser extent, for sensitivity (τ² = 0.80; I² = 31.77%). The negative covariance (–0.31) and correlation (ρ = –0.34) indicate the expected trade-off: studies reporting higher sensitivity tended to show lower specificity and vice versa. A highly significant likelihood-ratio test (χ² = 29,231, df = 3, p < 0.001) confirms that the random-effects (RE) specification fits the data far better than a fixed-effects (FE) approach, validating the choice to model heterogeneity explicitly.

After multivariable adjustment in a bivariate random-effects meta-regression, the pooled summary absolute accuracy of 115 tables of regulator-approved AI screeners remained high: summary sensitivity 0.93 (95% CI 0.91–0.95) and summary specificity 0.90 (95% CI 0.87–0.92) (Fig. 2a, Supplementary Fig. 2). These figures suggest that, on average, cleared deep-learning systems correctly identify more than nine out of ten true cases of diabetic retinopathy while correctly classifying roughly nine out of ten non-cases—performance levels broadly comparable to expert human graders. Deeks’ funnel-plot test revealed marginal evidence of publication bias across studies (p = 0.04, Supplementary Fig. 3).

Fig. 2. Hierarchical summary receiver operating characteristic curves of regulator-approved deep-learning systems for diabetic retinopathy detection.

a Patient-level HSROC curve showing the pooled model fit, 95% confidence and prediction regions, and study points scaled to weight. b Eye-level HSROC curve with the same visual elements, summarizing performance per eye.

Meta-regression of patient-wise regulator-approved deep learning system

Supplementary Table 6 presents the univariate meta-regression, revealing that while the overall diagnostic accuracy of regulator-approved deep-learning systems was high, several prespecified moderators contributed significantly to between-study heterogeneity. Studies conducted in low- and middle-income countries reported significantly higher specificity than those from high-income settings (0.93 vs 0.86; p < 0.01). Algorithm target influenced performance: systems detecting any DR showed reduced sensitivity and specificity (0.92 and 0.91, respectively) compared to those focused on referable or vision-threatening DR (p < 0.01 for both). Datasets with low image gradability yielded higher sensitivity (0.95 vs 0.92; p < 0.01) but lower specificity (0.82 vs 0.92; p < 0.01). The absence of vendor involvement produced a similar trade-off, with higher sensitivity (0.95 vs 0.87) but reduced specificity (0.85 vs 0.95; p < 0.01 for both). Single-field imaging was associated with higher specificity than two-field protocols (0.95 vs 0.88; p < 0.01). Portable fundus cameras also achieved higher specificity than desktop devices (0.95 vs 0.90; p < 0.01), though this comparison was underpowered (n = 3). Use of diagnostic criteria other than International Clinical Diabetic Retinopathy (ICDR), Early Treatment Diabetic Retinopathy Study (ETDRS) or National Health Service (NHS) was associated with higher specificity (0.92 vs 0.89; p < 0.01). Although healthcare setting and reference image modality significantly explained heterogeneity in sensitivity and specificity, respectively, they did not show statistically significant differences between subgroup means. Other factors—including grader discipline, study design, pupil dilation protocol, and article type—were not significantly associated with accuracy (p > 0.05 for all).

The multivariate meta-regression (Fig. 3a, Table 3) confirmed the independent contributions of DR category, national income level, diagnostic criteria, reference image modality, and image gradability to diagnostic performance, substantially improving model fit compared to the base model (AIC 3935 vs 5105 Supplementary Table 7). Portable fundus cameras and use of non-ICDR diagnostic criteria were consistently associated with higher specificity, while non-vendor-sponsored studies demonstrated a trade-off of higher sensitivity at the expense of lower specificity. In contrast, high-gradability datasets showed the opposite pattern, with slightly reduced sensitivity but improved specificity.

Fig. 3. Subgroup forest plots by diabetic-retinopathy subtype for regulator-approved deep-learning systems.

a Patient-level analysis displaying effect sizes and 95% confidence intervals for each DR-type subgroup, with pooled estimates and weights shown on the right; b Eye-level analysis built and displayed in the same manner.

Table 3.

Multivariate meta-regression of patient-wise regulator-approved deep learning system

Moderator	Category	N	Absolute Sensitivity	Relative Sensitivity	p-value (group-diff)	p-value (LR test)	Absolute Specificity	Relative Specificity	p-value (group-diff)	p-value (LR test)
Economic status	High income	61	0.94 (0.91 ~ 0.96)	1.01 (0.97 ~ 1.06)	0.52	<0.01*	0.86 (0.80 ~ 0.91)	0.94 (0.87 ~ 1.02)	0.05	<0.01*
	Low Middle income	54	0.93 (0.89 ~ 0.96)	1.00 (1.00 ~ 1.00)	-		0.92 (0.87 ~ 0.95)	1.00 (1.00 ~ 1.00)	-
Healthcare setting	Primary	54	0.91 (0.88 ~ 0.94)	0.95 (0.90 ~ 1.02)	0.15	0.43	0.91 (0.88 ~ 0.94)	1.18 (0.87 ~ 1.61)	0.29	0.46
	Secondary	19	0.97 (0.94 ~ 0.99)	1.02 (0.96 ~ 1.08)	0.64		0.81 (0.62 ~ 0.92)	1.05 (0.73 ~ 1.51)	0.79
	Tertiary	39	0.93 (0.90 ~ 0.96)	0.97 (0.92 ~ 1.04)	0.46		0.89 (0.83 ~ 0.94)	1.16 (0.85 ~ 1.58)	0.35
	Mixed	3	0.96 (0.89 ~ 0.99)	1.00 (1.00 ~ 1.00)	-		0.77 (0.47 ~ 0.93)	1.00 (1.00 ~ 1.00)	-
Diabetic retinopathy type	Any DR	35	0.92 (0.90 ~ 0.94)	0.98 (0.96 ~ 1.00)	0.03*	0.01*	0.90 (0.87 ~ 0.93)	0.99 (0.98 ~ 0.99)	<0.01*	<0.01*
	Referable DR	58	0.94 (0.93 ~ 0.96)	1.00 (0.99 ~ 1.02)	0.61		0.87 (0.84 ~ 0.90)	0.95 (0.94 ~ 0.97)	<0.01*
	Vison threatening DR	22	0.94 (0.91 ~ 0.96)	1.00 (1.00 ~ 1.00)	-		0.92 (0.89 ~ 0.94)	1.00 (1.00 ~ 1.00)	-
Camera type	Desktop	97	0.94 (0.92 ~ 0.95)	1.00 (1.00 ~ 1.00)	-	0.90	0.89 (0.85 ~ 0.92)	1.00 (1.00 ~ 1.00)	-	0.18
	Smartphone	15	0.92 (0.83 ~ 0.97)	0.98 (0.91 ~ 1.06)	0.65		0.91 (0.79 ~ 0.96)	1.03 (0.92 ~ 1.14)	0.63
	Portable	3	0.91 (0.83 ~ 0.95)	0.97 (0.91 ~ 1.03)	0.33		0.94 (0.91 ~ 0.97)	1.06 (1.03 ~ 1.10)	<0.01*
Diagnostic criteria	ICDR	75	0.93 (0.91 ~ 0.95)	1.00 (1.00 ~ 1.00)	-	0.84	0.89 (0.85 ~ 0.92)	1.00 (1.00 ~ 1.00)	-	<0.01*
	ETDRS	19	0.94 (0.91 ~ 0.96)	1.02 (0.97 ~ 1.07)	0.56		0.87 (0.75 ~ 0.94)	0.98 (0.87 ~ 1.10)	0.76
	NHS	7	0.95 (0.84 ~ 0.98)	1.02 (0.95 ~ 1.09)	0.94		0.81 (0.53 ~ 0.94)	0.91 (0.71 ~ 1.17)	0.47
	Other	14	0.94 (0.91 ~ 0.96)	1.01 (0.98 ~ 1.04)	0.46		0.94 (0.91 ~ 0.96)	1.05 (1.03 ~ 1.08)	<0.01
Reference image	Color fundus image	102	0.94 (0.92 ~ 0.95)	1.00 (1.00 ~ 1.00)	-	0.44	0.90 (0.87 ~ 0.92)	1.00 (1.00 ~ 1.00)	-	<0.01*
	With advanced image	13	0.93 (0.83 ~ 0.97)	1.00 (0.3 ~ 1.07)	0.85		0.80 (0.58 ~ 0.92)	0.89 (0.72 ~ 1.11)	0.30
Field strategy	1	19	0.95 (0.90 ~ 0.97)	1.00 (1.00 ~ 1.00)	-	0.07	0.92 (0.85 ~ 0.96)	1.00 (1.00 ~ 1.00)	-	0.16
	2	74	0.93 (0.90 ~ 0.95)	0.98 (0.94 ~ 1.02)	0.33		0.90 (0.85 ~ 0.93)	0.97 (0.90 ~ 1.05)	0.61
	>2	20	0.95 (0.90 ~ 0.98)	1.01 (0.96 ~ 1.06)	0.77		0.84 (0.69 ~ 0.93)	0.94 (0.80 ~ 1.09)	0.34
Vendor involved	No	79	0.95 (0.93 ~ 0.96)	1.09 (1.03 ~ 1.15)	<0.01*	0.61	0.86 (0.82 ~ 0.89)	0.90 (0.87 ~ 0.93)	<0.01*	0.13
	Yes	36	0.87 (0.82 ~ 0.91)	1.00 (1.00 ~ 1.00)	-		0.96 (0.94 ~ 0.97)	1.00 (1.00 ~ 1.00)	-
Gradable	High	73	0.92 (0.90 ~ 0.94)	0.97 (0.94 ~ 0.99)	0.01*	<0.01*	0.92 (0.89 ~ 0.94)	1.12 (1.08 ~ 1.17)	<0.01*	<0.01*
	Low	42	0.95 (0.93 ~ 0.97)	1.00 (1.00 ~ 1.00)	-		0.82 (0.77 ~ 0.86)	1.00 (1.00 ~ 1.00)	-

Values represent pooled sensitivity and specificity with 95% confidence intervals. P-values refer to differences in performance between subgroups within each moderator (Wald-type tests). Significant comparisons (p < 0.05) are marked with an asterisk ().*

DR diabetic retinopathy, ICDR International Clinical Diabetic Retinopathy severity scale, ETDRS Early Treatment Diabetic Retinopathy Study, NHS National Health Service, LR test Likelihood Ratio Test, p-value probability value, N number of studies, CI confidence interval.

Overall meta-analysis of eye-wise regulator-approved deep learning system

The hierarchical random-effects bivariate model showed considerable between-study heterogeneity—greater for specificity (τ² = 0.62; I² = 85.53%) and somewhat lower for sensitivity (τ² = 0.79; I² = 37.63%). A small covariance (0.04) and weak correlation (ρ = 0.05) illustrate the familiar trade-off: studies achieving higher sensitivity tended to report lower specificity, and vice versa. A likelihood-ratio test strongly favored the random-effects specification over a fixed-effects alternative (χ² = 127,870, df = 3, p < 0.001), confirming the need to model heterogeneity explicitly. After multivariable adjustment in a bivariate random-effects meta-regression, the pooled summary of 88 tables remained high. The pooled sensitivity was 0.92 (95% CI 0.89–0.94) and the pooled specificity was 0.93 (95% CI 0.91–0.94) (Fig. 2b, Supplementary Fig. 4), indicating that cleared DL algorithms correctly detect more than nine out of ten true cases of referable diabetic retinopathy while correctly excluding roughly nine out of ten non-cases—performance comparable to expert human graders. Deeks’ funnel-plot test suggested no evidence of publication bias for these regulator-approved systems (p = 0.23; Supplementary Fig. 5).

Meta-regression of eye-wise regulator-approved deep learning system

Most moderators resulted in only modest shifts in diagnostic accuracy, but several effects reached statistical significance. Sensitivity was significantly higher in studies using mixed pupil-dilation protocols—where pharmacologic dilation was applied selectively—showing an 8% increase (p = 0.01), and when the reference standard included advanced imaging modalities such as ultra-wide-field photography or OCT, with a 5% gain (p = 0.04). In contrast, specificity was lower in studies conducted in high-income countries (−4%), in those targeting any DR rather than referable or vision-threatening DR (−2%; p < 0.01), and in datasets with high image gradability (−6%; p < 0.01), consistent with the sensitivity–specificity trade-offs observed in patient-level evaluations. Other moderators—including healthcare setting, camera form factor, study design, diagnostic criteria, index-test grader, field strategy, and vendor involvement—did not significantly contribute to heterogeneity (all p > 0.05; Supplementary Table 8).

Multivariate meta-regression incorporating five moderators (Fig. 3b, Table 4)—economic status, DR category, pupil dilation, reference-image modality, and image gradability—substantially improved model fit over the base model (AIC 78570 vs 6550; Supplementary Table 9). Based on likelihood ratio tests, economic status (specificity, p < 0.01) and reference-image modality (both sensitivity and specificity, p < 0.01) showed overall significant effects, while DR category, pupil dilation, and gradability did not reach significance in the omnibus tests. However, individual group comparisons revealed important patterns: Compared to systems targeting vision-threatening DR, algorithms detecting any DR had 4% higher specificity (relative specificity 1.04; p = 0.01) with a corresponding 17% lower sensitivity (relative sensitivity 0.83; p < 0.01), suggesting threshold dispersion across severity definitions. Referable DR systems showed performance similar to vision-threatening DR systems (relative sensitivity 0.99, relative specificity 1.00; both p < 0.01). Mixed dilation protocols showed a trend toward 3-6% higher sensitivity compared to non-mydriatic or mydriatic imaging alone, though these differences did not reach statistical significance (p = 0.15). Advanced imaging methods (including OCT) showed a 4% sensitivity advantage (relative sensitivity 1.04; p = 0.18), though this was not statistically significant after adjustment. High-gradability datasets maintained significantly higher specificity than low-gradability datasets (6% difference; p < 0.01), with no difference in sensitivity (p = 0.98). High-income settings showed slightly lower specificity than low-middle-income settings (relative specificity 0.99; p < 0.01 by LR test).

Table 4.

Multivariate meta-regression of eye-wise regulator-approved deep learning system

Moderator	Category	N	Absolute Sensitivity	Relative Sensitivity	p-value (group-diff)	p-value (LR test)	Absolute Specificity	Relative Specificity	p-value (group-diff)	p-value (LR test)
Economic status	High income	35	0.93 (0.89 ~ 0.96)	1.01 (0.96 ~ 1.06)	0.69	0.98	0.94 (0.91 ~ 0.96)	0.99 (0.95 ~ 1.02)	0.50	<0.01*
	Low Middle income	53	0.92 (0.88 ~ 0.95)	1.00 (1.00 ~ 1.00)	-		0.95 (0.92 ~ 0.96)	1.00 (1.00 ~ 1.00)	-
Diabetic retinopathy type	Any DR	14	0.79 (0.72 ~ 0.84)	0.83 (0.78 ~ 0.87)	<0.01*	0.31	0.97 (0.96 ~ 0.98)	1.04 (1.03 ~ 1.05)	0.01*	0.29
	Referable DR	64	0.94 (0.92 ~ 0.96)	0.99 (0.98 ~ 1.01)	<0.01*		0.94 (0.92 ~ 0.95)	1.00 (1.00 ~ 1.00)	<0.01*
	Vison threatening DR	10	0.95 (0.93 ~ 0.96)	1.00 (1.00 ~ 1.00)	-		0.93 (0.91 ~ 0.95)	1.00 (1.00 ~ 1.00)	-
Pupil dilation	Yes	12	0.90 (0.81 ~ 0.95)	0.94 (0.87 ~ 1.02)	0.15	0.22	0.90 (0.83 ~ 0.95)	0.97 (0.90 ~ 1.04)	0.33	0.19
	No	57	0.92 (0.88 ~ 0.95)	0.97 (0.92 ~ 1.01)	0.15		0.95 (0.93 ~ 0.97)	1.02 (0.97 ~ 1.06)	0.48
	Mixed	19	0.95 (0.92 ~ 0.98)	1.00 (1.00 ~ 1.00)	-		0.94 (0.89 ~ 0.96)	1.00 (1.00 ~ 1.00)	-
Reference image	Color fundus image	84	0.93 (0.90 ~ 0.95)	1.00 (1.00 ~ 1.00)	-	<0.01*	0.94 (0.93 ~ 0.96)	1.00 (1.00 ~ 1.00)	-	<0.01
	With advanced image	4	0.96 (0.87 ~ 0.99)	1.04 (0.98 ~ 1.10)	0.18		0.90 (0.76 ~ 0.97)	0.96 (0.86 ~ 1.06)	0.41
Gradable	High	38	0.93 (0.89 ~ 0.95)	1.00 (1.00 ~ 1.00)	-	0.69	0.90 (0.87 ~ 0.93)	1.00 (1.00 ~ 1.00)	-	0.50
	Low	50	0.93 (0.89 ~ 0.95)	1.00 (0.96 ~ 1.04)	0.98		0.96 (0.95 ~ 0.97)	0.94 (0.92 ~ 0.96)	<0.01

Values represent pooled sensitivity and specificity with 95% confidence intervals. P-values refer to differences in performance between subgroups within each moderator (Wald-type tests). Significant comparisons (p < 0.05) are marked with an asterisk ().*

DR diabetic retinopathy, LR test Likelihood ratio test, p-value probability value, N number of studies, CI confidence interval.

Discussion

Our review screened 8404 records and ultimately meta-analyzed 82 eligible studies evaluating regulator-approved deep-learning systems for diabetic retinopathy screening. Pooled accuracy was high—patient-wise sensitivity 0.93 (95% CI 0.91–0.95) and specificity 0.90 (95% CI 0.87–0.92); eye-wise 0.92 (95% CI 0.89–0.94) and specificity was 0.93 (95% CI 0.91–0.94), respectively—yet substantial between-study heterogeneity (τ² ≈ 0.5–0.9) justified random-effects modeling. Multivariate meta-regressions revealed six consistent performance modifiers: DR severity tier, national income level, image gradability, pupil dilation status, diagnostic criteria, and reference image. In particular, “any-DR” tasks, low-income settings, and high-gradability datasets reduce sensitivity, while low-income settings, “vtDR” task, standard color fundus as a reference image, other diagnostic criteria than ICDR, ETDRS, NHS, and mixed dilated pupil protocol, boosted specificity. Eye-level analyses echoed these patterns but with smaller effect sizes; once confounding was controlled, only DR tier and gradability retained independent influence. Multivariate models halved unexplained variance (AIC 5439 → 3909; τ² 0.077 → 0.041). Deeks’ tests indicated marginal publication bias for patient-level results (p = 0.04) and none for eye-level data (p = 0.23). Together, these findings confirm that cleared DL algorithms perform comparably to expert graders while highlighting context-specific trade-offs that should guide real-world deployment.

Despite the uniformly high accuracy, we observed substantial heterogeneity in specificity across studies. Added evidence on co-pathology performance. A further source of real-world uncertainty is the presence of retinal co-morbidities—most commonly age-related macular degeneration (AMD)—whose lesions can mimic or obscure diabetic lesions. Four prospective or retrospective evaluations^38,43,62,77 deliberately enrolled mixed DR + AMD cohorts and reported AUCs of 0.88–0.98 with sensitivities ≥84%. However, specificities fell to 66–93%, highlighting a consistent false-positive penalty when exudates or hemorrhages overlap with drusen or geographic atrophy. He et al.⁴⁴ further showed that cross-camera domain shifts degrade agreement more for AMD than for DR, underscoring the need for multi-disease, multi-device validation before deployment. These findings confirm that current DL algorithms remain feasible in heterogeneous case-mix but that additional multi-center trials—including other look-alike pathologies such as vein occlusion or myopic choroidal neovascularization—are required to quantify workflow impact and cost-effectiveness. In practice, the reported sensitivities and specificities varied, reflecting differences in populations, settings, and implementation. A classic threshold trade-off was evident: studies that reported extremely high sensitivity tended to have lower specificity, and vice versa⁹⁸. This negative correlation suggests that some implementations tuned the algorithm to prioritize catching all possible DR (yielding very few false negatives at the expense of more false positives), whereas others set a stricter threshold to reduce false positives⁹⁸. Such threshold effects are common in diagnostic tests—if you “cast a wider net” to catch every case, you inevitably pull in a few healthy eyes as positive. In the UK, the UK National Screening Committee (UK NSC) emphasizes that service load is highly sensitive to specificity and supports risk-stratified recall intervals within the English Diabetic Eye Screening Program (DESP).

Meta-regression analyses revealed substantial heterogeneity in the diagnostic performance of regulator-approved deep learning (DL) systems for diabetic retinopathy (DR) detection across clinical, technical, and contextual moderators. Among these, disease severity tier emerged as a consistent and statistically significant source of variation. Systems evaluated for any DR demonstrated lower sensitivity (by 2 percentage points [pp]) and lower specificity (by 2 pp) compared to those targeting referable or vision-threatening DR (p < 0.01), with an additional 4 pp specificity gain at the eye level with 17% corresponding sensitivity loss. These findings reflect an inherent trade-off between early detection and false-positive burden. While broader screening thresholds may increase case-finding for mild DR, they also heighten the risk of false positives, potentially overwhelming follow-up services. Programs aiming to detect mild disease must therefore implement second-reader workflows, risk-based triage, or structured recall systems to prevent unnecessary escalation of low-risk findings. In the UK, UK NSC guidance recommends risk-stratified recall intervals (e.g., 12–24 months vs 6 months) within DESP to absorb the additional false-positive burden when “any-DR” thresholds are used⁹⁹.

The analysis also identified clear performance discrepancies across geographic and health system contexts. Studies conducted in low- and middle-income countries (LMICs) exhibited modestly higher sensitivity (+1 pp) but substantially reduced specificity (–6 pp; p ≤ 0.01). This likely reflects higher background disease prevalence, a wider spectrum of disease presentation, and variable image quality, all of which challenge systems primarily trained on high-fidelity datasets from high-income countries. In such contexts, the increased false-positive burden can pose a serious threat to already constrained healthcare infrastructures. This underscores the need for context-aware deployment strategies—including tiered referral systems, integration with teleophthalmology, and infrastructure reinforcement to accommodate the downstream impact of automated detection systems.

Image gradability moderated DL performance differently depending on the unit of analysis. At the patient level, low-gradability datasets (N = 42) showed +3-percentage-point higher sensitivity (0.95 vs 0.92; p = 0.01) but—9-percentage-point lower specificity (0.84 vs 0.93; p < 0.01), suggesting systems err toward caution with poor-quality images—a “safety-positive” behavior that may inflate referrals. At the eye level (Table 4), high-gradability datasets (N = 38) paradoxically showed—6-percentage-point lower specificity (0.90 vs 0.96; p < 0.01) with no significant sensitivity difference, likely reflecting that datasets excluding more ungradable images retain harder borderline cases. These findings support real-time image quality assessment and recapture prompts at point of care to ensure optimal DL performance and minimize unnecessary escalations. The UK NSC calls for standardized gradability reporting to enable cross-site benchmarking and automated audit pipelines⁹⁹.

Vendor involvement emerged as a significant moderator at the patient level: non-industry studies (N = 79) reported +7-percentage-point higher sensitivity (0.95 vs 0.88; p < 0.01) but—10-percentage-point lower specificity (0.86 vs 0.96; p < 0.01) compared to vendor-involved evaluations (N = 36). However, at the eye level (Table 4), vendor involvement showed no significant effect: non-vendor studies (N = 46) had only—3-percentage-point lower sensitivity (0.90 vs 0.93; p = 0.21) and identical specificity (0.93 vs 0.93; p = 0.61) compared to vendor-involved studies (N = 42). This discrepancy between patient-level and eye-level analyses suggests the patient-level differences may reflect study design factors rather than vendor influence per se. These findings underscore the need for preregistered, independently conducted validations to complement regulatory submissions. The UK National Screening Committee (UK NSC) lists ‘independent, head-to-head evaluations on identical datasets’ as a core evidence domain for adoption⁹⁹.

On the technical front, portable fundus cameras delivered significantly higher specificity (+5 pp; p ≈ 0.01) without a loss in sensitivity, affirming their value in mobile or community-based screening programs, particularly in resource-limited environments. Smartphone-based systems, in contrast, showed no significant deviation in performance relative to desktop platforms (p > 0.50), suggesting functional equivalence across capture modalities in adequately trained models. Reference imaging modality was significantly associated with between-study heterogeneity, although within-group differences were not statistically significant, suggesting that certain advanced imaging techniques (e.g., ultra-widefield [UWF], OCT) may offer marginal sensitivity benefits that dissipate after adjustment for confounders. Similarly, single-field imaging protocols demonstrated higher specificity in univariate analysis (+7 pp; p < 0.01), although this benefit did not persist in multivariate models, indicating that field strategy alone is not a dominant performance driver.

Other factors—including study design, healthcare tier, pupil dilation, diagnostic criteria, index test grader, field strategy, camera brand, and article type—were not significantly associated with performance in adjusted models (all p > 0.05). This suggests a reassuring robustness of pooled estimates across a wide range of operational conditions and affirms the validity of aggregated performance metrics for regulatory and policy decision-making. However, interaction modeling provided further nuance: performance gains associated with non-vendor studies were concentrated in high-gradability datasets, while specificity losses were most pronounced in low-gradability scenarios. Portable camera advantages persisted across interaction strata, highlighting their resilience to variable imaging conditions.

Taken together, these findings emphasize that the diagnostic performance of DL systems is not static, but rather deeply context-sensitive—shaped by disease target, image quality, geographic setting, technical infrastructure, and transparency of evaluation. As such, regulatory frameworks should move beyond one-size-fits-all benchmarks and adopt dynamic, context-aware approval and surveillance models that reflect real-world deployment conditions. Clinically, implementation should be accompanied by workflow adaptations to address false-positive burden and optimize human-AI collaboration. For health systems—especially those in LMICs—investment in triage capacity, image quality assurance, and portable imaging technology will be essential to realizing the full benefits of AI-assisted DR screening while maintaining efficiency and equity.

Our findings are broadly consistent with and extend previous meta-analyses of deep learning algorithms for diabetic retinopathy (DR) detection. Earlier reviews focusing on experimental, non-regulatory systems reported pooled sensitivities between 87–90% and specificities around 88–92%¹⁰⁰. In contrast, our analysis—restricted to regulator-approved systems—demonstrates slightly higher sensitivity (93%) with comparable specificity (90%), suggesting that the regulatory pathway may favor top-performing models. For instance, our pooled sensitivity surpasses the 87.4% reported in the pivotal trial of the first FDA-cleared system, IDx-DR¹⁴, and aligns closely with EyeArt’s U.S. FDA trial (96% sensitivity, 88% specificity)¹⁵ and UK real-world deployments showing 95–97% sensitivity⁶⁸.

Autonomous AI screening offers a scalable solution for expanding DR detection beyond ophthalmology clinics. It enables point-of-care use in primary care, endocrinology, and community settings—reaching patients who might otherwise miss annual exams. In a randomized trial, 100% of pediatric diabetes patients completed screening with autonomous AI compared to 22% under standard referral pathways¹⁰¹. In the UK, AI deployment halved the grading workload while maintaining >95% sensitivity⁶⁸. These findings support a safe shift toward upstream screening with manageable trade-offs. However, regulatory clearance is only the first step. The UK NSC sets out seven evidence domains—pathway-level accuracy, prospective clinical impact, cost-effectiveness, equity analysis, vendor-independent validation, interoperability testing, and post-market surveillance—for the UK context⁹⁹. These domains map onto our moderator findings and research gaps. Early engagement between developers and screening agencies, plus staged pilots using stepped-wedge or test-treat RCT designs, are highlighted as preferred evaluation routes⁹⁹. Policy shifts, such as U.S. Medicare reimbursement for Current Procedural Terminology (CPT) 92229, signal growing recognition of AI’s clinical value¹⁰². Scaling these tools globally will require national screening integration and reimbursement models that support sustainability. Importantly, our review highlights a persistent evidence gap in low-resource settings. Only one included study was from a low-income country, underscoring the need for greater geographic representation. Smartphone-based fundus imaging combined with AI shows promise for underserved regions^26,45,87, but robust validation, public-sector procurement, and international pilot programs will be essential. To avoid widening health inequities, global AI deployment must be accompanied by investments in image quality assurance, usability training, and infrastructure.

This is the first systematic review and meta-analysis focused exclusively on regulator-approved AI systems for diabetic retinopathy (DR) screening. Key strengths include the largest synthesis to date of diagnostic accuracy data for these systems and a uniquely detailed moderator analysis. Notably, we examined previously underreported variables such as vendor involvement and image gradability, which the latter contributed significantly to between-study heterogeneity. Our emphasis on these dimensions directly addresses four of the NSC’s seven evidence domains (accuracy, equity, independence, and image quality), underscoring the translational value of our findings. We also separated patient-wise and eye-wise outcomes to avoid conflating correlated units of analysis—a limitation seen in prior studies. Importantly, we applied a hierarchical bivariate random-effects model with extensions to account for within-study correlation between sensitivity and specificity, inter-moderator dependencies, and study-level clustering, representing one of the most comprehensive modeling frameworks applied to this field.

A key limitation is the inconsistent reporting of image gradability; future studies should adopt a standardized gradability metric to facilitate cross-study comparisons. In addition, patient-level factors that plausibly affect gradability—such as age, cataract, or neurocognitive disorders—could not be examined because most publications lacked individual-patient data; incorporating these covariates in upcoming datasets would enable a more granular exploration of accuracy modifiers. Software versioning was not uniformly documented across studies. Because different versions (e.g., v1 vs v2.2) may materially affect performance and thresholds, incomplete reporting prevented version-specific subgroup or sensitivity analyses. Also, Our publication-based search strategy may have missed some regulator-approved algorithms that lack peer-reviewed validation studies. While publication bias could not be ruled out, we observed no significant asymmetry in funnel plots. Additionally, although the inclusion of a broad range of algorithms enhances generalizability, it also introduces heterogeneity. Fewer than half of the included systems have been stress-tested outside the setting in which they were developed, underscoring the need for prospective, multi-regional validation, particularly in low-resource and ethnically diverse environments. Despite these limitations, our findings are robust and have strong implications for real-world deployment. Future work should include head-to-head trials of multiple AI systems, implementation research assessing clinical outcomes and workflow impact, and the development of automated audit pipelines. Standardized Automated Retinal Image Analysis Systems (ARIAS) application programming interfaces (APIs) and open audit dashboards, as advocated by the UK NSC in the UK program, would also streamline ongoing performance monitoring. Further studies should evaluate cost-effectiveness, prioritize low-resource settings, and explore multi-disease screening with fundus photography.

In conclusion, regulator-approved deep-learning systems have reached—and, in many contexts, surpassed—expert-grader accuracy for diabetic-retinopathy screening, correctly identifying ≈93% of referable cases while excluding ≈87–92% of non-cases. Yet their performance is not monolithic: disease-severity threshold, economic context, image gradability, vendor sponsorship, camera portability, and grader expertise each shift the sensitivity–specificity balance, explaining roughly half of the observed heterogeneity. Programs that target any DR, operate in low-/middle-income settings, or work with low-quality images must plan for a heavier false-positive load, whereas those using portable cameras and retina-specialist adjudication can expect higher specificity. Independent, preregistered evaluations and post-market surveillance are essential to mitigate sponsor bias and dataset drift, and future studies should adopt a standardized gradability metric and report patient-level covariates (e.g., age, cataract) to enable finer-grained moderator analyses. With robust quality-assurance loops, harmonized regulatory oversight, and equitable reimbursement models, autonomous AI can extend high-quality DR screening to the millions who currently lack access, helping to avert preventable vision loss on a global scale.

Methods

General guideline

This systematic review was prospectively registered with INPLASY (registration number: INPLASY202550099)¹⁰³. We followed the relevant items of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy Studies (PRISMA-DTA) checklist (see Supplementary Table 1,2 for item-by-item compliance)^104,105 to quantify the diagnostic accuracy of all regulator-approved deep-learning systems for fundus-based diabetic retinopathy detection and identify key factors influencing their performance across diverse global settings. Title and abstract screening, full-text eligibility assessment, data extraction, and risk-of-bias appraisal were each conducted independently and in duplicate by two reviewers (T-WW, L-WT); disagreements were resolved through discussion with a third reviewer. Because the review relied solely on previously published, de-identified data, no institutional ethical approval or informed consent was required.

Database searches and identification of eligible manuscripts

Two reviewers (T-W.W. and L-W.T.) systematically searched PubMed, Embase, and ClinicalTrials.gov from database inception to 3 April 2025 for studies that validated a regulator-approved deep-learning system for detecting diabetic retinopathy on color fundus photographs. For each database, we used a structured search strategy combining three concept blocks—condition (diabetic retinopathy and related retinal diseases), technology (artificial intelligence, machine learning, deep learning, automated detection), and system/regulatory context (named commercial algorithms and regulator-approval terms)—with database-specific subject headings (e.g., MeSH, Emtree) and title/abstract fields, without language restrictions. The comprehensive search strategy is detailed in Supplementary Table 3. Records were imported into the Nested Knowledge™ platform for automatic de-duplication and audit-trailed screening. Titles/abstracts and full texts were assessed in duplicate against prespecified criteria; disagreements were resolved by consensus.

We included prospective or retrospective diagnostic-accuracy studies of deep-learning algorithms for DR screening using color fundus photographs in adults (≥18 years); mixed-age cohorts were eligible if ≥85% were adults or if adult-only data allowed construction of a 2 × 2 table. Any clinical screening/diagnostic setting was eligible; country/region refers to the validation dataset location. Index test: DL autonomous or DL-assisted classifier on fundus photographs (excluded: non-DL methods, non-fundus modalities such as OCT/OCTA, and non-image risk models). Reference standard: human graders using recognized DR schemes (excluded: algorithm-only references). Target conditions: any DR grade; analyses prioritized prespecified thresholds for referable DR (≥ moderate non-proliferative diabetic retinopathy (NPDR) ± diabetic macular edema (DME)) and vision-threatening DR, using author-defined cut-points. Study design: must report, or permit calculation of, TP/FP/FN/TN; excluded: development-only studies without an external or distinct hold-out validation. Primary pooling used independent validation only, defined a priori as evaluations on datasets not used for model development. Report characteristics: searches from inception to 3 April 2025, no language restrictions; we meta-analyzed peer-reviewed full texts only, using preprints/abstracts for identification and citation chasing unless superseded before the cutoff. We also performed hand searching of references and backward/forward citation chasing. Where studies reported distinct strata with unique 2 × 2 tables (e.g., by camera, threshold, mydriasis, model version, or grader/reference image), each stratum was treated as an independent sub-study.

Data extraction and management

Titles and abstracts were screened independently by two reviewers, followed by a full-text eligibility assessment. Data were captured with a prespecified electronic extraction form, and every exclusion—together with its rationale—was documented in the PRISMA flow diagram. For each eligible study, we recorded (i) bibliographic details (first author, publication year, country (geographic location of the validation dataset)); (ii) study design and context (prospective/retrospective, recruitment period, clinical setting, regulatory-clearance pathway, external validation in different country from training set); (iii) participant characteristics (sample size, mean age, proportion female, economic status (World-Bank income category¹⁰⁶)); (iv) index-test specifications (device name, training data country and ethics, fundus-camera model, mydriatic vs non-mydriatic protocol, number and field of images, algorithm name, vendor involvement (participation in coauthor ship or provision of funding support by the algorithm manufacturer), regulatory certification, model architecture, image gradability); (v) reference standard (grading protocol and grader expertise); and (vi) diagnostic-accuracy data (2 × 2 confusion matrix, sensitivity, specificity, 95% CIs). We confirmed status in primary regulator databases (FDA De Novo/510(k), EU CE-MDR/MDD, UKCA, TGA-ARTG, HSA-SMDR, Philippines FDA, NMPA; MDALL/PMDA as needed). If no primary record was available, we corroborated manufacturer pages and press releases. When results were stratified by data source, camera type, AI model, DR severity, mydriasis status, and index test grader or reference image, we treated each stratum as a separate sub-study and entered it as an independent table in the meta-analysis.

Quality assessment

We appraised methodological quality using the QUADAS-2 framework¹⁰⁷. Two reviewers (T-W.W. and L-W.T.) first piloted the QUADAS-2 signaling questions on a random set of five studies to calibrate interpretation, then independently applied the tool to each included record. After independent scoring, disagreements were resolved through discussion; if consensus was not reached, a third author adjudicated. Each study was rated for risk of bias and applicability concerns in four domains: patient selection, index test, reference standard, and flow & timing, classified as “low,” “high,” or “unclear.” Where studies included multiple algorithm versions or imaging protocols, we evaluated the analysis with the highest detail level available to capture AI-specific bias, such as data leakage or threshold optimization.

Statistical analysis

We first reconstructed 2 × 2 contingency tables for each eligible dataset by extracting the counts of true positives, false positives, true negatives, and false negatives from the numerators and denominators reported in the original studies. Patient- and eye-level analyses were separated because the correlation between fellow eyes if treated as independent, can underestimate variance, inflate precision, and artificially heighten heterogeneity in pooled accuracy estimates. Diagnostic accuracy was synthesized using a hierarchical bivariate random-effects model, the recommended framework for meta-analyses of diagnostic test accuracy. Specifically, we used a bivariate hierarchical logistic-normal model that preserves the intrinsic correlation between sensitivity and specificity while allowing both within- and between-study variability.

$$\mathrm{Level}1(\mathrm{Within}-\mathrm{Study}):T{P}_j~\mathit{Bin}(S{e}_j,T{P}_j+F{N}_j)\mathrm{and}T{N}_j~\mathit{Bin}(S{p}_j,T{N}_j+F{P}_j)$$

$$\mathrm{Level}2(\mathrm{Between}-\mathrm{Study}):\left[\begin{array}{c}\mathit{logit}\left(S{e}_j\right)\hfill \\ \mathit{logit}\left(S{p}_j\right)\hfill \end{array}\right]~N\left[\begin{array}{cc}\begin{array}{c}{\mu }_1\hfill \\ {\mu }_2\hfill \end{array},& {\Sigma }_{\mathit{between}}\end{array}\right],\mathrm{and}{\Sigma }_{\mathit{between}}=\left[\begin{array}{cc}{\tau }_{\mathit{Se}}^2& \rho {\tau }_{\mathit{Se}}{\tau }_{\mathit{Sp}}\\ \rho {\tau }_{\mathit{Se}}{\tau }_{\mathit{Sp}}& {\tau }_{\mathit{Sp}}^2\end{array}\right]$$

Fixed Effects parameters (β_Se, β_Sp) represent the pooled logit-sensitivity and logit-specificity, estimated in Stata 18 (metadta) by maximum-likelihood with adaptive Gauss–Hermite quadrature (nine nodes); Laplace approximation was used as a sensitivity check. This model produced pooled sensitivity, and specificity estimates with corresponding 95% confidence intervals (CIs) and supported the construction of hierarchical summary receiver operating characteristic (HSROC) curves. Each HSROC plot displays the summary operating point surrounded by a 95% confidence ellipse and an outer 95% prediction ellipse, representing the range within which a future study’s accuracy is expected to fall. Study-level estimates were visualized using forest plots. Individual-study CIs used exact Clopper–Pearson limits. Pooled 95% CIs were derived analytically (Wald method) from the multivariate-normal sampling distribution of (β_Se, β_Sp) and back-transformed to the probability scale using the delta method. Prediction ellipses were generated in closed form from the random-effects variance–covariance matrix ${\Sigma }_{\mathit{RE}}$. as the locus ${\left(x-\widehat{\beta }\right)}^{\mathrm{T}}{\sum }_{RE}^{-1}\left(x-\widehat{\beta }\right)X_{\mathrm{2,0.95}}^2$. Because all variance components have analytic ML estimates, no bootstrapping was required.

To explore sources of heterogeneity, we performed univariate meta-regression analyses using prespecified moderators: study design (prospective vs retrospective), national income classification (high, upper-middle, lower-middle), healthcare setting (primary, secondary, tertiary), DR category (any diabetic retinopathy (aDR), defined as the presence of any detectable DR lesion including mild non-proliferative DR or worse; referable DR (rDR), defined as moderate NPDR or worse, diabetic macular edema (DME) or proliferative DR; and vision-threatening DR (vtDR), defined as severe NPDR, proliferative diabetic retinopathy (PDR), and/or clinically significant macular edema (CSME))(mapping of grading scales to these categories is provided in Supplementary Table 10), pupil dilation protocol (yes, no, mixed), camera form factor (desktop vs smartphone vs portable), diagnostic criteria (ICDR, ETDRS, NHS, other), index-test grader (ophthalmologist, retina specialist, other), reference-image modality (color fundus vs advanced imaging such as wide-field fundus), retinal field strategy (one, two, more than two), vendor involvement (yes vs no), and image gradability (high [≥95%] vs low [<95%]¹⁰⁸). Each covariate was simultaneously entered into both the sensitivity and specificity equations, and nested models were compared with likelihood-ratio tests; overall fit was evaluated by differences in Akaike and Bayesian information criteria (lower AIC/BIC = better fit). Between-study heterogeneity was summarized by ${\tau }_{\mathit{Se}}^2$, ${\tau }_{\mathit{Sp}}^2$, their correlation ρ, and the bivariate analog of $I^2$ (output by metadta). Moderators that significantly explained between-study variance (p < 0.05) in univariate models were included in a multivariate meta-regression to assess their combined effect. Publication bias was assessed for each AI system using Deeks’ funnel plot asymmetry test¹⁰⁹; robustness was evaluated by stratifying analyses by article type (peer-reviewed vs conference, abstract or supplement). All statistical analyses were conducted in Stata 18.0 using the metadta (bivariate DTA) and midas (HSROC visualization) packages. No resampling (e.g., bootstrapping) was required because closed-form variance expressions were available for all model components and transformations.

Author contributions

Wang T.-W.: Conceptualization, Methodology, Formal analysis, Data Curation, Writing – Original Draft, Visualization, Project administration; Luo W.-T.: Investigation, Data Curation, Writing – Review & Editing; Tu Y.-K.: Methodology, Validation, Formal analysis, Supervision, Writing – Review & Editing; Chou Y.-B.: Supervision, Writing – Review & Editing; Wu Y.-T.: Supervision, Resources, Writing – Review & Editing.

Data availability

The corresponding author had full access to all data in the study and final responsibility for the decision to submit the report for publication. The data used and analyzed during the current study are available from the corresponding author upon reasonable request.

Code availability

Study protocol and metadata are available from the corresponding author.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41746-025-02223-8.