. 2021 Apr 7;4:65. doi: 10.1038/s41746-021-00438-z

Table 2.

Characteristics of ophthalmic imaging studies.

Study	Model	Prospective?	Test set	Population	Test datasets	Type of internal validation	External validation	Reference standard	AI vs clinician?	Imaging modality	Pathology
Abramoff et al. 2016	AlexNet/VGG	No	1748	Photographs	Messidor-2	NR	No	Expert consensus	No	Retinal fundus photography	Referable DR
Abramoff et al. 2018¹⁴	AlexNet/VGG	Yes	819	Patients	Prospective cohort from 10 primary care practice sites in USA	NR	Yes	Expert consensus	No	Retinal fundus photography	More than mild DR
Ahn et al. 2018	(a) Inception-v3; (b) customised CNN	No	(a) 464; (b) 464	Images	Kim’s Eye Hospital, Korea	Random split	No	Expert consensus	No	Retinal fundus photography	Early and advanced glaucoma
Ahn et al. 2019	ResNet50	No	219	Photographs	Kim’s Eye Hospital, Korea	Random split	No	Expert consensus	No	Retinal fundus photographs	Pseudopapilloedema
Al-Aswad et al. 2019⁴⁶	Pegasus (ResNet50)	No	110	Photographs	Singapore Malay Eye Study	Random split	No	Existing diagnosis from source data	Yes	Retinal fundus photographs	Glaucoma
Alqudah et al. 2019²²	AOCT-NET	No	1250	Scans	Farsiu Ophthalmology 2013	Hold-out method	Yes	NR	No	OCT	(a) AMD; (b) DME
Arcadu et al. 2019	Inception-v3	No	(a) 1237; (b) 1798	Images	RISE/RIDE trials	Random split	No	Expert consensus	No	Retinal fundus photography	(a) DME—central subfield thickness >400 µm; (b) DME—central fovea thickness >400 µm
Asaoka et al. 2016	Deep feed-forward neural network with stacked denoising autoencoder	No	279	Eyes	University of Tokyo Hospital, Tokyo	Random split	No	Other imaging technique	No	Visual Fields	Preperimetric open-angle glaucoma
Asaoka et al. 2019	Customised CNN	No	196	Images	University of Tokyo Hospital, Tokyo	Random split	No	Expert consensus	No	OCT	Early open-angle glaucoma
Asaoka et al. 2019²³	ResNet50	No	(a) 205; (b) 171	Scans	(a) Iinan Hospital; (b) Hiroshiuma University	NR	Yes	Expert consensus	No	OCT	Glaucoma
Bellemo et al. 2019¹⁵	VGG/ResNet	Yes	3093	Eyes	Kitwe Central Hospital Eye Unit, Zambia	NA	Yes	Expert consensus	No	Retinal fundus photography	(a) Referable DR; (b) vision-threatening DR; (c) DME
Bhatia et al. 2019²⁴	VGG-16	No	(a) 4686; (b) 384; (c) 148; (d) 100; (e) 135; (f) 135; (g) 148; (h) 100	Scans	(a) Shiley Eye Institute of the UCDS; (b) Devers Eye Institute; (c) Noor Eye Hospital; (d) Ophthalmica Ophthalmology Greece; (e) Cardiff University; (f) Cardiff University; (g) Noor Eye Hospital; (h) Ophthalmica Ophthalmology Greece	NA	Yes	(a) Expert consensus; (b) NR; (c) NR; (d) NR; (e) Expert consensus + further imaging; (f) expert consensus + further imaging; (g) NR; (h) NR	No	OCT	(a) Abnormal scan; (b–f) AMD; (g–h) DME
Brown et al. 2018⁴⁷	Inception-v1 and U-Net	No	100	Photographs	i-ROP	Hold-out method	No	Expert consensus	Yes	Retinal fundus photography	Plus disease in ROP
Burlina et al. 2017⁴⁹	DCNN	No	5664	Images	AREDS 4 dataset	NR	No	Expert consensus	Yes	Retinal fundus photography	AMD-AREDS 4 step
Burlina et al. 2018⁴⁸	ResNet50	No	5000	Images	AREDS	Random split	No	Reading centre grader	No	Retinal fundus photographs	Referable AMD
Burlina et al. 2018⁵⁰	AlexNet	No	13,480	Photographs	NIH AREDS	NR	No	Reading centre grader	Yes	Retinal fundus photography	Referable AMD
Burlina et al. 2018	ResNet50	No	(a) 6654; (b) 58,978	Images	(a) AREDS 9 dataset; (b) AREDS 4 dataset	NR	No	Reading centre grader	Yes	Retinal fundus photography	(a) AMD-AREDS 4 step; (b) AMD-AREDS 9 step
Chan et al. 2018²⁵	AlexNet, VGGNet, GoogleNet	No	4096	Images	SERI	NR	Yes	Reading centre grader	No	OCT	DME
Choi et al. 2017	VGG-19	No	(a) 3000; (b) 3000	Photographs	STARE database	Random split	No	Expert consensus	No	Retinal fundus photographs	(a) DR; (b) AMD
Christopher et al. 2018¹⁶	(a) VGG-16; (b) Inception-v3; (c) ResNet50	Yes	1482	Images	ADAGES and DIGS	Random split	No	Expert consensus	No	Retinal fundus photography	Glaucomatous optic neuropathy
Das et al. 2019	VGG-16	No	1000	Images	UCSD	Hold-out method	No	Expert consensus	No	OCT	DME
De Fauw et al. 2018⁵¹	(a) U-Net (b) customised CNN	No	(a) 997; (b) 116	(a) Scans (Topcon device); (b) scans (Spectralis device)	Moorfields, London	Random split	No	Follow up	Yes	OCT	Urgent referral eye disease
ElTanboly et al. 2016	Deep fusion classification network (DFCN)	No	12	OCT scans		Hold-out method	No	NR	No	OCT	Early DR
Gargeya et al. 2017²⁶	CNN	No	(a) 15,000 (b) 1748; (c) 463	Photographs	(a) EyePACS-1; (b) Messidor-2; (c) E-Opthma	Random split	Yes	Expert consensus	No	Retinal fundus photography	DR
Gomez-Valverde et al. 2019⁵²	VGG-19	No	494	Photographs	ESPERANZA	Random split	No	Expert consensus	Yes	Retinal fundus photographs	Glaucoma suspect or glaucoma
Grassman et al. 2018²⁷	Ensemble:random forest	No	(a) 12,019; (b) 5555	Images	(a) AREDS dataset; (b) KORA dataset	Random split	Yes	Reading centre grader	No	Retinal fundus photography	AMD-AREDS 9 step
Gulshan et al. 2019¹⁷	Inception-v3	Yes	3049	Photographs	Prospective	NA	Yes	Expert consensus	Yes	Retinal fundus photographs	Referable DR
Gulshan et al. 2016²⁸	Inception-v3	No	(a) 8788; (b) 1745	Photographs	(a) EyePACS-1; (b) Messidor-2	Random split	Yes	Reading centre grader	Yes	Retinal fundus photography	Referable DR
Hwang et al. 2019²⁹	(a) ResNet50; (b) VGG-16; (c) Inception-v3; (d) ResNet50; (e) VGG-16; (f) Inception-v3	No	(a–c) 3872; (d–f) 750	Images	(a–c) Department of Ophthalmology of Taipei Veterans General Hospital; (d–f) External validation	Random split	Yes	Expert consensus	Yes	OCT	AMD-AREDS 4 step
Jammal et al. 2019⁵³	ResNet34	No	490	Images		Randomly drawn from test sample	No	Reading centre grader	Yes	Retinal fundus photographs	Glaucomatous optic neuropathy
Kanagasingham et al. 2018²¹	DCNN	Yes	398	Patients	Primary Care Practice, Midland, Western Australia	NA	Yes	Reading centre grader	No	Retinal fundus photography	Referable DR
Karri et al. 2017	GoogLeNet	No	21	Scans	Duke University	Random split	No	NR	No	OCT	(a) DME; (b) dry AMD
Keel et al. 2018¹⁸	Inception-v3	Yes	93	Images	St Vincent’s Hospital Melbourne and University Hospital Geelong, Barwon Health	NA	Yes	Reading centre grader	No	Retinal fundus photography	Referable DR
Keel et al. 2019³⁰	CNN	No	86,202	Photographs	Melbourne Collaborative Cohort Study	Hold-out method	Yes	Expert consensus	No	Retinal fundus photographs	Neovascular AMD
Kermany et al. 2018⁵⁴	Inception-v3	No	(a) 1000; (b–d) 500	Scans	Shiley Eye Institute of the University of California San Diego, the California Retinal Research Foundation, Medical Centre Ophthalmology Associates, the Shanghai First People’s Hospital, and Beijing Tongren Eye Centre	Random split	No	Consensus involving experts and non-experts	Yes	OCT	(a) Choroidal neovascularisation vs DME vs drusen vs normal; (b) choroidal neovascularisation; (c) DME; (d) AMD
Krause et al. 2018³¹	CNN	No	1958	Images	EyePACS-2	Hold-out method	Yes	Expert consensus	No	Retinal fundus photographs	Referable DR
Lee et al. 2017	VGG-16	No	2151	Scans		Random split	No	Routine clinical notes	No	OCT	AMD
Lee et al. 2019	CNN	No	200	Photographs	Seoul National University Hospital	Hold-out method	No	Other imaging technique	No	Retinal fundus photographs	Glaucoma
Li et al. 2018¹⁰⁸	Inception-v3	No	8000	Scans	Guangdong (China)	Random split	No	Expert graders	No	Retinal fundus photography	Glaucomatous optic neuropathy
Li et al. 2019⁵⁵	VGG-16	No	1000	Images	Shiley Eye Institute of the University of California San Diego, the California Retinal Research Foundation, Medical Centre Ophthalmology Associates, the Shanghai First People’s Hospital, and Beijing Tongren Eye Centre	Random split	No	Expert consensus	No	OCT	Choroidal neovascularisation vs DME vs drusen Vs normal
Li et al. 2019	OCT-NET	No	859	Scans	Wenzhou Medical University	Random split	No	Expert graders	No	OCT	Early DR
Li et al. 2019³³	Inception-v3	No	800	Images	Messidor-2	Random split	Yes	Reading centre grader	No	Retinal fundus photographs	Referable DR
Li et al. 2019	ResNet50	No	1635	Images	Shanghai Zhongshan Hospital and the Shanghai First People’s Hospital	Random split	No	Reading centre grader	Yes	OCT	DME
Lin et al. 2019¹⁰⁹	CC-Cruiser	Yes—multicentre RCT	350	Images	Multicentre RCT	NA	NA	Expert consensus	Yes	Slit-lamp photography	Childhood cataracts
Li F et al. 2018	VGG-15	No	300	Images	NR	Random split	No	NR	No	Visual Fields	Glaucoma
Li Z et al. 2018³³	CNN	No	35,201	Photographs	NIEHS, SiMES, AusDiab	Random split	Yes	Reading centre grader	No	Retinal fundus photographs	Referable DR
Liu et al. 2018³⁵	ResNet50	No	(a) 754; (b) 30	Photographs	(a) NR; (b) HRF	Random split	Yes	Reading centre grader	Yes	Retinal fundus photographs	Glaucomatous optic discs
Liu et al. 2019³⁴	CNN	No	(a) 28,569; (b) 20,466; (c) 12,718; (d) 9305; (e) 29,676; (f) 7877	Photographs	(a) Local Validation (Chinese Glaucoma Study Alliance); (b) Beijing Tongren Hospital; (c) Peking University Third Hospital; (d) Harbin Medical University First Hospital; (e) Handan Eye Study; (f) Hamilton Glaucoma Centre	Random split	Yes	Consensus involving experts and non-experts	No	Retinal fundus photographs	Glaucomatous optic neuropathy
Long et al. 2017⁵⁶	DCNN	No	57	Images	Multihospital clinical trial	Hold-out method	No	Expert consensus	Yes	Ocular images	Congenital Cataracts
MacCormick et al. 2019³⁶	DenseNet	No	(a) 130; (b) 159	Images	(a) ORIGA; (b) RIM-ONE	Random split	Yes	(a) NR; (b) expert consensus	No	Retinal fundus photography	Glaucomatous optic discs
Maetshke et al. 2019	3D CNN	No	110	OCT scans	Fivefold cross validation	Random split	No	Follow up	No	OCT	Glaucomatous optic neuropathy
Matsuba et al. 2018⁵⁷	DCNN	No	111	Images	Tsukazaki Hospital	NR	No	Expert consensus + further imaging	Yes	Retinal fundus photography (optos)	Exudative AMD
Medeiros et al. 2019	ResNet34	No	6292	Images	Duke University	Random split	No	Follow up	No	Retinal fundus photography	Glaucomatous optic neuropathy
Motozawa et al. 2019	CNN	No	382	Images	Kobe City Medical Centre	Random split	No	Routine clinical notes	No	OCT	AMD
Muhammad et al. 2017	AlexNet	No	102	Images	NR	NR	No	Expert consensus	No	OCT	Glaucoma suspect or glaucoma
Nagasato et al. 2019	VGG-16	No	466	Images	NR	K-fold cross validation	No	NR	No	Retinal fundus photography (optos)	Retinal vein occlusion
Nagasato et al. 2019⁵⁸	DNN	No	322	Scans	Tsukazaki Hospital and Tokushima University Hospital	K-fold cross validation	No	Expert graders	Yes	OCT	Retinal vein occlusion
Nagasawa et al. 2019	VGG-16	No	378	Images	Tsukazaki Hospital and Tokushima University Hospital	K-fold cross validation	No	Expert graders	No	Retinal fundus photography (optos)	Proliferative diabetic retinopathy
Ohsugi et al. 2017	DCNN	No	166	Images	Tsukazaki Hospital	Random split	No	Expert consensus	No	Retinal fundus photography (optos)	Rhegmatogenous retinal detachment
Peng et al. 2019⁵⁹	Inception-v3	No	900	Images	AREDS	Random split	No	Reading centre grader	Yes	Retinal fundus photography	Age-related macular degeneration-AREDS 4 step
Perdomo et al. 2019	OCT-NET	No	2816	Images	SERI-CUHK data set	Random split	No	Expert graders	No	OCT	DME
Phan et al. 2019	DenseNet201	No	828	Images	Yamanashi Koseiren Hospital		No	Expert consensus + further imaging	No	Retinal fundus photography	Glaucoma
Phene et al. 2019³⁷	Inception-v3	No	(a) 1205; (b) 9642; (c) 346	Images	(a) EyePACS, Inoveon, the United Kingdom Biobank, the Age-Related Eye Disease Study, and Sankara Nethralaya; (b) Atlanta Veterans Affairs (VA) Eye Clinic; (c) Dr. Shroff’s Charity Eye Hospital, New Delhi, India	Random split	Yes	Reading centre grader	Yes	Retinal fundus photographs	Glaucomatous optic neuropathy
Prahs et al. 2017	GoogLeNet	No	5358	Images	Heidelberg Eye Explorer, Heidelberg Engineering	Random split	No	Expert graders	No	OCT	Injection vs No injection for AMD
Raju et al. 2017	CNN	No	53,126	Images	EyePACS-1	Random split	No	NR	No	Retinal fundus photography	Referable DR
Ramachandran et al. 2018³⁸	Visiona intelligent diabetic retinopathy screening platform	No	(a) 485; (b) 1200	Photographs	(a) ODEMS; (b) Messidor	NA	Yes	Expert graders	No	Retinal fundus photographs	Referable DR
Raumviboonsuk et al. 2019³⁹	Inception-v4	No	(a–c) 25,348; (d) 24,332	Images	National screening program for DR in Thailand	NA	Yes	Expert consensus	Yes	Retinal fundus photography	(a) Moderate non-proliferative DR or worse; (b) severe non-proliferative DR or worse; (c) proliferative DR; (d) referable DME
Redd et al. 2018	Inception-v1 and U-Net	No	4861	Images	Multicentre i-ROP study	NR	No	Expert graders + further imaging	No	Retinal fundus photography	Plus disease in ROP
Rogers et al. 2019⁴⁵	Pegasus (ResNet50)	No	94	Photographs	EODAT	NA	Yes	Reading centre grader	Yes	Retinal fundus photographs	Glaucomatous optic neuropathy
Sandhu et al. 2018¹⁹	Deep fusion SNCAE	Yes	160	Scans	University of Waikato	NA	No	Clinical diagnosis	No	Retinal fundus photographs	Non-proliferative DR
Sayres et al. 2019⁴⁰	Inception-v4	No	2000	Images	EyePACS-2	NA	Yes	Expert consensus	Yes	Retinal fundus photographs	Referable DR
Shibata et al. 2018⁶⁰	(a) ResNet; (b) VGG-16	No	110	Images	Matsue Red Cross Hospital	Random split	No	Expert consensus	Yes	Retinal fundus photography	Glaucoma
Stevenson et al. 2019	Inception-v3	No	(a) 2333; (b) 2283; (c) 2105	Photographs	Publicly available databases	Random split	No	Existing diagnosis from source data	No	Retinal fundus photographs	(a) Glaucoma; (b) DR; (c) AMD
Ting et al. 2017⁴¹	VGGNet	No	(a) 71,896; (b) 15,798; (c) 3052; (d) 4512; (e) 1936; (f) 1052; (g) 1968; (h) 2302; (i) 1172; (j) 1254; (k) 7706; (l) 35,948; (m) 35,948	Images	(a) Singapore National Diabetic Retinopathy Screening Program 2014–2015; (b) Guangdong (China); (c) Singapore Malay Eye Study; (d) Singapore Indian Eye Study; (e) Singapore Chinese Eye Study; (f) Beijing Eye Study; (g) African American Eye Disease Study; (h) Royal Victoria Eye and Ear Hospital; (i) Mexican; (j) Chinese University of Hong Kong, (k, l) Singapore National Diabetic Retinopathy Screening Program 2014–2015	Random split	Yes	Expert consensus	No	Retinal fundus photography	Referable DR
Ting et al. 2019⁴²	VGGNet	No	85,902	Images	Combined eight datasets	NA	Yes	Consensus involving experts and non-experts	No	Retinal fundus photography	(a) Any DR; (b) referable DR; (c) vision-threatening DR
Treder et al. 2017	Inception-v3	No	100	Scans	NR	Hold-out method	No	NR	No	OCT	Exudative AMD
van Grinsven et al. 2016⁴⁴	(a) Ses CNN 60; (b) NSesCNN170	No	1200	Images	Messidor	Random split	Yes	Existing diagnosis from source data	Yes	Retinal fundus photographs	Retinal haemorrhage
Verbraak et al. 2019⁴³	AlexNet/VGG	No	1293	Images	Netherlands Star-SHL	NA	Yes	Expert consensus	No	Retinal fundus photography	(a) DR-vision-threatening; (b) DR- more than mild
Xu et al. 2017	CNN	No	200	Photographs	Kaggle	Random split	No	Existing diagnosis from source data	No	Retinal fundus photographs	DR
Yang et al. 2019	VGGNet	No	500	Photographs	Intelligent Ophthalmology Database of Zhejiang Society for Mathematical Medicine in China	Hold-out method	No	Expert consensus	No	Retinal fundus photographs	Referable DR
Yoo et al. 2019	VGG-19	No	900	Scans	Project Macula	Random split	No	NR	No	(a) OCT; (b) retinal fundus photographs	AMD
Zhang et al. 2019⁶¹	VGG-16	No	1742	Images	Telemed-R screening	Random split	No	Expert consensus	Yes	Retinal fundus photographs	ROP
Zheng et al. 2019²⁰	Inception-v3	Yes	102	Scans	Joint Shantou International Eye Centre of Shantou University and the Chinese University of Hong Kong (JSIEC)	Hold-out method	No	NR	No	OCT	Glaucomatous optic neuropathy