Table 3.
The performance comparison experiment of Interpretable Eye Diseases Screening System (IEDSS) and classic algorithms.
| Multi-class ResNetXt-50 |
Multi-label ResNetXt-50 |
Multi-label ViT |
CAFPN+SVM |
IEDSS |
|||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ACC (%)(95% CI) | AUROC(%)(95% CI) | F1 (95% CI) | ACC (%)(95% CI) | AUROC(%)(95% CI) | F1 (95% CI) | ACC (%)(95% CI) | AUROC(%)(95% CI) | F1 (95% CI) | ACC (%)(95% CI) | AUROC(%)(95% CI) | F1 (95% CI) | ACC (%)(95% CI) | AUROC(%)(95% CI) | F1 (95% CI) | |
| wet-AMD | 90·63(89·79-91·48) | 99·12(98·80-99·44) | 0·5052(0·4858-0·5247) | 92·08(91·16-93·00) | 93·39(92·54-94·23) | 0·6225(0·6036-0·6413) | 91·65(90·71-92·60) | 96·17(95·51-96·82) | 0·6416(0·6229-0·6602) | 92·20(91·28-93·11) | 96·62(96·00-97·24) | 0·6632(0·6448-0·6816) | 95·34(94·63-96·06) | 98·84(98·48-99·20) | 0·7813(0·7759-0·7866) |
| BRVO | 94·98(94·34-95·61) | 99·39(99·12-99·66) | 0·6514(0·6329-0·6700) | 95·71(95·02-96·40) | 98·23(97·78-98·68) | 0·7482(0·7313-0·7651) | 96·92(96·33-97·5) | 98·46(98·04-98·88) | 0·8097(0·7944-0·8250) | 95·62(94·92-96·31) | 98·62(98·22-99·02) | 0·7504(0·7336-0·7673) | 97·55(97·02-98·08) | 99·61(99·40-99·82) | 0·8469(0·8422-0·8516) |
| CRVO | 96·09(95·53-96·66) | 98·29(97·84-98·73) | 0·5762(0·5570-0·5954) | 90·23(89·22-91·24) | 94·14(93·34-94·94) | 0·4590(0·4396-0·4784) | 89·23(88·18-90·29) | 94·75(93·99-95·51) | 0·4385(0·4192-0·4578) | 91·53(90·58-92·48) | 96·97(96·39-97·56) | 0·5035(0·4841-0·523) | 94·25(93·46-95·05) | 98·61(98·21-99·01) | 0·6245(0·6182-0·6308) |
| CSCR | 91·97(91·18-92·76) | 98·23(97·78-98·68) | 0·3624(0·3437-0·3811) | 91·05(90·08-92·02) | 94·76(94·00-95·52) | 0·4825(0·4631-0·502) | 91·84(90·9-92·77) | 93·34(92·49-94·19) | 0·5018(0·4824-0·5213) | 93·62(92·79-94·45) | 96·56(95·94-97·18) | 0·5903(0·5711-0·6094) | 95·68(94·98-96·37) | 98·16(97·70-98·62) | 0·7088(0·7028-0·7147) |
| DR | 87·80(86·85-88·75) | 96·65(96·03-97·26) | 0·7470(0·7301-0·7640) | 89·23(88·18-90·29) | 94·11(93·30-94·91) | 0·8429(0·8287-0·8571) | 87·72(86·6-88·84) | 92·78(91·90-93·66) | 0·8227(0·8078-0·8376) | 91·53(90·58-92·48) | 96·35(95·71-96·99) | 0·8801(0·8675-0·8928) | 96·07(95·41-96·73) | 98·75(98·37-99·13) | 0·9443(0·9414-0·9473) |
| EMM | 92·26(91·48-93·03) | 93·55(92·71-94·38) | 0·5537(0·5344-0·5731) | 93·80(92·98-94·62) | 96·24(95·59-96·89) | 0·6861(0·6680-0·7041) | 91·84(90·9-92·77) | 95·98(95·31-96·65) | 0·6260(0·6072-0·6449) | 93·44(92·59-94·28) | 97·14(96·57-97·71) | 0·7031(0·6854-0·7209) | 97·67(97·16-98·19) | 99·35(99·08-99·62) | 0·8824(0·8783-0·8866) |
| MH | 97·30(96·83-97·77) | 93·97(93·16-94·78) | 0·6720(0·6537-0·6903) | 97·01(96·43-97·59) | 99·59(99·37-99·81) | 0·7227(0·7053-0·7401) | 97·64(97·12-98·16) | 99·21(98·91-99·51) | 0·7784(0·7622-0·7946) | 98·61(98·21-99·01) | 99·59(99·37-99·81) | 0·8571(0·8435-0·8708) | 99·09(98·77-99·42) | 99·74(99·57-100·00) | 0·9045(0·9006-0·9083) |
| RT | 95·46(94·85-96·06) | 97·10(96·53-97·68) | 0·7708(0·7544-0·7871) | 93·11(92·24-93·97) | 97·21(96·65-97·78) | 0·7625(0·7459-0·7791) | 92·44(91·54-93·34) | 97·43(96·90-97·97) | 0·7596(0·743-0·7763) | 93·95(93·14-94·76) | 97·48(96·95-98·02) | 0·7984(0·7828-0·814) | 96·73(96·13-97·34) | 99·26(98·97-99·70) | 0·8846(0·8805-0·8888) |
| VH | 97·19(96·71-97·67) | 94·43(93·65-95·21) | 0·7500(0·7331-0·7669) | NA | NA | NA | NA | NA | NA | NA | NA | NA | 99·39(99·16-99·61) | NA | 0·9333(0·9301-0·9366) |
| RD | 97·26(96·78-97·73) | 93·13(92·27-93·99) | 0·9297(0·9198-0·9397) | NA | NA | NA | NA | NA | NA | NA | NA | NA | 99·65(99·48-99·82) | NA | 0·9905(0·9893-0·9918) |
| RMO | 96·20(95·65-96·76) | 93·90(93·08-94·72) | 0·6826(0·6644-0·7007) | NA | NA | NA | NA | NA | NA | NA | NA | NA | 99·46(99·24-99·67) | NA | 0·9395(0·9364-0·9426) |
| ODE | 95·46(94·85-96·06) | 95·63(94·93-96·32) | 0·4838(0·4643-0·5032) | NA | NA | NA | NA | NA | NA | NA | NA | NA | 99·52(99·28-99·75) | NA | 0·9292(0·9259-0·9325) |
| OA | 93·73(93·02-94·43) | 98·80(98·43-99·17) | 0·3755(0·3567-0·3944) | NA | NA | NA | NA | NA | NA | NA | NA | NA | 99·73(99·55-99·91) | NA | 0·9557(0·953-0·9583) |
| SG | 89·38(88·49-90·28) | 97·79(97·29-98·29) | 0·3858(0·3668-0·4047) | NA | NA | NA | NA | NA | NA | NA | NA | NA | 99·24(98·95-99·54) | NA | 0·9383(0·9351-0·9414) |
The multi-label models including Multi-label ResNetXt-50, Multi-label ViT and CAFPN+SVM were not tested on the multi-class labels, including VH, RD, RMO, ODE, OA, SG.
wet-AMD, wet age-related macular degeneration; BRVO, branch retinal vein occlusion; CRVO, central retinal vein occlusion; CSCR, central serous chorioretinopathy; DR, diabetic retinopathy; EMM, epimacular membrane; MH, macular Hole; RT, retinal tears; VH, vitreous hemorrhage; RD, retinal detachment; RMO, refractive media opacity; ODE, optic disc edema; OA, optic atrophy; SG, suspected glaucoma; CAFPN, channel-attention feature pyramid network; SVM, support vector machine; NA, not applicable.