Table III.
Diagnostic performance of AI/ML protocols in the included studies
| Study | Sensitivity of the AI/ML protocol (%) | Specificity of the AI/ML protocol (%) | Accuracy of the AI/ML protocol (%) | PPV of the AI/ML protocol (%) | NPV of the AI/ML protocol (%) | Performance of AI/ML diagnosis as compared to human diagnosis |
|---|---|---|---|---|---|---|
| Kok et al13, 1996 | NA | NA | NA | NA | NA | AI is similar to human diagnosis |
| Chang et al14, 1999 | 92 | 84 | 88.4 | NA | NA | AI improves human diagnosis |
| Nieminen et al15, 2002 | NA | 92.5 | NA | 55 | NA | AI is similar to human diagnosis |
| de Veld et al16, 2004 | NA | NA | NA | NA | NA | Comparison not performed |
| Dreiseitl et al9, 2009 | 72 | 82 | NA | NA | NA | Depends on the user’s background |
| Lucidarme et al17, 2010 | 98 | 88 | NA | NA | NA | AI improves human diagnosis |
| Fink et al18, 2017 | 100 | 68.5 | 2.3 | 2.80 | 100 | Comparison not performed |
| Mori et al8, 2018 | NA | NA | 98.1 | NA | 93.7 to 96.5 | AI is better than human diagnosis |
| Walker et al19, 2019 | 86 (system B); 91 (system A)1 | 69 (system B)1 | NA | 88.9 | 88.9 | Comparison not performed |
| Wang et al20, 2019 | NA | NA | NA | NA | NA | AI improves human diagnosis |
| Su et al21, 2019 | NA | NA | NA | NA | NA | AI improves human diagnosis |
| Li et al22, 2019 | 86.2 | NA | NA | 57.0 | NA | AI is better than human diagnosis |
| Hollon et al23, 2020 | NA | NA | 94.6 | NA | NA | AI is better than human diagnosis |
| Wang et al24, 2020 | NA | NA | NA | NA | NA | AI is better than human diagnosis |
| Repici et al25, 2020 | NA | NA | NA | NA | NA | AI improves human diagnosis |
| Gong et al26, 2020 | NA | NA | NA | NA | NA | AI improves human diagnosis |
| Wang et al27, 2020 | NA | NA | NA | NA | NA | AI improves human diagnosis |
| Liu et al28, 2020 | NA | NA | NA | NA | NA | AI improves human diagnosis |
1System A is a deep learning classifier whose outputs from image processing of pigmented skin lesions were converted into sound waves, which were once again classified by system B. PPV: positive predictive value; NPV: negative predictive value