Table 4.
Category-wise performance of the human readers in session 1 (unaided session) and session 2 (aided session) in 2 external validation datasets.
| Hospital A | |||||||
|---|---|---|---|---|---|---|---|
| Reader | Category | AUC [95% CI] | Sensitivity [95% CI] | Specificity [95% CI] | |||
| Unaided Session | Aided Session | Unaided Session | Aided Session | Unaided Session | Aided Session | ||
| R1 | Lungs | 0.776 [0.738,0.815] | 0.762 [0.723, 0.802] |
0.573 [0.497, 0.651] |
0.537 [0.460, 0.616] |
0.979 [0.969, 0.989] |
0.987 [0.979, 0.994] |
| Pleura | 0.869 [0.819,0.917] |
0.873 [0.823, 0.921] |
0.767 [0.671, 0.863] |
0.786 [0.685, 0.882] |
0.971 [0.959, 0.981] |
0.9602 [0.946, 0.973] |
|
| Cardiac | 0.809 [0.713,0.897] |
0.916 [0.858, 0.956] |
0.640 [0.444, 0.815] |
0.929 [0.813, 1.000] |
0.978 [0.968, 0.987] |
0.9019 [0.884, 0.921] |
|
| Aggregate | 0.843 [0.812, 0.874] |
0.861 [0.831, 0.888] |
0.723 [0.662, 0.785] |
0.821 [0.767, 0.871] |
0.963 [0.949, 0.975] |
0.900 [0.878, 0.921] |
|
| R2 | Lungs | 0.804 [0.766,0.842] |
0.863 [0.830, 0.895] |
0.632 [0.557, 0.707] |
0.793 [0.731, 0.856] |
0.976 [0.965, 0.986] |
0.933 [0.914, 0.949] |
| Pleura | 0.839 [0.784,0.892] |
0.872 [0.819,0.919] |
0.687 [0.579, 0.795] |
0.786 [0.682, 0.881] |
0.989 [0.982, 0.996] |
0.959 [0.945, 0.971] |
|
| Cardiac | 0.889 [0.813,0.959] |
0.922 [0.867, 0.962] |
0.807 [0.655, 0.950] |
0.929 [0.821, 1.000] |
0.971 [0.959, 0.982] |
0.915 [0.897, 0.932] |
|
| Aggregate | 0.881 [0.854, 0.908] |
0.901 [0.879, 0.923] |
0.802 [0.749, 0.855] |
0.915 [0.876, 0.950] |
0.959 [0.945, 0.973] |
0.888 [0.864, 0.909] |
|
| R3 | Lungs | 0.755 [0.714,0.794] | 0.809 [0.771, 0.846] |
0.589 [0.510, 0.669] |
0.669 [0.593, 0.741] |
0.920 [0.901, 0.938] |
0.949 [0.934, 0.964] |
| Pleura | 0.815 [0.760,0.868] |
0.874 [0.824, 0.921] |
0.644 [0.535, 0.750] |
0.786 [0.687, 0.878] |
0.986 [0.979, 0.993] |
0.962 [0.949, 0.975] |
|
| Cardiac | 0.912 [0.844,0.967] |
0.8994 [0.831, 0.958] |
0.886 [0.750, 1.000] |
0.859 [0.722, 0.969] |
0.939 [0.923, 0.954] |
0.939 [0.923, 0.953] |
|
| Aggregate | 0.8352 [0.806, 0.867] |
0.869 [0.839, 0.896] |
0.786 [0.732, 0.844] |
0.815 [0.759, 0.866] |
0.884 [0.859, 0.906] |
0.923 [0.905, 0.941] |
|
| Hospital B | |||||||
| Reader | Category | AUC [95% CI] | Sensitivity [95% CI] | Specificity [95% CI] | |||
| Unaided Session | Aided Session | Unaided Session | Aided Session | Unaided Session | Aided Session | ||
| R1 | Lungs | 0.844 [0.762,0.919] |
0.789 [0.705, 0.870] |
0.723 [0.567, 0.871] |
0.613 [0.447, 0.776] |
0.966 [0.932, 0.992] |
0.966 [0.932, 0.992] |
| Pleura | 0.859 [0.724,0.987] |
0.901 [0.783, 0.987] |
0.740 [0.467, 1.000] |
0.840 [0.600, 1.00] |
0.979 [0.953, 1.000] |
0.962 [0.928, 0.987] |
|
| Cardiac | 0.678 [0.500,0.875] |
0.929 [0.779,0.994] | 0.357 [0.000, 0.750] |
0.893 [0.600, 1.000] |
0.998 [0.993, 1.000] |
0.965 [0.933, 0.993] |
|
| Aggregate | 0.884 [0.819, 0.942] |
0.885 [0.818, 0.944] |
0.791 [0.667, 0.903] |
0.816 [0.688, 0.927] |
0.976 [0.948, 1.000] |
0.953 [0.912,0.991] |
|
| R2 | Lungs | 0.783 [0.697,0.867] |
0.835 [0.751, 0.910] |
0.591 [0.419, 0.757] |
0.715 [0.556, 0.862] |
0.974 [0.942, 1.000] |
0.954 [0.915, 0.985] |
| Pleura | 0.749 [0.600,0.897] |
0.895 [0.764, 0.989] |
0.500 [0.200, 0.800] |
0.820 [0.571, 1.000] |
0.998 [0.987, 1.000] |
0.969 [0.938, 0.993] |
|
| Cardiac | 0.875 [0.684,0.990] |
0.926 [0.773, 0.993] |
0.786 [0.400, 1.000] |
0.893 [0.600, 1.000] |
0.964 [0.932, 0.987] |
0.959 [0.924, 0.987] |
|
| Aggregate | 0.881 [0.813, 0.942] |
0.923 [0.870, 0.969] |
0.798 [0.667, 0.915] |
0.902 [0.806, 0.977] |
0.964 [0.926, 0.992] |
0.945 [0.902, 0.983] |
|
| R3 | Lungs | 0.563 [0.499,0.632] |
0.816 [0.733, 0.892] |
0.183 [0.063, 0.314] |
0.729 [0.571, 0.871] |
0.944 [0.901, 0.977] |
0.901 [0.849, 0.951] |
| Pleura | 0.627 [0.500,0.769] |
0.894 [0.771, 0.996] |
0.260 [0.000, 0.546] |
0.800 [0.556, 1.000] |
0.995 [0.979, 1.000] |
0.988 [0.966, 1.000] |
|
| Cardiac | 0.670 [0.490,0.875] |
0.923 [0.769, 0.990] |
0.357 [0.000, 0.750] |
0.893 [0.600, 1.000] |
0.984 [0.960, 1.000] |
0.954 [0.918, 0.982] |
|
| Aggregate | 0.600 [0.532, 0.676] |
0.872 [0.807, 0.928] |
0.258 [0.128, 0.395] |
0.853 [0.738, 0.949] |
0.943 [0.901, 0.982] |
0.892 [0.835, 0.945] |
|