Table 3.
The classification performance of GraphSAGE and other machine learning models on the CAD-RADS model 1 and model 2, in terms of image-wise and subject-wise classification. Bolded values indicate the best performance across the models.
| Methods ** | Feature Selection | Sens. | Spec. | Accu. | AUC | F1-Score | Precision | p-Value * |
|---|---|---|---|---|---|---|---|---|
| CAD-RADS Model 1 (class 0: CAD-RADS ≤ 1; class 1: CAD-RADS ≥ 2) for image-wise classification | ||||||||
| GraphSAGE | all | 0.711 (0.621, 0.786) | 0.697 (0.605, 0.776) | 0.704 (0.644, 0.764) | 0.739 (0.675, 0.804) | 0.711 (0.672, 0.746) | 0.711 (0.621, 0.786) | - |
| LR | CFS | 0.509 (0.418, 0.599) | 0.541 (0.448, 0.632) | 0.525 (0.459, 0.59) | 0.521 (0.445, 0.596) | 0.514 (0.473, 0.555) | 0.537 (0.443, 0.628) | <0.01 |
| LDA | DISR | 0.553 (0.461, 0.641) | 0.468 (0.377, 0.561) | 0.511 (0.446, 0.577) | 0.507 (0.431, 0.583) | 0.546 (0.505, 0.586) | 0.521 (0.432, 0.608) | <0.05 |
| KNN | CFS | 0.158 (0.102, 0.236) | 0.862 (0.785, 0.915) | 0.502 (0.437, 0.568) | 0.527 (0.451, 0.603) | 0.184 (0.152, 0.221) | 0.545 (0.38, 0.702) | <0.01 |
| NB | CFS | 0.491 (0.401, 0.582) | 0.495 (0.403, 0.588) | 0.493 (0.428, 0.559) | 0.52 (0.444, 0.596) | 0.494 (0.453, 0.535) | 0.505 (0.413, 0.596) | <0.01 |
| SVM | all | 0.535 (0.444, 0.624) | 0.569 (0.475, 0.658) | 0.552 (0.486, 0.617) | 0.604 (0.53, 0.678) | 0.541 (0.5, 0.581) | 0.565 (0.471, 0.654) | <0.01 |
| CAD-RADS Model 1 (class 0: CAD-RADS ≤ 1; class 1: CAD-RADS ≥ 2) for subject-wise classification | ||||||||
| GraphSAGE | LAP | 0.747 (0.638, 0.831) | 0.571 (0.455, 0.681) | 0.662 (0.585, 0.739) | 0.769 (0.708, 0.831) | 0.725 (0.679, 0.768) | 0.651 (0.546, 0.743) | - |
| LR | CFS | 0.507 (0.396, 0.617) | 0.543 (0.427, 0.654) | 0.524 (0.443, 0.605) | 0.512 (0.436, 0.588) | 0.514 (0.463, 0.564) | 0.543 (0.427, 0.654) | < 0.01 |
| LDA | DISR | 0.453 (0.346, 0.566) | 0.5 (0.386, 0.614) | 0.476 (0.395, 0.557) | 0.526 (0.45, 0.601) | 0.461 (0.411, 0.512) | 0.493 (0.378, 0.608) | <0.05 |
| KNN | CFS | 0.387 (0.285, 0.5) | 0.657 (0.54, 0.758) | 0.517 (0.436, 0.599) | 0.531 (0.455, 0.607) | 0.411 (0.361, 0.463) | 0.547 (0.415, 0.673) | <0.01 |
| NB | CFS | 0.453 (0.346, 0.566) | 0.514 (0.4, 0.628) | 0.483 (0.401, 0.564) | 0.492 (0.416, 0.568) | 0.462 (0.412, 0.513) | 0.5 (0.384, 0.616) | <0.01 |
| SVM | SVMB | 0.653 (0.541, 0.751) | 0.614 (0.497, 0.72) | 0.634 (0.556, 0.713) | 0.697 (0.629, 0.765) | 0.652 (0.602, 0.698) | 0.645 (0.533, 0.743) | <0.05 |
| CAD-RADS Model 2 (class 0: CAT = 0; class 1: CAT = 1) for image-wise classification | ||||||||
| GraphSAGE | all | 0.544 (0.416, 0.666) | 0.681 (0.606, 0.747) | 0.646 (0.583, 0.709) | 0.692 (0.608, 0.776) | 0.497 (0.442, 0.552) | 0.369 (0.274, 0.476) | - |
| LR | CFS | 0.561 (0.433, 0.682) | 0.5 (0.425, 0.575) | 0.516 (0.45, 0.581) | 0.513 (0.426, 0.601) | 0.466 (0.414, 0.519) | 0.278 (0.205, 0.366) | >0.05 |
| LDA | CFS | 0.544 (0.416, 0.666) | 0.428 (0.355, 0.504) | 0.457 (0.392, 0.523) | 0.497 (0.41, 0.584) | 0.438 (0.387, 0.49) | 0.246 (0.179, 0.328) | >0.05 |
| KNN | CFS | 0.228 (0.138, 0.352) | 0.819 (0.754, 0.87) | 0.668 (0.606, 0.73) | 0.561 (0.473, 0.649) | 0.24 (0.193, 0.294) | 0.302 (0.186, 0.451) | >0.05 |
| NB | LAP | 0.544 (0.416, 0.666) | 0.422 (0.349, 0.498) | 0.453 (0.388, 0.518) | 0.498 (0.411, 0.585) | 0.437 (0.386, 0.489) | 0.244 (0.178, 0.326) | >0.05 |
| SVM | LAP | 0.544 (0.416, 0.666) | 0.488 (0.413, 0.563) | 0.502 (0.437, 0.568) | 0.514 (0.426, 0.601) | 0.451 (0.399, 0.503) | 0.267 (0.195, 0.354) | >0.05 |
| CAD-RADS Model 2 (class 0: CAT = 0; class 1: CAT = 1) for subject-wise classification | ||||||||
| GraphSAGE | CFS | 0.649 (0.488, 782) | 0.75 (0.661, 0.822) | 0.724 (0.651, 0.797) | 0.753 (0.674, 0.832) | 0.603 (0.534, 0.668) | 0.471 (0.341, 0.605) | - |
| LR | CFS | 0.568 (0.409, 0.713) | 0.444 (0.354, 0.538) | 0.476 (0.395, 0.557) | 0.501 (0.414, 0.588) | 0.459 (0.395, 0.523) | 0.259 (0.176, 0.364) | >0.05 |
| LDA | CFS | 0.541 (0.384, 0.69) | 0.463 (0.372, 0.557) | 0.483 (0.401, 0.564) | 0.501 (0.414, 0.588) | 0.442 (0.379, 0.508) | 0.256 (0.173, 0.363) | >0.05 |
| KNN | CFS | 0.243 (0.134, 0.401) | 0.759 (0.671, 0.83) | 0.628 (0.549, 0.706) | 0.572 (0.485, 0.66) | 0.246 (0.189, 0.313) | 0.257 (0.142, 0.421) | >0.05 |
| NB | CMIM | 0.568 (0.409, 0.713) | 0.417 (0.328, 0.511) | 0.455 (0.374, 0.536) | 0.52 (0.432, 0.607) | 0.453 (0.39, 0.517) | 0.25 (0.17, 0.352) | <0.05 |
| SVM | SVMB | 0.595 (0.435, 0.737) | 0.556 (0.462, 0.646) | 0.566 (0.485, 0.646) | 0.565 (0.477, 0.653) | 0.505 (0.439, 0.57) | 0.314 (0.218, 0.43) | >0.05 |
* p-value based on McNemar’s testing. ** Abbreviations: GraphSAGE—the graph sample and aggregate network; LR—logistic regression classifier; LDA: linear discrimation analysis classifier; KNN- K-nearest neightbour classifier; NB—naive Bayesian classifer; SVM—support vector machine classifier.