Table 2.
Predictive performance of six types of models
| Multimodal training cohort | Radiology testing cohort | Pathology testing cohort | Prospective validation cohort | |
|---|---|---|---|---|
| Single-R model | ||||
| AUC | 0.904 (0.880–0.924) | 0.847 (0.809–0.879) | – | 0.936 (0.874–0.973) |
| Accuracy | 0.838 (0.810–0.864) | 0.822 (0.782–0.857) | – | 0.860 (0.782–0.918) |
| Sensitivity | 0.814 (0.778–0.847) | 0.719 (0.639–0.790) | – | 0.818 (0.714–0.897) |
| Specificity | 0.862 (0.813–0.902) | 0.864 (0.818–0.901) | – | 0.946 (0.818–0.993) |
| PPV | 0.922 (0.896–0.941) | 0.729 (0.664–0.786) | – | 0.969 (0.891–0.992) |
| NPV | 0.699 (0.658–0.737) | 0.858 (0.822–0.887) | – | 0.714 (0.607–0.802) |
| Single-P model | ||||
| AUC | 0.867 (0.841–0.891) | – | 0.859 (0.807–0.901) | 0.883 (0.809–0.935) |
| Accuracy | 0.810 (0.781–0.838) | – | 0.787 (0.728–0.838) | 0.798 (0.713–0.868) |
| Sensitivity | 0.804 (0.767–0.838) | – | 0.680 (0.579–0.770) | 0.766 (0.656–0.855) |
| Specificity | 0.806 (0.752–0.853) | – | 0.877 (0.808–0.928) | 0.892 (0.746–0.970) |
| PPV | 0.893 (0.866–0.915) | – | 0.810 (0.725–0.873) | 0.937 (0.853–0.974) |
| NPV | 0.673 (0.631–0.713) | – | 0.781 (0.727–0.827) | 0.647 (0.546–0.736) |
| Dual-RC model | ||||
| AUC | 0.905 (0.882–0.925) | 0.846 (0.808–0.878) | – | 0.929 (0.865–0.968) |
| Accuracy | 0.839 (0.811–0.865) | 0.810 (0.770–0.846) | – | 0.860 (0.782–0.918) |
| Sensitivity | 0.844 (0.809–0.874) | 0.706 (0.624–0.778) | – | 0.818 (0.714–0.897) |
| Specificity | 0.850 (0.800–0.891) | 0.864 (0.818–0.901) | – | 0.946 (0.818–0.993) |
| PPV | 0.918 (0.893–0.938) | 0.725 (0.660–0.783) | – | 0.969 (0.891–0.992) |
| NPV | 0.731 (0.688–0.770) | 0.852 (0.817–0.881) | – | 0.714 (0.607–0.802) |
| Dual-PC model | ||||
| AUC | 0.874 (0.848–0.896) | – | 0.869 (0.819–0.910) | 0.876 (0.802–0.931) |
| Accuracy | 0.812 (0.782–0.839) | – | 0.783 (0.724–0.834) | 0.781 (0.694–0.853) |
| Sensitivity | 0.804 (0.767–0.838) | – | 0.870 (0.788–0.929) | 0.701 (0.586–0.800) |
| Specificity | 0.806 (0.752–0.853) | – | 0.739 (0.654–0.812) | 0.946 (0.818–0.993) |
| PPV | 0.893 (0.866–0.915) | – | 0.719 (0.655–0.775) | 0.964 (0.874–0.991) |
| NPV | 0.673 (0.631–0.713) | – | 0.881 (0.815–0.925) | 0.603 (0.517–0.684) |
| Dual-RP model | ||||
| AUC | 0.919 (0.898–0.938) | – | – | 0.942 (0.883–0.977) |
| Accuracy | 0.835 (0.807–0.861) | – | – | 0.868 (0.792–0.924) |
| Sensitivity | 0.765 (0.725–0.801) | – | – | 0.818 (0.714–0.897) |
| Specificity | 0.937 (0.899–0.963) | – | – | 0.973 (0.858–0.999) |
| PPV | 0.960 (0.938–0.975) | – | – | 0.984 (0.901–0.998) |
| NPV | 0.666 (0.629–0.700) | – | – | 0.720 (0.615–0.806) |
| Multi-RPC model | ||||
| AUC | 0.921 (0.899–0.939) | – | – | 0.939 (0.878–0.975) |
| Accuracy | 0.842 (0.814–0.867) | – | – | 0.860 (0.782–0.918) |
| Sensitivity | 0.796 (0.759–0.831) | – | – | 0.831 (0.729–0.907) |
| Specificity | 0.897 (0.853–0.932) | – | – | 0.973 (0.858–0.999) |
| PPV | 0.939 (0.915–0.957) | – | – | 0.985 (0.902–0.998) |
| NPV | 0.688 (0.649–0.725) | – | – | 0.735 (0.627–0.820) |
Data are presented as mean (95% CI).
AUC, area under the curve; NPV, negative predictive value; PPV, positive predictive value; Single-R model, single-modality radiomic model; Single-P model, single-modality pathologic model; Dual-RC model, radiomics + clinical indicators dual-modality model; Dual-PC model, pathology + clinical indicators dual-modality model; Dual-RP model, radiomics + pathology dual-modality model; Multi-RPC model, radiomics + pathology + clinical indicators multimodality model.