Table 3.
Comparison of sensitivity, specificity, positive predictive value, negative predictive value, accuracy, and positive prevalence between our INTEGRAL-Radiomics model and the established PanCan Model. We provide point estimates and 95% percentile-based bootstrap confidence intervals for each statistic.
| Probability Threshold | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | Accuracy (%) | Positive Prevalence (%) |
|---|---|---|---|---|---|---|
| INTEGRAL-Radiomics 1,2 | ||||||
| ≥2% | 89.7 (83.8-95.1) | 81.6 (80.3-82.9) | 13.8 (11.4-16.6) | 99.6 (99.4-99.8) | 81.9 (80.6-83.2) | 20.6 (19.4-22.0) |
| ≥5% | 82.2 (75.2-89.4) | 90.9 (89.9-91.8) | 23.0 (19.1-27.4) | 99.4 (99.1-99.6) | 90.7 (89.6-91.6) | 11.4 (10.4-12.5) |
| ≥10% | 74.8 (66.7-82.6) | 94.7 (94.0-95.5) | 31.9 (26.4-37.9) | 99.1 (98.8-99.5) | 94.1 (93.3-94.9) | 7.5 (6.6-8.4) |
| ≥15% | 64.5 (56.1-73.4) | 96.7 (96.1-97.3) | 39.2 (32.4-47.0) | 98.8 (98.4-99.2) | 95.7 (95.0-96.3) | 5.2 (4.5-6.0) |
| ≥20% | 61.7 (53.3-70.8) | 97.8 (97.2-98.3) | 47.5 (39.3-56.2) | 98.7 (98.4-99.1) | 96.6 (96.0-97.2) | 4.1 (3.4-4.8) |
| ≥25% | 55.1 (45.8-64.6) | 98.5 (98.0-98.9) | 54.6 (45.2-64.1) | 98.5 (98.1-98.9) | 97.1 (96.6-97.7) | 3.2 (2.6-3.8) |
| ≥30% | 55.1 (45.8-64.6) | 98.8 (98.5-99.2) | 60.8 (51.4-70.8) | 98.5 (98.1-98.9) | 97.4 (96.9-97.9) | 2.9 (2.3-3.5) |
| PanCan Model 3 | ||||||
| ≥2% | 87.7 (84.7-90.6) | 64.5 (63.5-65.5) | 10.9 (9.9-12.1) | 99.1 (98.8-99.3) | 65.6 (64.7-66.6) | 38.0 (37.0-39.0) |
| ≥5% | 80.6 (76.9-84.0) | 79.8 (78.9-80.7) | 16.6 (15.0-18.3) | 98.8 (98.5-99.0) | 79.9 (79.0-80.7) | 23.0 (22.2-23.9) |
| ≥10% | 72.3 (68.1-76.5) | 88.0 (87.3-88.7) | 23.0 (20.6-25.6) | 98.5 (98.2-98.7) | 87.3 (86.5-88.0) | 14.9 (14.1-15.6) |
| ≥15% | 65.0 (60.6-69.4) | 91.6 (91.0-92.2) | 27.7 (24.9-30.7) | 98.1 (97.8-98.4) | 90.3 (89.7-91.0) | 11.1 (10.4-11.7) |
| ≥20% | 58.3 (53.6-63.0) | 93.7 (93.2-94.2) | 31.5 (28.2-34.9) | 97.8 (97.5-98.2) | 92.0 (91.5-92.6) | 8.8 (8.1-9.3) |
| ≥25% | 51.7 (47.0-56.3) | 94.9 (94.5-95.4) | 33.7 (30.0-37.4) | 97.5 (97.2-97.9) | 92.9 (92.4-93.4) | 7.3 (6.7-7.8) |
| ≥30% | 43.9 (39.1-48.7) | 96.1 (95.6-96.5) | 35.6 (31.4-39.7) | 97.2 (96.8-97.5) | 93.6 (93.1-94.1) | 5.8 (5.3-6.3) |
Abbreviations: NPV, negative predictive value; PPV, positive predictive value.
Note: Sensitivity is the proportion of malignant nodules correctly identified as malignant. Specificity is the proportion of benign nodules correctly identified as benign. PPV is the proportion of positive predictions that are malignant nodules. NPV is the proportion of negative predictions that are benign nodules. Accuracy is the total number of correct predictions out of the total number of nodules. Positive prevalence is the proportion of positive predictions divided by the total number of predictions.
The radiomics model was evaluated in the 20% hold-out test data not used for model development (N = 3,363).
Our model was chosen based on K-fold cross-validations using the training sample, and performance metrics are reported based on the hold-out test sample. Our final model was a LASSO model with a penalty (lambda) value of 0.000442 and retained 142 predictors in the model.
The PanCan Model was evaluated in the entire eligible set of participants from IELCAP-Toronto, NLST, and PLuSS (N = 8,622).