Table 4.
Prediction ability of the reference and four machine learning models using consecutive 2-year data for HNHC patients.
| Outcome | c-statistics | P-valueb | Sensitivity | Specificity | PPV | NPV | PLR | NLR |
|---|---|---|---|---|---|---|---|---|
| Reference modela | 0.83 (0.82–0.84) | [Reference] | 0.67 (0.64–0.69) | 0.87 (0.86–0.87) | 0.21 (0.20–0.22) | 0.98 (0.98–0.98) | 4.9 (4.7–5.1) | 0.39 (0.36–0.41) |
| Logistic regression with Lasso regularization | 0.83 (0.82–0.84) | 0.93 | 0.69 (0.67–0.71) | 0.85 (0.84–0.85) | 0.19 (0.18–0.20) | 0.98 (0.98–0.98) | 4.5 (4.3–4.7) | 0.37 (0.35–0.40) |
| Random forest | 0.84 (0.83–0.85) | 0.22 | 0.66 (0.64–0.68) | 0.88 (0.88–0.89) | 0.23 (0.22–0.24) | 0.98 (0.98–0.98) | 5.6 (5.4–5.9) | 0.39 (0.36–0.41) |
| Gradient-boosted decision tree | 0.84 (0.83–0.86) | 0.05 | 0.68 (0.66–0.71) | 0.88 (0.87–0.88) | 0.22 (0.21–0.24) | 0.98 (0.98–0.98) | 5.5 (5.3–5.7) | 0.36 (0.34–0.39) |
| Deep neural network | 0.84 (0.83–0.85) | 0.18 | 0.69 (0.67–0.71) | 0.86 (0.86–0.87) | 0.21 (0.20–0.22) | 0.98 (0.98–0.98) | 5.0 (4.8–5.2) | 0.36 (0.34–0.39) |
PPV positive predictive value, NPV negative predictive value, PLR positive likelihood ratio, NLR negative likelihood ratio.
aWe used a non-penalized logistic regression model as the reference model.
bWe compared the area under the curve between each machine-learning-based prediction model and the logistic regression model (the reference model) using the DeLong’s test.