Table 4.
% of observed mortality patients correctly identified, F1 Score, sensitivity, and specificity performance of ASA; POSPOM; logistic regression model and DNN model with 87 features; logistic regression model and DNN model with reduced feature set and ASA; and logistic regression model and DNN model with reduced feature set and POSPOM at different thresholds. Results for best thresholds chosen by 1) highest % of observed mortality and 2) highest F1 score.
| ASA | POSPOM | ||||||||
|---|---|---|---|---|---|---|---|---|---|
|
| |||||||||
| Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) | Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) |
| 3 | 85 (97.7%) | 0.02 (0.02 – 0.03) | 0.98 (0.94 – 1) | 0.44 (0.43 – 0.45) | 10 | 81 (93.1%) | 0.02 (0.01 – 0.02) | 0.93 (0.87 – 0.98) | 0.23 (0.22 – 0.24) |
| 5 | 14 (16.1%) | 0.24 (0.14 – 0.35) | 0.16 (0.09 – 0.25) | 1 (1 – 1) | 20 | 31 (35.6%) | 0.05 (0.03 – 0.07) | 0.36 (0.25 – 0.47) | 0.91 (0.90 – 0.91) |
|
| |||||||||
| Logistic Regression with 87 features | DNN with 87 features | ||||||||
|
| |||||||||
| Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) | Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) |
|
| |||||||||
| 0.00015 | 86 (98.9%) | 0.01 (0.01 – 0.02) | 0.99 (0.96 – 1) | 0.003 (0.002 – 0.004) | 0.05 | 86 (98.9%) | 0.02 (0.01 – 0.02) | 0.99 (0.96 – 1) | 0.20 (0.20 -0.21) |
| 0.1 | 28 (32.2%) | 0.24 (0.16 – 0.30) | 0.32 (0.22 – 0.42) | 0.99 (0.99 – 0.99) | 0.3 | 35 (40.2%) | 0.23 (0.17 – 0.30) | 0.40 (0.30 – 0.51) | 0.99 (0.98 – 0.99) |
|
| |||||||||
| Logistic Regression with Reduced Feature Set & ASA | DNN with Reduced Feature Set & ASA | ||||||||
|
| |||||||||
| Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) | Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) |
|
| |||||||||
| 0.0034 | 84 (96.6%) | 0.04 (0.03 – 0.05) | 0.97 (0.92 – 1) | 0.64 (0.63 – 0.65) | 0.22 | 84 (96.6%) | 0.04 (0.03 – 0.05) | 0.97 (0.92 – 1) | 0.64 (0.64 – 0.65) |
| 0.1 | 30 (34.5%) | 0.26 (0.18 – 0.33) | 0.34 (0.24 – 0.44) | 0.99 (0.99 – 0.99) | 0.4 | 15 (17.2%) | 0.22 (0.12- 0.30) | 0.17 (0.09 – 0.25) | 1 (1 – 1) |
|
| |||||||||
| Logistic Regression with Reduced Feature Set & POSPOM | DNN with Reduced Feature Set & POSPOM | ||||||||
|
| |||||||||
| Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) | Threshold | # Obs. Mort. (%) | F1 (95% CI) | Sens. (95% CI) | Spec. (95% CI) |
|
| |||||||||
| 0.002 | 85 (97.7%) | 0.03 (0.02 – 0.03) | 0.98 (0.94 – 1) | 0.48 (0.48 – 0.49) | 0.2 | 84 (96.6%) | 0.04 (0.03 – 0.04) | 0.97 (0.92 -1) | 0.63 (0.63 – 0.64) |
| 0.1 | 26 (29.9%) | 0.22 (0.15 – 0.29) | 0.30 (0.20 – 0.39) | 0.99 (0.99 – 0.99) | 0.3 | 40 (46%) | 0.18 (0.13 – 0.22) | 0.46 (0.36 – 0.56) | 0.97 (0.97 – 0.98) |
POSPOM: Preoperative Score to Predict Postoperative Mortality; CI: Confidence Interval; DNN: Deep neural network.