Table 3.
The detailed information of performance comparison (TOP-Net vs other models).
| Forecast range and model | AUROCa (%) (95% CI) | Accuracy (%) | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | F1 score (%) | Precision (%) | |||||||
| 0 hours |
|
|
|
|
|
|
|||||||
|
|
TOP-Net | 95.5 (94.2-96.8) | 90.1 | 89.1 (81.9-91.8) | 92.1 (85.9-94.3) | 89.8 | 90.5 | ||||||
| CNNb | 94.5 (93.0-96.0) | 89.3 | 85.9 (80.7-89.4) | 92.3 (87.0-95.1) | 88.9 | 92.1 | |||||||
| LSTMc | 94.4 (92.9-96.0) | 89.8 | 88.9 (83.9-91.8) | 90.8 (81.1-93.8) | 89.4 | 89.9 | |||||||
| XGBoostd | 93.2 (91.5-94.9) | 88.3 | 81.9 (75.7-85.6) | 93.0 (87.4-96.0) | 87.9 | 94.9 | |||||||
| MLPe | 93.0 (91.3-94.8) | 87.9 | 85.6 (80.2-88.9) | 89.9 (84.6-93.2) | 87.5 | 89.5 | |||||||
| Random forest | 92.3 (90.5-94.2) | 87.3 | 85.1 (80.2-88.8) | 89.0 (82.1-92.8) | 86.8 | 88.6 | |||||||
| 2 hours |
|
|
|
|
|
|
|||||||
|
|
TOP-Net | 85.6 (83.2-88.0) | 79.6 | 77.6 (70.8-81.3) | 81.6 (74.2-85.1) | 79.1 | 80.6 | ||||||
| CNN | 84.6 (82.1-87.1) | 77.6 | 78.6 (71.3-83.2) | 77.8 (71.2-81.4) | 77.1 | 75.6 | |||||||
| LSTM | 85.1 (82.7-87.5) | 78.2 | 88.6 (81.0-92.0) | 67.4 (56.8-71.5) | 77.7 | 76.8 | |||||||
| XGBoost | 83.8 (81.2-86.3) | 78.0 | 74.5 (66.7-79.1) | 80.9 (73.9-84.5) | 77.4 | 80.5 | |||||||
| MLP | 83.9 (81.4-86.4) | 77.5 | 78.3 (71.3-82.2) | 77.7 (69.9-82.0) | 77.0 | 75.8 | |||||||
| Random forest | 82.8 (80.2-85.4) | 77.7 | 71.5 (63.6-76.6) | 82.3 (76.4-86.0) | 77.1 | 83.7 | |||||||
| 4 hours |
|
|
|
|
|
|
|||||||
|
|
TOP-Net | 83.3 (80.7-85.8) | 76.3 | 83.5 (75.5-85.9) | 72.2 (63.8-74.7) | 75.8 | 69.4 | ||||||
| CNN | 80.9 (78.2-83.7) | 75.2 | 71.5 (63.6-76.3) | 78.8 (70.0-82.5) | 74.5 | 77.8 | |||||||
| LSTM | 81.9 (79.2-84.5) | 74.2 | 73.1 (65.5-77.9) | 76.3 (69.6-80.1) | 73.6 | 74.1 | |||||||
| XGBoost | 80.4 (77.7-83.2) | 73.4 | 68.1 (60.2-72.7) | 78.5 (72.0-82.8) | 72.6 | 77.8 | |||||||
| MLP | 80.1 (77.3-82.8) | 72.9 | 73.9 (66.9-78.7) | 72.0 (65.1-76.3) | 72.2 | 70.6 | |||||||
| Random forest | 79.0 (76.1-81.9) | 73.3 | 64.5 (60.6-71.4) | 79.9 (73.4-84.8) | 72.4 | 82.5 | |||||||
| 6 hours |
|
|
|
|
|
|
|||||||
|
|
TOP-Net | 79.6 (76.8-82.4) | 72.1 | 75.3 (66.3-79.3) | 72.0 (64.5-75.8) | 71.8 | 68.6 | ||||||
| CNN | 78.3 (75.4-81.1) | 70.9 | 79.3 (72.8-83.7) | 64.1 (57.1-69.1) | 70.5 | 63.5 | |||||||
| LSTM | 78.7 (75.9-81.5) | 72.5 | 74.0 (67.0-78.4) | 71.8 (64.1-76.0) | 72.2 | 70.5 | |||||||
| XGBoost | 76.1 (73.1-79.0) | 69.1 | 76.3 (69.5-75.4) | 64.1 (55.1-68.9) | 68.4 | 62.0 | |||||||
| MLP | 76.7 (73.8-79.6) | 70.6 | 71.9 (65.1-76.7) | 69.4 (61.7-74.5) | 70.1 | 68.4 | |||||||
| Random forest | 74.4 (71.4-77.5) | 67.2 | 69.1 (59.0-74.3) | 66.7 (59.3-70.6) | 66.4 | 63.9 | |||||||
aAUROC: area under the receiver operating characteristic curve.
bCNN: convolutional neural network.
cLSTM: long short-term memory.
dXGBoost: extreme gradient boosting.
eMLP: multilayer perceptron.