Table 2.
p-values from pair-wise comparisons between the AUC-ROCs on the testing sets: 1) RNN versus ElasticNet, 2) NN versus ElasticNet. Three methods of comparing classifier performance were applied, namely Net Reclas sification Improvement (NRI), Integrated Discrimination Improvement (IDI) and pauc43. p-values below 0.05 are considered significant, indicating statistically significant improvement in classification performance using RNN over ElasticNet, etc.
Number of Days | RNN vs. ElasticNet Neural Network vs. ElasticNet | |||||
NRI | IDI | pAUC | NRI | IDI | pAUC | |
1 | 0.71 | 0.53 | 0.84 | 0.96 | 0.91 | 0.98 |
2 | 0.96 | 0.77 | 0.98 | 0.22 | 0.21 | 0.52 |
3 | 0.21 | 0.21 | 0.78 | 0.14 | 0.13 | 0.47 |
4 | 0.09 | 0.08 | 0.54 | 0.09 | 0.08 | 0.42 |
5 | 0.88 | 0.87 | 0.94 | 0.36 | 0.41 | 0.60 |
6 | 0.04 | 0.02 | 0.28 | 0.03 | 0.008 | 0.14 |
7 | 0.02 | 0.006 | 0.16 | 0.05 | 0.03 | 0.16 |
8 | 0.04 | 0.02 | 0.25 | 0.07 | 0.05 | 0.30 |
9 | 0.07 | 0.05 | 0.32 | 0.13 | 0.12 | 0.36 |
10 | 0.13 | 0.12 | 0.49 | 0.41 | 0.54 | 0.67 |
11 | 0.41 | 0.41 | 0.56 | 0.12 | 0.12 | 0.35 |
12 | 0.41 | 0.40 | 0.56 | 0.18 | 0.2 | 0.49 |
13 | 0.49 | 0.50 | 0.64 | 0.29 | 0.41 | 0.46 |
14 | 0.14 | 0.14 | 0.43 | 0.94 | 0.80 | 0.97 |
15 | 0.14 | 0.13 | 0.42 | 0.33 | 0.42 | 0.62 |