Table 2.
Performance of XGBa, RFb, and L1c obtained with 10-fold cross-validation.
|
|
AUC-ROCd, mean (95% CI) | AUC-PRCe, mean (95% CI) | Precision, mean (95% CI) | Recall, mean (95% CI) | F1-score, mean (95% CI) | ||||||
| XGB model | |||||||||||
|
|
Wf2 | 0.85 (0.84-0.85) | 0.57 (0.56-0.58) | 0.63 (0.62-0.64) | 0.41 (0.40-0.42) | 0.50(0.48-0.51) | |||||
|
|
W3 | 0.8 (0.89-0.90) | 0.78 (0.78-0.79) | 0.77 (0.76-0.78) | 0.66 (0.65-0.67) | 0.71(0.70 0.72) | |||||
|
|
W4 | 0.9 (0.90-0.91) | 0.85 (0.85-0.85) | 0.80 (0.80-0.80) | 0.74 (0.74-0.75) | 0.77 (0.77-0.78) | |||||
|
|
W5 | 0.92 (0.92-0.92) | 0.89 (0.89-0.89) | 0.83 (0.83-0.84) | 0.79 (0.79-0.80) | 0.81 (0.81-0.82) | |||||
|
|
W6 | 0.92 (0.92-0.93) | 0.91 (0.91-0.91) | 0.85 (0.84-0.85) | 0.83 (0.82-0.83) | 0.84 (0.83-0.84) | |||||
|
|
W7 | 0.93 (0.93-0.93) | 0.93 (0.92-0.93) | 0.86 (0.86-0.86) | 0.86 (0.85-0.86) | 0.86 (0.86-0.86) | |||||
|
|
W8 | 0.93(0.93-0.94) | 0.94 (0.94-0.94) | 0.87 (0.87-0.88) | 0.88 (0.88-0.88) | 0.88 (0.88-0.88) | |||||
|
|
W9 | 0.94 (0.94-0.94) | 0.95 (0.95-0.95) | 0.89 (0.88-0.89) | 0.89 (0.89-0.90) | 0.89 (0.89-0.89) | |||||
|
|
W10 | 0.94 (0.93-0.94) | 0.95 (0.95-0.95) | 0.89 (0.88-0.89) | 0.90 (0.90-0.91) | 0.89 (0.89-0.90) | |||||
|
|
W11 | 0.93 (0.93- 0.93) | 0.95 (0.95-0.95) | 0.89 (0.88-0.89) | 0.91 (0.91-0.91) | 0.90 (0.90-0.90) | |||||
|
|
W12 | 0.93 (0.93-0.93) | 0.95 (0.95-0.96) | 0.89 (0.89-0.90) | 0.92 (0.91-0.92) | 0.91(0.90-0.91) | |||||
| RF model | |||||||||||
|
|
W2 | 0.84 (0.83-0.84) | 0.55 (0.54-0.57) | 0.45 (0.43-0.47) | 0.72 (0.71-0.73) | 0.55 (0.54-0.57) | |||||
|
|
W3 | 0.88 (0.88-0.89) | 0.77 (0.76-0.78) | 0.69 (0.68-0.69) | 0.73 (0.72-0.74) | 0.71 (0.70-0.71) | |||||
|
|
W4 | 0.90 (0.90-0.90) | 0.84 (0.84-0.85) | 0.76 (0.76-0.77) | 0.78 (0.77-0.79) | 0.77 (0.77-0.77) | |||||
|
|
W5 | 0.91 (0.91-0.91) | 0.88 (0.88-0.89) | 0.81 (0.80-0.83) | 0.80 (0.78-0.82) | 0.81 (0.80-0.81) | |||||
|
|
W6 | 0.92 (0.92-0.92) | 0.91 (0.90-0.91) | 0.85 (0.84-0.85) | 0.81 (0.79-0.82) | 0.83 (0.82-0.83) | |||||
|
|
W7 | 0.92 (0.92-0.93) | 0.92 (0.92-0.93) |
0.87 (0.86-0.88) | 0.83 (0.82-0.84) | 0.85 (0.84-0.85) | |||||
|
|
W8 | 0.93 (0.93-0.93) | 0.94 (0.93-0.94) | 0.88 (0.88-0.89) | 0.85 (0.84-0.86) | 0.87 (0.86-0.87) | |||||
|
|
W9 | 0.94 (0.93-0.94) | 0.95 (0.95-0.95) | 0.90 (0.90-0.91) | 0.86 (0.85-0.86) | 0.88 (0.88 -0.88) | |||||
|
|
W10 | 0.93 (0.93-0.93) | 0.95 (0.95-0.95) | 0.91 (0.90-0.91) | 0.86 (0.85-0.86) | 0.88 (0.88-0.89) | |||||
|
|
W11 | 0.93 (0.93-0.93) | 0.95 (0.95-0.95) | 0.91 (0.91-0.91) | 0.86 (0.86-0.87) | 0.89 (0.88-0.89) | |||||
|
|
W12 | 0.93 (0.92-0.93) | 0.95 (0.95-0.96) | 0.92 (0.91-0.92) | 0.86 (0.85- 0.87) | 0.89 (0.88-0.89) | |||||
| L1 model | |||||||||||
|
|
W2 | 0.81 (0.80-0.81) | 0.44 (0.43-0.44) | 0.34 (0.34-0.35) | 0.85 (0.84-0.86) | 0.49 (0.49-0.49) | |||||
|
|
W3 | 0.88 (0.88-0.88) | 0.75 (0.74-0.76) | 0.75 (0.74-0.75) | 0.67 (0.66-0.68) | 0.71 (0.70-0.72) | |||||
|
|
W4 | 0.90 (0.89-0.90) | 0.83 (0.82-0.83) | 0.79 (0.78-0.79) | 0.75 (0.74-0.76) | 0.77(0.76-0.77) | |||||
|
|
W5 | 0.91 (0.90-0.91) | 0.87 (0.87-0.88) | 0.82 (0.81-0.82) | 0.80 (0.79-0.81) | 0.81 (0.80-0.81) | |||||
|
|
W6 | 0.91 (0.91-0.92) | 0.89 (0.89-0.90) | 0.83 (0.82-0.84) | 0.84 (0.83-0.84) | 0.83 (0.83-0.84) | |||||
|
|
W7 | 0.92 (0.92-0.92) | 0.91 (0.91-0.92) | 0.84 (0.84-0.85) | 0.87 (0.87-0.87) | 0.86 (0.85-0.86) | |||||
|
|
W8 | 0.93 (0.92-0.93) | 0.93 (0.93-0.93) | 0.87 (0.87-0.88) | 0.88 (0.87-0.88) | 0.87 (0.87-0.88) | |||||
|
|
W9 | 0.93 (0.93-0.93) | 0.94 (0.94 -0.94) | 0.89 (0.89-0.89) | 0.88 (0.88-0.89) | 0.89 (0.88-0.89) | |||||
|
|
W10 | 0.93 (0.92-0.93) | 0.94 (0.94-0.94) | 0.89 (0.89-0.90) | 0.88 (0.88-0.89) | 0.89 (0.89-0.89) | |||||
|
|
W11 | 0.92 (0.92-0.93) | 0.94 (0.94-0.94) | 0.90 (0.90-0.90) | 0.89 (0.88-0.89) | 0.89 (0.89-0.90) | |||||
|
|
W12 | 0.92 (0.92-0.93) | 0.95 (0.94-0.95) | 0.91 (0.90-0.91) | 0.89 (0.88-0.89) | 0.90 (0.89-0.90) | |||||
aXGB: extreme gradient boosting.
bRF: random forest.
cL1: logistic regression with L1 regularization.
dAUC-ROC: area under the receiver operating characteristic curve.
eAUC-PRC: area under the precision-recall curve.
fW: week; the number beside stands for the week of the program and ranges 2 to 12.