Table 2.
Evaluation metrics of the 5 machine learning models for the current week forecasts (nowcast), 1-week forecasts, 2-week forecasts, and 3-week forecasts in 2018.
|
|
Outpatient influenza-like illness visits | Emergency influenza-like illness visits | |||||||||
|
|
Model | RMSEa | MAPEb, % | Hit rate | Pearson correlation coefficient | RMSE | MAPE, % | Hit rate | Pearson correlation coefficient | ||
| Nowcast (current week) | |||||||||||
|
|
ARIMAc | 6422.8 | 5.5 | 0.727 | 0.958 | 1125.3 | 5.7 | 0.782 | 0.965 | ||
|
|
RFd | 6472.1 | 5.4 | 0.709 | 0.957 | 1305.8 | 6.2 | 0.709 | 0.952 | ||
|
|
SVRe | 5343.2 | 5.8 | 0.655 | 0.969 | 2079.6 | 8.0 | 0.582 | 0.875 | ||
|
|
XGBf | 7384.6 | 5.8 | 0.691 | 0.943 | 1643.6 | 6.2 | 0.673 | 0.923 | ||
|
|
Ensemble | 6170.7 | 5.3 | 0.600 | 0.962 | 1751.0 | 6.5 | 0.727 | 0.912 | ||
| 1-week | |||||||||||
|
|
ARIMA | 9874.7 | 9.0 | 0.741 | 0.897 | 1707.6 | 7.8 | 0.704 | 0.919 | ||
|
|
RF | 8644.1 | 7.2 | 0.833 | 0.921 | 1861.2 | 7.9 | 0.759 | 0.899 | ||
|
|
SVR | 7330.1 | 7.9 | 0.778 | 0.942 | 2643.7 | 10.6 | 0.759 | 0.798 | ||
|
|
XGB | 9738.7 | 9.0 | 0.741 | 0.903 | 2353.3 | 8.3 | 0.685 | 0.836 | ||
|
|
Ensemble | 9156.8 | 7.5 | 0.796 | 0.911 | 2363.0 | 8.4 | 0.722 | 0.832 | ||
| 2-week | |||||||||||
|
|
ARIMA | 11630.6 | 11.8 | 0.811 | 0.851 | 1922.2 | 8.6 | 0.755 | 0.893 | ||
|
|
RF | 10082.3 | 9.4 | 0.811 | 0.893 | 1975.5 | 7.7 | 0.736 | 0.888 | ||
|
|
SVR | 9292.7 | 10.1 | 0.830 | 0.905 | 3031.0 | 12.8 | 0.566 | 0.730 | ||
|
|
XGB | 11262.8 | 11.3 | 0.755 | 0.873 | 2371.6 | 7.9 | 0.698 | 0.843 | ||
|
|
Ensemble | 10078.8 | 8.6 | 0.774 | 0.889 | 2389.4 | 8.6 | 0.755 | 0.835 | ||
| 3-week | |||||||||||
|
|
ARIMA | 13656.7 | 13.5 | 0.865 | 0.787 | 1875.4 | 9.5 | 0.788 | 0.898 | ||
|
|
RF | 10258.0 | 10.1 | 0.769 | 0.892 | 2041.9 | 8.8 | 0.692 | 0.877 | ||
|
|
SVR | 9439.7 | 10.9 | 0.885 | 0.904 | 3106.0 | 13.5 | 0.596 | 0.721 | ||
|
|
XGB | 12789.3 | 13.0 | 0.788 | 0.830 | 2259.6 | 7.6 | 0.692 | 0.890 | ||
|
|
Ensemble | 9160.9 | 8.8 | 0.904 | 0.908 | 2478.4 | 9.6 | 0.769 | 0.814 | ||
aRMSE: root mean squared error.
bMAPE: mean absolute percentage error.
cARIMA: autoregressive integrated moving average.
dRF: random forest.
eSVR: support vector regression.
fXGB: extreme gradient boosting.