Table 1. Accuracy metrics between ARES and CDC’s ILI for all geographic regions, for the three flu seasons spanning 2012–2015.
| Algorithm | RMSE | Rel. RMSE (%) | Correlation | ||||||
|---|---|---|---|---|---|---|---|---|---|
| 2012–2013 | 2013–2014 | 2014–2015 | 2012–2013 | 2013–2014 | 2014–2015 | 2012–2013 | 2013–2014 | 2014–2015 | |
| National | |||||||||
| GFT | 2.16 | 0.39 | 0.36 | 62.73% | 13.21% | 12.31% | 0.932 | 0.968 | 0.986 |
| Linear (univariate) | 0.45 | 0.20 | 0.54 | 12.20% | 9.00% | 12.22% | 0.996 | 0.985 | 0.986 |
| AR(2) | 0.46 | 0.30 | 0.45 | 11.77% | 9.53% | 11.40% | 0.940 | 0.940 | 0.937 |
| SVM (linear) + AR(2) | 0.10 | 0.09 | 0.17 | 4.15% | 5.12% | 4.62% | 0.997 | 0.995 | 0.991 |
| Region 1 | |||||||||
| GFT | 2.87 | 0.27 | 0.41 | 107.01% | 21.20% | 26.01% | 0.789 | 0.881 | 0.951 |
| Linear (univariate) | 0.51 | 0.22 | 0.36 | 25.54% | 21.11% | 20.80% | 0.974 | 0.965 | 0.973 |
| AR(2) | 0.40 | 0.23 | 0.32 | 18.35% | 16.88% | 17.31% | 0.897 | 0.856 | 0.926 |
| SVM (linear) + AR(2) | 0.26 | 0.13 | 0.24 | 13.80% | 10.44% | 10.30% | 0.964 | 0.974 | 0.960 |
| Region 2 | |||||||||
| GFT | 2.22 | 0.64 | 1.18 | 46.54% | 27.37% | 38.64% | 0.960 | 0.833 | 0.938 |
| Linear (univariate) | 0.38 | 0.49 | 0.97 | 14.24% | 20.16% | 28.98% | 0.983 | 0.877 | 0.940 |
| AR(2) | 0.42 | 0.27 | 0.27 | 11.32% | 10.12% | 9.73% | 0.949 | 0.922 | 0.937 |
| SVM (linear) + AR(2) | 0.30 | 0.19 | 0.23 | 9.22% | 7.02% | 8.61% | 0.975 | 0.970 | 0.956 |
| Region 3 | |||||||||
| GFT | 1.97 | 0.33 | 0.63 | 78.06% | 24.18% | 21.19% | 0.914 | 0.984 | 0.983 |
| Linear (univariate) | 0.90 | 0.36 | 0.81 | 22.97% | 16.86% | 20.51% | 0.984 | 0.986 | 0.993 |
| AR(2) | 0.71 | 0.24 | 0.88 | 19.97% | 9.96% | 16.59% | 0.908 | 0.965 | 0.900 |
| SVM (linear) + AR(2) | 0.43 | 0.15 | 0.26 | 12.66% | 6.57% | 7.07% | 0.976 | 0.988 | 0.992 |
| Region 4 | |||||||||
| GFT | 1.84 | 0.36 | 0.45 | 58.99% | 27.05% | 16.71% | 0.891 | 0.958 | 0.974 |
| Linear (univariate) | 1.02 | 0.38 | 0.48 | 47.33% | 34.95% | 13.33% | 0.979 | 0.973 | 0.986 |
| AR(2) | 0.57 | 0.37 | 0.74 | 15.07% | 15.21% | 18.29% | 0.924 | 0.941 | 0.903 |
| SVM (linear) + AR(2) | 0.21 | 0.16 | 0.27 | 8.70% | 11.41% | 9.18% | 0.991 | 0.989 | 0.989 |
| Region 5 | |||||||||
| GFT | 2.18 | 0.36 | 0.46 | 63.18% | 20.44% | 21.70% | 0.887 | 0.962 | 0.970 |
| Linear (univariate) | 0.28 | 0.43 | 0.53 | 13.58% | 23.34% | 15.22% | 0.989 | 0.951 | 0.983 |
| AR(2) | 0.46 | 0.33 | 0.42 | 12.86% | 11.59% | 11.95% | 0.927 | 0.886 | 0.940 |
| SVM (linear) + AR(2) | 0.23 | 0.20 | 0.18 | 7.51% | 8.22% | 6.68% | 0.985 | 0.964 | 0.991 |
| Region 6 | |||||||||
| GFT | 3.74 | 0.75 | 1.49 | 66.62% | 14.31% | 25.84% | 0.921 | 0.968 | 0.923 |
| Linear (univariate) | 1.29 | 0.73 | 0.66 | 21.17% | 18.53% | 13.38% | 0.965 | 0.964 | 0.963 |
| AR(2) | 0.69 | 0.58 | 0.68 | 14.39% | 10.38% | 13.31% | 0.937 | 0.949 | 0.935 |
| SVM (linear) + AR(2) | 0.49 | 0.34 | 0.57 | 10.26% | 8.90% | 10.31% | 0.982 | 0.987 | 0.957 |
| Region 7 | |||||||||
| GFT | 0.77 | 0.92 | 1.67 | 22.54% | 85.14% | 72.62% | 0.942 | 0.968 | 0.695 |
| Linear (univariate) | 0.63 | 2.43 | 1.95 | 29.42% | 315.08% | 124.32% | 0.975 | 0.958 | 0.569 |
| AR(2) | 0.59 | 0.24 | 0.55 | 16.87% | 35.87% | 22.99% | 0.948 | 0.910 | 0.919 |
| SVM (linear) + AR(2) | 0.37 | 0.45 | 0.81 | 13.03% | 54.24% | 29.14% | 0.981 | 0.935 | 0.827 |
| Region 8 | |||||||||
| GFT | 0.84 | 0.39 | 0.43 | 27.29% | 28.97% | 24.09% | 0.920 | 0.951 | 0.953 |
| Linear (univariate) | 0.77 | 0.54 | 0.78 | 23.73% | 20.16% | 25.38% | 0.973 | 0.942 | 0.894 |
| AR(2) | 0.36 | 0.41 | 0.38 | 15.01% | 19.64% | 15.78% | 0.961 | 0.843 | 0.930 |
| SVM (linear) + AR(2) | 0.24 | 0.21 | 0.26 | 10.09% | 12.99% | 11.47% | 0.986 | 0.972 | 0.970 |
| Region 9 | |||||||||
| GFT | 2.84 | 0.80 | 0.42 | 72.09% | 24.93% | 15.59% | 0.922 | 0.946 | 0.934 |
| Linear (univariate) | 0.43 | 0.38 | 0.29 | 19.28% | 20.79% | 13.37% | 0.975 | 0.927 | 0.965 |
| AR(2) | 0.37 | 0.29 | 0.35 | 11.92% | 13.42% | 10.82% | 0.934 | 0.935 | 0.942 |
| SVM (linear) + AR(2) | 0.28 | 0.21 | 0.21 | 10.81% | 10.57% | 7.79% | 0.970 | 0.971 | 0.981 |
| Region 10 | |||||||||
| GFT | 2.85 | 0.73 | 0.50 | 181.88% | 91.96% | 30.46% | 0.866 | 0.953 | 0.955 |
| Linear (univariate) | 0.75 | 0.49 | 0.48 | 125.35% | 82.45% | 31.04% | 0.737 | 0.976 | 0.908 |
| AR(2) | 0.49 | 0.57 | 0.41 | 37.20% | 30.14% | 27.99% | 0.867 | 0.881 | 0.920 |
| SVM (linear) + AR(2) | 0.42 | 0.46 | 0.34 | 26.92% | 21.07% | 21.54% | 0.904 | 0.922 | 0.947 |
For comparison purposes, we have included GFT’s historical predictions, and the two baseline models: dynamic linear regression (mapping athenahealth’s ILI onto CDC’s ILI), and a two term autoregressive model, AR(2). Values with best performance appear in bold face.