Table 2.
Statistics for Tier 2 performance of the System-4 malaria forecast seasonal time series
| JAS (NE India) | SON (NW India) | ||||||
|---|---|---|---|---|---|---|---|
| ERAI | IMD | TRMM-ERAI | ERAI | IMD | TRMM-ERAI | ||
| (1981–2010) | (1981–2002) | (1998–2010) | (1981–2010) | (1981–2002) | (1998–2010) | ||
| Sensitivity | 1.00 | 0.86 | 0.75 | 0.80 | 0.86 | 0.75 | |
| Specificity | 0.75 | 0.73 | 0.56 | 0.75 | 0.67 | 0.89 | |
| Precision | 0.67 | 0.60 | 0.43 | 0.62 | 0.55 | 0.75 | |
| Negative predictive | 1.00 | 0.92 | 0.83 | 0.88 | 0.91 | 0.89 | |
| Above upper tercile | value | ||||||
| Accuracy | 0.83 | 0.77 | 0.62 | 0.77 | 0.73 | 0.85 | |
| ROC area | 0.90 | 0.86 | 0.69 | 0.89 | 0.80 | 0.92 | |
| ROC critical value | 0.73 | 0.77 | 0.86 | 0.73 | 0.77 | 0.86 | |
| Sensitivity | 0.93 | 1.00 | 0.83 | 0.93 | 0.91 | 1.00 | |
| Specificity | 0.40 | 0.55 | 0.43 | 0.53 | 0.55 | 0.57 | |
| Precision | 0.61 | 0.69 | 0.56 | 0.67 | 0.67 | 0.67 | |
| Negative predictive | 0.86 | 1.00 | 0.75 | 0.89 | 0.86 | 1.00 | |
| Above median | value | ||||||
| Accuracy | 0.67 | 0.77 | 0.62 | 0.73 | 0.73 | 0.77 | |
| ROC area | 0.84 | 0.88 | 0.57 | 0.82 | 0.88 | 0.95 | |
| ROC critical value | 0.71 | 0.83 | 0.84 | 0.71 | 0.83 | 0.84 | |
| Sensitivity | 0.80 | 0.71 | 0.75 | 0.80 | 0.71 | 0.75 | |
| Specificity | 0.55 | 0.47 | 0.44 | 0.75 | 0.80 | 0.67 | |
| Precision | 0.47 | 0.38 | 0.38 | 0.62 | 0.63 | 0.50 | |
| Negative predictive | 0.85 | 0.78 | 0.80 | 0.88 | 0.86 | 0.86 | |
| Below lower tercile | value | ||||||
| Accuracy | 0.63 | 0.55 | 0.54 | 0.77 | 0.77 | 0.69 | |
| ROC area | 0.75 | 0.89 | 0.86 | 0.88 | 0.84 | 0.89 | |
| ROC critical value | 0.73 | 0.77 | 0.86 | 0.73 | 0.77 | 0.86 | |
The System-4 seasonal malaria time series is compared against the three meteorological observation/reanalysis datasets used to drive LMM. The three events considered are high malaria occurrence (above the upper tercile), above average malaria (above the median) and low malaria (below the lower tercile). Using values from the confusion matrices in Table 1, the sensitivity, or “hit rate”, is the number of hits divided by the total number of hits and misses, the specificity, or “correct rejection rate”, is the number of correct rejections divided by the total number of false alarms and correct rejections, the precision is the number of hits divided by the total number of hits and false alarms, the negative predictive value is the number of correct rejections divided by the total number of correct rejections and misses and the accuracy, which is the number of hits and correct rejections divided by the total number of events. Also shown are values of area under the ROC curve and it’s critical value for significance (where the significance level, p, is 0.05) calculated from the two-tailed Mann-Whitney U-statistic [63].