Table 1.

Summary of semantic system used to interpret algorithm performance

	Performance
Measurement	Outstanding	Excellent	Acceptable
Outbreak detection and prediction
AUC, AUWROC, VUTROC	0.90	0.80	0.70
Sensitivity, specificity, PPV (weekly)	0.95	0.90	0.85
Sensitivity, specificity, PPV (daily)	0.90	0.85	0.80
Only outbreak prediction
Pearson's correlation (weekly)	0.90	0.80	0.70
Pearson's correlation (daily)	0.85	0.75	0.65

AUC, area under the curve; AUWROC, area under the weighted receiver operating characteristic curve; PPV, positive predictive value; VUTROC, volume under the time-ROC surface.