Table 3.
‘Best’ parameter sets based on range of earliest true positive (TP) early warning signal (EWS) lead time and false positive (FP) EWS criteria.
| Best values by wave across all parameter sets | Best parameters for individual ranking and filter criteria | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ranking criteria | |||||||||||||||||||||
| Rank by | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | Lead time | |
| Which waves included in ranking an filters | All | All | New variant driven | New variant driven | New variant driven | New variant driven | All | All | New variant driven | New variant driven | All | All | All | New variant driven | New variant driven | New variant driven | All | All | New variant driven | All | |
| False positive limit | All | All | All | All | 10 | 10 | 5 | 10 | 2 | 5 | 2 | 5 | 10 | 2 | 5 | 0 | 2 | 0 | 0 | 0 | |
| Which false positives | All | All | All | All | All | Before 1st TP | All | All | All | All | Before 1st TP | Before 1st TP | Before 1st TP | Before 1st TP | Before 1st TP | All | All | Before 1st TP | Before 1st TP | All | |
| Restriction on individual wave lead times? | No | Yes | No | Yes | No | No | No | No | No | No | No | No | No | No | No | No | No | No | No | No | |
| Number of parameter sets | 40,720 | 19 | 40,720 | 19 | 12,066 | 13,554 | 2134 | 6998 | 2924 | 7164 | 1473 | 3143 | 10,313 | 3700 | 7642 | 995 | 319 | 608 | 1087 | 20 | |
| TFP Scanner parameters | |||||||||||||||||||||
| Leading indicator | Mean simple LGR | Mean simple LGR | Mean LGR | Mean LGR | Mean GAM LGR | Mean GAM LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | Dominant Pango lineage max LGR | |
| Cluster min age | 14 | 14 | 7 | 7 | 14 | 14 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 14 | 14 | 14 | 28 | |
| Cluster max age | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 56 | 84 | 56 | 56 | 56 | 56 | |
| Cluster min descendants | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | 20 | |
| Cluster filter variables | |||||||||||||||||||||
| LGR p-value limit | No limit | No limit | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | 0.05 | 0.05 | 0.05 | 0.01 | |
| Parent/sub-cluster replacement LGR threshold | 85% | 90% | 60% | 60% | 80% | 80% | 85% | 85% | 85% | 85% | 85% | 85% | 85% | 85% | 85% | 75% | 85% | 90% | 90% | 90% | |
| EWS threshold | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.05 | 0.00 | 0.00 | 0.70 | |
| Earliest TP EWS lead (−ve) or lag (+ve) days | |||||||||||||||||||||
| B.1.177 | 6 | 6 | 6 | 6 | 6 | 10 | 10 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 9 | 9 | 9 | 23 |
| Alpha | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 | −6 |
| Delta (1) | −24 | −24 | −24 | −24 | −24 | −24 | −24 | −20 | −20 | −20 | −20 | −20 | −20 | −20 | −20 | −20 | −20 | −20 | −24 | −24 | −4 |
| Delta (2) | −1 | −1 | −1 | −1 | −1 | −1 | −1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 5 | 5 | 5 | 7 |
| Delta (3) | −9 | −9 | −9 | −1 | −1 | −1 | −1 | −9 | −9 | −9 | −9 | −9 | −9 | −9 | −9 | −9 | 0 | 2 | 2 | 2 | 2 |
| Omicron BA.1 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 6 | 8 | 6 | 6 | 6 | 8 |
| Omicron BA.2 | −19 | −17 | −17 | −19 | −19 | −19 | −19 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | −17 | 11 |
| Mean across all waves | −6.6 | −6.4 | −6.4 | −5.6 | −5.6 | −5.0 | −5.0 | −5.4 | −5.4 | −5.4 | −5.4 | −5.4 | −5.4 | −5.4 | −5.4 | −5.4 | −3.0 | −3.0 | −3.6 | −3.6 | 5.9 |
| Mean across variant driven waves | −10.8 | −10.3 | −10.3 | −10.8 | −10.8 | −10.8 | −10.8 | −9.3 | −9.3 | −9.3 | −9.3 | −9.3 | −9.3 | −9.3 | −9.3 | −9.3 | −8.8 | −9.3 | −10.3 | −10.3 | 2.3 |
| Number of FP EWS | |||||||||||||||||||||
| B.1.177 | 0 | 5 | 5 | 1 | 1 | 5 | 5 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 5 | 2 | 4 | 4 | 0 |
| Alpha | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Delta (1) | 0 | 14 | 14 | 17 | 17 | 6 | 6 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0 | 0 | 0 | 0 | 0 |
| Delta (2) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Delta (3) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Omicron BA.1 | 0 | 1 | 1 | 1 | 1 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0 | 2 | 2 | 2 | 0 |
| Omicron BA.2 | 0 | 0 | 0 | 3 | 3 | 3 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Mean across all waves | 0.0 | 2.9 | 2.9 | 3.1 | 3.1 | 2.3 | 2.3 | 1.1 | 1.1 | 1.1 | 1.1 | 1.1 | 1.1 | 1.1 | 1.1 | 1.1 | 0.7 | 0.6 | 0.9 | 0.9 | 0.0 |
| Mean across variant driven waves | 0.0 | 3.8 | 3.8 | 5.3 | 5.3 | 2.8 | 2.8 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.5 | 0.5 | 0.5 | 0.0 |
| Number of FP EWS before earliest TP | |||||||||||||||||||||
| B.1.177 | 0 | 1 | 1 | 0 | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Alpha | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Delta (1) | 0 | 14 | 14 | 14 | 14 | 6 | 6 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0 | 0 | 0 | 0 | 0 |
| Delta (2) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Delta (3) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Omicron BA.1 | 0 | 1 | 1 | 0 | 0 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0 | 0 | 0 | 0 | 0 |
| Omicron BA.2 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Mean across all waves | 0.0 | 2.3 | 2.3 | 2.1 | 2.1 | 1.7 | 1.7 | 0.6 | 0.6 | 0.6 | 0.6 | 0.6 | 0.6 | 0.6 | 0.6 | 0.6 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Mean across variant driven waves | 0.0 | 3.8 | 3.8 | 3.8 | 3.8 | 2.3 | 2.3 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Precision or positive predictive value (PPV) | |||||||||||||||||||||
| B.1.177 | 1.0 | 0.6 | 0.6 | 0.8 | 0.8 | 0.6 | 0.6 | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | 0.7 | 0.8 | 0.7 | 0.7 | 1.0 |
| Alpha | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Delta (1) | 1.0 | 0.7 | 0.7 | 0.5 | 0.5 | 0.6 | 0.6 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Delta (2) | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Delta (3) | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Omicron BA.1 | 1.0 | 0.8 | 0.9 | 0.8 | 0.8 | 0.8 | 0.8 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 1.0 | 0.9 | 0.9 | 0.9 | 1.0 |
| Omicron BA.2 | 1.0 | 1.0 | 1.0 | 0.7 | 0.7 | 0.3 | 0.3 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Mean across all waves | 1.0 | 0.9 | 0.9 | 0.8 | 0.8 | 0.7 | 0.7 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 0.9 | 1.0 | 1.0 | 0.9 | 0.9 | 1.0 |
| Mean across variant driven waves | 1.0 | 0.9 | 0.9 | 0.8 | 0.8 | 0.7 | 0.7 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
Rankings are determined by the sum of the earliest TP EWS lead times across different sets of SARS-CoV-2 epidemic waves and a range of filters based on the number of FP EWS. There is some subjectivity in selecting a single parameter set for each ranking criteria as more than one parameter set may achieve the same ranking score. Tables containing the highest ranked parameter sets for each of the criteria sets in this table can be found in the Supplementary Material.