Table 2.
Scenario 1: L1=20%; L2=25%; L3=30% | |||||
Model | Bias | RMSE | # of times selected (%) | CI includes N (%) | Average CI width |
1. Independence† | 0.3 | 52.8 | 332 (66.4) | 475 (95) | 205.7 |
2. L1*L2 | −0.2 | 61.3 | 50 (10) | 470 (94) | 235.0 |
3. L1*L3 | 0.3 | 61.0 | 50 (10) | 479 (95.8) | 245.3 |
4. L2*L3 | −3.9 | 66.6 | 43 (8.6) | 476 (95.2) | 266.7 |
5. L1*L2, L1*L3 | −1.6 | 76.3 | 6 (1.2) | 472 (94.4) | 302.1 |
6. L1*L2, L2*L3 | −9.3 | 88.9 | 7 (1.4) | 473 (94.6) | 347.3 |
7. L1*L3, L2*L3 | −11.4 | 92.9 | 12 (2.4) | 479 (95.8) | 386.4 |
8. L1*L3, L2*L3, L1*L2 | −30.2 | 195.0 | 0 | 469 (93.8) | 752.8 |
9. DGA | 0.7 | 54.9 | NA | 481 (96.2) | 225.6 |
10. SparseMSE | 0.5 | 53.0 | NA | 475 (95) | 298.9 |
11. | −18.8 | 196.5 | NA | 489 (97.8) | 1117 |
12. | −1.1 | 52.5 | NA | 473 (94.6) | 211.9 |
13. | −0.1 | 69.6 | NA | 481 (96.2) | 281.3 |
Scenario 2: L1=20%; L2=25%; L3=24% + (30%*L1) | |||||
Model | Bias | RMSE | # of times selected (%) | CI includes N (%) | Average CI width |
1. Independence | 193.5 | 196.9 | 0 | 3 (0.6) | 134.0 |
2. L1*L2 | 232.5 | 235.4 | 0 | 1 (0.2) | 130.8 |
3. L1*L3† | −0.6 | 66.2 | 371 (74.2) | 479 (95.8) | 271.7 |
4. L2*L3 | 275.3 | 277.3 | 0 | 0 | 124.8 |
5. L1*L2, L1*L3 | −3.4 | 87.8 | 56 (11.2) | 471 (94.2) | 355.8 |
6. L1*L2, L2*L3 | 333.8 | 335.0 | 6 (1.2) | 0 | 99.4 |
7. L1*L3, L2*L3 | −17.7 | 120.6 | 67 (13.4) | 482 (96.4) | 517.7 |
8. L1*L3, L2*L3, L1*L2 | −30.1 | 195.8 | 0 | 475 (95) | 789.0 |
9. DGA | 0.2 | 71.0 | NA | 487 (97.4) | 332.1 |
10. SparseMSE | 0.2 | 67.8 | NA | 166 (33) | 206.5 |
11. | −22.7 | 154.1 | NA | 492 (98.4) | 716.7 |
12. | 191.5 | 194.9 | NA | 4 (0.8) | 136.7 |
13. | 136.8 | 146.1 | NA | 173 (34.6) | 201.5 |
Scenario 3: L1=20%; L2=25%; L3=25% + (45%*L1) + (20%*L1*L2) | |||||
Model | Bias | RMSE | # of times selected (%) | CI includes N (%) | Average CI width |
1. Independence | 249.3 | 251.6 | 0 | 0 | 98.0 |
2. L1*L2 | 297.4 | 299.1 | 0 | 0 | 89.9 |
3. L1*L3 | −1.6 | 70.5 | 27 (5.4) | 478 (95.6) | 267.6 |
4. L2*L3 | 286.4 | 288.5 | 0 | 0 | 81.0 |
5. L1*L2, L1*L3 | −6.5 | 101.0 | 8 (1.6) | 476 (95.2) | 347.2 |
6. L1*L2, L2*L3 | 356.6 | 357.8 | 0 | 0 | 58.6 |
7. L1*L3, L2*L3 | −9.70e+9 | 1.39e+11 | 465 (93) | 134 (26.8) | ND** |
8. L1*L3, L2*L3, L1*L2 | −6.32e+11 | 8.29e+12 | 0 | 120 (24) | ND** |
9. DGA | 225.1 | 316.4 | NA | 499 (99.8) | 869.3 |
10. SparseMSE | −661.7 | 2,539.3 | NA | 473 (94.6) | 254.7 |
11. | −1,054.2 | 1,290.1 | NA | 236 (47.2) | 16,677.2 |
12. | 219.4 | 222.4 | NA | 2 (0.4) | 136.7 |
13. | 32.1 | 68.2 | NA | 448 (89.6) | 235.7 |
Scenario 4: U=50%; L1=40%*U; L2=50%*U; L3=30% | |||||
Model | Bias | RMSE | # of times selected (%) | CI includes N (%) | Average CI width |
1. Independence | 183.8 | 187.6 | 0 | 7 (1.4) | 137.5 |
2. L1*L2† | −9.8 | 66.3 | 372 (74.4) | 481 (96.2) | 266.9 |
3. L1*L3 | 237.9 | 240.9 | 0 | 0 | 133.5 |
4. L2*L3 | 266.0 | 268.4 | 0 | 0 | 128.9 |
5. L1*L2, L1*L3 | −16.8 | 95.0 | 61 (12.2) | 479 (95.8) | 379.3 |
6. L1*L2, L2*L3 | −22.3 | 116.8 | 66 (13.2) | 477 (95.4) | 464.0 |
7. L1*L3, L2*L3 | 347.1 | 348.3 | 1 (0.2) | 0 | 92.5 |
8. L1*L3, L2*L3, L1*L2 | −42.5 | 185.7 | 0 | 481 (96.2) | 771.7 |
9. DGA | −10.8 | 72.4 | NA | 488 (97.6) | 320.2 |
10. SparseMSE | −9.8 | 66.3 | NA | 498 (99.6) | 248.2 |
11. | −30.7 | 162.6 | NA | 490 (98) | 479.9 |
12. | 196.7 | 200.0 | NA | 5 (1) | 135.3 |
13. | 139.3 | 148.3 | NA | 168 (33.6) | 204.5 |
Scenario 5: U=50%; L1=2% + (20%*U); L2=3% + (20%*U); L3=6% | |||||
Model | Bias | RMSE | # of times selected (%) | CI includes N (%) | Average CI width |
1. Independence | 257.7 | 273.0 | 76 (15.2) | 168 (33.6) | 356.6 |
2. L1*L2† | −67.0 | 287.2 | 253 (50.6) | ND** | ND** |
3. L1*L3 | 300.9 | 314.5 | 42 (8.4) | ND** | ND** |
4. L2*L3 | 306.9 | 320.6 | 41 (8.2) | ND** | ND** |
5. L1*L2, L1*L3 | −7.11e+9 | 1.59e+11 | 28 (5.6) | ND** | ND** |
6. L1*L2, L2*L3 | −3.53e+10 | 4.96e+11 | 27 (5.4) | ND** | ND** |
7. L1*L3, L2*L3 | 365.9 | 377.1 | 33 (6.6) | ND** | ND** |
8. L1*L3, L2*L3, L1*L2 | −2.09e+11 | 2.69e+12 | ND** | ND** | |
9. DGA | 312.7 | 319.7 | NA | 0 | 338.7 |
10. SparseMSE | 258.6 | 296.1 | NA | 144 (28.8) | 359.1 |
11. | −454.6 | 3,373.5 | NA | 411 (82.2) | 67,275.0 |
12. | 206.8 | 229.4 | NA | 279 (55.8) | 426.2 |
13. | 175.4 | 217.4 | NA | 382 (76.4) | 536.6 |
L1= List 1; L2= List 2; L3= List 3; U= binary third variable; CI= confidence interval; DGA= Decomposable Graph Approach; = Sample Coverage model assuming independence; = Sample Coverage model allowing list dependence and sufficient sample coverage fraction (>=55%); = Sample Coverage model estimating lower(upper) bound estimate due to insufficient sample coverage fraction (<55%); RMSE = Root Mean Squared Error
N=1,000 (the true population size)
Indicates the correct log-linear model for each scenario
ND**=Not Defined. Upper limit of confidence interval not defined in simulations; therefore, average confidence interval width could not be calculated.
Bias = (); RMSE = ); where is the estimated population size from simulation i, m is the number of simulations, and N is the true population size.