Skip to main content
. 2014 Jan 15;9(1):e85180. doi: 10.1371/journal.pone.0085180

Table 2. Adaptive iterative reviewing: IED nominations are iteratively presented to the reviewer one-by-one using 10 events per iteration.

Subject #IEDs IEDs confirmed up and until iteration
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
S1 13 8 11 13 13 13 13 13 13 13 13 13 13 13 13 13
S2 11 4 6 6 8 8 8 8 8 9 9 9 9 9 9 9
S3 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
S4 36 4 8 14 17 20 24 27 29 30 31 32 32 33 33 35
S5 19 8 12 15 18 19 19 19 19 19 19 19 19 19 19 19
S6 6 4 5 5 5 5 5 5 5 5 5 5 5 6 6 6
S7 79 9 17 26 36 45 52 58 61 62 68 68 70 72 74 74
S8 5 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5
S9 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
S10 12 3 10 11 11 12 12 12 12 12 12 12 12 12 12 12
S11 7 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2
S12 7 3 4 5 6 6 6 7 7 7 7 7 7 7 7 7
S13 19 10 17 19 19 19 19 19 19 19 19 19 19 19 19 19
S14 14 9 12 12 12 13 14 14 14 14 14 14 14 14 14 14
S15 6 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6
TOTAL 241 78 120 144 164 179 191 201 206 209 216 217 220 224 226 228
MEAN 32% 50% 60% 68% 74% 79% 83% 85% 87% 90% 90% 91% 93% 94% 95%

Nomination certainties are updated after every iteration, thereby allowing nominations with higher likelihoods to be shown first. Template reliabilities were also updated at the end of each review. This was done by updating the true and false positives of contributing templates and recalculating their reliability scores accordingly, as shown in Appendix S1.