Table 2:
Difference in correctness percentages comparing the top choice with the top 2 and 3 choices
| Reviewer | ICA | Pair Comparison | Number (Percentage Differences) | P Valuea |
|---|---|---|---|---|
| 1 | 20 | Top 1 vs top 2 | 4 (8.30%) | .063 |
| 20 | Top 1 vs top 3 | 5 (11.63%) | .031a | |
| 20 | Top 2 vs top 3 | 1 (2.33%) | .500 | |
| 1 | 50 | Top 1 vs top 2 | 8 (18.60%) | .004a |
| 50 | Top 1 vs top 3 | 10 (23.26%) | <.001a | |
| 50 | Top 2 vs top 3 | 2 (4.65%) | .250 | |
| 2 | 20 | Top 1 vs top 2 | 7 (16.28%) | .008a |
| 20 | Top 1 vs top 3 | 8 (18.60%) | .004a | |
| 20 | Top 2 vs top 3 | 1 (2.32%) | .500 | |
| 2 | 50 | Top 1 vs top 2 | 9 (20.93%) | .002a |
| 50 | Top 1 vs top 3 | 13 (30.23%) | <.001a | |
| 50 | Top 2 vs top 3 | 4 (9.30%) | .063 | |
| 3 | 20 | Top 1 vs top 2 | 4 (9.30%) | .063 |
| 20 | Top 1 vs top 3 | 4 (9.30%) | .063 | |
| 20 | Top 2 vs top 3 | 0(0.00%) | 1.0 | |
| 3 | 50 | Top 1 vs top 2 | 6 (13.95%) | .016a |
| 50 | Top 1 vs top 3 | 8 (18.60%) | .004a | |
| 50 | Top 2 vs top 3 | 2 (4.65%) | .250 |
a
P value from the 1-sided sign test.