Table 3. Distribution of the violations according to the different percentile values for the overall violations, as well as, the structural and irrelevant difficulty violations (ranking of the resources was done with sources with the least violations were at the top).
| Percentile | Source a | No_Viol (%) | Percentile | Source a | No_S_Viol (%) | Percentile | Source a | No_D_Viol (%) |
|---|---|---|---|---|---|---|---|---|
| >75th (>35.5) | 10 | 68 | >75th (>74.8 | 18 | 83 | >75th (>49.5) | 9 | 84 |
| 9 | 52 | 10 | 78 | 10 | 81 | |||
| 13 | 45 | 17 | 77 | 8 | 70 | |||
| 11 | 43 | 20 | 76 | 11 | 53 | |||
| 8 | 36 | 11 | 75 | 17 | 50 | |||
| 50th–75th (15–35.5) | 17 | 34 | 50th–75th (>57.5–74.8) | 13 | 74 | 50th–75th (>31.5–49.8) | 13 | 49 |
| 20 | 24 | 19 | 65 | 6 | 40 | |||
| 12 | 23 | 9 | 63 | 12 | 38 | |||
| 15 | 16 | 3 | 62 | 20 | 35 | |||
| 19 | 15 | 7 | 58 | 4 | 34 | |||
| 25th–50th (8.5–14.5) | 4 | 14 | 25th–50th (44.5–57.5) | 15 | 57 | 25th–50th (8.5–14.5) | 19 | 29 |
| 18 | 13 | 8 | 51 | 3 | 26 | |||
| 3 | 12 | 4 | 50 | 15 | 24 | |||
| 6 | 11 | 12 | 49 | 18 | 17 | |||
| 1 | 10 | 5 | 46 | 5 | 12 | |||
| <25th (<8.5) | 14 | 8 | <25th (<44.5) | 14 | 44 | <25th (<12) | 14 | 12 |
| 16 | 7 | 1 | 42 | 16 | 12 | |||
| 2 | 4 | 16 | 39 | 1 | 10 | |||
| 5 | 4 | 2 | 38 | 2 | 8 | |||
| 7 | 4 | 6 | 37 | 7 | 8 |
No_D_Viol, items without irrelevant difficulty flaws; No_S_Viol, items without structural flaws; No_Viol, total good items without flaws.
Note: Test-wiseness violations were not included as they were minimal as seen in Table 1 .
a
Sources: • Study-book related multiple-choice questions (MCQs) are resources Coded: 1, 2, 3, 18, 19, and 20. • Review-book related MCQs are resources Coded: 4, 5, and 6. • Self-assessment books are resources Coded: 8, 9, 10, 11, 12, 13, 15, and 17. • Online shared MCQs are resources Coded: 7, 14, and 16.