Table 1. Distribution of the percentage of items having an overall item flaws (violations), structural, test wiseness, and irrelevant difficulty flaws in all MCQ resources.
Total item flaws (%) | Structural flaws (%) | Test-wiseness flaws (%) | Irrelevant difficulty flaws (%) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Source or book a | Good items b ( n = 537) | One flaw ( n = 690) | >1 flaw ( n = 1,073) | No flaws ( n = 537) | One flaw ( n = 690) | >1 flaw ( n = 1,073) | No flaws ( n = 537) | One flaw ( n = 690) | >1 flaw ( n = 1,073) | No flaws ( n = 537) | One flaw ( n = 690) | >1 flaw ( n = 1,073) |
1 | 10 | 14 | 76 | 42 | 8 | 50 | 68 | 28 | 4 | 10 | 64 | 26 |
2 | 4 | 16 | 80 | 38 | 6 | 56 | 70 | 28 | 2 | 8 | 68 | 24 |
3 | 12 | 28 | 60 | 62 | 28 | 10 | 80 | 18 | 2 | 26 | 54 | 20 |
4 | 14 | 30 | 56 | 50 | 46 | 4 | 78 | 20 | 2 | 34 | 26 | 40 |
5 | 4 | 18 | 78 | 46 | 34 | 20 | 60 | 32 | 8 | 12 | 34 | 54 |
6 | 11 | 29 | 60 | 37 | 34 | 29 | 81 | 18 | 1 | 40 | 36 | 24 |
7 | 4 | 44 | 52 | 58 | 42 | 0 | 84 | 14 | 2 | 8 | 76 | 16 |
8 | 36 | 19 | 45 | 51 | 17 | 32 | 90 | 9 | 1 | 70 | 26 | 4 |
9 | 52 | 23 | 25 | 63 | 35 | 2 | 82 | 16 | 2 | 84 | 15 | 1 |
10 | 68 | 19 | 13 | 78 | 21 | 1 | 97 | 3 | 0 | 81 | 18 | 1 |
11 | 43 | 27 | 30 | 75 | 11 | 14 | 92 | 6 | 2 | 53 | 38 | 9 |
12 | 23 | 21 | 56 | 49 | 31 | 20 | 90 | 8 | 2 | 38 | 38 | 24 |
13 | 45 | 17 | 38 | 74 | 21 | 5 | 94 | 6 | 0 | 49 | 31 | 20 |
14 | 8 | 18 | 74 | 44 | 36 | 20 | 90 | 10 | 0 | 12 | 44 | 44 |
15 | 16 | 21 | 63 | 57 | 22 | 21 | 91 | 9 | 1 | 24 | 36 | 40 |
16 | 7 | 15 | 77 | 39 | 27 | 34 | 85 | 14 | 1 | 12 | 51 | 37 |
17 | 34 | 42 | 24 | 77 | 18 | 5 | 89 | 9 | 2 | 50 | 48 | 3 |
18 | 13 | 59 | 28 | 83 | 16 | 2 | 89 | 11 | 1 | 17 | 77 | 7 |
19 | 15 | 35 | 50 | 65 | 22 | 14 | 89 | 10 | 1 | 29 | 44 | 27 |
20 | 24 | 39 | 37 | 76 | 14 | 10 | 93 | 6 | 1 | 35 | 44 | 20 |
Average | 23 | 30 | 47 | 62 | 23 | 15 | 87 | 11 | 1 | 36 | 44 | 20 |
Abbreviation: MCQ, multiple-choice question.
As denoting to the selected book and/or resource (List of sources was detailed in Supplementary Material S1 [available in the online version]).
Good items mean Items without any flaw (violation).