Table 5.
Measurement practices and evidence of validity in major clusters in the general journals.
| Cluster ID |
Cloze/Likert/ multiple choice |
Essay/writing | Oral/interview | Total |
|---|---|---|---|---|
| Construct operationalization | ||||
| 2 | 29 | 13 | 6 | 48 |
| 1 | 3 | 16 | 21 | 40 |
| 3 | 10 | 7 | 12 | 29 |
| 0 | 20 | 8 | 8 | 36 |
| 4 | 3 | 28 | 16 | 47 |
| 6 | 6 | 2 | 6 | 14 |
| 8 | 5 | 0 | 1 | 6 |
| 5 | 2 | 0 | 6 | 8 |
| 11 | 3 | 4 | 4 | 11 |
| Cluster ID | Reported reliability | Did not report reliability | Non-English | Total |
| Reliability | ||||
| 2 | 44 | 40 | 0 | 84 |
| 1 | 34 | 32 | 0 | 66 |
| 3 | 21 | 20 | 0 | 41 |
| 0 | 25 | 13 | 0 | 38 |
| 4 | 27 | 22 | 0 | 49 |
| 6 | 16 | 8 | 0 | 24 |
| 8 | 5 | 6 | 1 | 12 |
| 5 | 12 | 3 | 0 | 15 |
| 11 | 3 | 9 | 1 | 13 |
| Cluster ID | Reported generalizability evidence | Did not report generalizability evidence | Non-English | Total |
| Generalization | ||||
| 2 | 1 | 83 | 0 | 84 |
| 1 | 0 | 66 | 0 | 66 |
| 3 | 1 | 40 | 0 | 41 |
| 0 | 0 | 38 | 0 | 38 |
| 4 | 0 | 49 | 0 | 49 |
| 6 | 0 | 24 | 0 | 24 |
| 8 | 0 | 11 | 1 | 12 |
| 5 | 0 | 15 | 0 | 15 |
| 11 | 0 | 12 | 1 | 13 |
| Cluster ID | Yes | No | non-English | Total |
| Criterion Evidence of Validity | ||||
| 2 | 3 | 81 | 0 | 84 |
| 1 | 4 | 62 | 0 | 66 |
| 3 | 5 | 36 | 0 | 41 |
| 0 | 6 | 32 | 0 | 38 |
| 4 | 1 | 48 | 0 | 49 |
| 6 | 0 | 24 | 0 | 24 |
| 8 | 0 | 11 | 1 | 12 |
| 5 | 2 | 13 | 0 | 15 |
| 11 | 0 | 12 | 1 | 13 |
| Cluster ID | Yes | No | Non-English | Total |
| Explanation | ||||
| 2 | 2 | 82 | 0 | 84 |
| 1 | 4 | 62 | 0 | 66 |
| 3 | 4 | 37 | 0 | 41 |
| 0 | 6 | 32 | 0 | 38 |
| 4 | 1 | 48 | 0 | 49 |
| 6 | 0 | 24 | 0 | 24 |
| 8 | 0 | 12 | 0 | 12 |
| 5 | 0 | 15 | 0 | 15 |
| 11 | 0 | 13 | 0 | 13 |
| Cluster ID | Yes | No | Claimed without evidence | Total |
| Utilization | ||||
| 2 | 0 | 82 | 2 | 84 |
| 1 | 0 | 63 | 3 | 66 |
| 3 | 0 | 29 | 12 | 41 |
| 0 | 1 | 30 | 7 | 38 |
| 4 | 0 | 49 | 0 | 49 |
| 6 | 0 | 24 | 0 | 24 |
| 8 | 0 | 11 | 0 | 12 |
| 5 | 0 | 15 | 0 | 15 |
| 11 | 0 | 12 | 0 | 13 |