Table 3.
Quality Analysis of Reliability Studies Using the Quality Appraisal of Reliability Studies Tool.24,25
| Questions | Gribble et al. (2013) | Hertel et al. (2000) | Hyong et al. (2014) | Kinzey et al. (1998) | van Lieshout et al. (2016) | Lopez-Plaza et al. (2018) | Munro et al. (2010) | Plisky et al. (2009) | Shaffer et al. (2013) |
|---|---|---|---|---|---|---|---|---|---|
| 1. Representative sample | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 2. Representative raters | Yes | Unclear | Unclear | Unclear | Yes | Unclear | Unclear | Yes | Yes |
| 3. Blinding (other raters) | N/A | Unclear | Unclear | N/A | Yes | N/A | N/A | Yes | Yes |
| 4. Blinding (own findings) | Unclear | Unclear | Unclear | Unclear | No | Unclear | Unclear | No | N/A |
| 5. Blinding (reference/disease) | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 6. Blinding (clinical information) | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 7. Blinding (additional cues) | Unclear | Unclear | Unclear | Unclear | Unclear | Unclear | Unclear | Unclear | Unclear |
| 8. Examination order varied | Yes | Yes | No | Unclear | No | No | Yes | No | No |
| 9. Appropriate time interval | Yes | Yes | Unclear | Yes | Yes | Unknown | Yes | Yes | Yes |
| 10. Test appropriate | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| 11. Appropriate statistics | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Internal validity (%) (Q: 3-9) | 2/4 = 50.00 | 2/5 = 40.00 | 0/5 = 00.00 | 1/4 = 25.00 | 2/5 = 40.00 | 0/4 = 0.00 | 2/4 = 50.00 | 2/5 = 40.00 | 2/4 = 50.00 |
| External validity (%) (Q: 1-2, 10) | 3/3 = 100 | 2/3 = 66.67 | 2/3 = 66.67 | 2/3 = 66.67 | 3/3 = 100 | 2/3 = 66.67 | 2/3 = 66.67 | 3/3 = 100 | 3/3 = 100 |
| Percentage of Yes (%) | 6/8 = 75.00 | 5/9 = 55.56 | 3/9 = 33.33 | 4/8 = 50.00 | 6/9 = 66.67 | 3/8 = 37.50 | 5/8 = 62.50 | 6/9 = 66.67 | 6/8 = 75.00 |
| Level of Evidence | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |