Table 4.
Inter- and Intrarater Reliability and Associated COSMIN Scoresa
| Reliability, Mean (95% CI) | COSMIN Item | ||||||
|---|---|---|---|---|---|---|---|
| Study (Year) | Cutoff Score | Interrater | Intrarater | 6.4 | 6.5 | 6.6 | 6.7 |
| Aartun (2014)1 | ≥4/9 | κ = 0.65 (0.33 to 0.97) | κ = 0.66-1 (0.03 to 1) | NA | VG | D | A |
| ≥5/9 | κ = 0.56 (0.11 to 1.00) | κ = 1 | |||||
| Aslan (2006)3 | Composite | ICC = 0.82 Agreement = 42% |
ICC = 0.92 Agreement = 43% |
A | NA | NA | NA |
| Baumhauer (1995)5 | Composite | ρ = 1 | NA | I | D | A | |
| Boyle (2003)8 | Composite | ρ = 0.87 Agreement = 51% |
ρ = 0.86 Agreement = 69% |
D | NA | NA | NA |
| Bulbena (1992)11 | Each item | κ = 0.79-0.93 | D | VG | D | NA | |
| Cooper (2018)15 | ≥4/9 | κ = 0.96b (0.87 to 1.00) | κ = 1 | NA | VG | D | A |
| Erdogan (2012)18 | Each item | κ = 0.71-1.0 | κ = 0.81-1.0 | NA | VG | D | A |
| Erkula (2005)20 | ρ = 0.86 | ρ = 0.62 | D | NA | NA | NA | |
| Evans (2012)21 | Composite | ICC = 0.73 | ICC = 0.96-0.98 | VG | NA | NA | NA |
| Fritz (2005)22 | Composite | ICC = 0.72 (0.50 to 0.85) | VG | NA | NA | NA | |
| Glasoe (2002)23 | Composite | κ = 0.7 | NA | VG | D | A | |
| Hansen (2002)27 | ≥4/9 | κ = 0.44-0.82 | D | VG | D | A | |
| Hicks (2003)29 | Composite | ICC = 0.79 (0.68 to 0.87) | VG | NA | NA | NA | |
| Hirsch (2007)31 | ≥4/9 | ICC >0.84 | ICC > 0.89 | A | NA | NA | NA |
| Junge (2013)34 | Each item c | κ = 0.49-0.94, 0.30-0.84 | NA | VG | D | A | |
| ≥5/9c | κ = 0.64, 0.59d | ||||||
| Juul-Kristensen (2007)35 | Composite | ICC = 0.91 | VG | VG | D | A | |
| ≥5/9 | κ = 0.66 (0.30 to 1.02) 0.74 (0.46 to 1.02)d |
||||||
| Karim (2011)37 | NS | κ = 0.6 Agreement = 54%-100% |
NA | VG | D | NA | |
| Naal (2014)46 | Composite | κ = 0.82b (0.72 to 0.91) | NA | VG | VG | VG | |
| Pitetti (2015)49 | Composite | ICC = 0.88 | A | VG | D | A | |
| Each item | κ = 0.45-0.80 | ||||||
| Smith (2012)57 | Composite | κ = 0.00 (−0.16 to 0.17) | κ = 0.25 (0.03 to 0.51) | NA | VG | VG | A |
| Tarara (2014)57 | Modified compositee | κ = 0.64-0.69f
κ = 0.72g (0.62 to 0.82) |
Expert: κ = 0.69 (0.46 to 0.92) Novice: κ = 0.72-0.73 ([0.53-0.90] to [0.58-0.89]) |
NA | VG | VG | A |
| Vaishya (2013)62 | ≥4/9 | κ = 0.7 | NA | VG | D | A | |
| Vallis (2015)63 | Composite | ICC = 0.72-0.80 ([0.51-0.84] to [0.64-0.89]) κ = 0.71-0.82 ([0.67-0.90] to [0.50-0.84]) |
A | VG | VG | A | |
| van der Giessen (2001)65 | Composite | κ = 0.81 | NA | VG | D | A | |
aA, adequate; COSMIN, Consensus-Based Standards for the Selection of Health Measurement Instruments; D, doubtful; I, inadequate; ICC, intraclass correlation; NA, not available/applicable; VG, very good.
bObserver-participant reliability.
cPercentage agreement omitted.
dFor 2 distinct methods of performing Beighton score.
eModified composite scale: 0 = pain with test, 1 = 8-9 points, 2 = 6-7 points, 3 = 4-5 points, 4 = 2-3 points, 5 = 0-1 points.
fExpert-novice rater reliability.
gNovice-novice rater reliability.