Table 3.
Reliability and agreement measures. Absolute and relative contributions of the different sources of variation to work capacity ratings: work capacity ratings, total variance and variance components (psychiatrists, patients, residuals), reliability and agreement parameters for ‘last job’ and ‘alternative work’ in RELY 1 and RELY 2
| Reference for WC | WC Mean (95%CI) | Total variance | Variance components Absolute variance (Relative variance) |
Reliability | Agreement | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Proportion of WC ratings between two psychiatrists whose ratings differed equal or less than the ‘maximum acceptable difference’ of 25 percentage points WC | ‘Standard error of measurement’ (95%CI) | ‘Maximum acceptable difference’ (95%CI) | ||||||||
| Psychiatrists | Patients | Residuals | ICCabs.agree (95%CI) | reported in natural units | reported in natural units | |||||
| Last job |
RELY 1 N = 120 |
43.6% (34.1–53.2) |
1092 |
263 (24%) |
414 (38%) |
415 (38%) |
0.38 (0.19–0.55) |
52.2% (94/180) |
26.0% WC (21.5–31.0) |
72.2% WC (59.5–86.0) |
|
RELY 2 N = 160 |
46.3% (39.9–52.6) |
1064 |
76 (7%) |
495 (47%) |
493 (46%) |
0.47 (0.29–0.61) |
61.7% (148/240) |
23.9% WC (20.8–27.0) |
66.1% WC (57.7–74.9) |
|
| Alternative work |
RELY 1 N = 119 |
55.0% (47.3–62.8) |
1060 |
88 (8%) |
457 (43%) |
515 (49%) |
0.43 (0.22–0.60) |
61.6% (112/177) |
24.6% WC (20.9–28.4) |
68.1% WC (57.9–78.8) |
|
RELY 2 N = 155 |
62.9% (57.7–68.0) |
669 |
50 (7%) |
292 (44%) |
328 (49%) |
0.44 (0.25–0.59) |
73.6% (170/231) |
19.4% WC (16.9–22.0) |
53.8% WC (46.8–61.0) |
|
Legend: WC: work capacity, % WC = absolute percentage points in work capacity, ICCabs.agree = intraclass correlation coefficient (agreement variant); CI: confidence interval