Table 2.

Inter-rater reliability coefficients

Item	Weighted κ (95% CI)
Core criteria (n=135 MIDs)
Is the patient or necessary proxy responding directly to both the PROM and the anchor?	0.80 (0.64 to 0.95)
Is the anchor easily understandable and relevant for patients or necessary proxy?	0.70 (0.66 to 0.76)
Has the anchor shown good correlation with the PROM?	0.90 (0.86 to 0.94)
Is the MID precise?	0.80 (0.67 to 0.87)
Does the threshold or difference between groups on the anchor used to estimate the MID reflect a small but important difference?	0.74 (0.71 to 0.79)
Additional criterion for transition rating anchors (n=137 MIDs)
Is the amount of elapsed time between baseline and follow-up measurement for MID estimation optimal?	0.94 (0.91 to 0.96)

κ=Cohen’s kappa; MID=minimal important difference; PROM=patient reported outcome measure.

The difference in the number of MIDs included in the reliability analysis for the credibility criteria for the transition rating anchors is because of the additional random sample of MIDs retrieved to meet the required sample size (see sampling method section in the methods).