Table 7.

Results for the beam mammogram study⁹ where 104 radiologists (J =104) classified mammograms of 148 patients (I = 148) using an ordered BIRADS scale with C = 5 categories (1 = normal; 2=benign; 3=probably benign; 4=possibly malignant; 5 = probably malignant).

Parameter	Symbol	Estimate	S.E.	Z-value
Ordinal GLMM:
Thresholds: (α0 = −∞, α5 = +∞)
Between categories 1 and 2	α1	−0.897	0.135	−6.643
Between categories 2 and 3	α2	−0.197	0.135	−1.460
Between categories 3 and 4	α3	0.761	0.135	5.630
Between categories 4 and 5	α4	2.539	0.137	18.574
Subject random effect variance	${\sigma }_u^2$	2.442	0.427
Rater random effect variance	${\sigma }_v^2$	0.158	0.073
Rho	ρ	0.678	0.026
GLMM-based observed agreement	p0	0.430
GLMM-based observed association (quadratic weights)	p0a	0.907
Measures of agreement:
Model-based (unweighted) kappa  (Nelson and Edwards41)	κm	0.241	0.015
Fleiss’ kappa (Fleiss47)	κF	0.297	0.001
Light and Conger’s kappa  (Light,48 Conger49)	κLC	0.298
Measures of association:  (with quadratic weights)
Model-based weighted kappa	κma	0.475	0.022
Shrout and Fleiss’ ICC[2,1]  (Shrout and Fleiss28)		0.652	95% c.i. = (0.601, 0.706)
Cohen’s GLMM-based weighted kappa	κGLMM,a	0.611