Table 3.

Three-facet generalizability coefficients. Agreement on TMJ score with three sources of variance—patient, reader and aid or reader specialty. The 95% confidence interval of measurement reflects the measurement error interval around a measured score, in the units of the respective JAMRIS-TMJ domain. Abbreviations: A, aid; J, temporomandibular joint; Non-Rad, non-radiologist clinician reader; P, patient; R, reader; Rad, radiologist reader; S, specialty (binary, radiologist or non-radiologist clinician).

Pooling the Two Readings from the Same Readers	Generalizability Coefficient						95% CI of Measurement (+/− Percentage Points)
	Inflammatory Domain			Damage Domain			Inflammatory Domain			Damage Domain
Measurement Scenario (Main Effects Definitions)	+/− Atlas		Atlas +/− Tutorial Rad	+/− Atlas		Atlas +/− Tutorial Rad	+/− Atlas		Atlas +/− Tutorial Rad	+/− Atlas		Atlas +/− Tutorial Rad
	Non-Rad	Rad		Non-Rad	Rad		Non-Rad	Rad		Non-Rad	Rad
Between readers of the same calibration (R random, P and A fixed)	0.49	0.69	0.81	0.39	0.78	0.80	35	28	21	47	24	23
Same reader with vs. without aid (A random, P and R fixed)	0.65	0.86	0.83	0.59	0.77	0.88	29	17	20	33	24	18
Between readers with different calibration (A and R random, P fixed)	0.41	0.67	0.75	0.37	0.76	0.78	37	29	25	48	25	25
Between readers with different calibration, averaging two readings per reader (A and R random, P fixed, A/2)	0.50	0.73	0.83	0.43	0.86	0.84	31	26	20	42	18	20
Between readers with different calibration, averaging two readers at a time (A and R random, P fixed, R/2)	0.56	0.81	0.84	0.53	0.86	0.88	28	21	19	35	18	17
Between readers of the same calibration, averaging two readers at a time (R random, P and A fixed, R/2)	0.65	0.82	0.90	0.56	0.87	0.89	25	20	15	34	17	16
Between right and left joints when scored by different readers with different calibration (J, A and R random)	0.12	0.20	0.27	0.23	0.39	0.34	54	56	55	65	54	56
Between right and left joints when scored by the same reader with the same calibration (J random, A and R fixed)	0.30	0.39	0.43	0.39	0.47	0.44	48	47	47	51	50	50
Pooling the Readings from Same Calibration	Generalizability Coefficient						95% CI of Measurement
Measurement Scenario (Main Effects Definitions)	Inflammatory Domain			Damage Domain			Inflammatory Domain			Damage Domain
	Baseline	Atlas	Tutorial	Baseline	Baseline	Atlas	Tutorial	Baseline	Baseline	Atlas	Tutorial	Baseline
Between radiologists and non-radiologists (R and S random, P fixed)	0.56	0.59	0.63	0.64	0.61	0.70	36	31	29	30	33	28
Between readers of the same specialty (R random, S and P fixed)	0.67	0.69	0.67	0.68	0.70	0.76	29	27	28	29	29	26
Between right and left joints when scored by a radiologist and a clinician (J, R and S random)	0.12	0.25	0.26	0.35	0.32	0.30	60	53	53	56	55	56
Between right and left joints when scored by the same reader (J random, R and S fixed)	0.34	0.43	0.40	0.49	0.41	0.42	48	45	46	49	50	50