Table 3.

Group I—Agreement on Traditional Chinese Medicine Pattern Differentiation: Rank Order of Mean Cohen's Kappa^a

Trial	Kappa mean (range, CI or p-value)	Q score	Prior group discussion and practice	Outcome of post hoc analysis	Dx prompts on rating form	Rating categories included in analysis	Raters rating each subject	Subjects included in analysis
Zhang 2008b,c	[0.59] (0.49 to 0.76, p > 0.05)	8.7	*	*	*	9	3	42
Grant Primary	0.56 (CI 0.25 to 0.81, p < 0.001)	8.5			*	3	2	27
MacPherson Secondary	[0.48] (0.25 to 0.67)	8	*		*	3	2	48
Tangb	0.472 (p = 0.003)	5	*	*	*	3	2	22
Sung Phase IIIb	[0.36] (0.27 to 0.51)	6.7	*		*	4	4	65
MacPherson Primary	[0.31] (0 to 0.59)	8	*		*	3	2	87
Zhang 2004 More stringentb,c	[0.26] (0.23 to 0.30)	9.2	*	*	*	[10]	3	39
Birkeflet 4 categories	[0.19] (0.13 to 0.51)	8.5		*		4	2	54
Sung Phase Ib	[0.11] (0.03 to 0.32)	6.7			*	4	4	39
Birkeflet 12 categories	[0.07] (−0.01 to 0.22)	8.5		*		12	2	54
Mean	0.34
Median	0.34

[] Values calculated from tabular data in the publication (E.J.); p = values reported from chi square tests; median between ranks occurs between Sung and MacPherson; Dx, diagnosis; ^* indicates that the trial had the feature named at the head of the column.

^aThe range of kappa values for agreement by “at least two” raters reported in O'Brien 2009a²⁸ omitted because its calculation is not described.

^bkappa assumed to be Cohen's.

^cKappa values “weighted by number of pairs involved.”