Table 7.

Summary of IRR statistics for nominal variables.

Statistical Family	Variant	Uses	R command	SPSS command	Reference(s)
Kappa (two coders)	Cohen’s kappa	No bias or prevalence correction	cohen.kappa(…)$kappa.c (concord package)	None*	Cohen (1960)
	Siegel & Castellan’s kappa	Bias correction	cohen.kappa(…)$kappa.sc (concord package)	CROSSTABS \STATISTICS=KAPPA …	Siegel & Castellan (1988, pp. 284- 291)
	Byrt et al’s kappa	Prevalence correction	cohen.kappa(…)$kappa.bbc (concord package)	None*	Byrt et al. (1993)
	Cohen’s weighted kappa	Disagreements differentially penalized (e.g., with ordinal variables)	kappa2(…, weight = c(“equal”, “squared’)) (irr package)	None, but quadratic weighting is identical to a two-way mixed, single-measures, consistency ICC	Cohen (1968)
Kappa-like Pi-family statistics (three or more coders)	Fleiss’s kappa	Raters randomly sampled for each subject	kappam.fleiss(…) (irr package)	None*	Fleiss (1971); Gross (1986)
	Light’s kappa,	Average kappa across all rater pairs	kappam.light(…) (irr package)	None*, but two-rater kappa can be computed for each coder pair then averaged manually	Light (1971)
	Davies & Fleiss’s kappa	Kappa-like coefficient across all rater pairs using average P(e)	None*	None*	Davies (1982)

Note:

^*Macros and syntax files may be available for computing statistical variants that are not natively available in SPSS or the R concord or irr packages. The reader is referred to the SPSSX Discussion hosted by the University of Georgia (http://spssx-discussion.1045642.n5.nabble.com) as one example repository of unverified user-created macros and syntax files created for SPSS.