Table 4. Various measurements of reviewer agreement in three studies involving manuscript review at medical journals.
Study | Journal (# of manuscripts) | Overall agreement | Positive agreement | Negative agreement | Cohen's Kappa | Fleiss' Kappa | Gwet's Kappa |
Rothwell & Martyn 2000 | Journal A (179) | 57% | 65% | 45% | 0.10 | 0.09 | 0.18 |
Journal B (116) | 71% | 77% | 60% | 0.37 | 0.37 | 0.46 | |
Yadollahie et al. 2004 | Iranian J Med Sciences (28) | 46% | 44% | 48% | −0.07 | −0.07 | 0.12 |
Baethge et al. 2012 | Dtsch Arztebl Int (206) | 75% | 84% | 31% | 0.16 | 0.15 | 0.63 |
Positive agreement denotes agreement regarding acceptance, negative agreement refers to agreement regarding rejection of a manuscript.