Skip to main content
. 2023 Feb 24;23(Suppl 1):40. doi: 10.1186/s12911-023-02136-0

Table 2.

The interrater reliability of the reviews among the three reviewers

Version Procedure Percent agreement for 1st review Fleiss’ Kappa for 1st review Percent agreement for 2nd review Fleiss’ Kappa for 2nd review
CIT_V1.1 Concatenation 0.85 0.46 0.91 0.67
CIT_V1.2 Anchoring 0.90 0.60 0.94 0.78
CIT_V2.1 Concatenation 0.87 0.43 0.92 0.67
CIT_V2.2 Anchoring 0.91 0.54 0.94 0.72
CIT_V3.1 Concatenation 0.84 0.37 0.92 0.69
CIT_V3.2 Anchoring 0.92 0.66
CIT_V4.1 Concatenation 0.83 0.23
CIT_V4.2 Anchoring 0.96 0.58