Table 4.
Agreement Between Initial and Final Rating by Panelist*
Panelist | Overall agreement | Overall kappa** | Agreement if final rating is inappropriate | Agreement if final rating is not inappropriate |
---|---|---|---|---|
A | 90.2% | 0.80 | 87% | 93% |
B | 81.1% | 0.59 | 58% | 98% |
C | 78.1% | 0.53 | 57% | 94% |
For ratings with unanimous agreement between panelists by the final round;
P-value for all kappa calculations <0.0001