Table 4.
Reliability, inter and intra objectivity κ values and standard error
| Evaluator | C1 | C2 | C3 | C4 | C5 | C6 | C7 | C8 | C9 |
|---|---|---|---|---|---|---|---|---|---|
| T1-T2 (R) | .1 .00 | .1 .00 | .65 .32 | .86 .09 | .59 .14 | .65 .13 | .78 .21 | .1 .00 | .1 .00 |
| GS X B (IR) | .70 .16 | .79 .12 | .51 .16 | .77 .13 | .83 .13 | .79 .09 | .88 .12 | .68 .13 | 1 .00 |
| GS X C (IR) | .73 .18 | .79 .18 | .57 .16 | .60 .14 | .88 .08 | .75 .09 | .69 .16 | .47 .14 | .90 .06 |
| BX C (IR) | .70 .16 | .90 .17 | .78 .09 | .82 .09 | .94 .00 | .88 .07 | .78 .15 | .79 .09 | .90 .00 |
| GS (I) | .93 .72 | .1 .00 | .84 .16 | .86 .09 | .87 .09 | .83 .11 | .65 .32 | .80 .11 | .47 .30 |
| PA | 97% | 100% | 97% | 93% | 93% | 93% | 97% | 90% | 93% |
| B (I) | .80 .11 | .86 .09 | .81 .12 | .87 .09 | .77 .11 | .91 .09 | .89 .11 | .87 .09 | .73 .14 |
| PA | 90% | 93% | 93% | 93% | 90% | 97% | 97% | 93% | 90% |
| C (I) | .93 .06 | .73 .12 | 71 .18 | .63 .15 | .93 .06 | .90 .09 | .84 .16 | .93 .07 | .63 .23 |
| PA | 97% | 87% | 93% | 83% | 97% | 97% | 97% | 97% | 93% |
Note –R = Reliability; T1 = Time 1; T2 = Time 2; IR = Inter-raters; I = Intra-raters; C = Criteria; PA = Percentage Agreement