Table 2.
Game design element coding inter-rater reliability by session
| First session IRR |
Second session IRR |
Third session IRR |
Fourth session IRR |
Fifth session IRR |
||||||
|---|---|---|---|---|---|---|---|---|---|---|
| N | % | N | % | N | % | N | % | N | % | |
| Interventions | 5 | 100 | 5 | 100 | 5 | 100 | 10 | 100 | 10 | 100 |
| Gamification elements coded | 75 | 100 | 75 | 100 | 75 | 100 | 150 | 100 | 150 | 100 |
| Individual coding—agreement | 48 | 64.0 | 18 | 76.0 | 67 | 89.3 | 127 | 84.7 | 132 | 88.0 |
| Agreement after discussion | 75 | 100 | 75 | 100 | 75 | 100 | 150 | 100 | 150 | 100 |
| K | Std. error | K | Std. error | K | Std. error | K | Std. error | K | Std. error | |
| K statistic* | 0.24 | 0.08 | 0.26 | 0.11 | 0.65 | 0.12 | 0.54 | 0.08 | 0.65 | 0.08 |
*
All p < 0.05