Table 3.
Kappa Scores among Raters for Both Clinical Judgment and VA/DoD CPG Criteria Ratings (n = 64 Events)
| Rater 1 | Rater 2 | Rater 3 | Rater 4 | |
|---|---|---|---|---|
| Clinical judgment | ||||
| Rater 1 | — | 0.97 | 0.75 | 0.72 |
| Rater 2 | 0.97 | — | 0.72 | 0.75 |
| Rater 3 | 0.75 | 0.72 | — | 0.50 |
| Rater 4 | 0.72 | 0.75 | 0.50 | — |
| Algorithm | 1.00 | 0.97 | 0.75 | 0.72 |
| VA/DoD CPG criteria rating | ||||
| Rater 1 | — | 0.97 | 1.00 | 1.00 |
| Rater 2 | 0.97 | — | 0.97 | 0.97 |
| Rater 3 | 1.00 | 0.97 | — | 1.00 |
| Rater 4 | 1.00 | 0.97 | 1.00 | — |
| Algorithm | 1.00 | 0.97 | 1.00 | 1.00 |
Reported numbers are kappa outcomes between raters. Clinical judgment allowed raters to utilize the same criteria that they would use in clinic to determine if an event should be considered to have resulted in traumatic brain injury (TBI). Criteria rating instructed raters to only utilize the provided Veterans Administration/Department of Defense Clinical Practice Guideline (VA/DoD CPG) criteria for definition of a TBI. Algorithm refers to the Mid-Atlantic Mental Illness Research, Education, and Clinical Center (MIRECC) Assessment of TBI (MMA-TBI) algorithm based on VA/DoD CPG criteria.