Table 1.
Inter-rater reproducability results for the initial group of three trainee clinicians for 14 pathoanatomic categories across the four coding rounds, expressed as a Kappa co-efficient and percentage agreement with 95% confidence intervals (95% CI)
| Pathoanatomic category | Reliability index (95%CI) | Coding round 1 | Coding round 2 | Coding round 3 | Coding round 4 |
|---|---|---|---|---|---|
| Intervertebral disc degeneration | Prevalence* Kappa Percentage agreement |
71.7% (50.2-93.1) 0.66 (0.54-0.77) 80.8% (73.8-87.9) |
94.0 (92.0-96.0) 0.94 (0.84-1.00) 95.8 (92.3-99.4) |
80.0 (80.0-80.0) n/a 98.2 (95.8-100.0) |
83.3 (80.1-86.6) 0.91 (0.80-1.00) 97.5 (94.7-100.0) |
| Intervertebral disc bulge | Prevalence Kappa Percentage agreement |
73.3% (70.1-76.6) 0.91 (0.78-1.00) 95.0% (94.7-100.0) |
87.0 (83.1-90.9) 0.86 (0.76-0.90) 92.5 (87.7-97.2) |
70.0 (70.0-70.0) 1.00 (0.87-1.00) 100.0 |
48.3 (45.1-51.6) 0.89 (0.76-1.00) 99.2 (97.5-100.0) |
| Intervertebral disc herniation | Prevalence Kappa Percentage agreement |
40.0% (34.3-45.7) 0.91 (0.73-1.00) 97.5% (94.7-100.0) |
61.0 (59.0-63.0) 1.00 (0.85-1.00) 100.0 |
40.0 (40.0-40.0) 0.98 (0.87-1.00) 96.5 (93.1-99.9) |
45.0 (45.0-45.0) 1.00 (0.82-1.00) 100.0 |
| Nerve root compromise | Prevalence Kappa Percentage agreement |
13.3% (4.7-22.0) n/a 97.5% (94.7-100.0) |
42.0 (31.9-52.1) 0.74 (0.49-1.00) 94.2 (90.0-98.4) |
31.7 (23.0-40.3) 0.63 (0.45-0.81) 93.0 (88.3-97.7) |
18.3 (15.1-21.6) 1.00 (0.75-1.00) 99.2 (97.5-100.0) |
| Modic change type 1 | Prevalence Kappa Percentage agreement |
26.7% (23.4-29.9) 0.86 (0.68-1.00) 95.0% (91.1-98.1) |
18.0 (15.6-20.4) 0.86 (0.60-1.00) 98.3 (96.0-100.0) |
20.0 (20.0-20.0) 1.00 (0.82-1.00) 100.0 |
18.3 (15.1-21.6) 0.86 (0.60-1.00) 100.0 |
| Modic change type 2 | Prevalence Kappa Percentage agreement |
25.0% (19.3-30.7) 0.76 (0.51-1.00) 97.5% (94.7-100.0) |
10.0 (5.6-14.4) n/a 98.3 (96.0-100.0) |
20.0 (20.0-20.0) 1.00 (0.82-1.00) 100.0 |
5.0 (5.0-5.0) n/a 100.0 |
| Spondylolisthesis | Prevalence Kappa Percentage agreement |
20.0% (20.0-20.0) n/a 97.5% (94.7-100.0) |
19.0 (17.0-21.0) 1.00 (0.75-1.00) 100.0 |
10.0 (10.0-10.0) 1.00 (0.82-1.00) 100.0 |
0.0 (0.0-0.0) n/a 100.0 |
| Stenosis | Prevalence Kappa Percentage agreement |
6.7% (3.4-9.9) n/a 97.5% (94.7-100.0) |
23.0 (20.6-25.4) 0.84 (0.59-1.00) 99.2 (97.5-100.0) |
25.0 (25.0-25.0) 0.95 (0.74-1.00) 99.1 (97.4-100.0) |
0.0 (0.0-0.0) n/a 100.0 |
| Scoliosis | Prevalence Kappa Percentage agreement |
25.0% (25.0-25.0) 1.00 (0.82-1.00) 100.0% |
6.0 (4.0-8.0) n/a 95.0 (91.1-98.9) |
5.0 (5.0-5.0) n/a 99.1(97.4-100.0) |
5.0 (5.0-5.0) n/a 100.0 |
| Osteophytes | Prevalence Kappa Percentage agreement |
5.0% (5.0-5.0) n/a 100.0% |
1.0 (0.0-3.0) n/a 100.0 |
10.0 (10.0-10.0) n/a 99.1 (97.4-100.0) |
10.0 (10.0-10.0) 1.00 (0.75-1.00) 99.2 (97.5-100.0) |
| Facet joint arthrosis | Prevalence Kappa Percentage agreement |
0.0% (0.0) n/a 100.0% |
14.0 (12.0-16.0) 0.68 (0.42-0.93) 95.0 (91.1-98.9) |
20.0 (20.0-20.0) 1.00 (0.82-1.00) 100.0 |
5.0 (5.0-5.0) n/a 100.0 |
| Endplate irregularities | Prevalence Kappa Percentage agreement |
# | 1.0 (0.0-3.0) n/a 99.2 (97.5-100.0) |
25.0 (19.3-30.7) 1.00 (0.82-1.00) 98.2 (95.8-100.0) |
18.3 (11.8-24.9) 0.74 (0.56-0.92) 97.5 (94.7-100.0) |
| Red flags | Prevalence (%) Kappa Percentage agreement |
# | 0.0 (0.0-0.0) n/a 100.0 |
8.3 (1.8-14.9) n/a 98.2 (95.8-100.0) |
0.0 (0.0-0.0) n/a 100.0 |
| High intensity zone | Prevalence (%) Kappa Percentage agreement |
# | 15.0 (11.9-18.1) 0.84 (0.59-1.00) 99.2 (97.5-100.0) |
26.7 (20.1-33.2) 0.85 (0.67-1.00) 98.2 (95.8-100.0) |
25.0 (25.0-25.0) 1.00 (0.82-1.00) 100.0 |
* The prevalence estimates are calculated on a 'whole person' level. That is, the presence of a particular pathology in a person at one or more vertebral levels was counted as a single instance.
95%CI = 95% confidence interval in brackets.
# not included in coding round 1.
n/a = kappa co-efficient not calculated. Calculation of kappa in STATA requires the prevalence of the pathology to be 10% to 90% at each and every vertebral level being compared. This restriction applies to the findings of all raters being compared.