Table 4.
Sensitivity specificity, percent agreement, and kappa for direct observation of health surveillance assistant classification and treatment of illness compared with re-examination, Malawi*
| Illness | Sensitivity | Specificity | % Agreement† | Kappa statistic (95% CI) | |||
|---|---|---|---|---|---|---|---|
| No. | % (95% CI) | No. | % (95% CI) | ||||
| Uncomplicated illness | |||||||
| Fast breathing | Classification | 34/58 | 59 (46–72) | 209/256 | 82 (75–88) | 77 | 0.35 (0.23–0.47) |
| Treatment | 36/57 | 63 (50–77) | 173/230 | 75 (68–83) | 73 | 0.31 (0.19–0.43) | |
| Fever | Classification | 226/242 | 93 (90–97) | 71/72 | 99 (96–100) | 95 | 0.86 (0.79–0.92) |
| Treatment | 198/229 | 87 (81–92) | 52/53 | 98 (94–100) | 89 | 0.70 (0.60–0.79) | |
| Diarrhea | Classification | 86/94 | 92 (86–97) | 218/220 | 99 (98–100) | 97 | 0.92 (0.88–0.97) |
| Treatment | 63/70 | 90 (82–98) | 149/150 | 99 (98–100) | 96 | 0.91 (0.86–0.97) | |
| Severe illness | |||||||
| Fast breathing | Classification | 13/19 | 68 (41–95) | 42/49 | 86 (75–96) | 81 | 0.53 (0.31–0.76) |
| Treatment | 6/19 | 32 (3–60) | 23/45 | 51 (35–67) | 45 | −0.15 (−0.38 to 0.07) | |
| Fever | Classification | 44/51 | 86 (76–96) | 16/17 | 94 (82–100) | 88 | 0.72 (0.54–0.90) |
| Treatment | 17/47 | 36 (22–51) | 11/17 | 65 (38–91) | 44 | 0.01 (−0.18 to 0.19) | |
| Diarrhea | Classification | 8/14 | 57 (26–89) | 46/54 | 85 (74–96) | 79 | 0.40 (0.14–0.66) |
| Treatment | 5/10 | 50 (12–88) | 18/43 | 42 (23–61) | 43 | −0.05 (−0.24 to 0.15) | |
*
CI = confidence interval. See Figure 1 for definitions of sensitivity and specificity for classification and treatment.
†
Percent agreement is a weighted average of sensitivity and specificity. For example, 77% agreement for uncomplicated fast breathing (first row) was calculated as (34 + 209)/(58 + 256).