Table 2.
Performance Measures for All Subjects in the Study
| MD1 | MD2 | MD3 | MD4 | SymText | AAKS | KS | Lay1 | Lay2 | Lay3 | |
|---|---|---|---|---|---|---|---|---|---|---|
| Pneumonia (N = 47): | ||||||||||
| Recall | 0.98 (0.94–1.00) | 0.96 (0.90–1.00) | 1 (0.80–0.98) | 0.98 (0.94–1.00) | 0.94 (0.87–1.00) | 0.94 (0.87–1.00) | 0.83 (0.72–0.94) | 0.89 (0.80–0.98) | 0.72 (0.59–0.85) | 0.89 (0.80–0.98) |
| Precision | 0.96 (0.90–1.00) | 0.94 (0.87–1.00) | 0.96 (0.91–1.00) | 0.98 (0.94–1.00) | 0.96 (0.90–1.00) | 0.52 (041–0.63) | 0.83 (0.72–0.94) | 0.81 (0.70–0.92) | 0.87 (0.76–0.98) | 0.95 (0.89–1.00) |
| Specificity | 0.99 (0.98–1.00) | 0.99 (0.98–1.00) | 0.99 (0.98–1.00) | 0.99 (0.98–1.00) | 0.99 (0.98–1.00) | 0.83 (0.78–0.88) | 0.97 (0.95–0.99) | 0.96 (0.94–0.98) | 0.98 (0.96–1.00) | 0.99 (0.98–1.00) |
| Aspiration (N = 6): | ||||||||||
| Recall | 1 | 1 | 1 | 1 | 1 | 0.83 (0.53–1.00) | 1 | 0.66 (0.28–1.00) | 0.83 (0.53–1.00) | 0.83 (0.53–1.00) |
| Precision | 1 | 1 | 1 | 1 | 0.75 (0.45–1.00) | 0.62 (0.28–0.96) | 0.85 (0.59–1.00) | 0.80 (0.45–1.00) | 0.35 (0.10–0.60) | 1 |
| Specificity | 1 | 1 | 1 | 1 | 0.99 (0.98–1.00) | 0.98 (0.96–1.00) | 0.99 (0.98–1.00) | 0.99 (0.98–1.00) | 0.97 (0.95–1.00) | 1 |
| Infiltrate (N = 132): | ||||||||||
| Recall | 0.80 (0.73–0.87) | 0.93 (0.89–0.97) | 0.79 (0.72–0.86) | 0.77 (0.70–0.84) | 0.84 (0.78–0.90) | 0.56 (0.48–0.64) | 0.76 (0.69–0.83) | 0.045 (0.01–0.08) | 0.13 (0.07–0.19) | 0.14 (0.08–0.20) |
| Precision | 0.97 (0.94–1.00) | 0.92 (0.87–0.97) | 0.96 (0.92–1.00) | 0.93 (0.88–0.98) | 0.87 (0.81–0.93) | 0.92 (0.86–0.98) | 0.82 (0.75–0.89) | 0.75 (0.45–1.0) | 0.94 (0.83–1.00) | 0.63 (0.46–0.80) |
| Specificity | 0.98 (0.96–1.00) | 0.94 (0.90–0.98) | 0.97 (0.94–1.00) | 0.95 (0.92–0.98) | 0.90 (0.85–0.95) | 0.96 (0.93–0.99) | 0.86 (0.81–0.91) | 0.98 (0.96–1.00) | 0.99 (0.97–1.00) | 0.93 (0.89–0.97) |
| Support Pneumonia (N = 112): | ||||||||||
| Recall | 0.94 (0.90–0.98) | 0.93 (0.88–0.98) | 0.97 (0.94–1.00) | 0.91 (0.86–0.96) | 0.95 (0.91–0.99) | 0.79 (0.71–0.87) | 0.87 (0.81–0.93) | 0.38 (0.29–0.47) | 0.58 (0.49–0.67) | 0.41 (0.32–0.50) |
| Precision | 0.83 (0.76–0.90) | 0.92 (0.87–0.97) | 0.88 (0.82–0.94) | 0.84 (0.77–0.91) | 0.78 (0.71–0.85) | 0.63 (0.55–0.71) | 0.70 (0.62–0.78) | 0.86 (0.76–0.96) | 0.85 (0.77–0.93) | 0.97 (0.92–1.00) |
| Specificity | 0.88 (0.83–0.93) | 0.95 (0.92–0.98) | 0.92 (0.88–0.96) | 0.89 (0.84–0.94) | 0.85 (0.80–0.90) | 0.71 (0.64–0.78) | 0.77 (0.71–0.83) | 0.96 (0.93–0.99) | 0.93 (0.89–0.97) | 0.99 (0.98–1.00) |
note: The 95 percent confidence intervals (95% CI) are shown in parentheses. N is the number of reports containing the concept, as judged by the gold standard.