Table 2.
Accuracy of Algorithms Reviewed
| Study*† | Code system | Validation | Number of cases algorithm + | Number of cases algorithm − | Algorithm accuracy | Probability of pneumonia‡ | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pneumonia + | Pneumonia − | Pneumonia + | Pneumonia − | Sensitivity | Specificity | PPV | NPV | LR+ | LR− | Algorithm + | Algorithm − | |||
| Wiese18 | ICD-9-CM | No | 328 | 12 | 0 | 0 | - | - | 96.5% | - | - | - | - | - |
| Ahmed19 | ICD-9-CM | Yes¶ | 94 | 26 | 28 | 1295 | 77.0% | 98.0% | 78.3% | 97.9% | 38.5 | 0.2 | 49.7% | 0.5% |
| Holland-Bill20 | ICD-10-CM | No | 83 | 10 | 0 | 0 | - | - | 89.2% | - | - | - | - | - |
| Meropol21 | Read | No | 51 | 8 | 0 | 0 | - | - | 86.4% | - | - | - | - | - |
| Yu22 | ICD-9-CM (a) | No | 325 | 33 | 189 | 407 | 63.2% | 92.5% | 90.8% | 68.3% | 8.4 | 0.4 | 17.7% | 1.0% |
| ICD-9-CM (b) | No | 1214 | 150 | 651 | 870 | 65.1% | 85.3% | 89.0% | 57.2% | 4.4 | 0.4 | 10.1% | 1.0% | |
| Grijalva23 | ICD-9-CM (a) | No | 135 | 26 | 0 | 0 | - | - | 83.9% | - | - | - | - | - |
| ICD-9-CM (b) | No | 103 | 5 | 0 | 0 | - | - | 95.4% | - | - | - | - | - | |
| ICD-9-CM (c) | No | 32 | 21 | 0 | 0 | - | - | 60.4% | - | - | - | - | - | |
| Skull24 | ICD-10-CM | No | 1509 | 644 | 34 | 1156 | 97.8% | 64.2% | 70.1% | 97.1% | 2.7 | 0.03 | 6.5% | 0.1% |
| Gedeborg25 | ICD-9-CM (a) | No | 89 | 165 | 96 | 3831 | 48.1% | 95.9% | 35.0% | 97.6% | 11.7 | 0.5 | 23.1% | 1.3% |
| ICD-9-CM (b) | No | 40 | 62 | 88 | 3991 | 31.3% | 98.5% | 39.2% | 97.8% | 20.9 | 0.7 | 34.9% | 1.8% | |
| ICD10-CM (c) | No | 116 | 135 | 105 | 3078 | 52.5% | 95.8% | 46.2% | 96.7% | 12.5 | 0.5 | 24.3% | 1.3% | |
| ICD10-CM (d) | No | 71 | 45 | 115 | 3203 | 38.2% | 98.6% | 61.0% | 96.5% | 27.3 | 0.6 | 41.2% | 1.5% | |
| van de Garde10 | ICD-9-CM | No | 212 | 0 | 81 | 0 | 72.4% | - | - | - | - | - | - | - |
| Schneeweiss26 | ICD-9-CM | No | 16 | 7 | 0 | 0 | - | - | 69.6% | - | - | - | - | - |
| Aronsky8 | ICD-9-CM (a) | No | 109 | 20 | 90 | 10,529 | 54.8% | 99.8% | 84.5% | 99.2% | 274.0 | 0.5 | 87.5% | 1.3% |
| ICD-9-CM (b) | No | 136 | 23 | 63 | 10,526 | 68.3% | 99.8% | 85.5% | 99.4% | 341.5 | 0.3 | 89.8% | 0.8% | |
| ICD-9-CM (c) | No | 139 | 25 | 60 | 10,524 | 69.8% | 99.8% | 84.8% | 99.4% | 349.0 | 0.3 | 89.9% | 0.8% | |
| DRG (d) | No | 89 | 13 | 110 | 10,536 | 44.7% | 99.9% | 87.3% | 99.0% | 447.0 | 0.6 | 92.0% | 1.5% | |
| DRG (e) | Yes§ | 124 | 21 | 75 | 10,528 | 62.3% | 99.8% | 85.5% | 99.3% | 311.5 | 0.4 | 88.9% | 1.0% | |
| Whittle27 | ICD-9-CM (a) | No | 79 | 7 | 15 | 43 | 84.0% | 86.0% | 91.9% | 74.1% | 6 | 0.2 | 13.3% | 0.5% |
| DRG (b) | No | 70 | 5 | 24 | 45 | 74.5% | 90.0% | 93.3% | 65.2% | 7.5 | 0.3 | 16.1% | 0.8% | |
| Marrie28 | ICD-9-CM | No | 73 | 54 | 32 | 0 | 69.5% | - | 57.5% | - | - | - | - | - |
*For detailed description of the algorithms, please refer to Appendix 3. †Letters in (), if present, corresponded with the same letters in Appendix 3. ‡Assuming a true prevalence of pneumonia in population is 2.5%. ¶Data shown corresponds to the internal validation analysis of this study. §This algorithm is considered external validation of DRG-based algorithm of Whittle’s study