Table 3.
Comparison of patients in automated versus manual reviews and measures of agreement for individual responses for categorical (nominal) variables.
| Variable name | Percent agreement, automated vs. manual | Sensitivity | Specificity | Kappa coefficient (95% CI, SE) | Strength of agreement based on Kappa coefficient |
|---|---|---|---|---|---|
| Sex, N = 652 | 99.7 | 99.7 | 0.99 (0.99–1.0, 0) | Almost perfect | |
| Male | 359/360 (99.7) | ||||
| Female | 291/292 (99.7) | ||||
| Total | 650/652 (99.7) | ||||
| Coronary artery disease, N = 540 | 98.6 | 97.4 | 0.90 (0.85–0.96, 0.03) | Almost perfect | |
| Yes | 73/74 (98.6) | ||||
| No | 454/466 (97.4) | ||||
| Total | 527/540 (97.6) | ||||
| Hypertension, N = 540 | 92.0 | 93.5 | 0.85 (0.80–0.89, 0.02) | Almost perfect | |
| Yes | 298/324 (92.0) | ||||
| No | 202/216 (93.5) | ||||
| Total | 500/540 (92.6) | ||||
| Congestive heart failure, N = 540 | 88.0 | 97.8 | 0.82 (0.74–0.90, 0.04) | Almost perfect | |
| Yes | 44/50 (88) | ||||
| No | 479/490 (97.8) | ||||
| Total | 523/540 (96.7) | ||||
| Chronic obstructive pulmonary disease, N = 540 | 92.7 | 96.3 | 0.80 (0.72–0.88, 0.04) | Substantial | |
| Yes | 51/55 (92.7) | ||||
| No | 467/485 (96.3) | ||||
| Total | 518/540 (95.9) | ||||
| Asthma, N = 540 | 93.7 | 95.8 | 0.81 (0.73–0.88, 0.04) | Almost perfect | |
| Yes | 59/63 (93.7) | ||||
| No | 457/477 (95.8) | ||||
| Total | 516/540 (95.6) | ||||
| Chronic kidney disease, N = 540 | 81.2 | 96.2 | 0.79 (0.72–0.85, 0.03) | Substantial | |
| Yes | 95/117 (81.2) | ||||
| No | 407/423 (96.2) | ||||
| Total | 502/540 (93) | ||||
| Diabetes mellitus, N = 540 | 92.1 | 96.3 | 0.89 (0.85–0.93, 0.02) | Almost perfect | |
| Yes | 176/191 (92.1) | ||||
| No | 336/349 (96.3) | ||||
| Total | 512/540 (94.8) | ||||
| Dyslipidemia/Hyperlipidemia, N = 540 | 88.9 | 86.4 | 0.61 (0.53–0.69, 0.04) | Substantial | |
| Yes | 80/90 (88.9) | ||||
| No | 389/450 (86.4) | ||||
| Total | 469/540 (86.9) | ||||
| ICU admission rate, N = 611 | 90.3 | 95.2 | 0.86 (0.82–0.90, 0.02) | Almost perfect | |
| Yes | 215/238 (90.3) | ||||
| No | 355/373 (95.2) | ||||
| Total | 570/611 (93.3) | ||||
| IMV rate, N = 582 | 87.7 | 98 | 0.85 (0.79–0.92, 0.03) | Almost perfect | |
| Yes | 64/73 (87.7) | ||||
| No | 499/509 (98) | ||||
| Total | 563/582 (96.7) | ||||
| NIMV rate, N = 581 | 83.3 | 99.3 | 0.80 (0.66–0.95, 0.07) | Substantial | |
| Yes | 15/18 (83.3) | ||||
| No | 559/563 (99.3) | ||||
| Total | 574/581 (98.3) | ||||
| HFNC rate, N = 581 | 100 | 98.9 | 0.86 (0.75–0.97, 0.06) | Almost perfect | |
| Yes | 19/19 (100) | ||||
| No | 556/562 (98.9) | ||||
| Total | 575/581 (99) | ||||
| ECMO rate, N = 581 | 72.7 | 99.3 | 0.69 (0.47–0.91, 0.11) | Substantial | |
| Yes | 8/11 (72.7) | ||||
| No | 566/570 (99.3) | ||||
| Total | 574/581 (98.8) | ||||
| ICU discharge status, N = 172 | 100 | 100 | 1.0 (1–1, 0.0) | Almost perfect | |
| Death | 9/9 (100) | ||||
| Alive | 163/163 (100) | ||||
| Total | 172/172 (100) | ||||
| Hospital discharge status, N = 541 | 90 | 100 | 0.94 (0.88–1, 0.03) | Almost perfect | |
| Death | 27/30 (90) | ||||
| Alive | 511/511 (100) | ||||
| Total | 538/541 (99.4) |
CI, Confidence interval; ECMO, Extracorporeal membrane oxygenation; HFNC, High Flow Nasal Canula; ICU, Intensive Care Unit; IMV, Invasive Mechanical Ventilation; LOS, Length of stay; NIMV; Non-Invasive Mechanical Ventilation; PCC, Pearson Interclass Correlation Coefficient; SE, Standard Error.