Table 2.
Model performance on different CRWD test sets
| n |
In-hospital mortality |
Mechanical ventilation |
Prolonged hospital stay (> 7 days) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Logistic regression | Light gradient boost machine | CovRNN binary prediction | CovRNN survival prediction* | Logistic regression | Light gradient boost machine | CovRNN binary prediction | CovRNN survival prediction* | Logistic regression | Light gradient boost machine | CovRNN binary prediction | ||
| Multi-hospital test set | 48 781 | 90·3% (89·8–90·8) | 91·5% (91·1–92·0) | 93·0% (92·6–93·4) | 86·0% (85·1–86·9) | 89·5% (89·1–89·9) | 91·2% (90·8–91·5) | 92·9% (92·6–93·2) | 92·6% (92·2–93·0) | 80·0% (79·5–80·4) | 81·7% (81·3–82·2) | 86·5% (86·2–86·9) |
| Hospital 1 | 3469 | 88·8% (86·9–90·5) | 91·0% (89·5–92·4) | 91·8% (90·3–93·2) | 86·0% (83·2–88·5) | 86·7% (85·1–88·4) | 88·4% (87·0–89·9) | 91·5% (90·2–92·8) | 90·8% (89·4–92·2) | 77·3% (75·5– 79·1) | 78·5% (76·7–80·2) | 87·2% (85·8–88·4) |
| Hospital 2 | 706 | 94·6% (91·9–96·9) | 95·1% (92·7–97·2) | 97·0% (95·2–98·6) | 91·6% (87·5–94·8) | 93·5% (90·7–95·8) | 95·6% (93·8–97·1) | 96·0% (94·2–97·7) | 93·8% (91·4–96·0) | 80·9% (76·9– 84·7) | 84·3% (80·5–87·7) | 88·3% (85·6–90·9) |
Data are area under the receiver operating characteristic curve (95% CI), unless otherwise indicated.
*
Unlike the binary outcomes used in the other models, CovRNN survival prediction uses time-to-event outcomes and the concordance index (95% CI) is shown. CRWD=Cerner Real-World COVID-19 Q3 Dataset.