Table 4.
Prediction regions on the internal validation and external validation populations.
| Confidence level | Prediction regions | MIMIC-IVa internal validation | eICU-CRDb external validation | |||||
|
|
|
Survivors (n=3496), n (%) | Nonsurvivors (n=691), n (%) | Overall (n=4187), n (%) | Survivors (n=8605), n (%) | Nonsurvivors (n=1757), n (%) | Overall (n=10,362), n (%) | |
| 99.9% | ||||||||
|
|
Empty | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | |
|
|
Single error | 43 (1.2) | 1 (0.1) | 44 (1.1) | 81 (0.9) | 13 (0.7) | 94 (0.9) | |
|
|
Single correct | 868 (24.8) | 145 (21) | 1013 (24.2) | 999 (11.6) | 263 (15) | 1262 (12.2) | |
|
|
Multiple | 2585 (73.9) | 545 (78.9) | 3130 (74.8) | 7525 (87.4) | 1481 (84.3) | 9006 (86.9) | |
| 95% | ||||||||
|
|
Empty | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | |
|
|
Single error | 182 (5.2) | 28 (4.1) | 210 (5) | 461 (5.4) | 80 (4.6) | 541 (5.2) | |
|
|
Single correct | 1598 (45.7) | 306 (44.3) | 1904 (45.5) | 2704 (31.4) | 616 (35.1) | 3320 (32) | |
|
|
Multiple | 1716 (49.1) | 357 (51.7) | 2073 (49.5) | 5440 (63.2) | 1061 (60.4) | 6501 (62.7) | |
| 90% | ||||||||
|
|
Empty | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | |
|
|
Single error | 361 (10.3) | 77 (11.1) | 438 (10.5) | 997 (11.6) | 224 (12.7) | 1221 (11.8) | |
|
|
Single correct | 2123 (60.7) | 397 (57.5) | 2520 (60.2) | 4285 (49.8) | 852 (48.5) | 5137 (49.6) | |
|
|
Multiple | 1012 (28.9) | 217 (31.4) | 1229 (29.4) | 3323 (38.6) | 681 (38.8) | 4004 (38.6) | |
| 85% | ||||||||
|
|
Empty | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | |
| Single error | 502 (14.4) | 109 (15.8) | 611 (14.6) | 1454 (16.9) | 322 (18.3) | 1776 (17.1) | ||
| Single correct | 2390 (68.4) | 470 (68) | 2860 (68.3) | 5171 (60.1) | 1018 (57.9) | 6189 (59.7) | ||
| Multiple | 604 (17.3) | 112 (16.2) | 716 (17.1) | 1980 (23) | 417 (23.7) | 2397 (23.1) | ||
| AIc point predictiond | ||||||||
|
|
Error | 1372 (39.2) | 77 (11.1) | 1449 (34.6) | 4316 (50.2) | 224 (12.7) | 4540 (43.8) | |
|
|
Correct | 2124 (60.8) | 614 (88.9) | 2738 (65.4) | 4289 (49.8) | 1533 (87.3) | 5822 (56.2) | |
aMIMIC-IV: Medical Information Mart for Intensive Care database-IV.
beICU-CRD: eICU Collaborative Research Database.
cAI: artificial intelligence.
dFor comparison with the AI model with conformal prediction at a user-specified 90% confidence level, the AI point prediction without conformal prediction was made when setting the error rate in nonsurvivors as 11.1% for MIMIC-IV validation and 12.7% for eICU-CRD validation.