Table 2.

Performance metrics.

	Non-COVID-19 viral pneumonia full model	Non-COVID-19 viral pneumonia reduced model	APACHE2	SOFA	SAPS2	4C Mortality Score	Zhou COVID-19 model	Wang laboratory COVID-19 model	Wang clinical COVID-19 model
auROC	0.86 (0.86–0.87)	0.85 (0.84–0.86)	0.63 (0.61–0.65)	0.76 (0.75–0.77)	0.72 (0.71–0.74)	0.71 (0.70–0.72)	0.76 (0.73–0.78)	0.62 (0.59–0.65)	0.56 (0.55–0.58)
auPRC	0.69 (0.67–0.71)	0.68 (0.65–0.70)	0.41 (0.39–0.44)	0.53 (0.51–0.56)	0.46 (0.44–0.48)	0.46 (0.43–0.48)	0.46 (0.42–0.50)	0.39 (0.35–0.43)	0.32 (0.30–0.34)
F1 score	0.67 (0.66–0.68)	0.66 (0.64–0.67)	0.51 (0.49–0.52)	0.56 (0.54–0.58)	0.53 (0.51–0.54)	0.50 (0.48–0.51)	0.58 (0.55–0.61)	0.43 (0.41–0.47)	0.44 (0.43–0.46)
PPV/Precision	0.61 (0.59–0.63)	0.57 (0.55–0.62)	0.38 (0.35–0.39)	0.45 (0.43–0.50)	0.44 (0.39–0.47)	0.41 (0.40–0.43)	0.42 (0.40–0.46)	0.33 (0.28–0.43)	0.29 (0.28–0.32)
NPV	0.89 (0.88–0.90)	0.90 (0.88–0.91)	0.80 (0.79–0.84)	0.87 (0.83–0.88)	0.83 (0.82–0.86)	0.82 (0.81–0.83)	0.95 (0.90–0.96)	0.81 (0.79–0.84)	0.83 (0.80–0.85)
Sensitivity	0.74 (0.72–0.77)	0.77 (0.70–0.79)	0.76 (0.74–0.88)	0.75 (0.63–0.80)	0.65 (0.60–0.77)	0.62 (0.60–0.64)	0.93 (0.83–0.95)	0.62 (0.46–0.81)	0.92 (0.80–0.93)
Specificity	0.82 (0.80–0.84)	0.78 (0.76–0.84)	0.44 (0.28–0.47)	0.64 (0.59–0.74)	0.67 (0.53–0.73)	0.66 (0.65–0.67)	0.50 (0.49–0.61)	0.57 (0.31–0.78)	0.15 (0.14–0.32)
Threshold@max F1	0.15 (0.13–0.16)	0.16 (0.15–0.21)	20.00 (16.00–21.00)	7.00 (6.00–9.00)	43.00 (39.00–45.00)	13.00 (13.00–13.00)	21.75 (21.51–25.43)	− 15.82 (− 19.50–− 13.12)	5.53 (5.53–6.57)
n time bins	18,521	18,521	13,361	17,255	17,245	18,521	4774	4480	18,521
n patients	1054	1054	607	921	925	1054	278	253	1054
Brier score	0.15 (0.15–0.16)	0.15 (0.15–0.16)

The table shows the area under the ROC (auROC) and the area under the precision-recall curve (auPRC) as threshold-independent performance metrics and the F1 score, positive predictive value (PPV)/precision, negative predictive value (NPV), sensitivity/recall and specificity at a classifier threshold that maximizes the F1 score (Threshold@max F1) for each of the models/scores applied to the COVID-19 viral pneumonia patients test dataset for the prediction of mortality within the next 5 days across all 24 h time bins of each patient’s stay on the ICU, weighted inversely by the number of time bins per patient. Additionally shown are the number of included time bins (note that there are usually multiple time bins per patient) and the number of included unique patients for each of the models and the Brier score for the two models that output a probability score for the prediction.