Table 2.

Model performances evaluated on the holdout test set.^a

Name	Model	Area under the receiver operating characteristic curve (95% CI)	Threshold	Sensitivity (95% CI)	Specificity (95% CI)
A	Baseline model	0.655 (0.632-0.676)	N/Ab	0.551 (0.508-0.596)	0.759 (0.752-0.767)
B	Logistic regression	0.697 (0.689-0.711)	Youden’s J statistic	0.708 (0.625-0.768)	0.644 (0.574-0.706)
C	Random forest	0.727 (0.720-0.735)	Youden’s J statistic	0.755 (0.676-0.813)	0.629 (0.564-0.700)
D	Random forest	0.727 (0.720-0.735)	Specificity=0.25	0.921 (0.907-0.935)	0.250 (0.246-0.254)
E	Random forest	0.727 (0.720-0.735)	Specificity=0.75	0.576 (0.553-0.594)	0.750 (0.749-0.751)

^aThe area under the receiver operating characteristic curve column denotes the area under the receiving operator curve (Figure 4) for each model. The 3 rightmost columns display the sensitivity and specificity of models at predicting exacerbations with different thresholds used to dichotomize the predictions. The baseline model is already binary and only has 1 nontrivial configuration, but the threshold used to dichotomize the machine learning models (B-E) can be tuned to suit the intended context of the model. The maximum of Youden’s J statistic is used as a baseline criterion for dichotomizing the prediction (models B and C), and other cutoffs yielding fixed specificities are investigated for the random forest model. The area under the receiver operating characteristic curve for models C, D, and E are the same since they correspond to the same underlying model.

^bN/A: not applicable.