Table 3.

The Gradient Boosting Model's Performance and Correlation to LTCF Related COVID-19 Case and Death Rates by State Compared with the Performance of the Benchmark Logistic Regression and Neural Network Models

Dataset	Metric of Interest	Gradient Boosting Model	Benchmark Logistic Regression Model	Benchmark Neural Network Model
Training set∗ (via 10-fold cross validation)	AUC, mean (95% CI)	0.729 (0.690‒0.767)	0.653 (0.599‒0.706)	0.696 (0.657‒0.734)
	Sensitivity, mean (95% CI)	0.670 (0.477‒0.862)	0.610 (0.483‒0.738)	0.664 (0.484‒0.843)
	Specificity, mean (95% CI)	0.611 (0.412‒0.809)	0.592 (0.450‒0.733)	0.585 (0.410‒0.760)
Prospective validation set†	AUC	0.721	0.689	0.707
	Sensitivity	0.622	0.914	0.904
	Specificity	0.713	0.233	0.308
State LTCF outcome rates‡	Correlation between median risk index and state LTCF case rates by state, Pearson correlation coefficient	0.859	0.384	0.731
	Correlation between median risk index and LTCF deaths rates by state, Pearson correlation coefficient	0.856	0.335	0.705

^∗NHs from Massachusetts, Georgia, and New Jersey with outcomes reported on April 20, 2020.

^†NHs from California with outcomes reported on May 11, 2020.

^‡LTCF-related COVID-19 case and death rates reported on May 11th by states across the United States.