Table 3.

Results for the ML models.

Model	Validation year	Training				Test
		Accuracy	Sensitivity	Specificity	AUC *	Accuracy	Sensitivity	Specificity	AUC *
DT	2017	0.75	0.82	0.50	0.65	0.70	0.82	0.22	0.53
	2018	0.94	1.00	0.73	0.86	0.68	0.81	0.36	0.59
	2019	0.97	1.00	0.91	0.96	0.72	0.75	0.60	0.68
RF	2017	0.81	0.83	0.72	0.73	0.70	0.79	0.33	0.60
	2018	0.94	0.94	0.89	0.87	0.70	0.87	0.32	0.63
	2019	0.89	0.90	0.87	0.85	0.82	0.85	0.60	0.77
LR	2017	0.63	0.59	0.78	0.63	0.63	0.59	0.78	0.61
	2018	0.71	0.71	0.68	0.63	0.65	0.73	0.45	0.62
	2019	0.62	0.58	0.74	0.63	0.65	0.60	0.90	0.84
SVM	2017	0.99	0.98	1.00	0.97	0.65	0.74	0.33	0.45
	2018	0.94	0.92	1.00	0.86	0.61	0.75	0.27	0.56
	2019	0.89	0.86	0.97	0.85	0.68	0.69	0.60	0.68
MLP	2017	0.82	0.95	0.38	0.77	0.74	0.88	0.22	0.65
	2018	0.87	1.00	0.26	0.93	0.74	1.00	0.14	0.65
	2019	0.79	0.99	0.23	0.83	0.85	0.93	0.40	0.82

^*AUC, Area Under Receiver Operative Curve.