Table 2.

Model validation using overlapping attributes and annual observations.

Threshold	Model	Sensitivity		Specificity		Overall
		CLIMB	EPIC	CLIMB	EPIC	CLIMB	EPIC
0.5	SVM	0.63	0.81	0.75	0.70	0.72	0.74
	Logistic Regression	0.64	0.76	0.78	0.72	0.75	0.73
	Random Forest	0.62	0.83	0.77	0.65	0.74	0.71
	XGBoost	0.58	0.75	0.75	0.71	0.71	0.72
	LightGBM	0.56	0.62	0.75	0.83	0.71	0.76
	Meta-La	0.61	0.78	0.79	0.76	0.75	0.77
0.45	SVM	0.76	0.90	0.61	0.45	0.64	0.60
	Logistic Regression	0.69	0.83	0.69	0.65	0.69	0.71
	Random Forest	0.73	0.90	0.63	0.53	0.65	0.65
	XGBoost	0.68	0.79	0.70	0.66	0.70	0.70
	LightGBM	0.69	0.69	0.68	0.77	0.68	0.74
	Meta-La	0.70	0.85	0.68	0.70	0.68	0.75
0.4	SVM	0.84	0.93	0.47	0.42	0.55	0.59
	Logistic Regression	0.78	0.88	0.60	0.59	0.64	0.68
	Random Forest	0.85	0.92	0.54	0.39	0.61	0.56
	XGBoost	0.75	0.85	0.62	0.60	0.65	0.68
	LightGBM	0.75	0.73	0.61	0.73	0.64	0.73
	Meta-La	0.81	0.90	0.58	0.58	0.63	0.68
0.35	SVM	0.92	0.96	0.37	0.32	0.50	0.53
	Logistic Regression	0.86	0.92	0.51	0.51	0.59	0.64
	Random Forest	0.89	0.96	0.45	0.31	0.55	0.52
	XGBoost	0.85	0.87	0.54	0.60	0.61	0.69
	LightGBM	0.85	0.80	0.52	0.70	0.60	0.73
	Meta-La	0.88	0.93	0.49	0.52	0.58	0.65
0.3	SVM	0.93	0.98	0.25	0.23	0.40	0.47
	Logistic Regression	0.90	0.93	0.41	0.48	0.52	0.63
	Random Forest	0.95	0.95	0.30	0.24	0.45	0.47
	XGBoost	0.90	0.90	0.45	0.56	0.55	0.67
	LightGBM	0.92	0.86	0.42	0.62	0.53	0.70
	Meta-La	0.93	0.96	0.38	0.37	0.51	0.56
Regression coef. (p value) R-square (correlation)		1.08 (6.9E−08) 0.65 (0.81)		0.77 (8.6E−09) 0.70 (0.84)		0.88 (1.8E−08) 0.68 (0.83)

Bold numbers indicate models of high practical value.

^aEnsemble of SVM, Logistic Regression, and Random Forest.