Table 4.

Diagnostic performance comparison of different machine learning models

Method	Model	Dataset	Accuracy	Precision	Recall	F1 Score	AUC
Grid	RandomForest	train	1.000	1.000	1.000	1.000	1.000
Grid	RandomForest	validation	0.856	0.858	0.853	0.856	0.929
Grid	RandomForest	test	0.855	0.858	0.851	0.855	0.925
Grid	XGBoost	train	0.993	0.999	0.988	0.993	1.000
Grid	XGBoost	validation	0.875	0.908	0.835	0.870	0.930
Grid	XGBoost	test	0.862	0.894	0.822	0.857	0.924
Grid	CART	train	0.852	0.856	0.846	0.851	0.939
Grid	CART	validation	0.776	0.779	0.771	0.775	0.839
Grid	CART	test	0.754	0.758	0.745	0.752	0.832
Grid	MLP	train	0.922	0.888	0.966	0.926	0.984
Grid	MLP	validation	0.764	0.719	0.868	0.786	0.837
Grid	MLP	test	0.778	0.739	0.857	0.794	0.842
Original	RandomForest	train	0.837	0.811	0.879	0.844	0.942
Original	RandomForest	validation	0.753	0.735	0.793	0.763	0.857
Original	RandomForest	test	0.763	0.748	0.795	0.771	0.856
Original	XGBoost	train	0.909	0.956	0.857	0.904	0.968
Original	XGBoost	validation	0.858	0.907	0.799	0.849	0.917
Original	XGBoost	test	0.849	0.900	0.785	0.838	0.915
Original	CART	train	0.781	0.734	0.882	0.801	0.869
Original	CART	validation	0.704	0.669	0.807	0.731	0.762
Original	CART	test	0.715	0.685	0.799	0.737	0.762
Original	MLP	train	0.849	0.841	0.862	0.851	0.931
Original	MLP	validation	0.741	0.729	0.766	0.747	0.804
Original	MLP	test	0.741	0.725	0.776	0.749	0.799
Stacking	KNN	train	0.990	0.991	0.989	0.990	0.999
Stacking	KNN	validation	0.869	0.892	0.839	0.865	0.920
Stacking	KNN	test	0.863	0.886	0.834	0.859	0.914
Stacking	LogisticRegression	train	0.998	1.000	0.997	0.998	1.000
Stacking	LogisticRegression	validation	0.877	0.889	0.861	0.875	0.943
Stacking	LogisticRegression	test	0.877	0.893	0.856	0.875	0.938
Stacking	DecisionTree	train	0.992	0.992	0.991	0.992	0.999
Stacking	DecisionTree	validation	0.877	0.890	0.860	0.875	0.936
Stacking	DecisionTree	test	0.878	0.892	0.859	0.875	0.934
DL	DNN	train	0.745	0.728	0.783	0.754	0.830
DL	DNN	validation	0.702	0.683	0.755	0.717	0.771
DL	DNN	test	0.698	0.679	0.750	0.713	0.770
DL	CNN	train	0.739	0.726	0.769	0.747	0.817
DL	CNN	validation	0.689	0.675	0.730	0.701	0.761
DL	CNN	test	0.700	0.683	0.748	0.714	0.764
DL	Transformer	train	0.616	0.608	0.655	0.631	0.663
DL	Transformer	validation	0.602	0.598	0.620	0.609	0.646
DL	Transformer	test	0.618	0.611	0.650	0.630	0.659