Table 4.

The performance of prediction models in internal and external validation

Algorithm	Dataset	Model	Accuracy	Classification report			AUC
				Precision	Recall	f1-score
Random forest classifier (RFC)	10-Fold cross-validation	RFC	0.72 ± 0.10	–	–	–	0.77 ± 0.08
	Internal validation	RF-Model-O	0.83	0.84	0.82	0.83	0.87
	Bootstrap-internal validation	RF-Model-O	0.81 ± 0.05	0.83 ± 0.05	0.80 ± 0.07	0.82 ± 0.05	0.80 ± 0.06
	External validation	RF-Model-O	0.68	0.66	0.68	0.67	0.69
	Bootstrap-external validation	RF-Model-O	0.72 ± 0.04	0.77 ± 0.06	0.85 ± 0.03	0.81 ± 0.03	0.69 ± 0.05
Support vector classifier (SVC)	10-Fold cross-validation	SVC	0.69 ± 0.12	–	–	–	0.76 ± 0.09
	Internal validation	SV-Model-O	0.78	0.80	0.78	0.78	0.82
	Bootstrap-internal validation	SV-Model-O	0.78 ± 0.08	0.94 ± 0.07	0.68 ± 0.12	0.81 ± 0.03	0.81 ± 0.09
	External validation	SV-Model-O	0.57	0.62	0.57	0.58	0.61
	Bootstrap-external validation	SV-Model-O	0.57 ± 0.06	0.73 ± 0.07	0.57 ± 0.07	0.64 ± 0.06	0.61 ± 0.07
Gradient boosting classifier (GBC)	10-Fold cross-validation	GBC	0.73 ± 0.07	–	–	–	0.79 ± 0.08
	Internal validation	GB-Model-O	0.80	0.81	0.80	0.80	0.81
	Bootstrap-internal validation	GB-Model-O	0.72 ± 0.05	0.84 ± 0.06	0.67 ± 0.07	0.74 ± 0.06	0.81 ± 0.05
	External validation	GB-Model-O	0.68	0.66	0.68	0.67	0.72
	Bootstrap-external validation	GB-Model-O	0.70 ± 0.03	0.73 ± 0.04	0.86 ± 0.03	0.79 ± 0.03	0.67 ± 0.04
7-layer perceptron model	Internal validation	Simple-DL	0.76	0.7	0.73	0.71	0.72
		Dropout-DL	0.76	0.7	0.73	0.71	0.72
	External validation	Simple-DL	0.55
		Dropout-DL	0.55