Table 4.

Performance Metrics of the Automated Machine Learning Compared to Custom Models on the Validation Phase

ML Platform	Model Type	Dataset Characteristics			Discriminative Model Test Performance
		Dataset #	n	Outcome Ratio	F2 Score	AUROC	Accuracy (%)	PPV	Sensitivity	Specificity
MATLAB Classification Learner App	Linear Superior Vector Machine (SVM)	Dataset 1	483	2:1	0.01	0.53	68.3	1.00	0.01	1.00
		Dataset 1 prime	1449	2:1	0.02	0.60	68.5	0.67	0.01	1.00
		Dataset 2	660	1:1	0.60	0.66	61.4	0.62	0.59	0.63
		Dataset 3	1313	1:1	0.58	0.64	60.9	0.61	0.58	0.64
	Ensemble Random Undersampling Boosting (RUSBoosted)	Dataset 1	483	2:1	0.52	0.56	52.6	0.35	0.59	0.50
		Dataset 1 prime	1449	2:1	0.60	0.67	60.3	0.42	0.67	0.57
		Dataset 2	660	1:1	0.72	0.65	62.4	0.60	0.76	0.48
		Dataset 3	1313	1:1	0.62	0.66	61.2	0.61	0.62	0.61
Python	Support Vector Classification (SVC)	Dataset 1	483	2:1	0.71	0.61	59.6	0.76	0.57	0.43
		Dataset 1 prime	1449	2:1	0.71	0.61	59.6	0.76	0.57	0.43
		Dataset 2	660	1:1	0.61	0.62	61.5	0.64	0.54	0.60
		Dataset 3	1313	1:1	0.66	0.66	66.3	0.66	0.68	0.66

AUROC = area under the receiver operating characteristic curve; ML = machine learning; PPV = positive predictive value.

This table does not include validation results for the Google AutoML models as the software generates an output for performance metrics for the test phase only.