Table 2.

Predictive performance of the top 5 ML models in the test/holdout datasets. The abbreviations refer to: a baseline dummy classifier that predicts outcomes based on the most frequent class, logistic regression implementing an L2 penalty (hereafter abbreviated as LR), Light Gradient Boosting Machine (LightGBM), Gradient Boosting Classifier (GBC), and the CatBoost Classifier.

	ML model	# Input features	AUC	Balanced accuracy	Sensitivity	Specificity	Precision	NPV	F1
Question 1	Dummy	0	0.5	0.5	0	1	0	0.979	0
	LR	All	0.817	0.743	0.715	0.771	0.063	0.992	0.116
		40	0.813	0.741	0.711	0.771	0.062	0.992	0.114
		30	0.804	0.735	0.702	0.767	0.060	0.992	0.111
		20	0.799	0.711	0.658	0.763	0.056	0.991	0.103
		10	0.798	0.716	0.667	0.765	0.057	0.991	0.105
		5	0.750	0.679	0.566	0.792	0.055	0.988	0.100
	LightGBM	All	0.825	0.740	0.803	0.677	0.052	0.993	0.097
		40	0.817	0.730	0.785	0.675	0.049	0.993	0.092
		30	0.807	0.717	0.768	0.667	0.047	0.993	0.089
		20	0.802	0.717	0.781	0.653	0.048	0.994	0.090
		10	0.789	0.705	0.746	0.665	0.045	0.992	0.085
		5	0.744	0.678	0.750	0.605	0.039	0.985	0.075
	GBC	All	0.824	0.719	0.627	0.810	0.064	0.992	0.116
		40	0.810	0.719	0.627	0.810	0.064	0.992	0.117
		30	0.803	0.731	0.689	0.773	0.063	0.992	0.115
		20	0.804	0.719	0.675	0.763	0.057	0.999	0.105
		10	0.801	0.716	0.662	0.769	0.056	0.991	0.104
		5	0.752	0.681	0.570	0.791	0.057	0.988	0.104
	LDA	All	0.816	0.743	0.702	0.785	0.065	0.992	0.120
		40	0.811	0.742	0.697	0.786	0.065	0.992	0.118
		30	0.803	0.732	0.684	0.780	0.062	0.991	0.114
		20	0.797	0.716	0.654	0.779	0.059	0.991	0.109
		10	0.796	0.704	0.627	0.781	0.057	0.990	0.105
		5	0.748	0.677	0.526	0.828	0.061	0.988	0.110
	CatBoost	All	0.814	0.724	0.654	0.795	0.066	0.983	0.120
		40	0.809	0.728	0.662	0.794	0.065	0.983	0.118
		30	0.798	0.716	0.654	0.778	0.057	0.982	0.105
		20	0.798	0.704	0.640	0.767	0.055	0.984	0.101
		10	0.792	0.701	0.640	0.761	0.053	0.984	0.099
		5	0.739	0.679	0.575	0.784	0.043	0.986	0.079
Question 2	Dummy	0	0.5	0.5	0	1	0	0.987	0
	LR	All	0.808	0.732	0.619	0.846	0.049	0.994	0.090
		40	0.812	0.730	0.614	0.845	0.048	0.994	0.089
		30	0.808	0.716	0.574	0.859	0.049	0.994	0.091
		20	0.802	0.717	0.574	0.861	0.050	0.994	0.092
		10	0.796	0.722	0.614	0.831	0.044	0.994	0.083
		5	0.781	0.709	0.580	0.839	0.044	0.994	0.082
	LightGBM	All	0.810	0.717	0.676	0.757	0.034	0.995	0.064
		40	0.808	0.722	0.671	0.773	0.034	0.994	0.064
		30	0.810	0.730	0.688	0.773	0.036	0.995	0.068
		20	0.805	0.728	0.671	0.786	0.036	0.995	0.069
		10	0.796	0.718	0.705	0.731	0.032	0.995	0.062
		5	0.777	0.706	0.739	0.674	0.027	0.995	0.052
	GBC	All	0.810	0.715	0.585	0.846	0.049	0.994	0.091
		40	0.793	0.706	0.563	0.850	0.051	0.994	0.094
		30	0.808	0.722	0.591	0.854	0.047	0.994	0.087
		20	0.792	0.725	0.585	0.865	0.050	0.993	0.091
		10	0.784	0.724	0.608	0.840	0.043	0.994	0.079
		5	0.775	0.710	0.585	0.834	0.040	0.993	0.074
	LDA	All	0.809	0.713	0.585	0.842	0.050	0.994	0.093
		40	0.813	0.725	0.591	0.858	0.051	0.994	0.093
		30	0.808	0.712	0.557	0.868	0.049	0.994	0.091
		20	0.803	0.714	0.557	0.871	0.052	0.994	0.095
		10	0.797	0.726	0.608	0.844	0.047	0.994	0.088
		5	0.782	0.704	0.557	0.851	0.045	0.993	0.084
	CatBoost	All	0.800	0.720	0.574	0.866	0.053	0.990	0.097
		40	0.799	0.728	0.597	0.860	0.053	0.991	0.097
		30	0.799	0.725	0.580	0.871	0.082	0.990	0.125
		20	0.802	0.713	0.568	0.859	0.064	0.991	0.106
		10	0.795	0.709	0.591	0.827	0.038	0.991	0.070
		5	0.761	0.701	0.557	0.846	0.034	0.992	0.064