TABLE 5.

The performance of a series of models predicting preclinical disease status

		Multivariable logistic regression		SVM			Random forest	GBM	Logistic regression
		Backward stepwise elimination	Ridge regression	Linear	Polynomial	RBF		XGBoost	NT‐proBNP
Train (n = 848)	AUC (95% CI)	0.83 (0.80‐0.86)	0.85 (0.82‐0.88)	0.84 (0.81‐0.87)	0.84 (0.81‐0.87)	0.87 (0.84‐0.90)	0.97 (0.96‐0.98)	0.97 (0.96‐0.98)	0.78 (0.75‐0.82)
	Accuracy	0.81	0.82	0.81	0.81	0.83	0.89	0.91	0.78
	Bootstrap AUC (SD)	0.83 (0.017)	0.85 (0.016)	0.84 (0.017)	0.84 (0.017)	0.87 (0.015)	0.97 (0.005)	0.97 (0.006)	0.79 (0.019)
Test (n = 212)	AUC (95% CI)	0.86 (0.81‐0.91)	0.88 (0.83‐0.93)	0.88 (0.84‐0.93)	0.88 (0.83‐0.93)	0.87 (0.82‐0.92)	0.85 (0.80‐0.91)	0.86 (0.82‐0.91)	0.77 (0.70‐0.84)
	Accuracy	0.79	0.82	0.81	0.81	0.83	0.81	0.80	0.77
	Brier score	0.133	0.125	0.125	0.126	0.127	0.136	0.133	0.158
	Calibration in the large	−0.034	0.135	0.268	0.244	0.097	0.098	0.088	−0.056
	Calibration slope	1.129	1.441	1.492	1.454	1.383	1.275	1.217	0.950
n (variables required)		5	29	29	29	29	29	29	1
Interpretable		Yes	Yes	No	No	No	No	No	Yes

Notes: The 5 variables selected by backward stepwise elimination in the predictive logistic regression model were: appetite, body condition score, creatinine, murmur intensity, and NT‐proBNP. Model calibration was assessed using a locally weighted smoothing line (LOWESS) fitted to predicted probabilities plotted against the actual probability (prevalence) for each value. Support vector machine models were developed using linear, polynomial and radial basis function kernels. AUC, area under the receiver operating characteristic curve; CI, confidence intervals; GBM, gradient boosting machine; NT‐proBNP, N‐terminal propeptide of B‐type natriuretic peptide; RBF, radial basis function; SVM, support vector machine; XGBoost, extreme gradient boosting trees.