Table 3.

Performance metrics of the different models, A1, A2, and B, when trained with the different combinations of three out of the four centers and tested on the held-out center. AUC, area under the receiver operating characteristic curve; Acc, accuracy; Sens, sensitivity; Spec, specificity; MCC, Matthews correlation coefficient.

		Model A1		Model A2		Model B
		CV [95% CI]	Test [95% CI]	CV [95% CI]	Test [95% CI]	CV [95% CI]	Test [95% CI]
Train with centers 2, 3, and 4; test with center 1	AUC	0.95 [0.94; 0.97]	0.75 [0.25; 1.00]	0.93 [0.91; 0.95]	0.67 [0.25; 1.00]	0.81 [0.78; 0.84]	0.83 [0.50; 1.00]
	Acc	0.90 [0.88; 0.92]	0.71 [0.35; 0.91]	0.89 [0.86; 0.91]	0.71 [0.35; 0.91]	0.75 [0.72; 0.78]	0.71 [0.35; 0.91]
	Sens	0.84 [0.78; 0.90]	0.67 [0.20; 0.93]	0.86 [0.80; 0.92]	0.67 [0.20; 0.93]	0.55 [0.45; 0.65]	0.67 [0.20; 0.93]
	Spec	0.93 [0.91; 0.95]	0.75 [0.30; 0.95]	0.90 [0.87; 0.93]	0.75 [0.30; 0.95]	0.85 [0.82; 0.88]	0.75 [0.30; 0.95]
	MCC	0.77 [0.72; 0.82]	0.42 [0.19; 0.68]	0.75 [0.70; 0.81]	0.42 [0.19; 0.68]	0.42 [0.33; 0.51]	0.42 [0.19; 0.68]
Train with centers 1, 3, and 4; test with center 2	AUC	0.92 [0.90; 0.94]	0.64 [0.36; 0.91]	0.91 [0.89; 0.94]	0.82 [0.55; 1.00]	0.84 [0.81; 0.86]	0.55 [0.27; 0.82]
	Acc	0.85 [0.82; 0.87]	0.75 [0.46; 0.91]	0.85 [0.82; 0.87]	0.67 [0.39; 0.86]	0.77 [0.75; 0.80]	0.83 [0.55; 0.95]
	Sens	0.79 [0.75; 0.83]	0.00 [0.00; 0.73]	0.76 [0.69; 0.83]	0.00 [0.00; 0.73]	0.59 [0.53; 0.65]	0.00 [0.00; 0.73]
	Spec	0.87 [0.84; 0.91]	0.82 [0.52; 0.94]	0.89 [0.87; 0.91]	0.73 [0.43; 0.90]	0.87 [0.84; 0.90]	0.91 [0.62; 0.98]
	MCC	0.67 [0.61; 0.72]	-0.13 [-0.38; -0.04]	0.66 [0.60; 0.72]	-0.17 [-0.41; -0.06]	0.48 [0.42; 0.54]	-0.09 [-0.38; -0.02]
Train with centers 1, 2, and 4; test with center 3	AUC	0.91 [0.89; 0.93]	0.80 [0.62; 0.95]	0.91 [0.89; 0.93]	0.85 [0.38; 0.86]	0.83 [0.80; 0.86]	0.89 [0.56; 0.96]
	Acc	0.84 [0.80; 0.88]	0.75 [0.58; 0.86]	0.85 [0.83; 0.88]	0.78 [0.61; 0.88]	0.76 [0.72; 0.79]	0.81 [0.64; 0.90]
	Sens	0.75 [0.66; 0.83]	0.86 [0.48; 0.97]	0.82 [0.74; 0.89]	0.57 [0.25; 0.84]	0.54 [0.44; 0.64]	0.86 [0.48; 0.97]
	Spec	0.88 [0.84; 0.93]	0.72 [0.54; 0.85]	0.87 [0.83; 0.91]	0.83 [0.65; 0.92]	0.87 [0.82; 0.91]	0.79 [0.61; 0.90]
	MCC	0.64 [0.55; 0.73]	0.47 [0.41; 0.53]	0.69 [0.63; 0.74]	0.36 [0.30; 0.42]	0.44 [0.34; 0.54]	0.55 [0.48; 0.60]
Train with centers 1, 2, and 3; test with center 4	AUC	0.87 [0.84; 0.89]	0.77 [0.51; 1.00]	0.84 [0.81; 0.87]	0.72 [0.44; 0.93]	0.82 [0.78; 0.85]	0.75 [0.47; 0.97]
	Acc	0.82 [0.79; 0.85]	0.59 [0.38; 0.76]	0.83 [0.80; 0.86]	0.36 [0.19; 0.57]	0.77 [0.74; 0.80]	0.27 [0.13; 0.48]
	Sens	0.59 [0.50; 0.67]	0.53 [0.30; 0.75]	0.62 [0.53; 0.71]	0.13 [0.03; 0.37]	0.56 [0.46; 0.65]	0.00 [0.00; 0.15]
	Spec	0.93 [0.90; 0.97]	0.71 [0.35; 0.91]	0.94 [0.91; 0.96]	0.86 [0.48; 0.97]	0.88 [0.84; 0.92]	0.86 [0.48; 0.97]
	MCC	0.59 [0.52; 0.66]	0.23 [0.16; 0.31]	0.61 [0.55; 0.68]	-0,01 [-0.07; 0.00]	0.47 [0.39; 0.55]	-0.32 [-0.46; -0.20]