Table 2.

Model evaluation results for each of the leave-one-out validation analyses of the TO and TC prediction models, which used text only and both text and coded data, respectively

Model and test set	AUROC (95% CI)	AUPRC (95% CI)	Brier score (95% CI)
TO
AHON-UMCG	0.7187 (0.6910 to 0.7452)	0.0154 (0.0118 to 0.0271)	0.0046 (0.0043 to 0.0050)
ANH-VUmc	0.8235 (0.8051 to 0.8417)	0.0515 (0.0387 to 0.0664)	0.0052 (0.0048 to 0.0056)
JGPN-UMCU	0.8054 (0.7908 to 0.8189)	0.0371 (0.0310 to 0.0452)	0.0049 (0.0046 to 0.0052)
AHA-AMC	0.8133 (0.7951 to 0.8312)	0.0210 (0.0169 to 0.0257)	0.0038 (0.0035 to 0.0042)
Overall	0.7875 (0.7778 to 0.7965)	0.0282 (0.0248 to 0.0324)	0.0046 (0.0044 to 0.0048)
TC
AHON-UMCG	0.7167 (0.6923 to 0.7412)	0.0181 (0.0129 to 0.0271)	0.0044 (0.0041 to 0.0048)
ANH-VUmc	0.8192 (0.7997 to 0.8365)	0.0505 (0.0395 to 0.0658)	0.0051 (0.0047 to 0.0055)
JGPN-UMCU	0.8099 (0.7959 to 0.8243)	0.0421 (0.0345 to 0.0522)	0.0049 (0.0046 to 0.0052)
AHA-AMC	0.8043 (0.7855 to 0.8218)	0.0253 (0.0193 to 0.0332)	0.0038 (0.0035 to 0.0042)
Overall	0.7817 (0.7720 to 0.7906)	0.0219 (0.0195 to 0.0247)	0.0046 (0.0045 to 0.0048)

AHA-AMC = Academic general practitioners network database, Amsterdam UMC location University of Amsterdam. AHON-UMCG = Academic General Practitioners Network Northern Netherlands database, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands. ANH-VUmc = Academic Network of General Practice database, Amsterdam UMC location Vrije Universiteit, Amsterdam, the Netherlands. AUPRC = area under precision-recall curve. AUROC = area under the receiver-operator characteristic. JGPN-UMCU = Julius General Practitioner’s Network database, University Medical Centre Utrecht University, Utrecht, the Netherlands. TC = with text and coded data. TO = text only.