Table 4.

Results by model and outcome class for colorectal cancer validation and test datasets.

Datasets and metrics		Bio+ClinicalBERT									Clinical-BigBird					Clinical-Longformer
		2-class				3-class				2-class			3-class		3-class			3-class
		Head	Tail	Head and tail	Head		Tail	Head and tail
Validation
	Sensitivity, %	80.9	78.5	82.0	86.4		79.9	80.4	82.4			88.4		81.0			91.3 a
	PPVb, %	88.3	85.2	88.7	84.0		80.3	87.1	88.3			87.4		91.8			82.4
	AUCc	0.98	0.98	0.97	0.99		0.98	0.98	0.98			0.99		0.98			0.99
	Scaled Brier score	0.72	0.67	0.73	0.73		0.66	0.72	0.75			0.77		0.76			0.75
	Charts reduced, %	88.6	88.6	88.5	87.2		87.6	88.5	88.4			87.4		89			86.2
Test
	Sensitivity, %	74.1	62.8	71.9	77.0		63.7	69.7	78.5			75.4		73.1			78.9
	PPV, %	79.8	81.1	82.4	72.9		82.4	82.9	81.6			82.0		86.3			81.9
	AUC	0.96	0.95	0.96	0.96		0.95	0.96	0.97			0.96		0.96			0.97
	Scaled Brier score	0.56	0.49	0.54	0.61		0.54	0.58	0.63			0.61		0.65			0.64
	Charts reduced, %	90.1	91.7	90.7	88.7		91.8	91.0	89.7			90.2		91.0			89.7

^aItalicization indicates the highest value.

^bPPV: positive predictive value.

^cAUC: area under the curve.