Table 2.

Results by model and outcome class for breast cancer validation and test datasets.

Datasets and metrics		Bio+ClinicalBERT									Clinical-BigBird					Clinical-Longformer
		2-class				3-class				2-class			3-class		2-class			3-class
		Head	Tail	Head and tail	Head		Tail	Head and tail
Validation
	Sensitivity, %	71.9	71.2	80.7	79.0		84.2	74.0	84.0			86.0		76.7			86.7 a
	PPVb, %	85.1	90.5	86.8	88.1		84.7	87.5	88.7			84.2		93.0			84.1
	AUCc	0.96	0.97	0.97	0.97		0.98	0.97	0.98			0.98		0.97			0.98
	Scaled Brier score	0.60	0.62	0.68	0.68		0.70	0.64	0.73			0.71		0.72			0.72
	Charts reduced, %	86.2	87.2	84.8	85.4		83.8	86.2	84.5			83.3		86.5			83.2
Test
	Sensitivity, %	84.5	76.6	87.1	82.3		85.9	82.5	86.6			93.6		94.3			93.6
	PPV, %	85.1	87.1	81.8	85.5		80.6	83.6	92.3			77.9		78.7			81.6
	AUC	0.98	0.97	0.98	0.98		0.98	0.98	0.99			0.99		0.99			0.99
	Scaled Brier score	0.71	0.66	0.69	0.70		0.68	0.68	0.79			0.72		0.70			0.76
	Charts reduced, %	87.1	88.6	86.2	87.5		86.2	87.2	87.8			84.4		84.4			85.1

^aItalicization indicates the highest value.

^bPPV: positive predictive value.

^cAUC: area under the curve.