Table 3.

Performance of models for pancreatic cancer prediction

Models		AUROC		AUPRC
Without Pretraining	Logistic regression	73.64	±2.26	68.95	±2.14
	LSTM	76.98	±0.54	73.48	±0.55
	BERT without pretraining	77.27	±0.45	74.00	±0.31
With Pretraining	BERT	79.22	±0.47	76.89	±0.48
	TransformEHR (ours)	81.95	±0.90	78.64	±0.85

Result is calculated from best hyperparameters with 5 randomized seeds each. ± represents standard deviation.