Table 5.

Macroaveraged performance metrics and accuracy for task 2 (about suicide vs off-topic) on the validation and test sets.

Model	Validation set (n=513)					Test set (n=641)
	Precision	Recall	F 1	Accuracy	Precision		Recall	F 1	Accuracy
Majority classifier	0.37	0.50	0.43	0.75	0.37		0.50	0.43	0.75
TF-IDFa and SVMb	0.74	0.77	0.75	0.80	0.75		0.77	0.76	0.81
BERTc	0.85	0.81	0.83	0.88	0.85		0.81	0.83	0.88
XLNet	0.84	0.78	0.81	0.87	0.83		0.80	0.81	0.87

^aTF-IDF: term frequency-inverse document frequency.

^bSVM: support vector machine.

^cBERT: Bidirectional Encoder Representations from Transformers.