Table 6.

Intraclass performance metrics for deep learning models in task 2 (about suicide vs off-topic) on the test set.

Test set and model	About suicide (n=478)				Off-topic (n=163)
	Precision (95% CI)	Recall (95% CI)	F 1	Precision (95% CI)		Recall (95% CI)	F 1
TF-IDFa and SVMb	0.89 (85.74-91.71)	0.85 (80.96-87.64)	0.87	0.60 (53.03-67.49)		0.69 (61.63-76.30)	0.65
BERTc,d	0.90 (87.42-92.81)	0.94 (91.64-96.07)	0.92	0.80 (71.62-85.67)		0.68 (60.35-75.17)	0.73
XLNetd	0.90 (87.12-92.59)	0.93 (90.68-95.38)	0.92	0.76 (68.60-83.06)		0.67 (59.72-74.60)	0.71

^aTF-IDF: term frequency-inverse document frequency.

^bSVM: support vector machine.

^cBERT: Bidirectional Encoder Representations from Transformers.

^dScores are averages across 5 model runs for BERT and XLNet. Table S5 in Multimedia Appendix 1 shows separate runs.