Table 3.

Performance of conventional machine learning, deep learning, and transformer-based classifiers of preoperative cannabis use status documentation in unstructured narrative clinical notes

Class	LR			L-SVM			W2V CNN			BERT-Base			Bio_ClinicalBERT			Support (N)d
	Pa	Rb	Fc	Pa	Rb	F c	Pa	Rb	Fc	Pa	Rb	Fc	Pa	Rb	Fc
Not a true cannabis mention	.99	.98	.98	.80	.95	.87	.99	.98	.98	1.0	.96	.98	.98	.99	.99	44
True mention not reporting use status	.83	.72	.77	.77	.77	.77	.89	.71	.79	.89	.71	.79	.84	.76	.80	12
Positive past use	.37	.35	.36	.56	.41	.47	.71	.54	.61	.71	.56	.63	.70	.56	.62	10
Negative current use	.63	.77	.69	.80	.83	.81	.74	.70	.72	.64	.67	.65	.84	.83	.84	20
Positive current use	.92	.94	.93	.95	.95	.95	.91	.96	.93	.91	.99	.95	.94	.99	.96	108
Weighted average	.88	.90	.89	.90	.91	.91	.91	.91	.91	.90	.92	.91	.93	.95	.94	192

BERT: Bidirectional Encoder Representations from Transformers; CNN: convolutional neural networks; LR: logistic regression; L-SVM: linear support vector machines.

^aPrecision/positive predictive value.

^bRecall/sensitivity.

^cF score = 2×([Precision×Recall]/[Precision+Recall]).

^dNumber of snippets included in model evaluation.