Skip to main content
. Author manuscript; available in PMC: 2024 Sep 2.
Published in final edited form as: Proc Conf. 2024 Jun;2024:4338–4359.

Table 5:

This table presents the experimental results of ODD on BioClinicalBERT and BioBERT. Each value stands for the average and the standard deviation of test folds of the nested cross-validation results and average scores with higher values between Fine-tuning and Prompt-based Fine-tuning marked as bold.

Categories Fine-tuning
Prompt-based Fine-tuning
BioBERT
BioClinicalBERT
BioBERT
BioClinicalBERT
AUPRC F1 AUPRC F1 AUPRC F1 AUPRC F1
Confirmed Aberrant Behaviors 77.79±4.80 64.07±6.68 79.48±2.53 66.19±5.09 84.46±16.40 71.44±12.02 90.52±6.54 78.25±7.07
Suggested Aberrant Behaviors 25.80±4.10 23.62±4.18 25.83±2.36 29.39±5.16 46.07±17.87 46.93±12.77 46.04±16.27 44.37±13.02
Opioids 98.91±0.30 97.29±0.59 99.04±0.23 97.23±0.41 99.55±0.16 97.52±0.47 99.57±0.18 98.00±0.29
Indication 97.57±0.77 94.77±0.45 97.29±1.02 94.25±1.11 97.83±0.90 93.89±1.60 97.86±0.90 93.55±0.94
Diagnosed Opioid Dependency 60.54±6.96 45.04±4.86 61.54±4.49 49.63±4.45 88.67±10.83 80.90±10.09 90.15±7.22 79.24±12.97
Benzodiazepines 96.83±1.01 95.09±1.27 96.47±1.33 94.40±0.94 97.39±1.33 95.43±0.82 96.89±1.71 97.15±1.55
Medication Change 51.64±4.13 46.42±2.12 56.02±5.52 50.72±1.65 79.21±3.46 68.32±1.67 76.33±4.06 68.61±5.14
Central Nervous System Related 97.83±0.57 87.10±2.10 98.15±0.46 88.11±1.00 98.60±1.23 94.85±1.25 98.74±0.53 92.81±2.44
Social Determinants of Health 94.32±1.07 80.20±5.73 92.82±1.81 83.90±6.30 96.17±2.01 93.65±4.04 97.39±1.83 93.79±3.08
Macro Average 77.91±26.36 70.40±26.81 78.52±25.63 72.65±24.58 87.55±17.12 82.55±17.29 88.17±17.40 82.86±17.62