Table 5:
This table presents the experimental results of ODD on BioClinicalBERT and BioBERT. Each value stands for the average and the standard deviation of test folds of the nested cross-validation results and average scores with higher values between Fine-tuning and Prompt-based Fine-tuning marked as bold.
| Categories | Fine-tuning |
Prompt-based Fine-tuning |
||||||
|---|---|---|---|---|---|---|---|---|
| BioBERT |
BioClinicalBERT |
BioBERT |
BioClinicalBERT |
|||||
| AUPRC | F1 | AUPRC | F1 | AUPRC | F1 | AUPRC | F1 | |
| Confirmed Aberrant Behaviors | 77.79±4.80 | 64.07±6.68 | 79.48±2.53 | 66.19±5.09 | 84.46±16.40 | 71.44±12.02 | 90.52±6.54 | 78.25±7.07 |
| Suggested Aberrant Behaviors | 25.80±4.10 | 23.62±4.18 | 25.83±2.36 | 29.39±5.16 | 46.07±17.87 | 46.93±12.77 | 46.04±16.27 | 44.37±13.02 |
| Opioids | 98.91±0.30 | 97.29±0.59 | 99.04±0.23 | 97.23±0.41 | 99.55±0.16 | 97.52±0.47 | 99.57±0.18 | 98.00±0.29 |
| Indication | 97.57±0.77 | 94.77±0.45 | 97.29±1.02 | 94.25±1.11 | 97.83±0.90 | 93.89±1.60 | 97.86±0.90 | 93.55±0.94 |
| Diagnosed Opioid Dependency | 60.54±6.96 | 45.04±4.86 | 61.54±4.49 | 49.63±4.45 | 88.67±10.83 | 80.90±10.09 | 90.15±7.22 | 79.24±12.97 |
| Benzodiazepines | 96.83±1.01 | 95.09±1.27 | 96.47±1.33 | 94.40±0.94 | 97.39±1.33 | 95.43±0.82 | 96.89±1.71 | 97.15±1.55 |
| Medication Change | 51.64±4.13 | 46.42±2.12 | 56.02±5.52 | 50.72±1.65 | 79.21±3.46 | 68.32±1.67 | 76.33±4.06 | 68.61±5.14 |
| Central Nervous System Related | 97.83±0.57 | 87.10±2.10 | 98.15±0.46 | 88.11±1.00 | 98.60±1.23 | 94.85±1.25 | 98.74±0.53 | 92.81±2.44 |
| Social Determinants of Health | 94.32±1.07 | 80.20±5.73 | 92.82±1.81 | 83.90±6.30 | 96.17±2.01 | 93.65±4.04 | 97.39±1.83 | 93.79±3.08 |
| Macro Average | 77.91±26.36 | 70.40±26.81 | 78.52±25.63 | 72.65±24.58 | 87.55±17.12 | 82.55±17.29 | 88.17±17.40 | 82.86±17.62 |