Table 3. Comparative performance of fine-tuned Scientific Bidirectional Encoder Representations from Transformers (SciBERT) and Longformer models and baseline GPT-4o model on conclusiveness classification task.
| Metric | Fine-tuned SciBERT | Fine-tuned Longformer | Baseline GPT-4o |
|---|---|---|---|
| Precision | 0.34 | 0.57 | 0.74 |
| Recall | 0.34 | 0.44 | 0.74 |
| F1-score | 0.27 | 0.40 | 0.74 |
| Accuracy (%) | 34.3 | 44.1 | 74.2 |
| Cohen κa | 0.03 | 0.19 | 0.57 |
a
Predictions were compared against expert annotations, where each plain language summary was manually classified by 2 experts, with a third expert resolving any discrepancies.