Table B.5.
Results of (P)recision, (R)ecall, and F-1 on the MIMIC-III assertion dataset. The best scores are bolded.
Note type | Model | Present | Absent | Possible | ||||||
---|---|---|---|---|---|---|---|---|---|---|
P | R | F-1 | P | R | F-1 | P | R | F-1 | ||
Discharge summaries | Logistic Regression | 0.923 | 0.890 | 0.906 | 0.792 | 0.917 | 0.850 | 0.521 | 0.396 | 0.450 |
NegEx [5] | 0.876 | 0.961 | 0.917 | 0.881 | 0.878 | 0.879 | – | – | – | |
RadText [35] | 0.817 | 0.947 | 0.877 | 0.836 | 0.584 | 0.688 | 0.608 | 0.316 | 0.416 | |
BERT model [4] | 0.941 | 0.961 | 0.951 | 0.920 | 0.948 | 0.934 | 0.727 | 0.480 | 0.578 | |
Prompt-based | 0.949 | 0.948 | 0.949 | 0.916 | 0.951 | 0.933 | 0.678 | 0.580 | 0.625 | |
Nursing letters | Logistic Regression | 0.916 | 0.860 | 0.887 | 0.639 | 0.780 | 0.702 | 0.105 | 0.143 | 0.121 |
NegEx [5] | 0.931 | 0.966 | 0.948 | 0.839 | 0.881 | 0.860 | – | – | – | |
RadText [35] | 0.875 | 0.956 | 0.914 | 0.719 | 0.390 | 0.506 | 0.429 | 0.429 | 0.429 | |
BERT model [4] | 0.980 | 0.983 | 0.981 | 0.966 | 0.949 | 0.957 | 0.786 | 0.786 | 0.786 | |
Prompt-based | 0.983 | 0.980 | 0.981 | 0.950 | 0.966 | 0.958 | 0.714 | 0.714 | 0.714 | |
Physician letters | Logistic Regression | 0.912 | 0.814 | 0.860 | 0.628 | 0.952 | 0.756 | 0.682 | 0.469 | 0.556 |
NegEx [5] | 0.809 | 0.912 | 0.857 | 0.703 | 0.788 | 0.743 | – | – | – | |
RadText [35] | 0.781 | 0.980 | 0.870 | 0.897 | 0.530 | 0.667 | 0.889 | 0.235 | 0.372 | |
BERT model [4] | 0.908 | 0.971 | 0.938 | 0.934 | 0.864 | 0.898 | 0.880 | 0.647 | 0.746 | |
Prompt-based | 0.895 | 0.956 | 0.924 | 0.887 | 0.833 | 0.859 | 0.875 | 0.618 | 0.724 | |
Radiology reports | Logistic Regression | 0.887 | 0.856 | 0.871 | 0.876 | 0.971 | 0.921 | 0.516 | 0.478 | 0.496 |
NegEx [5] | 0.767 | 0.902 | 0.829 | 0.768 | 0.862 | 0.812 | – | – | – | |
RadText [35] | 0.821 | 0.951 | 0.881 | 0.923 | 0.783 | 0.847 | 0.558 | 0.358 | 0.436 | |
BERT model [4] | 0.886 | 0.979 | 0.930 | 0.978 | 0.957 | 0.967 | 0.900 | 0.537 | 0.673 | |
Prompt-based | 0.923 | 0.968 | 0.945 | 0.970 | 0.942 | 0.956 | 0.825 | 0.702 | 0.758 |