Table B.3.
Results of (P)recision, (R)ecall, and F-1 on the external evaluation datasets.
| Dataset | Model | Present | Absent | Possible | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| P | R | F-1 | P | R | F-1 | P | R | F-1 | ||
| i2b2 2010 | Logistic Regression | 0.934 | 0.918 | 0.926 | 0.835 | 0.900 | 0.866 | 0.490 | 0.447 | 0.468 |
| NegEx [5] | 0.881 | 0.975 | 0.925 | 0.885 | 0.792 | 0.836 | – | – | – | |
| RadText [35] | 0.859 | 0.939 | 0.897 | 0.792 | 0.637 | 0.706 | 0.599 | 0.323 | 0.420 | |
| BERT model [4] | 0.968 | 0.986 | 0.977 | 0.969 | 0.966 | 0.967 | 0.874 | 0.666 | 0.756 | |
| Prompt-based | 0.975 | 0.985 | 0.980 | 0.973 | 0.976 | 0.975 | 0.835 | 0.712 | 0.769 | |
| i2b2 2012 | Logistic Regression | 0.944 | 0.899 | 0.921 | 0.725 | 0.847 | 0.782 | 0.508 | 0.595 | 0.548 |
| NegEx [5] | 0.913 | 0.962 | 0.937 | 0.779 | 0.855 | 0.815 | – | – | – | |
| RadText [35] | 0.881 | 0.916 | 0.898 | 0.627 | 0.588 | 0.607 | 0.454 | 0.282 | 0.348 | |
| BERT model [4] | 0.959 | 0.951 | 0.955 | 0.831 | 0.905 | 0.866 | 0.693 | 0.616 | 0.652 | |
| Prompt-based | 0.961 | 0.951 | 0.956 | 0.846 | 0.906 | 0.875 | 0.671 | 0.641 | 0.656 | |
| BioScope | Logistic Regression | 0.904 | 0.989 | 0.945 | 0.724 | 0.847 | 0.780 | 0.919 | 0.592 | 0.720 |
| NegEx [5] | 0.784 | 0.999 | 0.879 | 0.658 | 0.587 | 0.621 | – | – | – | |
| RadText [35] | 0.804 | 0.871 | 0.836 | 0.495 | 0.870 | 0.631 | 0.912 | 0.283 | 0.432 | |
| BERT model [4] | 0.911 | 0.994 | 0.951 | 0.766 | 0.947 | 0.835 | 0.985 | 0.583 | 0.732 | |
| Prompt-based | 0.941 | 0.991 | 0.966 | 0.752 | 0.908 | 0.823 | 0.961 | 0.702 | 0.811 | |
| MIMIC-III | Logistic Regression | 0.920 | 0.879 | 0.899 | 0.782 | 0.921 | 0.846 | 0.507 | 0.411 | 0.454 |
| NegEx [5] | 0.867 | 0.954 | 0.908 | 0.855 | 0.871 | 0.863 | – | – | – | |
| RadText | 0.819 | 0.950 | 0.880 | 0.847 | 0.597 | 0.700 | 0.609 | 0.321 | 0.420 | |
| BERT model [4] | 0.937 | 0.965 | 0.951 | 0.929 | 0.945 | 0.937 | 0.775 | 0.518 | 0.621 | |
| Prompt-based | 0.946 | 0.953 | 0.950 | 0.922 | 0.945 | 0.933 | 0.722 | 0.611 | 0.662 | |
| NegEx | Logistic Regression | 0.985 | 0.874 | 0.926 | 0.725 | 0.945 | 0.821 | – | – | – |
| NegEx [5] | 0.977 | 0.988 | 0.983 | 0.951 | 0.912 | 0.931 | – | – | – | |
| RadText [35] | 0.901 | 0.748 | 0.817 | 0.434 | 0.680 | 0.530 | – | – | – | |
| BERT model [4] | 0.993 | 0.867 | 0.926 | 0.700 | 0.976 | 0.815 | – | – | – | |
| Prompt-based | 0.975 | 0.907 | 0.940 | 0.747 | 0.912 | 0.821 | – | – | – | |
| Chia | Logistic Regression | 0.606 | 0.810 | 0.693 | 0.798 | 0.408 | 0.540 | – | – | – |
| NegEx [5] | 0.639 | 0.946 | 0.763 | 0.896 | 0.465 | 0.612 | – | – | – | |
| RadText [35] | 0.570 | 0.916 | 0.703 | 0.803 | 0.293 | 0.430 | – | – | – | |
| BERT model [4] | 0.640 | 0.944 | 0.763 | 0.915 | 0.467 | 0.619 | – | – | – | |
| Prompt-based | 0.669 | 0.913 | 0.772 | 0.894 | 0.513 | 0.652 | – | – | – | |