Table 11.
Comparing F1-score of deep learning models (TREC-6 dataset) with focusing POS tags (N+V+Adj+Det: NVAD).
Input | Model | Micro-averaged |
Macro-averaged |
||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
P | R | (Mean) | (SD) | (Max) | P | R | (Mean) | (SD) | (Max) | ||
Original | CNN | 0.9600 | 0.9600 | 0.9600 | 0.0042 | 0.9680 | 0.9641 | 0.9652 | 0.9641 | 0.0037 | 0.9710 |
BiLSTM | 0.9732 | 0.9732 | 0.9732 | 0.0041 | 0.9800 | 0.9753 | 0.9760 | 0.9755 | 0.0038 | 0.9816 | |
Hybrid | 0.9718 | 0.9718 | 0.9718 | 0.0027 | 0.9780 | 0.9737 | 0.9746 | 0.9741 | 0.0024 | 0.9796 | |
+all POS Tags | CNN | 0.9594 | 0.9594 | 0.9594 | 0.0031 | 0.9640 | 0.9628 | 0.9641 | 0.9630 | 0.0029 | 0.9674 |
BiLSTM | 0.9753 | 0.9753 | 0.9738 | 0.0035 | 0.9780 | 0.9772 | 0.9775 | 0.9758 | 0.0033 | 0.9799 | |
Hybrid | 0.9720 | 0.9720 | 0.9720 | 0.0028 | 0.9760 | 0.9739 | 0.9748 | 0.9742 | 0.0025 | 0.9779 | |
+focusing Tags | CNN | 0.9644 | 0.9644 | 0.9644 | 0.0034 | 0.9700 | 0.9675 | 0.9688 | 0.9678 | 0.0029 | 0.9723 |
BiLSTM | 0.9738 | 0.9738 | 0.9738 | 0.0033 | 0.9780 | 0.9756 | 0.9764 | 0.9759 | 0.0030 | 0.9799 | |
Hybrid | 0.9694 | 0.9694 | 0.9694 | 0.0035 | 0.9740 | 0.9715 | 0.9729 | 0.9720 | 0.0030 | 0.9760 | |
+focusing Words | CNN | 0.9668 | 0.9668 | 0.9668 | 0.0045 | 0.9740 | 0.9697 | 0.9711 | 0.9699 | 0.0041 | 0.9762 |
BiLSTM | 0.9724 | 0.9724 | 0.9724 | 0.0040 | 0.9780 | 0.9745 | 0.9753 | 0.9748 | 0.0035 | 0.9798 | |
Hybrid | 0.9750 | 0.9750 | 0.9750 | 0.0056 | 0.9820 | 0.9767 | 0.9779 | 0.9770 | 0.0051 | 0.9834 |