Table 3:
Performance of T5 with domain adaptation pre-training using Assessment (Assmt) as input, under two mask policies: Token Masking and Concept Masking. We report Rouge-L F-score (RL-F), and BERTScore (BS), as well as Sentence embedding cosine similarity (Sent θ) and CUI F-score. Numbers with green background address the highest performance across all results, with subscript number (↑) denoting the improvements over rule-based results.
| Setting | Model | Token Masking | Concept Masking | ||||||
|---|---|---|---|---|---|---|---|---|---|
|
| |||||||||
| RL-F | Sent.θ | BS | CUI | RL-F | Sent.θ | BS | CUI | ||
| Explicit | Assmt | 32.66 | 61.34(↑2.53) | 56.68 | 47.10(↑8.13) | 29.86 | 55.87 | 53.91 | 40.27(↑2.14) |
| ++ | 26.94 | 59.40(↑0.59) | 55.05 | 42.73(↑3.76) | 32.82 | 58.21 | 56.80 | 43.16(↑4.19) | |
|
| |||||||||
| Direct | Assmt | 12.69 | 53.63 | 42.40(↑2.27) | 35.39(↑1.16) | 14.90(↑2.59) | 55.48(↑0.15) | 47.10(↑6.97) | 35.29(↑1.06) |
| ++ | 10.44 | 53.47 | 43.46(↑3.33) | 37.45(↑3.22) | 15.76(↑5.22) | 56.82(↑1.49) | 48.72(↑8.72) | 37.74(↑3.51) | |
|
| |||||||||
| Indirect | Assmt | 10.07(↑0.58) | 52.72 | 41.47 | 38.19(↑5.03) | 13.58(↑4.36) | 53.44 | 44.91(↑0.45) | 33.56(↑0.40) |
| ++ | 8.04 | 51.84 | 40.45 | 37.53(↑4.37) | 13.28(↑4.06) | 55.02 | 45.51(↑1.05) | 35.10(↑1.94) | |
|
| |||||||||
| All | Assmt | 14.49(↑1.04) | 62.40 | 49.62 | 40.44 | 18.72(↑5.27) | 64.69 | 54.03(↑3.71) | 42.69 |
| ++ | 12.12 | 63.08 | 50.20 | 45.58(↑1.65) | 18.80(↑5.35) | 66.08 | 55.29(↑4.86) | 44.56(↑0.63) | |