Table 7.
Generalizability evaluation results across different organizations.
| Method | Training organizations | Test organization | Precision (%) | Recall (%) | F1 score (%) | TP | FP | TN |
|---|---|---|---|---|---|---|---|---|
| Dictionary-based | N/A | MC | 73.4 | 47.9 | 58.0 | 1476 | 534 | 1605 |
| N/A | MCW | 70.7 | 49.2 | 58.0 | 3581 | 1487 | 3693 | |
| N/A | MDA | 72.7 | 47.4 | 57.3 | 1900 | 715 | 2113 | |
| CMM | MCW, MDA | MC | 83.0 | 79.2 | 81.0 | 2439 | 499 | 642 |
| MC, MDA | MCW | 78.9 | 67.7 | 72.9 | 4924 | 1319 | 2350 | |
| MC, MCW | MDA | 85.7 | 78.5 | 81.9 | 3150 | 527 | 863 | |
| CRF | MCW, MDA | MC | 83.3 | 78.9 | 81.0 | 2430 | 486 | 651 |
| MC, MDA | MCW | 78.4 | 68.2 | 72.9 | 4961 | 1369 | 2313 | |
| MC, MCW | MDA | 85.4 | 77.7 | 81.3 | 3117 | 535 | 896 |