Table 7.
Sentiment classification on Magazine, Sport, MR, Amazon, IMDB, and Yelp datasets.
| Magazine | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Metric\ model | RND | AFINN | RND + AFINN | JST | TJST | RJST | TS | WJST | WJST1 |
| Accuracy1 | 0.515 | 0.6522 | 0.5822 | 0.6705 | 0.705 | 0.5411 | 0.5022 | 0.8355 | 0.81 |
| Accuracy2 | — | — | — | — | — | — | — | 0.8372 | 0.8083 |
| Perplexity | — | — | — | 21.8506 | 23.0349 | 21.4593 | 21.3828 | 19.9914 | 21.2095 |
| Topic_coherency | — | — | — | −0.0548 | −0.0348 | −0.0946 | −0.0561 | −0.132 | −0.0077 |
|
| |||||||||
| Sport | |||||||||
| Accuracy1 | 0.5285 | 0.686 | 0.5725 | 0.653 | 0.709 | 0.5565 | 0.5155 | 0.798 | 0.802 |
| Accuracy2 | — | — | — | — | — | — | — | 0.782 | 0.795 |
| Perplexity | — | — | — | 22.874 | 23.1356 | 22.0264 | 22.3361 | 21.968 | 21.4821 |
| Topic_coherency | — | — | — | −0.2234 | −0.0876 | −0.1369 | −0.0544 | −0.1406 | −0.2242 |
|
| |||||||||
| MR | |||||||||
| Accuracy1 | 0.4895 | 0.601 | 0.5455 | 0.613 | 0.62 | 0.51 | 0.5 | 0.821 | 0.8445 |
| Accuracy2 | — | — | — | — | — | — | — | 0.818 | 0.843 |
| Perplexity | — | — | — | 33.8663 | 35.0695 | 35.2359 | 34.6704 | 33.222 | 33.7698 |
| Topic_coherency | — | — | — | −0.021 | −0.0139 | −0.0012 | −0.0409 | −0.001 | −0.0106 |
|
| |||||||||
| Amazon | |||||||||
| Accuracy1 | 0.491 | 0.731 | 0.574 | 0.611 | 0.645 | 0.609 | 0.54 | 0.779 | 0.796 |
| Accuracy2 | — | — | — | — | — | — | — | 0.829 | 0.798 |
| Perplexity | — | — | — | 12.6442 | 13.7316 | 13.349 | 14.2397 | 10.9211 | 12.9946 |
| Topic_coherency | — | — | — | −0.8318 | −4.3775 | −0.4224 | −0.5411 | −0.5874 | −0.2696 |
|
| |||||||||
| IMDB | |||||||||
| Accuracy1 | 0.498 | 0.698 | 0.575 | 0.605 | 0.616 | 0.546 | 0.545 | 0.76 | 0.77 |
| Accuracy2 | — | — | — | — | — | — | — | 0.761 | 0.774 |
| Perplexity | — | — | — | 21.1053 | 20.4419 | 20.8543 | 19.7124 | 14.6719 | 18.9035 |
| Topic_coherency | — | — | — | −1.3334 | −1.4868 | −0.8853 | −1.1246 | −0.9666 | −0.9438 |
|
| |||||||||
| Yelp | |||||||||
| Accuracy1 | 0.506 | 0.689 | 0.559 | 0.579 | 0.614 | 0.561 | 0.547 | 0.737 | 0.773 |
| Accuracy2 | — | — | — | — | — | — | — | 0.726 | 0.769 |
| Perplexity | — | — | — | 15.4565 | 16.7169 | 15.5965 | 15.3614 | 12.2145 | 13.3453 |
| Topic_coherency | — | — | — | −2.1865 | −2.5609 | −1.9632 | −1.1714 | −2.0815 | −2.3715 |