Table 10.
Correlation matrix (Pearson correlation coefficients) of similarity approaches among evaluators for summaries according to the three metrics.
Metric | Evaluator A | Evaluator B | Evaluator C |
M1a | 0.728 | 0.767 | 0.837 |
M2b | 0.826 | 0.924 | 0.841 |
M3c | 0.772 | 0.843 | 0.804 |
aM1: summary relevance to the inbound query.
bM2: aim, population, intervention, results, and outcome classification representation in the summary.
cM3: model summary better than the baseline summary.