Skip to main content
. 2026 Aug 7;13:1150. doi: 10.1038/s41597-026-07753-8

Table 3.

Agreement Matrix for Mention and Use.

Metric Mention (%) Use (%)
Concurrence Rate (LLM vs Human Majority) 76.0 56.0
Plausibility Rate (LLM vs Minimum 1 Rater) 80.0 82.0
Pairwise Agreement - Raters 1 and 2 98.0 68.0
Pairwise Agreement - Raters 1 and 3 94.0 74.0
Pairwise Agreement - Raters 2 and 3 92.0 74.0
Overall Human Pairwise Agreement 94.7 72.0

Note: Concurrence rate is the proportion of publication-dataset pairs in which the LLM classification matched the human majority vote (yes or no for mention; yes or no for use). Plausibility rate is defined as agreement with at least one of the three reviewers.