Table 3.
Coding categories, category prevalence, and coder agreement per category
| Dataset | Coding categories | % of texts coded for (min–max)a |
Fleiss’ kappa (min–max)b |
PABAK (min–max)b |
|---|---|---|---|---|
| Feedback data | Reach out to the seller, Share objective facts, Express feelings, Help the seller, Avoid harming the seller, Help other buyers, Reward or punish the seller | 3.70–69.80 | .25–.53 | .42–.90 |
| Election data | Care, Harm, Fairness, Cheating, Loyalty, Betrayal, Authority, Subversion, Purity, Degradation, Non-moral | 2.11–56.24 | .18–.45 | .73–91 |
| Reddit data | Anger, Disgust, Enjoyment, Fear, Sadness, Surprise | 1.34–40.08 | .27–.53 | .54–.95 |
| Hate speech data | Hate speech, Offensive language, Neither | 5.77–77.43 | .55 | .72 |
a The % of texts coded for the least and most frequently occurring coding categories, respectively
b The lowest and highest values, respectively, of Fleiss’ kappa/PABAK across the categories in the coding scheme