Table 1.
Interrater results from the creation of the training data. Tweets were randomly sampled from the regular expression-filtered data sets and duplicates were removed. Each rater was assigned a portion of overlapping tweets to allow for interrater evaluation.
Theory | Unique tweets labeled (n) | Tweets labeled by multiple authors (n) | Agreement | Cohen κ |
5Ga | 725 | 146 | 0.852 | 0.708 |
Gatesb | 711 | 143 | 0.893 | 0.782 |
Labc | 735 | 146 | 0.901 | 0.796 |
Vaxd | 775 | 199 | 0.915 | 0.751 |
a5G: conspiracy theories related to 5G technology.
bGates: conspiracy theories related to Bill Gates or the Bill & Melinda Gates Foundation.
cLab: conspiracy theories related to SARS-CoV-2 being laboratory-released or human-made.
dVax: conspiracy theories related to vaccines.