Table 2.

Sentiment tools based on type of tool: KNN: k-nearest-neighbors; N/A: not applicable; NB: Naïve Bayes; SVM; support vector machines.

Author	Tool	Annotators	Kappa	Manually annotated sample	Sample size	Manually annotated compared with total sample, n (%)
Cole-Lewis et al [29]	Produced for study: machine learning classifiers based on 5 categories (NB, KNN, and SVM)	6	.64	250	17,098	250 (1.46)
Desai et al [31]	Produced for study: rule based using AFINN (Named after the author, Finn Arup Neilsen)	N/A	N/A	N/A	993	N/A
Daniulaityte et al [30]	Produced for study: logistic regression, NB, SVM	2	.68	3000	N/A	N/A
Myslin et al [34]	Produced for study: machine learning (NB, KNN, SVM)	2	>.7	1000	7362	1000 (13.58)
Sofean and Smith [36]	Produced for study: 5-fold validation using support vector machines (SVM’s) model using Waikato Environment for Knowledge Analysis toolkit toolkit	N/A	N/A	500	N/A	N/A
Tighe et al [37]	Produced for study: rule based using AFINN	N/A	N/A	N/A	65,000	N/A
Bhattacharya et al [27]	Open source: SentiStrength	3	N/A	N/A	164,104	N/A
Hawkins et al [33]	Open source: machine learning classifier using Python library TextBlob	2+Amazon Mechanical Turk	>.79	2216	404,065	2216 (0.55)
Ramagopalan et al [26]	Open source: TwitteR R package + Jeffrey Breen’s sentiment analysis code	N/A	N/A	N/A	60,037	N/A
Black et al [28]	Commercial: radian6	N/A	N/A	N/A	N/A	N/A
Greaves et al [32]	Commercial: TheySay	N/A	N/A	250	198,499	250 (0.13)
Nwosu et al [35]	Open source: TopsyPro	N/A	N/A	N/A	683,500	N/A