Table 1.
Investigations by broad outcome groupings and ML parameters.
| Author | Year | Journal | Quality Rating | Clinical Sample | Outcome | Biomarker | NLP a | Classification | Specificity | Sensitivity | Accuracy | AUC b | N | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Kessler et al. [33] | 2017 | Int. J. Methods Psychiatr. Res. | 2 | x | 1 | x | 0.28 | 6360 | |||||||
| McCoy et al. [27] | 2016 | JAMA Psychiatry | 3 | x | 1 | x | x | 458,053 | |||||||
| Kessler et al. [34] | 2017 | Mol. Psychiatry | 2 | x | 1 | x | 0.95 | 0.46 | 0.70 | 0.70 | 975,057 | ||||
| Kessler et al. [15] | 2015 | JAMA Psychiatry | 3 | x | 1 | 0.89 | 53,769 | ||||||||
| Poulin et al. [35] | 2014 | PLoS One | 4 | x | 1 | x | x | x | 0.65 | 210 | |||||
| Pestian et al. [36] | 2008 | AMIA Annu. Symp. Proc. | 3 | 1 | x | x | 0.74 | 66 | |||||||
| Pamer et al. [37] | 2008 | AMIA Annu. Symp. Proc. | 3 | 1 | x | 1204 | |||||||||
| Haerian et al. [38] | 2012 | AMIA Annu. Symp. Proc. | 3 | x | 1 | x | x | 280 | |||||||
| Ilgen et al. [39] | 2009 | J Clin. Psychiatry | 3 | x | 1 | 887,859 | |||||||||
| Adamou et al. [40] | 2019 | Crisis | 3 | x | 1 | x | x | 0.70 | 130 | ||||||
| Rossellini [41] | 2018 | Depress. Anxiety | 3 | x | 1 | x | 0.86 | 9488 | |||||||
| De Avila Berni [42] | 2018 | PLOS One | 5 | 1 | x | 0.91 | 0.69 | 0.80 | |||||||
| Metzger et al. [43] | 2017 | Int. J. Methods Psychiatr. Res. | 3 | 2 | x | x | 444 | ||||||||
| Passos et al. [44] | 2016 | J. Affect. Disord. | 4 | x | 2 | 0.71 | 0.72 | 0.77 | 144 | ||||||
| Kessler et al. [45] | 2016 | Mol. Psychiatry | 2 | 2 | x | 0.70 | 0.76 | 1056 | |||||||
| Modai et al. [46] | 2004 | J. Nerv. Ment. Dis. | 2 | x | 2 | 0.85 | 0.94 | 987 | |||||||
| Modai et al. [47] | 2002 | JMIR Med. Inform. | 3 | x | 2 | 0.85 | 1.00 | 0.82 | 197 | ||||||
| Modai et al. [48] | 1999 | Med. Inform. Internet Med. | 3 | 2 | 0.94 | 0.94 | 0.94 | 198 | |||||||
| Modai et al. [49] | 2002 | Crisis | 2 | x | 2 | 250 | |||||||||
| Modai et al. [50] | 1998 | Med. Inform. | 4 | x | 2 | x | 0.97 | 0.83 | 161 | ||||||
| Hettige et al. [51] | 2017 | Gen. Hosp. Psychiatry | 3 | x | 2 | x | x | 0.8 | 0.65 | 0.67 | 0.71 | 345 | |||
| Walsh et al. [52] | 2017 | Clin. Psychol. Sci. | 3 | x | 2 | x | 0.96 | 0.84 | 5167 | ||||||
| Venek et al. [53] | 2017 | IEEE Trans. Affect. Comput. | 3 | x | 2 | x | 0.90 | 60 | |||||||
| Baca-Garcia et al. [54] | 2007 | Prog. Neuropsych.. Biol Psychiatry | 4 | x | 2 | x | x | 0.99 | 0.99 | 0.97 | 0.99 | 539 | |||
| Tiet et al. [55] | 2006 | Alcohol Clin. Exp. Res. | 3 | x | 2 | x | x | x | 0.87 | 0.89 | 0.88 | 34,251 | |||
| Baca-Garcia et al. [56] | 2010 | Am. J. Med. Genet. B Genet. | 3 | x | 2 | x | x | 0.82 | 0.50 | 0.67 | 0.66 | 277 | |||
| Lopez-Castroman et al. [57] | 2011 | J. Psychiatry Res. | 4 | x | 2 | x | x | 0.97 | 0.76 | 0.71 | 1349 | ||||
| Modai et al. [58] | 2004 | Med. Inform. Internet Med. | 3 | x | 2 | 0.70 | 0.83 | 0.77 | 612 | ||||||
| Mann et al. [59] | 2008 | J. Clin. Psychiatry | 3 | x | 2 | x | 0.92 | 0.89 | 0.80 | 408 | |||||
| Bae et al. [60] | 2015 | Neuropsychiatr. Dis. Treat. | 3 | 2 | x | 2754 | |||||||||
| Choo et al. [61] | 2014 | Asian J. Psychiatr. | 3 | x | 2 | 0.90 | 418 | ||||||||
| Oh et al. [62] | 2017 | Front. Psychol. | 3 | x | 2 | x | 0.99 | 0.78 | 0.97 | 573 | |||||
| Benton et al. [63] | 2017 | Proc. 15th Conf. EACL | 3 | x | 2 | x | 9611 | ||||||||
| Ruderfer et al. [64] | 2019 | Mol. Psychiatry | 3 | 2 | x | 0.82 | 0.92 | 0.94 | 512,639 | ||||||
| Lyu and Zhang [65] | 2019 | J. Affect. Disord. | 3 | x | 2 | 0.94 | 0.68 | 0.85 | 1318 | ||||||
| Coppersmith et al. [66] | 2018 | Biomed. Inform. Insights | 3 | 2,6 | x | x | 0.94 | 418 | |||||||
| Dargel et al. [67] | 2018 | Acta Psychiatr. Scand. | 3 | x | 2 | x | 0.84 | 635 | |||||||
| Jordan et al. [68] | 2018 | Psychiatry Res. | 3 | x | 2 | x | 0.69 | 0.79 | 0.72 | 218 | |||||
| Setoyama et al. [69] | 2016 | PLoS One | 2 | x | 3 | x | x | 0.70 | 90 | ||||||
| Pestian et al. [70] | 2017 | Suicide Life Threat. Behav. | 2 | x | 3 | x | x | 0.93 | 0.85 | 379 | |||||
| Cook et al. [71] | 2016 | Comput. Math Meth. Med. | 2 | x | 3 | x | x | 0.57 | 0.56 | 0.85 | 0.61 | 1453 | |||
| Pestian et al. [26] | 2016 | Suicide Life Threat. Behav. | 2 | x | 3 | x | x | 0.96 | 0.97 | 60 | |||||
| Gradus et al. [72] | 2017 | J. Trauma Stress | 4 | x | 3 | x | 0.92 | 2240 | |||||||
| Birjali et al. [73] | 2017 | Procedia Comput. Sci. | 3 | 3 | x | x | |||||||||
| Just et al. [74] | 2017 | Nat. Hum. Behav. | 3 | x | 3 | x | x | 0.94 | 79 | ||||||
| Ryu et al. [75] | 2018 | Psychiatric Invest. | 3 | 3 | x | 0.81 | 0.84 | 0.80 | 11,628 | ||||||
| Desjardins et al. [76] | 2016 | J. Clin. Psychiatry | 4 | x | 4 | x | 0.93 | 879 | |||||||
| Tzeng [77] | 2006 | Worldview Evid. Base. Nurs. | 2 | x | 4 | x | 63 | ||||||||
| Baca-García et al. [78] | 2006 | J. Clin. Psychiatry | 3 | x | 4 | x | 1.00 | 0.99 | 509 | ||||||
| Quan et al. [79] | 2014 | PLoS One | 2 | 4 | x | ||||||||||
| Litvinova et al. [80] | 2017 | Comput. y Sistemas | 3 | x | 4 | x | x | 0.72 | 1000 | ||||||
| Zhang et al. [81] | 2019 | Health Inform. J. | 3 | x | 4 | x | x | 409 | |||||||
| McKernan et al. [82] | 2019 | Arthritis Care Res. | 3 | x | 2,3 | x | 1.00 | 1.00 | 0.82 | 8879 | |||||
| DelPozo-Banos et al. [83] | 2018 | JMIR Ment. Health | 3 | x | 1 | x | 0.85 | 0.65 | 0.80 | 2604 | |||||
| Burke et al. [84] | 2018 | Psychiatry Res. | 3 | 4 | x | 0.89 | 359 | ||||||||
| Cheng et al. [85] | 2017 | J. Med. Internet Res. | 4 | 5 | x | x | 0.61 | 974 | |||||||
| Braithwaite et al. [86] | 2016 | JMIR Ment. Health | 4 | 5 | x | 0.97 | 0.53 | 0.91 | 135 | ||||||
| Guan et al. [87] | 2015 | JMIR Ment. Health | 4 | 5 | x | 909 | |||||||||
| Woo et al. [88] | 2015 | Int. J. Environ. Res. Public Health | 4 | 5 | x | ||||||||||
| O’Dea et al. [89] | 2015 | Internet Interv. | 3 | 5 | x | 0.76 | 14,701 | ||||||||
| Nguyen et al. [90] | 2017 | Multimed. Tools Appl. | 3 | x | 5 | x | 0.88 | ||||||||
| Burnap et al. [91] | 2017 | Online Soc. Netw. Media | 3 | 5 | x | x | 0.85 | ||||||||
| Vioules et al. [92] | 2018 | IBM J. Res. Dev. | 3 | 5 | x | x | 120 | ||||||||
| Zalar et al. [93] | 2018 | Psychiatr. Danub. | 3 | 1, 2 | x | 0.91 | 78,625 | ||||||||
| Tran et al. [94] | 2015 | J. Biomed. Inform. | 3 | x | 1,2 | x | 7578 | ||||||||
| Leiva-Murillo et al. [95] | 2013 | Comput. Math. Methods Med. | 3 | x | 1,2 | x | 8699 | ||||||||
| Tran et al. [96] | 2014 | BMC Psychiatry | 3 | x | 1,2 | 0.97 | 0.79 | 7399 | |||||||
| Bernecker et al. [97] | 2019 | Behav. Res. Ther. | 3 | x | 2, 3 | x | 0.83 | 27,501 | |||||||
| Zhong et al. [98] | 2019 | Euro. J. Epidemiol. | 3 | 2, 3 | x | x | 0.96 | 0.34 | 0.83 | 275,843 | |||||
| Morales et al. [99] | 2017 | Front. Psychiatry | 4 | x | 2,3 | x | 0.79 | 0.71 | 707 | ||||||
| Barros et al. [100] | 2017 | Braz. J. Psychiatry | 4 | x | 2,3 | x | 0.79 | 0.77 | 0.78 | 707 | |||||
| Kuroki [101] | 2015 | Am. J. Orthopsychiatry | 3 | 2,3 | x | x | 624 | ||||||||
| Anderson et al. [102] | 2015 | J. Am. Board Fam. Med. | 3 | x | 2,3,4 | x | x | 0.96 | 0.94 | 0.95 | 15,761 | ||||
| Levey et al. [103] | 2016 | Mol. Psychiatry | 3 | x | 1,3,4 | x | 0.94 | 114 | |||||||
| Colic et al. [104] | 2018 | Conf. Proc. IEEE Eng. Med. Biol. Soc. | 3 | x | 3 | 0.84 | 738 | ||||||||
| Aladag et al. [105] | 2018 | J. Med. Internet Res. | 3 | 5 | x | x | 0.92 | 785 | |||||||
| Choi et al. [106] | 2018 | S. Korean J. Affect. Disord. | 3 | 1 | x | 0.72 | 819,951 | ||||||||
| Downs et al. [107] | 2018 | AMIA Annu. Symp. Proc. | 3 | x | 4 | x | x | 1906 | |||||||
| Fahey [108] | 2018 | Soc. Sci. Med. | 3 | 5 | x | x | 0.80 | 974,891 | |||||||
| Zhong et al. [109] | 2018 | BMC Med. Inform. Decis. Mak. | 3 | x | 2-4 | x | x | 275,843 | |||||||
| Fernandes et al. [110] | 2018 | Sci. Rep. | 3 | 2,3 | x | x | 0.87 | ||||||||
| Jordan et al. [111] | 2018 | Gen. Hosp. Psychiatry | 3 | x | 3 | x | 0.83 | 0.87 | 6805 | ||||||
| Carson et al. [112] | 2019 | PLOS one | 3 | x | 2 | x | x | 0.22 | 0.83 | 0.47 | 0.68 | 73 | |||
| McCoy et al. [113] | 2018 | Depress. Anxiety | 3 | 1 | x | 444,317 | |||||||||
| Connolly et al. [114] | 2017 | BMC Bioinform. | 3 | x | 3 | x | 314 | ||||||||
| Modai et al. [115] | 2002 | Euro. Psychiatry | 4 | x | 2 | 0.73 | 0.65 | 250 | |||||||
| Rossellini et al. [116] | 2017 | Psychol. Med. | 3 | 2 | x | 0.74 | 21,832 | ||||||||
Investigations by broad outcome groupings and ML parameters. Outcomes: 1 = suicide death, 2 = suicide attempt/medically serious suicide attempt, 3 = suicidal ideation/state, 4 = other-undifferentiated, 5 = other-social media risk outcomes. Notes: NLP = natural language processing a; AUC = area under the curve b; empty cells indicate missing/unreported data in articles; ML = machine learning.