Social media based surveillance systems for healthcare using machine learning: A systematic review

Graphical abstract

Cloud computing, patient diagnosis, video surveillance, mobile computing, microorganisms, patient monitoring, wireless sensor networks.

Abstract

Background

Real-time surveillance in the field of health informatics has emerged as a growing domain of interest among worldwide researchers. Evolution in this field has helped in the introduction of various initiatives related to public health informatics. Surveillance systems in the area of health informatics utilizing social media information have been developed for early prediction of disease outbreaks and to monitor diseases. In the past few years, the availability of social media data, particularly Twitter data, enabled real-time syndromic surveillance that provides immediate analysis and instant feedback to those who are charged with follow-ups and investigation of potential outbreaks. In this paper, we review the recent work, trends, and machine learning(ML) text classification approaches used by surveillance systems seeking social media data in the healthcare domain. We also highlight the limitations and challenges followed by possible future directions that can be taken further in this domain.

Methods

To study the landscape of research in health informatics performing surveillance of the various health-related data posted on social media or web-based platforms, we present a bibliometric analysis of the 1240 publications indexed in multiple scientific databases (IEEE, ACM Digital Library, ScienceDirect, PubMed) from the year 2010–2018. The papers were further reviewed based on the various machine learning algorithms used for analyzing health-related text posted on social media platforms.

Findings:

Based on the corpus of 148 selected articles, the study finds the types of social media or web-based platforms used for surveillance in the healthcare domain, along with the health topic(s) studied by them. In the corpus of selected articles, we found 26 articles were using machine learning technique. These articles were studied to find commonly used ML techniques. The majority of studies (24%) focused on the surveillance of flu or influenza-like illness (ILI). Twitter (64%) is the most popular data source to perform surveillance research using social media text data, and Support Vector Machine (SVM) (33%) being the most used ML algorithm for text classification.

Conclusions

The inclusion of online data in surveillance systems has improved the disease prediction ability over traditional syndromic surveillance systems. However, social media based surveillance systems have many limitations and challenges, including noise, demographic bias, privacy issues, etc. Our paper mentions future directions, which can be useful for researchers working in the area. Researchers can use this paper as a library for social media based surveillance systems in the healthcare domain and can expand such systems by incorporating the future works discussed in our paper.

1. Introduction

Syndromic surveillance systems aim for the collection of data, which can help in providing a basic scenario for all communicable infectious diseases. These systems may vary depending on the source of data, their planned duration, and how data is recorded and acquired. These systems may use traditional data and real-time data from various social media platforms. In the field of healthcare, these systems usually focus on the early identification of illness clusters and the symptom period before the confirmation of a particular disease by any clinical unit or laboratory and to mobilize the rapid response. Surveillance systems are usually concerned with the systematic collection, analysis, and interpretation of the collected data along with the detection, confirmation, and reporting of disease, and also considering the public health response. Objectivized definitions, algorithmic diagnosis, and electronic databases have made surveillance systems more user-friendly and effective over time [1]. Traditional biosurveillance relies on clinical encounters to collect information, which is a time-consuming process. Also, traditional pandemic surveillance is mostly a manual process that causes a delay of one to two weeks in the availability of the data by clinical diagnosis [2], [3]. In the last few years, the availability of web-based data sources emerged as an extension to traditional surveillance systems [4] and has sustainably contributed to infectious disease surveillance by providing real-time statistics and reducing the cost of public health [5]. It can be noted how rapidly events can be detected in real-time using internet-based surveillance data when an ordinary individual's social media post has led to a tremendous increase in public engagement with skin cancer prevention [6]. Despite the other uses of social media [7], the role of monitoring social media can be explored in making healthcare decisions [8]. Also, the early detection of diseases and immediate public health response has led to the need for new approaches and technologies to reinforce the capacity of traditional syndromic surveillance systems. Several research papers/articles have been appeared in reviewing the various surveillance systems using social-media data [5], [9], [10], [11], [13], [14], [15], [16], [17], [18]. These papers have covered the applications, technologies, algorithms, data sources, and their evaluation. However, a recent review of surveillance systems in the health informatics domain through social media is not available to the best of our knowledge. In this paper, we review machine learning technologies and approaches published in this domain, mainly in the past few years, and also mention the challenges and future directions.

This review paper examines a set of research questions that would allow us to get the latest trends followed by social media based surveillance systems in the field of healthcare. Moreover, it also helps us to get an overview of recently used machine learning algorithms for analyzing the data used by these systems. These research questions (RQ) are stated as follows:

• •RQ1: Which machine learning techniques are popular among authors of research papers when developing a social media based surveillance systems in the health sector?
• •RQ2: What are the most commonly used sources of social media data for the surveillance of health-related topics?
• •RQ3: What can be the applications of social media based surveillance systems in the area of health informatics?
• •RQ4: Are there any limitations or challenges faced by syndromic surveillance systems with the inclusion of social media data?

To answer the research questions mentioned above, we extracted 1240 research articles that published studies related to our research, from various scientific digital libraries, and inspected them. The methodology for the selection of these articles is discussed in Section 2.

When looking at the surveillance systems in the field of health informatics, various machine learning algorithms like Deep Neural Network (DNN), Naive Bayes (NB), Multinomial Naive Bayes (NBM), SVM, etc. have been chosen and proposed for epidemic prediction classification approach. We will answer RQ1 by discussing and explaining a few machine learning approaches in Section 3, though the actual work is much broader. To answer RQ2, Section 4 will identify the social media sources involved in the collection of data, followed by the applications of such surveillance systems in the next section, i. e. Section 5. This section holds the answer to question RQ3. Section 6 aims to answer RQ4 by discussing the limitations and challenges, and Section 7 mentions the original contribution of this writing, followed by the conclusion, results, and future work in Section 8.

2. Articles selection method

The scope of this paper involves studies of the social media-based surveillance systems that predict the disease in real-time or near real-time using machine learning approaches. The selection criteria for the research articles were set to incorporate papers published in the year 2010–2018.

The following scientific databases were explored to provide a comprehensive bibliography of research papers on social media based surveillance systems in the healthcare domain:

• •ACM Portal
• •IEEE Xplore
• •Science Direct
• •PubMed

IEEE Xplore database was advanced searched to form the following query: (((“Abstract”: surveillance) OR “Document Title”: surveillance OR “Abstract”: outbreak) AND (“Abstract”: health OR “Abstract”: disease)) and 656 articles (Conferences and Journals & Magazines) were retrieved when filters mentioned in Fig. 1 were applied. Similarly, an ACM Digital Library searched for query: recordAbstract:(((outbreak OR surveillance) OR acmdlTitle:(+surveillance)) AND (health* OR disease)) retrieved 265 articles.Also, we advanced searched ScienceDirect database for query: (surveillance OR outbreak) AND (health* OR disease) AND “social media” in title, abstract, and keywords. As a result, 75 articles were extracted based on the search terms. Lastly, PubMed, which accesses the MEDLINE database, was searched for publications. On searching this resource with query: (((surveillance[Title/Abstract]) OR outbreak[Title/Abstract]) AND((health[Title/Abstract]) OR disease[Title/Abstract]) AND social media[Title/Abstract]), we get 244 articles . For further analysis, a total of 1240 articles were identified in the initial query of the knowledge sources.

Fig. 1.

Search methodology for the selection of relevant articles. * Cloud computing, patient diagnosis, video surveillance, mobile computing, microorganisms, patient monitoring, wireless sensor networks.

Each of the 1240 articles, were screened independently by each author of the paper, based on abstract and title. If the abstract or title or both are explaining social media or web-based surveillance, then we considered them for further research else they were rejected. While following this step, an article was included in the corpus only when both authors agreed it was relevant; disagreements were handled using consensus. The next step we performed was to consider the articles that had utilized machine learning approaches in their methodology. Fig. 1 describes the steps followed in selecting the relevant articles for this study.

Besides, we searched Google Scholar to get the statistics related to our research area, reflecting the trends of the past few years. Fig. 2 shows the count of recent research papers and patents published since 2010 till 2018. The terms involved for research were the “surveillance system”, “social media”, “machine learning” and “health informatics”. This plot clearly shows an increase in the number of publications about surveillance systems involving social media data and machine learning algorithms in the healthcare domain over time.

Fig. 2.

Distribution per year of articles on Google Scholar.

3. RQ1: machine learning methods used by surveillance systems for processing social media data

In this section, we will explain the most commonly used machine learning based classification methods employed to analyze health-related text from social media platforms. As mentioned in section 2, a total of 26 articles were found relevant for studying the ML-based text classification algorithms.

In recent years, machine learning has gained much attention, especially in analyzing the patterns in images or raw data. M. Bates [19] addressed how the progress in machine learning allows epidemiologists to mine through a broad set of digital data. A. Mike and C. Daniel [11] reviewed the conjunction of natural language processing and machine learning with social media platforms to support the analysis of massive dataset for population-level mental health research. Among different methodological variations of machine learning, some architecture stands out in popularity. For instance, we noted that the k-Nearest Neighbor (k-NN) classifier's precision was superior to several other machine learning classifiers such as NBM Modal, NB, and SVM [20], for classifying the tweets between two classes-i.e., real occurrences of allergy or awareness tweets. Similarly, K. Lee, A. Agrawal, A. Choudhary et al. [21] showed that best text classification performance was obtained using Multinomial Naive Bayes Modal with F-measure of 0. 811 when compared against the other classifiers such as NB, Random Forest (RF), and SVM. The author [22] exhibited a model using the multilayer perceptron with backpropagation algorithm on Twitter data to predict the weekly status of the US population infected with ILI. Even several supervised machine learning algorithms were studied for detecting the personal health experience tweets [23] and used the deep gramulator approach to improve precision when applied to independent test sets.

The unsupervised classification algorithms do not require labeled data sets to predict the output, like supervised algorithms. Because of this reason, the unsupervised classification methods seem to be a more attractive alternative in the process of analyzing the text, but they could be more challenging in achieving a similar accuracy as supervised methods. The same can be observed [24] when L. Sousa, R. de Mello, D. Cedrim et al. performed the classification of tweets using supervised and unsupervised methods. They concluded that the topic modeling (LDA), one of the unsupervised techniques, presents less control over the content of topics in comparison to a traditional classifier, particularly on a naturally noisy media channel. Hence, Multinomial Naive Bayes, a supervised classification approach, was considered for classifying the Twitter content.

Extensive use of machine learning algorithms to process and distinguish health-related social media data includes [20], [21], [23], [25].

Fig. 3 represents the commonly used machine learning methods for health-related text classification in selected papers.

Fig. 3.

Distribution of ML methods for health-related text classification by social media based surveillance systems. NB: Naive Bayes, NBM: Multinomial Naive Bayes, k-NN: k-Nearest Neighbor, ME: Maximum Entropy, LR: Logistic Regression, DT: Decision Tree, DNN: Deep Neural Network, RF: Random Forest, SVM: Support Vector Machine.

3.1. Support vector machine

SVM is a popular binary classifier built upon the concept of decision planes that define decision boundaries. In this approach, original training data is transformed into a higher dimension using a nonlinear mapping. Within this new dimension, a linear optimal separating hyperplane is searched to minimize the distance between hyperplane points and maximize the margin between the classes [26], [27]. It has been known for its superior performance in text classification with word features. Although the performance of classification algorithms highly depends on the input parameters and application, yet for binary classification tasks, SVM was observed to be highly suitable. V. K. Jain and S. Kumar [28] reported the SVM classifier as a good performer in terms of accuracy in predicting the class of tweets(disease-related tweets/irrelevant tweets). Similarly, the SVM algorithm was able to achieve an accuracy of 90.09% when tweets were classified as infodemiological or non-infodemiological [29]. N. Yang, X. Cui, C. Hu C et al. [30] were able to classify 'sick microblog' and 'not sick microblog' posts using the SVM classification model. They also showed that time consumption by SVM for classification task did not get much affected when the micro-blogs needed to be arranged increased by 100 times, though there is a huge increase in the time consumption by KNN to complete the classification task. SVM turned out to be the best tweet classification method when compared with other machine learning techniques and has been utilized to classify social media data on a range of physical health issues: [20], [21], [23], [24], [25], [31], [32], [33], [34], [35], [36], [37], [38], [39].

3.2. Naive Bayes

It is a classification algorithm for binary and multiclass classification problems. This algorithm makes a naive assumption that there are no predictors i. e. features are independent of each other, and one feature's impact on predicting class does not depend on the presence of another feature [27]. This method is based on the Bayes Theorem, shown as below:

$$P\left(H|X\right)=\frac{P\left(X|H\right)P\left(H\right)}{P\left(X\right)}$$ (1)

Where:

H: Hypothesis that data X belongs to a specific class C

X: Data with the class yet known

P(H|X): Posterior probability of hypothesis H conditioned on X

P(H): Prior probability of hypothesis H

P(X|H): Posterior probability of X conditioned based hypothesis H

P(X): Prior Probability of X.

V. K. Jain and S. Kumar [28] classified datasets into mosquito-borne disease-relevant and irrelevant tweets using SVM and Naive Bayes, and further relevant tweets were classified into three classes: symptoms, fear, and prevention using same classifiers. V. Kumar and S. Kumar [25] performed the classification task on tweets to differentiate swine flu-related text from noise or irrelevant tweets using various ML techniques such as Decision Tree, Naive Bayes, SVM, and Random Forest. They considered every relevant swine flu-related word as a feature and found both Naive Bayes and SVM provided the best classification result, with F-measure of 0. 77. The Naive Bayes classifier gave the best performance when dengue suspected tweets were classified as irrelevant or relevant, considering emojis, location information, unigrams, bigrams, and trigrams [40]. Naive Bayes is a popular classification algorithm and is used by many authors for text classification: [13], [20], [21], [24], [36], [41], [39] showing an average performance when compared to other classifiers.

3.3. Multinomial Naive Bayes:

Multinomial Naive Bayes networks, a variant of Naive Bayes networks, are better for text documents. Multinomial networks consider the frequency of words, and underlying calculations of probability are adjusted accordingly, while NB networks do not consider the frequency count.

X. Ji, S.A. Chun and J. Geller [36] used different ML methods (NB, NBM, and SVM) to classify tweets as either personal or news-related. They further classified personal tweets into two categories: negative or neutral tweets. Among all the methods used for classification and sentiment analysis, NBM accomplished overall the best outcome and turned out to be better than the other two machine learning classifiers regarding time consumed to build the classifier. Similarly, K. Lee, A. Agrawal, and A. Choudhary [21] developed a real-time allergy surveillance system that distinguished tweets as either positive or negative, where a positive tweet mentions about the author or someone around the author having allergy symptoms. And, the tweet is labeled negative if it talks about news, advertisement, or general awareness of allergies. They showed that best text classification performance was obtained using Naive Bayes Multinomial Modal with F-measure of 0. 811 when compared against the other classifiers such as NB, RF, and SVM. Similarly, this classifier gave the best recall and f-measure of 0. 859 and 0. 857 when the tweets were classified among the multiple allergy hidden classes [20].

3.4. k-Nearest neighbor

k-Nearest-Neighbor is an instance-based statistical analysis method to perform classification. Its implementation requires an integer k, a set of training data and measure closeness metric. The given training set is used as an input vector to form different regions for different classes. When given an unlabeled object, this classifier searches for k training sets in pattern space that are closest to the unlabeled object. These k training sets are the k “nearest neighbors” of the unlabeled object. “Closeness” is defined in terms of a distance metric, such as Euclidean distance, which is given by:

$$dist\left(X1,X2\right)=\sqrt{{\sum }_{i=1}^n{(x1i-x2i)}^2}$$ (2)

where, X1 = (x11, x12, …, x1n) and X2 = (. x21, x22, …, x2n) representing two objects or points [27], [42]. Nargund K, Natarajan S. [20] used k-NN alongside Naive Bayes, SVM, and Naive Bayes Multinomial to identify and recognize messages reporting and discussing different types of allergies. They noted that k-NN has better precision than other approaches in the identification and assignment of tweets as either actual incident of allergy or awareness tweets. This classifier was observed as having a good precision for text classification than other classifiers such as k-means, but not the best [30]. Other research papers that have used the k-NN are: [23], [33].

3.5. Logistic regression

Logistic regression was proposed in the late 1960s and early 1970s and became routinely available in statistical packages in the early 1980s [43]. LR is a statistical technique for analyzing a dataset for a binary classification problem. It helps in discovering the relationship between a dependent binary variable and at least one independent variable. Each independent variable is multiplied with weights and summed up. This outcome will sum up to a sigmoid function to get the result in the range of 0 and 1. The values below 0.5 are considered as 0, and those above 0.5 are considered as 1. In this manner, optimization techniques aim to find the best regression coefficients and weights. Logistic regression is mathematically constrained to produce probabilities in the range [0,1]. Also, it can converge on parameter estimates relatively easily.

Along with different classification algorithms, logistic regression is also preferred for the data classification task. For instance, the logistic regression gave a better recall and F1 measure than SVM in the classification of asthma relevant and irrelevant tweets [32]. Logistic regression showed excellent precision for the analysis of personal and non-personal tweets [23]. Among the research papers, we have reviewed [33], [44] also utilized logistic regression classifier. Maximum Entropy classifier, sometimes called Multinomial Logistic Regression [45], is also used for the text classification task. Tweets related to illness were identified using Maximum Entropy [46]. Another study that used Maximum Entropy for tweet classification includes [40].

3.6. Decision tree

Decision Tree is a flowchart-like tree structure, where each non-leaf node represents a test on an attribute, each branch denotes an outcome of the test, and each terminal node holds a class label. Attributes values of an unlabeled sample, X, are tested against the decision tree to predict its class. A unique path is traced from root(topmost node) to a terminal node based on attributes' values, which holds the predicted class for the unlabeled sample [47], [27]. DTs are easy to assimilate and have good accuracy. They can handle real-valued items, categorical features items, and items with a mixture of both. They are flexible enough to handle items with some missing features. Unfortunately, decision trees are poor at handling changes as a minor change in input data may lead to massive changes in the constructed tree. They are good at naturally supporting classification problems with more than two classes and capable of handling regression problems. Finally, once constructed, new items can be classified quickly.

J-48-Decision Trees classifier performed well in predicting positive and negative tweets related to personal health experience [33]. Similarly, R. A. Calix and A. General achieved an average result using Decision Trees classifiers for classifying Personal health experience tweets [23]. Even, DT classifier was experimented for distinguishing tweets related to swine flu [25].

3.7. Deep Neural Network:

A standard neural network (NN) consists of many simple, connected neurons, whereas Deep Neural Network(DNN) employs a deep architecture in NNs with a certain level of complexity and an increased number of layers in a single layer [48], [49]. In the past few years, Deep Neural Networks (DNNs) have gained much popularity in text classification. DNN classifiers outperform every other conventional classifier experimented such as IB1-k-Nearest Neighbor, J48-Decision Tree, LR, and SVM when tweets were classified as personal and non– personal health experience tweets [33]. CNNs have achieved remarkable performance in computer vision and deep learning. It is a class of Neural Network that is proven very useful in the areas of text processing, image recognition, and classification. Recently, CNNs are actively exploited for text classification in the health domain. J. Du, L. Tang, Y. Xiang et al. have also used different types of DNN, i.e., Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (Bi-LSTM), along with other machine learning approaches for measles-related tweet classification tasks, where CNN has shown a remarkable performance [50]. To study the prediction of chickenpox and eliminate delays in disease reporting of existing surveillance systems, S. Chae, S. Kwon and D. Lee [51] optimized parameters of DNN and LSTM (a special kind of RNN) algorithms.

3.8. Random forest

Random Forest is an ensemble learner which improves the accuracy of the model by combining a collection of decision tree classifiers(forest) to generate the aggregated result. It uses classification and regression trees (CART) methodology to grow the trees. At each node, attributes are randomly selected to determine the split and generate individual decision trees. The values of the random vector sampled are responsible for determining each tree. During classification, votes are cast by each tree, and the class with maximum votes is returned [52]. RF can handle multidimensional data and is capable of estimating missing data. It also considers the importance of the variables used in classification. RFs consider many fewer attributes for each split, and they are efficient on vast databases. Random Forest approach is used along with other conventional machine learning approaches for social media text classification, such as [21], [23], [24], [25].

Some authors have explored other popular machine learning approaches for text mining, such as k-means [30], [35], clustering [13], etc. Dai X, Bikdash M and Meyer B. [13] proposed a word embedding based clustering technique to classify health-related tweets. A tweet can be grouped on the grounds of similar words and can be classified based on the similarity measure. They compared their proposed method with Naive Bayes classifier and found the former is superior to the Naive Bayes method. Topic modeling, a well-known algorithm of semantic clustering, has indicated a valuable outcome for classification [24].

4. RQ2: popular social media data sources for data collection

Since there is a considerable increase in social media users to share information, there is significant traction among researchers to analyze social media activities for public health purposes. Also, social media can cover various topics in addition to those covered by traditional data sources of public-health.

Social media has emerged as a feasible source for health communication [10], [53], [54], [55]. Although some research has cast doubt on whether social media data would have utility for detecting outbreak [56], [46], analysis of social media content for healthcare data has been a topic of broad interest [57], [58], [59], [60]. To track and forecast health events, [61] explains an urgent need for including social media data sources while disseminating epidemic outbreak advisories using their Facebook or Twitter pages to address a more extensive public base in no time. Hence, social media posts and online search behavior could be useful sources of information about health outbreaks.

4.1. Twitter

Twitter is one of the leading micro-blogging services, where registered users can post tweets or retweet other posts that can be read by unregistered users. With over 300 million monthly active users, Twitter has become a reliable and fast source to evaluate the incidence of diseases in a population. H. Kwak, C. Lee, H. Park et al. [62] conducted a study to examine the potential of Twitter as a new source of information sharing. As social media postings from Twitter have become a reliable and fast source to evaluate the incidence of diseases in a population, effective and efficient methods must be developed to process and examine health-related tweets. Usually, dimensions like location, volume, time [63], and public perceptions are considered for disease surveillance. In a recent work [37], data collected from Twitter was utilized to find information during different epidemics that were useful for various health organizations. C. Bosley et al. used a set of seven terms to collect more than 60 thousand tweets and then examine and classify them, focusing on cardiac arrest and resuscitation [64]. For early detection of influenza activity, [22] proposed a model that uses real-time Twitter data streams and U.S. Centers for Disease Control and Prevention (CDC) historical datasets to foresee future influenza activities. Lee K, Agrawal A, Choudhary A. [21] examined the allergy activities. In their work, they collected tweets that mentioned allergy-related tweets. A natural language processing approach is adopted [65] to reach Ebola-related tweets considering four main topics based on clusters of keywords: risk factors, prevention education, disease trends, and compassion. An unsupervised method was used [13] to partition vectors represented tweets into clusters of similar words. The tweet is then classified as disease-related or unrelated based on the clusters' similarity measures. N. El-bathy, C. Gloster, M. El-bathy et al. [66] proposed a surveillance lifecycle architecture using a novel genetic algorithm to get relevant data from the large set of online data accessible faster at a lower cost. Health problems such as respiratory, gastrointestinal, heat-related illness, and ILI symptoms circulating among the population during mass gathering were also detected using Twitter [67]. The use of Twitter data has been done to examine a variety of public health incidences such as allergy:[20], mosquito-borne disease: [24], dengue: [29], [68], flu/influenza: [13], [69], [70], [71], [38], [72], [73], H1N1: [25], [74], [75], [76] and other disease [77], [78], [79], [80], [81], [82], [83], [84], [85].

4.2. Instagram:

Instagram is a photo and video-sharing service founded in 2010, with over 800 million registered users [86]. Guidry JPD, Jin Y, Orr CA et al. [87] examined the Ebola-related social media posts on Twitter and Instagram, and results suggested that Instagram can be an optimal platform for communication and reaching the public in times of worldwide health crises. Studies have also examined Zika-focused messages on Instagram [88]. As Instagram is a photo and video sharing platform, the image and video data available at this platform can be a promising source for disease surveillance.

4.3. Crowdsourcing

Crowdsourcing is a process in which a large, undefined group of volunteers or part-time workers are involved to provide services, ideas, or content through a flexible open call [89]. Usually, a large group of people with varying degrees of knowledge and experience are involved in contributing to a common goal. Most well-known methods of crowdsourcing include Google Consumer Surveys, Amazon Mechanical Turk, and proprietary websites. N. EO. , K. SA. and B. JS [90] studied the impact of reviews on foodservice on Yelp.com, a business review site on foodborne illness surveillance efforts, and observed that sickness reports generated online could complement traditional surveillance systems. M. O. Lwin, S. Vijayakumar, O. Noel et al. [91] have collected the crowdsourced information regarding mosquito bites, symptoms, and suspected mosquito breeding sites and reported the same to help health authorities by early warning them of dengue outbreaks. A tool combining the use of crowdsourcing and text classification technique was proposed to track misinformation about the Zika virus on Twitter [92], [93].

4.4. Other microblogs

A microblog represents a stream of text written by an individual comprising periodic and brief updates presented to the readers in reverse-chronological order. Compared with other social media, the information on microblogs is transferred in a truncated manner as the length of the post is limited. The short posts reduce users' time, and on an average, a microblogger can post many updates in a day. Various popular microblog services other than Twitter also exist. For example, Tumblr, Reddit, Sina Weibo, Pinterest, etc. N. Yang, X. Cui, C. Hu C et al. [30] analyzed the content of a famous Chinese social medium (Sina micro-blog) and then used them to predict the flu outbreak in a region of Beijing. To forecast seasonal flu, Z. Ertem, D. Raymond and L.A. Meyers collected data from WordPress flu-related blogs (WordPress Flu), along with other online data sources such as Wikipedia and Twitter [94]. Most of the papers that have collected data from microblogs, other than Twitter, used Chinese microblogs such as [95] (Sina), [96](Sina), [97] (Sina, Tencent), [98] (Sina).

4.5. Internet search query

One of the most well-known sources of data for Internet-based surveillance in the field of healthcare is Internet Search Query analysis, especially Google Trends (GT). Google Flu Trends was one of the earliest examples, which started providing real-time data to the public in 2008. It observed flu outbreaks around the world, based on flu-related terms people searched on the Internet. It hailed in providing the data one to two weeks faster and almost as precise as the CDC's [99]. J.D. Sharpe, R.S. Hopkins, R.L. Cook et al. compared the predictive performance of Google, Wikipedia, and Twitter-based surveillance with each other. As a result, Google Flu Trends showed superiority in sensitivity rates and positive predictive values [100].

Interestingly, online search behaviour successfully predicted the sudden increment in asthma-related emergency visits [101]. Apart from using Twitter data, this paper has combined internet users' search interests from Google with the environmental data to collect information on asthma-related visits. Google Trends' data has also been utilized to track infectious diseases such as tuberculosis [102], influenza [103], [104], [105]. Some studies considered web-based search queries [56], [106], [107] to analyze and predict the dissemination of an outbreak.

Some studies have searched other social media platforms for data collection. For example, S. Chaudhary and S. Naaz have accessed the digital health data for various diseases from Facebook pages of Practo, National Portal of India, and the Integrated Disease Surveillance Program (IDSP) [61]. Y. A. Strekalova [108] studied the Facebook users' characteristics, commenting about the Ebola outbreak on CDC's Facebook posts for more than seven months. S. Gittelman, V. Lange, C. Crawford et al. [109] employed Facebook “likes”, which proved to be an effective predictor of mortality, diseases, and lifestyle behaviors. YouTube is another popular source for analyzing videos on various health topics such as Ebola virus disease [110]. Another study analyzed public responses from YouTube videos related to the Zika virus to determine the video content [111]. S. Choi, J. Lee, M. Kang et al. [112] have mentioned that short-text comments on news articles are top-rated in Korea and behaves as a platform to express personal emotions and thoughts. They studied the public emotions using the comments on these news articles about the Middle East Respiratory Syndrome (MERS) outbreak.

Fig. 4 represents the most searched social media sources for data collection by the selected research articles, and it can be easily noted that Twitter is the most popular social media source for health-related data collection.

Fig. 4.

Types of social media platforms searched for health-related data collection.

Fig. 5 shows the share of the health topic or diseases studied by the authors of the selected research papers with ILI or flu as the most common disease for research related experiments.

Fig. 5.

Categorization of most common diseases on social media platforms.

Table 1 represents the machine learning techniques adopted for classification of health-related text and also the social media sources that were used along with corresponding research papers from the set of the selected relevant articles.

Table 1.

Summary of ML Classification Approaches used in Health Surveillance Systems based on Social Media.

Health Topic	ML Approach*	Social Media Data Source	Article Citation	Year
Chickenpox	DNN, RNN	Internet Search Query	[51]	2018
Measles	CNN, RNN, SVM, k-NN, NB, RF	Twitter	[50]	2018
Mosquito-borne diseases (Dengue, Zika, Chikungunya)	SVM, NB	Twitter	[24]	2018
Dengue	SVM, k-means	Twitter	[35]	2018
Personal-Health Experience	SVM, LR, k- NN, DT, DNN	Twitter	[23]	2017
MERS, Ebola	SVM,NB.LR	Twitter	[37]	2017
Personal-Health Experience	k- NN, DT, LR, SVM, DNN	Twitter	[33]	2017
Mosquito-borne diseases (Dengue, Malaria, Chikungunya)	SVM, NB	Twitter	[28]	2017
Flu	NB	Twitter	[13]	2017
Dengue, Typhoid	SVM	Twitter	[29]	2017
Influenza	ME	Twitter	[46]	2016
Dengue	SVM,NB,ME	Twitter	[40]	2016
Flu	SVM	Twitter	[38]	2016
Asthma	SVM, LR	Twitter	[32]	2016
Allergy	SVM, NB, NBM, k- NN	Twitter	[20]	2016
Influenza	SVM	Twitter	[34]	2016
Health-Related	SVM NB	Twitter	[39]	2016
Flu/Influenza, H1N1	LR	Twitter	[44]	2016
Influenza-A (H1N1)	NB, SVM,RF, DT	Twitter	[25]	2015
Allergy	NBM, NB, RF, SVM	Twitter	[21]	2015
Asthma	Artificial Neural Network (ANN)	Twitter	[101]	2015
ILI	SVM	Twitter	[73]	2014
Flu	k-means, k-NN, SVM	Sina Microblog	[30]	2014
Influenza	SVM	Twitter	[72]	2013
Personal health	NB, NBM, SVM	Twitter	[36]	2013
Health Related	SVM,NB,NBM,RF	Twitter	[31]	2013

*Abbreviations are defined in the body of the paper

5. RQ3: applications of social media based surveillance systems

This section discusses the various recent applications of surveillance systems in the area of health informatics. These include disease prediction, tracking misinformation, global awareness, etc.

5.1. Syndromic surveillance-based disease prediction

Syndromic surveillance has emerged as a potential tool to predict outbreaks for public health purposes through data gathered from various sources before clinically confirmed data is available. The desired result is to minimize the spread in the population and take preventive measures. In the past few years, social media information has been widely used to estimate disease incidences and to detect disease outbreaks. Such data would be beneficial for public health officials in earlier detection of outbreaks than the traditional methods. Usually, the data is in the form of self-reported symptoms. Studies have shown that surveillance systems in the healthcare domain can be used to predict the diseases for public health concerns. Twitter data was used as a tool for early warning and outbreak detection, such as to predict syphilis [113], swine flu [75], tuberculosis [102], flu [78], and Ebola [56]. Another study that was able to examine disease incidences of dengue and typhoid fever in a region was suggested by the Philippines [29]. Similarly, there is another study [41] where surveillance systems have used social media data for disease detection.

5.2. Magnitude estimation of disease over some time

Surveillance systems can be used to estimate the magnitude of the problem. The planning, resource allocation, treatments, and prevention can be done by estimating the future of disease levels. The analysis provided by the surveillance systems can be useful to determine the level of the disease over some time, and assessments can be made accordingly.

5.3. Event-based surveillance and disease prediction

Event-based surveillance involves fast capturing of data in an organized manner about events that are at potential risk to public health. The data can be from diverse Internet sources such as media reports, online discussion platforms, routine reporting systems, personal information, or rumors. In the web forum context, an event is characterized as excessive news postings. The significance of the event can be considered proportional to the number of postings about it. Therefore, the event effect can be identified on topic diffusion from the amount of posting on the topic. When the number of postings on a topic surpasses the usual number of postings, we can expect that there is an event at that time. The well-known examples of event-based surveillance systems are HealthMap, a system that tracks news related to health events, EpiSPIDER, etc. The use of social media-based public health intelligence monitoring techniques to provide situational awareness of potential health threats to support surveillance activities has grown tremendously over the last few decades [58], [14]. There is a study that analyzed the Zika virus epidemic using Twitter corpus [114]. N. Thapen, D. Simmie, C. Hankin et al. [39] proposed DEFENDER, a software system with potential health events detection functionality that monitors Twitter stream and then outputs the generated events to the users in Front-end UI.

5.4. Analyze user's reactions to health events:

Surveillance systems in the field of healthcare can be used to measure social media users' reactions to health promotion messages or events. T. Tran and K. Lee [115] have gathered 2 billion tweets in 90 languages from Twitter between August 2014 and December 2014 to understand citizen reactions towards Ebola by extracting geotagged Ebola-related tweets. Another study collected Twitter data to express emotions during different stages of an outgoing health event [50]. S. Choi, J. Lee, M. Kang et al. [112] analyzed the relationship between mass media and emotional public reactions during a nation-wide outbreak of MERS in 2015 in Korea. E. K. Seltzer, E. Horst-Martz, M. Lu et al. [88] proposed that public sentiments can be described using Instagram and also highlighted areas of concern for public health. Another study [116] that reflected the interests of Twitter users to share Zika virus news such as symptoms, stories of Zika infected pregnant women, and worries of parents to be.

Moreover, R. Gaspar, S. Gorjão, B. Seibt et al. have shown that, during the EHEC/Escherichia coli bacteria outbreak in Europe in 2011, amplification of public reactions over Twitter has a severe political and economic impact [117]. K. Liu, L. Li, T. Jiang et al. analyzed variations in responses to different disease outbreaks by analyzing the public's search behavior [95]. In another study, I.C. Fung, K. Fu, Y. Ying et al. gained insight into Chinese people's reactions to different outbreaks by analyzing social media data [96].

5.5. Global awareness of events

Once the event has been detected, surveillance systems can be used to monitor the general public's awareness and perception towards health events. User-generated sentiments on social media platforms towards an outbreak situation reflect their knowledge, attitudes, and perception. Social media allow the sharing of public sentiments, opinions, and responses during outbreaks [118]. The high uncertainty related to Zika may not only lead individuals to social media to search for essential information [119], but also resulted in creating and spreading conspiracy theories.

5.6. Tracking misinformation

The capability of social media to provide the information in real-time also allows the fast flow of misinformation among people during an ongoing epidemic that can have severe reactions. However, in our knowledge, limited research has been done to detect the spread of misinformation about such events, yet few researchers have worked on widespread misinformation posted on social media platforms [98]. Ghenai A, Mejova Y. [92] have used machine learning techniques to track misinformation regarding the Zika virus on the Twitter platform. Also, another study [12] examined ways to correct misinformation about the measles vaccination by studying the reactions of two social media groups. Later, they also examined the effect of correcting the misinformation on the users who reacted to measles vaccination. There is a study [120] that presented an analysis of hoax medical news in social media. Another study based on YouTube data indicated that the spreading of both informational and conspiracy theory took place in a similar manner [111], which was a striking result for health organizations. To counter the spread of such misinformation, more attention is required towards the content posted on YouTube or any other online platform. Another study analyzed 1680 microblog posts and found that more than 20% of posts were misleading messages [97]. To fill the gap on how health organizations should get a response and correct misinformation, E. Hagg, V. S. Dahinten, and L. M. Currie [59] discussed that the potential for misinformation could be a barrier to use social media data in the area of healthcare.

5.7. Uncovering the topics popular during the outbreak

One more application of health-related surveillance systems can be to extract the topics popular during the outbreak. The data collected for the surveillance purpose can be used to obtain multiple concurrent events during the outbreak [121]. Even different topics, including government/politics and economy, were also captured to analyze public reactions during the MERS outbreak in Korea in 2015 [112]. K. Rudra, A. Sharma, N. Ganguly et al. mentioned that during epidemics, opinions, and sentiments related information is mostly contained in the non-disease tweets (tweets that do not convey any information about the epidemic situation) [37]. Similarly, data collected for public health surveillance purposes can be utilized to uncover ongoing topics during health events.

6. RQ4: limitations and challenges of social media based surveillance systems

This section aims to highlight some of the limitations and challenges faced in using social media data by surveillance systems.

While the surveillance systems based on social media data for outbreak detection or health events has led to the advancement in early detection of epidemics and associated events, still some studies have challenged the outcome of these surveillance systems for some of the following reasons:

6.1. Noise

One of the challenges faced while collecting the data is Noise. The data collected from social media sites may contain data that are irrelevant to the task. Such data referring to illness terms have no relation to health. For instance, a large number of activities related to flu may be caused by posts containing the term “Irish Flu.” Unfortunately, sometimes the user may post a status, and they are suspected to be infected when they are not. In this manner, such deceptive information can influence disease management for the public health department. To prevent such noise in data essential text processing techniques such as feature weighting, tokenization, stemming, frequency-based methods, etc. are available, still more training would be to get the relevant data for further analysis.

6.2. Data validation

Another challenge associated with social media data is its validation. The use of unofficial data from social media may lead to the problem of standardization, verification, and control [122]. T. Bodnar and M. Salathé focused on validating the ground truth associated with a large amount of heterogeneous social media data [123]. The dataset is one of the most critical components while creating prediction models. The outcomes of the prediction models highly depend on the dataset. The dataset for the prediction models includes historical information, training data, and testing data. A large amount of training data is required to train the forecasting models and testing data for valid testing of predictions based on the training of models. Hence, the correctness of data is essential as much as the volume and quality of data from reliable sources [124].

6.3. Low confidence

One more challenge that arises with social media-based data is low confidence. There is a study [125] showing that government websites are a more trustworthy resource for obtaining vaccination information than social media. Online data related to health is of varying quality. Many social media data analysis methods may show spikes showing something noticeable is going on, but that can be a reflection of panic and not the real incidences of a disease outbreak. Also, users may post that they have flu when they have a common cold, or people might talk about disease due to increased media hype. There is a study presenting conspiracy about the Zika virus outbreak on Reddit during a public health crisis [126]. Such behaviors are the reasons behind the low confidence associated with online data. To prevent this issue, more training would be required as an approach to modify the classification algorithms.

6.4. Demographic bias

Although information such as location, username, number of followers, user profiles, etc. can be collected along with the tweet content, demographic information such as age, gender, and race are hard to figure out through tweets, making it difficult to determine who is posting about the outbreak and to whom public health efforts should be directed. Among the very few studies which have analyzed the users' profile [108], studied the Facebook comments in terms of sex and found that men composed a greater number of posts per person than women. Another bias is that younger people are the most common users of social media. Also, the bias is strongly supported by the observation that social media data are skewed towards active users who are often young adults, well-educated, and persons having a good income [40]. This paper used machine learning techniques to support that sociodemographic factors have a significant role in reporting disease-related posts on social media. Therefore, it is considered that social media users do not represent the whole population [127]. Any outcome based on social media data excludes people who do not use such platforms, who are probably going to be the most vulnerable in population, and even those who often do not want to share their health experiences publicly. Also, it can be speculated that those who are in discomfort, sick, elderly, or disabled would be less likely to be active users.

Language is another bias due to the demographics of a population. Most studies focus on a single language (English) even though the epicenter of the epidemic/event is a country where English is not an official language [128]. Very few studies have exhibited research relying on languages other than English to deal with disease tracking. For instance, M. U. Ilyas [129] has geographically filtered disease names in tweets both in Arabic and English language. Moreover, the author observed that tweets in English were in a very small fraction. Their approach was verified by having a high degree of correlation between the actual occurrence of MERS-Coronavirus cases and disease-related tweets in the Arabic language. Another study [83] has shown that there might be a difference in the reactions of people belonging to different linguo-cultural backgrounds towards the same outbreak.

6.5. Privacy issues

There is also discussion about ethical concerns while retrieving data from social media [130], including the privacy of the datasets collected using social media for health purposes [131]. Although there is the public availability of social media data, users may not want their posts or data to be used for research. Social media platform hosts must consider users' privacy expectations, such as diagnoses from public data or concern over algorithms that reveal unstated user demographics. Only a few studies [130], [132] have shown interest so far. And hence, there is a broader scope for considering privacy while performing research on social media data.

6.6. Lexical and linguistic variability

Though communication over social media helps extract healthcare information, it is challenging in terms of semantic interpretation of the language. In particular, social media texts are informal and ambiguous, and hence, just matching keywords may not capture their semantic interpretation, resulting in achieving the incomplete result. N. Limsopatham and N. Collier [133] have discussed and researched this limitation.

7. Original contributions of the paper

Our paper has the following contributions that have not been considered carefully in the previous papers:

• •Provides the article selection queries from different digital libraries databases for selecting the relevant articles.
• •Presents an overview of popular ML classification algorithms in the context of social media based surveillance systems in the healthcare domain.
• •Statistical analysis of most scrutinized social media platforms and health topics studied by the selected articles.
• •Provides a summarized view of the most used online participatory media platforms for data collection, along with the publication year of the research paper, health topic studied, and machine learning methods used for analysis.

8. Findings, conclusions, and future directions

Among all the types of social media platforms studied by the selected articles, Fig. 4 has clearly shown that Twitter (64%) was undoubtedly the most searched platform. Also, SVM (33%) was the most used classification technique among all the recently used machine learning based classifiers. In addition, it was observed that SVM was the most promising classifier when the data needs to be classified between two classes.

This paper aims to study the latest trends in surveillance systems using social media and machine learning algorithms in the area of public health. We have noted that in comparison to traditional surveillance systems, social media based surveillance systems show superiority. Also, we have discussed the applications of using dynamic natured social media data for the advancement of surveillance systems in the area of public health. Also, we can expect the challenges mentioned in this paper will lead to the development of new technologies, new capabilities for public health research.

Some of the future works that we have observed can be:

• (1)The conjunction of online data with other physical conditions like weather conditions, demographic information, etc. to get better prediction results.
• (2)Incorporating the input features such as sentiment content, comments, locations, etc. of users' posts along with the text content for the better analysis and prediction of health events and other related events.
• (3)Classification of users' posts into different categories such as health, news, ads, etc. and assigning different weights to different categories so that forecasting accuracy can be improved.
• (4)The text analysis and image analysis approach can be extended to video content analysis to enhance the disease prediction performance.
• (5)We observed there is a wide scope of improvement in the area of topic modeling to get more accurate results.
• (6)Implementation of predictive models that can use various social media platforms simultaneously to get accurate and timely predictions of epidemics.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.