Acute respiratory infections prediction through social media data and machine learning: a systematic review

Abstract

Background

Acute respiratory infections (ARIs) represent a major global public health burden, requiring timely surveillance and early detection to mitigate their impact. Traditional epidemiological monitoring systems often suffer from reporting delays, motivating the exploration of alternative data sources such as social media combined with machine learning techniques.

Methods

This study presents a systematic review of the literature on ARI prediction using social media data and machine learning models. Relevant studies were identified through structured searches of major scientific databases following established systematic review guidelines, PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). The selected studies were classified into four levels of complexity and subsequently analyzed in terms of data sources, feature extraction strategies, machine learning algorithms, evaluation metrics, and prediction objectives.

Results

The reviewed studies demonstrate that social media platforms, particularly Twitter (now X), can provide valuable signals correlated with ARI incidence. A wide range of machine learning methods have been employed, including regression models, support vector machines, ensemble methods, and deep learning approaches. Overall, the results indicate that machine learning models leveraging social media data can achieve competitive predictive performance, often complementing or enhancing traditional surveillance systems. However, challenges related to data noise, population bias, and model generalization remain.

Conclusions

The findings highlight the potential of integrating social media data and machine learning techniques for ARI prediction and public health surveillance. While promising, future research should focus on improving data quality, model interpretability, and robustness, as well as on validating these approaches across different geographic regions and respiratory diseases.

Supplementary information

The online version contains supplementary material available at 10.1186/s12911-026-03390-8.

Background

Infectious diseases contribute significantly to the global burden of illness. Moreover, epidemic outbreaks can lead to substantial economic and social consequences, as demonstrated by the COVID-19 pandemic caused by the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2).

Acute respiratory infections (ARI) comprise a group of disorders ranging from mild (such as the common cold and pharyngitis) to severe illness (such as bronchiolitis and pneumonia)  [1]. These are caused by a wide array of microorganisms including viruses (for example, influenza, SARS-CoV-2, and respiratory syncytial virus), bacteria (for example, Streptococcus pneumoniae, Streptococcus pyogenes, Haemophilus influenzae, and Moraxella catarrhalis), and fungi. COVID-19 is a term used to define infections that are caused specifically by SARS-CoV-2. While some patients with SARS-CoV-2 infection (i.e. COVID-19) may present with non-respiratory symptoms, most COVID-19 cases are a subset of ARIs  [2, 3].

As such, ARI encompass diseases affecting diverse sites within the respiratory system. According to the International Statistical Classification of Diseases and Related Health Problems, 10 Revision (ICD-10) by the World Health Organization (WHO), respiratory diseases are classified under codes J00–J99, which cover 356 distinct conditions. Within this group of disorders, those caused by respiratory pathogens are considered ARI. Additionally, COVID-19 is included under codes U07.0 and U07.1. The complete ICD-10 classification can be consulted at: https://icd.who.int/browse10/2019/en.

Traditional methods for monitoring the spread of infectious diseases rely on Indicator-Based Surveillance (IBS) systems. These systems analyze structured data gathered through surveillance and monitoring protocols tailored to specific diseases. The standardized nature of IBS allows health authorities to systematically record information and track trends over time. However, their dependence on case confirmation, often requiring laboratory testing, and the need for proactive identification limit the volume of data that can be collected, especially under budgetary constraints  [4]. Another limitation of these systems is the time lag between data collection and publication of official reports. Furthermore, IBS systems typically lack the ability to detect outbreaks caused by newly emerging pathogens  [4].

To address some of these challenges, Event-Based Surveillance (EBS) systems have been developed as a complementary approach. According to the World Health Organization, EBS involves the organized and rapid capture of information regarding events that may pose a threat to public health  [5]. Unlike IBS, EBS collects data in real time directly from observers, using diverse sources such as social media, news outlets, and public health networks. Although the data in EBS systems is often unstructured, noisy, and harder to verify, these systems offer important advantages: they are cost-effective and capable of quickly identifying potentially hazardous events, even in areas lacking traditional surveillance infrastructure. As a result, EBS can detect outbreaks and emerging threats in real time, which might go unnoticed by conventional systems  [4].

On the other hand, in a world where 59% of the population has internet access and 49% are active users of social media, the information generated by individuals represents a valuable resource for the welfare of society. In the context of this work, such welfare is reflected in the enhanced surveillance of respiratory infectious disease outbreaks. As of early 2020, over 4.5 billion people were using the internet, and social media users had surpassed 3.8 billion. Nearly 60% of the global population was online. The continuous expansion of social media usage ensures a growing volume of data  [6].

Therefore, the steady increase in the use of social media has led researchers worldwide to assess the potential use of publicly available information on these platforms for epidemiological studies. Systematic reviews consistently identify Twitter (now known as X) as the most widely used source in studies exploring the intersection of social media and public health  [7, 8]. Among the health conditions studied most frequently, ARI (including influenza and influenza-like illness (ILI)) stand out due to their high relevance and data availability  [7].

However, the application of social media data in research varies widely. Some studies focus on sentiment analysis, others examine the volume of posts on trending topics, and some analyze visual content such as images  [9]. Notably, previous systematic reviews that incorporate machine learning (ML) for monitoring social media activity tend to refer to ML techniques in a general sense and do not assess the complexity of the models applied  [10].

The goal of this work is to examine recent research that uses ML techniques to identify and quantify messages posted on Twitter related to infectious respiratory diseases. Taking into account that the period of time used to collect data and the association measures used influence the precision of ML techniques, the research questions related to our objective are the following:

1. What analysis methodologies are used in studies related to the use of social networks for the prediction of acute respiratory infections (ARI)?

2. What are the main ML techniques used in investigations related to the extraction of information on Twitter?

3. What metrics are used to measure the performance of the ML techniques?

4. What are the main countries where studies related to the extraction of information on Twitter with ML techniques are carried out?

5. In which time periods have the largest number of studies related to our topic been published?

First, we investigate the analysis methodologies used across studies to understand the approaches applied in processing social media data to create prediction models for ARI. Second, we identify the ML techniques used most commonly to extract information from Tweets. Third, we examine the performance metrics used to evaluate the effectiveness of these methodologies and ML models, aiming to assess the rigor and comparability of results across studies. Additionally, we explore the geographic distribution of the research, highlighting the countries that have contributed the most to the development of this field. Finally, we analyze the temporal trends in publication, identifying the periods during which the highest volume of relevant studies has emerged, thus providing insight into how interest in this topic has evolved over time.

Methods

The methodology used in the development of this systematic literature review was the PRISMA Statement for Reporting Systematic Reviews, developed by Moher et al. [11]. This methodology, which is based on research question, allows structuring a literature review in an orderly manner, and presenting the results under certain predefined criteria.

The results obtained from the broad search are not solely statistical in nature (quantitative); rather, the in-depth analysis of the selected studies (qualitative) constitutes the primary contribution of this systematic review. We categorize the studies according to the algorithms used in their mathematical models, considering both the specific task addressed (classification, regression, or clustering) and the complexity of the models in relation to how they process information from social media. In addition, we report the performance metrics and highlight the best outcomes presented in the reviewed literature.

In summary, the PRISMA methodology is important for systematic reviews because it promotes standardization, transparency, comprehensive reporting, bias reduction, enhanced quality, ease of peer review, increased credibility, and supports evidence-based decision-making. It has ample acceptance in studies related to health issues  [7, 12, 13].

Search strategy

A search for scientific literature was conducted using the EBSCO Discovery Service (EDS) database, covering publications from January 2006 to December 2020. The starting point was selected based on the creation of Twitter in 2006, while the end date was chosen to exclude the exponential surge in COVID-related publications between 2021 and 2023. To validate our findings and ensure relevance, results from 2021 to 2024, particularly from systematic reviews, are discussed in the Discussion section. Additionally, a filter was applied to include only peer-reviewed publications. Table  1 outlines the search terms used, organized into three thematic sections; each selected article was required to include at least one term from each section. The EDS platform also provided metadata attributes for each article, which are described below:

1. Article Title

2. Author

3. Journal Title

4. ISSN

5. ISBN

6. Publication Date

7. Volume

8. Issue

9. First Page

10. Page Count

11. Accession Number

12. DOI

Table 1.

Search terms applied on the EDS database, including title and abstract during the search

Terms related to diseases	Twitter related terms	Terms related to prediction models
influenza OR coronavirus OR	social media OR twitter	forecasting OR
SARS OR epidemic OR	OR tweets	machine learning OR
healthcare OR disease OR		predict OR deep
outbreak OR surveillance OR		learning OR predict OR
epidemiology OR pandemic		predicting OR
OR influenza-like illness OR		classification OR
pneumonia OR acute		recognition OR
respiratory infection OR		artificial intelligence
COVID OR public health OR		OR classifiers
surveillance systems OR flu
OR ILI OR SARS-CoV-2 OR
COVID-19

The databases consulted by EDS are listed in Table  2.

Table 2.

Total articles () identified in the search and classified by content provider

Content Provider	Number of Records
MEDLINE	527
Complementary Index	395
Academic Search Ultimate	282
Directory of Open Access Journals	176
IEEE Xplore Digital Library	163
Supplemental Index	125
ScienceDirect	100
Library, Information Science & Technology Abstracts with Full Text	98
Springer Nature Journals	80
Business Source Complete	55
Emerald Insight	8
ERIC	6
JSTOR Journals	5
GreenFILE	4
Dentistry & Oral Sciences Source	4
SciELO	2
SciTech Connect	1
Academic Source	1

Study selection

For the study selection process, the titles of the articles were first reviewed, and the following were excluded: (i) duplicate entries, (ii) articles published in a language other than English, and (iii) systematic reviews. After identifying relevant titles, the abstracts were examined, and studies were excluded if they were not related to (i) human subjects, (ii) social networks, or (iii) respiratory diseases or prediction models.

Only peer-reviewed publications, such as journal articles and conference papers, were included. Informal sources, including technical reports, handouts, and non–peer-reviewed content, were excluded from the review. Additionally, articles focused solely on sentiment analysis were removed, as the measurement of people’s emotions falls outside the scope of this research. To minimize bias, three reviewers independently carried out the study selection.

Data collection process

For each selected article, we extracted key information such as the authors, year of publication, predictive models used, evaluation metrics and outcomes, as well as the time period, geographic region, and the specific respiratory disease examined in the study.

Due to the diverse approaches available for classifying methodologies and algorithms, we adopted the framework proposed by Dai et al. [14] to address the first two research questions, as it is specifically designed for the context of public health and social media data. This framework defines the following categories:

1. Learning-based Approaches. These methods require labeled data for training, after which a ML classifier is used to predict whether a tweet is related to ARI. Some of the most commonly used techniques include Naive Bayes, KNN and SVM.

2. Lexicon-Based Approaches. This is a knowledge-based unsupervised learning approach that does not require annotated labels for training; instead, it leverages domain-specific dictionaries to assign labels based on predefined knowledge.

3. Word Embedding Based Approach. This is an unsupervised learning method that does not require annotated data; it learns optimal vector representations from the context of surrounding words. A tweet can be represented by a set of such vectors and grouped into clusters of semantically similar words, capturing the underlying meaning of the text.

4. Keywords-based Approaches. This methodology establishes a correlation between the number of respiratory infection–related keywords identified and the number of confirmed cases reported.

Results

The results are organized into two sections: Study Selection and Study Characteristics. The Study Selection section outlines the process by which articles were identified and included in this systematic review, while Study Characteristics presents the findings corresponding to each research question outlined in Section 1.

Study selection

The search yielded 2032 records, with their distribution by content provider shown in Table  2. After removing 1034 duplicates, the titles and abstracts of the remaining 998 articles were reviewed. This screening involved verifying whether each study addressed: i) the social network Twitter, ii) respiratory diseases, and iii) machine learning techniques or prediction models. Articles focusing solely on sentiment or emotion analysis of tweets were excluded. As a result, 883 articles were discarded because they did not meet the aforementioned criteria, along with 9 non-English publications and 12 systematic literature reviews. This left 94 articles for full-text review, where the same three inclusion criteria were re-evaluated. Ultimately, 46 articles met the eligibility requirements. Details of the reviewed literature are provided in the Appendix.

Figure 1 summarizes the whole process using the PRISMA flow diagram.

Fig. 1.

PRISMA flow diagram for the selection of literature reviewed

Study characteristics

Based on the data extracted from each article, tables and graphs were generated to address the research questions. At this stage, the grouped information was analyzed to provide answers to each question.

What analysis methodologies are used in studies related to the use of social networks for the prediction of acute respiratory infections (ARI)?

To address the first research question, the methodologies were grouped into four categories, following the classification proposed by Dai et al. [14]. The most represented category was the Learning-based Approach, accounting for 20 out of 46 studies, followed by the Keywords-based Approach with 15 out of 46, as shown in Table  3. The years with the highest number of Learning-based studies were 2015 and 2018, each with four publications. In the Keywords-based category, 2018 stands out with the highest number of studies (five).

Table 3.

Analysis methodologies in social networks

Approach	Number of studies
Learning-bas ed	20
Keywords-based	15
Word embedding-based	9
Lexicon-based	2

What are the main ML techniques used in studies related to the extraction of information on twitter?

To address the second research question, we analyzed the models employed within each of the four methodological categories. The most frequently used ML techniques were found in the Learning-based Approach category. In contrast, the Keywords-based Approach featured the fewest ML techniques, primarily due to the simplicity of its models. While some correlation methods are included in this category, they are not considered machine learning techniques.

Learning-based approaches

Tables  4 and 5 summarize the key characteristics of studies that employed learning-based approaches. Table  4 presents the machine learning technique used and the type of respiratory disease studied, listed in the second and third columns, respectively. Based on the disease of interest included in the studies, ARI were categorized as Flu, Influenza, H1N1, Middle East Respiratory Syndrome (MERS), and COVID-19. The first column indicates the study number out of the 46 studies identified in this systematic review. The fourth column provides the reference citation for each study—note that the study number is not to be confused with the reference number.

Table 4.

Learning-based approaches: techniques and respiratory diseases

Study	Technique	Respiratory Disease	Ref.
1	SVR	H1N1	[15]
2	Linear Regression and SVR	ILI	[16]
3	SVM, Naive Bayes, Random Forest,	ILI	[17]
	Decision Trees and K-Nearest Neighbour
4	SVM, Naive Bayes,	Flu	[18]
	Decision Trees and K-Nearest Neighbour
5	Multilayer Perceptron	Flu	[19]
6	SVM, C4.5 Decision Trees and Naive Bayes	Influenza	[20]
7	Stacked Linear Regression, SVM,	Influenza	[21]
	and AdaBoost with Decision Trees Regression
8	Naive Bayes	Influenza	[22]
9	SVM and Naive Bayes	ILI	[23]
10	SVM	ILI	[24]
11	Least Absolute Shrinkage and Selection Operator	Influenza	[25]
12	SVM, AdaBoost and Long Short Term Memory (LSTM)	ILI	[26]
13	SVM, Naive Bayes, Random Forest	H1N1	[27]
	and Decision Trees
14	SVM, Logistic Regression and Naive Bayes	MERS	[28]
15	Backpropagation Neural Network	ILI	[29]
16	SVM	Influenza	[30]
17	Autoregressive Models, Deep Multilayer	ILI	[31]
	Perceptron and Convolutional Neural Network (CNN)
18	FastText, Random Forest, Naive Bayes,	ILI	[32]
	SVM, C 4.5 Decision Trees, k-Nearest Neighbors (KNN)
	and AdaBoost
19	SVR	ILI	[33]
20	XGBoost, Decision Trees	COVID-19	[34]

Table 5.

Learning-based approaches: region and period

Study	Region	Period Analyzed
1	USA	October 4, 2009 to May 16, 2010
2	USA	September, 2009 to May, 2010
3	Portugal	March, 2011 to February, 2012
4	Portugal and Spain	October 30, 2012 to November 30, 2012
5	USA	January, 2011 to April, 2015
6	Victoria, Australia	May, 2011 to August, 2011
7	USA	2011 to 2013
8	USA	Not Defined
9	UK	February, 2014 to August, 2014
10	Cincinnati, USA	November 1, 2014 to May 1, 2015
11	USA, Brazil, Paraguay,	July, 2012 to May, 2013
	Mexico, and Venezuela
12	31 geolocations (25 in	January, 2011 to December, 2014
	USA and 6 in other countries)
13	India	Not Defined
14	Global	April 27, 2014 to July 16, 2014
15	USA	October, 2016 to Octuber, 2017
16	Japan	November, 2012 to May, 2013;
		November, 2013 to May, 2014;
		November, 2014 to May, 2015
17	USA	2009 to 2010 and 2011 to 2014
18	USA	January, 2018 to May, 2018
19	USA	October, 2016 to October, 2017
20	China	January 22, 2020 to April 13, 2020

Table  5 complements this information by showing the geographical region and the time period analyzed in each of the studies listed in Table  4. In many of the selected studies, more than one ML technique was evaluated and the results of different models were compared.

Table  6 contains the most frequently used ML methods among the reviewed studies. Support Vector Machines (SVM) was the most frequently used method, accounting for 11 out of 37 cases, followed by Naive Bayes with 8 out of 37 cases, and Decision Tree-based approaches, including AdaBoost and XGBoost.

Table 6.

Main ML techniques for learning-based approaches

ML technique	Number of studies
SVM	11
Naive Bayes	8
Decision Tree	6
AdaBoost	3
K-Nearest Neighbour	3
Random Forest	3
SVR	3

While both SVM and SVR are based on kernel methods, SVM was originally developed for classification tasks  [35]. Some studies explicitly use the term SVR when referring to regression tasks, whereas others use the term SVM interchangeably for both classification and regression applications.

Lexicon-based approaches

The analysis corresponding to this classification is presented in Tables  7 and 8. The results include the applied technique, the targeted respiratory disease, the geographical region, and the time period analyzed. Only two studies using this methodology were identified  [36, 37].

Table 7.

Lexicon based approach: techniques and respiratory diseases

Study	Technique	Respiratory Disease	Ref.
21	Algorithm developed by authors	ILI	[36]
22	Conditional Random fields	Influenza, Common Cold	[37]
	and a Log Linear Model	and Listeria

Table 8.

Lexicon based approach: region and period

Study	Region	Period Analyzed
21	USA	January 262,013 to May 6, 2013
22	USA	Not defined

This approach is considered a knowledge-based unsupervised learning method, as it does not require annotated data for training. Instead, it relies on dictionary-based information to assign labels  [14]. Both studies analyzed data from the United States. Velardi et al. [36] developed a custom prediction model, while Nabende et al. [37] employed Conditional Random Fields (CRFs) and a Log-Linear Model.

Word embedding based approach

The results of the studies within this classification are summarized in Table  9 and Table  10, which include details on the technique used, the respiratory disease studied, the region analyzed, and the time period covered. The techniques used most frequently in these studies are shown in Table  11. This analysis focuses specifically on the methods used to convert words into continuous vector representations, excluding techniques applied in other stages of the modeling process.

Table 9.

Word embedding based approach: techniques and respiratory diseases

Study	Technique	Respiratory Disease	Ref.
23	LDA	Flu	[38]
24	LDA	ILI	[39]
25	LDA	Flu	[40]
26	Word2Vec	Influenza	[14]
27	Word2Vec	Swine and Avian Influenza	[41]
	(CBOW)
28	Word2Vec, GloVe	Not Defined	[42]
29	LaBSE, BERT	COVID-19	[43]
30	Biterm Topic Model	COVID-19	[44]
31	K-Means Clustering	COVID-19	[45]

Table 10.

Word embedding based approach: region and period

Study	Region	Period Analyzed
23	15 Countries in South America	December, 2012 to January, 2014
24	USA	May, 2009 to October, 2010
25	South America	December, 2012 to August, 2014
26	USA	Not Defined
27	Global	2009 to 2012
28	Not Defined	July 12, 2018 to July 12, 2019
29	Global	January 4, 2020 to April 5, 2020
30	Global	March 3, 2020 to March 20, 2020
31	Global	January 6, 2020 to April 15, 2020

Table 11.

Main techniques for word embedding based approaches

ML technique	Number of studies
LDA	3
Word2Vec	3
LaBSE	1
BERT	1
Biterm Topic Model	1
GloVe	1
K-Means Clustering	1

Word embedding has become one of the most prominent trends in Natural Language Processing (NLP) [14]. In this category, the two most commonly used techniques were Latent Dirichlet Allocation (LDA) and Word2Vec, each appearing in 3 out of 11 studies. Word2Vec enables the computation of vector representations of words using two main learning algorithms: continuous bag-of-words (CBOW) and continuous skip-gram. CBOW predicts a target word based on its surrounding context, while skip-gram predicts surrounding words from a given target word [46].

Although Table  9 includes three studies that utilized Word2Vec, only one—[47]—explicitly specifies the learning algorithm applied, identifying it as CBOW model.

Keywords-based approaches

The studies classified under the Keywords-based Approach are presented in Tables  12 and 13. As with the other approaches, each study is summarized by its modeling technique, the respiratory disease investigated, the study region, and the time period analyzed.

Table 12.

Keywords-based approaches: techniques and respiratory diseases

Study	Technique	Respiratory Disease	Ref.
32	Autoregressive Model	ILI	[48]
33	Linear Regression Model	ILI	[49]
34	Linear Regression Model	ILI	[50]
35	Linear Regression Model	ILI	[51]
36	Correlation Coefficient	Influenza	[52]
37	Correlation Coefficient	ILI	[53]
38	Autoregressive Model	ILI	[54]
39	Not Defined	Not Defined	[55]
40	Autoregressive Model	ILI	[56]
41	Autoregressive Model	Influenza	[57]
42	Linear Regression Model	Influenza	[58]
43	Correlation Coefficient	ILI	[59]
44	Autoregressive Model	Influenza	[60]
45	Nonlinear Gaussian Process	ILI	[61]
46	Linear Regression Model	COVID-19	[62]

Table 13.

Keywords-based approaches: region and period

Study	Region	Period Analyzed
32	USA	October 18, 2009 to October 31, 2010
33	USA	December, 2011 to April, 2012
34	Korea	October, 2011 and September, 2012
35	New York, USA	October 15, 2012 to May 10, 2013
36	USA	December 2, 2012 to April 7, 2013
37	11 cities in USA	September 29, 2013 to March 1, 2014
38	USA	November 27, 2011 to April 5, 2014
39	Global	June, 2014 and December, 2014
40	Maryland, USA	November 20, 2011 to March 16, 2014
41	Boston metropolis, USA	September 6, 2009, to May 15, 2016,
		and May 22, 2016, to May 7, 2017
42	United Arab Emirates	November 2016 and January 2017
43	Toronto, Canada	June 26, 2015 to September 10, 2015
44	Italy	October 2016 to April 2017
		and October 2017 to April 2018
45	UK	March, 2012 to August, 2015
46	USA	January 12, 2020 to April 5, 2020

Table  14 presents the most commonly used techniques within this category, with both Autoregressive Models and Linear Regression Models each accounting for 5 out of 14 cases. The Keywords-based Approach features the least complex algorithms among all categories. Its primary aim is to assess whether there is a correlation between the frequency of disease-related keywords in tweets and the actual incidence of those diseases.

Table 14.

Main prediction techniques for keywords-based approaches

Technique	Number of studies
Autoregressive Model	5
Linear Regression Model	5
Correlation Coefficient	3
Nonlinear Gaussian Process	1

Linear regression methods—both autoregressive and non-autoregressive—were the most frequently applied. These included simple linear models with a single predictor, as well as multiple regression models incorporating data from Twitter and other external sources.

What metrics are used to measure the performance of the ML techniques?

The models and algorithms used in the studies included in this review varied depending on the specific objectives of each investigation. While the classification proposed by Dai et al. [14] provides a useful framework for grouping models by complexity, differences in research goals exist even within the same category. For instance, within the Word Embedding-Based Approach, some studies focus on tweet content classification, while others use time series models to predict patterns based on the volume of classified tweets.

Overall, the models identified in the literature can be grouped into three main categories of machine learning tasks: i) Classification, ii) Regression and iii) Clustering. These categories were used to classify both the algorithms and their corresponding evaluation metrics. Due to the diverse objectives of the studies, several papers applied multiple ML techniques and reported metrics classified into more than one category.

For classification algorithms, the most commonly reported metrics were Accuracy, Precision, Recall, F-Measure, and Area Under the ROC Curve (AUC). The Correlation metric was also included in this group, as it assesses the presence and direction of relationships between variables—effectively classifying them as positively, negatively, or not correlated.

For regression algorithms, the main evaluation metrics identified were Root Mean Squared Error (RMSE), Mean Squared Error (MSE), Mean Absolute Percentage Error (MAPE), Root Mean Squared Percentage Error (RMSPE), and Mean Absolute Error (MAE).

Clustering algorithms were applied in seven of the nine studies categorized under the Word Embedding-Based Approach. However, most of these studies did not report evaluation metrics for assessing clustering quality. An exception is Chen et al. [45], who reported and visualized the within-cluster sum of squares. While Paul et al. [39] and Mackey et al. [44] mentioned the use of coherence scores to optimize clustering, they did not report the final results. The remaining studies did not include any clustering performance metrics.

To evaluate the performance of the models reported across the selected studies, a comparative analysis of methods and metrics was conducted. The performance of the metrics used in each study are presented in four tables, organized by the methodological classification previously described. Direct comparison of metric values across studies is challenging due to differences in datasets, model configurations, temporal scope, and geographical context. Nonetheless, a descriptive comparison allows us to highlight models that appear more promising across different research settings.

Table  15 presents the results for the Learning-based Approach. Within this category, 11 studies employed classification algorithms, and 9 used regression models. The most frequently reported metrics for classification were Accuracy, Precision, and Recall. The best-performing models for each metric were:

• Accuracy (84.2%): A Hybrid Naive Bayesian Classifier with NLP.

• Precision (94.38%): SVM.

• Recall (90%): Naive Bayes.

Table 15.

Learning-based approaches, best model and metrics. The third column contains the Model category (cat.): regression (R) or classification (C)

Study	Best Model	Cat.	Metrics	Metrics values
1	SVR	R	Average error	0.37%
			Standard Deviation	0.26%
2	SVM	C	Accuracy / F1	83.98% / 90.01%
			Precision / Recall	94.38% / 86.63%
3	Naive Bayes + Queries Model	C	F-Mesuare / Precision	83% / 78%
			Recall / AUC	90% / 94.1%
4	Naive Bayes	C	F-Mesuare, Precision,	Best overall
			Recall, AUC
5	Regularized Multi-task Feature	C	AUC	83.07%
	Learning Model, Paraguay data
6	SVM with linear kernel and	C	F-Mesuare, Precision, Recall	Best overall
	stochastic gradient descent
7	SVM with Radial Basis	R	Pearson Correlation / RMSE	0.989 / 0.176%
	Function (RBF)		RMSPE / MAPE	8.27% / 23.6%
			Hit Rate	69.4
8	Hybrid Approach with NLP	C	Accuracy	84.2%
9	Autoregressive Integrated Moving	R	MAE	8.20
	Average combined with tweet
	count regression
10	SVM based on a linear kernel	C	Accuracy	78%
11	Social Media Nested Epidemic	R	MSE / Pearson Correlation	Best overall
	Simulation (SimNest)		P-value
12	LSTM model only Social	R	Pearson Correlation / RMSE	0.79 / 0.01%
	Media predictors		RMSPE / MAPE	29.52% / 69.54%
13	Naive Bayes	C	F-Measure / Precision	77% / 70%
			Recall	86%
14	SVM classifier with RBF kernel	C	Accuracy	88.26%
	bag-of-words features, MERS data
15	IAT-BPNN	R	MSE / RMSE / MAPE	Best overall
16	Model with NLP and	C	Accuracy	70%
	High-population areas data
17	CNN	R	RMSE / MAE	3.12 / 4.43
	(Twitter data)
18	Random Forest	C	Precision / F-mesuare	90.5% / 90.1%
			Recall	90.2%
19	SVR with improved particle swarm	R	MSE / RMSE	0.8401 / 0.8225
	optimization (data Region 10)		MAPE	0.7082
20	XGBoost, Feature Sets:	R		5862
	Original + Event + SEIR
	(Susceptible Exposed Infected
	Recovered)

¹ Best overall performance, mentioned by the authors

² TRAP = It is the name of a model that detects disease epidemics

³ The MSE values correspond to the scale of the predicted variable

For regression algorithms, the most commonly used metrics were RMSE and MAPE. The best-performing models were:

• Lowest RMSE (0.01%): Long Short-Term Memory (LSTM).

• Lowest MAPE (23.6%): SVM.

In the Lexicon-Based Approach category, only two studies were identified, both employing classification algorithms (see Table  16). Precision was the only evaluation metric reported in both studies. The highest precision value, 79%, was achieved using a Conditional Random Fields (CRF) algorithm combined with a Log-Linear Model.

Table 16.

Lexicon based approaches, best model and metrics. The third column contains the Model category (cat.): classification (C)

Study	Best Model	Cat.	Metrics	Metrics values
21	Model with a threshold	C	Precision / MRR	69% / 82%
			Coverage	60%
22	Algorithm Conditional Random Fields,	C	Precision / Recall	79% / 58%
	(Measles + pneumonia + mumps) data		F1 score	67%

In the Word Embedding-Based Approach category, several studies employed combined algorithms, as shown in Table  17. These combinations include regression models paired with clustering models, as well as classification models combined with clustering models. Specifically, two articles used regression algorithms, six used classification algorithms, and seven incorporated clustering algorithms. The evaluation metrics reported in this category vary widely, with only one metric—Accuracy—appearing in three studies. The highest accuracy reported was 87.1%, achieved by the Word2Vec embedding model  [14].

Table 17.

Word embedding based approach, best model and metrics. The third column contains the Model category (cat.): regression (R), classification (C) or Grouping/Clustering (G)

Study	Best Model	Cat.	Metrics	Metrics values
23	model	R	RMSE	Best overall
		G	Not specified	Not specified
24	Ailment Topic AspectModel	C	Pearson Correlation	Best overall
		G	Coherence score	Not specified
25	HFSTM-A model	R	RMSE	Best overall
		G	Not specified	Not specified
26	Word Embedding Clustering,	C	Precision / Recall	96.2% / 75.6%
	similarity threshold = 0.6		F1 / Accuracy	84.6% / 87.1%
		G	Cosine Similarity	Not specified
27	Algorithm II + CNN, Influenza Dataset	C	Accuracy	72.84%
28	BiLSTM-CRF (word + char + ,	C	Precision / Recall	94.93% / 81.98%
	word embedding method)		F-score	87.52%
29	SVM classifier applied on LaBSE	C	Accuracy / F1 (Micro)	86.92% / 87.6%
			F1 (Macro)	88.1%
		G	Not specified	Not specified
30	Model with tweets that included	C	Spearman	0.45
	users self-reporting symptoms		Correlation
	and self-reported recovery	G	Coherence score	Not specified
31	Twitter data: Kmeans(6),	G	Sum of squares	Range: 1000
	Weibo data: Kmeans(5)		within a group	to 1100

¹ POS tagging is an additional word feature

While accuracy does not directly measure the quality of clustering, it can assess the classification of clusters and the formation of new clusters. For example, Dai et al. [14] use cosine similarity to determine the number of clusters in their model. This metric quantifies the similarity between two numerical vectors. In their approach, each word in a tweet is converted into a numerical vector and compared against vectors of keywords such as “influenza,” with the goal of classifying tweets based on their proximity to these keywords.

Most models in the Keywords-Based Approach category correspond to regression or classification (correlation) tasks, with eleven and four articles, respectively (see Table  18). The most commonly used metric for regression algorithms was RMSE, with the lowest reported value being 0.001897 for a multiple linear regression model. However, comparing RMSE values across different studies is unreliable, as RMSE depends on the scale of the predicted variable. For classification algorithms, the Correlation Coefficient was the primary metric used, with a maximum reported value of 0.79.

Table 18.

Keywords-based approaches, best model and metrics. The third column contains the Model category (cat.): regression (R) or classification (C)

Study	Best Model	Cat.	Metrics	Metrics values
32	Model without retweets, Syndrome	R	Correlation Coefficient	0.9846
	Elapse (time = one week)		RMSE	0.318
33	Multiple linear regression model	R	RMSE	0.001897
	with ridge regularization unweighted
34	LASSO with a marker ‘novel flu’	R	Coefficient of Determination	0.853
35	Daily infection tweets model	R	Correlation Coefficient	0.763
36	Google Flu Trends and USA	C	Pearson Correlation	0.79
	tweets model
37	Correlation model (San Diego data)	C	Correlation Coefficient	0.93
38	Model with Twitter data	R	MAE	0.1866
39	Not Defined	C
40	Model with ILI and	R	Log-likelihood	Not Defined
	Google Flu Trends data
41	ARGO Model	R	RMSE / MAE / MAPE /	Best overall
	(athena+Google+Flu-Near-You data)		Pearson Correlation
42	Experiment 3 (January data)	R	RMSE	0.14
43	Model with Twitter data related	C	Cross-Correlation Coefficient	0.5
	with heat syndrome and temperature
44	Model M4 (nowcast)	R	RMSE / MAE	0.1972 / 0.1191
			MAPE / Correlation Coefficient	10.75 / 0.9867
			Coefficient of Determination	0.9704
45	Google model	R	Pearson Correlation / MSE	0.96 /3.86
	(national level: England data)		RMSE / MAE	1.96 / 1.47
			MAPE / Mean Error	14.10% / 0.54
46	Google Trends model	R	Coefficient of Determination	0.9209

Overall, among the 46 selected articles, 23 used classification algorithms, 22 used regression algorithms, and 7 applied clustering algorithms. The most common evaluation metrics were precision for classification, RMSE for regression, and coherence score for clustering.

What are the main countries where studies related to the extraction of information on twitter with ML techniques are carried out?

Analyzing the countries where studies were conducted is important to understand the regions in which Twitter has been applied for epidemiological surveillance and to evaluate the generalizability of their findings. This analysis also highlights areas that need further research, such as adapting text analysis methods to different languages. Additionally, it helps assess whether research activity aligns with the intensity of Twitter usage in a region or reflects particular health concerns within specific populations.

To address this question, only studies focusing on a single country were considered; those involving multiple countries were excluded. A total of 32 studies were analyzed to determine their geographical distribution. The United States led with 21 studies, representing 66% of the total. UK followed with two studies (6%), while each of the remaining countries accounted for a single study (3%), as shown in Figure 2.

Fig. 2.

Distribution of the selected studies worldwide. Only studies focusing on a single country are shown; n = number of studies

In which time periods have the largest number of studies related to our topic been published?

Analyzing the time periods covered by the studies is important to assess whether findings remain consistent over time and if models require adjustments due to evolving language use. Most studies were conducted over relatively short durations (one year or less), highlighting the need for further research on adapting models as terminology changes. For example, the emergence of new terms such as COVID-19 and SARS-CoV-2 during epidemic periods requires updates in text analysis to reflect changing epidemiological contexts.

To address this, we examined the publication years of the selected studies. Figure 3 shows that the studies span from 2011 to 2020, with peaks in 2014 and 2018 (nine studies each). The Keywords-Based Approach was more common in the early years (2011–2014), while the Learning-based Approach has been predominant since 2015. Use of the Word Embedding-Based Approach has increased in recent years, whereas the Lexicon-Based Approach remains infrequent.

Fig. 3.

Studies classified by publication year and type of analysis methodology

Regarding publication peaks by category: Keywords-Based studies peaked in 2018; Learning-Based in 2015 and 2018; Lexicon-Based in 2014 and 2017; and Word Embedding-Based in 2020, accounting for 60% of that year’s studies, making it one of the most recent approaches to gain popularity.

Discussion

Respiratory infectious diseases continue to pose significant threats to human health. Traditional surveillance systems are labor intensive and frequently require laboratory testing for disease confirmation. Information derived from social media is increasingly considered a potential source of data that could contribute to epidemiological surveillance. Twitter has been identified as the most attractive platform to provide information for this purpose [8, 9].

Processing data from social networks to extract knowledge and selecting the most appropriate ML techniques is not a simple task. In this systematic review, we classified ML techniques according to the objectives of the analysis and the level of natural language processing on social media posts, as well as the most used metrics depending on the problem that the models solve.

Our results expand on findings from previous literature reviews. During the literature search for the present work, we identified twelve Systematic Reviews, and seven fulfilled the thematic criteria for our study. However, since these publications are considered secondary research (since they do not provide primary data), they were analyzed separately to avoid counting two or more times the results reported by primary research publications. A summary and main findings of these Systematic Reviews are presented in Tables 19 and 20.

Table 19.

Selected systematic reviews. The first column is the identifier (Id); the third column contains the number of studies analyzed (#) and the fourth column the methodology used

Id	Title	#	Methodology	Ref.
a	Social Media and Internet-Based Data in Global Systems for Public Health Surveillance: A Systematic Review.	32	Not referenced	[4]
b	Using Social Media for Actionable Disease Surveillance and Outbreak Management: A Systematic Literature Review	60	PRISMA	[7]
c	Social media based surveillance systems for healthcare using machine learning: A systematic review	26	Not referenced	[8]
d	Twitter as a Tool for Health Research: A Systematic Review	137	PRISMA	[9]
e	Adoption of Digital Technologies in Health Care During the COVID-19 Pandemic: Systematic Review of Early Scientific Literature.	124	PRISMA	[63]
f	Big data analytics as a tool for fighting pandemics: a systematic review of literature	45	Methodi Ordinatio	[64]
g	Sentiment analysis and its applications in fighting COVID-19 and infectious diseases: A systematic review.	28	PRISMA	[65]

Table 20.

Selected systematic reviews (databases, analysis period and notable findings). The first column is the identifier (Id)

Id	Databases	Period	Notable findings
a	PubMed, Scopus and Scirus	1990–2011	There is a need for technologies that monitor health-related issues on the Internet. Although the acceptance of the scientific use of information from social networks generates diverse opinions.
b	PubMed, Embase, Scopus, and Ichushi-Web	-Feb. 2013	The analysis of the literature demonstrates that information from social networks improves public health mechanisms and helps identify vulnerable populations.
c	IEEE, ACM Digital Library, ScienceDirect and PubMed	2010–2018	Most of the studies found focus on flu or influenza-like illness (ILI). Twitter is the social network with the largest number of studies developed, and the most used ML technique is Support Vector Machine.
d	PubMed, Embase, Web of Science, Google Scholar and CINAHL	-Sep. 2015	Health research based on Twitter data is a growing field. To describe the use of Twitter in health areas, a taxonomy with six categories was created.
e	MEDLINE and medRxiv	Jan. 2020 -Apr. 2020	AI algorithms based on image recognition and clinical data are a promising field. User health tracking applications are effective, but there is a debate about data privacy.
f	Web of Science and Scopus	2014–2020	The two main sources of Big data information are internet search engines and social networks; and some common techniques for analyzing this data are correlation and regression.
g	ScienceDirect, PubMed, Web of Science, IEEE Xplore and Scopus	Jan. 2010 - Jun. 2020	The use of technologies such as AI and ML, in the field of sentiment analysis on social networks, contributes to an improvement in results.

In addition, we conducted a review of the literature to identify additional Systematic Reviews published between 2021 and 2024 regarding the topic of interest of this review. The search was conducted using the same key words as described in the Methods section, but limiting the results to systematic reviews. Seven new reviews were identified; a summary of these is presented in Table  21 and Table  22.

Table 21.

Systematic reviews published between 2021 and 2024. The first column is the identifier (Id), the third column contains the number of studies analyzed (#), the fourth column the methodology used, and the fifth column has the year of publication

Id	Title	#	Methodology	Year	Ref.
h	The application of artificial intelligence and dataintegration in COVID-19 studies: a scoping review	794	PRISMA	2021	[66]
i	Surveillance of communicable diseases using social media: A systematic review	23	PRISMA	2023	[67]
j	Applications of machine learning for COVID-19 misinformation: a systematic review	43	PRISMA	2022	[68]
k	Classification of COVID-19 misinformation on social media based on neuro-fuzzy and neural network: A systematic review	34	Kitchenham [69, 70]	2022	[71]
l	The Impact and Applications of Social Media Platforms for Public Health Responses Before and During the COVID-19 Pandemic: Systematic Literature Review	678	Not referenced	2021	[72]
m	Promoter or barrier? Assessing how social media predicts COVID-19 vaccine acceptance and hesitancy: A systematic review of primary series and booster vaccine investigations	113	PRISMA	2024	[73]
n	Utilizing natural language processing and large language models in the diagnosis and prediction of infectious diseases: A systematic review	15	PRISMA	2024	[74]

Table 22.

Systematic reviews published between 2021 and 2024 (databases, analysis period and notable findings). The first column is the identifier (Id)

Id	Databases	Period	Notable findings
h	National Institutes of Health (NIH) LitCovid (part of PubMed) and the World Health Organization (WHO) COVID-19 database	-Mar. 2021	Identification of research areas related to COVID-19 in which AI is applicable. In the AI applications found, there is a lack of integration of heterogeneous data.
i	ACM Digital Library, IEEE Xplore, PubMed, and Web of Science	-Mar. 2020	Data mining and NLP analysis of health-related content on social networks offer valuable tools for monitoring public health and remotely predicting contagious diseases.
j	Scopus, Web of Science, and Google Scholar	-July 2021	Deep learning methods are more effective than traditional ML in detecting COVID-19. Challenges: absence of standardized datasets, limited multilingual and multimodal information services.
k	IEEE Xplore, SpringerLink, ScienceDirect, Scopus, Taylor and Francis, Wiley, Google Scholar	2018–2021	Since the COVID-19 pandemic, research on classifying misinformation on social networks has grown. Methods such as Adaptive Neural-Based Fuzzy Inference Systems (ANFIS) and Deep Neural Networks have proven effective, with studies recommending the use of hybrid algorithms that combine both approaches.
l	PubMed, Medline and Institute of Electrical and Electronics Engineers Xplore	Dec. 2015 - Dec. 2020	Since the COVID-19 pandemic, there has been a growing number of studies on misinformation classification, highlighting the role of social network data in enhancing public health monitoring and surveillance.
m	PubMed,Scopus, and Web of Science	Jan. 2020 - Feb. 2023	Although negative perceptions of vaccination were more common, studies show an increasing trend in positive sentiment, particularly in content related to booster doses.
n	PubMed, Embase, Web of Science, and Scopus	-Dec. 2023	Research shows very promising results using Large Language Models (LLMs) like GPT-4 and BERT for analyzing social media data and detecting conditions such as urinary tract infections and Lyme disease surveillance. Challenge: in-depth studies to apply LLMs in disease diagnosis, surveillance, and prognosis.

While Systematic Reviews published up to now are within the same broad interest area of our review, several differences were observed, including the main focus of interest (for instance, sentiment analysis  [65] or vaccine acceptance  [73]) and the time period of studies included in the analysis. In this context, our systematic review provides a contribution in some areas which have previously not been analyzed completely, such as the classification of the studies according to the algorithms used in the mathematical models, either based on the objective they want to achieve (classification, regression or clustering) or on the complexity of the models according to the analysis of information from social networks. The systematic review of Gupta & Katarya  [8] identified the ML techniques from its selected research, but does not delve into the results obtained from them. In our research we analyzed the algorithms used based on the problem they solve and we also analyzed the metrics used, comparing them with other research. Alamoodi et al. [65] performed a classification on the selected studies, but do not provide information about the algorithms used per study; in addition, their research focused exclusively on Sentiment Analysis associated with diverse infectious diseases, and not on potential use on epidemiological surveillance. More recent reviews pay particular attention to COVID-19  [65, 68, 71]. Of note, in general, they do not address disease surveillance or forecasting, but assess other aspects (such as social challenges, misinformation, and vaccine acceptance).

In the present work, we used the methodology outlined in the PRISMA statement. As noted in Tables  19 and 21, this methodology is frequently used in health-related research. PRISMA was the methodology used in nine of the 14 related reviews, while other methodologies were less frequently reported.

Summary of evidence and perspectives

In our review, which was centered in studies that assess the correlation between publications on Twitter and the actual number or rate of ARI, we identified two areas of major importance during the conduction of these studies: 1) identification of relevant messages and extraction of information; and 2) comparison of social media information with disease data. Regarding data extraction methods, a Learning-based approach is the most frequently used methodology. While the categorization approach proposed by Dai et al. [14] provides very precise divisions to differentiate studies corresponding to each category, the description of the methodology in some studies is unclear or ambiguous. In other studies there is a mix of characteristics of two different methodologies. As such, future research should provide clear information regarding the methodological approach that is selected by the authors to allow to assess the advantages and drawbacks of each method. In addition, patterns of Twitter usage across time need to be considered in message analysis methods, including trends in language use, language contagion, amplification patterns, as well as misinformation contents  [75, 76]. Since these factors have been reported to vary in different countries and different languages, text analysis methods that may be applied across several cultures would be useful in order to avoid the need to tailor these procedures to specific settings.

Another aspect that is noticeable in our study is that temporal trends in the use of different text analysis methodologies do not allow to clearly establish a preferred method due to the relative short time frame (2011–2020) in which studies were carried out, as well as the overall number of publications. However, it is of note that the Word Embedding based approach has been used preferentially most recently; due to the use of unsupervised models and non-manual classification (in training datasets), it is foreseeable that the preference for models of this category might increase in the coming years.

With respect to the countries analyzed by study, the largest number of studies was carried out in the USA (66%). As discussed in the limitations section, this might be explained in part due to the inclusion of only studies written in English in our review; this reduces studies performed in countries with other official languages. In addition, the intent for Twitter use may vary in different countries and, therefore, the predictive ability of message data analysis requires to be assessed in diverse regions in order to generalize its use. In this regard, another important issue that needs to be addressed is the convenience of using surveillance data based on case-definitions agreed upon internationally. This is not always an easy task, as even when case-definitions are similar, financial, social, and health system organization characteristics affect the implementation of surveillance activities.

Overall, most studies reported good results when data from Twitter was used in models to assess the occurrence of respiratory infection outbreaks. In general, analyses that focused on correlations showed better results than predictive models. Of note, most studies assessed several models, showing that even with the same data, different models can provide diverse results. This highlights the fact that comparing models between studies may be difficult and should be interpreted with caution. As such, studies that assess the performance of models in diverse regions and through different time frames would be valuable to determine whether the proposed approaches can be applied in different scenarios. This will be of particular interest, since most studies that were identified in this review were carried out in a limited geographical area and during a relatively short period of time.

Research challenges

In addition to the identified issues that merit attention in future research, there are some additional challenges for studies that use Twitter data for epidemiological surveillance, which can be divided in two categories: technical and ethical.

Technical challenges:

• Tweet analysis presents a challenge starting with cleaning the data: misspelled words, idioms, emoticons, post context, identification repeated tweets (retweets) and knowing whether or not it has been published by bots  [45].

• Considering that in each country the information in the tweets may be in a different language, this is a significant challenge for researchers who want to carry out global studies and comparisons between countries or regions.

• In addition, Twitter’s recent changes in policies and costs of using its API may limit research using data derived from this social network, as massive downloading of tweets to carry out large-scale projects will represent high costs.

• The restriction on access to publications on social networks is a limitation to carry out comparative research between different platforms; this can preclude identifying the more suitable source of information to guide public health interventions.

Ethical challenges:

• There is a debate about data privacy on social networks, and the importance of anonymize data for analysis. These policies may vary depending on the digital platform, and the permissions that users have granted. In the case of Twitter (now X), users’ profiles and publications can be viewed without any restrictions; this is not the case for Facebook, which restricts access to publications according to user preferences.

• The mechanisms for obtaining data from social networks is another point to consider prior to starting a research project. The restriction on access to data from social networks results on the need of alternative sources of information; these could be public repositories, other research, or even techniques such as web scraping, which could generate ethical or legal conflicts. Therefore, researchers must create a safe, legal and ethical strategy for extracting data from social networks.

Limitations

In the study carried out, several limitations were found, which are summarized below:

1. Only studies in English were selected, so studies published in countries with a different language could be left out of this analysis.

2. A list of keywords were used to do the Boolean search in EDS, abstract and title of the studies; while some relevant studies might have been missed, we included a large number of relevant words to encompass a wide range of ARI. The words used to refer to the social network Twitter may vary in each research, so it is possible that some relevant studies may have not been identified.

3. Only studies related to respiratory diseases were analyzed, and classification was based on the four categories proposed by Dai et al. [14]. Therefore, studies related to other health topics were not considered.

4. The start of the COVID-19 pandemic at a global scale in 2020; investigations after this year were not reviewed, and research in this area of study is expected to increase. However, systematic reviews carried out between 2021 and 2024 were included to address this limitation.

5. Studies that used data from social networks other than Twitter were not considered in this literature review.

6. The identification of all the models in each investigation is a quite complex task, because the results of the models are not always mentioned explicitly; in some studies it is difficult to identify the architecture of the algorithms used, and it was not possible to classify them.

7. The comparison among the studies included in this systematic review is a difficult task, because the circumstances, parameters and data for each research project were too diverse, many of them in completely different contexts. Therefore, it is worth clarifying that the previous comparison exercise aims only to provide a general idea of the metrics used, and the results obtained with each model.

Conclusions

Unlike previous systematic reviews, this review focuses on studies related to Twitter posts on ARI. In addition, the diversity and temporal trends in the use of different ML techniques has not been previously performed. Learning-based Approach was identified as the most widely used methodology to understand the relationship between information provided by Twitter users and the actual number of infections. Within this category the most used ML technique is SVM, but we expect that in the following years models based on deep learning may become more popular, for example models based on LSTM and CNN. According to the objective of the algorithms, the most used category is classification, and its most used metric is precision.

Future research

During the development of this systematic review several research opportunities were identified:

• Although we did not find evidence of an increase in the use of unsupervised algorithms, it is likely that use of the Word embedding based approach might increase in future years due to the automation of classification and the speedup during training of this kind of ML techniques.

• Future research involves developing clusters with more granular categories (topics) [32]; for example, it is possible to identify not only if a tweet is related to COVID-19, but also classifying its relationship with this infectious disease.

• Most of the algorithms found in this systematic review were related to variations of linear regressions. However, in recent years, ML techniques have made considerable advances in time series forecasting algorithms, such as Recurrent Neural Network (RNN) (DeepAR, developed by Amazon, is an example of this model) and Transformer Neural Networks (developed by Google).

• Delving deeper into the classification of tweets, algorithms to identify the context of the tweets are needed, so context awareness is another important area for future research.

• The identification of false positives (tweets that seem to be related to a disease when this is not true) helps to filter out tweets that are not related to the analyzed disease and leave them out of the models training data  [20], so this is another research opportunity.

• Regarding anonymizing data, even if user information is public or accessible, since research related to data from specific users, companies or institutions may require their prior consent.

• Finally, most of the studies found were carried out in a limited geographical area, which represents an opportunity for research with greater scope and representativeness at a global level; however, this involves several challenges.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

AI

Artificial Intelligence

ARGO

Autoregression with General Online Information

ARI

Acute Respiratory Infections

AUC

Area Under Curve

BERT

Bidirectional Encoder Representations from Transformers

BiLSTM-CRF

Bidirectional Long Short-Term Memory - Conditional Random Field

CBOW

Continuous Bag-Of-Words

CNN

Convolutional Neural Network

COVID-19

Coronavirus disease

CRFs

Conditional Random fields

EBSCO/EDS

EBSCO Discovery Service

HFSTM

Hidden Flu-State from Tweet Model

IAT-BPNN

Improved Artificial Tree - Back Propagation neural network

ILI

influenza and influenza-like illness

KNN

k-Nearest Neighbors

LaBSE

Language-agnostic BERT Sentence Embeddings

LASSO

Least Absolute Shrinkage and Selection Operator

LDA

Latent Dirichlet Allocation

LSTM

Long Short Term Memory

MAE

Mean Absolute Error

MAPE

Maximum Absolute Percent Error

MERS

Middle East Respiratory Syndrome

ML

Machine Learning

MRR

Mean Reciprocal Rank

MSE

Mean Squared Error

NLP

Natural Language Processing

PRISMA

Preferred Reporting Items for Systematic reviews and Meta-Analyses

RBF

Radial Basis Function

RMSE

Root Mean Squared Error

RMSPE

Root Mean Squared Percent Error

RNN

Recurrent Neural Network

ROC

Receiver Operating Characteristic

SARS-CoV-2

Severe Acute Respiratory Syndrome Coronavirus 2 (COVID-19)

SVM

Support Vector Machine

SVR

Support Vector Regression

UK

United Kingdom. It comprises England, Scotland, Wales and Northern Ireland.

USA

United States of America

Author contributions

Conceptualization, J.M.R.V. and J.C.C.T.; data curation, J.M.R.V.; writing—original draft preparation, J.M.R.V., J.C.C.T. and D.N.; writing—review and editing, J.M.R.V., J.C.C.T. and D.N.; visualization, J.M.R.V. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethical approval

Not applicable.

Consent for publication

Not applicable.

Competing interest

The authors declare no competing interests.