Abstract
The use of artificial intelligence (AI) to generate automated early warnings in epidemic surveillance by harnessing vast open-source data with minimal human intervention has the potential to be both revolutionary and highly sustainable. AI can overcome the challenges faced by weak health systems by detecting epidemic signals much earlier than traditional surveillance. AI-based digital surveillance is an adjunct to—not a replacement of—traditional surveillance and can trigger early investigation, diagnostics and responses at the regional level. This narrative review focuses on the role of AI in epidemic surveillance and summarises several current epidemic intelligence systems including ProMED-mail, HealthMap, Epidemic Intelligence from Open Sources, BlueDot, Metabiota, the Global Biosurveillance Portal, Epitweetr and EPIWATCH. Not all of these systems are AI-based, and some are only accessible to paid users. Most systems have large volumes of unfiltered data; only a few can sort and filter data to provide users with curated intelligence. However, uptake of these systems by public health authorities, who have been slower to embrace AI than their clinical counterparts, is low. The widespread adoption of digital open-source surveillance and AI technology is needed for the prevention of serious epidemics.
Keywords: Artificial intelligence, public health, epidemic intelligence, pandemic, early warning system, digital surveillance
Introduction
Artificial intelligence (AI) has been adopted in a wide spectrum of clinical medicine applications;1 however, the uptake of AI technologies in public health remains slow.2 The coronavirus disease 2019 (COVID-19) pandemic has led to substantial investment in AI tools for epidemic surveillance. With rapid advancements in AI and machine learning algorithms, state-of-the-art epidemic surveillance systems have been developed to detect early signs of epidemics by processing open-source data including news reports and social media data. The implementation of AI in outbreak detection requires 1) natural language processing (NLP) of a large quantity of multi-dimensional open-source data to detect early warning signals, 2) identifying local and regional patterns in the detected signals, 3) modelling and simulating outbreak behaviours and 4) rapidly identifying misinformation and disinformation that can cripple pandemic responses.3,4 In this narrative review, we focus on the role of AI in the early detection of and response to outbreaks and summarise the current Internet-based epidemic intelligence systems available. This review aims to update the knowledge of AI in epidemic surveillance and assess the need for widespread adoption of AI-driven open-source surveillance in public health.
The use of AI to generate automated early warnings for epidemics by harnessing vast open-source data with minimal human intervention can be revolutionary and highly sustainable. In low-income countries, AI has the potential to overcome the shortfall in human resources for traditional labour-intensive disease surveillance, which relies on doctors or laboratories to report infections and is a passive and untimely system that requires multi-level reporting structures.3 AI can further address politically sensitive issues such as data censorship. Epidemics grow exponentially and often spread by the time health authorities become aware of them. Although notifications based on confirmations from laboratories and healthcare systems are valid, early detection can be expedited and enhanced by using as early epidemic signals open-source data such as news reports, social media and geospatial, temporal, environmental and meteorological satellite data.3 Time is critical in an epidemic. For example, the severe acute respiratory syndrome coronavirus (SARS-COV-2) epidemic in Wuhan, China may have started with a single case that rapidly increased to a handful of cases in a short timeframe.5 Before the spread of the virus outside of China, transmission could have been contained through case isolation, contact tracing and quarantine, and the global pandemic could have been prevented.
Infectious disease outbreaks are further characterised by non-linear complex dynamics that are not well captured by conventional statistical approaches. AI technology applied to open-source data and followed up with formal outbreak investigation enables rapid epidemic signals that can detect and prevent serious epidemics. Open-source systems generate large quantities of unfiltered data with uncertain meaning and may overwhelm users or lead to misleading conclusions. AI can be used to curate, filter and decipher such data to provide more valid early warning signals. AI technology can also predict spread at a granular scale, guiding data-driven early local responses that can be critical in the initial stages of a pandemic.6 Given the non-linear complex spread dynamics and the uncertainty inherent in early epidemic evolution, complex dynamic AI-based modelling frameworks such as multi-agent models can help temporally and geospatially simulate the evolution of epidemics, allowing targeted and effective public health responses.6 In addition, these frameworks can be used to identify the most effective interventions and their importance in limiting spread. Finally, AI technology can overcome the challenges faced by weak health systems and issues such as poor surveillance and the censorship of outbreak reporting.3
Early detection through AI-based surveillance and monitoring
A vast array of uncurated, open-source data including social media and news reports capture the concerns and discussions of the community. If mined regularly, these data can provide early signals of epidemics before official detection by health authorities.3 Public health has been slow to adopt AI technology and utilise AI capability to enable rapid and early detection of epidemic signals. AI can further address the issue of data censorship and the challenges faced by weak healthcare systems in low-income countries with limited human resources for public health surveillance. Figure 1 illustrates the genesis and exponential growth of epidemics and the importance of early detection to prevent global spread. The early detection of epidemics affords the best prospect of preventing global spread. Currently, epidemics are detected using traditional surveillance methods that rely on disease surveillance data formally reported by healthcare workers and laboratories. Given the time lag between the onset of a symptom and the formal laboratory confirmation report, traditional surveillance methods are not rapid enough to allow for the early detection of serious epidemics.3 The COVID-19 pandemic is an example: the SARS-CoV-2 virus had already spread worldwide before the virus was first reported. The first COVID-19 cases presenting severe pneumonia of unknown origin were officially reported in China on approximately 8 December 2019. However, a retrospective study using open-source intelligence data identified another COVID-19 case in China in mid-November 2019.7 In Spain, the first COVID-19 case was officially reported on 25 February 2020; however, the virus was detected in sewage water in Barcelona 41 days before that date.5 The COVID-19 pandemic demonstrates the critical need for the early detection of epidemics using AI technology in public health.5 Currently, investment in epidemic preparedness predominantly supports the development of drugs and vaccines. Although essential, drugs and vaccines tend to become available a considerable time after a serious new infection has emerged and spread widely.8 Figure 1 highlights currently missing system capability—rapid epidemic intelligence, recognition and risk analysis—and the potential gains of AI-based epidemic intelligence.
As demonstrated during the 2020 COVID-19 pandemic and the 2014 Ebola epidemic, non-pharmaceutical interventions are critical during epidemics, especially when drugs or vaccines are unavailable.9 Epidemic growth was exponential in both cases, making rapid intervention critical to mitigating spread.9 The Ebola epidemic could have been detected in late 2013, 3 months before the World Health Organisation (WHO) was informed, using rapid social media-based intelligence—even in Guinea, which has relatively low smartphone penetration.10 Similarly, a retrospective study using open-source data on Weibo detected a signal of unknown severe pneumonia in Hubei Province in mid-November 2019, a month before the COVID-19 outbreak was officially reported.7
The most widely used outbreak alert system is the Program for Monitoring Emerging Diseases (ProMED-mail), a qualitative reporting system to which clinicians report unusual outbreaks.11 This system relies on health professionals notifying moderators of unusual outbreaks. While the system has improved the speed of traditional health system surveillance and has been the first to detect many important epidemics, it is still largely dependent on human reporting and does not harness the full capabilities of open-source data and AI. The ideal system must harness and process multiple sources of unstructured data and display data in a curated, filtered and structured format that can inform rapid public health action.3,12
Another system used for outbreak alerts is Google Flu Trends, which was in use from 2008 to 2015 but was terminated because of errors in estimation.13 Another tool, DEFENDER,14 was an outbreak detection, surveillance, forecasting and nowcasting system developed as a research tool by the United Kingdom. The system integrated geocoded symptom data from Twitter with news reports and used this information for outbreak detection, situational awareness and nowcasting.14 DEFENDER was not a public system and is not currently in use. This review focuses on systems in current use.
Current Internet-based epidemic intelligence systems
Several digital systems for the early identification of important public health events are currently available.15 The systems vary in scope based on the type of event covered; event types range from solely human diseases to a combination of human, animal and plant diseases and infectious diseases to all health events including non-communicable diseases, natural disasters and humanitarian emergencies.16 A One Health approach is ideal for infectious diseases.17 This approach describes that an end-user, whether a public health official from a national agency or a lay person, should be able to access a range of services such as news reports, geographical heat maps, risk analysis tools on the web or via mobile applications. Some services are commercially available and others are internal systems accessible only to public health agencies.
Internet-based surveillance systems are breaking new ground in the surveillance of public health events that otherwise rely exclusively on laboratory diagnostic capability and timely notification by health professionals. Although vast open-source data are freely available online, they may be irrelevant or lead to false positive reports. Research on the level of human moderation required to operate a valid epidemic intelligence system is currently limited.3,18 However, the incorporation of AI technology such as machine learning and text mining using NLP is undeniably essential for processing and filtering large amounts of unstructured data and generating valid early warning signals of serious epidemics.19 We describe eight known early warning systems below.
ProMED-mail
ProMED-mail is a system established by the International Society for Infectious Diseases in 1994 to identify unusual health events affecting humans, animals and plants.20,21 This free service is moderated by expert staff who review reports from health professionals, the Internet (e.g., official government websites) and traditional media.22 This system largely relies on a network of health personnel to provide qualitative reports of events of interest and an expert assessment of risk. In the absence of an automated data collection process, the importance of articles is judged on a case-by-case basis, introducing the potential for human error and personal bias that are inherent in human moderation.21,23 With a network of staff from at least 30 countries working in different time zones and nearly 80,000 subscribers from approximately 200 countries, ProMED-mail operates 24/7 and has identified several important epidemics.24 ProMED-mail has ongoing collaborations with other services such as HealthMap, the United States Agency for International Development and Public Health England.24,25 Human moderation continues while research to automate the data collection and curation processes is currently in progress.23 On average, six posts are displayed daily by ProMED-mail.26
HealthMap
HealthMap is a fully automated system that does not rely on human moderation. It reports all health events including non-communicable diseases and is therefore not specific to epidemics.27 Developed in 2006, HealthMap uses Fisher–Robinson Bayesian filtering in a Linux/Apache/MySQL/PHP application with other products such as Google Maps, GoogleMapAPI for PHP, Google Translate API and a single AJAX library in PHP.28 HealthMap uses a text processing algorithm to automate identifying, classifying and overlaying relevant information on a map.29 This system contains multiple modules including a data acquisition engine, a classification engine, a back-end web application, a database and a front-end web application that enable the system’s smooth operation.30 HealthMap processes approximately 80 infectious disease alerts daily.31
Epidemic Intelligence from Open Sources (EIOS)
The EIOS was developed as a collaboration between the WHO and the Joint Research Committee of the European Commission.17,32 Developed in 2017, the EIOS is an automated system with roots in the Global Public Health Intelligence Network and the Global Health Security Initiative and endorses the Early Alerting and Reporting system and the Hazard Detection and Risk Assessment System.33 The EIOS is estimated to have a system capacity of at least 40 million news items from 12,000 web sources including social media in multiple languages.34 The system includes NLP recognition technology, article classification and priority algorithms to identify, tag and categorise reports.33 Additionally, the EIOS includes human review before reports are made available to users33 and reports a broad scope of events that range from human health, natural disasters, conflicts and mass gatherings. During the COVID-19 pandemic, the EIOS provided a public COVID-19 news map35 and access to COVID-19 data with accompanying graphs through a public dashboard36 that displayed data from the WHO, Johns Hopkins University, the European Centre for Disease Prevention and Control (ECDC) and Worldometer.32 The EIOS was operational in 2019 but did not contribute to the early detection of the COVID-19 pandemic. System access is granted exclusively to the WHO and specific agencies or countries. The EIOS system is integrated with the INFORM suite, which includes risk analysis tools.37
BlueDot
BlueDot is a commercial system that began as a transport network modelling tool and later added open-source intelligence and clustering tools to allow the identification of potential hot spots for infectious disease outbreaks.38 BlueDot uses both AI and human moderation and includes a search capacity in multiple languages. However, this system is not available for public use and is only available to paying clients.38 In addition, the system has access to closed-source information such as government data, which is usually provided by clients.
Metabiota
Accessible to the public, the Metabiota Epidemic Tracker provides a heat map to show the geographic distribution of event-based epidemics for 208 pathogens.39 Metabiota couples disease impact with economic impact by calculating a Pathogen Sentiment Index, a unique feature of potential interest to insurance companies and the travel and tourism industry.40 Metabiota provides a large set of simulated events—with up to 18 million simulations for a single pathogen—using big data and cloud computing platforms.41 Metabiota further has a validated library with various disease models that facilitate risk analysis and inform planning and response activities using a peer-review process.
Global Biosurveillance Portal (GBSP)
The web-based information-sharing system GBSP42 facilitates timely responses and decision-making to support the detection and management of natural and unnatural biological hazards. Based on the Ozone Widget Framework architecture, an open-source data integration framework, the GBSP provides end-users with a single web front-end to access reports in HTML frames from multiple web applications.43 The GBSP further integrates systems in a whole-of-government approach by including data from the United States Department of Defense and other government agencies. To support the COVID-19 response, the GBSP provides near real-time data sharing, mapping and AI-based predictive analysis models for users and has partnerships with various health organisations and industry partners across countries.44 However, the GBSP is not publicly available, and its early warning sensitivity is currently unknown.
Epitweetr
Epitweetr, an R-based tool developed in 2018 by the ECDC, is an open-source system that monitors tweets on infectious diseases.45 To identify potential public health threats, individual detection signals can be sorted by geolocation, time and language.46 The rationale for creating this system was based on the proven value to public health responses of monitoring tweets and social media.3,10,47 This system is publicly available, provides open-source code and can be customised by users.
EPIWATCH
EPIWATCH is an AI-based system that harnesses open-source data to generate automated early warnings of epidemics worldwide. A public dashboard provides analytics with a searchable and sortable table of outbreak reports, analytics capability and geographic information systems mapping functionality free of charge. EPIWATCH provides AI-based event filtering, prioritisation, curation and human review of reports. These features ensure that the user is not overwhelmed with an impractical volume of data and provides a more reliable and trustworthy prediction of disease outbreaks.
EPIWATCH captures specific infectious diseases and clinical syndromes that may signal new and emerging infections. The system uses AI techniques that incorporate contemporary NLP and named entity recognition algorithms to automatically detect data points within scanned articles. A second AI sub-system of classification and prioritisation is empowered by bidirectional encoder representations derived from transformers (BERT)48,49 and can assess with 88.2% accuracy whether articles contain relevant outbreak information. BERT allows the articles to maintain contextualised representations49 and achieves state-of-the-art results in many downstream tasks including text classification, named entity recognition and text summarisation in NLP fields.50 Use of pre-trained BERT on datasets such as Google News and fine-tuning on a smaller dataset using transfer learning techniques have proven effective in increasing the robustness of the model.51 This AI system is trained and validated on article datasets.
In addition to EPIWATCH’s public dashboard, extra functionality is provided with an internal dashboard. EPIWATCH also has a suite of risk analysis tools such as FLUCAST,52 EPIRISK53 and ORIGINS54 that are designed to forecast the severity of an emerging influenza season, prioritise serious developing epidemics and provide insights into the origins of epidemics, respectively.
Using the Centers for Disease Control and Prevention’s guideline for evaluating a public health surveillance system55, we compared available systems using 15 specific parameters (Table 1).
Table 1.
ID | Parameter | ProMED Mail | HealthMap | Epidemic Intelligence from Open Sources (EIOS) | BlueDot | Metabiota Epidemic Tracker | Global Biosurveillance Portal | Epitweetr | EPIWATCH |
---|---|---|---|---|---|---|---|---|---|
Commencement year | 1994 | 2006 | 2017 | 2013 | 2008 | 2014 | 2018 | 2016 | |
1 | Epidemic events under surveillance | • Unusual health events in a broad range of infectious diseases56• Clinical syndromes56 | • A broad range of infectious diseases• A broad range of syndromes32 | • A broad range of infectious diseases57• All main syndromes included, and is customisable | • A broad range of infectious diseases56,58• Began as a transport network modelling tool, with open-source intelligence added more recently. | • A broad range of infectious diseases59 | • New or undefined infectious diseases• Syndromic surveillance60• COVID-19,44MERS-CoV61• Other epidemics: not mentioned | • Default setting is the topic of infectious diseases | • New or undefined infectious diseases• A broad range of infectious diseases• A broad range of syndromes |
2 | Types of automated or primary sources• Validated health sources (e.g., WHO, CDC, health professions)• Unvalidated web sources• Social media (e.g., Twitter) | • Health professionals• Formal (e.g., official government websites)• Informal (media reports)62 | • WHO• ProMED mail• News media: Google News, Yahoo News, Factiva, LexisNexis—blogs, etc.27 | • >12000 web sources (WHO, news channels, news feeds, others)63• ∼240 twitter accounts63• Facebook: Sprinklr, ministry of health sites | • Official health notifications (e.g., WHO)• Health forums (e.g., ProMED mail)• News articles• Travel data• Remote sensing data64 | • >240 data sources including official and unofficial sources59 | • United States Department of Defense, other government agencies, industry, CDC, intelligence community42 | • Tweets from Twitter45 | • 9 curated sources (e.g., WHO, CDC, Outbreak News Today)• Non-curated sources (Google News using over 4,000 search terms) |
3 | Number of languages of information sources | • At least 6 languages:English, French, Spanish, Portuguese, Russian and Arabic21 | • 15 languages 65 | • 80 languages63 | • 65 languages 58 | • Not mentioned | • Not mentioned | • >20 languages | • 42 languages |
4 | Number of languages in which dashboard is published | • 6 languages:English, French, Spanish, Portuguese, Russian and Arabic21 | • 7 languages: English, Chinese, Spanish, Russian, French, Portuguese and Arabic32 | • One language: English | • Not mentioned | • One language: English | • Not mentioned | • One language: English | • One language: English |
5 | Back translation of reports from original language into English | • Posts are in both original language and English translation56 | • Reports are in 7 languages: English, Chinese, Spanish, Russian, French, Portuguese and Arabic32 | • Reports are in original language• GPHIN content is translated | • Not mentioned | • Reports: brief information is provided in English only | • Not mentioned | • Tweets are in original languages | • Reports are in both original language and English translation |
6 | Frequency of item collection | • Near real- time21 | • Hourly66 | • Near real-time (check web sources every 5 minutes)33 | • Not mentioned | • Near real- time39 | • Near real- time44 | • Daily | • Curated sources: near real-time• Non-curated sources (e.g., Google News): daily |
7 | Filter out irrelevant items | • No automatic filtration21 | • Automatic categorisation, filtration and integration of reports with 84% accuracy27 | • Automatically filtering of articles for reports using AI: news article credibility algorithm67• Filtering out duplicates, named entity recognition, categorisation and tagging/filtering out noise (483 categories) | • Use of AI for automated filtering58 | • Use of AI for automated filtering68 | • Not mentioned | • No automatic filtration | • Automatic filtration using AI: priority algorithm improved to 88% using machine learning• Human moderation: analysts review all filtered articles to send the relevant reports |
8 | Time, location, disease information and link to original source provided in all reports | • Minimum information includes 1) time (publication time or event time), 2) outbreak location, 3) diseases/syndromes and 4) link to original article56 | • Include 1) publication time, 2) event location, 3) disease/syndrome, 4) link to original article32 | • For COVID-19, the minimum information includes 1) time (publication time or event time), 2) event location, 3) disease: COVID35 and 4) link to original article | • Not mentioned | • Includes 1) time (publication time or event time), 2) outbreak location and 3) diseases/syndrome | • Not mentioned | • Includes 1) tweet time, 2) event location and 3) keywords (diseases/syndromes) set by users | • Minimum information includes 1) time (publication time or event time), 2) outbreak location, 3) diseases/syndromes and 4) link to original article |
9 | Mapping of reported events | • With mapping, linked reports and no search or filter function56 | • With mapping, linked reports and searchable function69 | • With mapping, linked reports and searchable function35 | • With mapping70• Searchable function not mentioned• Linked reports not mentioned | • Mapping with linked reports, without searchable function39 | • Mapping44 without searchable function• Linked reports not mentioned | • With mapping, no linked tweets and, no search or filter function | • With mapping, linked reports, and searchable function |
10 | Range of data analytics | • No analytics56 | • Number of reports over time by disease and syndrome, location (timeseries)69 | • Number of COVID-19 cases for global and selected countries36• Other epidemics-number of reports over time• Risk analysis tool: INFORM risk index34,71 | • Number of cases/deaths over time by disease and location72 | • Regions with low to high number of epidemics39• Risk assessment (produces epidemic risk models for insurance, commercial and government sectors)61,73 | • MERS-CoV: number of cases and deaths in South Korea61• Other epidemics: not specified• Predictive modelling using AI45 | • Number of tweets over time by keyword and location45 | • Number of reports over time by disease syndrome and location• Risk analysis tools: EPIRISK, FLUCAST |
11 | Public data access | • Free to public, no downloadable dataset | • Publicly accessible | • COVID-19 dashboard and webpage: online view of the latest reports only35• Other epidemics: no public access | • No public access• Appears to be a paid service for selected clients | • Website (map): online view of the latest reports only39• System: no public access | • No public access | • Free to public | • Website: free access to recent 1-month data, online view only• System: guest user registration for free, access to recent 6 months dataset (Excel), provides both online view and downloadable data |
12 | Ability to customise | • No | • Not mentioned | • A generic event management system module67 that can be customised according to needs will be released to EIOS communities• The system can be customised according to user needs | • Infectious disease surveillance can be customised70 | • Not mentioned | • Can integrate additional software applications44 | • Data collection can be customised by setting up keywords, location and time45 | • The system, data collection and analysis can be customised based on requests (e.g., radiation) |
13 | Number of publications to validate system | • >1074 | • >1024,28,30,75–82 | • >1017,34,83–92 | • >10 | • <10 (7 papers)68,93–98 | • <10 (7 papers)42,60,69,99–102 | • <10 (3 papers)46,56,103 | • >10104 |
14 | Timeliness of detecting early signals of COVID-19 | • ProMED identified the early signal of the COVID-19 outbreak on 30 December 2019,105 one day before the Chinese government officially acknowledged the outbreak. | • HealthMap detected the unidentified pneumonic cases in Wuhan Province on 30 December 30 2019,106 one day before the Chinese government officially acknowledged the outbreak. | • The EIOS system detected the first article reporting on a pneumonia cluster in Wuhan Province on the day that the outbreak was declared by the WHO.32 | • BlueDot identified undiagnosed pneumonia in Wuhan Province on the same day that the outbreak was declared by the WHO.38 | • Not mentioned | • Not mentioned | • Not mentioned | • EPIWATCH detected early signals of COVID-19 on the day of the WHO announcement. |
15 | Web and Mobile App development | • Web App developed56• Mobile App developed107 | • Web App developed69• Mobile App developed (i.e., Outbreaks Near Me)108 | • Web App: COVID-19 web app accessible to public.35,36 Other epidemics web app internally accessible• Mobile App: not mentioned | • Web App (not mentioned) • No Mobile App | • Web APP developed39• No Mobile App | • Web App developed44• No Mobile App | • No Web App• No Mobile App | • Web App developed• Mobile App in development, will be available in app store. |
WHO, World Health Organization; CDC, United States Centers for Disease Control; GPHIN, Global Public Health Intelligence Network; app, application.
Geospatiotemporal forecasting
Understanding and predicting the geospatial risk of outbreaks and the evolution and spread of epidemics can further inform public health responses. In this context, machine learning methods can identify and predict the risk of outbreaks at a granular level—both geospatially and temporally. Processing such multi-dimensional (i.e., geospatiotemporal) data for analysis and forecasting requires machine learning approaches that can utilise these features to develop prediction models without losing salient information that carries important signals. Convolutional neural networks,109 transfer learning, support vector machines, random forest,110 deep learning and gradient boosting machine learning have been applied with high accuracy to these challenges in different contexts. Used in research, these models typically utilise regional data on past outbreaks, environmental factors, travel data, social factors, vector distribution and satellite meteorological data (e.g., temperature and rainfall). These data can be highly predictive of the occurrence and timing of regional outbreaks, providing a framework for early preparedness and response. None of the available epidemic intelligence systems has automated capability for geospatial risk prediction.
Modelling of interventions and response
Early warning systems can be enhanced by modelling pandemic growth and the effectiveness of interventions. This requires modelling complex dynamic systems with non-linearities that can be applied to time-series data with lags between interventions and responses. A variety of AI-based approaches have been applied to develop frameworks for these data and can be automated, customised and added to early warning systems. These approaches include long-short term memory networks that are ideal for modelling temporal data trends and can be trained to retain memory for features that are important for prediction at a given point in time whilst ‘forgetting’ features that are unimportant.13 These networks retain ‘memory’, as needed, over time and can account for lags between intervention and response. Support vector machine models and transformers have also been used to flexibly model the impact of interventions on pandemic growth globally; these systems have identified the most effective interventions employed during the SARS-CoV-2 pandemic.
Agent-based simulation models can further provide a flexible alternative to conventionally used susceptible-infectious-removed models to model geospatial dynamics and spread. These models can create synthetic populations and use available granular data on geospatial context, contact rates, behaviour, mobility and infrastructure to model the spread and impact of interventions at a fine scale.
The potential of AI in public health
The potential of AI in public health is illustrated through the development and use of the epidemic intelligence systems described in this review. Nevertheless, AI is not widely implemented at an operational level in the everyday practice of public health compared with the use of AI in clinical medicine. By generating early epidemic warnings even in low-resource settings or in areas in which data are censored by governments, AI can be revolutionary. AI enables early identification and intervention, allowing the early management of newly emergent epidemics to feasibly result in eradication. When added to late-stage interventions such as diagnostics, drugs and vaccines, AI can considerably improve health security and the prospect of preventing pandemics. AI can be used to identify not only specific diseases but also clinical syndromes that may predict new and emerging infections. Innovations in novel digital syndromic surveillance systems using open-source data can support the early detection of serious emerging infectious epidemics. The key features of an optimised AI system are:
Rapid intelligence drawn from open-source data to generate higher-level and earlier epidemic alerts compared with traditional surveillance without the need for human reporting. These alerts can be followed up with formal investigation and traditional surveillance methods such as laboratory confirmation by public health authorities.
The capability to rapidly and globally identify key serious syndromes that may result from new emerging infections or biowarfare events.
The ability to address the issues of censorship of reporting and reliance on human reporting and the challenges faced by weak health systems.
The capability to predict in real time the likelihood of serious outcomes of identified events using a suite of decision support tools (e.g., risk analysis, modelling and simulation), prioritise responses and determine the urgency of intervention.
Tailored user interfaces on the Web, mobile applications for real-time decision support and tools that can be adapted for use in health and defence across government and non-government sectors that require early warning and intelligence on serious epidemics.
The COVID-19 pandemic has prompted substantial investment in AI tools for epidemic surveillance. The United States Centers for Disease Control and Prevention established the Center for Forecasting and Outbreak Analytics in 2021,111 and the United Kingdom announced its Global Pandemic Radar the same year.112 In Berlin, the WHO further established the Pandemic Hub,113 which is co-funded by the German government. The ECDC established an open-source tool called Epitweetr in August 2018.45,46 The unprecedented 2022 monkeypox epidemic in non-endemic countries provided a test case for using learnings from epidemic intelligence systems’ responses to the COVID-19 pandemic. All systems reported on monkeypox; however, we are aware of only two special initiatives. HealthMap created a monkeypox dashboard with daily updates of case counts,114 and EPIWATCH created a weekly summary of syndromic surveillance for rash and fever illnesses that could be monkeypox misdiagnosed as other illnesses.115
The past decade has seen the emergence of epidemics such as the novel zoonotic influenza,116,117 the Middle East respiratory syndrome coronavirus, Ebola, Zika virus and SARS-CoV-2.118 These events highlight the increasing risk of emerging infectious diseases and the need for early warning signals. Strategies to adopt open-source early warning systems and provide the source code for such systems would allow for collaborative design on a global scale. Harnessing the creative talents of health and software engineering experts working collaboratively in interdisciplinary teams could support optimising global early warning systems. The creation of tools with free availability and user interfaces in all major international languages can increase access—including for the community and local health authorities—to open-source intelligence.
Conclusion
The widespread adoption of AI technology in public health and clinical medicine can revolutionise disease prevention and control. Currently, the use of available systems is not widespread at the grassroots level of public health practice. AI technology can generate early epidemic warnings without reliance on passive human reporting, enable intervention early in the timeline of an epidemic and allow newly emergent epidemics to be identified and eradicated as quickly as possible. In this review of existing epidemic intelligence systems, EPIWATCH is identified as having substantial value in epidemic intelligence collection, the identification of outbreak alerts and early epidemic signal detection. Widespread adoption of digital surveillance by public health agencies at the global, national and local operational levels offers the best prospect of preventing the next pandemic.
Footnotes
Author contributions: CRM: Conception and design of the study, manuscript drafting and critical revision of the article. XC: manuscript drafting, data collection, revision and manuscript submission. MPK: manuscript drafting, data collection and revision. AQ: manuscript drafting, data interpretation and revision. SL: manuscript drafting and revision. HS, HP, LY, DH, WW, and IS: manuscript drafting. DG: manuscript drafting and study conception.
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors are part of the EPIWATCH system.
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Medical Research Future Fund Research Grant (ID RFRHPI000280), Stage 1 from the Australian Government. Raina MacIntyre is supported by an NHMRC Principal Research Fellowship, grant number 1137582, and from The Balvi Filantropic Fund.
ORCID iDs: Xin Chen https://orcid.org/0000-0002-9905-2307
Samsung Lim https://orcid.org/0000-0001-9838-8960
References
- 1.Rajpurkar P, Chen E, Banerjee O, et al. AI in health and medicine. Nat Med 2022; 28: 31–38. [DOI] [PubMed] [Google Scholar]
- 2.Panch T, Pearson-Stuttard J, Greaves F, et al. Artificial intelligence: opportunities and risks for public health. Lancet Digit Health 2019; 1: e13–e14. [DOI] [PubMed] [Google Scholar]
- 3.Yan SJ, Chughtai AA, MacIntyre CR. Utility and potential of rapid epidemic intelligence from internet-based sources. Int J Infect Dis 2017; 63: 77–87. [DOI] [PubMed] [Google Scholar]
- 4.EPIWATCH: Prevent the next pandemic with epidemic intelligence. 2022. [cited 2022 January 20]. Available from: https://www.epiwatch.org/.
- 5.Chavarria-Miró G, Anfruns-Estrada E, Guix S, et al. Sentinel surveillance of SARS-CoV-2 in wastewater anticipates the occurrence of COVID-19 cases. MedRxiv 2020. [Google Scholar]
- 6.Heslop DJ, Chughtai AA, Bui CM, et al. Publicly available software tools for decision-makers during an emergent epidemic—Systematic evaluation of utility and usability. Epidemics 2017; 21: 1–12. [DOI] [PubMed] [Google Scholar]
- 7.Kpozehouen EB, Chen X, Zhu M, et al. Using open-source intelligence to detect early signals of COVID-19 in China: descriptive study. JMIR Public Health Surveill 2020; 6: e18939. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.MacIntyre R. The risk of selective investment in downstream pandemic planning. Global Biosecurity 2019; 1. [Google Scholar]
- 9.Ajisegiri WS, Chughtai AA, MacIntyre CR. A risk analysis approach to prioritizing epidemics: Ebola virus disease in West Africa as a case study. Risk Anal 2018; 38: 429–441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Joshi A, Sparks R, Karimi S, et al. Automated monitoring of tweets for early detection of the 2014 Ebola epidemic. PLoS ONE 2020; 15: e0230322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rolland C, Lazarus C, Giese C, et al. Early detection of public health emergencies of international concern through undiagnosed disease reports in ProMED-mail. Emerg Infect Dis 2020; 26: 336–339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kaiser R, Coulombier D, Baldari M, et al. What is epidemic intelligence, and how is it being improved in Europe? Weekly releases (1997–2007) 2006; 11: 2892. [DOI] [PubMed] [Google Scholar]
- 13.Chae S, Kwon S, Lee D. Predicting Infectious Disease Using Deep Learning and Big Data. Int J Environ Res Public Health 2018; 15: 1596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Thapen N, Simmie D, Hankin C, et al. Defender: detecting and forecasting epidemics using novel data-analytics for enhanced response. PLoS One 2016; 11: e0155417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Collier N. Uncovering text mining: A survey of current work on web-based epidemic intelligence. Global public health 2012; 7: 731–749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Barboza P, Vaillant L, Le Strat Y, et al. Factors influencing performance of internet-based biosurveillance systems used in epidemic intelligence for early detection of infectious diseases outbreaks. PLoS ONE 2014; 9: e90536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Abdelmalik P, Peron E, Schnitzler J, et al. The Epidemic Intelligence from Open Sources initiative: a collaboration to harmonize and standardize early detection and epidemic intelligence among public health organizations/L'initiative≪Epidemic Intelligence from Open Sources≫: une collaboration visant a harmoniser et a standardiser les procedures de detection precoce et de renseignement epidemiologique entre les organisations de sante publique. Weekly Epidemiological Record 2018; 93: 267–270. [Google Scholar]
- 18.O'Shea J. Digital disease detection: A systematic review of event-based internet biosurveillance systems. Int J Med Inform 2017; 101: 15–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Fisichella M, Stewart A, Denecke K, et al. , editors. Unsupervised public health event detection for epidemic intelligence. Proceedings of the 19th ACM international conference on Information and knowledge management 2010. [Google Scholar]
- 20.Yu VL, Madoff LC. ProMED-mail: an early warning system for emerging diseases. Clin Infect Dis 2004; 39: 227–232. [DOI] [PubMed] [Google Scholar]
- 21.Carrion M, Madoff LC. ProMED-mail: 22 years of digital surveillance of emerging infectious diseases. Int Health 2017; 9: 177–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Madoff LC, Woodall JP. The internet and the global monitoring of emerging diseases: lessons from the first 10 years of ProMED-mail. Arch Med Res 2005; 36: 724–730. [DOI] [PubMed] [Google Scholar]
- 23.You J, Expert P, Costelloe C. Using text mining to track outbreak trends in global surveillance of emerging diseases: ProMED-mail. medRxiv 2020. 2020-01. [Google Scholar]
- 24.Madoff L, Brownstein J. ProMED and HealthMap: Collaboration to improve emerging disease surveillance. Int J Infect Dis 2010; 14: e184.19726214 [Google Scholar]
- 25.ProMED-mail. Partners.: International Society of Infectious Diseases (ISID); 2022 [updated January 2022. Available from: https://promedmail.org/partners/.
- 26.ProMED [cited 2022. April 12]. Available from: https://promedmail.org/.
- 27.Freifeld CC, Mandl KD, Reis BY, et al. HealthMap: global infectious disease monitoring through automated classification and visualization of Internet media reports. J Am Med Inform Assoc 2008; 15: 150–157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Madoff LC, Li A. Web-based surveillance systems for human, animal, and plant diseases. Microbiol Spectr 2014; 2: OH-0015-2012. [DOI] [PubMed] [Google Scholar]
- 29.Roberts SL, Elbe S. Catching the flu: syndromic surveillance, algorithmic governmentality and global health security. Security Dialogue 2017; 48: 46–62. [Google Scholar]
- 30.Kapadia R, Walker M. HealthMAP-A Model-Based Framework for On-Line Prognostics and Health Management (PHM). AIAA Infotech@ Aerospace 2010; 2010: 3500. [Google Scholar]
- 31.HealthMap [cited 2022. January 12]. Available from: https://www.healthmap.org/en/.
- 32.World Health Organization (WHO). Epidemic Intelligence from Open Sources (EIOS). 2022 [updated Jan 2022; cited 2022. February 10]. Available from: https://www.who.int/initiatives/eios.
- 33.World Health Organization (WHO). EIOS Technology 2022 [updated Jan 2022; cited 2022. February 10]. Available from: https://www.who.int/initiatives/eios/eios-technology.
- 34.Spagnolo L, AbdelMalik P, Doherty B, et al. Integration of the Epidemic Intelligence from Open Sources (EIOS) system and the INFORM suite. 2020.
- 35.World Health Organisation (WHO). EIOS COVID-19 News Map 2022 [updated February 10, 2022; cited 2022 February 10]. Available from: https://portal.who.int/eios-coronavirus-newsmap/.
- 36.World Health Organisation (WHO). COVID-19 case and death count dashboard 2022 [updated February 10, 2022; cited 2022. February 10]. Available from: https://portal.who.int/report/eios-covid19-counts/.
- 37.World Health Organization (WHO). Coronavirus news as reported through publicly available websites: Epidemic Intelligence of Open Sources (EIOS); 2020 [updated 30 Jan 2022. Available from: https://portal.who.int/eios-coronavirus-newsmap/.
- 38.BlueDot 2022 [updated Jan 2022. Available from: https://bluedot.global/.
- 39.Metabiota. Metabiota Epidemic Tracker 2017 [updated 30 Jan 2022. Available from: https://www.metabiota.com/epidemic-tracker.
- 40.Singer AW. The Evolution of Parametric Insurance. Risk Management 2019; 66: 32–36. [Google Scholar]
- 41.Metabiota 2017. [updated Jan 2022. Available from: https://metabiota.com/.
- 42.Anderson JA, Rosenzweig CN, Roos J, et al. The Global Biosurveillance Portal: Biosurveillance for the Department of Defense. Online J Public Health Inform 2015; 7: e63. [Google Scholar]
- 43.Ruchkin I, Dwivedi V, Garlan D, et al. Architectural Modeling of Ozone Widget Framework End-User Compositions (CMU-ISR-14-108). 2014.
- 44.JPL CBRN Information Management/Information Technology. Global Biosurveillance Portal [cited 2022. February 12]. Available from: https://www.jpeocbrnd.osd.mil/Portals/90/jpm-fact-sheet-gbsp-final.pdf.
- 45.European Centre for Disease Prevention and Control. Epitweetr tool 2020. [updated October 1, 2020; cited 2021 October 28]. Available from: https://www.ecdc.europa.eu/en/publications-data/epitweetr-tool.
- 46.Espinosa L, Wijermans A, Orchard F, et al. Epitweetr: Early warning of public health threats using Twitter data. Euro Surveill 2021; 27: 2200177. Available at SSRN 3811673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Joshi A, Sparks R, McHugh J, et al. Harnessing tweets for early detection of an acute disease event. Epidemiology 2020; 31: 90–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Lan Z, Chen M, Goodman S, et al. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. arXiv:190911942 [cs] 2020. [Google Scholar]
- 49.Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:181004805 [cs] 2018. [Google Scholar]
- 50.Tay Y, Dehghani M, Bahri D, et al. Efficient Transformers: A Survey. ACM Comput Surv 2022; 55: 1–28. [Google Scholar]
- 51.Rasmy L, Xiang Y, Xie Z, et al. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ Digit Med 2021; 4: 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Moa A, Muscatello D, Chughtai A, et al. Flucast: A Real-Time Tool to Predict Severity of an Influenza Season. JMIR Public Health Surveill 2019; 5: e11780. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Lesmanawati DA, Veenstra P, Moa A, et al. A rapid risk analysis tool to prioritise response to infectious disease outbreaks. BMJ Global Health 2020; 5: e002327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Chen X, Chughtai AA, MacIntyre CR. Recalibration of the Grunow–Finke Assessment Tool to Improve Performance in Detecting Unnatural Epidemics. Risk Anal 2019; 39: 1465–1475. [DOI] [PubMed] [Google Scholar]
- 55.German RR, Lee LM, Horan JM, et al. Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001; 50: 1–35; quiz CE1–7. [PubMed] [Google Scholar]
- 56.Trotta A, Marinaro M, Cavalli A, et al. African Swine Fever—How to Unravel Fake News in Veterinary Medicine. Animals (Basel) 2022; 12: 656. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.EIOS Newsletter July 2021 2021 [cited 2022. April 12]. Available from: https://cdn.who.int/media/docs/default-source/eios/eios-july-2021-newsletter.pdf?sfvrsn=16a5cb51_3&download=true.
- 58.AWS. Powered by AWS, BlueDot Uses Machine Learning to Detect and Respond to Infectious Disease Risks 2020. [cited 2022 April 12]. Available from: https://aws.amazon.com/solutions/case-studies/bluedot/.
- 59.Metabiota. Metabiota Brochure [cited 2022. April 12]. Available from: https://metabiota.com/sites/default/files/presentation_files/metabiota_brochure_sept_2017.pdf.
- 60.Rhee C, Burkom H, Yoon CG, et al. Syndromic Surveillance System for Korea–US Joint Biosurveillance Portal: Design and Lessons Learned. Health Secur 2016; 14: 152–160. [DOI] [PubMed] [Google Scholar]
- 61.Colonel POK, Brent B, Michael EM. The global biosurveillance portal: A One Health Approach. 2019. [cited 2022 May3]. Available from: https://www.jwc.nato.int/images/stories/threeswords/OneHealth2019.pdf.
- 62.Bhatia S, Lassmann B, Cohn E, et al. Using digital surveillance tools for near real-time mapping of the risk of infectious disease spread. NPJ Digit Med 2021; 4: 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.GitHub. How the WHO uses technology for early detection, verification, and risk assessment of pandemics 2020. [cited 2022 April 12]. Available from: https://www.youtube.com/watch?v=clm5Ee6O_4o.
- 64.Tang V. How BlueDot Leverages Data Integration to Predict COVID-19 Spread 2020 [cited 2022. April 12]. Available from: https://www.safe.com/blog/2020/10/bluedot-leverages-data-integration-predict-covid-19-spread/.
- 65.Lee S, Moscoso S. How open data helps detect disease outbreak 2014. [cited 2023 February 1]. Available from: https://blogs.worldbank.org/governance/how-open-data-helps-detect-disease-outbreak.
- 66.Jenara N. HealthMap: The Crowdsourced Infectious Disease Tracker That Was Built on Nights and Weekends 2011. [cited 2022 April 10]. Available from: https://www.fastcompany.com/1716151/healthmap-crowdsourced-infectious-disease-tracker-was-built-nights-and-weekends.
- 67.World Health Organisation. EIOS NEWSLETTER January 2022 2022 [cited 2022. April 2]. Available from: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/viewer.html?pdfurl=https%3A%2F%2Fcdn.who.int%2Fmedia%2Fdocs%2Fdefault-source%2Feios%2Feios-newsletter_january-2022.pdf%3Fsfvrsn%3D84f843f_1%26download%3Dtrue&clen=1826731&chunk=true.
- 68.Allam Z, Dey G, Jones DS. Artificial intelligence (AI) provided early detection of the coronavirus (COVID-19) in China and will influence future Urban health policy internationally. Aids Behav 2020; 1: 156–165. [Google Scholar]
- 69.Berger KM. Emerging and Enabling Technologies in Biodefense. Defense Against Biological Attacks: Springer; 2019: 253–281.
- 70.Bluedot. 5 Useful Tips to Get the Most Out of BlueDot Insights [cited 2022. April 12]. Available from: https://bluedot.global/5-useful-tips-to-get-the-most-out-of-bluedot-insights/.
- 71.Doherty B, Marin-Ferrer M, Vernaccini L. INFORM Epidemic Risk Index. JRC Technical Reports. Retrieved from https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk/INFORM-Epidemic-Risk-Index, 2018.; 2018.
- 72.Bluedot. BlueDot for Healthcare [cited 2022. April 12]. Available from: https://bluedot.global/healthcare/.
- 73.Oppenheim B, Gallivan M, Madhav NK, et al. Assessing global preparedness for the next pandemic: development and application of an epidemic preparedness index. BMJ global health 2019; 4: e001157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.One Health initiative. ProMED Publications [cited 2022. April 12]. Available from: http://dev.onehealthinitiative.com/promed-publications/.
- 75.Nelson R. HealthMap: the future of infectious diseases surveillance? The Lancet Infectious Diseases 2008; 8: 596. [Google Scholar]
- 76.Majumder MS, Santillana M, Mekaru SR, et al. Utilizing nontraditional data sources for near real-time estimation of transmission dynamics during the 2015-2016 Colombian Zika virus disease outbreak. JMIR public health and surveillance 2016; 2: e5814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Lyon A, Nunn M, Grossel G, et al. Comparison of Web‐Based biosecurity intelligence systems: BioCaster, EpiSPIDER and HealthMap. Transbound Emerg Dis 2012; 59: 223–232. [DOI] [PubMed] [Google Scholar]
- 78.Hossain N, Househ MS, editors. Using HealthMap to Analyse Middle East Respiratory Syndrome (MERS) Data. ICIMTH; 2016. [PubMed]
- 79.Chen H, Zeng D, Yan P. HealthMap. Infectious Disease Informatics: Springer; 2010; 183–186.
- 80.Brownstein JS, Freifeld CC, Reis BY, et al. Surveillance Sans Frontieres: Internet-based emerging infectious disease intelligence and the HealthMap project. PLoS Med 2008; 5: e151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Brownstein JS, Freifeld C. HealthMap: the development of automated real-time internet surveillance for epidemic intelligence. Weekly releases (1997–2007) 2007; 12: 3322. [DOI] [PubMed] [Google Scholar]
- 82.Thomas MC, Kite-Powell A, Atrubin D, et al. Evaluating the utility of HealthMap as a supplementary surveillance tool. Online J Public Health Inform 2015; 7. [Google Scholar]
- 83.Williams GS, Impouma B, Mboussou F, et al. Implementing epidemic intelligence in the WHO African region for early detection and response to acute public health events. Epidemiol Infect 2021; 149: e261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Wark W. Building a better global health security early-warning system post-COVID: The view from Canada. International Journal 2021; 76: 55–67. [Google Scholar]
- 85.Ng V, Rees EE, Niu J, et al. Artificial intelligence in public health: Application of natural language processing algorithms for extracting information from news articles in event-based surveillance. Can Commun Dis Rep 2020; 46: 186–191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 86.Mercier A. An integrated approach to epidemic intelligence: the importance of pluridisciplinary collaborations in the framework of the development of methods to collect and analyze health data: Université Paris-Est; 2020.
- 87.Mboussou F, Ndumbi P, Ngom R, et al. Infectious disease outbreaks in the African region: overview of events reported to the World Health Organization in 2018. Epidemiol Infect 2019; 147: e299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88.Luengo-Oroz M, Hoffmann Pham K, Bullock J, et al. Artificial intelligence cooperation to support the global response to COVID-19. Nature Machine Intelligence 2020; 2: 295–297. [Google Scholar]
- 89.Laxmi Lydia E, Moses Gummadi J, Ranjan Pattanaik C, et al. , editors. Interdependence in Artificial Intelligence to Empower Worldwide COVID-19 Sensitivity. International Conference on Communication, Computing and Electronics Systems; 2021: Springer. [Google Scholar]
- 90.Impouma B, Wolfe CM, Mboussou F, et al. Use of electronic tools for evidence-based preparedness and response to the COVID-19 pandemic in the WHO African region. Lancet Digit Health 2020; 2: e500–e502. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 91.Fanelli A, Awada L, Caceres-Soto P, et al. Sensitivity of an international notification system for wildlife diseases: A case study using the OIE-WAHIS data on tularemia. Zoonoses Public Health 2022; 69: 286–294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.De Lusignan S, Liyanage H, McGagh D, et al. COVID-19 surveillance in a primary care sentinel network: in-pandemic development of an application ontology. JMIR Public Health Surveill 2020; 6: e21434. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Badker R, Miller K, Pardee C, et al. Challenges in reported COVID-19 data: best practices and recommendations for future epidemics. BMJ Glob Health 2021; 6: e005542. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Gallivan M, Oppenheim B, Madhav NK. Using social media to estimate Zika's impact on tourism:# babymoon, 2014–2017. PLoS ONE 2019; 14: e0212507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 95.Allam Z. The rise of machine intelligence in the COVID-19 pandemic and its impact on health policy. Surveying the COVID-19 Pandemic its Implications 2020: 89. [Google Scholar]
- 96.Alhasan M, Hasaneen M. Digital imaging, technologies and artificial intelligence applications during COVID-19 pandemic. Comput Med Imaging Graph 2021; 91: 101933. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97.Jamison DT, Lau LJ, Wu KB, et al. Country performance against COVID-19: rankings for 35 countries. BMJ Glob Health 2020; 5: e003047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98.Meadows AJ, Oppenheim B, Guerrero J, et al. Estimating infectious disease underreporting at the country level: a model and application to the COVID-19 pandemic. Available at SSRN 3706059. 2020. [Google Scholar]
- 99.Cho HW, Chu C. A Joint Exercise against Intentional Biothreats. Osong Public Health Res Perspect 2018; 9: 1–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Toffoli CA. Optimizing Mission-Specific Medical Threat Readiness and Preventive Medicine for Service Members. US Army Med Dep J 2018; 49–54. [PubMed] [Google Scholar]
- 101.Rhee C, Burkom H, Yoon C, et al. Military Real-time Syndromic Surveillance System for Biosurveillance Portal in Korea. Online J Public Health Inform 2015; 7: e48. [Google Scholar]
- 102.Tak S, Jareb A, Choi S, et al. Enhancing ‘whole-of-government’ response to biological events in Korea: able response 2014. Osong Public Health Res Perspect 2018; 9: 32–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 103.Sökmen S, Sevinc F, Yolal M. COVID-(Mis) Infodemic as a Nascent Concept for Tourism and the Case of Epitweetr. Co-Editors 2021: 136.
- 104.EPIWATCH. Curated data at your fingertips [cited 2022. April 12]. Available from: https://www.epiwatch.org/publications.
- 105.ProMED. PRO/AH/EDR> Undiagnosed pneumonia – China (HU): RFI 2019. [cited 2022 April 19]. Available from: https://scholar.harvard.edu/files/kleelerner/files/20191230_promed_-_undiagnosed_pneumonia_-_china_hu-_rfi_archive_number-_20191230.6864153.pdf.
- 106.O'brien M, Larson C. Can AI flag disease outbreaks faster than humans? Not quite 2020. [cited 2023 February 1]. Available from: https://apnews.com/article/united-nations-us-news-ap-top-news-international-news-virus-outbreak-100fbb228c958f98d4c755b133112582.
- 107.Apptricker. Download ProMED-mail App for IOS & Android Phones, How to [cited 2022. April 12]. Available from: https://apptricker.in/download-promed-mail-app-for-ios-android-phones-how-to/.
- 108.Chunara R, Andrews JR, Brownstein JS. Social and news media enable estimation of epidemiological patterns early in the 2010 Haitian cholera outbreak. Am J Trop Med Hyg 2012; 86: 39–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 109.Anno S, Hara T, Kai H, et al. Spatiotemporal dengue fever hotspots associated with climatic factors in Taiwan including outbreak predictions based on machine-learning. Geospat Health 2019; 14. [DOI] [PubMed] [Google Scholar]
- 110.Ong J, Liu X, Rajarethinam J, et al. Mapping dengue risk in Singapore using Random Forest. PLoS Negl Trop Dis 2018; 12: e0006587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111.Center for Disease Control and Prevention (CDC). CDC Stands Up New Disease Forecasting Center 2021 [updated August 18, 2021; cited 2022 February 10]. Available from: https://www.cdc.gov/media/releases/2021/p0818-disease-forecasting-center.html.
- 112.GOV.UK. PM announces plan for ‘Global Pandemic Radar’ 2021 [updated 21 May 2021; cited 2022 February 8]. Available from: https://www.gov.uk/government/news/pm-announces-plan-for-global-pandemic-radar.
- 113.World Health Organisation (WHO). WHO, Germany open Hub for Pandemic and Epidemic Intelligence in Berlin 2021 [updated 1 September 2021. cited 2022 February 8]. Available from: https://www.who.int/news/item/01-09-2021-who-germany-open-hub-for-pandemic-and-epidemic-intelligence-in-berlin.
- 114.Global. health Map. Monkeypox Line List Cases [cited 2022. September 19]. Available from: https://map.monkeypox.global.health/country.
- 115.EPIWATCH. EPISCOPE-SEP 13 2022. [cited 2022 September 19]. Available from:5https://www.epiwatch.org/media/EPISCOPE_SEPTEMBER_13_2022.pdf.
- 116.Yang Y, Halloran ME, Sugimoto JD, et al. Detecting human-to-human transmission of avian influenza A (H5N1). Emerg Infect Dis 2007; 13: 1348–1353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 117.Muscatello DJ, Cretikos MA, MacIntyre CR. All-cause mortality during first wave of pandemic (H1N1) 2009, New South Wales, Australia, 2009. Emerg Infect Dis 2010; 16: 1396–1402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118.Bernstein AS, Ando AW, Loch-Temzelides T, et al. The costs and benefits of primary prevention of zoonotic pandemics. Sci Adv 2022; 8: eabl4183. [DOI] [PMC free article] [PubMed] [Google Scholar]