Abstract
Background
The search for valid information was one of the main challenges encountered during the COVID-19 pandemic, which resulted in the development of several online alternatives.
Objectives
To describe the development of a computational solution to interact with users of different levels of digital literacy on topics related to COVID-19 and to map the correlations between user behavior and events and news that occurred throughout the pandemic.
Method
CoronaAI, a chatbot based on Google's Dialogflow technology, was developed at a public university in Brazil and made available on WhatsApp. The dataset with users’ interactions with the chatbot comprises approximately 7,000 hits recorded throughout eleven months of CoronaAI usage.
Results
CoronaAI was widely accessed by users in search of valuable and updated information on COVID-19, including checking the veracity of possible fake news about the spread of cases, deaths, symptoms, tests and protocols, among others. The mapping of users' behavior revealed that as the number of cases and deaths increased and as COVID-19 became closer, users showed a greater need for information applicable to self-care compared to following the statistical data. In addition, they showed that the constant updating of this technology may contribute to public health by enhancing general information on the pandemic and at the individual level by clarifying specific doubts about COVID-19.
Conclusion
Our findings reinforce the potential usefulness of chatbot technology to resolve a wide spectrum of citizens' doubts about COVID-19, acting as a cost-effective tool against the parallel pandemic of misinformation and fake news.
Keywords: Coronavirus, Data Mining, Natural Language Processing, Brazil
1. Introduction
In December 2019, a new infectious disease caused by a specific subtype of coronavirus (SARS-CoV-2) was reported in Wuhan City of Hubei Province of China [1]. In March 2020, this disease was named COVID-19 and was declared a public health emergency of international concern by the World Health Organization. COVID-19 rapidly reached a pandemic status and is responsible for severe acute respiratory syndrome, leading to nearly two million deaths worldwide as of March 2021 [2].
The rapid global spread, disease severity and absence of effective treatments generated a race of clinical trials aimed at finding drugs to treat and vaccines to prevent COVID-19. In parallel, a race through information on the statistics of contagious and related deaths also began. There was also an intense and inconclusive news storm of information (COVID-19 infodemic) about possible preventive measures, although there was no supporting scientific evidence [3]. In this context of information demand to support strategies for controlling and supporting people’s activities, the importance of adaptable and easily updated information tools became a necessity.
The term fake news does not yet have a well-defined concept, but it has often been used to express the dissemination of wrong or intentionally manipulated information to divert the truth from the facts. Although the term fake news is not new, the current most concerning point is that it is easily and speedily disseminated through social networks [4], [5]. The emergence of fake news, even if unintentional, can generate panic in the population, deviating from the correct application of strategies for the prevention and control of the disease.
Strategies aimed at preventing disinformation (i.e., deceptive advertising, forged documents, fake maps, internet fraud, fake websites, etc.) may have a substantial role in reducing the impact of COVID-19 on peoples’ lives and health. Considering today’s real-time lifestyle, easy-to-use, robust, and efficient tools are needed to address both fake news and disinformation [6]. Computational solutions based on information systems and artificial intelligence techniques have provided interactive technologies able to meet the requirements of real-time access and information processing in recent decades. In particular, natural language processing (NLP) and text mining methodologies provide automated techniques that emulate human communication through text or voice toward knowledge extraction for automatic indexing and abstracting [7]. Conversational agents grounded on NLP, known as “chatbots”, are important tools that perform advanced information retrieval, in which responses are generated based on the analysis of a rich dataset. In a simplistic way, chatbots can translate input phrases into output responses automatically.
In the scenario of high demand for information observed in the first weeks after the first case of COVID-19 in Brazil (March 2020), we proposed applying chatbot technology to answer the main questions of users based on reliable information obtained from government agencies, public health agencies such as the World Health Organization and the Pan American Health Organization, peer-reviewed scientific articles, and fake news verification agencies. Since this information is often difficult to find and understand for average users, NLP techniques are a great fit for democratizing its access. For example, older adults are not as likely as younger people to search for information on websites or less popular applications [8]. However, older people frequently use WhatsApp, a well-known application that encourages a friendlier mode of communication, as a socializing channel [9]. In this sense, a previous study on machine learning techniques for COVID-19 information and misinformation reported that older adults were more inclined to trust traditional media than new media [10].
Therefore, the objectives of this manuscript were to describe CoronaAI, a chatbot developed to interact with users of different levels of digital literacy on topics related to COVID-19 in WhatsApp, and to map the correlations between user behavior, query topics, and pandemic aspects (i.e., incidence of events and news in the period) over eleven months of its usage.
2. Methods
A chatbot named CoronaAI was developed based on Google Dialogflow technology and using Brazilian Portuguese as the language. CoronaAI was built following the framework proposed by Chung and Park [11], who applied a chatbot for use in the context of a healthcare service. Although the technology was already developed, we proposed its application for an innovative purpose, which was to answer user questions about fake news and other queries related to the COVID-19 pandemic. The chatbot consists of a design with four different levels: data level, information level, knowledge level, and service level. In our context, the data level represents the raw resources of textual content, such as technical papers, spreadsheets, medical reports, and websites. The information level represents the layer where the processed data can be delivered to the users, simulating a dialog. Knowledge level covers the voice and textual process engine. This level is the interface between user questions and reliable responses, where the textual data are processed using text mining and information retrieval methods. It is responsible for fake news detection in the user's input.
Instead of returning a verdict, the tool compares the user's input with the indexed false claim using a textual similarity score and answers with the link for the fact-checking article. Thus, it relies on specialized fact-checking agencies to make verdicts and only matches the user's interaction with the already checked fake news. The last level, the service level, has the information system layer, connecting a web application to the knowledge level to provide a rich user experience for voice or text exchange of messages.
Fig. 1 shows an overview of the CoronaAI platform with different levels and objects. From bottom to top, the data level represents the different sources of information used by CoronaAI. The primary sources of sanitary recommendations include medical research papers and governmental websites, which determine the foundations of the chatbot answers. To fetch newly checked claims regarding COVID-19, we scraped fact-checking agencies' websites with an automated update. Those claims were processed and stored in a database for further retrieval. The last object on the data level is the COVID-19 reports, which are retrieved from public APIs containing statistics concerning the pandemic evolution.
Fig. 1.
Overview of the CoronaAI platform composed of four levels: Data Level, Information Level, Knowledge Level and Service Level.
On top of the data level, the information level systematizes all data on a structured data storage to be accessed on higher levels. The periodically fetched fact-checked claims give rise to a fake news repository, with every claim processed and indexed to be retrieved during interaction time. Finally, the statistical information contains the current statistics on cities, states, and the country's level of disaggregation.
The knowledge level refers to the intents designed for user interactions. Those intents were created and classified through Google Dialogflow, which, in the CoronaAI architecture, serves as the middleware between the service and information levels. There are three categories of intent:
-
●
Health intents, which contain questions about health information;
-
●
Fake news intents, which check the claims against our fake news repository for similarity;
-
●
Statistical intents, which describe the user's intention of seeking information about the latest statistical reports.
Our architecture's highest level is the service level, which refers to the user interface and an auxiliary service. The user interface is a web application with a conversational interface that allows users to input written text and voice. The web service serves as an auxiliary service for knowledge level that is accessed through API to match fake news claims and fetch statistical information. Several divulgations on local TV channels, universities, and newsletters were made in the local context during the data acquisition period. The chatbot was developed and hosted using the resources of the State University of Londrina, a public university located in the southern region of Brazil. For the present study, the data were acquired over eleven months using its first version.
CoronaAI did not induce usage; all users accessed the system and interacted with the app spontaneously, focusing on finding answers to their questions as anonymous users. All the information used in the study was provided by Google API, following Brazilian data protection law boundaries.
3. Results
The characteristics that make it possible to map the interactions of users with CoronaAI throughout the observation period are presented below.
In our chatbot architecture, the conversation always starts with a user interaction classified on a series of defined intents. Table 1 presents each intent category identified by Dialogflow intention classification and its contribution to the total user interactions. Among the chatbot's total interactions (∼7,000 interactions), the majority (58%) were classified as health intent. The health intent category includes general questions about sanitary recommendations and directions concerning precautions to be taken and symptoms. The second most frequent intent, responsible for 21% of interactions, was related to the fake news repository, with statements and questions similar to those fact-checked and debunked claims in the repository. The less frequent interaction intention, considering the total number of interactions, was about the current COVID-19 statistics. This lack of interest in specific numbers of pandemic evolution statuses can be explained by the exposure of those numbers in different media sources (e.g., TV and online news). Another possibility is that the user behavior changed over time from statistics to new events and status evolutions, as explained below.
Table 1.
Relation between each intent category and the percent of interactions that resulted on it considering the whole history of CoronaAI.
| Intent category | Percentage (%) of interactions |
|---|---|
| Health | 58 |
| Fake News | 21 |
| Welcome | 12 |
| Statistics | 9 |
To map user behavior over time, Table 2 presents the most frequent terms relative to each month from March 2020 to January 2021. Those terms were extracted after preprocessing of stopword removal and normalization of similar terms. The change of interest is evident in those most frequent terms, where the prevalence of statistics-related terms at the beginning contrasts with the test and symptoms terms in the following months.
Table 2.
Most frequent terms grouped by month from March 2020 to January 2021.
| Year | Month | Most frequent terms |
|---|---|---|
| 2020 | March | how many, cases, coronavirus, dead, Londrina |
| April | coronavirus, cases, how many, covid, Londrina | |
| May | Londrina, cases, covid, coronavirus, ache | |
| June | ache, covid, do, Londrina, good | |
| July | covid, symptoms, ache, do, coronavirus | |
| August | ache, covid, symptoms, do, good | |
| September | covid, people, Londrina, contact, person | |
| October | coronavirus, covid, result, exam, test | |
| November | result, covid, exam, test, symptoms | |
| December | result, exam, covid, test, know | |
| 2021 | January | result, exam, covid, know, test |
As shown in Fig. 2 , at the beginning of the pandemic, the users' interest was to know how many people were infected, about the virus, and about the disease status in the city and in the country. This was expected since it was a new variant of the coronavirus and consequently a new disease. On the other hand, this paves the way for the increase in the dissemination of fake news and its credibility, since not much was known about the disease until then.
Fig. 2.
The term’s frequency decreased throughout the pandemic evolution, indicating an initial high interest in case counting and statistics about the stage of the disease spreading in specific places (e.g., Londrina and Brazil), which has been decreasing over time.
In January 2021, it was possible to observe a further increase in the population's interest in knowing about the cases in the city of Londrina, Brazil, since a new wave of coronavirus has been observed in the country and worldwide, renewing and reinforcing the concerns and fear about the spread of the disease. This increase also occurred around the months of September and October 2020, although this behavior was possibly because of greater dissemination of the application by the developers through a regional television channel, which pointed out the interest of viewers in the CoronaAI and its functions (Fig. 3 ).
Fig. 3.
The increasing frequency of symptom-related and test-related terms indicates a low interest in the pandemic's initial stage due to the low incidence of cases. With the increase in cases of COVID-19 in Londrina and Brazil, the search for help with symptoms and test results increased concomitantly.
Since the creation of the tool in March 2020, the “symptoms” term was the most searched, according to Fig. 3. We observed two peaks of increased interest from users about the “symptoms” of the disease. The first increase occurred between July and August 2020, coinciding with the first peak of the disease in Brazil, as shown in Fig. 4 . In January 2021, there was an increasing trend in the interest of users in the symptoms of coronavirus, coinciding with the increase in cases in late 2020 and early 2021 (Fig. 4). The figures overlapping trends in interactions with trends in cases and deaths are presented in the Supplementary material (Figures S1 and S2).
Fig. 4.
Historical series of coronavirus cases in Brazil from January 2020 to January 2021. https://coronavirus.jhu.edu/data/new-cases (accessed in 05/17/2021).
4. Discussion
In this manuscript, we report the experience of using a chatbot called CoronaAI as an auxiliary tool in the fight against fake news related to COVID-19 in Brazil, one of the most affected countries and with the highest number of deaths caused by the disease. To respond to users, the CoronaAI related the questions with data and information from official sources of government bodies and public health agencies, as well as peer-reviewed scientific articles and webpages whose proposal was to identify and correct possible fake news. By mapping more than 7,000 interactions between March 2020 and January 2021, we observed that inquiries about statistics decreased, and those related to the results of detection tests and symptoms of the disease increased. This change in user behavior corresponded to the substantial increase in the number of cases and deaths from COVID-19 in Brazil and with the potential approach of the disease to the circles of people close to the users.
The present work is grounded in empirical analysis related to the information required and replies using a chatbot. We consider as related works those recent research papers focused on proposing a chatbot for health care, AI-based healthcare, e-health chatbots, and NLP solutions in the scope of interaction between user and machine. Using the social network platform Facebook, Brixey et al. [12] created SHIHbot, a chatbot committed to providing information about sexual health questions on HIV/AIDS. The authors created a data repository of questions and their respective answers from the Center for Disease Control and the New York State Department of Health HIV. The results and discussion were focused on the proposed architecture lacking analysis of the users’ interactions and mentioning the drawbacks posed by the usage of the Facebook Application Programming Interface (API).
Dealing with health subjects, some human aspects related to the expression of sympathy and empathy can soften the angst for information. This topic was addressed by Liu and Sundar [13]. Considering that machines are unable to express affective or cognitive empathy, the authors explored the outcomes provided by chatbots and human-user interaction in two different experiments to evaluate individuals’ perceptions of the service and the chatbot, such as a Turing Test. The results prove that human users are more pleasant and detectable from chatbot replies. Similarly, Powell [14] raised the discussion about the Turing test on user-facing artificial intelligence systems in health care. Arguments reinforced by the demand for value judgments and the doctor-patient relationship grounded in empathy and understanding to arrive at a shared decision across uncertainty and balancing competing risks brought important insights. Chatbots, mainly in health care, need to have more wisdom than intelligence. Finally, Chung and Park [11] proposed a framework for constructing chatbot solutions particularly related to AI-based healthcare. The authors went through important points from an information system perspective, delivering layers of data interoperability, and creating a cloud computing-based environment for health care.
Research addressing topics related to chatbots on COVID-19 and coronavirus has appeared since the beginning of the pandemic period [15]. Judson et al. [16] mentioned the importance of creating automatic solutions to support healthcare workers in dealing with COVID-19 symptoms and exposures prior to every clinical shift. The authors developed a chatbot using the Conversa platform with an additional challenge, the demand for fast deployment. Using a mobile interface, the proposed solution was able to answer doubts and collect subject health situations to help in contamination detection, enabling physical distancing, and providing real-time workforce data for health system leaders. Focusing on the opportunities offered by telemedicine, Barthi et al. [17] proposed a multilingual conversational bot based on NLP to provide primary healthcare education, information, and advice to chronic patients. The solution, called MedBot, was built using Google Cloud technology with several information system integrations; in particular, the authors mentioned WhatsApp and different integrations using web services. The development and deployment of other chatbot experiences using a deep learning-based NLP have been reported by Lee, Kang and Yeo [18].
A chatbot named Symptoma was compared to phone hotlines created by the public service to support the COVID-19 diagnosis toward reducing the traditional drawbacks of phone calls. The proposed solution achieved an accuracy of 96.32% in distinguishing COVID-19 from other pathologies [19]. In fact, in a comparison study of web-based COVID-19 symptom checkers, Symptoma showed the highest sensitivity and specificity [20]. Maniou and Veglis [21] proposed a chatbot architecture based on new articles to support the user looking for reliable news. User reactions to COVID-19 chatbots have been explored by Dennis et al. [22], and the authors concluded that when chatbots are perceived to provide the same service quality as human agents, users are more likely to see them as persuasive, be more satisfied, and be more likely to use them. In addition to chatbots, which provide reliable and up-to-date information to reduce the social impact of the pandemic, alternative therapies (e.g., therapeutic filmmaking, virtual reality, music therapy) utilizing low-cost technologies have been proposed as home-based solutions to mitigate the impact on physical and mental well-being during the COVID-19 lockdown [23]. By serving as a complementary tool alongside the mentioned therapies, chatbots provide information, guidance, and support to individuals seeking mental health during the COVID-19 lockdown, while their integration with these alternative strategies allows for the development of a comprehensive and holistic approach to address the well-being of individuals during this challenging period.
Our findings have implications both for the development of public health policies and for the application of health technologies related to COVID-19. First, interactions operated by artificial intelligence, such as CoronaAI, can be beneficial to the general population by increasing general knowledge about pandemic data and individually to the citizens themselves by contrasting fake news and correcting erroneous or inaccurate recommendations. This implies a potential saving of public resources since part of the funds that would be destined for these purposes can be used in other measures to address the pandemic, such as the acquisition of health equipment and supplies or the expansion and maintenance of hospital infrastructures. Second, considering that it was possible to detect a change in user behavior as COVID-19 spread and news was disseminated, it could be suggested that this type of technology has sufficiently dynamic applicability and user acceptance to deliver the most needed information in with a reasonably short update time. Finally, considering that chatbots are available in widely used applications, such as WhatsApp, public policy developers, health service managers and other stakeholders would have with these technologies a friendly and useful alternative channel for information and health education to answer citizens' questions about COVID-19, even for those with lower digital literacy.
5. Conclusion
In conclusion, our findings contribute to the state of the art by showing that the maintenance and constant updating of a machine learning tool made available on a friendly and widely used platform such as WhatsApp has the potential to generate public health benefits. These positive impacts may occur both at the collective level, by addressing the parallel epidemic of misinformation and fake news, and at the individual level, by clarifying concrete doubts about people's COVID-19 screening and prevention measures. Empowering individuals to make well-informed decisions based on reliable information holds the potential to diminish the harmful impact of misinformation and, consequently, mitigate the negative social consequences of COVID-19. In this regard, educational institutions and online platforms should collaborate to develop comprehensive programs that promote media literacy, fact-checking techniques (such as the CoronaAI chatbot), and responsible sharing practices. Moreover, fostering collaboration between social media platforms and public health authorities is essential for reducing the spread of fake news and promoting critical thinking skills necessary to discern reliable sources. We believe that the integration of artificial intelligence solutions into daily life, particularly in the context of public health scenarios, has taken place naturally. However, ensuring successful adoption requires the implementation of robust ethical constraints, a thorough comprehension of the potential benefits and risks involved, and continuous monitoring of its outcomes.
6. Summary points
Previous research on the topic revealed that chatbot artificial intelligence tools provide an alternative for addressing inquiries related to COVID-19 preventive measures and symptoms. Additionally, these solutions have the potential to alleviate the burden on healthcare systems by countering the spread of fake news and misinformation during public health crises. This study contributes by revealing the changing behavior of users of the CoronaAI chatbot throughout the course of the COVID-19 pandemic, particularly as it approached individuals in close proximity to the users. It underlines the importance of regularly expanding and updating the chatbot's content on a regular basis to ensure its credibility and effectiveness. Moreover, by being available on a widely used and user-friendly platform such as WhatsApp, tools such as CoronaAI could serve as a feasible alternative for individuals with limited digital literacy and health knowledge, including the elderly population.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.ijmedinf.2023.105134.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
References
- 1.Singhal T. A Review of Coronavirus Disease-2019 (COVID-19) Indian J Pediatr. 2020;87:281–286. doi: 10.1007/s12098-020-03263-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Xu X.-W., Wu X.-X., Jiang X.-G., et al. Clinical findings in a group of patients infected with the 2019 novel coronavirus (SARS-Cov-2) outside of Wuhan, China: retrospective case series. BMJ. 2020;368 doi: 10.1136/bmj.m606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hu Z., Yang Z., Li Q., et al. The COVID-19 Infodemic: Infodemiology Study Analyzing Stigmatizing Search Terms. J Med Internet Res. 2020;22:e22639. doi: 10.2196/22639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Quandt T., Frischlich L., Boberg S., et al. Fake News. The International Encyclopedia of Journalism Studies. 2019:1–6. doi: 10.1002/9781118841570.iejs0128. [DOI] [Google Scholar]
- 5.Apuke O.D., Omar B. Fake news and COVID-19: modelling the predictors of fake news sharing among social media users. Telemat Inform. 2021;56 doi: 10.1016/j.tele.2020.101475. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Koonin L.M., Hoots B., Tsang C.A., et al. Trends in the Use of Telehealth During the Emergence of the COVID-19 Pandemic - United States, January-March 2020. MMWR. 2020;69:1595–1599. doi: 10.15585/mmwr.mm6943a3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Chowdhury G.G. Natural language processing. Annual Review of Information Science and Technology. 2003;37:51–89. doi: 10.1002/aris.1440370103. [DOI] [Google Scholar]
- 8.Banerjee S, Kapetanaki AB, Dempsey L. January. Older People’s Online Information Search During the Pandemic. In 16th IMCOM. IEEE. 2022;2022:1–6. doi: 10.1109/IMCOM53663.2022.9721773. [DOI] [Google Scholar]
- 9.Fernández-Ardèvol M., Rosales A. Smartphone Cultures. Routledge; London: 2017. Older people, smartphones and WhatsApp; pp. 55–68. [Google Scholar]
- 10.Choudrie J., Banerjee S., Kotecha K., et al. Machine learning techniques and older adults processing of online information and misinformation: A covid 19 study. Comput Human Behav. 2021;119 doi: 10.1016/j.chb.2021.106716. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Chung K., Park R.C. Chatbot-based heathcare service with a knowledge base for cloud computing. Cluster Computing. 2019;22:1925–1937. doi: 10.1007/s10586-018-2334-5. [DOI] [Google Scholar]
- 12.Brixey J, Hoegen R, Lan W, et al. {SHIH}bot: A {F}acebook chatbot for Sexual Health Information on {HIV}/{AIDS}. In: Proceedings of the 18th Annual {SIG}dial Meeting on Discourse and Dialogue. Saarbrücken, Germany: Association for Computational Linguistics 2017. 370–3. doi:10.18653/v1/W17-5544.
- 13.Liu B., Sundar S.S. Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot. Cyberpsychol Behav Soc Netw. 2018;21(10) doi: 10.1089/cyber.2018.0110. [DOI] [PubMed] [Google Scholar]
- 14.Powell J. Trust me, I’m a Chatbot: How Artificial Intelligence in Health Care Fails the Turing Test. J Med Internet Res. 2019;21:e16222. doi: 10.2196/16222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Almalki M., Azeez F. Health Chatbots for Fighting COVID-19: a Scoping Review. Acta Inform Med. 2020;28:241–247. doi: 10.5455/aim.2020.28.241-247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Judson T.J., Odisho A.Y., Young J.J., et al. Implementation of a digital chatbot to screen health system employees during the COVID-19 pandemic. J Am Med Inform Assoc. 2020;27:1450–1455. doi: 10.1093/jamia/ocaa130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bharti U, Bajaj D, Batra H, et al. Medbot: Conversational Artificial Intelligence Powered Chatbot for Delivering Tele-Health after COVID-19. 2020. doi:10.1109/ICCES48766.2020.9137944.
- 18.Lee H., Kang J., Yeo J. Medical Specialty Recommendations by an Artificial Intelligence Chatbot on a Smartphone: Development and Deployment. J Med Internet Res. 2021;23:e27460. doi: 10.2196/27460. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Martin A., Nateqi J., Gruarin S., et al. An artificial intelligence-based first-line defence against COVID-19: digitally screening citizens for risks via a chatbot. Sci Rep. 2020:10. doi: 10.1038/s41598-020-75912-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Munsch N., Martin A., Gruarin S., et al. Diagnostic Accuracy of Web-Based COVID-19 Symptom Checkers: Comparison Study. J Med Internet Res. 2020;22:e21299. doi: 10.2196/21299. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Maniou T.A. Design. Implementation and Evaluation; Employing a Chatbot for News Dissemination during Crisis: 2020. Veglis A; p. 12. [Google Scholar]
- 22.Dennis A.R., Kim A., Rahimi M., et al. User reactions to COVID-19 screening chatbots from reputable providers. J Am Med Inform Assoc. 2020;27:1727–1731. doi: 10.1093/jamia/ocaa167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Radanliev P., De Roure D. Alternative mental health therapies in prolonged lockdowns: narratives from Covid-19. Health Technol. 2021;11:1101–1107. doi: 10.1007/s12553-021-00581-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.