Enhancing risk and crisis communication with computational methods: A systematic literature review

Abstract

Recent developments in risk and crisis communication (RCC) research combine social science theory and data science tools to construct effective risk messages efficiently. However, current systematic literature reviews (SLRs) on RCC primarily focus on computationally assessing message efficacy as opposed to message efficiency. We conduct an SLR to highlight any current computational methods that improve message construction efficacy and efficiency. We found that most RCC research focuses on using theoretical frameworks and computational methods to analyze or classify message elements that improve efficacy. For improving message efficiency, computational and manual methods are only used in message classification. Specifying the computational methods used in message construction is sparse. We recommend that future RCC research apply computational methods toward improving efficacy and efficiency in message construction. By improving message construction efficacy and efficiency, RCC messaging would quickly warn and better inform affected communities impacted by current hazards. Such messaging has the potential to save as many lives as possible.

1. INTRODUCTION

Risk and crisis communication (RCC) is a powerful tool for improving hazard preparedness. RCC encourages individuals to take protective actions to keep themselves and their community safe (Fischhoff & Downs, 1997; Jones et al., 2014; Reinhold, Munro, et al., 2023; Reynolds & Seeger, 2005). Such communication encompasses both risk messaging deployed before a hazard and crisis messaging deployed during a disaster. Effective RCCs motivate as many individuals as possible in a target population to adopt protective actions.

Efficacy (definition in Table 1) is essential in RCC. Achieving messaging efficacy involves bridging knowledge gaps between hazard domain experts and affected populations (Reinhold, Raile, et al., 2023). Bridging such gaps relies on how the message is developed. This development frames RCC objectives in ways such that affected populations are receptive to and engaged with the messaging. Receptivity and engagement motivate affected populations to take action to protect themselves from a hazard (Heath et al., 2018; Shanahan et al., 2018).

TABLE 1.

Definitions for key terms used in this systematic literature review (SLR), as defined by the authors and other sources.

Term	Definition	Definitions inferred from other sources
Efficacy	The extent to which risk messaging changes individuals’ risk perceptions and mitigation behavior when faced with a hazard	Measuring persuasive outcomes of risk messages, including changes in attitudes, behaviors, intentions, and knowledge of individuals (Vos et al., 2018) Risk communication that incorporates honesty, reassurance, and actionable items in its messaging, usually guided by a risk communication framework (Abrams et al., 2022)
Efficiency	Combines the speed with which messages can be created with optimal use of resources	A general set of standards, procedures, guidelines, norms, reference points, or principles that are designed to improve performance (Sellnow et al., 2008) The ability for a computational method to complete its function that is both computationally inexpensive and scalable for large‐scale data (Li et al., 2017) Improvements to work or task performance through automation and aggregation of tasks (Wang et al., 2019)

RCC also relies on the timely delivery of messages (Lwin et al., 2018). Messages disseminated promptly ensure individuals have as much lead time as possible to prepare for a risk or to respond during a crisis. For this reason, the time it takes to develop a message is an important concern. Efficient RCCs are created with optimal use of computational resources and time.

Efficiency (definition in Table 1) is a crucial aspect of RCCs. Achieving efficiency involves constructing RCC messages using semi‐automated or fully automated computational methods (Karinshak et al., 2023). Such methods enable shorter message development timeframes compared to manually constructing messages, thus resulting in timelier distribution of messages to populations impacted by hazards. Successful RCC depends on how researchers create effective and efficient messaging.

Recent developments in RCC research blend the worlds of psychology, policy process, and data science to construct effective risk messages efficiently (Gore et al., 2024; Reinhold, Munro, et al., 2023). Although there is growing emphasis on improving message efficacy using computational methods (Nelson, 2020; Reinhold, Raile, et al., 2023), RCC research prioritizes social science aspects informing effective message development. This prioritization is specifically on analysis of effective message elements (Bartolucci et al., 2023; Fathollahzadeh et al., 2023; Hannes & Thyssen, 2022).

Computational methods for analyzing message efficacy are well researched (Guetterman et al., 2018; Ogie et al., 2018), but effective computational message construction is largely ignored (Reinhold, Raile, et al., 2023). This is not to say that message efficacy has been overlooked, but that the use of computational methods improving message efficacy is limited. In addition, little research attention has been paid to efficient message construction—both in using computational or manual methods. One reason for this gap in efficiency research is because computational methods have recently become prominent in the last few years (Kalyan, 2024). Overall, research into computational message construction is sparse, a problem that is reflected in systematic literature reviews (SLRs) on RCC research (Bartolucci et al., 2023; Fathollahzadeh et al., 2023; Hannes & Thyssen, 2022; Ogie et al., 2018).

Existing SLRs aggregate research on message efficacy, specifically on manual (Bartolucci et al., 2023; Fathollahzadeh et al., 2023; Hannes & Thyssen, 2022) or computational (Ogie et al., 2018) classification and analysis of message elements. However, none of them focus on message construction. To date, no SLRs investigate methods used in message construction for improving efficacy, nor do any SLRs investigate methods for improving the efficiency of message construction.

To address these critical research gaps, we conduct an SLR to investigate what, if any, research focuses on efficacy and efficiency in message construction. We highlight existing gaps present in research on computational message construction. Further, we highlight usages of computational methods in RCC research. The next section details the methodology this SLR coincides with.

2. METHODS

The methodology for this SLR aligned with the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses¹ (PRISMA) guidelines (Page, McKenzie, et al., 2021; Page, Moher, et al., 2021). We primarily adhered to guideline steps aligning with data synthesis. These steps include developing research questions (RQs), developing a search strategy, assessing eligibility, performing data meta‐analysis, and selecting our final literature to report.

2.1. Guideline selection and research questions

The first step in PRISMA facilitated the development of RQs on computational methods used in message construction. The following RQs drove the direction of this SLR:

1. What established and emerging computational methods improve efficacy in RCC message construction?

2. What established and emerging computational methods improve efficiency in RCC message construction?

3. What are the trade‐offs between efficient and effective computational methods for message construction?

All three RQs reflect our research attention on effective and efficient computational methods used in RCC. After RQ development, we focused on the next step in the PRISMA guidelines: developing a search strategy.

2.2. Selection, search, and screening

2.2.1. Database selection and search

We constructed search strings from both Boolean logical operators and terms derived from the defined RQs. Terms within the strings cover RCC, messaging, and computational methods. Search string formatting varied across selected databases (Table 2).

TABLE 2.

Searched databases and respective strings used to query results.

Database used	Search string	Filters applied	Results	Date searched
ACM Digital Library	[[All: “risk communication”] OR [All: “crisis communication”] OR [All: “hazard communication”]] AND [All: messag*] AND [[All: analysis] OR [All: computational analysis] OR [All: “natural language processing”] OR [All: “nlp”] OR [All: “artificial intelligence”] OR [All: “ai”]] AND [E‐Publication Date: (01/01/2018 TO 12/31/2023)]	Research articles only	138	2/7/2024
IEEE Xplore	(“Full Text & Metadata”:“risk communication” OR “Full Text & Metadata”: “crisis communication" OR “Full Text & Metadata”:“hazard communication”) AND (“Full Text & Metadata”: messag*) AND (“Full Text & Metadata”: analysis OR “Full Text & Metadata”: computational analysis OR “Full Text & Metadata”: “natural language processing” OR “Full Text & Metadata”: “nlp” OR “Full Text & Metadata”: “artificial intelligence” OR “Full Text & Metadata”: “ai”)	Published between 2018 and 2023, journals and conferences only	166	2/7/2024
PubMed	((“risk communication” OR “crisis communication” OR “hazard communication”) AND (messag*)) AND (analysis OR computational analysis OR “natural language processing” OR “nlp” OR “artificial intelligence” OR “ai”)	Published between 2018 and 2023	144	2/7/2024
Web of Science	((AB=(“risk communication” OR “crisis communication” OR “hazard communication”)) AND AB=(messag*)) AND AB=(analysis OR computational analysis OR “natural language processing” OR “nlp” OR “artificial intelligence” OR “ai”)	Published between 2018 and 2023, articles only	192	2/7/2024

Note: The table also mentions any filters applied to results, the number of results after applying filters, and the search date for each database consulted. The search strings differed from each other since each database utilized different query formats. Any *’s present in search strings indicated that the term preceding it was stemmed.

We utilized the research databases IEEE Xplore,² ACM Digital Library,³ Web of Science,⁴ and PubMed ⁵ to find relevant RCC message literature. IEEE Xplore and ACM Digital Library were chosen for their collection of multidisciplinary computer science research; Web of Science and PubMed were chosen for their collection of multidisciplinary social science research. The database searches uncovered 1385 potentially relevant literature on message construction (548 from IEEE Xplore, 299 from ACM Digital Library, 313 from Web of Science, and 225 from PubMed).

2.2.2. Results screening

Literature identified from each database underwent automatic and manual screening. Filters for publishing year and publication type narrowed the number of articles down to 640. Filtered articles had their titles and DOI links web scraped into dataframes corresponding to the database the article was identified from. The scraping was done both manually and using the R ⁶ package rvest.⁷ R was also used for postprocessing to filter out duplicate and invalid entries in each dataframe, reducing the number of articles from 640 to 627. Articles selected after the screening were subsetted and dispersed to six reviewers recruited for manual evaluation.

2.3. Manual eligibility analysis

2.3.1. Inclusion criteria

We developed inclusion criteria to assess the relevance of the screened literature (Table 3). The criteria specified which topics and characteristics research‐relevant literature needed to include. For example, a relevant literature source needed to discuss computational RCC messaging, have an English publication, be published between January 2018 and December 2023, and be a primary literature source. Such characteristics were chosen to ensure that the most recent, state‐of‐the‐art, and high‐impact literature was captured. The criteria assisted reviewers with assessing article relevance based on their abstracts.

TABLE 3.

Inclusion and exclusion criteria developed for the systematic literature review (SLR).

Criteria	Status
Study/Research topic is on risk, crisis, or hazard communication	Inclusion
Article covers computational or mixed methods used in research on the above topic	Inclusion
Article covers computational or mixed methods used specifically in risk message construction/development	Inclusion
Article is published between 2018 and 2023	Inclusion
Article is published in English	Inclusion
Article is a research article from primary literature	Inclusion
Article is not retracted, outdated, or pre‐published	Inclusion
Article comes from a journal or conference proceeding	Inclusion
Article does not meet all the above criteria	Exclusion

2.3.2. Abstract reviews

Disbursement of literature to reviewers occurred as follows. For this study, the lead author reviewed 317 abstracts, 2 co‐authors reviewed 100, and 1 co‐author reviewed 70. Two additional reviewers each read 20 abstracts to lessen the workload of the main reviewers. Three reviewers with RCC knowledge took on most of the abstract review load (at least 100 abstracts), and the remaining three were doctoral students; every reviewer had knowledge about computational sciences. All reviewers worked independently of each other with no overlap, meaning that no ties among reviewers could occur.

Prior to evaluating abstracts, all reviewers received a copy of the inclusion criteria (Table 3). Reviewers read each article's abstract to determine research relevance. If the abstract was unclear, reviewers assessed the introduction for relevance. Reviewers recorded each criterion that each article met in their spreadsheets. Criteria recorded as “YES” indicated that the article was potentially relevant; criteria recorded as “NO” indicated that the article should be excluded from the SLR. Reviewers returned their spreadsheets to the lead author after completion. The lead author did a brief quality check to ensure the spreadsheets were filled in properly, removing any duplicate entries. A total of 124 articles met the inclusion criteria and thus formed our corpus. These articles underwent meta‐analysis to further assess relevance.

2.4. Meta‐analysis

We conducted the meta‐analysis primarily using the R package tidytext.⁸ As part of the meta‐analysis, titles and hyperlinks for our corpus of articles were aggregated into one dataframe. Each entry contained full text that was manually scraped from corresponding hyperlinks. The text underwent data preprocessing and cleaning before text analysis and term tokenization. Tokenized terms classified as stopwords (e.g., “and,” “the,” and “2020”) were filtered out before calculating term frequencies.

We calculated the frequency of each term across all articles by dividing the occurrence of each term in each by the total number of terms present in the same article. Term frequency calculations were foundational to calculating term frequency‐inverse document frequency⁹ (TF‐IDF) scores for terms. The top 50 TF‐IDF scores corresponded to the terms used most in our corpus (Figure 1). We then manually tagged terms that were relevant to our RQs (e.g., “communication,” “computational,” and “efficacy”; herein, “research‐relevant terms”). Terms with both a high TF‐IDF score and relevance to our RQs determined the next selection of literature. In total, 51 articles from our corpus contained research‐relevant terms and were thus selected to undergo manual quality assessment.

FIGURE 1.

Top 50 terms based on highest term frequency‐inverse document frequency (TF‐IDF) scores across our corpus of articles. Research‐relevant terms are shaded dark blue. Determining research relevance involved manual tagging of terms in R.

2.5. Article quality assessment

Selected articles underwent manual quality assessment of topical findings and credibility. Regarding topical findings, final inclusion criteria required that each article presented quantitative assessments of computational methods improving RCC message efficacy and/or efficiency. If an article did not present such quantitative assessments, the article was excluded. Regarding credibility, final inclusion criteria required that the presentation of the article was coherent and well‐reasoned. If grammatical errors, misspellings, illogical structure, or vagaries prevented the lead author from understanding the study, the article was excluded. Based on these criteria, 25 articles were excluded from the study. Twenty‐six articles made up the final selection of literature on computational risk message construction (Figure 2 shows how many articles were filtered out for each step in the SLR; also see Table 4 for article titles, authors, and topics addressed in the final selection of literature). Findings are summarized in the next section.

FIGURE 2.

Flowchart detailing each step of the systematic literature review (SLR) process for selecting relevant literature. The figure is based on the flowchart structure specified in the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines.

TABLE 4.

Final selection of literature on computational risk message construction identified in this systematic literature review (SLR).

Article title	Authors	Article topic
Arabic Twitter Corpus for Crisis Response Messages Classification	Adel and Wang (2020)	Researchers developed a corpus in the Arabic language to classify crisis communication messages/tweets based on what crisis category they fell under
Comparing the Effectiveness of Text‐based and Video‐based Delivery in Motivating Users to Adopt a Password Manager	Albayram et al. (2021)	Researchers compared the efficacy of text‐ and video‐based risk communication for motivating users to adopt password managers
The Saudi Ministry of Health's Twitter Communication Strategies and Public Engagement During the COVID‐19 Pandemic: Content Analysis Study	Alhassan and AlDossary (2021)	Researchers aimed to evaluate the Saudi Arabia Ministry of Health's use of Twitter and the public's engagement during different stages of the COVID‐19 pandemic in Saudi Arabia. They classified tweets based on the CERC framework
Hacked Time: Design and Evaluation of a Self‐Efficacy Based Cybersecurity Game	Chen et al. (2020)	Researchers developed a game that provided an interactive risk communication approach to improve risk perception of cybersecurity threats and self‐efficacy in users affected
Platform Effects on Public Health Communication: A Comparative and National Study of Message Design and Audience Engagement Across Twitter and Facebook	DePaula et al. (2022)	Researchers analyzed risk communication messages dispersed by government accounts on Facebook and Twitter, specifically looking at how the public engaged with these messages
Emotionality in COVID‐19 crisis communication from authorities and independent experts on Twitter	Drescher et al. (2023)	Researchers analyzed the sentiment (negative, neutral, or positive) of tweets from German health organizations during the early stages of COVID‐19
Knowing Your Audience: A Typology of Smoke Sense Participants to Inform Wildfire Smoke Health Risk Communication	Hano et al. (2020)	This study explored perspectives on wildfire smoke as a health risk among participants of Smoke Sense, a citizen science project with an objective to engage affected individuals on wildfire smoke. Researchers then developed effective health risk communication strategies to motivate individual‐level behavior change
Developing a gist‐extraction typology based on journalistic lead writing: A case of food risk news	Ju and You (2018)	This study aimed to construct a journalistic gist extraction typology to improve the development of risk communication messages. Researchers aimed to translate expert jargon into a format that was easy to read and digest for the public at large
Validation of mobile phone text messages for nicotine and tobacco risk communication among college students: A content analysis	Khalil et al. (2018)	Researchers constructed text messages for tobacco risk communication based on three main structures: framing (gain‐ or loss‐framed messages), depth (simple or complex messages), and appeal (emotional or rational messages)
Canadian COVID‐19 Crisis Communication on Twitter: Mixed Methods Research Examining Tweets from Government, Politicians, and Public Health for Crisis Communication Guiding Principles and Tweet Engagement	MacKay, Cimino et al. (2022)	This study described how crisis actors used guiding principles in COVID‐19 tweets and how the use of these guiding principles related to tweet engagement. Researchers classified tweets based on said guiding principles
Examining Social Media Crisis Communication during Early COVID‐19 from Public Health and News Media for Quality, Content, and Corresponding Public Sentiment	MacKay et al. (2021)	Researchers aimed to evaluate the quality and content of Canadian public health and news media crisis communication during the first wave of the COVID‐19 pandemic on Facebook and the subsequent emotional response to messaging by the public
A content analysis of Canadian influencer crisis messages on Instagram and the public's response during COVID‐19	MacKay, Ford et al. (2022)	Researchers examined COVID‐19‐related crisis messages across Canadian influencer accounts on Instagram to examine their efficacy based on message constructs outlined by the Health Belief and Extended Parallel Processing models. Researchers also analyzed audience sentiment
Machine Learning Framework for Analyzing Disaster‐Tweets	Manimegalai et al. (2023)	This study analyzed the performance of computational classifier models when classifying types of disaster crisis tweets
Build community before the storm: The National Weather Service's social media engagement	Olson et al. (2019)	This study examined crisis communication on social media by observing how 12 National Weather Service (NWS) offices used Twitter to facilitate engagement with stakeholders during threat and nonthreat periods
Narrative Risk Communication as a Lingua Franca for Environmental Hazard Preparation	Raile et al. (2022)	Researchers developed a new risk communication framework that guides the construction of risk messages both using narrative structure and invoking narrative transport
Investigating the presentation of uncertainty in an icon array: A randomized trial	Recchia et al. (2022)	Researchers analyzed the efficacy of visual risk communication about the risks of breast and ovarian cancer for individuals carrying the BRCA1 pathogenic variant
User‐Generated Crisis Communication: Exploring Crisis Frames on Twitter during Hurricane Harvey	Riddell and Fenner (2021)	Researchers analyzed user‐generated crisis communication—as well as crisis communication distributed by organizations—to get a well‐founded understanding of how the public views risks and crises and what information was sought after
Communicating risk of medication side‐effects: role of communication format on risk perception	Sawant and Sansgiry (2018)	This study assessed the interaction effects of message format and contextual factors (rate of occurrence and severity) on risk perception of medication side‐effects after considering message format and contextual factors influencing risk perception
Characters matter: How narratives shape affective responses to risk communication	Shanahan et al. (2019)	Researchers analyzed the use and effectiveness of narrative elements in flood risk messaging. Subsequent messages were constructed, aiming to improve individual affective response and changes in intended behavior and risk perception
A machine learning approach to flood severity classification and alerting	Sharma et al. (2021)	Researchers leveraged several machine learning models and assessed their performance at classifying flood risk message types (advisory, information, warning, and watch)
Examining Tweet Content and Engagement of Canadian Public Health Agencies and Decision Makers During COVID‐19: Mixed Methods Analysis	Slavik et al. (2021)	This study examined the content and engagement of COVID‐19 tweets authored by Canadian public health agencies and decision makers, making suggestions on how to improve the efficacy of crisis communication based on the results
Qualitative analysis of visual risk communication on twitter during the Covid‐19 pandemic	Sleigh et al. (2021)	Researchers investigated how visual risk communication was used on Twitter to promote the World Health Organization's (WHO) recommended preventative behaviors and how this communication changed over time
Story mapping and sea level rise: listening to global risks at street level	Stephens and Richards (2020)	This study described the development of an interactive tool that juxtaposed coastal residents’ video‐recorded stories about sea level rise and coastal flooding with an interactive map that showed future sea level rise projections
An application of the extended parallel process model to protective behaviors against COVID‐19 in South Korea	Yoon et al. (2022)	This study applied the EPPM to understand factors that affect an individual's participation in protective behaviors against COVID‐19. Such factors included the effect of public perception of threat, the efficacy of fatalism, and undertaking protective behaviors
Sharing health risk messages on social media: Effects of fear appeal message and image promotion	Zhang and Zhou (2020)	This study examined how fear appeal and individuals’ image promotion consideration drove users’ intentions to share fear appeal messages on social networking sites
Understanding motivated publics during disasters: Examining message functions, frames, and styles of social media influentials and followers	Zhao et al. (2019)	Researchers analyzed how different message functions in risk and crisis communication were employed by Twitter users, both general users and popular influencers, using the Ariana Grande concert bombing event as the hazard subject

Note: Included are the 26 article titles, authors, and the topics discussed therein.

Abbreviation: CERC, Crisis and Emergency Risk Communication; EPPM, Extended Parallel Process Model.

3. RESULTS

We report the final selection of literature identified from the SLR. Findings presented within the first three subsections provide answers to our three RQs. These RQs ask what computational methods improve message construction efficacy and efficiency as well as what trade‐offs between them exist. We also present findings from the selected literature discussing other applications of computational methods in RCC.

3.1. Computational methods and message construction efficacy

Risk and crisis message construction utilizing computational methods is rarely discussed in RCC research. This SLR identified seven studies explicitly discussing computational methods used in RCC message construction (Chen et al., 2020; Khalil et al., 2018; Raile et al., 2022; Recchia et al., 2022; Sawant & Sansgiry, 2018; Shanahan et al., 2019; Stephens & Richards, 2020). Furthermore, studies on computational methods used in message construction discussed only efficacy, not efficiency.

Studies on message efficacy focused on combining social science theory and computational tools to construct risk messages (Chen et al., 2020; Raile et al., 2022; Sawant & Sansgiry, 2018; Shanahan et al., 2019), to analyze risk messages (Albayram et al., 2021; Alhassan & AlDossary, 2021; DePaula et al., 2022; Drescher et al., 2023; Ju & You, 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Olson et al., 2019; Riddell & Fenner, 2021; Slavik et al., 2021; Sleigh et al., 2021; Zhao et al., 2019), or to analyze audience interaction with RCC messaging (Albayram et al., 2021; DePaula et al., 2022; Hano et al., 2020; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Riddell & Fenner, 2021; Slavik et al., 2021; Yoon et al., 2022; Zhang & Zhou, 2020; Zhao et al., 2019). Natural language processing (NLP) and content analysis were the most prevalent computational methods addressed for message construction, message analysis, and audience response analysis (Table 5).

TABLE 5.

Effective and efficient computational methods used for risk message construction, classification, and analysis across all selected articles.

Message aspect addressed	Computational methods used	Articles
Message classification efficiency	Random forest, Naïve Bayes, support vector machine, logistic regression, extreme gradient boost, decision tree	Adel and Wang (2020), Manimegalai et al. (2023), Sharma et al. (2021)
Message classification efficacy	Content analysis, natural language processing, random forest, Naïve Bayes, support vector machine, logistic regression, Extreme Gradient Boost, decision tree, cluster analysis, linguistic inquiry, and word count	Adel and Wang (2020), Albayram et al. (2021), Alhassan and AlDossary (2021), DePaula et al. (2022), Khalil et al. (2018), MacKay et al. (2021), MacKay, Cimino et al. (2022), MacKay, Ford et al. (2022), Manimegalai et al. (2023), Olson et al. (2019), Sharma et al. (2021), Slavik et al. (2021)
Message construction efficacy	Content analysis, natural language processing, transformational game design and programming, icon arrays, interactive story map development, linguistic inquiry, and word count	Chen et al. (2020), Khalil et al. (2018), Raile et al. (2022), Recchia et al. (2022), Sawant and Sansgiry (2018), Shanahan et al. (2019), Stephens and Richards (2020)
Message analysis efficacy	Content analysis, chi‐squared analysis, natural language processing, cluster analysis, logistic regression, multiple regression, ANOVA/ANCOVA	Albayram et al. (2021), Alhassan and AlDossary (2021), DePaula et al. (2022), Drescher et al. (2023), Hano et al. (2020), Ju and You (2018), MacKay et al. (2021), MacKay, Cimino et al. (2022), MacKay, Ford et al. (2022), Olson et al. (2019), Riddell and Fenner (2021), Slavik et al. (2021), Sleigh et al. (2021), Yoon et al. (2022), Zhang and Zhou (2020), Zhao et al. (2019)

Note: Some articles discussed more than one aspect of message development.

Computational methods helped operationalize theoretical frameworks for effective RCC messaging. Theoretical frameworks were embedded in codebook creation and generally had two aims: (1) to improve the efficacy of RCC messaging or (2) to identify effective elements in RCC messages. These codebooks were used for message construction (Chen et al., 2020; Khalil et al., 2018; Raile et al., 2022; Sawant & Sansgiry, 2018; Shanahan et al., 2019) and message analysis (Albayram et al., 2021; Alhassan & AlDossary, 2021; DePaula et al., 2022; Drescher et al., 2023; Hano et al., 2020; Ju & You, 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Olson et al., 2019; Riddell & Fenner, 2021; Sleigh et al., 2021; Yoon et al., 2022; Zhang & Zhou, 2020; Zhao et al., 2019). The Crisis and Emergency Risk Communication (CERC) model, the Narrative Policy Framework (NPF), the extended parallel process model (EPPM), and protection motivation theory (PMT) were the most prevalent operationalized theoretical frameworks (Table 6).

TABLE 6.

Theoretical frameworks discussed and implemented in risk message development across all final selections of articles.

Theoretical framework	Article(s)
Crisis and Emergency Risk Communication model	Alhassan and AlDossary (2021), MacKay et al. (2021), MacKay, Cimino et al. (2022)
Extended parallel process model	MacKay, Ford et al. (2022), Yoon et al. (2022)
Fuzzy‐trace theory	Ju and You (2018)
Hermann's crisis model	Riddell and Fenner (2021)
Narrative Policy Framework	Raile et al. (2022), Shanahan et al. (2019)
Narrative Risk Communication Framework	Raile et al. (2022)
Precaution adoption process model	Hano et al. (2020)
Prospect theory	Sleigh et al. (2021)
Protection motivation theory	Albayram et al. (2021), Chen et al. (2020)
Rhormann's risk communication process model	Sawant and Sansgiry (2018)
Self‐efficacy design framework	Chen et al. (2020)
Social media analytics framework	Drescher et al. (2023)
Social‐mediated crisis communication model	Zhao et al. (2019)
Unspecified or combined frameworks	DePaula et al. (2022), Khalil et al. (2018), MacKay et al. (2021), MacKay, Cimino et al. (2022), MacKay, Ford et al. (2022), Olson et al. (2019), Riddell and Fenner (2021), Slavik et al. (2021), Zhang and Zhou (2020)

Note: Some articles addressed or combined multiple frameworks; some frameworks were not specified explicitly.

3.2. Computational methods and message construction efficiency

No studies in this SLR addressed using computational methods to improve message construction efficiency. Any coverage of efficient computational methods analyzed and compared various machine learning models on how well risk messages were classified based on framework or hazard keywords (Adel & Wang, 2020; Manimegalai et al., 2023; Sharma et al., 2021). Support vector machines (SVMs), Extreme Gradient Boost (XGB), Naïve Bayes, and random forest were commonly used for efficient classification of message elements (Table 5).

3.3. Trade‐offs between method efficacy and efficiency

No articles analyzed trade‐offs between efficacy and efficiency in computational message construction. Limitations with effective and efficient computational methods were rarely discussed as well. Rather, discussions mainly focused on a lack of NLP term dictionaries for low‐resource languages (Adel & Wang, 2020) and NLP tools inconsistently analyzing sentiment in text (Drescher et al., 2023).

3.4. Message classification with computational methods

Classification of messages was the predominant application of computational methods in RCC. Computational methods helped identify and classify message elements that were effective in changing individuals’ risk perceptions, mitigation behavior, and self‐efficacy (Adel & Wang, 2020; Albayram et al., 2021; Alhassan & AlDossary, 2021; DePaula et al., 2022; Khalil et al., 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Manimegalai et al., 2023; Olson et al., 2019; Sharma et al., 2021; Slavik et al., 2021). The most prevalent computational methods used for message and element classification were NLP, content analysis, logistic regression, Naïve Bayes, SVMs, and XGB (Table 5). All the above computational methods contributed toward improving the efficacy and efficiency of message classification.

Operationalized theoretical frameworks were also used to classify messages on their efficacy (Alhassan & AlDossary, 2021; DePaula et al., 2022; Ju & You, 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Olson et al., 2019; Slavik et al., 2021; Zhao et al., 2019). The CERC model was the most common theoretical framework operationalized for message classification. Combined or unspecified frameworks were more common in message classification studies than in studies that used computational methods to analyze or construct RCC messages. These combined or unspecified frameworks were covered in nine articles (DePaula et al., 2022; Khalil et al., 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Olson et al., 2019; Riddell & Fenner, 2021; Slavik et al., 2021; Zhang & Zhou, 2020) (Table 6).

4. DISCUSSION

We report current and emerging aspects in RCC research that improve message construction. Discussions on specific methods used to improve message construction efficacy and efficiency, as well as their limitations, are also present in each subsection below. We also share insight into how cultural differences within affected populations can impact RCC message development.

4.1. Theoretical frameworks in risk and crisis communication

Effective RCC depends on the theoretical framework chosen for computational message development. The most commonly referenced frameworks used in message construction and analysis are the NPF (Raile et al., 2022; Shanahan et al., 2018, 2019) and PMT (Albayram et al., 2021; Chen et al., 2020; Maddux & Rogers, 1983), as described in Sections 4.1.1 and 4.1.2. Choosing between the two frameworks depends on how researchers want to motivate individuals to change risk mitigation behavior and risk perception.

4.1.1. Protection motivation theory (PMT)

RCC developed using PMT targets individuals’ fear‐appeal when faced with a hazard through threat and coping appraisal (Boss et al., 2015; Maddux & Rogers, 1983). Fear‐appeal in PMT messages is surmounted when an individual's response efficacy and self‐efficacy outweigh the costs of taking protective action against the hazard communicated. The most common hazard domain applying PMT in messaging is cybersecurity, specifically RCC on increasing and encouraging cybersecure behaviors in users (Albayram et al., 2021; Chen et al., 2020). Visual and interactive messaging, embedded with PMT tenets, improves user self‐efficacy, response efficacy, and changes in cybersecure behavior (Albayram et al., 2021; Chen et al., 2020).

4.1.2. The narrative policy framework (NPF)

The NPF is another theoretical framework used to improve message efficacy. The NPF asserts that narrative elements such as plot, setting, moral, and characters‐in‐action play an important role in the policy process (Shanahan et al., 2018). Specific characters that appear in narratives are heroes, villains, and victims (Shanahan et al., 2018). Hero characters can improve the efficacy of messages created using the NPF (Raile et al., 2022; Shanahan et al., 2019). Furthermore, communicating risk using narratives invokes narrative transportation (Green & Brock, 2000) and makes the messaging more personable and memorable for individuals (Dahlstrom, 2014; Raile et al., 2022; Shanahan et al., 2019; Stephens & Richards, 2020). Messages created with the NPF can heighten affective response and thereby have a greater impact on intended behavior as compared to strict science messages (Raile et al., 2022; Shanahan et al., 2019).

4.1.3. Trade‐offs between PMT and NPF

Inducing individual affective response differs among messages developed using PMT and the NPF. With PMT, negative affective response is influenced using fear‐appeal (Albayram et al., 2021). With the NPF, positive affective response is influenced using character selection (Raile et al., 2022). Risk messages that induce a positive valence of affect motivate individual risk mitigation behavior better than messages that induce a negative valence of affect; however, negative affect appears to have as much of an impact on individual risk perceptions as positive affect (Raile et al., 2022). For crisis communications, the magnitude of affective response may be more important than the valence of affect in crisis messaging because individuals need to take protective actions promptly (Albayram et al., 2021; Chen et al., 2020). However, in risk communications, we posit that the valence of affect may be more important if the messages are deployed frequently.

RCC developed with either framework is effective at inducing affective response. Although messages developed with PMT can induce affective responses, these messages motivate risk mitigation behavior with varying degrees of success (Albayram et al., 2021; Chen et al., 2020). Factors that impact the success of these messages are individual risk perception of a hazard and increased self‐efficacy (Chen et al., 2020). Messages developed using the NPF, on the other hand, motivate risk mitigation behavior consistently when compared to conventional RCC messaging (Raile et al., 2022; Shanahan et al., 2019).

Messaging on a specific hazard, as opposed to a generalized hazard, changes how individuals perceive associated risks. Individuals focusing their attention on one hazard at a time limits cognitive overload and is correlated with changes in perception (Albayram et al., 2021; Chen et al., 2020). The medium through which a message is communicated also impacts risk perception in individuals. Visuals or interactive elements derived from PMT tenets improve response efficacy and self‐efficacy in individuals (Albayram et al., 2021; Chen et al., 2020). Researchers have also developed visual messaging adhering to the NPF, showing such messaging to be as effective at inducing affective responses in individuals as text‐based messaging (Guenther & Shanahan, 2021; Shanahan et al., 2023). By visualizing the risks of a hazard, individuals understand how a given hazard affects them, thus improving risk perception.

4.2. Effective computational methods in risk communication

Applications of and discussions on computational message construction are scant in RCC research. The research focuses on classifying or analyzing RCC messages, specifically message elements like calls to action, hazard information, and visual media (Adel & Wang, 2020; Albayram et al., 2021; Alhassan & AlDossary, 2021; DePaula et al., 2022; Drescher et al., 2023; Hano et al., 2020; Ju & You, 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Manimegalai et al., 2023; Olson et al., 2019; Riddell & Fenner, 2021; Sharma et al., 2021; Slavik et al., 2021; Sleigh et al., 2021; Yoon et al., 2022; Zhang & Zhou, 2020; Zhao et al., 2019). Sometimes, articles include discussions on how analyzed elements improve message construction in the future, but how to construct messages is never specified (Albayram et al., 2021; DePaula et al., 2022; Drescher et al., 2023; Hano et al., 2020; Khalil et al., 2018; MacKay et al., 2021; MacKay, Cimino, et al., 2022; MacKay, Ford, et al., 2022; Manimegalai et al., 2023; Olson et al., 2019; Yoon et al., 2022). Failure to specify how messages are constructed represents a larger problem in message development as a whole, not just computational message construction (Reinhold, Raile, et al., 2023).

4.2.1. Natural language processing (NLP)

Little research delves into using computational methods to improve text‐based messaging (Reinhold, Raile, et al., 2023). When message construction methods are discussed, messages are constructed either with visual or video elements and compared against textual risk messaging (Chen et al., 2020; Recchia et al., 2022; Sawant & Sansgiry, 2018; Stephens & Richards, 2020). From the limited body of work, computational methods used in text‐based message construction are linguistic computer science tools such as Linguistic Inquiry and Word Count (Khalil et al., 2018) or NLP (Raile et al., 2022; Shanahan et al., 2019).

NLP proves to be effective at improving message development. NLP tools have been applied in operationalizing theoretical frameworks (Raile et al., 2022; Reinhold, Raile, et al., 2023; Shanahan et al., 2019) and analyzing message efficacy through audience engagement (Drescher et al., 2023; MacKay et al., 2021; MacKay, Ford, et al., 2022). Messages developed using NLP can automate content analysis (Nelson, 2020; Raile et al., 2022; Shanahan et al., 2019) and help select terms associated with framework elements (Raile et al., 2022; Reinhold, Raile, et al., 2023; Shanahan et al., 2019). Additionally, NLP tools such as sentiment analysis can help determine how individuals respond to RCC messaging by examining the sentiment expressed in RCC messages (Drescher et al., 2023; MacKay et al., 2021; MacKay, Ford, et al., 2022).

4.2.2. Limitations with NLP

Although NLP is a powerful tool for textual analysis, limitations emerge with text classification, sentiment analysis, and topic modeling techniques. All three techniques have difficulties assessing term sentiment, classification, and topic relevance of context‐specific words or sentences (Drescher et al., 2023; Guetterman et al., 2018; Reinhold, Raile, et al., 2023; Silberztein, 2024). In contrast, manual classification and sentiment analysis can contextualize words better than equivalent NLP tools because humans can intuit situational context better than computers (Guetterman et al., 2018; Silberztein, 2024). Difficulties in contextualizing terms can be attributed to generalized word dictionaries used when working with NLP tools.

Sole reliance on generalized word dictionaries contributes to the disparity between manual text analysis and NLP tools. For example, sentiment analysis performed on RCC messages sometimes calculate a negative polarity score (i.e., a message is interpreted to have a negative sentiment) for the whole message even if terms used are “neutral” for the given context (Drescher et al., 2023). Additionally, Part of Speech (POS) tagging with generalized word dictionaries often results in poor accuracy. Typical problems include distinguishing different noun types, ignoring multiword units, and assigning wrong POS tags (Silberztein, 2024). Nuances of human language are challenging for computational methods to fully analyze or classify terms, a problem further exacerbated when applying NLP on low‐resource languages (Farghaly & Shaalan, 2009).

NLP lacks substantial support for low‐resource languages (Ghafoor et al., 2021). Typical strategies take messages written in a low‐resource language (e.g., Arabic) and translate them into a high‐resource language (e.g., English) before applying word classification, term frequency analysis, or sentiment analysis (Adel & Wang, 2020; Farghaly & Shaalan, 2009; Ghafoor et al., 2021). Translating the original text often loses the context or meaning of the message (Ghafoor et al., 2021). However, this is not the only challenge present with NLP for low‐resource languages. These languages can contain linguistic and semantic ambiguities, and some do not adhere to punctuation or capitalization rules present in high‐resource languages (Farghaly & Shaalan, 2009; Ghafoor et al., 2021). Messages deployed lose their effectiveness if the wrong words are chosen or if the language structure is incoherent for message recipients.

4.3. Efficient computational methods in risk and crisis communication

RCC research largely overlooks applications of efficient computational methods in message development. Current applications of computational message construction, while improving construction and message efficacy, are time‐ and resource‐consuming (Reinhold, Munro, et al., 2023). Hence, researchers focus their attention toward large language models (LLMs) for efficient message construction (Karinshak et al., 2023; Lynch et al., 2023; Reinhold, Munro, et al., 2023). Current work on constructing messages using LLMs combines zero‐shot learning and prompt engineering to develop accurate, quality, and impactful messaging (Filippi, 2023; Lim & Schmälzle, 2023; Lynch et al., 2024, 2023). Additionally, advancements in generating non‐textual, multimedia forms of communication through LLMs are also ongoing (Meskó, 2023; Moor et al., 2023).

4.3.1. Limitations with efficiency

LLMs have become popular and powerful tools for text generation and communication research (Kalyan, 2024; Lynch et al., 2023). However, LLMs have technical and ethical limitations with respect to accountability, responsibility, safety, and honest use (Lynch et al., 2023; Sallam, 2023; Stokel‐Walker & van Noorden, 2023; van Dis et al., 2023). LLMs perform well when fed multimodal (Thirunavukarasu et al., 2023) and validated information (Gilbert et al., 2023; Karabacak & Margetis, 2023), and LLM prompts created with explicit guidance can result in consistent, well‐structured outputs (Filippi, 2023). Although these methods are implemented to mitigate concerns over validity, uncertainty, bias, and accountability when generating LLM outputs, challenges still present themselves. On the technical side, generating multiple messages utilizing the same prompt can result in temporal mismatches within the message content across a set of messages (Lynch et al., 2023). Issues also arise when irrelevant contextual or personal information is introduced into the prompt (Lynch et al., 2024; Shi et al., 2023). On the ethical side, LLMs can generate messages embedded with social, scientific, and psychological biases (Hagendorff & Danks, 2023; Zhang et al., 2020) or provide morally inconsistent advice (Krügel et al., 2023). In addition, responses given by popular LLM ChatBots, such as OpenAI's ChatGPT,¹⁰ can contain inaccurate or overgeneralized information, stemming from limited or biased training sets (van Dis et al., 2023). However, solutions are being explored to mitigate the impacts of position, verbosity, and self‐enhancement biases (Zheng et al., 2024).

These limitations with LLM message generation threaten the objectivity and accuracy of RCC messaging. Therefore, it is important that researchers take full consideration of the trade‐offs between timely message deployment and precise message content. With these concerns in mind, our position is that message construction should not be fully reliant on LLMs. Rather, LLM‐generated messages should involve human validation to ensure message content is accurate (Lynch et al., 2024) and to judge whether linguistic nuances are properly reflected (Nasution & Onan, 2024).

4.4. Impact of culture on risk and crisis communication messaging

Computational tools and theoretical frameworks are instrumental for improving RCC message construction efficacy and efficiency. However, variation in cultural contexts impacts the efficacy of computational tools. A significant cultural barrier is the predominance of the English language in computational linguistic tools and dictionaries (Adel & Wang, 2020; Farghaly & Shaalan, 2009; Ghafoor et al., 2021). For all other languages, the tools’ reliance on English requires that text be translated. Because translation can ignore or introduce semantic ambiguities, we posit that RCC messages will be less effective if they require translation. Consequently, we expect that linguistic barriers reduce the efficacy of messages when dispersed to non‐English‐speaking populations. Therefore, language is an important cultural consideration.

Language is not the only cultural barrier that impacts RCC messaging. Culture is inextricably linked to geolocation. For example, Asian countries tend to be more collectivist than Western countries like the United States (Yoon et al., 2022; Zhang, 2021). Collectivism is an example of a cultural factor that can impact message receptivity. Message receptivity is also impacted by intercultural differences such as race, ethnicity, political affiliation and ideology, cultural norms and prevailing personal beliefs and attitudes (e.g., religious and philosophical) (Chen et al., 2020; Raile et al., 2022; Shanahan et al., 2023; Zhang, 2021). Therefore, cultural influences on message receptivity cannot be tackled with computational methods alone because linguistic and other cultural considerations influence message efficacy.

5. THREATS TO VALIDITY

5.1. Construct validity

We identified construct validity as a potential threat. Construct validity refers to the extent to which an instrument or test reflects the construct being investigated (Reinhold, Raile, et al., 2023). Our investigation into computational message construction methods involved both manual and automatic filtering of literature. The use of manual filtering methods can threaten construct validity. Both the abstract reviews and quality assessments used manual filtering methods via reading key findings of a given article and recording criteria for SLR inclusion. The inclusion criteria served as a guide for assessing article relevance. Yet, differences in how reviewers assessed relevance could have introduced some inconsistencies in the inclusion of articles meeting the inclusion criteria in Table 3 by a slim margin.

We selected reviewers that had a solid knowledge base on computational sciences to ensure literature on computational RCC was included in the SLR. Of the six reviewers, all were familiar with RCC, but only half of them were RCC experts. Although it would have been better if all reviewers had been experts in RCC, our stringent and well‐defined inclusion criteria mitigated threats to construct validity resulting from some of the reviewers’ limited expertise in RCC.

In addition, we did not conduct overlapping abstract reviews, meaning distributed articles were unique for each reviewer. Distributing articles with reviewer overlap would have improved the reliability and construct validity of the study. However, the large volume of articles (Figure 2) was a significant undertaking for our team; hence, our decision to not distribute overlapping articles. We perceive threats resulting from this to be limited to articles that were included or excluded by a narrow margin, as mentioned earlier in this section.

Computational filtering methods may have also introduced threats to construct validity. The TF‐IDF analysis potentially threatens construct validity by assuming frequently occurring terms correlate with an article's relevancy to computational risk message construction. Most frequently occurring terms across all vetted articles were not deemed research‐relevant, so it is likely that research‐relevant articles were falsely excluded or included.

Another potential threat to construct validity stems from the database searches, specifically with the search string construction. The strings went through a month‐long refinement process to best capture relevant literature on RCC messaging. However, it is possible that the search strings did not capture all relevant literature on the topic. For example, the studies of Reinhold, Raile et al. (2023) and Lynch et al. (2023) were sources not captured in our SLR despite their discussion of computational RCC message construction. The search strings also yielded some results irrelevant to RCC messaging, which we manually filtered out both in the database search and results screening steps of this SLR (Figure 2).

5.2. External validity

External validity refers to whether results from this study can be generalized beyond the specific research content (Bryman, 2016). We searched for literature on RCC messaging across four distinct databases. For each database, we limited our search to include literature published between the years 2018 and 2023. This range was chosen to best capture any RCC research covering LLMs or NLP. However, our results from this research scope could be too specific to generalize to most RCC research available.

6. CONCLUSION

The primary application of computational methods in RCC is for message classification. Computational methods help classify effective RCC message elements, and similar methods classify RCC messages based on hazard domains efficiently. However, computational methods are seldom used to improve the efficacy and efficiency of message construction. Although some RCC research leverages computational methods to improve message construction efficacy, improving construction efficiency is a nascent area of research but one that is rapidly growing with the maturation of LLMs. Yet, by improving message construction efficacy and efficiency, RCC messaging would have greater potential to quickly warn and better inform affected communities impacted by hazards. Thus, we recommend that future RCC research focus on the development of computational methods for improving efficacy and efficiency in message construction.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

Munro, M. H , Gore, R. J. , Lynch, C. J. , Hastings, Y. D. , & Reinhold, A. M. (2025). Enhancing risk and crisis communication with computational methods: A systematic literature review. Risk Analysis, 45, 1683–1697. 10.1111/risa.17690