Impact of information timeliness and richness on public engagement on social media during COVID-19 pandemic: An empirical investigation based on NLP and machine learning

Abstract

This paper investigates how information timeliness and richness affect public engagement using text data from China's largest social media platform during times of the COVID-19 pandemic. We utilize a similarity calculation method based on natural language processing (NLP) and text mining to evaluate three dimensions of information timeliness: retrospectiveness, immediateness, and prospectiveness. Public engagement is divided into breadth and depth. The empirical results show that information retrospectiveness is negatively associated with public engagement breadth but positively with depth. Both information immediateness and prospectiveness improved the breadth and depth of public engagement. Interestingly, information richness has a positive moderating effect on the relationships between information retrospectiveness, prospectiveness, and public engagement breadth but no significant effects on immediateness; meanwhile, it has a negative moderating effect on the relationship between retrospectiveness and depth but a positive effect on immediateness, prospectiveness. In the extension analysis, we constructed a supervised NLP model to identify and classify health emergency-related information (epidemic prevention and help-seeking) automatically. We find that public engagement differs in the two emergency-related information categories. The findings can promote a more responsive public health strategy that magnifies the transfer speed for critical information and mitigates the negative impacts of information uncertainty or false information.

1. Introduction

Social media, such as Facebook, Twitter, and Weibo, offers an effective channel for the public to engage with information dissemination during major emergencies, such as health crises and natural disasters [15,63,64]. Through social media, the public can understand the current situation about an emergency [69,74], and governments or organizations can also improve their capabilities in processing emergency information and providing better public social services [14,15,24,77]. Due to the openness and participatory nature of social media, it offers vital benefits in information dissemination between citizens during emergencies and brings new ways to engage citizens with emergency management [4].

According to the extant literature, the public is defined as groups of people who are able to affect decisions but who engage with the issues to which decisions pertain through discussion [42]. During health emergencies, the public engages with the emergencies on social media through three behaviors: sharing, commenting, and like [15]. When a post is presented on social media, anyone (e.g., social media agencies, individuals) who has registered can engage with it through these three actions. Thus, public engagement is a theoretically intriguing and practically valuable phenomenon, and social media users are the primary participants of engagement in this study. Encouraging public engagement in emergencies is increasingly important for stakeholders, such as departments of emergency management, social media agencies, and individuals, to alleviate public anxiety and reduce damage [65]. For example, during the 2012 Hurricane Sandy disaster, the United States government used Twitter to engage the public in information service development [14]. Another study analyzed engagement behaviors during COVID-19 by examining individuals' sharing behaviors on social media [15]. They found that individuals' sharing behavior was an important way for the government to know how the relevant COVID-19 message was spreading on social media.

Nevertheless, public social media engagement in emergencies is not always optimistic [15]. First, many social media agencies (e.g., emergency management departments) or individuals post emergency-related information, but few consider how to promote public engagement [71,81]. The interaction between the public on social media often stays on a superficial level, such as simple shares and limited comments [79]. Second, media agencies may experience challenges in employing social media in this way, such as security issues, lack of sufficient information, and the digital divide [28,38].

In recent years, some researchers have attempted to examine how public engagement can be increased during emergencies using content analysis [61]. During an emergency, people are eager to find updated information about the emergency from social media, so information timeliness is an important cause for public concern. Surprisingly, little research has attempted to examine its role in enhancing public engagement. This provides us with a new opportunity to fill this research gap. For this reason, the first research question in this study is: Does information timeliness always promote public engagement during emergencies? In this study, natural language processing (NLP) is used to extract the timeliness of emergency-related information. We choose NLP for the following reasons: first, emergency-related information is mainly in text form on social media, and NLP is widely used to extract meaning from various texts [70]. Second, social media gathers large volumes of emergency-related information. NLP allows researchers to easily extract beneficial insights contained in textual datasets while avoiding burdensome computational work [48]. Third, some studies on health emergencies have utilized NLP to capture individuals' behaviors in the context of social media, for example, Zhou et al. [92] explored the mechanism of COVID-19-related misinformation on social media based on NLP. Chen et al. [15] analyzed the posts about the COVID-19 crisis in order to promote citizen engagement. Li et al. [64] drew on COVID-19 textual data to characterize the propagation of situational Information on social media. Thus, we believe that NLP outpaces other instrumental means to achieve the optimal outcome.

Media richness theory (MRT) indicates that richness can affect information credibility and argument quality [89]. Nowadays, technologies enable people to create and use multimedia content easily. Contents can be posted in various formats on social media, such as text only, text + images, or text + videos. Individuals are more likely to be attracted to that vivid information and repost it [15,92]. Information richness can encourage greater involvement in information processing [88]. Shi et al. [75] found that information richness was positively related to individuals' information dissemination behavior. The extant literature indicated that MRT is based on the information processing model to reduce uncertainty and resolve equivocality [29]; thus, information richness increases individuals' perceived richness regarding textual data. In fact, individuals often post text with multimedia to facilitate the conveyance of interpretation information [23]. As aforementioned, NLP has been widely used to extract meaning from textual data, and information richness can facilitate shared meaning and understanding. Plain text is the leanest medium but cannot provide more information compared to images or videos [57]. Thus, information richness comingles with NLP's distinct strengths in dissecting and synthesizing textual data. Accordingly, this study proposes that information richness, which is defined as “the display format of information on social media,” be explored in the second research of this study: Are the effects of information timeliness on promoting public engagement contingent on information richness?

Since context is considered a critical driver of individuals' behavior [9,41]. In this study, we mainly focus on health emergencies like COVID-19 and the unit of analysis are those people who can post, share, or comment on social media, such as social media agencies and individuals. In addition, emergency-related information posted on social media during health emergencies is our boundary condition. NLP, text mining, and machine learning techniques enable us to further study these research issues. We used a similarity calculation method based on NLP and text mining to evaluate information timeliness; meanwhile, public engagement is divided into breadth and depth. To automatically identify and classify multimedia posts, we constructed a supervised machine learning model used specifically for epidemic text analysis. Data collected from Sina Weibo, a leading social media platform in China, is used to test the hypotheses. The contributions of this paper are as follows: (1) This study takes the first step to classify public engagement into two dimensions: public engagement breadth and depth, which extends the health emergency management literature. (2) This study identifies the determinants of promoting public engagement from an information quality perspective and thus extends information quality research in the health emergency context. (3) This study confirms information richness's positive effects on promoting public engagement on social media and, more interestingly, finds its negative effects. In addition, we also with our findings can promote a more responsive public health strategy that magnifies the transfer speed for critical information and mitigates the negative impacts of information uncertainty or false information.

The remainder of this study is organized as follows. In the second section, we summarize public engagement on social media during emergencies and introduce some determinants of public engagement based on the information quality framework and MRT. In the third section, the research model and hypotheses of this study are proposed. The fourth section is related to the research methodology and empirical results of the data analysis. Finally, we discuss the main findings, theoretical and practical implications, limitations, and possible future research.

2. Theoretical background and hypotheses

2.1. Public engagement on social media during emergencies

There are many ways to describe various types of public engagement. First, bottom-up may be a critical characteristic of public engagement (initiated by the public with limited formal decision-making power) [72]. Second, public engagement may be driven by different motivations or outcomes. The public is more likely to participate in activities that can affect them [72]. Third, engagement typically exists in an information or knowledge flow and seeks feedback from the public [73].

During an emergency, the public is eager to seek information from various sources and constantly update it [90]. This drives them to engage with the emergency. In addition, the proliferation of social media has changed the way the public accesses information and engages with crisis management, and played a critical role in health-related information exchange during public health crisis events [40]. For example, during the H1N1 virus, Twitter was used as a public sphere for people to exchange opinions and experiences [17,76]. During the COVID-19 crisis, another study further analyzed people's engagement behaviors on social media, including shares, comments, and likes [15].

To date, an increasing number of studies have discussed the measurement of public engagement. Bonsón and Ratkai [11] first proposed three dimensions of public engagement, including popularity, virality, and commitment, and have received widespread attention. These three subdimensions are measured by the number of likes, shares, and comments, respectively, and have been widely used in the literature on public engagement [12,13,45,55,64]. Besides the number of behaviors aforementioned, detailed content is another important measurement of the degree of public engagement, and has not been examined in the literature. On social media, the public expresses their opinions through comments. If public comments are highly related to the post, it means that the public is deeply engaged. Thus, this study tries to use two indicators to measure public engagement: public engagement breadth, which is measured by the sum of shares, comments, and likes, and public engagement depth, which is measured by the relevance of corresponding comments to a post.

2.2. Health emergency information quality

During an emergency, information is considered a critical need of stakeholders [39]. In order to respond to an emergency, effective public engagement requires access to the right information at the right time [34]. Such requirements are often based on information quality dimensions, such as timeliness and correctness. Dawes et al. [23] indicated that a lack of information quality was a serious issue in emergency practice.

Information quality is a multidimensional concept. Wang and Strong [81] categorized it into four dimensions: intrinsic, contextual, representational, and accessibility quality. Of these dimensions, contextual quality has been examined most often in previous studies [8,34,43,80]. The most commonly mentioned factors of contextual quality are completeness [16], accuracy [56], timeliness [8], relevance [2], and security [54]. Further, timely information is still current situation about an event. Timeliness means a dynamic process where old information is replaced by new. Information timeliness is constantly changing, because of changes in people's perceptions caused by the external environment [68]. In this study, we mainly focus on information timeliness for the following reasons: first, in times of health emergencies, people are eager to seek information in a timely manner [91], they thus are more likely to engage with various information about the emergencies in order to alleviate their concerns and information uncertainty; second, as Minller [68] indicated, the concept of what is timely is usually defined by people, if a piece of information on social media is discussed by a large of people, it suggests the information with high timeliness. As for health emergencies, timely information is more likely to arise people's perceptions and then increase the possibility of engagement. Meanwhile, social media is highly dynamic encouraging instantaneous communication. It can be observed from the context that people's attention in a topic, thus, information with high timeliness is more likely to attract people's engagement; third, information on social media is highly contextual [3], in times of health emergencies, people may be more likely to pay more attention to health-related information. Fourth, this study assumes that information accuracy and completeness are automatically got. The extant literature indicated that these two information quality dimensions could be used to evaluate the reduce the consequences of communication breakdowns about the emergency [34]; however, there are difficulties that how to extract accuracy and completeness from text data. In addition, during times of emergency, individuals prefer to post or share accurate and complete information in order to more details of the emergency can be obtained by others. Thus, we assume that information accuracy and completeness are organically combined with information timeliness when a piece of information is posted on social media.

In this study, information timeliness refers to the extent to which the information is sufficiently up to date for a health emergency. People cannot process all the information they encounter, and tend to give most of their attention to only a small fraction. Accordingly, when the public perceives that health emergency information is relevant or new, their subsequent engagement behavior may be affected. For example, Kim et al. [55] indicated that health emergency information was highly time-sensitive. Eckert et al. [27] revealed that health emergency information had the characteristics of uncertainty, social publicity, and suddenness, which required more people to know in time. Doll and Torkzadeh [26] suggested that information timeliness is one of the most important factors that influence an individual's perception of product quality. Similarly, timely health emergencies-related information may present high information quality. Further, Information timeliness is conducive to public engagement. Lee and Kwak [60] indicated that timely information is highly related to public engagement (such as sharing and commenting) for the government on social media. Fu et al. [32] proved that the influence of eWOM information timeliness on consumer purchase intention. Further, the timeliness of information updated by the website is identified as a critical factor that has a positive impact on customer satisfaction [66]. In the event of health emergencies, if the information is timely, reflects current public concerns, and even predicts the future situation, users are more likely to forward the information and expand its social influence. However, if the information is highly similar to previous content, users may feel bored and less likely to engage.

The existing literature has primarily theorized the timeliness concept [34,55], but it has not provided a detailed view of how information timeliness promotes public engagement. To narrow this research gap, this study further divides information timeliness into three dimensions (see Fig. 1 ): information retrospectiveness, immediateness, and prospectiveness. Information retrospectiveness refers to the similarity to which a post is compared to previous posts released on social media. A high level of retrospectiveness indicates that the content of the post may be out of date, and thus users will be less willing to engage. Information immediateness refers to the similarity of a post to others that have been released on the same day. Information prospectiveness refers to the similarity of a post to others that will be released on the following day. High levels of immediateness and prospectiveness indicate that the content is timely and new, and users are more likely to propagate it. Prior studies have also indicated that people prefer to read something new [19]. The relationships between these three dimensions are shown in Fig. 1 and we thus hypothesize that:

H1

Information retrospectiveness negatively influences public engagement breadth and depth.

H2

Information immediateness positively influences public engagement breadth and depth.

H3

Information prospectiveness positively influences public engagement breadth and depth.

Fig. 1.

Concept of information timeliness.

2.3. Media richness theory

MRT, proposed by Daft and Lengel [21], emphasizes the ability of communication media on facilitating understanding. Media richness is considered to be information load. Daft et al. [22] later divided media richness into four types: face-to-face communication, telephone calls, written documents, and unprocessed documents. Advances in technology have enabled people to create and share multimedia content more easily. For example, individuals can post content with plain text, images, or videos, and in this way, media richness varies from low to high [15,25]. Due to word limits on social media (such as Twitter), individuals are likely to extend what they want to post by including extra material, such as special tags or symbols, URLs [13], images, and videos [36,47]. Thus, we used information richness to capture the features of information expression and further investigated its moderating role in the relationship between information quality and public engagement.

In this study, information richness was defined as “the display format of emergency-related information.” In general, social media services allow users to share various types of information such as plain texts, images, or videos. The information delivered through different message types has different abilities to deliver information [59]; accordingly, people can determine their perception of richness of the information and subsequently behaviors. Information richness may provide additional information cues that can enhance the social engagement of people [30]. Some studies have indicated that posts with images receive 89% more likes and 150% more forwards on Twitter [46]. Visual information, such as videos and images, is more easily used to attract individuals' attention than plain text [35]. Richer travel-related messages are more likely to be propagated by travelers because they are more engaging [59]. However, the role of such visual information in public engagement during an emergency remains debated. Compared with text-only information, the public may pay more attention to emergency information with pictures, videos, and links. Richer information is more interesting and can decrease uncertainty, and encourage people to make decisions. For example, Goh et al. [33] found that the information richness of user-generated content was positively related to users' purchasing behavior. Xu and Zhang [86] analyzed tweets about the Malaysian Airlines Flight 370 disaster and found that whether a post contained video or pictures had a significant impact on the number of retweets.

In addition, some scholars have examined the moderating effects of information richness on individual engagement in the context of social media. For example, Zhou et al. [92] find that information richness positively moderates the relationship between the relevance of message and its dissemination on social media. They indicate that COVID-19-related information with images is more likely to be forwarded by people. Lee et al. [59] indicated that information richness strengthens the relationship between organizations' social media efforts and individual engagement. Following the same logic, high information richness is seen as being of high quality and more likely to affect public engagement.

H4a

Information richness strengthens the relationships between informational retrospectiveness on public engagement breadth and depth.

H4b

Information richness strengthens the relationships between informational immediateness on public engagement breadth and depth.

H4c

Information richness strengthens the relationships between informational prospectiveness on public engagement breadth and depth.

In addition to information timeliness, other factors can also affect public engagement, such as the length of information, whether it contains URL(s), the number of views, and the number of followers [65,67,75]. These are all included in control variables. Our research model is presented in Fig. 2 .

Fig. 2.

Research model.

3. Research methodology

3.1. Research context

This research focuses on the current health emergency: the COVID-19 pandemic. According to the real-time data released by Johns Hopkins University, by July 1, 2020, the number of cumulative confirmed cases of COVID-19 totaled over 30 million globally, with more than 200 countries/territories/areas being affected, and the number of deaths at over 900,000. After the outbreak of COVID-19, a variety of relevant information was posted on social media, such as the current situation of the virus and how to prevent it. Social media has become one of the most important sources of emergency information. This study focuses on Sina Weibo, a leading social media platform in China. The platform provides an ideal setting for exploring public engagement for two reasons. (1) Sina Weibo is one of the most influential micro-blogging platforms in China. At the end of 2019, there were over 510 million monthly active users. (2) The platform gathers and generates a large volume of unstructured text data, which provides data support for our research.

3.2. Data collection

We developed a Python-based web crawler to randomly collect posts (the keyword “COVID-19 Wuhan” was used to keep concerns about this research context and to narrow the scope of the data) during the period of January 23 to 30, 2020. On January 23, 2020, Wuhan was locked down to control the spread of COVID-19. This was an important time for the Chinese government and the public to fight the disease, and the public was eager to receive helpful information quickly. Kim et al. [52] indicated that when an emergency happens, online user search trends decrease significantly after about seven days. For each post, multimedia content, number of reposts, likes, corresponding comments, and the detailed content of all comments were captured. After collecting the initial data, we dropped non-original posts. On Sina Weibo, a post begins with the //@ symbol, indicating that the post is forwarded or reposted. Finally, 87,540 posts and 1,073,606 comments over seven days were collected. The required sample size for NLP applied to health emergencies-related data is often unknown. Zhou et al. [92] used 12,101 posts to explore the mechanism of the COVID-19-related misinformation on social media. A total of 1441 posts about the COVID-19 crisis were obtained by Chen et al. [15] to explore how to promote citizen engagement. Li et al. [64] used 36,746 posts to characterize the propagation of situational Information on social media during the COVID-19 epidemic. In this study, our sample size is far more than these studies; therefore, we believe that our sample size is sufficient for the NLP approach in this type of research.

3.3. Operationalization for variables

3.3.1. Dependent variable

The dependent variable of this study was public engagement. We measured it from two perspectives. First, similar to prior studies [15,46], public engagement was measured by the number of reposts, comments, and likes of each post. This dimension is called public engagement breadth (PEB). Second, as previously mentioned, the existing literature lacks research into the detailed content of comments during health emergencies. The content of the corresponding comments presents public opinions or arguments toward a post [53]. If the content of the corresponding comments is highly related to the post, it reflects a high level of public engagement. Thus, this study proposes public engagement depth (PED) to measure public engagement, which is measured by the relevance of natural language between the post and the corresponding comments. In this study, cosine similarity was used to calculate relevance. Cosine similarity is advantageous because even if the two similar texts are far apart by Euclidean distance due to the size of the document, they may still be oriented close together [37]. This method has been widely used in NLP and text mining [64].

3.3.2. Independent variables

The independent variables are three dimensions of information timeliness: retrospectiveness (Retro), immediateness (Imme), and prospectiveness (Prosp). Information retrospectiveness was measured by comparing the calculated relevance between the posts on the day and all the posts on the previous day. Similarly, information immediateness was measured using the relevance between the post of the day and other posts of the day. Information prospectiveness was measured using topical relevance between the post of the day and all posts the day after. Cosine similarity was also used to calculate the three independent variables.

3.3.3. Moderator variable

Different from information timeliness, this study classifies information richness into three levels according to the complexity of the post-presentation form. The three levels ascending with richness are plain text, text + images, and text + video. Text-only is marked as a low level of information richness, text + images are marked as a moderate level of information richness, and text + video is marked as a high level of information richness. Low, moderate, and high information richness were coded as 1, 2, and 3, respectively. This measurement is consistent with previous arguments [15,44,92].

3.3.4. Control variables

The number of followers of each post creator, the content length, whether it contained a URL, and the number of views [65,67,75], are included in our study to eliminate the interference of other factors on the results. The definitions of the main variables in this study are presented in Table 1 .Table 2 presents the statistics for the variables included in this study.

Table 1.

Description of Variables.

Variable	Measure item	Description
Dependent variable
Public engagement	Public engagement breadth (PEB)	The sum of the number of reposts, comments, and likes of each post.
	Public engagement depth (PED)	The relevance between the content of comments below each post and the content of each post.
Independent variables
Information timeliness	Information retrospectiveness (Retro)	The relevance between a post at time t and all posts at time t-1.
	Information immediateness (Imme)	The relevance between a post at time t and the other posts at time t.
	Information prospectiveness (Prosp)	The relevance between a post at time t and all posts at time t + 1.
Moderate variable
Information richness	Richness	The richness level of each post.
Control variables
Followers	Fans	The number of followers of the poster.
URL	Url	Whether the post contains URL(s).
Length	Length	The content length of each post.
Views	Views	The number of views of each post.

Table 2.

Variable statistics.

Variables	Mean	S.D.	Min	Max	VIF
PEB	68.429	2190.974	0	318,277
PED	0.063	0.232	0	1
Retro	0.010	0.060	0	1	1.14
Imme	0.090	0.275	0	1	1.17
Prosp	0.063	0.232	0	1	1.19
Richness	1.227	0.465	1	3	1.21
Fans	22.810	730.325	0	106,092	1.16
Url	0.013	0.081	0	1	1.53
Length	21.423	855.658	0	154,695	4.24
Views	101.052	4019.441	0	1,119,347	2.12

3.4. Research procedures based on text mining and NLP

The text data processing process of the main analysis included two sections, one focusing on text mining for information timeliness (see the left panel in Fig. 3 ), the other on NLP for information type (see the right panel in Fig. 3). The main purpose of text mining is to extract information retrospectiveness, immediateness, and prospectiveness from the unstructured text information of health emergency-related posts. The main purpose of NLP is to identify the category of the text content. The detailed procedures of text mining and NLP are described in Appendix A.

Fig. 3.

The process of the research procedures.

3.5. Data analysis and results

STATA 15 was used to analyze our data samples. The summary statistics of the variables are presented in Table 3 . We then checked for multicollinearity in STATA. As shown in Table 3, the variance inflation factor (VIF) values for all variables in this study were below 6 (no bigger than the threshold of 10), indicating that multicollinearity was not an issue in our data sample. An empirical analysis based on the regression model was conducted. We developed a model with the moderating role of emotional valence:

$$\begin{array}{c}{\mathit{PEB}}_i\left({\mathit{PED}}_i\right)={\beta }_0+{\beta }_1{\mathit{Retro}}_i+{\beta }_2{\mathit{Imme}}_i+{\beta }_3{\mathit{Prosp}}_i++\\ {\beta }_4{\mathit{Retro}}_i\ast {\mathit{Richness}}_i+{\beta }_5{\mathit{Imme}}_i\ast {\mathit{Richness}}_i+{\beta }_6{\mathit{Prosp}}_i\ast {\mathit{Richness}}_i+\beta Z_i\end{array}$$ (1)

Table 3.

Hierarchical regression results.

	Model 1	Model 2	Model 3	Mode 4	Model 5	Model 6
	Ln(PEB + 1)	Ln(PEB + 1)	Ln(PEB + 1)	PED	PED	PED
Retro		−0.057**	−0.009		0.010***	0.015***
Imme		0.055***	0.032**		0.227***	0.229***
Prosp		0.039**	0.045*		0.034*	0.023**
Richness			0.198***			0.176**
Retro* Richness			0.016*			−0.009*
Imme* Richness			0.006			0.116***
Prosp* Richness			0.041**			0.506***
ln(Fans + 1)	1.099***	1.116***	1.117***	0.022***	0.006***	0.001***
Url	−0.006*	−0.007*	−0.006*	−0.004***	−0.001***	−0.000***
ln(Length + 1)	0.157***	0.152***	0.151***	0.004***	0.002***	0.001***
ln(views + 1)	0.101***	0.103***	0.103***	0.004***	0.005***	0.001***
R-squared	0.299	0.334	0.363	0.122	0.287	0.296

Note: * p < 0.05; ** p < 0.01; ***p < 0.001.

where β parameters are the coefficients to be estimated and Z is the vector controlling the number of followers, whether the post contained URL(s), the content length of each post, and the number of views.

Hierarchical regression was used in our study [15,67]. Given that the variables varied in their initial magnitudes, we standardized the control and dependent variables for the main analysis; that is, we drew on the log transformation of the control and dependent variables. The results are shown in Table 4 . Models 2 and 5 results indicate that the information retrospectiveness had a negative effect on public engagement breadth (β =  − 0.057, p < 0.01); however, it had a positive effect on public engagement depth (β = 0.010, p < 0.01), partially supporting H1. Information immediateness (β = 0.055, p < 0.001 in Model 2; β = 0.227, p < 0.001 in Model 5) and prospectiveness (β = 0.039, p < 0.01 in Model 2; β = 0.034, p < 0.05 in Model 5) had a positive effect on public engagement. Hence, H2 and H3 were supported.

Table 4.

Robustness check using n-grams.

	Model 1	Model 2	Model 3	Mode 4	Model 5	Model 6
	Ln(PEB + 1)	Ln(PEB + 1)	Ln(PEB + 1)	PED	PED	PED
Retro		−0.200***	−0.165*		0.003***	0.007*
Imme		0.036***	0.024		0.134**	0.122***
Prosp		0.085***	0.120***		0.021***	0.027***
Richness			0.206***			0.145***
Retro* Richness			0.128***			−0.007*
Imme* Richness			−0.002			0.091***
Prosp* Richness			0.165***			0.412***
ln(Fans + 1)	0.431***	0.330***	0.231***	0.321***	0.034***	0.023***
Url	−0.006*	−0.009	−0.003	−0.002***	−0.000***	−0.001*
ln(Length + 1)	0.503***	0.564***	0.641***	0.301***	0.342***	0.239**
ln(views + 1)	0.154***	0.170***	0.201***	0.023***	0.012***	0.004***
R-squared	0.385	0.453	0.443	0.145	0.209	0.254

Note: * p < 0.05; ** p < 0.01; ***p < 0.001.

Information richness positively moderated information retrospectiveness and public engagement breadth (β = 0.016, p < 0.05, in Model 3), whereas its effect on PED was negative (β =  − 0.009, p < 0.05, in Model 6). Thus, H4a was partially supported. The moderating effect of information richness on the relationship between information immediateness and PEB was not significant (β = 0.006, p > 0.05, in Model 3), whereas its effect on PED was positive (β = 0.116, p < 0.001, in Model 6), partially supporting H4b. The moderating effect of information richness on the relationship between information prospectiveness and public engagement was positive (breadth: β = 0.041, p < 0.01, in Model 3; depth: =0.506, p < 0.001, in Model 6). Thus, H4c was supported.

In order to better understand the moderating effects, based on previous studies' procedures [6], the interaction diagram was plotted. The results are shown in Fig. 4, Fig. 5 . Fig. 4(a) shows that the effect of information retrospectiveness on PEB was stronger with high information richness (dotted line). Fig. 4(b) shows that at high information richness (dotted line), PEB increased more quickly than at low information richness (solid line), indicating that high information richness strengthened the positive effect of information prospectiveness. Fig. 5(a) reveals that for high information richness, the slope of the dotted line was smaller than for posts with low information richness (solid line), indicating a negative interaction between information richness and information retrospectiveness. However, Fig. 5(b-c) indicates that, for high information richness, the slope of the dotted line is steeper than for posts with low information richness (solid line), indicating a positive interaction between information richness and information immediateness and prospectiveness.

Fig. 4.

The moderating effects of information richness on information timeliness on public engagement breadth.

Fig. 5.

The moderating effects of information richness on the relationship between information timeliness and public engagement depth.

3.6. Robustness check

In the main analysis, Jieba was used to segment words. We further drew on the data with the n-gram approach [5] to test the robustness of our findings. The results are presented in Table 5 . Table 5 shows that most of the effects we studied were quantitatively consistent with the results reported in Table 3. Therefore, we are more confident that the results of our analysis are solid and robust.

Table 5.

Information category of posts and example posts.

Category	Urgency degree	Example post
Help-seeking	Important and urgent	#Medical help# My friend is suffering from COVID-19, which hospital should he go to?
Epidemic prevention	Important but not urgent	#Health science# During COVID-19, four critical measures to protect yourself: wearing masks, washing hands frequently, more ventilation, and less gathering.

3.7. Extension

The existing literature has confirmed the differentiated effect of information categories, such as cultural activities and sports [11] and health-related messages [20] on social media engagement. For example, Rahim et al. [1] analyzed 2123 posts from the Malaysian Health Department and revealed that content associated with health education promoted citizen engagement (calculated using reposts, comments, and likes), whereas posts about organizational propaganda had no effect on engagement. Another study analyzed 1018 tweets about Breast Cancer Awareness Month, and found that content about propaganda had fewer reposts [20].

According to the uses and gratifications theory [49], people's perceived satisfaction is positively related to their subsequent media usage. The public often has different personal information needs during health emergencies. Xie et al. [85] categorized emergency information published on social media into three categories: supervising the government, seeking truth, and caring for self-interest. They found that supervising the government positively influenced public reposting behavior. Thus, we posit that the emergency-related information category will have significantly differentiated effects on public engagement.

According to Li et al. [64] and Xie et al. [85], the most in-demand information during health emergencies is health warnings and advice and help-seeking. In this study, we identified two information categories based on the urgency of information: health help-seeking and epidemic prevention. The example post for each category is presented in Table 5. Given the massive scale of the data, we need to develop a scalable approach to classify the posts into health help-seeking or other categories. The identification procedures are shown in the right panel in Fig. 3 and the detailed identification procedures are also described in Appendix A.

Based on the results of the NLP algorithm, we classified all posts into help-seeking (26,231 posts and 429,451 comments) or epidemic prevention categories (61,309 posts and 644,155 comments). Then, we conducted analysis procedures consistent with the main analysis. The results are presented in Table 6, Table 7 . As shown in Table 6, Table 7, there are differences between help-seeking posts and epidemic prevention posts. The information retrospectiveness of help-seeking posts induced public engagement breadth (β = 0.009, p < 0.05), whereas it reduced PEB for epidemic prevention posts (β =  − 0.017, p < 0.001). The information prospectiveness of epidemic prevention posts was positively related to public engagement breadth (β = 0.000, p < 0.01), but its effect on help-seeking posts was not significant (β = 0.000, p > 0.05). The information retrospectiveness of help-seeking posts also induced public engagement depth (β = 0.004, p < 0.001), whereas the impact was not significant on epidemic prevention posts (β = 0.002, p > 0.05). Further, the influence of information prospectiveness was reversed for information retrospectiveness. Information prospectiveness of epidemic prevention posts induced public engagement depth (β = 0.004, p < 0.001), whereas the influence was not significant for help-seeking posts (β = 7.08e − 06, p > 0.05).

Table 6.

Results of health help-seeking data set.

	Model 1	Model 2	Model 3	Mode 4	Model 5	Model 6
	Ln(PEB + 1)	Ln(PEB + 1)	Ln(PEB + 1)	PED	PED	PED
Retro		0.009*	0.005*		0.004***	0.003***
Imme		0.001***	0.001**		0.000***	0.001***
Prosp		0.000	0.002		7.08e-06	1.34e-06
Richness			0.109***			0.098**
Retro* Richness			0.003*			0.000*
Imme* Richness			0.001*			0.002*
Prosp* Richness			0.002*			9.83e-06
ln(Fans + 1)	0.919**	1.006***	1.112**	0.031**	0.011***	0.000***
Url	0.002	−0.004*	−0.006*	−0.000**	−0.001**	−0.000
ln(Length + 1)	0.101***	0.141**	0.130***	0.000***	0.003*	0.001**
ln(views + 1)	0.131***	0.122***	0.130***	0.002*	0.004**	0.006***
R-squared	0.347	0.369	0.403	0.101	0.197	0.206

Note: * p < 0.05; ** p < 0.01; ***p < 0.001.

Table 7.

Results of the epidemic prevention data set.

	Model 1	Model 2	Model 3	Mode 4	Model 5	Model 6
	Ln(PEB + 1)	Ln(PEB + 1)	Ln(PEB + 1)	PED	PED	PED
Retro		−0.017***	−0.009*		0.002	0.002*
Imme		0.002*	0.016**		0.000*	0.001***
Prosp		0.000**	0.015*		0.004***	0.001*
Richness			0.087*			0.134***
Retro* Richness			0.003*			−0.003***
Imme* Richness			0.001			0.002**
Prosp* Richness			0.007***			6.02e-06***
ln(Fans + 1)	0.301**	0.352***	0.341***	0.301***	0.029**	0.034***
Url	−0.002	−0.006*	−0.003	−0.003**	−0.000	−0.001*
ln(Length + 1)	0.223*	0.294**	0.301***	0.312***	0.331**	0.307***
ln(views + 1)	0.167***	0.187**	0.210***	0.021***	0.031***	0.027**
R-squared	0.281	0.293	0.343	0.151	0.190	0.204

Note: * p < 0.05; ** p < 0.01; ***p < 0.001.

There were also differences in the contingent role of information richness. The moderating effect of information richness on the relationship between information immediateness and PEB was positive in the case of help-seeking posts (β = 0.001, p < 0.05), but not for epidemic prevention posts (β = 0.001, p > 0.05). Information richness positively moderated the relationship between PED and information prospectiveness for help-seeking posts (β = 0.000, p < 0.05), and negatively for epidemic prevention posts (β =  − 0.003, p < 0.001). Information richness positively moderated information prospectiveness and PED for epidemic prevention posts (β = 6.02e − 06, p < 0.001), whereas the effect on PED was not significant for help-seeking posts (β = 9.83e − 06, p > 0.05).

4. Discussion

4.1. Key findings

This study aims to examine the relationships between information timeliness and public engagement through social media during COVID-19. We explore the contingent role of information richness in moderating these relationships and present several key findings. First, regarding information timeliness, we find that information retrospectiveness has a negative impact on public engagement breadth. Previous studies suggest that information timeliness is positively related to individuals' behavior [31,52] because the recency of information is a critical factor that draws their attention [84]. We conducted regressions regarding the number of likes, comments, and shares. The results show that information retrospectiveness is negatively associated with shares and likes, whereas its effect on comments is not significant. This finding further suggests that outdated emergency-related information may reduce public engagement, such as sharing and liking. Information retrospectiveness has a positive impact on PED. One possible explanation for this discrepancy is that although the content of information is relatively old, the public may have new thoughts or similar experiences toward that information over time. Thus, the public is more likely to provide comments related to the information and to expect their comments to be helpful for others. We conduct a post hoc analysis (see Appendix B) by setting the time window at t±2 and t±3. The results show that information retrospectiveness has a negative impact on PED. It indicates that if emergency-related information is flawed to some extent, it still be beneficial to disseminate them during times of health emergency. If a piece of emergency-related information is significantly out of date, the public is reluctant to comment on other related information. In addition, the effects of immediateness and prospectiveness on PEB and depth are both positive and significant, indicating that the timeliness and novelty of health emergency information can promote public engagement. This finding is also consistent with the results of Filieri and McLeay [31] and Kim et al. [54].

Second, information richness strengthens the effect of information retrospectiveness and prospectiveness on PEB, but the effect of that information immediateness is positive but not significant. This finding shows that for outdated and novel information in health emergencies, images or videos can provide supplementary information. The public can learn more about the past situation of health emergencies, and hope that more people can understand this information, so they are more likely to share, comment, and like. In terms of posts with high information immediateness, such posts may need to be known to others; the public thus pays more attention to text content. Meanwhile, the public needs to spend additional time and effort processing the content of images or videos; therefore, multimedia may not promote their share, comment, and like behaviors. In addition, information richness weakens the impact of information retrospectives on PED but strengthens the effects of information immediateness and prospectiveness. For timely and novel information, multimedia content can provide extra information to support and confirm it. The public is thus more likely to be convinced, and will be more willing to comment. On the other hand, the public may already know the content of the outdated information, and may not need extra information from images or videos. Therefore, they may comment less on outdated emergency-related information that includes images or videos.

Finally, the results of the additional analysis show the differences between help-seeking posts and epidemic prevention posts. In terms of outdated emergency-related information, the public is more willing to engage in help-seeking posts. Help-seeking-related posts spread information about important and urgent help or aid during health emergencies [64]. Due to the emergency with suddenness and occasionality, both the public and healthcare administration are usually unready for preventing it. When some people have an infection, they prefer to post their situation on the Internet to seek help. If their help-seeking has been supported, but they do not state it on the Internet. Others may think such help-seeking has not been supported and then they are more likely to engage with help-seeking-related posts to help healthcare administration and individuals obtain the help or aid [87], even if the posts are outdated to some extent. Thus, we claim that outdated information does not always reduce public engagement on social media, especially during the health epidemic. However, the effects of information prospectiveness on PEB and depth are not significant in the case of help-seeking posts. One possible explanation is that the public may not be convinced by such posts because the detailed content is too new. Surprisingly, information richness strengthens the effect of information prospectiveness on PEB. This finding indicates that multimedia content can increase information credibility [89] and encourage public engagement on social media during health emergencies.

4.2. Theoretical implications

This study provides several theoretical insights into the existing literature. First, it contributes to the health emergency management literature by exploring public engagement through social media. Previous studies have mainly measured public engagement through the number of reposts, comments, and likes [12,15,45,52,65], but they have neglected the effect of the content of comments. This study proposed two measurements of public engagement: PEB (measured by the sum of reposts, comments, and likes) and depth (measured by the relevance between the detailed content of comments and the content of the post). In doing so, this study extends the dimensions of measuring public engagement through social media.

Second, this study extends the contextual dimension of the information quality framework by measuring timeliness from specific categories to reconcile the framework better with the health emergency context. The existing literature has mainly theorized information timeliness as a whole concept [31,34,55,84]. Few studies have focused on different dimensions of information timeliness and explored how each dimension can promote public engagement during health emergencies. Drawing on the information quality framework, this study measured information timeliness from three dimensions: information retrospectiveness, immediateness, and prospectiveness. Our empirical findings provide new insights that further enrich the application of the information quality framework in promoting public engagement during health emergencies.

Finally, this study highlights the role of information richness in shaping the relationship between information timeliness and public engagement. On social media, although prior studies have examined the positive role of information richness in public engagement [11,52,62,75,86], our findings reveal that it has a negative effect. This counterintuitive finding shows that while information richness has been widely verified to be a positive factor, it may play a negative role and have a dark side. Therefore, this study enriches MRT by disentangling the positive and negative sides of information richness in the health emergency context.

4.3. Practical implications

This research has several important practical implications. First, both information immediateness and prospectiveness had significant impacts on public engagement during an emergency. This finding suggests that social media agencies or individuals should publish novel and timely emergency-related information. High levels of immediateness and prospectiveness may be more helpful to the public to decrease their uncertainty about the emergency, as well as to alleviate their concerns and anxiety. Social media platforms should recommend up-to-date information to the public because people prefer to engage with timely and novel emergency-related information. Second, due to the positive and negative sides of information richness, social media agencies or individuals should fully consider how information richness matches information timeliness when creating posts during times of health emergencies. When posts are out of date, images or videos are more suitable.

Finally, this paper also provides some decision supports to public healthcare administration and social media providers. For example, help-seeking posts are more likely to attract public attention and engagement during health emergencies. Therefore, the administration could post such messages on its social media account or encourage other posters to post more such messages in order to embed public engagement into health disease prevention activities and social media providers should spend greater effort recommending relevant help-seeking information in times of health emergencies like COVID-19. Compared to the public, health administration knows more about the health emergency; thus, the administration should post timely information to promote transparent decision-making and social media providers also should recommend such information to users. The information richness of posts should not be simply considered “the richer the better”. Health administration should encourage posters to create posts that information richness matches the content type.

5. Avenues for future IS research and conclusions

It is important for the public to receive timely information during health emergencies. Previous studies have indicated that social media can promote public engagement. Drawing on the information quality framework and MRT, this study uses NLP, text mining, machine learning technologies, and empirical analysis methods to examine the roles of information timeliness in promoting public engagement on social media.

Although this paper investigated the underlying mechanisms of public engagement during COVID-19 from vast quantities of text data from text mining, we only basically substantiated our hypotheses. There are still some limitations that should be noted for future research. First, due to the scope of this study, we only focused on information timeliness. Other dimensions of information quality, such as completeness [16], accuracy [56], and relevance [2], may also facilitate public engagement. Future research could explore whether additional information quality factors have important influences on promoting public engagement during health emergencies. Second, we focused only on a health emergency situation. Further research could examine other emergency situations, such as earthquakes and hurricanes. Third, although the two types of health-related emergency information proved effective and credible in this study, some biases cannot be avoided in the process of manual labeling work. Future research could improve the accuracy of the coding method. Fourth, multimedia data is simply possessed as three levels. We do not extract the details of the data by using other instrumental means such as computer vision. Future work might try to extract the contents from images or videos.

Credit author statement

The first four authors equally contributed to the design and development of this research. Dr. Qinyu Liao (the fifth coauthor) joined the research project after the original submission and made a significant contribution to the revision of the manuscript.

Biographies

Kai Li is a Professor of IS at Business School, Nankai University, Tianjin, China. His research interests include online reviews, online privacy, social media analysis, and business intelligence. His research has been published in leading IS journals such as Decision Support Systems and Information & Management.

Cheng Zhou is a Ph.D. Candidate of IS at Business School, Nankai University, Tianjin, China. His research interests include social media analysis, online privacy, and electronic platforms. His research has been published in Information Processing & Management and Telematics and Informatics.

Xin (Robert) Luo is an Endowed Regent's Professor and a Full Professor of Management Information Systems and Information Assurance in the Anderson School of Management at the University of New Mexico, Albuquerque, USA. He received his Ph.D. in Management Information Systems from Mississippi State University, Starkville, USA. His research will appear or has been published in leading IS or business journals, including the Information Systems Research, Journal of Operations Management, Journal of Management Information Systems, Journal of the Association for Information Systems, European Journal of Information Systems, Information Systems Journal, Journal of Strategic Information Systems, Decision Sciences, Decision Support Systems, Information & Management, and IEEE Transactions on Engineering Management. He has served as an ad hoc Associate Editor for MIS Quarterly and Associate Editor for the European Journal of Information Systems. He currently serves as an Associate Editor for the Journal of the Association for Information Systems, Decision Sciences, Information & Management, Electronic Commerce Research, Journal of Electronic Commerce Research, and Journal of Intellectual Capital. His research interests center around information assurance, innovative technologies for strategic decision-making, and global IT management. He is the Co-Editor-in-Chief for the International Journal of Accounting and Information Management.

Jose Benitez is a Full Professor of Management Information Systems at EDHEC Business School, Lille, and Paris, France. Previously, Jose held senior faculty member and academic managerial and leadership positions (i.e., Associate Dean, Department Chair, and Endowed Chair) at Rennes School of Business (Rennes and Paris, France) and the University of Granada (Granada, Spain). His research interests cover the study of how firms select digital technologies and develop unique digital capabilities to execute digitally-driven business transformation initiatives to improve firm performance, the bright and dark IT impact on individuals, and the development of quantitative research methods in the field of IS. His research has been published in top journals in IS and other Business Administration disciplines such as MIS Quarterly, Journal of Management Information Systems, Journal of Operations Management, Journal of the Association for Information Systems, European Journal of Information Systems, Journal of Information Technology, Information & Management, Decision Support Systems, Decision Sciences, and Journal of Business Research. Jose's engagement with the IS discipline has been recognized, designating him as an Association for Information Systems Distinguished Member Cum Laude. He currently serves as Senior Editor of the European Journal of Information Systems and Decision Support Systems and as Associate Editor for the Journal of the Association for Information Systems and Information & Management. In addition, Jose has served as Guest Editor of Decision Sciences. Jose got a Ph.D. in Business Administration (with a concentration in IS) from the University of Granada, Spain, with a double exchange with the University of Minnesota, USA, and Maastricht University, The Netherlands. His teaching interests and instructional expertise cover managing digital business transformation, digital innovation, the business value of digital technologies, IT management, IT strategy, IT and sustainable development, and quantitative research methods at Ph.D., Executive MBA, Global MBA, MIM, and BBA. Jose is a passionate speaker that enjoys working with students, colleagues, and executives. He has provided consulting services and worked on IT development and digital transformation projects with many leading companies in Spain, France, and the USA. Jose can be contacted at jose.benitez@edhec.edu.

Qinyu Liao is a Professor of Management Information Systems in Robert C. Vackar College of Business and Entrepreneurship at the University of Texas Rio Grande Valley, Brownsville, USA. She earned her Ph.D. in IS from Mississippi State University, Starkville, USA. Her research interests include e-commerce, information security, and e-education. Her research has been published in IS journals including Decision Support Systems, Information & Management, Journal of Computer Information Systems, Electronic Commerce Research, and Computers & Security.

Appendix A. The detailed procedures of text mining and NLP

A.1. Information timeliness extraction based on text mining

First, we extracted plain text from original posts. After cleaning the posts, Jieba1 in Python language is used to segment the sample posts. Jieba is popular software in Chinese text segmentation and keyword extraction. Therefore, we used the Jieba word segmentation tool to extract the vectors of this paper's data word segmentation and then removed the relevant stop words based on the external reference of the stop words dictionary. Second, we used the TF-IDF algorithm to find the keywords. Finally, according to the results of the TF-IDF algorithm, we obtained the value of the three variables based on cosine similarity.

A.2. Information type identification based on NLP

In the extension, we need to identify the category of health emergency-related information. The help-seeking category is measured using the total number of help-seeking linguistic words such as donate, help, sponsor, aid, support, benefact, etc. The epidemic prevention category is measured using the total number of professional medical words such as diets, drug prescriptions, masks, washing hands, exercises, moods, open windows, etc.

In order to ensure the quality of manual labeling, three research assistants are invited to help label the samples. Moreover, before manual annotation, the three assistants are informed of the tagging rules. The detailed process of labeling for the assistants: In step 1, we randomly sampled 10,000 posts as the training set and sorted out a preliminary dictionary based on the extant literature, health emergency knowledge, and understanding of the posts. In step 2, three assistants annotated the posts in parallel. Step 3, through the Cohen's Kappa value, we evaluated the accuracy of the 10,000 data annotation, and then updated the tagging rules. Steps, repeat steps 2 and 3 until the Cohen's Kappa value is larger than 0.61. Landis and Koch [51] suggest that the Cohen's Kappa value within the substantial interval [0.61, 0.80] indicates a high agreement in manual annotation results.

After labeling the sample posts, the labeled posts were randomly divided into training sets (accounting for 75%) and test sets (accounting for 25%). Next, the support vector machines algorithm was used to identify the training data, and the test data was used to evaluate the performance of the classifier. The performance evaluation result shows that our classifier obtained a good average accuracy of 81.3%. Finally, based on the supervised NLP algorithm, we automatically labeled the remaining posts.

Appendix B. The results of post-hoc analysis

Table B-1.

The results at time t±2.

	Model 1	Model 2	Model 3	Mode 4	Model 5	Model 6
	Ln(PEB + 1)	Ln(PEB + 1)	Ln(PEB + 1)	PED	PED	PED
Retro		−0.043**	−0.007		−0.014**	−0.000
Imme		0.051**	0.047**		0.198***	0.201**
Prosp		0.027*	0.036**		0.056**	0.032**
Richness			0.165**			0.019*
Retro* Richness			0.020**			0.010*
Imme* Richness			0.015*			0.124**
Prosp* Richness			0.056***			0.320**
ln(Fans + 1)	1.081**	1.120***	1.170***	0.030***	0.026***	0.022***
Url	−0.005	−0.006*	−0.007**	−0.002**	−0.002**	0.000
ln(Length + 1)	0.145***	0.131*	0.139**	0.014**	0.016**	0.027***
ln(views + 1)	0.113**	0.091*	0.109***	0.020***	0.018**	0.012*
R-squared	0.267	0.298	0.286	0.201	0.234	0.246

Table B-2.

The results at time t±3.

	Model 1	Model 2	Model 3	Mode 4	Model 5	Model 6
	Ln(PEB + 1)	Ln(PEB + 1)	Ln(PEB + 1)	PED	PED	PED
Retro		−0.025***	−0.031*		−0.027***	−0.001**
Imme		0.054*	0.067***		0.149**	0.107***
Prosp		0.035**	0.031*		0.069*	0.042*
Richness			0.167***			0.017**
Retro* Richness			0.028***			0.015*
Imme* Richness			0.011*			0.111*
Prosp* Richness			0.058**			0.432***
ln(Fans + 1)	1.078*	1.068**	1.082***	0.032***	0.030***	0.028***
Url	−0.002	−0.002*	−0.001**	−0.002*	0.001	0.001
ln(Length + 1)	0.183***	0.137**	0.140**	0.024***	0.022**	0.021***
ln(views + 1)	0.101*	0.087	0.111**	0.021***	0.028**	0.022**
R-squared	0.203	0.215	0.221	0.113	0.178	0.189