Association of “#covid19” Versus “#chinesevirus” With Anti-Asian Sentiments on Twitter: March 9–23, 2020

Abstract

Objectives. To examine the extent to which the phrases, “COVID-19” and “Chinese virus” were associated with anti-Asian sentiments.

Methods. Data were collected from Twitter’s Application Programming Interface, which included the hashtags “#covid19” or “#chinesevirus.” We analyzed tweets from March 9 to 23, 2020, corresponding to the week before and the week after President Donald J. Trump’s tweet with the phrase, “Chinese Virus.” Our analysis focused on 1 273 141 hashtags.

Results. One fifth (19.7%) of the 495 289 hashtags with #covid19 showed anti-Asian sentiment, compared with half (50.4%) of the 777 852 hashtags with #chinesevirus. When comparing the week before March 16, 2020, to the week after, there was a significantly greater increase in anti-Asian hashtags associated with #chinesevirus compared with #covid19 (P < .001).

Conclusions. Our data provide new empirical evidence supporting recommendations to use the less-stigmatizing term “COVID-19,” instead of “Chinese virus.”

In 2015, the World Health Organization (WHO) wrote

Disease names really do matter. . . . We’ve seen certain disease names provoke a backlash against members of particular religious or ethnic communities.¹

Consequently, the WHO recommended using the phrase “COVID-19” to describe the disease associated with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) on February 11, 2020.² On February 24, 2020, the WHO stated, “Don’t attach locations or ethnicity to the disease, this is not a ‘Wuhan Virus,’ ‘Chinese Virus’ or ‘Asian Virus.’”³ Other organizations, such as the US Centers for Disease Control and Prevention (CDC), issued similar guidelines.

The pandemic provides a natural experiment to evaluate the consequences of not adhering to these recommendations. One such test comes from a comparison of the phrase “COVID-19” versus “Chinese Virus,” which was tweeted by previous US president Donald J. Trump at 18:51:00 on March 16, 2020, from his official verified Twitter account @realDonaldTrump, which has since been banned by Twitter as of January 8, 2021 (https://blog.twitter.com/en_us/topics/company/2020/suspension.html):

The United States will be powerfully supporting those industries, like Airlines and others, that are particularly affected by the Chinese Virus. We will be stronger than ever before!⁴

Because the former president used the platform often, and because of the power of his office, his tweets could be highly influential. This was the first time he used “Chinese Virus,” and, according to newspaper reports, there was a rise in hate crimes against Asians after the president tweeted.^5,6 Yet, many have claimed that the terms are not discriminatory. For example:

It’s not racist. . . . it comes from China. . . . I want to be accurate.⁷

—President Trump

Anyone who complains that it’s racist or xenophobic to call this virus the Chinese coronavirus or the Wuhan virus is a politically correct fool.⁸

—Senator Tom Cotton, R-AR

Others downplayed the words’ importance, as seen in a newspaper editorial:

Hurling the racism charge over such minor issues such as disease names is silly.⁹

—David Mastio

Thus, although the scientific community agrees that “COVID-19” should be used instead of “Chinese virus,” influential voices in the government and press argue otherwise.

Social media data, such as from Twitter, may provide evidence regarding these claims. Twitter is an online platform for publicly expressing thoughts and feelings, making it useful for examining real-world behaviors.¹⁰ For example, Twitter has been used to identify political sentiment to predict election results.^11–13 In addition, this information can be used to conduct ecological momentary assessment (ongoing evaluation of in-the-moment experiences)¹⁴ and has been used to study shifts in emotions as a result of natural disasters.¹⁵ Therefore, data from Twitter (tweets and hashtags) have the potential to detect changes in attitudes that lead to the formation of mass public opinions,¹⁶ including hate toward specific groups.

People typically use hashtags to signify agreement and solidarity, but typically do not add hashtags to statements that they find disagreeable (similar to how people use bumper stickers on cars). Furthermore, hashtags can proliferate allied hashtags (e.g., #blacklivesmatter can inspire use of #blackpower, #buyblack, and #sayhername). Thus, hashtags allow information to travel beyond the initial social network and can form collations of speech.¹⁷ This has led researchers to examine how hate-speech hashtags are associated with hate crimes.¹⁸ In this research, the variable that best predicted real-world violence was the hashtag used in the tweet.^17,18

One study examined 69 470 tweets and “#chinavirus” and “#chinesevirus,” which the authors considered to be “representative racist hashtags.”¹⁹ The study found temporal fluctuations in use of these hashtags between January and March 2020, and suggested that these fluctuations coincided with worldwide changes in the policy response to the pandemic.¹⁹ This study provided an important foundation, but left unanswered the question of whether the phrase “Chinese virus” is inflammatory in comparison with “COVID-19.” This question is important for identifying and describing the consequences of attaching locations or ethnicity to diseases. Accordingly, we investigate these hypotheses:

• Hypothesis 1: The tweets with the hashtag #chinesevirus will contain a greater proportion of anti-Asian hashtags than the tweets with the hashtag #covid19.

• Hypothesis 2: Anti-Asian hashtags will rise after the president’s tweet of “Chinese Virus.”

• Hypothesis 3: The rise in anti-Asian hashtags will be more pronounced among tweets with #chinesevirus compared with #covid19.

METHODS

We collected data from Twitter’s Application Programming Interface, which procures tweets from Twitter’s public stream that included the hashtags #covid19 or #chinesevirus. Data were from March 9 to 23, 2020, corresponding to the week before and week after the president’s tweet with the phrase, “Chinese Virus.” After excluding non-English tweets and hashtags, our analysis sample consisted of 668 597 tweets and 1 273 141 hashtags. In addition, we collected the timestamp of tweets and users (i.e., tweeters).

Analyses focused on hashtags because previous research indicates that hashtags are related to the formation of hate groups and hate crimes and because hashtags can be predictive of behaviors.^11,17,18

Anti-Asian Hashtags

We studied whether the hashtags associated with #covid19 differed in terms of anti-Asian expressions compared with hashtags associated with #chinesevirus. Tweets with both #covid19 and #chinesevirus were included in each of the groups’ analysis. Tweets containing only #covid19 or only #chinesevirus without any other hashtags were excluded in the hashtag analysis. To characterize anti-Asian expressions, the hashtags were independently coded by 2 trained research assistants who were blinded as to whether the hashtags belonged to #covid19 or #chinesevirus.¹⁷

The characterization of the hashtag was done through a qualitative investigation of the tweet and its neighboring hashtags. A hashtag was considered anti-Asian if it (1) was opposed to or hostile toward the region, the people, or culture of Asia; (2) demonstrated a general fear, mistrust, and hatred of Asian ethnic groups; (3) supported restrictions on Asian immigration; or (4) used derogatory language or condoned punishments toward Asian countries or their people. Examples of anti-Asian hashtags included #bateatingchinese, #yellowmanfever, #makethecommiechinesepay, #disgustingchinese, #commieflu, #chopstickchins, and #chinkflu.

We coded as “other” the remaining hashtags, including those that

1. were neutral (e.g., #washhands) or positive (e.g., #saferathome);

2. demonstrated hostility toward other racial groups (e.g., #nonrentingtoblacks);

3. were antiimmigrant (e.g., #secureourborders) but not specific to Asians;

4. criticized policies implemented by the Chinese government about Hong Kong, Taiwan, and Tibet (e.g., #tibetpolicestate); and

5. were conspiracy stories (e.g., #wuhancoverup).

Disagreements in coding between the 2 raters was minimal; the interrater reliability between them was 93.7%. Disagreements were adjudicated by a third coder.

Temporal Trends

A daily accumulation of the number of hashtags from tweets with #covid19 and #chinesevirus was calculated from March 9 to 23, 2020. In addition, we calculated and compared the daily growth of anti-Asian hashtags.

Statistical Analysis

We used the t test to compare the mean number of tweets per day, users per day, hashtags per day, and anti-Asian hashtags per day between the #covid19 and #chinesevirus group tweets. We used the χ² test to test the difference in the change in proportion of anti-Asian and non–anti-Asian hashtags between the #chinesevirus and #covid19 groups before versus after Trump’s tweet, which occurred at 18:51:00 on March 16, 2020. We used the t test to evaluate the difference in means before versus after Trump’s tweet for tweets per day, hashtags per day, anti-Asian hashtags per day, and users per day. We conducted analyses with R version 3.6 (R Core Team, Vienna, Austria).

RESULTS

Table 1 compares the #covid19 and #chinesevirus groups. For the #covid19 group, the total number of tweets was 247 958, the mean number of tweets per day was 6340.8 (SD = 6410.5), the mean number of users per day was 1816.9 (SD = 1427.9), the mean number of hashtags per day was 33 019.3 (SD = 31 366.0), and the mean number of anti-Asian hashtags per day was 6524.6 (SD = 6337.9). For the #chinesevirus group, the total number of tweets was 495 287, the mean number of tweets per day was 16 530.53 (SD = 19 471.7), the mean number of users per day was 4264.2 (SD = 4953.2), the mean number of hashtags per day was 51 856.8 (SD = 60 717.8), and the mean number of anti-Asian hashtags per day was 26 130.5 (SD = 31 174.81). We saw no significant differences in these descriptive statistics between the #covid19 and #chinesevirus groups. A significantly higher mean number of anti-Asian hashtags per day was seen in tweets within the #chinesevirus group compared with tweets within the #covid19 group.

TABLE 1—

Descriptive Comparisons of #covid19 and #chinesevirus Twitter Hashtags: March 9–23, 2020

Total Tweets	#covid19 (n = 247 959), Mean (SD)	#chinesevirus (n = 495 287), Mean (SD)	Difference in Mean (95% CI)	t Test
Tweets per day	6 340.80 (6410.52)	16 530.53 (19 471.68)	10 189.73 (–21357.05, 977.58)	−1.93
Users per day	1 816.93 (1427.90)	4 264.20 (4953.23)	2 447.27 (–5264.49, 369.96)	−1.84
Hashtags per day	33 019.27 (31 365.97)	51 856.80 (60 717.80)	18 837.53 (–55536.06, 17861.00)	−1.07
Anti-Asian hashtags per day	6 524.60 (6337.90)	26 130.50 (31 174.80)	19 605.90 (–37 097.90, –2113.90)	−2.39*

Note. CI = confidence interval.

^*P < .001.

Table 2 compares the #covid19 with #chinesevirus groups across the study period. Overall, there was a significantly higher proportion of anti-Asian hashtags in tweets within the #chinesevirus group compared with tweets within the #covid19 group (P < .001). From March 9 to 23, 2020, the total number of hashtags in the #covid19 group was 495 289 with 97 869 (19.8%) of those hashtags being coded as anti-Asian, and the total number of hashtags in the #chinesevirus group was 777 852 with 391 957 (50.4%) of those hashtags being coded as anti-Asian.

TABLE 2—

Comparison of Hashtags #covid19 Versus #chinesevirus on Twitter Before and After 18:51:00 on March 16, 2020

	#covid19 (n = 495 289)			#chinesevirus (n = 777 852)
Total Hashtags	Pre, No. or Mean (SD)	Post, No. or Mean (SD)	% Change or Difference in Mean (95% CI)	Pre, No. or Mean (SD)	Post, No. or Mean (SD)	% Change or Difference in Mean (95% CI)	Difference in % Change Between #chinesevirus vs #covid19
Total no.a
All hashtags	48 642	446 647	818.20	3 956	773 896	19 462.60	371 935
Anti-Asian hashtags	9 813	88 056	797.30	2 227	389 730	17 400.20	309 260
Non–anti-Asian hashtags	38 829	358 591	823.50	1 729	384 166	22 119.00	62 675
No. per day
Tweets	927.50 (168.58)	10 961.50 (5 472.4)	10 034.00 (–14 609.45, −5 458.55)	125.50 (101.35)	30 869.25 (15 960.88)	30 743.75 (–44 087.43, −17 400.07)	. . .
Hashtags	6 080.25 (1 208.12)	55 830.88 (26 367.34)	49 750.63 (–71 798.71, −27 702.54)	494.50 (423.01)	96 737.00 (49 397.71)	96 242.50 (–137 540.31, −54 944.69)	. . .
Anti-Asian hashtags	1 431.25 (321.75)	11 694.12 (5 988.17)	10 262.87 (–15 270.51, −5 255.24)	304.63 (264.53)	51 085.00 (27 828.04)	50 780.38 (–74 045.40, −27 515.35)	. . .
Users	588.50 (103.77)	2818.25 (1 279.66)	2 229.75 (–3 300.27, −1 159.23)	93.50 (69.65)	7 901.88 (4 079.67)	7 808.38 (–11 219.16, −4 397.59)

Note. CI = confidence interval.

^aX²(1, n = 1 167 945) = 112 586; P< .001.

Table 2 and Figure 1 also show the changes in hashtags between the groups for #covid19 and #chinesevirus pre–post 18:51:00 on March 16, 2020. For the #covid19 group, the number of hashtags rose by 818.2% (398 005 tweets), and anti-Asian hashtags rose by 797.3% (78 243 tweets). For the #chinesevirus group, the number of hashtags rose by 19 462.6% (769 940 tweets), and anti-Asian hashtags increased by 17 400.2% (387 503 tweets). There was significantly higher proportion of the change in the occurrence of anti-Asian hashtags in tweets in the #chinesevirus group compared with tweets in the #covid19 group (P < .001). Viewed another way, within the group of #covid19, the percentage of anti-Asian hashtags declined from 20.2% to 19.7%, whereas in the #chinesevirus group, the percentage declined from 56.3% to 50.4%. Before Trump’s message, there were more hashtags in the #covid19 group than the #chinesevirus group. After his message, both hashtags increased in prevalence. However, there was a significantly larger (P < .001) increase in the proportion of hashtags in the #chinesevirus group compared with the #covid19 group. Furthermore, the number of #chinesevirus hashtags surpassed that of #covid19.

FIGURE 1—

Timeline of All Twitter Hashtags Under #covid19 (n = 495 289) and #chinesevirus (n = 777 852): March 9–23, 2020

Table 2 shows the same patterns for changes in daily averages. For example, the mean number of users per day in the #covid19 group rose from 559 to 2818 (404.11%) after Trump’s tweet, whereas in the #chinesevirus group, it climbed from 94 to 7902 (8306.38%). Similarly, the mean number of anti-Asian hashtags per day in the #covid19 group rose from 1431 to 11 694, (717.19%), but the #chinesevirus group soared from 305 to 27 828 (9023.93%).

Figure 2 depicts the dramatic divergence between anti-Asian hashtags in the #chinesevirus group compared with the #covid19 group. There were slightly fewer anti-Asian hashtags associated with the #chinesevirus group than #covid19 group before Trump’s message. Anti-Asian hashtags rose on March 16, and by March 17, there were more anti-Asian hashtags associated with the #chinesevirus group. There was a significantly higher difference in the change in the proportion of anti-Asian hashtags from tweets within the #chinesevirus group compared with the #covid19 group (χ² [1, n = 1 167 945] = 112 586; P < .001).

FIGURE 2—

Timeline of Anti-Asian Twitter Hashtags Under #covid19 (n = 97 869) and #chinesevirus (n = 391 957): March 9–23, 2020

Although we had high interrater reliability (93.7%) between the 2 raters, we wanted to ensure that our analyses were robust to modeling assumptions. We performed 2 sensitivity analyses to examine if differences in coding changed our results. We reanalyzed the data assuming that (1) the disagreements were all anti-Asian and (2) the disagreements were all not anti-Asian. The results of these analyses are similar to those reported.

DISCUSSION

A rise in discrimination against people of Asian descent during the COVID-19 pandemic has been reported around the world.^20,21 The United Nations Secretary-General António Guterres announced, “the pandemic continues to unleash a tsunami of hate and xenophobia, scapegoating and scare-mongering.”²² To mitigate this discrimination, the WHO has recommended avoiding terms that connect diseases to countries or specific people, and instead promoted the use of neutral scientific terms. Our research on 1.2 million hashtags buttresses their recommendation by showing that the hashtag #chinesevirus is connected to more anti-Asian hashtags than #covid19. Approximately 1 in 5 hashtags with #covid19 were anti-Asian, whereas half of the hashtags with #chinesevirus were anti-Asian.

In the week beginning March 9, 2020, the hashtag #covid19 was more prevalent than #chinesevirus. Also, the number of anti-Asian hashtags associated with these phrases was relatively low and stable.

However, the president’s tweet on March 16 coincided with several major changes. First, there was a massive increase in the volume of tweets for both the #covid19 and #chinesevirus groups and in the number of users. Both hashtags together climbed from about 53 000 to 1.2 million in the period studied. Trump’s tweet appeared to increase discussion about the pandemic in general, as shown by these example tweets:

Still seeing a lot of depleted shelves around the Milwaukee area. #covid19 #notoiletpaper (March 16, 2020)

Best part of working from home? Turning that damn morning alarm off #quarantine #chinesevirus (March 19, 2020)

Second, there was a differential effect on the hashtag #chinesevirus. It overtook the hashtag #covid19 as the more popular hashtag and coincided with a major growth in the number of people using the phrase. The mean number of daily users from the #covid19 group rose by 379%, compared with an increase of 8351% for the #chinesevirus group. Furthermore, the phrase “Chinese virus” may have served as a rallying cry to some supporters, as seen in this example:

The Coronavirus Outbreak Shows Clearly That President Trump Was Right All Along About Borders, Trade And Most Of All, He Was Right About China #coronaviruspandemic

#chinavirus #chinacoronavirus #trump2020 #chinesevirus (March 17, 2020)

The proportion of anti-Asian hashtags attached to the groups of #covid19 and #chinesevirus declined slightly after March 16 (by 0.46% and 5.9%, respectively). Although statistically significant, we do not view this decline as substantively meaningful. Even with the decline, more than half of the #chinesevirus hashtags were associated with anti-Asian sentiment, compared with 1 in 5 of the #covid19 hashtags.

More importantly, the number of anti-Asian hashtags rose by 797% and 17 400% for #covid19 and #chinesevirus, respectively. This represents a combined increase from about 12 000 to almost a half a million anti-Asian hashtags. This finding aligns with previous studies that suggest a rise in prejudicial language following some of the president’s tweets and that racist attitudes may be reinforced by institutional support.^23,24

The growing chorus of hateful words possibly contributed to the rise in hate incidents.²³ We do not have the data to investigate whether these sentiments translated directly to hate incidents. However, indirect evidence comes from the StopAAPIhate.org Web site. From March 19 to 25, 2020, they reported more than 600 anti-Asian hate incidents.

Although we were unable to assess the relationship between hateful hashtags and hate crimes, our results provide a plausible connection because many tweets and hashtags implied violence. For example,

Fuck the ding dongs. Fuck the ching chongs. And most definitely, fuck the god damn chinks. #chinaliedpeopledied #coronavirus #fuckchina #chinesevirus #wuhanvirus #burnwuhan #bombchina (March 20, 2020)

#chinesevirus please #nukechina (March 17, 2020)

Furthermore, even if the probability of a hashtag leading to a hate crime is low, the large volume of new hashtags might translate to a noticeable increase in incidents. Indeed, even a single hate crime is 1 too many. Previous studies have documented the link between racist discussion on social media like Twitter and Facebook and hate crimes.¹⁸ One study found a relationship in the use of racist hashtags such as #banislam with hate crimes targeting Muslims.¹⁸

One other study documented the association of negative sentiment in tweets with #chinesevirus. Their study examined a smaller sample (n = 174 488) over a longer period of 3 months.¹⁹ They identified temporal variation in sentiment of tweets with #chinesevirus, although their methodology differed from ours. They assumed that the phrase #chinesevirus was itself discriminatory, whereas we did not make that assumption. Rather, we wanted to provide some objective evidence as to whether this term might be considered biased through its connection with other prejudicial terms. Thus, our studies provide complementary information.

Previous studies have used sentiment analysis to identify opinions toward the topic of interest.^25,26 However, sentiment analysis is used to detect the polarity of the tweet (e.g., positive or negative opinion) and cannot detect anti-Asian expressions, inappropriate references, nuances, slang, or sarcasm.²⁷ For instance, sarcastic tweets without hashtags have been shown to be difficult to distinguish because hashtags convey an extralinguistic that is the equivalent of nonverbal expressions in live interactions.^28,29 Hence, hashtag extraction and the manual labeling of hashtags has been shown to be more effective at accurately identifying the position of users toward the topic of study.²⁷ In an example, the hashtags “#batmaneatingflu” or “#yellowmanflu” could be classified as a neutral sentiment using a lexicon and rule-based sentiment analysis tool, whereas we considered these hashtags anti-Asian in our study. Thus, a strength of our study was the use of qualitative assessments to directly code for anti-Asian sentiment.

Asian Americans face the dual stereotypes of being a “model minority” and the “yellow peril.”³⁰ The former refers to the purported successes of Asian communities, and, as a consequence, Asians are viewed as easy targets for scapegoating. By contrast, the latter refers to the invasion by a foreign threat. The pandemic further illustrates how a disease can garner support for the yellow peril stereotype. We caution that even the model minority stereotype leads to problems and may generate a false idea that Asians are immune to prejudice and discrimination. It also ignores the many needs within the community and is used as a foil against other racial groups (i.e., If Asians are successful, why cannot other groups be too?). The more fundamental problem of both stereotypes is that they divert attention away from the broader issues of structural racism and White supremacy.

Limitations

We focused on hashtags and not tweets in this study. While a future study could code each individual tweet, we opted to use hashtags because of their categorical function, whose occurrence can become a trending topic.²⁷ A hashtag acts like a summary of the tweet, a global overview of the content in the text of the tweet. For instance, analysis of hashtags has been shown to be more effective at determining political affiliation of a user than analysis of the tweet because of its ability to better capture the real position of the user.²⁷ Therefore, hashtags allow us to identify what topics and groups the user intends to be connected to more than the tweet itself. Furthermore, hashtags archive messages and allow messages to be found by organizations and spread virtually to users outside of their direct network. Hashtags help access new audiences, maximize reach, and increase likelihood of viral attention to posts.³¹ Thus, hashtagging increases the level of engagement of users.³¹

We also caution that opinions on Twitter may not be generalizable to the population, and there are potential selection biases on who uses the platform. Furthermore, our analyses do not extend to other social media platforms (e.g., Facebook) or modes of communication (e.g., newspapers). It would be useful for future research to study the other sources.

In addition, we did not code hashtags targeted to the Chinese government and conspiracy theories as anti-Asian. We took this approach because some hashtags are used to categorize information (e.g., curate a list of theories related the pandemic’s origins). This likely made our analyses more conservative by underestimating antipathy directed toward Asians.

Public Health Implications

These results imply some possible suggestions for research and action. First, it has been encouraging that many agencies have issued statements against stigmatizing language. However, communication strategies have not been well-coordinated or thoughtfully planned. As we move into the second year of this pandemic, public health agencies should coordinate with policy makers, communication experts, and media outlets to not only avoid words that carry pejorative connotations but also to design countermessaging strategies to reverse the harm that has already been done to Asian communities.

Second, the monitoring, prevention, and prosecution of hate crimes is usually the purview of the Department of Justice. Yet, the pandemic clearly illustrates how such crimes are interwoven with diseases and other health issues, and, hence, relegating hate crimes to a single agency is suboptimal. The Department of Justice should partner with the CDC and other agencies to create a coordinated response to quell the rise in discriminatory speech, hate crimes, and other forms of discrimination.

Third, in consideration of future outbreaks, scientific names should be used to describe pathogens, but it will take time to identify them. In advance, public health officials can create generic templates and talking points that can be provided to the media from which to describe new outbreaks.

Fourth, more basic research should be conducted to understand stigma and medical terminology. It is clear that we should not label people with their diseases, but how medical terms absorb negative or positive connotations and how it shapes behaviors needs further investigation.

Fifth, our research provides a framework from which to study related phenomena. For example, recent reports have surfaced on the Vespa mandarinia, more popularly known as the “Asian Giant Hornet” or the “murder hornet,” with reports that echo the trope of the “yellow peril” from Asia coming to invade the United States.³² The lessons learned from COVID-19 could inform how we describe invasive insects, animals, and plants.

Our analyses suggest that the simple descriptor of a disease can carry racial overtones. Everyone—scientists, community members, and politicians—should use neutral, nonjudgmental language to avoid stigmatizing communities and perpetuating discrimination. Our analyses are consistent with recommendations to use neutral terminology.³³ “Chinese virus” was related to more than twice as many hate expressions compared with “COVID-19.” However, we caution that even the more neutral term of COVID-19 was associated with anti-Asian sentiment in a fifth of the hashtags. Thus, scientific language alone is not enough to erase prejudicial sentiments. Rather, we need to focus on the broader social determinants that perpetuate structural racism.