How do Canadian public health agencies respond to the COVID-19 emergency using social media: a protocol for a case study using content and sentiment analysis

Abstract

Introduction

Keeping Canadians safe requires a robust public health (PH) system. This is especially true when there is a PH emergency, like the COVID-19 pandemic. Social media, like Twitter and Facebook, is an important information channel because most people use the internet for their health information. The PH sector can use social media during emergency events for (1) PH messaging, (2) monitoring misinformation, and (3) responding to questions and concerns raised by the public. In this study, we ask: what is the Canadian PH risk communication response to the COVID-19 pandemic in the context of social media?

Methods and analysis

We will conduct a case study using content and sentiment analysis to examine how provinces and provincial PH leaders, and the Public Health Agency of Canada and national public heath leaders, engage with the public using social media during the first wave of the pandemic (1 January–3 September 2020). We will focus specifically on Twitter and Facebook. We will compare findings to a gold standard during the emergency with respect to message content.

Ethics and dissemination

Western University’s research ethics boards confirmed that this study does not require research ethics board review as we are using social media data in the public domain. Using our study findings, we will work with PH stakeholders to collaboratively develop Canadian social media emergency response guideline recommendations for PH and other health system organisations. Findings will also be disseminated through peer-reviewed journal articles and conference presentations.

Strengths and limitations of this study.

• We will understand the challenges with current public health (PH) social media approaches to communicating with the public in Canada and develop recommendations about how to enhance social media engagement with the public during an emergency response.

• Social media content will be collected from PH agencies, PH stakeholders and the public in Canadian provinces and at the national level at milestone points in time during COVID-19.

• Descriptive, content and sentiment analysis will be used to describe social media usage and engagement.

• The findings will help PH organisations communicate effectively and avoid misinformation, stigma and fear in the context of social media during an emergency response.

• Social media communication is not representative of all voices.

Introduction

Social media is a communication channel with widespread appeal and reach. Online social media platforms represent mainstream sources of information about health, wellness and risk to good health. Social media can be defined as information and communication technologies that allow ‘…individuals, communities, and organisations to collaborate, connect, interact, and build community by enabling them to create, co-create, modify, share, and engage with user-generated content that is easily accessible’ (McCay et al, p17)[1]. Mobile digital device use (eg, smartphone) is increasing globally with an estimated 3.3 billion users in 2019 and an anticipated 3.8 billion users by 2021.2 Almost all (97%) of Canadians 16–64 years of age are online; 85.0% of Canadians 65–69 years of age, 75% of individuals 70–74 years of age, 62.0% of individuals 75–79 years of age and 40.8% of individuals aged 80 or older reported online access.3 Over 70% of individuals online are seeking health information.4 Furthermore, healthcare organisations and governmental sectors offer a range of health-related resources (eg, policies, forms, patient instruction and health education materials) online—some exclusively. Social media is a powerful tool for communication to the public by public health (PH) agencies.

From the experience with SARS, we learnt that communication to the public during emergencies needs to be multifaceted, evidence-informed, timely, comprehensive, accurately targeted, credible and coordinated in order to sustain public confidence and to maintain order while minimising panic and stigma.5–7 During the 2009 H1N1 pandemic, researchers identified the internet as the most frequently used source of information by the public.8 Chew and Eysenbach9 performed an analysis on Twitter content during the 2009 H1N1 outbreak and found that message content consisted generally of information from credible sources, opinions and experiences. The authors also found that only 1.5% of the sources shared by social media users were from government and health agencies, demonstrating room for improvement in message content but also the prevalence of user-generated media content. Wong et al10 studied the dissemination of Ebola information via Twitter by US local health departments and found that a majority of posts consisted of informational resources and visuals such as infographics about Ebola; messages were most likely to be rapidly shared if they contained a hashtag. A systematic review on the social media literature on the Ebola outbreak found that social media may enhance PH communications during outbreak management; however, none of the included studies evaluated the utility of social media specifically for PH response.11

In terms of information accuracy, researchers found that out of 5395 Twitter posts about H1N1, only 4.5% were identified as misinformation.9 This was a similar finding in a study on the Ebola outbreak and the use of Twitter and Weibo (a Chinese-based microblogging platform), where approximately 2% of the assessed information was considered to be erroneous.12 In contrast, Oyeyemi et al13 reported that more than 55% of social media content is misinformation in relation to an Ebola emergency.

Although the ‘push out’ information feature of social media is valuable during emergencies, a key aspect of social media is the ability for PH to interact with the public, enabling engagement and relationship development.14–17 Engaging the public via social media in two-way communication generates benefits such as increased trust and positive feelings towards institutions,18–21 potentially counteracting misinformation during PH emergencies.22 However, there is evidence that PH agencies are not using social media to its full interactive potential.14 23

In 2015, a planning meeting with 20 of 36 Ontario PH agencies revealed that most have steadily adopted the technology since the H1N1 pandemic of 2009.24 For example, the majority of PH agencies are using Twitter, and some are using Facebook to engage directly with the community. Yet it is unclear what information is communicated and whether PH messaging was crafted to engage with (interactive communication) or simply inform (passive health communication) audiences.24 Following this, a review of existing literature reviews (ie, umbrella review) of research and grey literature was done to determine how social media is used by PH agencies.25 Findings from 38 reviews, which included emergency and non-emergency social media communication, found that social media is used in a variety of ways for PH research and practice, but primarily targeting individual health behaviour change and disease surveillance. The use of social media is a low-cost alternative for PH programme delivery and facilitates the ability to increase health message reach, as well as the development of supportive online communities. Important barriers to social media use include a lack of trustworthiness in online information, the digital divide and privacy concerns among social media users.25

Effective risk communication is particularly relevant to PH during an emergency response. In December 2019, Canadian performance indicators for PH emergency preparedness were developed.26 Of the 67 identified indicators, 11 were related to communication, representing a domain with the second highest number of indicators after leadership. Social media was explicitly named as a communication platform for (1) PH messaging, (2) monitoring misinformation and (3) responding to questions and concerns raised by the public. This study builds on this need for PH to effectively and efficiently use social media and will investigate the use of social media as a PH communication strategy. From the demand perspective, 83% of Canadians who are online reported at least one social media account in 2020.27 Eighty-three per cent of Canadian online adults have a Facebook account; 65% have messaging apps; 64% have YouTube; 51% have Instagram; and 42% have Twitter.27 In Canada, Facebook is used daily by 77% of its account holders. Twitter, in comparison, is visited daily by about half of its users27 (refer to Gruzd and Mai27 for further details about social media platform use in Canada). In this study, Twitter and Facebook were the selected social media platforms based on use, frequency of use, efficiency of open information distribution (reach), and ability to engage with users directly and in real time.

The COVID-19 pandemic is a natural experiment, and this study will allow for rapid turnaround in the identification of early challenges and opportunities to guide evidence-informed social media communication policy and organisational decision-making. From this study, we will (1) seek to understand the challenges and strengths with the current PH social media approach to communicating with the public in Canada and (2) develop recommendations about how to enhance social media engagement with the public during an emergency response in the healthcare system in Canada and globally. Our study is relevant and timely and will contribute to the limited scholarly literature on how and to what extent Canadian PH agencies are using social media to engage with the public, particularly with a focus on emerging infectious diseases.24 The findings of this research will inform PH communication practices to effectively engage with the public in times of emergency events, like COVID-19, to mitigate misinformation, stigma and fear among Canadians.

Research question

What is the Canadian PH risk communication response to the COVID-19 pandemic in the context of social media?

Objectives

The objectives of this study were as follows:

1. Identify the social media level of engagement with the public by provincial PH agencies and PH leaders during active periods of the COVID-19 outbreak.

2. Identify the social media level of engagement with the public by the national PH agency and PH leaders during active periods of the COVID-19 outbreak.

3. Determine the strengths and weaknesses of provincial and national social media engagement content during COVID-19.

4. Characterise the public’s response to PH agencies and PH leaders on social media during active periods of the COVID-19 outbreak.

5. Collaboratively develop a Canadian social media emergency response set of guideline recommendations for PH and other health system organisations.

Methods and analysis

The proposed social media analysis will draw on Yin’s exploratory case study approach28 where the overall case is defined as the Canadian PH system social media response to COVID-19. The PH system is defined as provincial/territorial PH agencies, Medical Officers of Health, the Public Health Agency of Canada (PHAC), the Chief Medical Officer of Health and the public; embedded cases are represented by each provincial/territorial PH agency, PH actors and its public. The case is bounded by the time period 1 January 2020 until 3 September 2020; this represents the first wave of the pandemic in Canada, estimated to be an appropriate time frame to capture emergency milestones, given that SARS had an 8-month run during an earlier evolution of social media use.29 The case study approach will generate a holistic, deep understanding of the social media PH response with the public during COVID-19.

Conceptual framework

We will draw on the Centers for Disease Control and Prevention (CDC)’s Crisis Emergency and Risk Communication guidelines for social media and mobile media devices.30 The publicly available PHAC strategic risk communication framework does not address social media communication.

Data collection and sampling

Data collection is purposive; embedded cases allow for an examination of variation in PH response, and the public’s response, across the country. Social media postings on Twitter and Facebook focusing on COVID-19 from each PH authority will be pulled for the period of 1 January 2020–3 September 2020. Specifically, from each province, data from the PH agency, the Medical Officer of Health, and corresponding comments and replies from the public will be collected. Data will not be collected from Quebec due to limited funds for translation of French social media material. Nationally, data from the PHAC, the Chief Medical Officer of Health and corresponding posts from the public will be collected. Internationally, data will be collected from the WHO. The programming language Python 3 and Twitter’s application programming interface will be used to collect postings from Twitter, and NVivo’s NCapture program will be used to collect posts and comments from Facebook. Members of the public’s handles or other identifiable information will be deidentified from the dataset.

Sampling from this dataset will rely on the Canadian Institute for Health Information’s COVID-19 Intervention Scan (https://www.cihi.ca/en/covid-19-intervention-scan), a continuously updated database of government interventions and announcements related to the pandemic. Interventions are related to case finding and management, openings and closures, physical distancing, health workforce capacity, health services, and travel restrictions from federal and provincial governments. We anticipate choosing key interventions, defined as those that will have impact on a broad number of individuals, and selecting Twitter and Facebook data 3 days before and 3 days after the intervention was announced. It is expected that this will provide a sufficient understanding of how public mood might have anticipated or shifted with a new COVID-19 intervention.

Objectives 1 and 2: methods to identify the social medial level of engagement with the public by provincial and national PH agencies and leaders

Data

The data collected and sampled as described previously will be used, with the public’s response kept bundled with each COVID-19 government intervention announcement.

Variables

Neiger et al’s hierarchy for social media engagement, developed by a coalition at a social media metrics and standards summit, assesses an organisation or government’s stage of engagement with its audience.14 Early implementation of social media, seen as low engagement, is marked by one-way communication from the sender as a way to create a presence for followers. Metrics include the number of tweets posted and the number of followers (reach) and second-level followers (potential reach). Medium engagement signifies engaging in conversation through retweets or shares. Metrics include mutual followers, posters mentioning the government body or person and messaging directly to them, frequencies of responses to questions, etc. High engagement reflects the development of a partnership with the public and is measured by participation by the public in programmes, delivery of programmes, advocacy, etc. Neiger et al provide a comprehensive list of measures for each engagement level.14 Thus, engagement classification is based on systematic coding of social media content.

Analysis

Content analysis will measure engagement level from the text posted by the provincial, national and PH agencies and leaders (including the WHO) and the public.31 NVivo software will be used to organise the analysis.

Objective 3: methods to determine the strengths and weaknesses of provincial and national social media engagement content during COVID-19

Data

The data collected and sampled as described earlier will be used. In addition, the official website profiles of each PH authority (agency and actor) from across Canada will be analysed for COVID-19 information.

Variables/concepts

Drawing from the CDC’s Crisis Emergency and Risk Communication guidelines, we are interested in whether the message (1) provided adequate scientific expertise, (2) provided messages of self-efficacy, (3) used social media to provide emotional support, (4) used social media to listen to the public, and (5) collaborated and coordinated with credible sources. Descriptive variables of interest include content type (eg, type of social media platform, number of social media applications adopted, overall profile page and COVID-19 information included, and messaging services available); number of social media applications adopted by each PH authority and accessibility (ie, overall online presence, links on website homepage to social media platforms for ease of use and increased access).

Analysis

The directed content analysis will be conducted using NVivo software to organise the data and emerging findings. Descriptive variables will be collected and summarised using Excel 2013 (15.0).

Objective 4: methods to characterise the public’s response to PH agencies and PH leaders on social media during COVID-19

Data

The public response Twitter data collected and sampled as described earlier will be used to understand the sentiment to PH interventions.

Variables

Our initial list of emotions, which will be refined as we develop a test batch of tweets, include sadness, anger, disgust, surprise, fear, concern, distrust, confusion and positivity.

Analysis

A machine learning approach will be used to classify the postings and replies according to their emotional content using a subset of manually labelled postings and replies as training data.

Comparison with pre-COVID-19

In 2019, we carried out a similar analysis during non-emergency times. To compare findings, the ratio of message content focusing on COVID-19 to non-COVID-19-related messaging will be analysed descriptively. If the study sample is large enough, independent sample t-tests will be used to compare whether COVID-19 communications received more engagement than other messaging. We will consider paired t-tests for analysis on those government accounts that are the same before and after COVID-19.

Comparison with WHO

As described earlier, we will use WHO social media messaging as a benchmark from which to compare the Canadian content response.

Case study analysis

This case study report will present findings that discuss (1) provincial social media COVID-19 activity, (2) national social media COVID-19 activity, (3) the content of the activity, (4) how the public responded to the activity and (5) recommendations for the PH sector arising from the analysis.

Patient and public involvement

The public was not involved in the conceptualisation or design of this research protocol and will not be invited to comment on the results of this study.

Ethics and dissemination

As the data are in the public domain, research ethics approval is not required.32 Western University’s research ethics boards confirmed that this study does not require research ethics board review.

We have three primary knowledge translation goals: (1) to stimulate reflection about study findings among PH actors in Canada; (2) to align, inform and coordinate PH organisational and government social media (SM) communications practice by developing a Canadian social media emergency response set of guideline recommendations; and (3) to contribute to the science of both PH systems and social media research by disseminating findings. We will reach out to Canadian PH stakeholders (eg, Medical Officers of Health, PHAC, champion practitioners) to assist with messaging based on our findings (which we will situate in the larger risk communications literature). Our team of researchers and invited PH stakeholders will come together for dialogue and sense-making about the findings to then craft a social media emergency response set of guideline recommendations for a strong and feasible emergency response. We will disseminate a one-page summary and infographic of recommendations to provincial and federal government departments (eg, British Columbia Regional Health Authorities and PHAC) and other agencies to inform their emergency response SM strategies. We will initiate face-to-face meetings with key individuals in informal dialogue about the work. We will disseminate findings using relevant newsletters, websites and conferences (eg, International Conference on PH) to practitioners and researchers to raise interest in the research findings. Traditional scholarly dissemination, like journals (eg, Bulletin of the WHO), will target other researchers. We will reach out to our established PH contacts to assist with dissemination efforts.