Getting beyond impressions: an evaluation of engagement with breast cancer-related Facebook content

Abstract

Background

Reaching young adults with health messages has been a documented challenge in public health. Public health researchers have initiated studies to assess how social media are changing health communication. In 2014, the Centers for Disease Control and Prevention (CDC) launched social media-based health education initiatives on Facebook to increase knowledge of breast health and breast cancer among women under age 45 and those at higher risk for developing the disease. The current study used digital analytics and metrics to describe the impact of these social media efforts on health communication.

Methods

Engagement rate was calculated by taking the average engagement rate for 574 posts published by the CDC Breast Cancer Facebook page in multiple categories, including CDC campaign specificity, content type, time of day, and year posted. Linear regression was used to model the effect of campaign content.

Results

Engagement rate (ER) was highest for content shared for the Know:BRCA campaign posts (ER =6.4), followed by the non-campaign related posts (ER=5.5), and the Bring Your Brave posts (ER =4.6). Overall engagement rate decreased from 2014–2016. Photos consistently produced the most significant engagement rate overall.

Conclusions

We found that users were more likely to click, share, comment, or like the content of the post that had photos. These data suggest that that branded, visual content is more effective in facilitating engagement. These findings will be used to adjust both free and paid social media efforts for the CDC Breast Cancer Facebook page.

Introduction

Efforts to reach adolescents and young adults with health messages have been challenging in public health (1). However, there has been recently documented success in engaging these audiences with social media and online videos, which has increased in the past decade, especially among 18–29 years old regardless of race and ethnicity (2-4). Social media sites can provide a space for to engage users in dialogue, build or engage communities on specific topics, and encourage individuals to interact with one another about information pertaining to disease prevention, early detection, treatment, and survivorship (5,6). These sites enable users to share information in a timely manner and empower them to make health decisions informed by contributions of other visitors (6). Social media sites, including Facebook that has over 1.65 billion users (7), allows for an increase in the bidirectional flow of information, specifically regarding health information (8).

As familiarity with digital media has developed for those planning and implementing campaigns, an increasing number of health communicators are using social media as a part of communications strategy (9,10), but there are limited measurement methods used to understand the communication landscape and to critically assess intervention effectiveness (11). Some studies have described how social media are changing health communication in public health settings, including those that focus on cancer-related information (11-13).

In 2009, the Centers for Disease Control and Prevention (CDC) launched initiatives to increase knowledge of breast health and breast cancer among women, particularly among those under age 45 and those at higher risk for developing the disease. Efforts focused, in part, on the development of social media, public education campaigns that targeted young women with increased risk for developing the disease. CDC launched the Know:BRCA and Bring Your Brave (14,15) social media campaigns to educate young women about breast health and risk factors for early onset breast cancer. Know:BRCA focused primarily on factors related to hereditary breast and ovarian cancer (HBOC), promoting the Know:BRCA (13) web tool for young women to use to assess their personal risk for carrying BRCA1/2 gene mutations. The Bring Your Brave campaign tells real stories via photos and videos about young women whose lives have been affected by breast cancer in an effort to inspire young women to learn their risk for breast cancer, talk with their health care provider about their risk, and live a healthy lifestyle (14).

In this study, we sought to understand the impact of social media content type on Facebook user engagement for the Know:BRCA and Bring Your Brave public education campaigns.

Methods

The CDC Breast Cancer Facebook page was created on August 7, 2014. The Know:BRCA campaign was launched via social media in September, 2014. Content from Know:BRCA was featured on the CDC Breast Cancer Facebook page, the CDC Cancer Twitter account, CDC website, partner websites, as well as paid advertisements on Facebook. The first phase of the Bring Your Brave campaign launched in early May, 2015, with a dedicated website. Extensive promotion of the campaign website and videos was carried out on digital media, including on the CDC Breast Cancer Facebook page, CDC YouTube account, the CDC Cancer Twitter page, and the social media pages of partners, as well as paid advertisements on Facebook.

Analytics data (N=574 posts) from the CDC Breast Cancer Facebook page for the time periods August 2014 to mid-April 2016 were used. We reviewed engagement rate based on whether a post utilized campaign content, the type of post (video, status/link, and photos), time of day, and year posted. Engagement rate was calculated by dividing the number of unique users that liked, clicked, shared, and or commented by the number of unique users reached.

Engaged users included those unique users that liked, clicked, commented, and/or shared the content. Reached users included those unique users who received impressions of the content on their Facebook pages, via newsfeed, timeline, or directly on the CDC Breast Cancer Facebook page.

Through Facebook Insights, users can see the number of page likes added, actions taken (such as clicks, likes, comments, and shares), unique people who engaged during a given time period, unique people who could have seen CDC Breast Cancer Facebook posts, and the number of times those people could have seen CDC Breast Cancer Facebook posts (15). These actions were quantified as engagements for the purpose of this evaluation.

Linear regression was used to model the effect of campaign content [coded as “Know:BRCA”, “Bring Your Brave”, and “Non-Campaign” (referent category)] and content type [“Photo”, “Video (including YouTube link)”, “Link/Status” (referent category)] on engagement rate. Links and status updates were grouped together given similar characteristics. No gross violations of assumptions were detected.

Given that time of day and year were related to engagement rate, campaign versus non-campaign content, and media type, we controlled for these variables (treated as categorical variables). As day of the week was not related to either engagement rate or campaign content, we did not include day of the week in final models. An alpha of 0.05 was used for all tests, and no co-linearity problems were detected.

The final model used was:

Engagement = β₀ + β₁Campaign + β₂Type + β₃Time + β₄Year + ε

These data were analyzed using the R environment for statistical computing (16).

Results

Engagement rate (ER) was highest for content shared for the Know:BRCA campaign posts (ER=6.4), followed by the non-campaign related posts (ER=5.5), and the Bring Your Brave posts (ER=4.6) (Table 1). For post type, posts that included a photo, regardless of campaign type, had the highest engagement rate (ER=5.6) followed by posts which were status/link (ER=4.7) and video (ER=3.8). Post released in the early a.m. and 2–6 p.m. hour had the highest engagement rates at 6.2 and 5.7, respectively. Posts shared in 2014 had the highest engagement rate 6.9, whereas those posted in 2016 had the lowest ER (3.8).

Table 1. Number of Facebook posts and engagement rate by post characteristic (N=574), 2014–2016.

Post characteristic	N	Mean engagement rate
Campaign vs. non-campaign
Campaign—Bring Your Brave	166	4.6
Campaign—Know:BRCA	29	6.4
General, not campaign-related	379	5.5
Post type
Photo	405	5.6
Video (including YouTube links)	39	3.8
Status or link	130	4.7
Time of day posted by CDC
12:00–7:59 am	156	6.2
8:00–9:59 am	130	4.7
10:00–11:59 am	182	5.0
12:00–1:59 pm	73	5.0
2:00–5:59 pm	29	5.7
6:00–9:59 pm	4	4.6
10:00–11:59 pm	0	0
Year posted
2014	139	6.9
2015	361	5.0
2016	74	3.8

CDC, Centers for Disease Control and Prevention.

Although campaign content was not significantly related to change in engagement rate, there was a significant association of media type and year with change in engagement rate (Table 2). When compared to links and status updates, photos had significantly higher change in engagement (0.9%, P<0.0001). The change in engagement rate for videos was not significantly different from that of status updates and links. Change in engagement rate was not significant different at different times of day. From 2014 through 2016, overall engagement rate for all content types decreased (Table 2).

Table 2. Change in engagement rate by Facebook post characteristic, 2014–2016.

Post characteristic	Change in engagement rate (% points)	95% CI	P value*
Post campaign vs. non
Campaign—Bring Your Brave	−0.25	(−0.69, 0.19)	0.27
Campaign—Know:BRCA	0.37	(−0.41, 1.15)	0.35
Not campaign	Referent	Referent	Referent
Post type
Photo	0.90	(0.47, 1.33)	<0.0001
Video (including YouTube links)	−0.36	(−1.15, 0.42)	0.37
Status or link	Referent	Referent	Referent
Time of day posted
12:00–7:59 am	−0.07	(−0.88, 0.75)	0.87
8:00–9:59 am	−0.73	(−1.56, 0.09)	0.080
10:00–11:59am	−0.64	(−1.44, 0.16)	0.12
12:00–1:59 pm	−0.65	(−1.52, 0.22)	0.14
2:00–5:59 pm	Referent	Referent	Referent
6:00–9:59 pm	−0.81	(−2.91, 1.29)	0.45
10:00–11:59 pm	N/A	N/A	N/A
Year posted
2014	Referent	Referent	Referent
2015	−1.43	(−1.86, −1.00)	< 0.0001
2016	−2.46	(−3.10, −1.83)	< 0.0001

*, Wald Chi-square. N/A, not applicable.

Without parsing out Know:BRCA and Bring Your Brave content, there was still no significant difference between campaign-specific content and general, non-campaign content. Photos continued to have a significantly higher change in engagement rate than other content types (Table 3).

Table 3. Change in engagement rate by Facebook post characteristic, all content; campaign vs. non-campaign, 2014–2016.

Post characteristic	Change in engagement rate (% points)	95% CI	P value*
Campaign
Campaign	0.35	(0.75, 0.06)	0.095
Non-campaign	Referent	Referent	Referent
Post type
Photo	0.93	(0.50, 1.37)	<0.0001
Video	0.39	(1.19, 0.41)	0.34
Link/status	Referent	Referent	Referent
Time of day posted
AM post	0.13	(0.56, 0.31)	0.57
PM post	Referent	Referent	Referent
Year posted
2014	Referent	Referent	Referent
2015–2016	1.82	(2.23, 1.40)	<0.0001

*, Wald Chi-square.

Within the Know:BRCA campaign specifically, no significant difference was observed in the year posted (Table 4). Yet within the Bring Your Brave campaign, photos produced a significantly higher engagement than all other content types (Table 5). The same was true for non-campaign, general breast cancer content (Table 6).

Table 4. Change in engagement rate by Facebook post characteristic, within Know:BRCA campaign, 2014–2016.

Post characteristic	Change in engagement rate (% points)	95% CI	P value*
Post type
Photo	0.66	(3.23, 1.90)	0.60
Video/link status	Referent	Referent	Referent
Time of day posted
AM post	1.29	(3.58, 0.99)	0.26
PM post	Referent	Referent	Referent
Year posted
2014	Referent	Referent	Referent
2015–2016	2.34	(4.40, 0.29)	0.027

*, Wald Chi-square.

Table 5. Change in engagement rate by Facebook post characteristic, within Bring Your Brave campaign, 2015–2016.

Post characteristic	Change in engagement rate (% points)	95% CI	P value*
Post type
Photo	1.50	(0.78, 2.21)	<0.0001
Video/link/status	Referent	Referent	Referent
Time of day posted
AM post	0.14	(0.61, 0.89)	0.71
PM post	Referent	Referent	Referent
Year posted
2015	Referent	Referent	Referent
2016	0.34	(1.00, 0.33)	0.32

*, Wald Chi-square.

Table 6. Change in engagement rate by Facebook post characteristic, within non-campaign content, 2014–2016.

Post characteristic	Change in engagement rate (% points)	95% CI	P value*
Post type
Photo	1.00	(0.54, 1.46)	<0.0001
Video/link/status	Referent	Referent	Referent
Time of day posted
AM post	0.11	(0.66, 0.44)	0.70
PM post	Referent	Referent	Referent
Year posted
2014	Referent	Referent	Referent
2015–2016	1.69	(2.16, 1.23)	<0.0001

*, Wald Chi-Square.

Within the Bring Your Brave campaign and non-campaign content, photos continued to have the most significant engagement rate (P<0.0001) (Tables 5,6), yet did not seem to have a significant change in engagement rate relative to other content within the Know:BRCA campaign (Table 4). Non-campaign content also seemed to have increased engagement in 2015 and 2016 (Table 6).

Discussion

The findings from this study suggests that factors such as content/media type affect user engagement. Within each campaign and among non-campaign content, photos produced the most significant engagement rate, meaning that users were most likely to engage with photos on CDC Breast Cancer’s Facebook page. While the overall number of users reached increased for posts from 2014 through 2016, we did not find that engagement changed meaningfully during this time. Engagement rate seemed to be highest on average for content posted by CDC between 2 and 6 p.m.

Our study showed that photos produced a significantly higher engagement rate, meaning users were more likely to click, share, comment, or like the content of the post. These findings are similar to those reported in the trade literature (17,18). Our study also showed that while content that was specifically related to a campaign was not significantly associated with change in engagement rate, there was a significant association of content type and year posted with engagement rate. Specifically, those photos were significantly associated with engagement rate. These data suggest that highly visual content may be more effective in facilitating engagement in public health social and digital media campaigns. This knowledge will assist public health practitioners in developing targeted health messages and potentially enhance engagement with selected audiences.

Prior public health campaigns have used social media as a component complimented with other more traditional media, such as print products (1,4), however, few have utilized social media-only formats to deliver campaign messages. The novelty of this format has limited the number of published studies to help ensure its efficacy as a public health communication method. Thus, there is little empirical evidence regarding the impact and utility of social media on health promotion (1,2). The dearth of evidence is due in part to the lack of innovative methods for analyzing campaign-generated data. Despite this, the emergence of digital health campaigns, like Bring Your Brave and Know:BRCA which focus on discrete and targeted audiences may have significant impact on health marketing and communication (2,6). As Web 2.0 and social media make the communication landscape increasingly participatory (10), the need for innovative methods for assessing the effectiveness of this communication method for health messaging has increased. Our study used novel methods to assess the impact of certain social media content characteristics on user engagement and thus contributed to evidence regarding the utility of this communication method.

The association between year and engagement rate makes the case for further evaluation of engagement as it may be related to changes in Facebook’s timeline algorithm. For example, in 2015, Facebook changed the way page content was presented on a user’s timeline (19). Priority was given to a user’s “friends” and paid advertisements. Overall reduction in engagement rate may have been impacted by changes in the Facebook algorithm that controlled how and when Facebook page content appeared in a user’s Newsfeed. Among trade publications and the social media community, it has been noted several times that Facebook changes in its newsfeed algorithm have led to decreased engagement overall (20,21).

Limitations

These results should be interpreted according to the following limitations.

While engagements as defined for Facebook (clicks, likes, comments, and shares) do illustrate that visitors are taking in and sharing content, it is not clear if this led to changes in behavior (conversations with family or healthcare providers or further research).

The Bring Your Brave and Know:BRCA campaign content specifically seeks to create awareness and change or improve behaviors that occur offline, for example with a healthcare provider. As there is a lack of data on these offline conversations and behaviors, the complete understanding of the effectiveness of CDC’s Breast Cancer content cannot be fully assessed.

Finally, some cell sizes in the analyses were too small for individual analysis, requiring that they be combined with other strata (e.g., status updates and links).

Conclusions

The CDC Breast Cancer Facebook page is being primarily used to disseminate health messages about breast health, breast cancer awareness and prevention, and risk reduction, particularly in young women (<45 years old). To expand the engagement with breast cancer content, future efforts could consider using more photos in their programs.

Future studies could include qualitative assessments or surveys about our content in order to fill the gap in understanding the effect of the campaign materials. Future studies could also look further into year-over-year differences, and explore effectiveness of this content across other social media channels.

As the campaign continues, Bring Your Brave will adjust both its use of free and paid social media channels to best reflect what resonates with the target audience. Because of the immediacy of social media versus more traditional media that require longer-term planning (such as magazines or television), it is possible to quickly adjust the course of a planned media placement and use the ongoing input to tailor messages and presentation.

Disclaimer: The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.