Abstract
Background
Literature on health promotion evaluation and public understanding of health suggests the importance of investigating behaviour over time in conjunction with information environment trends as a way of understanding programme impact. We analysed population response to online promotion of an educational tool built by the Ask About Aspirin campaign in the USA to inform people about aspirin as a preventive aid.
Methods
We collected 156 weeks of time series data on audience behaviour, namely use of a self-assessment tool. We then used the Autoregressive Integrated Moving Average (ARIMA) modelling to predict that outcome as a function of paid search engine advertising, paid social media promotion and general search interest in aspirin.
Results
Through ARIMA modelling of tool engagement data adjusted for outcome series autocorrelation, we found a significant effect of online promotional effort on audience behaviour. Total paid search advertising positively predicted weekly total of individuals who started using the self-assessment tool, coefficient=0.023, t=3.28, p=0.001. This effect did not appear to be an artefact of broader secular trends, as Google search data on the topic of aspirin use did not add explanatory power in the final model nor did controlling for general search interest eliminate the significant coefficient for paid search promotion.
Conclusion
Results hold implications both for educational tool development and for understanding health promotion campaign effects. We witnessed substantial but ephemeral effects on tool use as a function of paid search efforts, suggesting prioritisation of efforts to affect search engine results as a dissemination tactic.
INTRODUCTION
Although the Internet affords access to health information for many, actual patient engagement with information reflects more than the simple existence of information online. For example, search behaviour and social media interaction respond to government announcements and news coverage and a primary explanation for media campaign success and failure has been investment in campaign message exposure.1–4 Weeks and colleagues found correspondence between the volume of news regarding United States Preventive Services Task Force (USPSTF) mammography screening guidelines and online search activity related to mammography.3 Similarly, Southwell and colleagues found social media postings related to Zika virus in three countries to be a function of news coverage.1 We need to investigate what promotional strategies can encourage patient engagement with information on the assumption that availability of educational tools and aids alone will not guarantee attention.
An evidence-based recommendation warranting promotion is the use of aspirin to prevent heart attack or stroke, a recommendation not equally suitable for all people but which could offer important advice to certain men and women over 45 years old.5 Internet-based self-assessment tools to educate people about the use of aspirin could help people understand whether the strategy is relevant for them, but such tools are only useful if people find and use them.
Literature on media coverage and campaign effects yields three observations for our discussion. First, because people regularly encounter a wide array of information, investigation of any single promotional effort should acknowledge potential influence of secular trends beyond the promotional campaign.6 Media campaigns do not operate in a vacuum. Second, social science investigation focused on cross-sectional data alone risks eliding an opportunity for understanding communication processes and dynamics.7 Asking about patterns of aggregate behaviour over time can illuminate how populations respond to macro-level variation in an information environment.8 Third, all information exposure is not equal, per se; research suggests different patterns associated with active seeking of health information versus the serendipity of information encountered incidentally.9
The prevalence of an online tool designed to educate people about aspirin hypothetically could help us explain population-level response to the tool. We also can ask whether efforts to make information about a tool salient in search engines such as Google and efforts to promote the tool via social media platforms such as Facebook have equal effects. To accomplish such inquiry, we organised longitudinal data with a time-based unit of analysis so that aggregate patient engagement and campaign efforts could be studied on the same plane.
METHODS
Site for research
In 2012, the Minnesota Heart Health Programme launched the Ask About Aspirin campaign to promote USPSTF recommendations on aspirin use for prevention of cardiovascular disease. See Oldenburg et al for discussion of campaign development and an initial community-based intervention trial.10 Subsequent to campaign launch, staff developed a website-based tool to allow people to assess whether they should discuss a daily aspirin regimen with a healthcare professional; users described their circumstances and received a tailored assessment. Intended audiences for the tool included men 45 to 79 years old and women 55 to 79 years old. In 2015, the campaign began substantive online promotion of the tool, meaning longitudinal assessment of 2014 to 2017 data included both a pre-promotion baseline period and a period of campaign promotion on online platforms.
Procedure
We organised data indicating explicit engagement with the online tool—not just Internet page views but actual initiated use of the tool—along the dimension of time, with weekly data for the period from 7 September 2014 through 2 September 2017 (a 3-year, 156-week span of time). We aligned the dependent variable time series indicating tool engagement and a series of independent variables, including weekly campaign promotion expenditures and general online search behaviour that could potentially explain tool use separately from campaign activity. The advertising firm responsible for online promotion and research university staff organising the campaign shared accounting records for advertising expenditures and Internet site traffic and activity. As described below, we also used Google Trends to obtain a general indicator of search activity on a relevant topic as an indicator of trend outside of campaign activity.
Measures
Our task required both a dependent variable time series and various independent variable time series. Our dependent variable for analysis was engagement with the self-assessment tool created by the Ask About Aspirin campaign. Drawing on recorded initiation of the tool, our measure of tool engagement was the number of unique visitor self-assessment starts in a week.
To measure online promotional effort by the campaign, we organised advertising expenditures corresponding to four venues: Google, Yahoo!, Facebook and Twitter. Of note are the two different categories of online venues covered by that list: search engine and content platforms (where people can encounter information when searching) and social media platforms (where peers connect, correspond and share content). Although there is overlapping functionality between the two types of venues, we also sought to assess the potentially independent contribution of promotional efforts in each space.
We created three indicators of promotional activity by the campaign, each again organised by week. We computed a total weekly expenditure variable, in dollars, comprising expenditure on advertising via Google, Yahoo!, Facebook (including advertisements on the margin of the frame as well as sponsored posts appearing in a person’s main content feed) and Twitter advertising appearing in people’s feeds on that platform. We also separately computed (in dollars) a paid search promotion total for each week (using Google and Yahoo! expenditure) and a paid social media promotion total (using Facebook and Twitter expenditure) for each week.
We also wanted to include a measure of general population interest in aspirin, which could stand as a proximal indicator of potential secular variation in the public salience of the preventive strategy. We used Google Trends to assess total searches originating for ‘aspirin’ relative to total Google searches overall for the same period (using a scale of 0 to 100, where 50 represents half the volume as 100 but not a specific absolute number).
Analysis
Using a week as our unit of analysis (meaning n=156 for all analysis), we used time-series analysis to understand the relationship of campaign activity and engagement with the online tool. In using a time-series approach, we avoided a potential pitfall— spuriousness resulting from trend autocorrelation—that correlational analysis alone would not have avoided. Because time-series data can reflect autocorrelation that makes observed relationships spurious, interpretation of bivariate correlations alone to link time-series data is inadvisable.11 To assess the relationship between independent variables and self-assessment starts, we used time-series analysis to predict self-assessment starts as a function of other observed trends and date. To do that, we fit an Autoregressive Integrated Moving Average (ARIMA) model. We first looked at the simple 156-week time series of self-assessment starts without any adjustment for secular trend, which generated a Ljung-Box statistic that indicated whether significant autocorrelation resided in the dependent time series. As described in the Results section, we then added an autoregressive component to our base model to eliminate evidence of autocorrelation and included independent variables as predictors.
RESULTS
Over the 3-year period observed, each of the variables in our analysis demonstrated substantive variation. Self-assessment starts in a week ranged from 0 to 627. Online promotion expenditures each week on search engine and social media platforms varied considerably, ranging from 0 to US$10 502. The campaign spent almost US$200 000 on such advertising, with $186 642 paid during the period in question. Google search index scores also varied to some extent, ranging from 72 to 100. The main question before us is how those trends related to one another.
As a first step, we analysed the dependent variable time series without any predictors or additional model components and found evidence of autocorrelation. Specifically, the Ljung-Box statistic (452.58, df=18, p<0.001) suggested significant autocorrelation. Secular trend accounted for variance in the outcome, in other words; that suggested controlling for autocorrelation would allow us to see an outcome trend attributable to factors other than secular trend. We then fit a simple ARIMA (1, 0, 0) model with no predictors aside from an autoregressive component to address this dependence between residuals, resulting in a statistically insignificant Ljung-Box statistic, 15.69, df=17, p=0.55, meaning we sufficiently reduced the time series to white noise to assume no autocorrelation in residuals. That ARIMA model offered a base into which we could add predictors.
Our next step was to compare the predictive power of total online promotion expenditure over and above the autoregressive component and general search behaviour linked to aspirin during the 156 weeks in question. That analysis revealed two important findings: a significant prediction coefficient for campaign online promotion expenditure and an insignificant coefficient for general search behaviour. That model achieved an R2 of 0.72 and stationary R2 of 0.72, and root mean square error was 78.69. The Ljung-Box statistic again was not significant, 13.28, df=17, p = 0.72. Total campaign online promotion expenditure emerged as a significant and positive predictor, coefficient=0.024, t=3.48, p=0.001. Total promotion expenditure predicted departures from expected trend in self-assessment starts, in other words. Google search index for aspirin did not garner significance as a predictor, −1.2, t=−0.56, p=0.58.
We also analysed two components of the total expenditure time series, total paid search promotion and total paid social media promotion, with a model including those two predictors, Google search index score for aspirin and the autoregressive component. That model also explained the dependent time series in similar fashion. The model achieved an R2 of 0.72 and stationary R2 of 0.72, and root mean square error was 78.75. The Ljung-Box statistic was not significant, 13.26, df=17, p = 0.72. Total paid search promotion was a significant and positive predictor, coefficient=0.023, t=3.28, p =0.001. Total paid social media promotion was not a significant predictor, 0.059, t=1.31, p=0.19, nor was Google search index for aspirin, −1.02, t=−0.49, p=0.62.
The positive relationship over time of total paid search promotion and engagement with the self-assessment tool is apparent visually as well as in ARIMA statistics. Figure 1 shows the correspondence of two separate time series: the dependent variable and campaign expenditure for each week on paid search promotion (adjusted by a factor of 10 to allow similar ranges in values for visual comparison). As paid search expenditure rises, self-assessment tool starts increase as well and when paid search expenditure declines so do self-assessment tool starts.
Figure 1.
Self-assessment tool engagement and campaign paid search expenditure. Paid campaign search expenditure in increments of US$10. Self-assessment tool engagement is the weekly total number of tool initiations by unique visitors. Horizontal axis is week and vertical access is value for the trend in question in a given week.
DISCUSSION
The importance of encouraging patient engagement with a health information tool through active promotion of that tool is evident here. With promotion, we see substantive corresponding tool use; without promotion, tool use waned noticeably. Beyond the use of promoting the tool to generate patient engagement, however, these data also underscore ideas about the nature of patient behaviour, information salience and the ephemeral effects of promotional efforts.
Note in figure 1 both the rise and fall of self-assessment starts. During an initial stage, the tool remained relatively unused prior to campaign promotion. Over time, we also can see at least one point during which self-assessment starts essentially return to zero. That period corresponds to a dramatic decline in paid online promotion. In other words, what we do not see here is a plateau effect whereby self-assessment starts reach a level in conjunction with campaign promotion and then remain at that level beyond campaign promotion efforts. Instead, we see a rise and fall in population tool engagement that appears to reflect the rise and fall of campaign promotional effort. That is simultaneously heartening for those who wish to use online promotion as a way of encouraging information engagement and a cautionary tale.
Clearly, in the case of encouraging novel information tool engagement, campaign organisers need to anticipate doing more than an initial promotional effort to achieve sustained use of the tool. Although some campaigns may be able to achieve peer-to-peer sharing of campaign information or to encourage news coverage that could extend the social salience of the tool beyond initial paid promotion, sustained information engagement appears to need sustained audience prompting in some form. (A difficult-to-achieve exception to that pattern may occur in the case of a tool that elicits repeated and habitual use, but in this case self-assessment is not likely a recurring behaviour for most people.) This pattern of sharp and immediate impact yet also decline in effect soon after exposure is consistent with an emerging picture from the health communication literature that highlights effects of promotional efforts via electronic media on health behaviour as both noteworthy but also vulnerable to decay over a relatively short period.1,12–14
This pattern of punctuated, short-term effect offers both hope and the need for caution for public health practitioners. The ephemeral nature of exposure-prompted behaviour is understandable given the torrent of information in which we regularly swim; today’s focused attention on a topic can be replaced by a variety of other spotlights the next day. In some ways, this tendency dates back almost a century, although Gitlin also has called attention to the increased volume of available information in recent years.15,16 That the Ask About Aspirin achieved apparent traction in audience engagement during the observed period is notable, but we also should understand the observed behaviour as likely vulnerable to decline absent continued promotional support.
Limitations
What we demonstrate in this analysis suggests the use of online promotion of a self-assessment tool to encourage understanding of aspirin as a heart attack and stroke prevention aid. Although we found a striking relationship over and above what secular trend and general online search interest in aspirin can explain, we also should acknowledge the nature of the campaign effort in this case. The partnership responsible for the campaign included a major research university with a long history of heart health promotion research and practice and a professional advertising team. Perceived source credibility likely enhanced the likelihood of self-assessment tool engagement. Moreover, outcomes other than tool use are beyond our scope here, although future work could investigate patient perceptions or additional prevention behaviour.
The scope of campaign investment also warrants comment. Ask About Aspirin campaign expenditure on search engine and social media promotion was not trivial, with almost US$200 000 spent primarily over a 2-year window of the 3-year time series. Such expenditure may be beyond the available budget for some community efforts and yet also is less than larger amounts routinely spent in the commercial advertising arena. Whether such spending is necessary to reach a minimal prevalence threshold is a question for future inquiry. The present results also illustrate an effect across 156 weeks of data. We need to remain agnostic as to the specific impact of any single-week expenditure, although visual inspection of figure 1 suggests the relationship between amount spent and self-assessment starts is not a perfect 1:1 ratio of dollars to starts. The relationship may harbour effect thresholds: the effect of promotion may not be linear or even monotonically increasing at all levels of expenditure, for example.
We also faced limitation in ascertaining the relative contribution of paid search promotion and paid social media promotion. Results suggested paid search promotion outperformed paid social media promotion in explaining self-assessment starts. That analysis, though, may have been constrained by the joint timing of paid search promotion and paid social media promotion in some weeks, as paid search promotion and paid social media promotion trends corresponded modestly but significantly, Pearson’s r=0.24, p=0.003, n=156. Under different circumstances, paid social media may independently explain engagement with online health tool information. Nonetheless, the prominence of paid promotion in search engine venues as an explanatory factor here suggests that at least for the adult population for whom aspirin use as a preventive tool is most relevant search results may be a more useful site for heart health campaigns to engage than social media platforms.
Available data offered an adequate opportunity to conduct time-series analysis, as we had 3 years of weekly data for independent and dependent variables. In considering time, however, it is also clear that our data are situated in a historical moment. In the context of decades of health promotion, online health assessment tools are a relatively novel phenomenon as is the recommendation for aspirin use. Conceivably, we could see an interaction with broad secular history over time such that the same analysis a decade or two into the future may not yield the same response to promotional effort. Nonetheless, our results suggest important verification that online promotion prompted engagement with an educational tool.
CONCLUSION
We collected time-series data on audience behaviour to understand the influence of paid search engine advertising and social media promotion on engagement with an educational tool describing aspirin as a preventive aid. Through ARIMA modelling of a 3-year series of tool engagement data, we discovered a discernible effect of online promotional effort on audience behaviour. Total paid search advertising was a significant and positive predictor of the weekly total of individuals who started using a self-assessment tool built by the campaign, coefficient=0.023, t=3.28, p=0.001. This effect did not appear to be an artefact of broader secular trends, as a proxy for general interest in the topic of aspirin use (in the form of Google search data) did not add explanatory power in the final model nor did controlling for general search interest eliminate the significant coefficient for paid search promotion.
Our exploration suggested several dimensions for consideration and avenues for future inquiry. The effect of online promotion of the tool was short-lived; when weekly expenditure on online promotion declined close to zero, so did the number of people using the tool. In addition, people other than the adult population for whom preventive aspirin use is most relevant might experience different patterns of online promotion effects than evident here. Results nonetheless illustrate potential for generating audience interaction with educational materials about cardiovascular disease risk reduction through an online promotion campaign.
What is already known on this subject.
-
►
New media technologies now afford Internet-based tools to engage patients and encourage patient-provider conversations about preventive strategies like aspirin use.
-
►
At the same time, we do not yet know whether patients will use such tools and how to optimally encourage use.
-
►
In conjunction with literature on public understanding of health and science, time-series methods using administrative data from a heart health promotion campaign offer one approach for testing whether investment in promotion of online tools through Internet search engines or social media can yield patient use and encourage conversations with healthcare professionals.
What this study adds.
-
►
We present time-series analysis of aggregate patterns in patient engagement with an online self-assessment tool built for the Ask About Aspirin campaign.
-
►
We offer an innovative approach that draws together data on secular trend, general search behaviour and campaign promotion over time to answer questions about Ask About Aspirin campaign impact as well as about the responsiveness of patient populations to search engine initiatives.
Policy implications.
-
►
Online promotion of Internet-based patient assessment tools appears vital to increasing use by populations. Without such promotion, use of the tool in our study declined to almost zero.
-
►
Practitioners should consider prioritising efforts to make tools and materials prominent and salient in search engine results rather than solely promoting tools via social media.
-
►
Longitudinal analysis of administrative data offers a novel method of exploring health campaign impact.
Acknowledgments
Funding This work was supported by the U.S. National Heart, Lung and Blood Institute (award no. 1R01HL126041).
Footnotes
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Readers interested in the weekly data reported in this manuscript can contact the corresponding author for a complete report of count totals used in analysis.
REFERENCES
- 1.Southwell BG, Dolina S, Jimenez-Magdaleno K, et al. Zika virus-related news coverage and online behavior, United States, Guatemala, and Brazil. Emerg Infect Dis 2016;22:1320–1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Towers S, Afzal S, Bernal G, et al. Mass media and the contagion of fear: the case of ebola in America. PLoS One 2015;10:e0129179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Weeks BE, Friedenberg LM, Southwell BG, et al. Behavioral consequences of conflict-oriented health news coverage: the 2009 mammography guideline controversy and online information seeking. Health Commun 2012;27:158–66. [DOI] [PubMed] [Google Scholar]
- 4.Hornik RC, ed. Public health communication: evidence for behavior change. Mahwah, NJ: Lawrence Erlbaum Associates, 2002. [Google Scholar]
- 5.Ittaman SV, VanWormer JJ, Rezkalla SH. The role of aspirin in the prevention of cardiovascular disease. Clin Med Res 2014;12:147–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Viswanath K, Finnegan JR. Community health campaigns and secular trends: insights from the Minnesota Heart Health Program and Community Trials in Heart Disease Prevention Hornik R, ed. Public health communication: evidence for behavior change. New York: Lawrence Erlbaum, 2002:289–312. [Google Scholar]
- 7.Stephenson MT, Southwell BG, Yzer M. Advancing health communication research: issues and controversies in research design and data analysis In: Thompson TL, Parrott R, Nussbaum JF, eds. Routledge handbook of health communication. 2nd edn. New York: Routledge, 2011:560–77. [Google Scholar]
- 8.Soumerai SB, Ross-Degnan D, Kahn JS. Effects of professional and media warnings about the association between aspirin use in children and Reye’s syndrome. Milbank Q 1992;70:155–82. [PubMed] [Google Scholar]
- 9.Kelly B, Hornik R, Romantan A, et al. Cancer information scanning and seeking in the general population. J Health Commun 2010;15:734–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Oldenburg NC, Duval S, Luepker RV, et al. A 16-month community-based intervention to increase aspirin use for primary prevention of cardiovascular disease. Prev Chronic Dis 2014;11:130378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.McCleary R, Hay RA, Meidinger EE, et al. Applied time series analysis for the social sciences. Thousand Oaks, CA: Sage Publications, 1980. [Google Scholar]
- 12.Fan DP. Predictions of public opinion from the mass media. Westport, CT: Greenwood, 1988. [Google Scholar]
- 13.Foss KA, Southwell BG. Infant feeding and the media: the relationship between Parents’ Magazine content and breastfeeding, 1972–2000. Int Breastfeed J 2006;1:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Southwell BG, Hornik RC, Fan DP, et al. , 2000. Can news coverage predict mammography use? A time series analysis to predict health behavior using the ideodynamic model. International Communication Association annual conference. Acapulco, Mexico. [Google Scholar]
- 15.Lippmann W Public opinion. New York: Penguin Books, 1922. [Google Scholar]
- 16.Gitlin T Media unlimited: how the torrent of images and sounds overwhelms our lives. New York: Henry Holt and Company, 2002. [Google Scholar]