Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Sep 2.
Published in final edited form as: J Health Commun. 2023 Aug 1;28(9):573–584. doi: 10.1080/10810730.2023.2236976

Evaluation of the “We Can Do This” Campaign Paid Media and COVID-19 Vaccination Uptake, United States, December 2020–January 2022

Benjamin Denison a,*, Heather Dahlen a, Jae-Eun C Kim a, Christopher Williams a, Elissa Kranzler a, Joseph N Luchman a, Sarah Trigger b, Morgane Bennett b, Tyler Nighbor b, Monica Vines b, Elizabeth L Petrun Sayers b, Allison N Kurti b, Jessica Weinberg b, Leah Hoffman a, Joshua Peck b
PMCID: PMC10529331  NIHMSID: NIHMS1917808  PMID: 37528606

Abstract

Public education campaigns are promising methods for promoting vaccine uptake. In April 2021, the U.S. Department of Health and Human Services launched the We Can Do This COVID-19 public education campaign. This study is one of the first evaluations of this COVID-19 public education campaign. We tested associations between channel-specific campaign exposure (i.e., digital, TV, radio, print, and out-of-home advertising) and COVID-19 first-dose vaccinations among a nationally representative online sample of 3,278 adults. The study introduces novel ways to simultaneously evaluate short- and long-term cumulative media dose, filling an important gap in campaign evaluation literature. We observed a positive, statistically significant relationship between the short-term change in digital media dose and the likelihood of first-dose vaccination, and a positive, statistically significant relationship between long-term cumulative TV dose and the likelihood of first-dose vaccination. Results suggest that both digital and TV ads contributed to vaccination, such that digital media was associated with more immediate behavioral changes, whereas TV gradually shifted behaviors over time. As findings varied by media channel, this study suggests that public education campaigns should consider delivering campaign messages across multiple media channels to enhance campaign reach across audiences.

Keywords: Media campaign evaluation, COVID-19 vaccination, multilevel modeling

Introduction

The COVID-19 pandemic has resulted in nearly 1.1 million deaths in the United States as of January 2023 (Centers for Disease Control and Prevention [CDC], 2023), representing one of the largest public health challenges of our time. Despite freely available vaccines, many adults remain vaccine hesitant, citing safety concerns related to the accelerated speed of the vaccines’ development and testing (Cassata, 2021) and distrust of government, medicine, and science (Okoro et al., 2022; Latkin et al., 2022). In April 2021, the U.S. Department of Health and Human Services (HHS) introduced the We Can Do This campaign (the Campaign; Weber et al., 2022), with the goal of promoting COVID-19 vaccination uptake. In the April-September 2021 period, the Campaign reached over 90% of U.S. adults with billions of advertising impressions (Nielsen, 2021).

Public education campaigns can help activate positive behavior change and prevent audiences from engaging in harmful behaviors, resulting in improved health-related outcomes for large segments of the population (Wakefield et al., 2010) and leading to behavior changes such as smoking cessation, consistent condom use, consistent seat belt use, reduced binge drinking, and increased vaccination (Hornik, 2002; Noar, 2006; Noar et al., 2009; Swanton et al., 2015; Yadav & Kobayashi, 2015; Wakefield et al., 2010). In this study, we asked the following research questions: (1) What is the relationship between probable exposure to the Campaign, measured by media dose across multiple channels, and first-dose COVID-19 vaccination uptake? (2) Does the relationship between probable Campaign exposure and COVID-19 vaccination uptake vary by media channel and the period over which media dose is administered?

Theoretical framework

The development of the Campaign was informed by several theories of behavior change, notably the theory of planned behavior (TPB; Ajzen, 1985), the transtheoretical (stages of change) model (TTM; Prochaska & DiClemente, 1983), and the health belief model (HBM; Rosenstock et al., 1988). The TPB states that volitional behavior is predicted by one’s intention to engage in that behavior, which is in turn predicted by beliefs about the advantages and disadvantages of the behavior, beliefs about whether other people are or are not engaging in the behavior, and perceptions of control over one’s ability to perform or abstain from the behavior. The TTM posits that individuals do not move through behavior change quickly and proposes six stages of change through which individuals can advance: precontemplation, contemplation, preparation, action, maintenance, and termination. According to the HBM, behavior change is informed by one’s perceptions of behavioral susceptibility, severity, benefits, and behaviors; behavior change is also reliant on a cue to action, which triggers behavioral decision-making.

Together, these theories suggest that behavior change is predicted by changes in behavioral beliefs, attitudes, and intentions. Exposure to the We Can Do This Campaign messages through routine media use is expected to induce change in these cognitions (e.g., increased pro-vaccination beliefs), resulting in downstream effects on targeted behavior. These theories also suggest that the further one is along the cognition pathway (i.e., further away from being vaccine hesitant and closer to being vaccine confident), the closer they are to changing their behavior. For example, a person who is thinking about getting vaccinated (e.g., in the contemplation stage of the TTM) is expected to be closer to behavior change than a person for whom vaccination is not yet a consideration (e.g., in the precontemplation stage); accordingly, exposure to the Campaign could encourage vaccine uptake in a shorter amount of time for individuals who are further along the cognition pathway.

In recent years, public health campaigns have evolved to reach audiences through different media channels, with an emphasis on effecting change in targeted outcomes through digital (including social) media (Zeller, 2019; Brown et al., 2014; Boles et al., 2014; Alhabash et al., 2015). Dissemination of the We Can Do This campaign used multiple channels—TV, digital, out-of-home (OOH), radio, and print advertising—to reach diverse audiences, with an emphasis on reaching the Movable Middle—individuals not yet vaccinated but open to vaccination. Evidence from related research has shown that there is variation within vaccine-hesitant populations with respect to their awareness of and beliefs about COVID-19 vaccination (Edwards et al., 2021; Murphy et al., 2021). In line with behavior change theory, we expected similar differences in the audience we aimed to reach through the Campaign, such that some individuals within this group would be more open to vaccination than others (e.g., contemplation versus precontemplation stages), for whom behavior change would require considerable time and messaging. The type of media channel through which individuals receive messaging, in conjunction with the extent of their vaccine openness, may influence behavior change. Leveraging the different media types and lengths of exposure in this evaluation provided an opportunity to explore these associations within this study.

Relevant campaign evaluation literature

The Centers for Disease Control and Prevention (CDC) guidelines for public health campaign implementation and evaluation—largely informed by tobacco control and prevention programs—indicate that a campaign is expected to run for at least 3 to 6 months to achieve awareness of the health issue, 6 to 12 months to influence behavioral attitudes, and 12 to 18 months to influence behavior (CDC, 2018). Evidence from some campaign evaluations has supported these guidelines, with observed changes in cognitions and behavioral outcomes over similar periods (Duke et al, 2015; Farrelly et al., 2017). In contrast, evaluations of other campaigns have suggested that behavior change can occur more rapidly, such as within 1 to 6 months after a campaign’s launch (Hall et al., 2012; Cates et al., 2011; Davis et al., 2011; Wakefield et al., 2011).

Extant literature thus has provided support for how campaigns are associated with behavioral changes; however, the targeted behavior is often not vaccine uptake, and research linking campaign exposure and COVID-19 vaccine uptake is nascent. Furthermore, evaluations of campaigns that have addressed other behaviors may have limited application to COVID-19 vaccination given differences in behavioral attributes (e.g., temporality, the recipient of behavioral benefits, the required frequency of the behavior) (Rimal & Lapinski, 2021). In addition, there is variation with respect to the length of time between initial campaign exposure and change in targeted outcomes, so we expect there might be interplay between media type and time periods over which effects may occur, particularly for a behavior that is less studied in the context of campaign evaluation. The lack of a clear theoretical guidance concerning the expected length of time needed between different types of media exposure and behavior change meant it was important to test the association of both short-term change in media and cumulative media exposure on vaccine uptake.

In line with understanding the differential effects of media channels and the length of time between campaign exposure and behavior change, the purpose of this study was to evaluate the relationship between the We Can Do This Campaign channel-specific media dose from April 2021 through January 2022 and individual first-dose COVID-19 vaccination uptake at the designated market area (DMA) level.1 This analysis built upon prior Campaign research that found a positive relationship between digital media dose and COVID-19 vaccination uptake (Williams et al., 2023). In the current study, we expanded on that work by including four additional types of Campaign paid media (TV, radio, print, OOH) and modeled both the short-term change and the long-term cumulative influence of these media on first-dose vaccinations. Further, although prior national public education campaign evaluations have limited assessments of the differential impact by media channel, this study is one of the first evaluations assessing the relationship between media channel-specific dose across multiple channels and COVID-19 vaccine uptake (Davis et al., 2016; Dens et al., 2018).

Methods

Data

To evaluate the potential relationship between Campaign exposure and vaccine uptake, this study used data from two sources: weighted individual-level survey data and media market-level Campaign paid media dose data. Survey data came from the COVID-19 Attitudes and Beliefs Survey (CABS), a nationally representative, probability-based longitudinal survey of U.S. adults (ages 18+) that was administered every 4 months through the NORC at the University of Chicago’s (NORC) AmeriSpeak research panel. The final analytic sample consisted of 3,278 unique respondents who completed at least three CABS waves between January 2021 and January 2022. Market-level Campaign paid media dose data included measures of five types of general market (i.e., U.S. residents ages 18+) paid media placed per DMA in a given week: local digital impressions (including social media), local TV gross rating points (GRP), local radio impressions, local print impressions, and local OOH advertising spending. See Supplement for additional CABS and Campaign media information, along with details of how these data sources were combined for analysis.

Measures

Dependent variable

The dependent variable was a dichotomous indicator of whether the survey respondent self-reported receiving either the first dose of a two-dose COVID-19 vaccination or a single dose vaccine in the current broadcast week. Specifically, respondents were asked about their vaccination status in all four waves, but only Waves 3 and 4 asked respondents to report the date that they were vaccinated. For individuals who reported being vaccinated as of Wave 1, 2, or 3, the vaccination date in Wave 3 was used. For individuals vaccinated as of Wave 4 but not Wave 3, vaccination dates provided in Wave 4 were used. The earliest first-dose vaccination date reported by any respondent was in December 2020. See Supplement for more information on the quality assurance process used to review self-reported first-dose vaccination data.

Independent variables

Paid media dose by DMA represents the likelihood that an individual in a DMA was exposed to the Campaign and the likely frequency of exposure. Each Campaign media channel was evaluated individually to determine the association of channel-specific dose with vaccination. Campaign digital, print, and radio doses were measured using DMA-level impressions, the publishers’ estimates of the number of times a Campaign advertisement was seen, read, or heard, normalized to DMA market size. Digital impressions included site direct (buys made directly on websites), programmatic (buys made through ad exchanges), and social media impressions (buys made directly on social media platforms) that were combined into the total number of digital impressions per 100,000 population in the DMA for each broadcast week across all three forms of digital media dose. DMA-level Campaign paid TV dose (broadcast and cable TV) was measured using TV GRPs, a composite measure of estimated audience reach and frequency. Reaching 1% of the audience once is equivalent to 1 GRP. Campaign paid OOH dose was measured using billed costs,2 also normalized by DMA population size, to estimate audience exposure to advertising like billboards and bus shelters. Respondents were assigned paid media dose by broadcast week based on their DMA of residence.

We examined two pathways through which media may influence vaccination behavior. We operationalized two types of variables to test these pathways for each media channel. The first set of media variables was a lagged change in weekly media dose between weekt-2 and weekt-1, which measures the shock, that is, the short-term change in media dose on vaccine uptake. We used the lagged change between weekt-2 and weekt-1 to account for time between the exposure and subsequent behavior to account for logistics needed to plan for and have a vaccine appointment. The short-term pathway treated increased media dose as a catalyst, which may influence more immediate vaccination action.

The second pathway used a lagged, cumulative decay measure to capture the potential influence of long-term accumulated paid media dose on vaccinations. Cumulative measures were included to represent exposure over a longer time period to foster belief and attitudinal change necessary for behavior change, such as someone who is vaccine hesitant becoming vaccine confident and getting vaccinated (Breuer & Brettel, 2012; Gerber et al., 2011; Burke & Srull, 1988). As studies have indicated that cumulative advertising impact may be the most powerful in the immediate and that its influence decays over time, it is important to assess older dose differently than recent dose (Gerber et al., 2011; Chong & Druckman, 2010). To account for differential influence of dose based on when it was delivered, we used the concept of a media dose “half-life” within the cumulative measure, by which an older dose’s influence decays over time. Within the context of Campaign advertisements, the half-life was the time we theorize it would take for the influence of half of the week’s media dose to dissipate. Campaign advertising studies have used a range of half-lives varying by campaign topic (Hill et al., 2013). In line with Chong and Druckman (2010), we used a media half-life of 3 weeks. This approach assumed that the influence of the media campaign aggregated over time but decayed exponentially each week, with 50% of the previous weeks’ values removed after every 3-week period. To ensure that the cumulative half-life measures were acting on the survey respondents during the same time as the change variables described above, we also lagged these decay variables by 2 weeks.3

Covariates

To account for exogenous factors potentially confounding the relationship between campaign exposure and vaccine uptake, we controlled for changes in CDC-reported COVID-19 cases and deaths by DMA between weekt-2 and weekt-1.4 We controlled for individual respondent characteristics measured at CABS Wave 1, including age, gender, race, ethnicity, education, household income, political ideology, rurality, essential worker status, and preexisting health conditions.5 We also controlled for other COVID-19 vaccination campaign TV GRPs from non-HHS state and local health agencies; companies; and non-profit organizations. Finally, we controlled for baseline, self-reported COVID-19 vaccine uptake and confidence at CABS Wave 1 and DMA-level vaccinations. See Supplement for information on covariates.

Study design / model specification

We used a multilevel logistic regression model to evaluate the relationship between Campaign paid media dose and the likelihood of first-dose COVID-19 vaccination. We considered a multilevel structure of individuals nested within DMAs with random intercepts for each DMA, a common design in neighborhood and health studies (Mulaga et al., 2021; Merlo et al., 2016; Larsen & Merlo, 2005). The respondent-DMA-week panel spanned up to 61 weeks, from December 2020 to January 2021. For the model and the additional robustness checks (see Supplemental), the unit of analysis was the respondents–broadcast week, with each respondent-week nested within a DMA. Figure 1 shows the number of first-dose vaccinations over time among study respondents. Weekly new vaccinations among respondents peaked in the February–April 2021 period. The Campaign officially launched April 1, 2021. The earliest that a survey respondent reported having received a first-dose COVID-19 vaccination was December 2, 2020, which may indicate vaccine clinical trial participation. Note that respondents who were vaccinated before December 2, 2020, were dropped from the analysis due to lagged differentials.

Figure 1. Histogram of Reported First Dose of COVID-19 Vaccination Dates, United States, November 30, 2020–January 31, 2022.

Figure 1.

Open in a new tab

Note: The first broadcast week for this analysis began the week of November 30, 2020 (Monday). The earliest that a survey respondent reported having received a first-dose COVID-19 vaccination was December 2, 2020, which may indicate vaccine clinical trial participation. For comparison, a graph of reported first dose COVID-19 vaccination uptake from the CDC can be found in Figure S1.

Recognizing that there may have been time periods in which a DMA was more likely to see changes in paid media advertising and periods in which individuals were more likely to get vaccinated, we included time fixed effects (weekly dichotomous variables) in the model. The model and the robustness check models were weighted for known population values and vaccine uptake values from CDC data, using sampling weights from Wave 3, in Stata 17 (StataCorp, 2021).

Results

Multilevel logistic regression model results

Results from the model (Table 1) showed a positive, statistically significant relationship between the weekly change in Campaign digital impressions and the likelihood of first-dose vaccination (β = 0.000014; p < 0.01; 95% CI, [0.000006, 0.00002]). This result indicates that short-term increases in Campaign digital impressions were associated with a higher likelihood of first-dose COVID-19 vaccination in the subsequent week. The long-term cumulative Campaign digital impressions coefficient was not statistically significant.

Table 1.

Relationship Between Paid Advertising Media Dose and the Likelihood of Vaccination, United States, December 21, 2020–January 30, 2022

First-dose COVID-19 Vaccination (Sth. Err.)
Short-Term Change (Paid Local Media)
 HHS Campaign Digital Impressions 1.43e-05***
(4.24e-06)
 HHS Campaign TV GRPs −0.000195
(0.00676)
 HHS Campaign Radio Impressions −5.49e-06
(7.26e-06)
 HHS Campaign Print Impressions −9.72e-06
(6.30e-06)
 HHS Campaign OOH Spending 1.20e-05
(1.14e-05)
 Other Campaign TV GRPs (Non- HHS) −0.00293
(0.00215)
Long-Term Accumulation (Paid Local Media)
 HHS Campaign Digital Impressions −1.58e-06
(1.28e-06)
 HHS Campaign TV GRPs 0.00475*
(0.00219)
 HHS Campaign Radio Impressions −1.53e-06
(3.64e-06)
 HHS Campaign Print Impressions −6.33e-07
(2.23e-06)
 HHS Campaign OOH Spending 1.25e-05
(9.74e-06)
 Other Campaign TV GRPs (Non- HHS) 0.000760
(0.000582)
Exogenous Factors
Change in COVID-19
 Cases per 100,000 people −0.000310
(0.000280)
 Deaths per 100,000 people 0.00732
(0.00891)
Demographics
Income 0.0782**
(0.0275)
Female −0.0234
(0.0625)
Age 0.414***
(0.0445)
Education 0.177***
(0.0363)
Essential Worker Status 0.0107
(0.0822)
Political Ideology −0.240***
(0.0422)
Preexisting Health Condition 0.0902
(0.0579)
Rurality −0.132**
(0.0482)
Non-Hispanic Black/African American −0.0204
(0.0874)
Hispanic/Latino 0.231**
(0.0829)
Non-Hispanic Other Race 0.125
(0.127)
Initial Vaccine Confidence and DMA-level Vaccination
Wave 1 Vaccine Confidence 1.082***
(0.0536)
Share of DMA That Was Vaccinated 0.0106**
(0.00391)
Week Dummy Variables Not Reported for Brevity
Constant −10.508***
(1.092)
DMA Variance 0.137
(.300)
Observations 84,188
DMAs 202
Open in a new tab

Note: The dependent variable is a dichotomous measure of whether a respondent received the first dose of a COVID-19 vaccination each week. As we lag our independent variables by two time periods, the earliest vaccination date in our models is December 21, 2020. The gender variable is based on NORC’s survey methodology. The data used in this analysis derives from multiple sources, including the CABS. See the Supplemental Material section of this paper for further discussion of media data source information.

***

p < 0.001;

**

p < 0.01;

*

p < 0.05.

Model results also suggested a positive and statistically significant relationship between long-term cumulative Campaign TV GRPs and vaccine uptake (β = 0.0048; p = 0.028; 95% CI, [0.0005, 0.0091]). Increased cumulative Campaign TV GRPs were associated with an increased likelihood of first-dose vaccination in the subsequent week. The short-term change in Campaign TV GRPs was not statistically significant.

We did not observe significant relationships between vaccine uptake and short-term change or long-term cumulative media dose for the additional HHS media channels—radio, print, and OOH media—suggesting they were not associated with changes in weekly vaccinations in this model. Additionally, there were no observed significant relationships between exposure to other COVID-19 vaccination TV advertising campaigns and vaccine uptake.

With respect to covariates included in the model, both vaccine confidence and the vaccinated share of the DMA were significantly related to vaccine uptake (Table 1). As expected, respondents with greater vaccine confidence in Wave 1 were more likely to report receiving a first-dose COVID-19 vaccination respondents to respondents with less vaccine confidence (β = 1.08; p < 0.01; 95% CI, [0 .97736, 1.18712]). The share of the DMA with a first-dose vaccination was also a statistically significant predictor of vaccine uptake (β = 0.0103; p < 0.01; 95% CI,[(0.00259, 0.017971]).

Our model indicated that when holding all media channels at zero (i.e., in the absence of any public media campaign on vaccinations), respondents who were older, more affluent, and more educated were more likely to get first-dose vaccinations compared to respondents who were younger, less affluent, and less educated. Our results also indicated that respondents who live in urban or suburban areas compared to more rural areas and those who identify their political ideology as moderate or liberal compared to conservative were more likely to have received a COVID-19 first-dose vaccination. Compared to non-Hispanic White respondents, Hispanic respondents were more likely to have received a COVID-19 vaccination. These results align with recent studies on COVID-19 vaccine willingness and intentions (Kelly et al., 2021; Berg & Lin, 2021).

Predicted probabilities of first-dose vaccination

Although weekly changes in digital impressions and the long-term accumulation of Campaign TV GRPs were significantly associated with vaccine uptake, it remained essential to examine their marginal effects to observe the substantive relationship between the digital and TV advertising campaigns and the likelihood of vaccination. To do this, we estimated the predicted probabilities of a first-dose vaccination across both the short-term weekly change in digital impressions and the long-term weekly cumulative TV GRPs, while holding all other variables at their means. Figure 2 presents the predicted probabilities for the short-term change in digital impressions. With more than 95% of the values of the weekly change in digital impressions variable falling between –30,000 and 30,000, we found that a decrease in digital impressions of 30,000 was associated with a 0.8% probability of vaccination. In contrast, an increase in digital impressions by 30,000 was associated with a 2.1% probability of vaccination in the current week, more than doubling the predicted probability of vaccine uptake. The probability of vaccination was positive because all other model variables were held at their means, and decreased impressions still mean that impressions were delivered.

Fig 2. Average Change in Weekly Local Digital Impressions on the Likelihood of Individual First-Dose Vaccination, United States, December 21, 2020–January 30, 2022.

Fig 2.

Open in a new tab

Note: Because we lag our independent variables by two time periods, the earliest vaccination date in our models is December 21, 2020. This figure is derived from analysis reported in Table 1.

Figure 3 presents the predicted probabilities for long-term cumulative TV GRPs. When examining the predicted probabilities for cumulative TV GRPs, 95% of the observations ranged from zero to 100 cumulative GRPs. When the cumulative half-life value of TV GRPs was zero, the predicted probability of vaccination in the current week was 1.3%, holding all other variables at their means. Increasing to 100 cumulative TV GRPs was associated with a 2.0% probability of vaccination in the current week, a 53.4% increase in the probability of vaccination. This represents expected effects over the range of GRPs delivered. Robustness checks that were run to test model sensitivity supported the main model findings (Supplement Table S4).

Fig 3. Average Change in Weekly Local TV GRPs on the Likelihood of Individual First-Dose Vaccination, United States, December 21, 2020–January 31, 2022.

Fig 3.

Open in a new tab

Note: Because we lag our independent variables by two time periods, the earliest vaccination date in our models is December 21, 2020. This figure is derived from analysis reported in Table 1.

Discussion

In this study, we evaluated the relationship between Campaign paid media exposure and COVID-19 first-dose vaccinations across a nationally representative sample of U.S. adults. We found that short-term increases in digital advertising and greater levels of long-term accumulation of TV advertising were associated with an increased likelihood of self-reported first-dose COVID-19 vaccinations, controlling for sociodemographic characteristics and other DMA-level trends. These findings complement the Campaign’s previous evaluation work which found a positive association between recalled exposure to Campaign advertisements and COVID-19 vaccine confidence (Kranzler et al, 2023), and a positive association between paid digital Campaign impressions and the likelihood of first-dose COVID-19 vaccinations (Williams, 2023). We expanded on these previous findings by modeling both the short-term change and the long-term, cumulative influence of the Campaign on first-dose vaccinations, and by including four additional types of Campaign paid media.

Our findings regarding short- and long-term effects suggest that TV-based vaccination campaign advertising may take longer than digital advertising or may require an accumulation of messages to elicit behavior change. These results align with findings from evaluations of other health campaigns, which have shown associations between cumulative TV expenditures and changes in targeted behaviors (Duke et al., 2019; Farrelly et al., 2009). It is possible that the accumulation of long-term TV advertising was more effective for individuals who were initially less open to vaccination, prompting them to move through one of the theorized pathways to behavior change (Prochaska & DiClemente, 1983; Ajzen, 1985; Rosenstock et al., 1988) over a longer period. The differential impact of digital compared to TV Campaign exposure may also be explained by the differences in how messages delivered across these two mediums influence behavior. An abundance of evidence suggests public education campaigns delivered through TV are successful at changing behavioral social norms, leading to subsequent behavior change – a process that may unfold over time (Kranzler & Hornik, 2019; Murphy-Hoefer, 2020; Palmgreen et al., 2001).

Compared to TV advertisements, digital Campaign advertisements may be more likely to elicit behavior change in the short term. Digital advertisements often include clickable links or other interactive elements to guide individuals to resources, such as those needed to schedule a vaccination appointment, thereby facilitating the behavior. According to the health belief model, “cues to action” (Prochaska & DiClemente, 1983) are expected to have a greater influence on behavior in situations where the perceived benefits and threats are high, or when the perceived barriers are low (Champion & Skinner, 2008). Individuals who have more pro-vaccination attitudes and beliefs (e.g., the belief that COVID-19 vaccination is effective) may respond to advertisements, digital or otherwise (e.g., TV), as cues to action that prompt behavior change. Accordingly, digital Campaign advertisements may both cue individuals with pro-vaccination attitudes to get vaccinated and provide the tools (e.g., clickable links with resources to schedule a vaccination appointment) to immediately move forward with this behavior, thereby eliciting short-term effects on vaccination. The unique combination of a cue to action and the provision of resources may be more immediately actionable such that it facilitates shorter-term behavior change among individuals with pro-vaccination attitudes.

Importantly, this does not mean that TV campaign advertisements cannot be cues to action, nor that digital campaign advertisements are unlikely to encourage longer-term change in outcomes. Rather, the findings from this study suggest that Campaign effects reflect the distinct but complementary ways in which different media channels, like TV and digital, influence behavior change. Moreover, the ways in which media channels complement each other in their effects on behavior may depend on the behavior of interest, audience characteristics, and the timing of a campaign.

Regarding the other media platforms examined in this study, significant associations between vaccine uptake and paid media dose through radio, print, and OOH channels were not observed, perhaps due to less investment in these resources than in digital and TV. The majority of Campaign media budget was spent on digital (43%) and TV (41%), as digital and TV channels were assumed to be the best way to reach the audience, whereas the radio, print, and OOH budgets were smaller (4%–7% each). This assumption was informed by media usage trends that show digital use is increasing and TV usage remains high, whereas fewer individuals are reading print media and listening to traditional radio than in the past (Shearer, 2021; Rainie, 2021; Auxier & Anderson, 2021). Although the results of this evaluation demonstrated significant associations between digital and TV advertising and vaccine uptake, it is unclear if the nonsignificant findings for radio, print, and OOH advertising were due to a lack of impact or lower monetary investment. Public education campaigns, including the one described in this study, typically leverage multiple channels to surround the target audience with messaging on a variety of platforms and to reach people where they routinely consume media. Additional research exploring additive and multiplicative effects of multichannel use may be helpful for demonstrating possible cross-channel synergies.

The current results were robust across a variety of model specifications and variable constructions, providing confidence in our modeling approach and the observed relationships between both media channels and COVID-19 vaccine uptake. However, our findings of short-term influences of digital dose and long-term influences of TV dose on vaccine uptake may be specific to COVID-19 vaccination, the general market adult scope of assessed campaign dose, or the distribution of the audience in terms of their level of readiness to initiate a given behavior. Advertisements that promote other behaviors (e.g., smoking cessation, physical activity) or are tailored to specific audiences (e.g., young adults or parents) may show a different relationship, as the unique combination of attributes that define a given behavior and the characteristics of a particular audience may influence the effectiveness of corresponding campaign messaging. Future research could explore Campaign influence differences by audience and assess the potential relationship between media tailored for and focused on specific audiences and COVID-19 vaccination.

These findings offer evidence that a mass media public education campaign can play an important role in educating the public about vaccination. Of note, a large public education campaign is just one of many factors that may have influenced an individual’s decision to get a COVID-19 vaccination and that may work in collaboration with other efforts such as policy changes, vaccine logistics, conversations with doctors, and messaging from government and private companies.

This study also presented a novel method to evaluate both short-term and long-term effects of TV and digital paid Campaign dose simultaneously using multilevel modeling and weekly panel data. By incorporating the timing and geographic footprint of the Campaign, this methodological strategy provided an opportunity to examine multiple pathways that paid Campaign dose can impact individuals across different media streams and at similar time points. This evaluation strategy can be similarly applied to other public health campaigns with complementary media streams to examine how different health behaviors respond to the presence of multiple media channels being leveraged at the same moment.

Limitations

This study is not without limitations. The dependent variable—date of first-dose vaccination—was self-reported and subject to recall bias. However, recent research has supported the validity of self-reported measures to assess COVID-19 vaccine status (Siegler et al., 2021). This current study also analyzed paid media data during a discrete period of the Campaign, and thus the results may not reflect dose–response relationships at other times. Further, probable Campaign exposure for each respondent was represented by dose in each respondent’s DMA, in line with previous studies (Williams, 2023; Davis et al., 2018; Duke et al., 2019). Our use of exogenous measures of Campaign exposure complements previous work demonstrating an association between self-reported Campaign recall (an endogenous measure of exposure) and vaccine confidence (Kranzler, 2023). However, it is possible that actual exposure differed from DMA-level media dose due to variations in respondents’ media consumption.

Results were also subject to omitted variable bias, or when an unmeasured confounder was related to both the Campaign dose and vaccination uptake. We addressed this possible bias by controlling for various DMA- and individual-level characteristics. However, not all potential external confounders, such as policy changes or vaccine logistics were able to be included in this study. Another limitation is nonresponse bias, which may limit inferences that can be made about the general population; we addressed nonresponse bias using survey weights and design adjustments. Survey respondents were recruited using English- and Spanish-language materials only, so adults in the United States who are not fluent in either language are underrepresented. Accordingly, findings may not apply beyond the populations examined in this study.

Importantly, it should be noted that this evaluation pertains to only part of the We Can Do This Campaign, focusing on the relationship between paid media dose and first-dose COVID-19 vaccination during the period under analysis. As the pandemic has continued, the Campaign has evolved to reach different populations and target different vaccination behaviors (such as booster uptake) that are outside the scope of this study, and these findings may not apply to future periods in the Campaign. The Campaign relied on approaches beyond paid media (e.g., earned media, social media influencers) to encourage first-dose vaccination. Other periods during which the Campaign was disseminated and approaches for delivering Campaign content, as described above, may be the topics of future analyses.

Conclusion

Our findings suggest that the HHS COVID-19 public education campaign successfully increased COVID-19 first-dose vaccination uptake. Long-term cumulative TV dose and short-term increases in digital dose were associated with an increased likelihood of COVID-19 first-dose uptake.

This analysis also enhances our understanding of the differential relationships between channel-specific campaign dose and health behavior. The above findings suggest that digital and TV dose may influence behavior differently. Specifically, results suggest that TV dose may drive attitudinal change over time, eventually leading to a specific behavior, whereas digital dose may encourage those already predisposed to a behavior to undertake the behavior due to the ways in which individuals engage with each form of media. The findings from this analysis suggest that public health campaigns may benefit from disseminating messages through multiple media channels, thereby reaching audiences across platforms.

Supplementary Material

Supp 1

Footnotes

1

Nielsen DMA (i.e., media markets) are regions that are exclusive geographic areas based on the footprint of home market traditional TV stations.

2

OOH spending was used rather than impressions since OOH advertising impressions are particularly hard to calculate.

3

Note that we also estimated the model using a 2-week half-life and a 4-week half-life, as well as with a 1-week lag and a 3-week lag.

4

Note that we lagged two periods to capture the short-term change pathway.

5

Descriptions of how these variables are measured and constructed are in the Supplement.

REFERENCES

  1. Ajzen I. (1985). From intentions to actions: A theory of planned behavior. In Kuhl J & Beckmann J (Eds.), Action Control (pp.11–39). Springer. https://link.springer.com/chapter/10.1007/978-3-642-69746-3_2 [Google Scholar]
  2. Alhabash S, McAlister AR, Quilliam ET, Richards JI, & Lou C. (2015). Alcohol’s getting a bit more social: When alcohol marketing messages on Facebook increase young adults’ intentions to imbibe. Mass Communication and Society, 18(3), 350–375. 10.1080/15205436.2014.945651 [DOI] [Google Scholar]
  3. Allen JA, Duke JC, Davis KC, Kim AE, Nonnemaker JM, & Farrelly MC. (2015). Using mass media campaigns to reduce youth tobacco use: A review. American Journal of Health Promotion, 30(2), e71–e82. 10.4278/ajhp.130510-LIT-237 [DOI] [PubMed] [Google Scholar]
  4. Auxier B, & Anderson M. (2021). A majority of Americans say they use YouTube and Facebook, while use of Instagram, Snapchat and TikTok is especially common among adults under 30. Pew Research Center. https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/ [Google Scholar]
  5. Berg MB, & Lin L. (2021). Predictors of COVID-19 vaccine intentions in the United States: The role of psychosocial health constructs and demographic factors. Translational Behavioral Medicine, 11(9), 1782–1788. 10.1093/tbm/ibab102 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Boles M, Adams A, Gredler A, & Manhas S. (2014). Ability of a mass media campaign to influence knowledge, attitudes, and behaviors about sugary drinks and obesity. Preventive Medicine, 67, S40–S45. 10.1016/j.ypmed.2014.07.023 [DOI] [PubMed] [Google Scholar]
  7. Brown J, Kotz D, Michie S, Stapleton J, Walmsley M, & West R. (2014). How effective and cost-effective was the national mass media smoking cessation campaign ‘Stoptober’? Drug and Alcohol Dependence, 135, 52–58. 10.1016/j.drugalcdep.2013.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Breuer R, & Brettel M. (2012). Short- and long-term effects of online advertising: Differences between new and existing customers. Journal of Interactive Marketing, 26(3),155–166. 10.1016/j.intmar.2011.12.001 [DOI] [Google Scholar]
  9. Burke RR, & Srull TK. (1988). Competitive interference and consumer memory for advertising. Journal of Consumer Research, 15(1), 55–68. 10.1086/209145 [DOI] [Google Scholar]
  10. Cassata C. (2021, June 22). Doctors debunk 9 popular COVID-19 vaccine myths and conspiracy theories. Healthline. Retrieved December 13, 2022, from https://www.healthline.com/health-news/doctors-debunk-9-popular-covid-19-vaccine-myths-and-conspiracy-theories [Google Scholar]
  11. Cates JR, Shafer A, Diehl SJ, & Deal AM. (2011). Evaluating a county-sponsored social marketing campaign to increase mothers’ initiation of HPV vaccine for their preteen daughters in a primarily rural area. Social Marketing Quarterly, 17(1), 4–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Centers for Disease Control and Prevention. (2022). COVID data tracker. U.S. Department of Health and Human Services. Retrieved January 13, 2023, from https://covid.cdc.gov/covid-data-tracker
  13. Centers for Disease Control and Prevention. (2018). Best Practices User Guides: Health Communications in Tobacco Prevention and Control. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health. https://www.cdc.gov/tobacco/stateandcommunity/guides/pdfs/health-communications-508.pdf [Google Scholar]
  14. Chong D, & Druckman JN. (2010). Dynamic public opinion: Communication effects over time. American Political Science Review, 104(4), 663–680. 10.1017/S0003055410000493 [DOI] [Google Scholar]
  15. Davis KC, Uhrig J, Rupert D, Fraze J, Goetz J, & Slater M. (2011). Effectiveness of a mass media campaign in promoting HIV testing information seeking among African American women. Journal of Health Communication, 16(9), 1024–1039. 10.1080/10810730.2011.571342 [DOI] [PubMed] [Google Scholar]
  16. Davis KC, Patel D, Shafer P, Duke J, Glover-Kudon R, Ridgeway W, & Cox S. (2018). Association between media doses of the Tips From Former Smokers campaign and cessation behaviors and intentions to quit among cigarette smokers, 2012–2015. Health Education & Behavior, 45(1), 52–60. 10.1177/1090198117709316 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Davis KC, Shafer PR, Rodes R, Annice K, Hansen H, Patel D, Coln C, & Beistle D. (2016). Does digital video advertising increase population-level reach of multimedia campaigns? Evidence from the 2013 Tips From Former Smokers campaign. Journal of Medical Internet Research, 18(9), Article e235. 10.2196/jmir.5683 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Dens N, De Pelsmacker P, Goos P, Aleksandrovs L, & Martens D. (2018). How consumers’ media usage creates synergy in advertising campaigns. International Journal of Market Research, 60(3), 268–287. 10.1177/1470785317751333 [DOI] [Google Scholar]
  19. Duke JC, Alexander TN, Zhao X, Delahanty JC, Allen JA, MacMonegle AJ, & Farrelly MC. (2015). Youth’s awareness of and reactions to The Real Cost national tobacco public education campaign. PLOS ONE, 10(12), Article e0144827. 10.1371/journal.pone.0144827 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Duke JC, MacMonegle AJ, Nonnemaker JM, Farrelly MC, Delahanty JC, Zhoa X, Smith AA, Roa P & Allen JA. (2019). Impact of The Real Cost media campaign on youth smoking initiation. American Journal of Preventive Medicine, 57(5), 645–651. 10.1016/j.amepre.2019.06.011 [DOI] [PubMed] [Google Scholar]
  21. Edwards B, Biddle N, Gray M, & Sollis K. (2021). COVID-19 vaccine hesitancy and resistance: Correlates in a nationally representative longitudinal survey of the Australian population. PLOS ONE, 16(3), Article e0248892. 10.1371/journal.pone.0248892 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Farrelly MC, Nonnemaker J, Davis KC, & Hussin A. (2009). The influence of the national truth® campaign on smoking initiation. American Journal of Preventive Medicine, 36(5), 379–384. 10.1016/j.amepre.2009.01.019 [DOI] [PubMed] [Google Scholar]
  23. Farrelly MC, Duke JC, Nonnemaker J, MacMonegle AJ, Alexander TN, Zhao X, Delahanty JC, Rao P, & Allen JA. (2017). Association between The Real Cost Media campaign and smoking initiation among youths—United States, 2014–2016. Morbidity and Mortality Weekly Report, 66(2), 47–50. 10.15585/mmwr.mm6602a2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Garaus M, Wagner U, & Bäck A-M. (2017). The effect of media multitasking on advertising message effectiveness. Psychology & Marketing, 34(2), 138–156. 10.1002/mar.20980 [DOI] [Google Scholar]
  25. Gerber AS, Gimpel JG, Green DP, & Shaw DR. (2011). How large and long-lasting are the persuasive effects of televised campaign ads? Results from a randomized field experiment. American Political Science Review, 105(1), 135–150. 10.1017/S000305541000047X [DOI] [Google Scholar]
  26. Hall IJ, Rim SH, Johnson-Turbes CA, Vanderpool R, & Kamalu NN. (2012). The African American women and mass media campaign: A CDC breast cancer screening project. Journal of Women’s Health, 21(11), 1107–1113. 10.1089/jwh.2012.3903 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hill SJ, Lo J, Vavreck L, & Zaller J. (2013). How quickly we forget: The duration of persuasion effects from mass communication. Political Communication, 30(4), 521–547. 10.1080/10584609.2013.828143 [DOI] [Google Scholar]
  28. Hornik RC (Ed.). (2002). Public health communication: Evidence for behavior change (1st ed.). Routledge. 10.4324/9781410603029 [DOI] [Google Scholar]
  29. Kelly BJ, Southwell BG, McCormack LA, Bann CM, McDonald PDM, Frasier AM, Bevc CA, Brewer NT, & Squiers LB. (2021). Predictors of willingness to get a COVID-19 vaccine in the U.S. BMC Infectious Diseases, 21, Article 338. 10.1186/s12879-021-06023-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kranzler EC, Luchman JN, Williams CJ, Model TA, Ostby R, Vines M, Weinberg J, Petrun Sayers EL, Kurti AN, Trigger S, Hoffman L, & Peck J. (2023). Recalled Exposure to COVID-19 Public Education Campaign Advertisements Predicts COVID-19 Vaccine Confidence. Journal of Health Communication, 28(3), 144–155. 10.1080/10810730.2023.218189 [DOI] [PubMed] [Google Scholar]
  31. Larsen K, & Merlo J. (2005). Appropriate assessment of neighborhood effects on individual health: Integrating random and fixed effects in multilevel logistic regression. American Journal of Epidemiology, 161(1), 81–88. 10.1093/aje/kwi017 [DOI] [PubMed] [Google Scholar]
  32. Latkin C, Dayton L, Miller J, Yi G, Balaban A, Boodram B, Uzzi M, & Falade-Nwulia O. (2022). A longitudinal study of vaccine hesitancy attitudes and social influence as predictors of COVID-19 vaccine uptake in the US. Human Vaccines & Immunotherapeutics, 18(5), Article 2043102. 10.1080/21645515.2022.2043102 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Llupià A, Mena G, Olivé V, Quesada S, Aldea M, Sequera VG, Ríos J, García-Basteiro AL, Varela P, Bayas JM, & Trilla A. (2013). Evaluating influenza vaccination campaigns beyond coverage: A before–after study among health care workers. American Journal of Infection Control, 41(8), 674–678. 10.1016/j.ajic.2013.04.006 [DOI] [PubMed] [Google Scholar]
  34. Merlo J, Wagner P, Ghith N, & Leckie G. (2016). An original stepwise multilevel logistic regression analysis of discriminatory accuracy: The case of neighbourhoods and health. PLOS ONE, 11(4), Article e0153778. 10.1371/journal.pone.0153778 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mulaga AN, Kamndaya MS, & Masangwi SJ. (2021). Examining the incidence of catastrophic health expenditures and its determinants using multilevel logistic regression in Malawi. PLOS ONE, 16(3), Article e0248752. 10.1371/journal.pone.0248752 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Murphy J, Vallières F, Bentall RP, Shevlin M, McBride O, Hartman TK, McKay R, Bennett K, Mason L, Gibson-Miller J, Levita L, Martinez AP, Stocks TVA, Karatzias T, & Hyland P. (2021). Psychological characteristics associated with COVID-19 vaccine hesitancy and resistance in Ireland and the United Kingdom. Nature Communications, 12, Article 29. 10.1038/s41467-020-20226-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Nielsen (2021). Digital Ad Ratings and Total Ad Ratings. We Can Do This Campaign, P18+ Reach for campaign running April 5, 2021–September 26, 2021. [Google Scholar]
  38. Noar SM. (2006). A 10-year retrospective of research in health mass media campaigns: Where do we go from here? Journal of Health Communication, 11(1), 21–42. 10.1080/10810730500461059 [DOI] [PubMed] [Google Scholar]
  39. Noar SM, Palmgreen P, Chabot M, Dobransky N, & Zimmerman RS. (2009). A 10-year systematic review of HIV/AIDS mass communication campaigns: Have we made progress? Journal of Health Communication, 14(1),15–42. 10.1080/10810730802592239 [DOI] [PubMed] [Google Scholar]
  40. NORC. (2019, February 18). Technical overview of the AmeriSpeak panel NORC’s probability-based household panel. Retrieved May 10, 2022, from https://amerispeak.norc.org/content/dam/amerispeak/research/pdf/AmeriSpeak%20Technical%20Overview%202019%2002%2018.pdf
  41. Okoro O, Kennedy J, Simmons G Jr., Vosen EC, Allen K, Singer D, Scott D, & Roberts R. (2022). Exploring the scope and dimensions of vaccine hesitancy and resistance to enhance COVID-19 vaccination in Black communities. Journal of Racial and Ethnic Health Disparities, 9, 2117–2130. 10.1007/s40615-021-01150-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Prochaska JO, & DiClemente CC. (1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51(3), 390–395. 10.1037/0022-006X.51.3.390 [DOI] [PubMed] [Google Scholar]
  43. Rabe-Hesketh S, & Skrondal A. (2006). Multilevel modelling of complex survey data. Journal of the Royal Statistical Society Series A, 169(4), 805–827. 10.1111/j.1467-985X.2006.00426.x [DOI] [Google Scholar]
  44. Rainie L. (2021). Cable and satellite TV use has dropped dramatically in the US since 2015. Pew Research Center. https://www.pewresearch.org/fact-tank/2021/03/17/cable-and-satellite-tv-use-has-dropped-dramatically-in-the-u-s-since-2015/ [Google Scholar]
  45. Rimal RN, & Lapinski MK. (2021). Attribute-centred theorizing to address behavioural changes. Bulletin of the World Health Organization, 99(11), 828–833. 10.2471/BLT.20.285363 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Rosenstock IM, Strecher VJ, & Becker MH. (1988). Social learning theory and the health belief model. Health Education Quarterly, 15(2),175–183. 10.1177/109019818801500203 [DOI] [PubMed] [Google Scholar]
  47. Sagy I, Novack V, Gdalevich M, & Greenberg D. (2018). Mass media effect on vaccines uptake during silent polio outbreak. Vaccine, 36(12), 1556–1560. 10.1016/j.vaccine.2018.02.035 [DOI] [PubMed] [Google Scholar]
  48. Shearer E. (2021). More than eight-in-ten Americans get news from digital devices. Pew Research Center. https://www.pewresearch.org/fact-tank/2021/01/12/more-than-eight-in-ten-americans-get-news-from-digital-devices/ [Google Scholar]
  49. Siegler AJ, Luisi N, Hall EW, Bradley H, Sanchez T, Lopman BA, & Sullivan PS. (2021). Trajectory of COVID-19 vaccine hesitancy over time and association of initial vaccine hesitancy with subsequent vaccination. JAMA network open, 4(9), e2126882-e2126882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC. [Google Scholar]
  51. Swanton R, Allom V, & Mullan B. (2015). A meta-analysis of the effect of new-media interventions on sexual-health behaviours. Sexually Transmitted Infections, 91, 14–20. 10.1136/sextrans-2014-051743 [DOI] [PubMed] [Google Scholar]
  52. Wakefield MA, Loken B, & Hornik RC. (2010). Use of mass media campaigns to change health behaviour. Lancet, 376(9748), 1261–1271. 10.1016/S0140-6736(10)60809-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Weber MA, Backer TE, & Brubach A. (2022). Creating the HHS COVID-19 public education media campaign: Applying systems change learnings. Journal of Health Communication, 27(3), 201–207. 10.1080/10810730.2022.2067272 [DOI] [PubMed] [Google Scholar]
  54. Williams CJ, Kranzler EC, Luchman JN, Denison B, Fischer S, Wonder T, Ostby R, Vines M, Weinberg J, Petrun Sayers EL, Kurti AN, Trigger S, Hoffman L, & Peck JF. (2023). The Initial Relationship Between the United States Department of Health and Human Services’ Digital COVID-19 Public Education Campaign and Vaccine Uptake: Campaign Effectiveness Evaluation. Journal of Medical Internet Research, 25, e43873. 10.2196/43873 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Yadav R-P, & Kobayashi M. (2015). A systematic review: Effectiveness of mass media campaigns for reducing alcohol-impaired driving and alcohol-related crashes. BMC Public Health, 15, Article 857. 10.1186/s12889-015-2088-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Zeller M. (2019). Evolving “The Real Cost” campaign to address the rising epidemic of youth e-cigarette use. American Journal of Preventive Medicine, 56(2), S76–S78. 10.1016/j.amepre.2018.09.005 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supp 1
NIHMS1917808-supplement-Supp_1.docx (150.5KB, docx)

RESOURCES