Abstract
Purpose
Understanding reactions to health communication campaigns is essential to designing effective messaging. Multi-year campaigns through The Helping to End Addiction Long-term® Initiative (HEALing) Communities Study (HCS) were implemented from April 2020 – June 2022. Responses to HCS campaign messages and statewide campaign messages were evaluated.
Design
Cross-sectional, self-reported surveys of different respondents at 3 timepoints.
Setting
N = 33 communities across Kentucky, Ohio, Massachusetts, and New York.
Subjects
N = 2368 respondents recruited from Facebook/Instagram.
Measures
Message acceptability survey items assessed attention, understandability, and visuals/text match. Message influence items assessed whether the message made the respondent want to carry naloxone, learn more about medications for opioid use disorder (MOUD), or seek help for OUD. Items were scaled from 1 (strongly disagree) to 5 (strongly agree).
Analysis
All outcomes were analyzed using linear mixed models adjusted for community as a random effect.
Results
Among naloxone campaigns, HCS message acceptability was higher than statewide messages (3.98, 95% CI = 3.94-4.02 vs 3.93, 95% CI = 3.89-3.97, P = 0.002). HCS message influence was also higher than those of statewide messages (3.44, 95% CI = 3.38-3.50 vs 3.40, 95% CI = 3.34-3.46, P = 0.034). Among HCS campaigns, naloxone messaging had the highest acceptability (3.96, 95% CI = 3.92-4.00), followed by MOUD stigma (3.73, 95% CI = 3.70-3.77) and MOUD awareness (3.71, 95% CI = 3.68-3.75, P < 0.001). The HCS naloxone campaign was also highest rated in terms of influence (3.47, 95% CI = 3.43-3.51, P < 0.001).
Conclusion
This analysis provides insight into acceptability and potential influence of locally-tailored, opioid-related health campaign messaging.
This work was performed under the HEALing Communities Study, ClinicalTrials.gov Identifier: NCT04111939, https://clinicaltrials.gov/study/NCT04111939.
Keywords: education/communications, awareness, strategies, opioid use disorder, stigma reduction, community, specific settings, health campaigns
Purpose
Opioid-related overdose deaths have become a major public health concern in the United States (US), with annual mortality increasing from 8050 in 1999 to 82 136 in 2022.1,2 A systematic review of the effectiveness of various strategies to reduce opioid use disorder (OUD) and opioid overdose deaths found communication campaigns to be one of the most promising strategies, and communication campaigns to reduce overdose deaths have proliferated.3,4 Most campaigns focus on awareness-raising messaging about opioid misuse and few address evidence-based treatment interventions compared with prevention. 3 Among addiction-related health messaging, there is evidence that use of non-stigmatizing language and communication through real stories of individuals with substance use disorder are most effective. Exposure to visual campaigns and narrative vignettes featuring individuals with OUD with non-stigmatizing language reduced stigmatizing attitudes among an online panel of healthcare providers, and increased the belief that medications for opioid use disorder (MOUD) are valuable. 5 Interviews with young adults in Illinois who use drugs found that personal narratives and encouragement to carry naloxone to protect friends are favorable messaging strategies.3,6 In Rhode Island, interviews with adults who had either experienced or witnessed an overdose and reported non-medical use of opioids found that participants value non-judgmental, plain-language communication and action-oriented steps to prevent overdoses in campaign messaging. 7
The Helping to End Addiction Long-term Initiative® (HEALing) Communities Study (HCS) was a multi-site, parallel-group, cluster randomized wait-list controlled trial testing the Communities That HEAL (CTH) intervention. 8 Implementation and evaluation of a series of messages within a multi-year communications campaign was 1 part of the three-pronged CTH approach aiming to decrease opioid-involved deaths in intervention (ie, Wave 1, n = 34) compared to wait-list (ie, Wave 2, n = 33) communities across Ohio, Kentucky, Massachusetts, and New York. 8
Previous research suggested the HCS media campaign did not have a measurable impact; the authors of that evaluation suggested the reason was inadequate message exposure. 9 However, an alternative explanation is that the messages were poorly received in the study population. In this analysis, we assess the plausibility of that alternative explanation.
To develop a better understanding of how community members perceived the HCS campaigns, we assess message acceptability and influence for 3 HCS campaigns (naloxone, MOUD stigma, and MOUD awareness) and other statewide naloxone campaigns running simultaneously to the HCS campaigns (for example, from a state department of health) in 33 Wave 1 communities (1 intervention community withdrew from the study immediately after randomization). We compare message acceptability and influence for (1) the HCS naloxone campaign vs statewide naloxone campaigns and (2) across the 3 HCS campaigns. We also evaluated the acceptability and influence of naloxone messaging in HCS campaigns compared to state campaigns within each state to determine whether there was variation that could be attributable to differences among statewide campaigns.
Methods
Design
The HCS communication campaigns supported the adoption of evidence-based practices (EBPs) by (1) improving awareness of and access to naloxone and MOUD, (2) driving demand for these EBPs, and (3) reducing stigma, a barrier to treatment and recovery for individuals with substance use disorder. 10 These objectives, rooted in social cognitive theory, aimed to enhance self-efficacy and confidence in behaviors such as asking about MOUD and carrying naloxone, increase positive outcome expectancies that MOUD can help with OUD recovery, and provide cues to action to increase demand for EBPs and stimulate behavior change. The campaign also prioritized reduction of stigma toward people with OUD, because stigma can prevent people from seeking treatment and receiving support from friends and family. 11 The HCS health communications campaigns were unique in their focus, designed to encourage adoption of specific behaviors (eg, carry naloxone, seek MOUD treatment) related to the EBPs of the CTH intervention.
The campaign framework and evaluation were developed by a workgroup of communications experts from the research sites, the data coordinating center, and an outside consultant. 10 The workgroup conducted formative research and developed a standardized strategy (ie, topics, objectives, priority audiences) and approach for the campaigns. Priority audiences included local community leaders and influencers, people at risk for an overdose or with OUD, and family and friends of those at risk. Each site had the latitude to craft community-tailored messages/imagery, but all messages contained similar calls to action (eg, Carry naloxone. Save a life.)
Campaigns, like the rest of the CTH, were implemented with local substance use coalitions, which were either existing or created for the purpose of the study. Before each campaign, research sites provided communities (eligibility criteria and definition in HEALing Communities Study Consortium, 2020) with the campaign topic, objectives, priority audience groups, and a set of messages that included a main message (information or story), tagline, call-to-action, webpage URL to visit for more information, and a choice of sample images to accompany digital and print media. 8 Communities were also given a set of 8 campaign playbooks which provided practical campaign implementation tips (eg, How to Build Relationships with Local Media, How to Create Your Campaign’s Distribution Plan) and a campaign dissemination template to help communities plan, customize, implement, and monitor campaign dissemination activities.
Building from these common elements, community coalitions planned and implemented unique, data-informed, and community-tailored campaigns while ensuring the overarching campaign framework and core messages were consistent across communities for evaluation comparability. From April 2020 – June 2022, 5 campaigns were implemented in intervention communities: overdose education and naloxone distribution (Naloxone, April – June 2020); stigma related to MOUD (MOUD Stigma, July – September 2020); MOUD Awareness (October – December 2020); MOUD Treatment Retention (June – December 2021); and Community Choice (ie, communities chose to repeat a prior campaign, February 2022 – June 2022). Further details are described in Lewis et al, 2024. 9
Sample
Campaign message acceptability and influence were assessed with the Campaign Evaluation Questionnaire (CEQ). Recruitment for and administration of cross sectional CEQ surveys occurred in n = 33 intervention communities for approximately 5-6 weeks for each CEQ survey, which were conducted at regular intervals until the end of the last campaign. A total of 6 CEQ iterations were conducted, but we include only 3 in the present analysis, as these included identical measures and the variables of interest. Recruitment within a community was stopped at n = 20 respondents for each CEQ to avoid oversampling of urban areas. Potential CEQ survey participants were recruited with Facebook/Instagram advertisements targeted in zip codes corresponding to HCS communities. HCS community residents ≥18 years were eligible to participate in the survey. Online consent was obtained for participants. Upon completion of the CEQ, each participant could enter a drawing to win a $100 electronic gift card. At the close of each of CEQ, 1 gift card was distributed to 1 winner in each of the 33 communities. A total of n = 2368 participants were included in this analysis.
The CEQ collected demographic information, including race/ethnicity (coded as non-Hispanic White, non-Hispanic Black, Hispanic, or other); sex (male, female) age (18-34, 35-49, 50-64; 65-74, 75 and over); and education (high school or less, some college/associate degree, bachelor’s degree, graduate degree). It also gathered information about perceived community attitudes toward opioid addiction and OUD, and about experience with someone who has an opioid addiction/OUD.
Each CEQ survey included a section about campaign message assessment. In CEQs A, B, and C, participants were shown still images from each of 3 HCS campaigns (naloxone, MOUD stigma, and MOUD awareness), and from statewide naloxone campaigns that were running at the time of survey [Supplemental Material], that were considered by state HCS staff to be the most frequently used and recognizable image from each campaign. Statewide campaign images selected by each state were: Massachusetts – Stop overdose campaign from the Massachusetts Department of Public Health; New York - Combat addiction campaign from the New York State Office of Addiction Services and Supports; Ohio – Stop Overdoses. Carry naloxone campaign from the Ohio Department of Health; and Kentucky – Carry the Spray campaign from the Lexington-Fayette County Health Department [Supplemental Material]. Of note, each research site selected 1 HCS campaign message for each of the 3 HCS campaigns and 1 statewide campaign message to show to all communities in their corresponding state. Each image included a set of questions about the message’s overall acceptability and influence on various behaviors and attitudes relating to OUD.
Measures
HCS naloxone campaigns and statewide naloxone campaigns were compared using a (1) message acceptability composite measure and (2) naloxone message influence measure [Supplemental Material]. The message acceptability composite was calculated using 3 items: this message grabbed my attention; this message is easy for me to understand; and this message has a picture and text that match. This acceptability measure was adapted from the message evaluation item in Sutfin et al, 2019. 12 The naloxone message influence measure was the mean response to “this message makes me want to carry naloxone when out in public.” Participants provided responses of strongly disagree, disagree, neither agree nor disagree, agree, or strongly agree (1-5) to each item. The naloxone message acceptability composite value was calculated by averaging response values of the corresponding 3 survey items. The naloxone message influence measure value was the mean of responses to the 1 survey item. The Cronbach’s Alpha for each measure exceeded 0.8 (α > 0.8) in all CEQ surveys, indicating good internal consistency.
The message influence composite measure was calculated with responses to the following items: this message makes me want to carry naloxone when out in public; this message makes me want to learn more about medication for opioid use disorder (MOUD); this message recommends medication for opioid use disorder; this message would encourage me to seek help if I had an opioid addiction/opioid use disorder; and this message would encourage me to seek help for a loved one if they had an opioid addiction/opioid use disorder. These items were created and scaled for the study based on the goals of the HCS campaigns. This composite measure was calculated using the same method as above with the mean of responses from 1 (strongly disagree) to 5 (strongly agree). The Cronbach’s Alpha for each message influence composite measure exceeded 0.8 (α > 0.8), indicating good internal consistency.
Because each state submitted unique statewide campaign images for the CEQ, we hypothesized that the effects may differ by state. We tested the moderating effects of state when comparing HCS naloxone messages to statewide naloxone messages. When comparing HCS messages to each other (naloxone, MOUD stigma, and MOUD awareness), we tested the moderating effects of both urban/rural classification and personal experience with OUD. Urban/rural classification was a binary variable determined by the respondent’s community of residence and its designation of urban or rural by HCS. Communities were assigned rural or urban status using the CDC NCHS urban-rural classification scheme where applicable. 13 When not applicable, the rural designation was assigned to communities in which there are fewer than 500 people per square mile while the rest were considered urban.14,15 The HCS utilized RE-AIM as an outcome framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance,) which included assessment of access to the intervention by urban/rural status; therefore, we include it as a potential moderator in this analysis. 8 We expected that individuals with personal OUD experience may be more accepting of the HCS messages due to their closeness to the topic, as personal experience with OUD is associated with lower levels of stigma. 16 Personal OUD experience was also a binary variable and assessed by participants indicating yes or no to the following prompts: I have had personal issues with opioid addiction/OUD, a relative has had personal issues with opioid addiction/OUD, or a close friend has had personal issues with opioid addiction/OUD. If respondents indicated “yes” to any of the experience-related prompts, they were coded as having personal OUD experience. This measure was adapted from Livingston et al, 2013. 17
Analysis
All outcomes were analyzed using a complete case analysis framework, and missing data was assumed to be at random. Due to the use of covariate-constrained randomization in HCS, all models were adjusted for baseline (2019) opioid overdose death rates per 100 000 adults, rural-urban status, and state (KY, MA, NY, or OH). Additionally, models included adjustments for sex and CEQ surveys (CEQ A, B, or C). Linear mixed models with community as a random effect were employed. We tested for moderation effects of the CEQ by including interaction terms in the primary analyses; however, none were significant at the 0.05 level, and the interaction terms were subsequently removed.
To study the different effects across strata of state, urban/rural classification, and personal OUD experience, subgroup analyses were performed by including an interaction term between the main exposure and the subgroup variable. To account for multiple comparisons arising from subgroup analyses within this manuscript, all P-values for the subgroup analyses were grouped and the Benjamini-Hochberg (1995) procedure was applied to obtain false discovery rate (FDR) adjusted P-values. 18 Ninety-five percent confidence intervals for estimated response differences were not adjusted for multiple comparison. Results are considered statistically significant if the 2-sided P-values are <0.05. All analyses were conducted using SAS version 9.4.
Results
A total of 2368 individuals participated in the CEQs, with CEQ A n = 817, CEQ B n = 794, and CEQ C n = 757 (Table 1). Given that population sizes varied across states, researchers anticipated that the sample sizes would correspondingly reflect these differences in state representation. Ohio had the largest number of respondents at n = 838, and Massachusetts the smallest at n = 340. Most respondents lived in an urban county (70.5%), were between ages 18 and 64 years (80.4%), and female (75.8%). About 80% of respondents were non-Hispanic white and 75% of the participants reported having at least some college education. Over half (63.3%) of participants reported personal experience with opioid use disorder, whether it be themselves, a relative, or a close friend.
Table 1.
Participant Characteristics Stratified by Iteration of Campaign Evaluation Questionnaire (CEQ)
| Characteristic, statistic | Iteration | Overall | ||
|---|---|---|---|---|
| CEQ A | CEQ B | CEQ C | ||
| Number of respondents a | 817 | 794 | 757 | 2368 |
| Number of communities represented | 33 | 33 | 33 | 33 |
| Research site, n(%) | ||||
| Kentucky | 188 (23.0%) | 193 (24.3%) | 190 (25.1%) | 571 (24.1%) |
| Massachusetts | 104 (12.7%) | 106 (13.4%) | 130 (17.2%) | 340 (14.4%) |
| New York | 237 (29.0%) | 202 (25.4%) | 180 (23.8%) | 619 (26.1%) |
| Ohio | 288 (35.3%) | 293 (36.9%) | 257 (33.9%) | 838 (35.4%) |
| Geographic location, n(%) | ||||
| Rural | 229 (28.0%) | 237 (29.8%) | 232 (30.6%) | 698 (29.5%) |
| Urban | 588 (72.0%) | 557 (70.2%) | 525 (69.4%) | 1670 (70.5%) |
| Age, n(%) | ||||
| 18-34 years | 195 (23.9%) | 152 (19.1%) | 131 (17.3%) | 478 (20.2%) |
| 35-49 years | 227 (27.8%) | 192 (24.2%) | 205 (27.1%) | 624 (26.4%) |
| 50-64 years | 255 (31.2%) | 276 (34.8%) | 269 (35.5%) | 800 (33.8%) |
| 65-74 years | 120 (14.7%) | 122 (15.4%) | 121 (16.0%) | 363 (15.3%) |
| 75+ years | 13 (1.6%) | 44 (5.5%) | 23 (3.0%) | 80 (3.4%) |
| Missing | 7 (0.9%) | 8 (1.0%) | 8 (1.1%) | 23 (1.0%) |
| Race/Ethnicity, n(%) | ||||
| Non-Hispanic White | 668 (81.8%) | 640 (80.6%) | 617 (81.5%) | 1925 (81.3%) |
| Non-Hispanic Black | 54 (6.6%) | 54 (6.8%) | 54 (7.1%) | 162 (6.8%) |
| Non-Hispanic other | 28 (3.4%) | 32 (4.0%) | 36 (4.8%) | 96 (4.1%) |
| Hispanic | 33 (4.0%) | 38 (4.8%) | 32 (4.2%) | 103 (4.3%) |
| Missing | 34 (4.2%) | 30 (3.8%) | 18 (2.4%) | 82 (3.5%) |
| Gender, n(%) b | ||||
| Male | 154 (18.8%) | 207 (26.1%) | 153 (20.2%) | 514 (21.7%) |
| Female | 641 (78.5%) | 572 (72.0%) | 582 (76.9%) | 1795 (75.8%) |
| Missing | 22 (2.7%) | 15 (1.9%) | 22 (2.9%) | 59 (2.5%) |
| Education, n(%) | ||||
| <High school | 25 (3.1%) | 45 (5.7%) | 45 (5.9%) | 115 (4.9%) |
| High school diploma | 132 (16.2%) | 157 (19.8%) | 165 (21.8%) | 454 (19.2%) |
| Some College/Associate’s | 335 (41.0%) | 318 (40.1%) | 316 (41.7%) | 969 (40.9%) |
| College degree | 186 (22.8%) | 131 (16.5%) | 134 (17.7%) | 451 (19.0%) |
| Graduate/Professional degree | 129 (15.8%) | 135 (17.0%) | 89 (11.8%) | 353 (14.9%) |
| Missing | 10 (1.2%) | 8 (1.0%) | 8 (1.1%) | 26 (1.1%) |
| Personal experience with opioid addiction/Opioid use disorder | ||||
| Yes | 502 (61.4%) | 486 (61.2%) | 511 (67.5%) | 1499 (63.3%) |
| No | 306 (37.5%) | 288 (36.3%) | 228 (30.1%) | 822 (34.7%) |
| Missing | 9 (1.1%) | 20 (2.5%) | 18 (2.4%) | 47 (2.0%) |
| Baseline opioid overdose death rate c | ||||
| Mean (SD) | 38.6 (20.3) | 38.4 (18.6) | 40.4 (20.5) | 39.1 (19.8) |
| Median (Q1, Q3) | 35.7 (21.8, 49.3) | 35.7 (21.8, 49.3) | 37.3 (27.4, 51.3) | 36.8 (21.8, 49.3) |
aRespondent defined as a participant who has consented to the CEQ iteration and is in a Wave 1 community.
bSex is reported in this table, but gender was recorded in the questionnaire. “Other gender” (1.4% overall) was combined with missing responses.
cCommunity-level rate per 100 000 residents ages 18+.
Mean naloxone message acceptability and influence scores were mostly between 3 and 4, indicating that respondents generally selected either “neither agreed nor disagreed” or “agreed” for the survey items. Message acceptability scores were higher than message influence scores for all HCS campaigns and for statewide naloxone campaigns.
In adjusted models for HCS naloxone messages compared to state naloxone messages, overall message acceptability scores were slightly but statistically significantly better for HCS naloxone campaigns (3.98, 95% CI = 3.94-4.02, Table 2) compared to the state naloxone campaigns (3.93, 95% CI = 3.89-3.97, P = 0.002). Naloxone message influence scores were also significantly better for the HCS campaign (3.44, 95% CI = 3.38-3.50) compared to the state naloxone campaigns (3.40, 95% CI = 3.34-3.46, P = 0.034).
Table 2.
Mean Acceptability and Influence Ratings for HCS Naloxone Campaign Messaging and Statewide Campaign Messaging a
| HCS campaign (95% CI) b | State campaign (95% CI) b | Response difference (95% CI) c | P-value d | |
|---|---|---|---|---|
| Naloxone message acceptability | 3.98 (3.94, 4.02) | 3.93 (3.89, 3.97) | 0.05 (0.02, 0.09) | 0.002 |
| Naloxone message influence | 3.44 (3.38, 3.50) | 3.40 (3.34, 3.46) | 0.04 (0.00, 0.07) | 0.034 |
*Rating scales were 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree.
aModels using linear mixed models and adjusted for survey iteration (CEQ), state, urban/rural status, sex, and baseline opioid overdose death rate.
bModel estimated marginal means.
cResponse difference calculated as HCS campaign - State campaign.
dP-value reported for interaction between different survey iterations and type of campaign.
In adjusted models comparing HCS messages to each other, the HCS naloxone messages had the highest estimated mean acceptability score (3.96, 95% CI = 3.92-4.00, Table 3) compared to MOUD stigma (3.73, 95% CI = 3.70-3.77) and MOUD awareness acceptability scores (3.71, 95% CI = 3.68-3.75, P < 0.001). Similarly, message influence score for the HCS naloxone campaign was the highest (3.47, 95% CI = 3.43-3.51), followed by MOUD stigma (3.42, 95% CI = 3.38-3.46) and MOUD awareness (3.39, 95% CI = 3.35-3.43, P < 0.001).
Table 3.
Mean Acceptability and Influence Ratings for HCS Campaign Messaging (Naloxone, MOUD, and Stigma) a
| Naloxone (95% CI) b | MOUD (95% CI) b | Stigma (95% CI) b | P-value c | |
|---|---|---|---|---|
| Acceptability composite | 3.96 (3.92, 4.00) | 3.71 (3.68, 3.75) | 3.73 (3.70, 3.77) | <0.001 |
| Influence composite | 3.47 (3.43, 3.51) | 3.39 (3.35, 3.43) | 3.42 (3.38, 3.46) | <0.001 |
*Rating scales were 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree.
aModels using linear mixed models and adjusted for survey iteration (CEQ), campaign topic, state, urban/rural status, sex, and baseline opioid overdose death rate.
bModel estimated marginal means.
cP-value reported for interaction between different survey iterations and type of campaign.
In subgroup analyses, the difference in naloxone message acceptability between HCS naloxone campaigns and state naloxone campaigns differed significantly by state (P < 0.001, Table 4). The same was true for naloxone message influence (P < 0.001). HCS naloxone campaigns were considered more acceptable than state naloxone campaigns in all states except for New York, though the difference between the 2 was not significant in Massachusetts. For message influence, HCS naloxone campaigns had significantly higher scores than state naloxone campaigns in Kentucky and New York. The scores for HCS campaigns were lower in Ohio and Massachusetts, though the differences were not significant.
Table 4.
Mean Acceptability and Influence Ratings for HCS Naloxone Campaign Messaging and Statewide Campaign Messaging by State a
| Group | Level | HCS campaign (95% CI) b | State campaign (95% CI) b | Response difference (95% CI) c | P-value d | |
|---|---|---|---|---|---|---|
| Naloxone message acceptability | State | Kentucky | 4.09 (4.03, 4.16) | 3.71 (3.65, 3.78) | 0.38 (0.31, 0.44) | <0.001 |
| Massachusetts | 4.07 (3.98, 4.15) | 4.00 (3.92, 4.09) | 0.06 (−0.02, 0.15) | |||
| New York | 3.87 (3.80, 3.94) | 4.15 (4.08, 4.22) | −0.28 (−0.34, −0.21) | |||
| Ohio | 3.89 (3.83, 3.95) | 3.83 (3.77, 3.88) | 0.06 (0.01, 0.12) | |||
| Naloxone message influence | State | Kentucky | 3.52 (3.42, 3.61) | 3.41 (3.32, 3.51) | 0.10 (0.03, 0.17) | <0.001 |
| Massachusetts | 3.51 (3.38, 3.64) | 3.54 (3.41, 3.68) | −0.04 (−0.13, 0.06) | |||
| New York | 3.40 (3.29, 3.51) | 3.26 (3.16, 3.37) | 0.14 (0.07, 0.21) | |||
| Ohio | 3.32 (3.24, 3.41) | 3.37 (3.28, 3.46) | −0.05 (−0.11, 0.01) |
*Rating scales were 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree.
aModels using linear mixed models and adjusted for survey iteration (CEQ), state, urban/rural status, sex, baseline opioid overdose death rate, and interaction between state and different campaigns.
bModel estimated marginal means.
cResponse difference calculated as HCS campaign - State campaign, 95% CI are not adjusted for multiple comparison.
dP-value reported for interaction between state and type of campaign are adjust for multiple comparison using Benjamini-Hochberg.
Comparing the 3 HCS campaigns, message acceptability was significantly different by urban/rural classification (P = 0.002, Table 5). Acceptability scores were slightly higher for all 3 campaigns in rural communities. The level of HCS message acceptability did not differ significantly across campaigns as a function of personal OUD experience (P = 0.281). Message influence was not significantly different between HCS campaigns by either urban/rural classification (P = 0.430) or personal OUD experience (P = 0.310).
Table 5.
Mean Acceptability and Influence Ratings for HCS Campaign Messaging (Naloxone, MOUD, and Stigma) by Urban/Rural Classification and Personal OUD Experience a
| Group | Level | Naloxone (95%CI) b | MOUD (95%CI) b | Stigma (95%CI) b | P-value c | |
|---|---|---|---|---|---|---|
| Acceptability composite | Urban/Rural classification a | Urban | 3.94 (3.90, 3.98) | 3.66 (3.62, 3.70) | 3.71 (3.67, 3.75) | 0.002 |
| Rural | 3.97 (3.91, 4.03) | 3.80 (3.74, 3.86) | 3.74 (3.68, 3.80) | |||
| Personal OUD experience d | Yes | 4.03 (3.99, 4.08) | 3.77 (3.73, 3.82) | 3.78 (3.74, 3.82) | 0.281 | |
| No | 3.84 (3.78, 3.89) | 3.61 (3.55, 3.66) | 3.64 (3.59, 3.70) | |||
| Influence composite | Urban/Rural classification a | Urban | 3.43 (3.38, 3.47) | 3.34 (3.30, 3.39) | 3.38 (3.34, 3.43) | 0.430 |
| Rural | 3.52 (3.45, 3.58) | 3.44 (3.37, 3.50) | 3.44 (3.38, 3.51) | |||
| Personal OUD experience d | Yes | 3.55 (3.50, 3.60) | 3.46 (3.41, 3.51) | 3.48 (3.43, 3.53) | 0.310 | |
| No | 3.33 (3.27, 3.39) | 3.26 (3.20, 3.32) | 3.31 (3.25, 3.37) |
*Rating scales were 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree.
aModels using linear mixed models and adjusted for survey iteration (CEQ), campaign topic, state, urban/rural status, sex, baseline opioid overdose death rate, and interaction between state and different campaigns.
bModel estimated marginal means.
cP-value reported for interaction between type of campaign and groups are adjust for multiple comparison using Benjamini-Hochberg.
dModels using linear mixed models and adjusted for survey iteration (CEQ), campaign topic, state, urban/rural status, sex, baseline opioid overdose death rate, personal OUD experience, and interaction between state and different campaigns.
Discussion
Overall, HCS naloxone messages had higher acceptability and influence scores than statewide naloxone messages. This helps answer a key question concerning the lack of evidence for population impact of the HCS messages previously published. 9 Explanations offered for the lack of effect focused on limited evidence that respondents had actually seen the messages (as well as some other possible methodological issues). However, 1 possibility we could not at that time exclude was that the HCS messages were somehow substandard relative to messages created by agencies under contract to the states. Clearly, the HCS messages appear to have been slightly better received than the state messages, based on the present analysis, eliminating that explanation. This analysis provides supporting evidence that the campaign did not have measurable impact due to inadequate exposure, which is consistent with Hornik’s (2002) argument that difficulty in obtaining significant levels of exposure is a primary reason for failure of population-level effects in communication efforts. 19 Lewis et al (2024) found some evidence that HCS campaigns reduced individual stigma and increased respondent willingness to carry naloxone, but this finding was reported with caution due to inadequate levels of campaign exposure. Had the HCS messages reached greater levels of exposure, it is more likely that an impact would have been measured, given the positive reception of the message content described here.
Possible reasons for the HCS naloxone messages doing slightly but significantly better than the statewide messaging can only be addressed speculatively. Local coalitions could tailor key messaging and materials to reflect unique community characteristics and needs. This may have resulted in the more favorable assessments of these messages. In the actual campaigns, outside of images presented to CEQ respondents, communities selected even more specific versions of the message (eg, the race/ethnicity of models, some text differences) that they thought were best suited to their community. If community coalitions had a reasonably accurate sense of their community preferences, it is likely that the reception of the messages reported here at least slightly underestimate the reception of the actual HCS messages that any given community member would have seen.
In the comparison between HCS naloxone, MOUD stigma, and MOUD awareness campaigns, the HCS naloxone campaign had both the highest acceptability and influence scores. It should be noted that the general “look and feel” of the messages were similar for all topics, and that the main difference accounting for differences in ratings were the topics themselves. This suggests continuing challenges in communicating as effectively about MOUD treatment and stigma as about the importance of naloxone. The naloxone messages averaged ratings of just below 4, with 4 representing agreement regarding acceptability and influence. The MOUD treatment and stigma messages averaged below 3.5, closer to the neutral point of the scale than to agreement, though still on average above the neutral point.
The difference in both message acceptability and influence between HCS naloxone and state campaigns varied by state, as one might expect given that the state campaigns were created independently and likely varied in their reception. Of particular interest is the lack of evidence for difference in receptivity between persons who had close contact with someone with OUD and persons without. We were concerned that persons without such contact might be less attentive to or persuaded by the personal images and portrayals in the messages, and this was not the case in our data.
Across all measures and strata, message acceptability scores were higher than message influence scores. This suggests that messages were more likely to capture attention, be understood, and have a picture and text that match than to influence desire to change behavior. The message acceptability scores in this analysis are similar to those of a separate campaign about cigarillos and waterpipe tobacco that used the same measures, with respondents rating the message attention capturing capability and ease of understanding generally between 3 and 4. 12 Behavior changes through health communications, however, are more complex than simple message acceptability, and highly dependent on the topic. 5 It is somewhat expected that respondents would rate the message influence lower than acceptability, since the influence items assessed here would require the message to influence a belief or behavior.
Study limitations include sampling and message selection issues. The Facebook sample was a convenience sample, with persons with some interest regarding the OUD issue being arguably more likely to self-select in. That might have led to higher evaluative scores on average than would be found using a random population sample. Both methods may also be problematic relative to a particular concern reaching people with OUD or their families, which this sample did not attempt to address. However, the lack of evidence regarding different responses from people with or without close contact with persons with OUD suggests that might not be a major issue. Additionally, the majority of respondents were female, white, and had some college education, which is not representative of the demographic composition of participating communities. Respondents were also only presented with single images from both the HCS and statewide campaigns. Other images from these campaigns might have had other effects. However, we did have different images in each state, providing more variability and increasing confidence in the robustness of results. Another technical strength of this study is that each survey respondent was provided images of both the HCS and statewide messages to assess. This within-person design inherently controls for between-person differences and is supported by a robust sample size for an approach.
This analysis provides useful insight into acceptability and influence of opioid-related health communications campaign messaging. More specifically, it further contextualizes previously published findings about the HCS communications campaigns. 9 Given the more positive assessment of HCS messages by community members compared to statewide campaigns, future opioid-related health communications messaging should consider utilizing similar creatives to those disseminated by HCS communities. The lower acceptability and influence of MOUD-related messaging may indicate a lack of widespread awareness and/or acceptance of MOUD as an intervention for reducing opioid-related deaths and treating OUD. Opioid-related health campaigns should promote the benefits of MOUD and not just naloxone. With mean message acceptability scores nearing and exceeding 4 in some cases, this analysis supports the utility of health-related messaging for capturing attention and promoting understanding of an issue. The higher rating of message acceptability vs message influence suggests that additional campaign elements may be required to shift attitudes, beliefs, and action beyond simply message comprehension. Future campaigns should conduct formative research with randomized, representative evaluations of specific elements of campaign messages to determine which parts are most acceptable and influential, eg, the images used, calls to action, colors, etc. This insight can be used to craft targeted campaigns that drive shifts in attitudes and behaviors to build communities supportive of individuals with substance use disorder and their recovery. Though HCS campaigns had limited recognition in participating communities, modest effect sizes for large-scale mass media campaigns still have the potential to catalyze behavior change.9,20 Significant investments in future health communication campaigns to reach high levels of recognition among target populations are necessary to achieve the desired impact of messaging.
So what?
- • What is already known on this topic?
- ○ Non-stigmatizing, personalized narratives about opioid use disorder are effective means of messaging. Previous research on HEALing Communities Study (HCS) campaigns did not show impact in a treatment/control design, and presented evidence that limited message recognition (resulting from insufficient message exposure) may have been responsible.
- • What does this article add?
- ○ This article provides a quantitative evaluation of the HCS, allowing us to assess the alternative explanation for lack of population effect (ie, messages not being well-received in the study population). Messages were generally well-received, and with greater levels of exposure, likely would have had a measurable impact.
- • What are the implications for health promotion practice or research?
- ○ This analysis supports the potential utility of health communications campaigns for capturing attention and promoting understanding of an issue. However, without sufficient message exposure, even well-designed and well-received messages will have little or no impact.
Supplemental Material
Supplemental Material for A Cross-Sectional Study of Acceptability and Influence of HEALing Communities Study Communications Campaign Messaging Among Community Members in Four U.S. States by Jamie Luster, Jennifer Reynolds, Rouba Chahine, Nicky Lewis, Michael D. Stein, R. Craig Lefebvre, Lauren D’Costa, Kat Asman, Kara Stephens, Nishita Dsouza, Michael D. Slater in American Journal of Health Promotion
Acknowledgements
We wish to acknowledge the participation of the HEALing Communities Study communities, community coalitions, community partner organizations and agencies, and Community Advisory Boards and state government officials who partnered with us on this study. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the Substance Abuse and Mental Health Services Administration or the NIH HEAL Initiative®.
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Institutes of Health and the Substance Abuse and Mental Health Services Administration through the NIH HEAL (Helping to End Addiction Long-term®) Initiative under award numbers UM1DA049394, UM1DA049406, UM1DA049412, UM1DA049415, UM1DA049417 (ClinicalTrials.gov Identifier: NCT04111939).
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material: Supplemental material for this article is available online.
ORCID iDs
Jamie Luster https://orcid.org/0000-0002-8484-1215
Rouba Chahine https://orcid.org/0000-0002-6061-064X
Ethical Approval
This study protocol (Pro00038088) was approved by Advarra Inc., the HEALing Communities Study single Institutional Review Board.
Consent to Participate
Online consent was obtained for all participants involved in the Campaign Evaluation Questionnaire (CEQ). Each individual had the opportunity to review the information related to the study, understand the purpose and procedures, and provide their agreement to participate voluntarily through a secure online platform.
References
- 1.National Institute of Drug Abuse . Drug Overdose Deaths: Facts and Figures. Bethesda, MD: National Institutes of Health. https://nida.nih.gov/research-topics/trends-statistics/overdose-death-rates (2022). [Google Scholar]
- 2.Ahmad F, Cisewski J, Rossen L, Sutton P. Provisional Drug Overdose Death Counts. Atlanta, GA: U.S. Centers for Disease Control and Prevention; 2023. [Google Scholar]
- 3.Frkovich J, Hedrick H, Anakaraonye AR, Bornkessel A, Lefebvre RC. Opioid-related public health communication campaigns: an environmental scan. Am J Health Promot. 2022;36(6):913-919. doi: 10.1177/08901171221082471 [DOI] [PubMed] [Google Scholar]
- 4.Furlan AD, Carnide N, Irvin E, et al. A systematic review of strategies to improve appropriate use of opioids and to reduce opioid use disorder and deaths from prescription opioids. Can J Pain. 2018;2(1):218-235. doi: 10.1080/24740527.2018.1479842 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Snyder LB. Health communication campaigns and their impact on behavior. J Nutr Educ Behav. 2007;39(2 Suppl):32-S40. doi: 10.1016/j.jneb.2006.09.004 [DOI] [PubMed] [Google Scholar]
- 6.Fernandez P, Azucar D, Zambole K. A dose of truth: a qualitative assessment of reactions to messages about fentanyl for people who use drugs. Subst Use Misuse. 2023;58(4):520-527. doi: 10.1080/10826084.2023.2177112 [DOI] [PubMed] [Google Scholar]
- 7.Waye KM, Yedinak JL, Koziol J, Marshall BDL. Action-focused, plain language communication for overdose prevention: a qualitative analysis of Rhode Island’s overdose surveillance and information dashboard. Int J Drug Pol. 2018;62:86-93. doi: 10.1016/j.drugpo.2018.08.010 [DOI] [PubMed] [Google Scholar]
- 8.HEALing Communities Study Consortium . The HEALing (helping to end addiction long-term (SM)) communities study: protocol for a cluster randomized trial at the community level to reduce opioid overdose deaths through implementation of an integrated set of evidence-based practices. Drug Alcohol Depend. 2020;217:108335. doi: 10.1016/j.drugalcdep.2020.108335 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lewis N, Eggleston B, Chandler RK, et al. Stigma and efficacy beliefs regarding opioid use disorder treatment and naloxone in communities participating in the HEALing communities study intervention. PLoS One. 2024;19(12):e0308965. doi: 10.1371/journal.pone.0308965 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lefebvre RC, Chandler RK, Helme DW, et al. Health communication campaigns to drive demand for evidence-based practices and reduce stigma in the HEALing communities study. Drug Alcohol Depend. 2020;217:108338. doi: 10.1016/j.drugalcdep.2020.108338 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kennedy-Hendricks A, McGinty EE, Summers A, Krenn S, Fingerhood MI, Barry CL. Effect of exposure to visual campaigns and narrative vignettes on addiction stigma among health care professionals: a randomized clinical trial. JAMA Netw Open. 2022;5(2):e2146971. doi: 10.1001/jamanetworkopen.2021.46971 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Sutfin EL, Cornacchione Ross J, Lazard AJ, et al. Developing a point-of-sale health communication campaign for cigarillos and waterpipe tobacco. Health Commun. 2019;34(3):343-351. doi: 10.1080/10410236.2017.1407277 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Ingram DD, Franco SJ. NCHS urban-rural classification scheme for counties. Vital Health Stat 2. 2013;2014(166):1-73. https://www.ncbi.nlm.nih.gov/pubmed/24776070 [PubMed] [Google Scholar]
- 14.MacDougall C. Rural Massachusetts: A statistical overview. [Report]. Massachusetts: State Government; 1995. [Google Scholar]
- 15.Sanders A, Cromartie J. Rural Classifications - What is Rural? Washington, DC: Economic Research Service U.S. Department of Agriculture Web site; 2025. Updated 2025. https://www.ers.usda.gov/topics/rural-economy-population/rural-classifications/what-is-rural [Google Scholar]
- 16.Taylor BG, Lamuda PA, Flanagan E, Watts E, Pollack H, Schneider J. Social stigma toward persons with opioid use disorder: results from a nationally representative survey of U.S. adults. Subst Use Misuse. 2021;56(12):1752-1764. doi: 10.1080/10826084.2021.1949611 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Livingston JD, Tugwell A, Korf-Uzan K, Cianfrone M, Coniglio C. Evaluation of a campaign to improve awareness and attitudes of young people towards mental health issues. Soc Psychiatr Psychiatr Epidemiol. 2013;48(6):965-973. doi: 10.1007/s00127-012-0617-3 [DOI] [PubMed] [Google Scholar]
- 18.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J Roy Stat Soc B. 1995;57(1):289-300. https://www.jstor.org/stable/2346101 [Google Scholar]
- 19.Hornik R, (Ed.). Public Health Communication: Evidence for Behavior Change. 1st ed. New York: Routledge. 10.4324/9781410603029 (2002). [DOI] [Google Scholar]
- 20.Zhao X. Health communication campaigns: a brief introduction and call for dialogue. Int J Nurs Sci. 2020;7(Suppl 1):S11-S15. doi: 10.1016/j.ijnss.2020.04.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Material for A Cross-Sectional Study of Acceptability and Influence of HEALing Communities Study Communications Campaign Messaging Among Community Members in Four U.S. States by Jamie Luster, Jennifer Reynolds, Rouba Chahine, Nicky Lewis, Michael D. Stein, R. Craig Lefebvre, Lauren D’Costa, Kat Asman, Kara Stephens, Nishita Dsouza, Michael D. Slater in American Journal of Health Promotion