Testing warning messages on smokers’ cigarette packages: A standardized protocol

Abstract

Purpose

Lab experiments on cigarette warnings typically use a brief one-time exposure that is not paired with the cigarette packs smokers use every day, leaving open the question of how repeated warning exposure over several weeks may affect smokers. This proof of principle study sought to develop a new protocol for testing cigarette warnings that better reflects real-world exposure by presenting them on cigarette smokers’ own packs.

Methods

We tested a cigarette pack labeling protocol with 76 US smokers ages 18 and older. We applied graphic warnings to the front and back of smokers’ cigarette packs.

Results

Most smokers reported that at least 75% of the packs of cigarettes they smoked during the study had our warnings. Nearly all said they would participate in the study again. Using cigarette packs with the study warnings increased quit intentions (p<.05).

Conclusion

Our findings suggest a feasible pack labeling protocol with six steps: (1) schedule appointments at brief intervals; (2) determine typical cigarette consumption; (3) ask smokers to bring a supply of cigarette packs to study appointments; (4) apply labels to smokers’ cigarette packs; (5) provide participation incentives at the end of appointments; and (6) refer smokers to cessation services at end of the study. When used in randomized controlled trials in settings with real-world message exposure over time, this protocol may help identify the true impact of warnings and thus better inform tobacco product labeling policy.

INTRODUCTION

For much of the world, tobacco product packaging is a key part of marketing efforts to make tobacco use appealing.[1–3] In contrast, warnings on tobacco product packages accurately convey the many health risks of tobacco to discourage smoking initiation and increase cessation.[4] Information on the effectiveness of these warnings can guide policymakers as they consider new warnings and develop regulations. In the more than 100 countries that already have cigarette pack labeling policies in place,[5] testing the impact of potential cigarette pack warnings can guide selection of warnings with the greatest impact.

Research on cigarette pack warnings currently focuses on population-based observational studies of smoking behavior and on laboratory experiments. Longitudinal observational studies have found increased cessation behavior after countries introduced new health warnings on cigarette packs.[6–9] These observational studies have high external validity, but they do not rule out alternative explanations for the association of the changes in labeling and behavior change. Experiments on exposure to cigarette pack warnings in laboratory settings can offer much stronger evidence that warnings cause observed changes in beliefs and behavior.[10, 11] While a large experimental literature has developed, these experiments use very brief exposure to warnings; assess non-behavioral short-term outcomes, such as attitudes or quit intentions, instead of smoking behavior; and, yield results that need corroboration outside of a controlled research setting.

It will be helpful to develop a way to deliver new messages on cigarette packs that smokers use daily in order to expand the external validity of experimental research on cigarette warnings. This will ensure a meaningful warning “dose” that replicates the frequency and duration of warning exposure in the real world. For example, a pack-a-day smoker sees a cigarette pack at least an estimated 7,300 times per year (>20 views/day × 365 days/year).

Several studies have tested warnings by providing mocked-up cigarette packs to participants, but most of these studies used a one-time, brief exposure that does not replicate message dose in the real world.[10, 12–18] At least five studies have used mocked-up packs for a longer period of time.[19, 20] A 2001 study conducted in France, Switzerland, and Belgium mailed 485 adult smokers cardboard boxes with one of four antismoking messages selected by the smoker at recruitment. Study materials instructed participants to place their cigarette packs in the boxes for four weeks, but few did so.[20] A 2010 study in Scotland provided 140 adult smokers with 14 plain cigarette packs that had warnings but no brand information or logos, asking participants to transfer their own cigarettes into the supplied packs for two weeks. Only 34% of participants completed the study as intended.[19] A recent US study provided smokers with free cigarettes and labeled the packs with health warnings for four weeks. During the study, participants collected their cigarette butts and brought them to study appointments. This study used an innovative method for labeling cigarette packs, but the results may have been influenced by receipt of free cigarettes and saving the cigarette butts.[21] Two additional recent studies in the US had smokers apply warning labels to their own cigarette packs, which may be an intervention in its own right, and both studies had short follow-up periods (three to seven days).[22, 23] While these studies’ methods are promising because they more closely replicate real-world message dose, we need strategies with high protocol adherence and longer follow-up periods. Tobacco control research would benefit from a way to test health warnings on cigarette packs that allows for random assignment to warning conditions, adds realism, and maintains high protocol adherence.

To that end, we tested a new cigarette pack labeling and carrying protocol with adult smokers in two proof-of-principle pilot studies. We aimed to assess whether the pack labeling and carrying protocol (1) is feasible and has high adherence for assessing smokers’ reactions to cigarette pack warnings in a real-world setting and (2) is sensitive to changes in psychosocial outcomes.

MATERIALS AND METHODS

Participants

Pilot Study 1

From February to March 2013, we recruited 30 smokers ages 18 or older who we observed smoking cigarettes in public places in North Carolina, USA, or who received referrals from study participants. We defined current smoking as having smoked at least 100 cigarettes during one’s lifetime and now smoking every day or some days, and we excluded pregnant women, people who smoke only roll-your-own cigarettes, and cigarillo-only smokers.

Pilot Study 2

In July and August 2014, we recruited an additional 48 smokers using the same eligibility criteria as Pilot Study 1. We used a variety of recruitment methods, including newspaper ads, flyers, and email lists.

Procedures

Pilot Study 1

A research assistant explained the study to participants and obtained written informed consent. We provided participants with an $80 financial study incentive in cash and accompanied them to a nearby store, where we asked them to purchase the amount of cigarette packs they would normally smoke over two weeks as determined in the initial screening. We randomly assigned participants to receive one of nine graphic warnings (Figure 1). In 2011, the US Food and Drug Administration (FDA) proposed these nine warnings for implementation, but the warnings were later challenged in court and are not currently in use.[24] At the time of the study, US cigarette packs also included one of four text-only warnings on one side of the pack; these warnings have been in circulation since 1985. Then, participants completed a baseline survey on a tablet computer while staff removed the package cellophane and applied the same graphic warning labels to the top half of the front and rear panels of participants’ cigarette packs, in accordance with the proposed FDA requirements (Figure 1). As applying the labels on the top half of the front panel sealed the flip-top box shut, staff cut through the label to allow the box to again open freely. Participants returned to the study offices to complete a follow-up computer survey two weeks later and received an additional $50 cash incentive. After completing the follow-up visit survey, each participant received information about a local smoking cessation program.

Figure 1.

Example of graphic warning label on pack

Pilot Study 2

Procedures for Pilot Study 2 were similar to Pilot Study 1. After obtaining written informed consent, we randomly assigned participants to receive one of five graphic warnings, four of which were also used in Pilot Study 1. Participants visited our study offices at baseline and then weekly for four weeks, completing a survey on the computer at each visit. Smokers brought eight days’ worth of cigarettes to the first four appointments. While participants were taking the survey, we labeled their packs using the same procedures as in Pilot Study 1. Participants received a cash incentive at the end of each visit in an envelope marked “payment for survey completion.” Cash incentives totaled $185. At the final appointment, each participant received information about a local smoking cessation program. Two of the 48 participants withdrew from the study. The University of North Carolina Institutional Review Board approved the procedures for both pilot studies.

Measures

Prior to the studies, we refined the survey instruments by conducting cognitive interviews with 17 adult smokers.[25] Baseline and follow-up surveys applied measures adapted from prior studies, including knowledge of smoking health risks (3 items),[26] perceived likelihood of harm from smoking (2 items),[27] worry about the harms of smoking (1 item),[28] subjective norms (3 items),[29] positive smoker prototypes (7 items),[30–32] negative smoker prototypes (5 items),[30–32] self-efficacy to quit smoking (3 items),[29] and quit intentions (3 items).[33] Cronbach’s alpha for multi-item scales was 0.70 or higher in both studies, except for self-efficacy in Pilot Study 1 (α=0.69). We also measured whether participants attempted to quit smoking (defined as not smoking for at least 24 hours because they were trying to quit smoking) or successfully stopped smoking during the study (defined as not smoking for at least seven days).

The follow-up surveys assessed noticing the warning, whether people talked to others about the warning, avoidance of the warning, emotional reactions to the warning,[34] participant satisfaction with study procedures, the number of cigarettes smoked from labeled and unlabeled packs, and unprompted recall of the warning image. Two raters independently coded whether each participant correctly recalled the image in their assigned warning, with perfect agreement.

Finally, in Pilot Study 1, several open-ended questions assessed attitudes toward the study protocol. Two raters independently coded whether open-ended responses indicated that smokers thought the study was giving them free cigarettes. A coauthor resolved all coding discrepancies. The survey instruments are available online: www.unc.edu/~ntbrewer.

Data analysis

We calculated descriptive statistics to summarize demographic information and process measures. We assessed changes between baseline and follow-up using paired t-tests or McNemar chi-squared tests. For continuous variables with non-normal distribution (by Q-Q plots and Kolmogorov-Smirnov tests of normality), we used Wilcoxon signed-rank tests, a non-parametric equivalent of the paired t-test. Analyses used SPSS Statistics version 20 and Stata/IC version 13.1. The two Pilot Study 2 participants who withdrew from the study were excluded from analyses. We set test critical alpha value to 0.05 and used two-tailed tests.

RESULTS

Smokers’ mean age was 30 years in Pilot Study 1 and 43 years in Pilot Study 2 (Table 1). Fewer than half of participants were white and about one-third were African American. Most smokers (69% in Pilot Study 1 and 59% in Pilot Study 2) were low-income, defined as being at or below 200% of the US federal poverty level. At baseline, participants in Pilot Study 1 smoked an average of 12 cigarettes per day (SD=9) and participants in Pilot Study 2 smoked an average of 11 cigarettes per day (SD=8).

Table 1.

Participant demographics

	Pilot Study 1 (n=30) %	Pilot Study 2 (n=46) %
Age
18–24 years	40	4
25–39 years	33	37
40–54 years	27	33
55+ years	0	26
Mean (SD)	30 (11)	43 (12)
Sex
Female	40	57
Sexual orientation
Straight	90	89
Gay	3	9
Bisexual	7	2
Hispanic ethnicity
No	90	87
Yes	10	13
Race
Asian	7	4
Black	27	35
White	43	44
Other/Multiracial	23	17
Education
High school degree or less	33	20
Some college	50	50
College graduate	10	24
Graduate or professional degree	7	7
Low income
No	31	41
Yes	69	59
Cigarettes smoked per day
Mean (SD)	12 (9)	11 (8)

Exposure to labels

Study participants reported high rates of adherence to smoking cigarettes only from labeled packs. Around 90% of smokers reported that at least three quarters of the cigarettes smoked during the study were from labeled packs (Figure 2). Ninety percent of smokers in Pilot Study 1 and 93% of smokers in Pilot Study 2 correctly recalled the image on their assigned labels. At follow-up, most (93% in Pilot Study 1 and 89% in Pilot Study 2) smokers reported noticing the warning sometimes, often, or all of the time (Table 2). About half of smokers tried to avoid thinking about the labels sometimes, often, or all the time. Similarly, 43% of smokers in Pilot Study 1 and 46% of smokers in Pilot Study 2 tried to avoid looking at the labels sometimes, often, or all of the time.

Figure 2.

Process evaluation measures for cigarette pack labeling proof of principle studies

Table 2.

Outcomes at follow-up.

	Pilot Study 1 (n=30)	Pilot Study 2 (n=46)
	2 week follow-up	2 week follow-up	4 week follow-up
	%	%	%
Psychosocial
Noticed the warning1	93	89	78
Talked to anyone about the warnings1	97	93	96
Tried to avoid thinking about the warning1	43	50	30
Tried to avoid looking at the warning1	43	46	35
Felt (as a result of viewing the warning)2:
Depressed	10	4	4
Disgusted or grossed out	20	11	7
Guilty	10	11	9
Sad	13	7	9
Worried or anxious	20	5	2
Warning caused them to think about the harmful effects of smoking	50	41	39
Behavioral
Quit attempt, 1 or more days	10	17	20
Quit smoking	0	4	2

¹Responded “sometimes,” “often,” or “all of the time” rather than “never” or “almost never”

²Responded “very” or “completely,” rather than “not at all,” “a little,” or “somewhat”

Outcomes at follow-up

About half (50% in Pilot Study 1 and 41% in Pilot Study 2) of participants reported that the warnings caused them to think about the harmful effects of smoking somewhat, quite a bit, or very much. Almost all participants (97% in Pilot Study 1 and 93% in Pilot Study 2) reported talking to someone about the warnings during the study.

Changes in outcomes

Endorsement of positive smoker prototypes decreased between baseline and follow-up in both studies (p<.05, Table 3). Likewise, quit intentions increased in both studies (p<.05). Worry about the harms of smoking increased in Pilot Study 1 (p=.01) but not in Pilot Study 2. Knowledge and perceived likelihood of harm increased in Pilot Study 2 (p<.05), but not in Pilot Study 1. The two-week outcomes for Pilot Study 2 showed the same pattern as the four-week outcomes (data not shown), except for cigarettes smoked per day which dropped from baseline (mean=9.10, SD=6.81, p=.02).

Table 3.

Changes in outcomes

	Pilot Study 1 (n=30)			Pilot Study 2 (n=46)
	Baseline	2-week follow-up		Baseline	4-week follow-up
	mean (SD)	mean (SD)	p	mean (SD)	mean (SD)	p
Knowledge of smoking health risks	2.47 (0.82)	2.63 (0.72)	.17	2.67 (0.73)	2.93 (0.44)	<.01*
Perceived likelihood of harm	2.93 (0.12)	3.10 (1.30)	.39	3.20 (1.09)	3.59 (1.13)	.02*
Worry	2.86 (1.06)	3.38 (1.08)	.01*	3.02 (1.00)	3.09 (1.05)	.42
Subjective norms	4.07 (0.95)	4.16 (0.87)	.69	4.30 (0.83)	4.42 (0.77)	.19
Smoker prototypes – positive	2.14 (1.07)	1.88 (0.92)	.03*	1.98 (0.83)	1.47 (0.60)	<.01*
Smoker prototypes – negative	1.93 (0.86)	1.83 (0.85)	.15	1.68 (0.60)	1.73 (0.97)	.76
Self-efficacy	3.27 (0.96)	3.53 (0.81)	.10	3.46 (0.99)	3.73 (1.03)	.06
Quit intentions	2.52 (1.06)	2.89 (1.10)	.04*	2.26 (0.97)	2.93 (1.19)	<.01*
Cigarettes smoked per day	11.59 (9.31)	11.58 (9.78)	.99	11.11 (7.44)	9.98 (8.29)	.24

^*p<.05

The mean number of cigarettes smoked per day did not change between baseline and follow-up in either study. Ten percent of Pilot Study 1 participants and 20% of Pilot Study 2 participants stopped smoking for at least 24 hours during the study. No participants successfully quit smoking during Pilot Study 1. During Pilot Study 2, two participants had quit smoking by week 2 but they resumed by week 4, and one participant quit between weeks 2 and 4.

Process measures

Retention was very high in both studies (100% in Pilot Study 1 and 96% in Pilot Study 2). Nearly all participants said that they would probably or definitely participate in the study again, and 100% said they would probably or definitely recommend the study to a friend (Figure 2). Most smokers (90% in Pilot Study 1 and 78% in Pilot Study 2) found participating in this study to be easy or very easy.

About a third (11/30) of Pilot Study 1 smokers said in exit interviews that they thought the study was buying them cigarettes as our staff had prepaid them and escorted them to the store so they could buy cigarettes at the baseline interview. To address these concerns, we asked Pilot Study 2 participants to bring an eight-day supply of cigarettes with them to the study visits every week for four weeks. Most participants did not find this task to be burdensome: 15% of participants reported that it was difficult to bring in eight days’ worth of cigarettes, 30% said it was neither difficult nor easy, and 54% said it was easy or very easy.

DISCUSSION

Our proof of principle studies of placing warnings on real-world cigarette packages for an extended period of time suggests that our proposed protocol is feasible. The protocol had low attrition, high adherence, and allowed assessment of impact over time. Most smokers complied with the study protocol and reported high levels of satisfaction with the procedures. Thus, this new pack carrying protocol offers the potential for both experimental control and greater real-world dose than exposures in brief lab studies.

The original protocol (Pilot Study 1) provided a financial incentive immediately before the smoker purchased cigarettes because we were concerned that low-income smokers could not afford the up-front costs of buying two weeks’ worth of cigarettes at one time. However, we believe that our revised protocol (Pilot Study 2) may pose fewer risks to smokers. In the US, lower-income populations smoke at higher rates than their higher-income counterparts,[35] and, accordingly, research shows that most smokers do not purchase cigarettes in bulk, but rather by the pack.[36] Losing large numbers of smokers who cannot afford to purchase cigarettes in bulk would, in turn, limit the generalizability of cigarette pack labeling studies using such a protocol. However, many Pilot Study 1 participants believed that the study gave them free cigarettes, which could potentially increase smoking. Studies suggest lower prices encourage cigarette consumption.[37] Fortunately, cigarette consumption did not increase, while quit intentions did increase.

Another potential concern is whether having more cigarettes on hand might make participants smoke more, especially if they are not accustomed to purchasing packs in bulk. The empirical impact of stockpiling cigarettes on smoking behavior is unknown. Smokers can save money by purchasing cigarettes in bulk (e.g., cartons) rather than by the pack, but many smokers elect to ration their purchase quantities and impose additional transaction costs to limit their smoking.[38, 39] We suggest keeping the length of time between study appointments at one week, which will limit the number of cigarette packs that participants will have on-hand at one time, thus reducing the potential risk that participants will smoke more. Future research should try to assess unintended consequences of stockpiling packs.

Having participants bring their own cigarettes to study appointments decouples cigarette purchasing from receiving payment. Moreover, it appears that low-income smokers in our study were not deterred by having to purchase cigarettes in advance as part of the protocol as more than half of Pilot Study 2 participants were at or below 200% of the US federal poverty level. It remains possible that other low income smokers were unable to participate in the study.

Our findings point to a new pack labeling protocol with six steps: (1) schedule appointments one week apart; (2) determine daily cigarette consumption through smokers’ self-report; (3) ask smokers to purchase and bring to the study appointments the amount of cigarette packs they would normally smoke between appointments, with an extra supply to buffer against cancelations and allow for greater than expected use; (4) during study appointments, apply labels to smokers’ cigarette packs in-person by removing the top of the cellophane wrapping from the cigarette packs and applying self-adhesive labels with warnings directly to the packs, to prevent smokers from inadvertently removing the label with the wrapping; (5) provide study participation incentives at the end of the appointments and clearly communicate that the payment is for survey completion to reduce the possibility that participants will believe that the payments equate to receiving free cigarettes; and (6) refer smokers to cessation services at study completion or directly help smokers who want to quit. Table 4 gives the rationale for each protocol step. We implemented the six steps of this protocol successfully in Pilot Study 2.

Table 4.

Recommended pack labeling protocol for studies that place warnings on smokers’ cigarette packs

Protocol Step	Rationale
1. Schedule appointments weekly	A week is feasible for smokers who would purchase cigarettes for the study, and it limits the number of cigarette packs a smoker has on hand.
2. Determine cigarette consumption	Knowing how much a smoker typically smokes allows the researcher to instruct the smoker on how many packs to bring to the first and subsequent study visits.
3. Ask smokers to purchase and bring cigarettes to study appointments	Asking smokers to purchase cigarettes on their own can help prevent smokers from thinking that the study is giving smokers “free cigarettes.” Asking them to bring extra packs allows for a buffer against missed appointments or in case they smoke more than expected in a given week.
4. Apply labels to smokers’ cigarette packs.	Researchers label the packs, rather than smokers self-labeling packs, as self-labeling may serve as an active intervention. Also, researchers labeling packs is likely to lead to higher protocol compliance as compared to asking smokers to self-label their packs.
5. Provide participation incentives for survey completion	Communicating that study incentives are for survey completion may reduce the possibility that participants will perceive that payments equate to receiving free cigarettes.
6. Refer smokers to cessation services at study completion	Referral to cessation services may help smokers to quit at the end of the study, if they have not already.

Strengths of the study include establishing the feasibility of a new pack labeling protocol in a field study with current smokers, followed over time. As we studied a convenience sample recruited in a single location, the generalizability of our findings remains to be established. By labeling cigarette packs that smokers were currently using, we tested novel warnings alongside the standard US text-only warnings that we did not obscure. Depending on the brand, the warnings may have covered some of the branding or color of the cigarette packs. In these cases, we may have overestimated the impact of the warning, as obscuring the branding may make cigarettes less appealing to smokers.[40, 41] We relied on self-report to measure protocol adherence, but the extent to which self-reported adherence reflects actual adherence is unknown. We tested only methods for labeling cigarette packs, and we are unable to determine the utility of this protocol for other tobacco products. Finally, the small sample size limited statistical power to detect changes in psychosocial and behavioral outcomes.

Given the compelling need for better strategies to test the efficacy of warnings on tobacco products,[42, 43] this feasible protocol holds promise for improving our ability to determine the utility of warnings on cigarette packages. Our recommended protocol developed through this proof of principle study mimics real-world exposure to tobacco product packaging while allowing for experimental control needed in randomized trials. The protocol could be used to test the impact of other cigarette pack warning characteristics such as size and format, as well as the impact of modified-risk tobacco product warnings.[44, 45] Moreover, researchers can adapt the protocol to test graphic or text warnings for non-cigarette tobacco products. Future research should examine the utility of our pack labeling protocol in randomized controlled trials.

Rigorous evidence is required to identify the best ways to communicate the risks of tobacco use, de-normalize tobacco products, and reduce the extent to which tobacco product packaging glamorizes and promotes the deadly product inside. Our proof of principle studies provide a feasible strategy for testing warning messages that ensures high adherence, and can generate stronger evidence of the real-world impact of warnings on smoking-related outcomes.

WHAT THIS PAPER ADDS.

Experimental studies on graphic cigarette pack warnings typically use a brief one-time exposure, often in a laboratory setting. Naturalistic pack labeling studies hold promise for replicating the frequency and duration of warning exposure in the real world while maintaining experimental control. We developed and tested a six-step protocol for labeling smokers’ actual cigarette packs. The protocol was feasible to implement, was acceptable to smokers, and had high retention over four weeks. Using the protocol can yield stronger evidence on the impact of warnings to better inform the development of cigarette pack warning policies around the world.