Does Winning an Experimental Auction Change People's Behavior? An Application to E-cigarettes

Abstract

Experimental auctions allow researchers to estimate demand for products like e-cigarettes in a non-hypothetical environment where participants face real and immediate consequences for their bids. However, because auction winners actually purchase the product they bid on, participants may be introduced to a product they otherwise would not have discovered. Based on an experimental auction where 432 participants bid to buy e-cigarettes, we found that auction winners are significantly more likely to be using e-cigarettes two weeks, six weeks, and six months after the study but are no less likely to be daily cigarette smokers. This result holds even after controlling for prior e-cigarette use, strength of participants’ initial demand for e-cigarettes, and demographic characteristics.

Introduction

People behave differently when being observed by researchers. The lighting experiments at the Hawthorne Works in the 1920s and ‘30s (Mayo 1933) are the most famous example of this phenomenon. And though the causes of this “Hawthorne effect” are still debated (e.g., Levitt and List 2011), researchers continue to find ways in which participating in a study changes participants’ behavior. Sherman (1980), for example, finds early evidence of a “question-behavior effect,” where asking participants to predict how they will behave changes their future behavior. Sherman asked a control group if they would volunteer three hours of their time for a well-known charity; only 4% agreed. Sherman asked participants in a treatment group to predict if they would be willing to volunteer; nearly half said they would. When subsequently asked to actually volunteer, 31% of participants in the treatment group agreed. In a more recent study, Chandon et al. (2004) found that completing a survey on intent to make an online grocery purchase increased the likelihood of making a purchase.

The “survey effect” is closely related but more general. Here, asking participants questions on a topic changes their future opinions or actions even when the original question did not involve predicting future actions. Zwane et al. (2011), for example, found that Kenyan households surveyed more frequently about their use of home drinking water disinfectant actually used the disinfectant more regularly and reported better health outcomes.

We propose an “auction effect,” where winning a good in an experimental auction changes the likelihood a participant will continue to use the good in the future. The auction effect is of special concern in studies dealing with addictive substances or harmful behaviors. Experimental auctions, traditionally used to estimate willingness to pay for ordinary consumer goods (Lusk and Shogren 2007), have recently been used in a series of studies focused on tobacco products, including labeling differences between cigarettes (Rousu and Thrasher 2013), novel smokeless tobacco products (Rousu et al. 2014), and e-cigarettes (Rousu et al. 2017). A potential concern with this design is that exposing users to a tobacco product that is new to them, as well as allowing them to win it via auction, could encourage further use of that product or forestall quit attempts.

Until now, this has been difficult to assess since auction studies are typically one-session affairs with little to no follow-up. A study by Shogren et al. (2000) is one exception. The authors asked participants to bid on goods in a series of auctions spread over two weeks. Those who won a good in early auctions tended to decrease their bids in later auctions. The authors attributed this to the participants’ desire to learn about their preferences for unfamiliar goods. The authors did not, however, present evidence about whether winners are more likely to continue using a product after the two-week study period.

Our study proposes to fill this gap in the literature by building on recent work examining willingness to pay for e-cigarettes (Rousu et al. 2017). Our goal is to assess whether winning an e-cigarette in an experimental auction affects participants’ use of e-cigarette and cigarette two weeks, six weeks, and six months after their study session. We find that participants who won e-cigarettes were significantly more likely to use e-cigarettes at follow up but were no less likely to use cigarettes.

Methods

Institutional Review Boards at Roswell Park Comprehensive Cancer Center and Susquehanna University approved these procedures.

Sample.

Four hundred thirty-two smokers from the Buffalo, NY, and Selinsgrove, PA, areas took part in experimental auctions in 2014 and 2015. Participants were 18 or older, were not currently using e-cigarettes, and had no major medical issues. Participants received $80 for participating in the study with the understanding that they might win, and have to pay for, up to one tobacco product. According to Harrison and List’s (2004) taxonomy, this would be considered an artefactual field experiment – a laboratory experiment with a non-student population.

Products.

Three products were offered in an nth-price auction – Blu single-use e-cigarettes (Blu-D), Blu rechargeable e-cigarettes (Blu-R), and Camel cigarettes. Both e-cigarettes contained the nicotine equivalent of two packs of cigarettes. At the time of the study, both e-cigarette products were widely available at traditional cigarette retailers (e.g., convenience stores) as well as online (Haardörfer et al. 2017). Twenty-eight percent of participants won one of the e-cigarettes (Blu-D or Blu-R) at their auction sessions.

Experimental Design.

The experimental auction had six steps.

Step 1: When participants arrived, we checked their identification to ensure they were18 or older. Participants then completed a consent form, pre-auction questionnaire, and received $80 for their participation.

Step 2: Participants received detailed oral and written instructions on the random nth-price auction (Shogren 2001) and asked any questions they might have about the auction.

Step 3: Participants bid on two candy bars in two potentially binding practice rounds. After collecting bids from both rounds, we randomly selected the binding round and the binding nth price for that round. While the practice rounds were hypothetical, we emphasized that the auctions for tobacco products would be real.

Step 4: Participants saw e-cigarette advertisements based on their information group. Participants in the Control Group (N=97) saw no ads, participants in the Print Group (N=115) read a print ad for e-cigarettes, participants in the Video Group (N=118) watched a television commercial for e-cigarettes, and participants in the Print + Video Group (N=102) saw both the print ad and the TV commercial. Both ads were being used by Blu at the time of the experiment. See Rousu et al. (2017) for further discussion of the ads.

Step 5: Participants bid on the three tobacco products in three potentially binding auctions, only one of which would be realized. (While we randomized the order in which participants bid on e-cigarettes, participants always bid on the cigarettes last.) After collecting bids from all auctions, we randomly selected which of the three products would be auctioned and the binding nth price for that product. High bidders purchased the product from the binding auction.

Step 6: Participants completed a post-auction questionnaire.

Follow up.

Participants were contacted by telephone two weeks, six weeks, and six months after completing the auction. Questions during these brief telephone interviews focused on smoking (“Do you now smoke cigarettes every day, some days, or not at all?”) and e-cigarette use (“Do you now use an e-cigarette, e-hookah, or vape pen every day, some days, or not at all?”).

Data analysis

To examine the impact of winning a product on future use, we used a probit model. For all data analysis, we used Stata/SE 14.2. If a smoker reported using an e-cigarette occasionally or daily at the follow-up call, we define this as H_i =1. Otherwise H_i = 0. To examine the impact of winning a product, a participant’s bid in the auction, and other demographic characteristics on the probability of e-cigarette use, we use the following probit model:

$$\text{Prob}(H_i=1)=f(\alpha +\delta \text{'}{W}_i+\beta \text{'}{X}_i+{\epsilon }_i).$$ 1)

Here, α is an intercept term. W_i is a dummy variable that equals one if the participant won the auction, and δ’is the associated coefficient vector. X_i is a vector that represents auction bid and demographic and background characteristics of smoker i, and β’ is the associated coefficient vector. ε_i is the error term.

Results

Table 1 shows that of the initial 432 participants, 232 completed the two-week follow up, 226 completed the six-week follow up, and 260 completed the six-month follow up. More than three quarters of participants reported using cigarettes daily at two weeks, six weeks, and six months. These smoking rates did not vary statistically significantly by auction-winner status. E-cigarette use rates, however, varied substantially by auction winner status, with winners being up to 197% more likely to be using e-cigarettes than non-winners. A two-sample t-test assuming unequal variance rejects the null hypotheses of equal e-cigarette use across winner status at the 0.01 level for two weeks and six weeks and at the 0.05 level for six months.

Table 1.

Bivariate comparisons of daily smoking and any e-cigarette use at two weeks, six weeks, and six months post auction by auction winning status.

	Use cigarettes daily	Use e-cigarettes daily or some days
Two-week follow up
All (N=232)	82.3%	19.0%
Won e-cigarettes (N=62)	75.8%	33.9%
Did not win e-cigarettes (N=170)	84.7%	13.5%
Six-week follow up
All (N=226)	88.9%	17.7%
Won e-cigarettes (N=66)	87.9%	33.3%
Did not win e-cigarettes (N=160)		11.2%
Six-month follow up
All (N=260)	77.7%	16.5%
Won e-cigarettes (N=73)	74.0%	24.7%
Did not win e-cigarettes (N=187)	79.1%	13.4%

Because the product sold and the number of auction winners were randomly determined, the results from Table 1 have many of the desirable features of a randomized trial. However, higher bidders in any group were still more likely to win e-cigarettes than lower bidders. This introduces endogeneity because winning an e-cigarette at auction and using e-cigarettes at follow-up may be jointly determined by participants’ underlying interest in e-cigarettes. In our probit analysis (Table 2), we control for underlying interest by including (i) a dummy variable indicating prior e-cigarette use and (ii) the bid a participant placed for the single-use e-cigarette, which is a demand-revealing measure of that good’s value to the participant. The results from Table 2 suggest that having won an e-cigarette device at the auction and prior e-cigarette use were the only consistent predictors of e-cigarette use at each follow-up point. The coefficient associated with having won an e-cigarette can be interpreted as the effect of winning an e-cigarette on the likelihood of using e-cigarettes at follow-up, holding prior e-cigarette use, e-cigarette bid, and other demographic and treatment variables constant. We find that having won an e-cigarette increases the likelihood of using e-cigarettes at follow-up by 10 to 21 percentage points, again holding constant underlying interest in e-cigarettes as measured by prior use and auction bid. Information group, actual e-cigarette bid, and demographics were generally not associated with the probability of continued e-cigarette use. None of the examined factors were consistently associated with the probability of daily post-auction cigarette smoking.

Table 2.

Probit average marginal effects estimating the probability of e-cigarette use and daily smoking at two weeks, six weeks, and six months post-auction.

		E cigarette use			Smoking
	2 weeks (N=219)a	6 weeks (N=216)	6 months (N=251)	2 weeks (N=219)	6 weeks (N=216)	6 months (N=251)
Won e-cig	0.14**	0.21***	0.10**	−0.11*	−0.03 (0.05)	−0.06 (0.06)
(Ref=No)	(0.06)b	(0.05)	(0.05)	(0.06)	(0.05)	(0.06)
Used e-cig before	0.12**	0.04	0.12***	0.09*	0.01	0.02
(Ref=No)	(0.05)	(0.05)	(0.05)	(0.05)	(0.04)	(0.05)
Bid for single e-cig	0.01	−0.00	−0.00	−0.00	0.00	−0.00
($)	(0.01)	(0.00)	(0.01)	(0.02)	(0.00)	(0.00)
Print Group	−0.07	−0.01	0.02	−0.05	−0.01	−0.13*
(Ref= Control)	(0.08)	(0.07)	(0.07)	(0.07)	(0.06)	(0.08)
Video Group	0.08	−0.01	0.08	0.07	−0.02	−0.10
(Ref= Control)	(0.07)	(0.07)	(0.07)	(0.07)	(0.06)	(0.08)
Print+Video Group	0.00	−0.10	0.01	0.15*	0.05	−0.08
(Ref= Control)	(0.07)	(0.08)	(0.07)	(0.08)	(0.07)	(0.08)
Age	−0.00	0.00	−0.00	0.00	0.00	0.00
(years)	(0.00)	(0.00)	(0.00)	(0.00)	(0.00)	(0.00)
Female	0.03	−0.02	−0.05	0.04	0.01	0.09
(Ref=Male)	(0.05)	(0.05)	(0.05)	(0.05)	(0.04)	(0.05)
Race	−0.08	−0.11**	−0.03	−0.05	−0.01	−0.07
(Ref = White)	(0.05)	(0.05)	(0.05)	(0.05)	(0.04)	(0.05)

^aSample sizes differ from Table 1 because some participants did not complete the demographic survey.

^bStandard errors in parentheses.

^*Significant at 0.10 level.

^**Significant at 0.05 level.

^***Significant at 0.01 level.

Discussion

Seventy-eight percent of auction participants were contacted at least once in the six months post-auction. Among these, 21% reported using e-cigarettes in at least one follow-up call. Smokers who won e-cigarettes in the auction were significantly more likely to use e-cigarettes than those who did not win, and these effects were large. According to our probit analysis, participants who won e-cigarettes were between 10 to 21 percentage points more likely to be using e-cigarettes at follow-up. For those surveyed two weeks after the auction, this result seems natural, as winners would have just received either a single-use e-cigarette or an e-cigarette kit. But six weeks and six months out, participants who smoke e-cigarettes would have had to spend money on refills, suggesting that the experience of winning the auction, and thereby being exposed to a product trial, influenced longer-term behavior.¹ This holds even after controlling for prior e-cigarette use, auction bid (an index of demand), information condition, and demographics, suggesting that gaining access to the product per se encouraged use.

At the same time, winning an e-cigarette did not seem to have a lasting effect on cigarette smoking behavior – there was a marginally statistically significant negative effect at two weeks and no discernable effect at six weeks or six months.

The positive correlation (r = 0.27) between winning an e-cigarette and bid for the single-use e-cigarette raises the possibility of multicollinearity, which while not biasing the coefficient estimates themselves, would upwardly bias the coefficients’ standard error estimates, leading to lower t-statistics and p-values. However, this does not appear to be a serious problem with our data. We calculated the variance inflation factor (VIF), a measure of the severity of multicollinearity in OLS analysis, for having won an e-cigarette and bid for the single-use e-cigarette for all three timeframes. All six VIF estimates were in the range of 1.00 and 1.20, well below the cutoff of 10 commonly used to indicate a serious multicollinearity (O’Brien 2007).

Conclusions

We found that smokers who won an e-cigarette in an experimental auction were significantly more likely to be e-cigarettes users up to six months later but no less likely to use conventional cigarettes. A larger question is whether this impact on e-cigarette use is the result of preference learning or self-generated validity. In the case of preference learning, participants who won an e-cigarette may find that they like it due to convenience, risk reduction, or some other factor. Non-winners do not learn these lessons. As a result, winners are more likely to become regular e-cigarette users. In the case of self-generated validity, winners may continue to use e-cigarettes in order to validate their original purchase. In a review of the question-behavior effect literature, Dholakia (2010) explains that a participant may not have formed an opinion about a good before being asked to do so by the researcher. Once the participant has formed the opinion and shared it with the researcher, the opinion becomes more “accessible,” or easier for the participant to recall. Auction winners presumably formed relatively positively opinions of e-cigarettes. Because this positive opinion is now easily accessible, winners are more likely to become regular e-cigarette users than non-winners. Unfortunately, this study was not designed to determine whether preference learning, self-generated validity, or a combination of the two, drives results like ours.

From the perspective of encouraging smokers to transition to potentially less hazardous alternative nicotine delivery systems, our results suggest promoting trials of e-cigarettes should result in greater adoption, which mirrors the market experience thus far (e.g., Burbank et al. 2018, Kong et al. 2017, Lee et al. 2017). At the same time, the 2016 “deeming” regulation extended an existing ban on free sampling to all tobacco products, including electronic cigarettes, e-liquids, and accessories (Federal Register 2016). This does not include free products included with purchase of another tobacco product in the same transaction (e.g., buy one-get one discounts or coupons), potentially providing a means for manufacturers to promote trial by pairing novel e-products with traditional products (U.S. HHS 2017). However, these results are also consistent with the observation that many smokers are adopting dual use patterns (adding e-cigarette use to continued smoking) rather than a full substitution pattern, which does not appear to be associated with exposure reduction (Shahab et al. 2017). Such dual use could be furthered by the type of free trials permissible under deeming. This suggests that messaging around e-cigarettes should explicitly promote complete switching, and that policies could facilitate this via large price differentials, stricter access regulation for cigarettes, and making cigarettes less appealing (e.g., nicotine reduction).

Highlights.

• Subjects bid for e-cigarettes and cigarettes in an experimental nth-price auction.

• We conducted telephone follow-up interviews at two weeks, two months, and six months.

• E-cigarettes winners were significantly more likely to use e-cigarettes at follow up.

• E-cigarettes winners were no less likely to use cigarettes at follow up.